← 返回 关于

用 Rust 重写 Bun

2026-07-09 · 原文链接

披露:Bun 于 2025 年 12 月被 Anthropic 收购。我和 Bun 团队的其他成员现在都在 Anthropic 工作。在这次 Rust 重写的大部分过程中,我使用了 Claude Fable 5 的预发布版本。

Bun 最初是把 esbuild 的 JavaScript 和 TypeScript 转译器从 Go 逐行移植到 Zig。我在 2021 年 4 月 16 日写下了第一行 Zig。当时我在 Hacker News 上看到单页的 Zig Language Reference,被它对底层控制和性能的重视打动,于是押注了 Zig。

从一开始,Bun 的范围就非常大:

Bun 的初版是我在一个狭小的 Oakland 公寓里,用 Zig 花 1 年写出来的,那时还没有 LLM。像 Bun 这种范围野心很大的项目,默认结局往往是躺进 GitHub 个人页上一堆死掉 side project 的坟场。Zig 让 Bun 成为了可能。如果没有 Zig,我绝不可能在 1 年里做出这么多东西。

今天,Bun CLI 每月下载量超过 2200 万。Claude Code 和 OpenCode 这样的流行工具选择 Bun 作为运行时。Vercel、Railway、DigitalOcean 等平台也都提供了对 Bun 的一方支持。

Bun 的范围也一直给稳定性带来挑战。下面是我们在 Bun v1.3.14 中修复的一小部分 bug:

我们当然可以永远一个一个地修这些 bug,但依赖我们的用户值得更好的结果:我们应该系统性地防止这类 bug 反复出现。

我们已经在做什么

这已经比很多项目做得更多了。

只要足够聪明、不要犯错就行?

那份 bugfix 列表让我很难受。我也厌倦了睡前还在担心 Bun 会崩溃。我不怪 Zig,其他 Zig 用户并没有我们遇到的这些 bug,而把 GC 和手动管理内存混在一起,本来就是软件里少见到足以让语言专门为它设计的需求。如果没有 Zig,我们走不到今天,我也会一直感谢它。直到很近之前,对 Bun 这样的项目来说,编程语言选择还是一条单行道。

JavaScript 是一门带垃圾回收的语言,JavaScriptCore(以及 V8)这样的现代 JavaScript 引擎对异常处理和垃圾回收器有严格规则。Zig 和 C 一样,不替你管理内存;对很多项目来说,这种取舍正是使用 Zig 的好理由。Zig 没有构造函数/析构函数,大多数清理逻辑都预期在每个调用点用 defer 显式写出来。

对 Bun 来说,正确处理垃圾回收值和手动管理值的生命周期,一直是稳定性问题的主要来源,最常见的是小内存泄漏,偶尔是崩溃。每一次内存分配都必须仔细 review:这些字节在哪里释放?怎么保证只释放一次?JavaScript 异常有没有正确检查?这个 GC 指针对 conservative stack scanner 可见吗?这块内存是 GC 管理的,还是手动管理的?

对稳定性问题来说,越早知道越好。Fuzzing 发生在代码合并之后。CI 发生在代码 push 之后。运行时 safety check 和 address sanitizer 发生在代码运行时(理想情况下是在开发阶段、CI 之前)。

减少这类问题的常见方法之一,是确保需要清理的代码永远恰好运行一次。Zig 被设计成一门简单、没有隐藏控制流的语言,所以它更偏向用显式的 defer 关键字在作用域结束时运行代码,而不是 C++ 隐式的 ~Destructor 或 Rust 隐式的 Drop

Language

Cleanup

Zig

defer, errdefer

C++

~Destructor, &&Move

Rust

Drop

对 Zig 代码来说,我们到底应该在什么时候运行清理代码?如果把同一个 *T 传给很多不同函数,怎么知道它已经不再可访问,可以清理了?如果某些函数在被调用后还需要继续引用那段内存,又该怎么办?我们目前的方法是几种东西混在一起:

很多项目选择用 style guide 回答这类问题。TigerBeetle 的 TigerStyle 是 Zig 中的一个例子,Google 31000 词的 C++ style guide 是另一个例子。style guide 的挑战在于执行。你怎么确保 style guide 被遵守?历史上,答案是 code review,再辅以 linter 和 static analyzer 的 best-effort 检查。

对 Bun 来说,写一份严格的 style guide,并把清晰的 ownership 预期显式编码进类型系统,确实是一个现实选择。由于 Zig 没有 operator overloading,我们很可能最终会得到大量类似这样的代码:

fn foo(a_ptr: SharedPtr(TCPSocket)) !void {
  const a: *TCPSocket = a_ptr.get();
  defer a_ptr.deref();

  const b = try do_something_with_a(a);
  defer b.deref();

  // ...
}

这比我们期望的 Zig 写法更不顺手:

fn foo(a: *TCPSocket) !void {
  const b = try do_something_with_a(a);
  // ...
}

那 C/C++ 呢?

Bun 大约 20% 的代码是 C++,并且 Bun 嵌入了几个 C/C++ 库:

对 Bun 来说,用 C++ 代替 Zig 会是一个合理选择。我们会得到构造函数和析构函数,也可以删除大量 extern "C" wrapper 代码。

但是,我们仍然会依赖通过 code review 执行的 style guide。即使用了 ASAN,内存破坏和内存泄漏仍然会发生。

为什么是 Rust?

上面那份列表里的大量 bug 都是 use-after-free、double-free,以及错误路径上“忘记 free”。在 safe Rust 里,这些都是编译器错误,并且 Drop 会提供类似 RAII 的自动清理。编译器错误比 style guide 有更好的反馈循环。

历史上,重写通常是个糟糕主意。不算注释,Bun 有 535496 行 Zig。用另一门语言重写,需要一个小工程师团队花整整一年。这意味着在这段时间里冻结 bugfix、安全修复或功能开发。想得到一个可发布版本,风险最低的方法是从 Zig 到 Rust 做机械式移植,尽量少改行为,并使用我们已经用于测试 Bun 的同一套测试套件。

幸运的是,Bun 自己的测试套件是用 TypeScript 写的,这意味着它不依赖 runtime 的实现语言。

一年没有任何用户可见影响不是我们能考虑的现实选项。所以,通过 code style 强制约束来修稳定性问题,是我们最好的选择;这也是我们给 Bun 代码库加入 Rust 风格 smart pointer 时的计划。

但说实话,我不想这么做。自研 smart pointer 的 ergonomics 比 Rust 更差,却没有 Rust 的保证。

如果我换个方向,花一周测试 Anthropic 的新模型能不能把 Bun 重写成 Rust 呢?

一开始,我并不期待它能成功。几天后,测试套件开始有很高比例通过,我也看到新的 Rust 代码和原来的 Zig 代码库高度对应。我的看法从“这值得一试”变成了“我要把它 merge 进去”。

Claude,把 Bun 重写成 Rust。

有很多方法可以把这件事做得很糟。比如 prompt Claude:“Rewrite Bun in Rust. Don't make any mistakes.” 然后祈祷它能工作。这不是我做的事。

想想一个人会怎么做。第一个大问题是:

增量重写?还是一次性全部重写?

以我当初在没有 LLM 的情况下,把 esbuild 的 transpiler 从 Go 移植到 Zig 来做 Bun 初版的经验看,一次性全部重写更好。增量重写会加入临时代码,你希望它们最终被删除,但在短期到中期都会很痛苦。

第二个大问题是:怎么做?

我们怎样让 Rust 版 Bun 仍然是以前那个 Bun,保持同样的架构、性能和功能集,同时获得 Rust 的 borrow checker 等语言特性?怎样确保团队在重写后仍然能够维护它?

做一次看起来像把 Zig 代码转译成 Rust 的重写。Bun v1.4 发布后,我们可以逐步重构,减少 unsafe 使用,让它更像惯用 Rust。

大的问题只有这两个。其他都是战术。

写代码和 review 代码的循环

软件工程师的大量日常工程工作,都可以被过度简化成循环。

// Pseudocode, not real code:
let task;
while ((task = todoList.pop())) {
  const result = task();
  const feedback = await Promise.all([review(result), review(result)]);
  await apply(feedback, result);
}

一个 task 带着一些上下文(Jira ticket、GitHub issue 等)。result 是你为修它写的代码。code reviewer 调用 review 检查变更是否引入回归、是否正确。然后你处理反馈。

我用 Claude Code 中大约 50 个 dynamic workflow 持续运行了 11 天,把 Bun 重写成了 Rust。

每个 dynamic workflow 都是这样的循环,例如:

在这 11 天的大部分时间里(以及之后),我都在监控 workflow:手动阅读输出,检查问题和 bug,并 prompt Claude 修改循环来修复问题。

你要怎么 review 一个新增超过 100 万行的 PR?你要如何建立足够信心,负责任地合并大量 LLM 写出的代码?

答案是:一个和语言无关、拥有百万级断言的测试套件;adversarial code review;以及当问题发生时,修复生成代码的流程,而不是手动修代码。

Adversarial review

Adversarial review 会要求 Claude(在单独的 context window 中)穷尽式地提出这些变更会产生 bug 或无法工作的理由。

拆分 context window

通常对人来说,review 代码的人不是写代码的人。写代码的人想把代码 merge 进去,这可能让他们倾向于在代码还没准备好时就发出去。

Claude 也是这样。写代码的 Claude 想让代码被接受。review 的 Claude 想找出代码里的问题。

每个 implementer 配 2 个或更多 adversarial reviewer。reviewer 的唯一任务是:找 bug,找代码为什么不能工作的理由。implementer 不 review。reviewer 不 implement。

✻ claude code · dynamic workflowadversarial review3 of the many bugs adversarial review caught before merge

bug 1 of 3 · the async close

✻claudeimplementer

its context: the .zig original, the port plan, its own reasoning

✻claudeadversarial reviewer

its context: only the diff. told to assume the code is wrong.

src/runtime/api/bun/js_bun_spawn_bindings.rs · compiles clean

for stdio in [spawned_stdout, spawned_stderr] {

match stdio {

StdioResult::Buffer(mut pipe) => {

// pipe: Boxuv::Pipe — hand it to libuv to close

pipe.close(Subprocess::on_pipe_close)

}

StdioResult::Fd(fd) => fd.close(),

StdioResult::Unavailable => {}

}

}

uv_close 是异步的:libuv 会一直持有 raw handle pointer,直到下一个 loop tick,然后调用 on_pipe_close 释放这次分配。但 pipe 是一个 Box,它会在这个 match arm 结束时 drop。于是 libuv 手里留下了已经释放的内存,而 close callback 又会再释放一次。先 use-after-free,再 double-free。

Box::leak(pipe).close(Subprocess::on_pipe_close)

f0a454376c7 · win-review: js_bun_spawn_bindings.rs leak Boxuv::Pipe before async uv_close to avoid UAF/double-free in on_pipe_close

这是 adversarial reviewer 真正抓到的三个 bug。每个被引用的 commit subject 里都带着 review attribution。这三个 bug 都能编译,三个看起来都合理。reviewer 是另一个 Claude,运行在自己的 context window 里:它只拿到 diff,除此之外什么都没有,不知道 implementer 的推理;它被要求找出代码错在哪里。这里的代码从引用 commit 中压缩而来,bug 和修复是同一个。

这看起来是什么样?

如果你准备做一件又大又贵的事,先降低风险能省时间也省钱。

准备工作

在写任何代码之前,我花了大约 3 个小时和 Claude 讨论如何把我们 Zig 代码库里的 pattern 尽量贴近地映射到 Rust。Claude 把这次讨论序列化成一份 PORTING.md 文档,后来还上了 Hacker News

下一个问题是:怎样给手动管理内存的代码加上 Rust lifetime?

这时我大概这样 prompt Claude:

我:让我们启动一个 dynamic workflow,分析代码库里每个 struct field 的正确 lifetime。这个 workflow 应该读取每个文件里的每个 struct field,并追踪控制流。首先找出那些在 Rust 中难以表达复杂 lifetime 的 struct field,然后为这个 field 提出一个 lifetime,再用 2 个 adversarial review agent review 这个 lifetime,然后应用所有反馈,并序列化成 LIFETIMES.tsv,供其他 Claude 查看。

然后对 PORTING.mdLIFETIMES.tsv 一起做一轮 adversarial review,修复所有互相冲突的建议,并再次检查所有内容。我也手动读了一遍。

试运行

在要求 Claude 把全部 1448 个 .zig 文件翻译成 .rs 文件之前,我先只试了 3 个文件。对这 3 个文件中的每一个,1 个 implementer 写新的 .rs 文件,2 个 adversarial reviewer 检查这个 .rs 文件是否和 .zig 文件行为一致,且是否遵循 PORTING.mdLIFETIMES.tsv。之后,1 个 fixer 应用所有建议。

几次误启动

我让 Claude 在全部 1448 个 .zig 文件上循环跑 workflow。大约 2 分钟后,一个 Claude 在 commit 前运行了 git stash。另一个运行了 git stash pop。然后又来了 git reset HEAD --hard。它们互相踩脚了!如果我把每个 Claude 放进单独的 worktree,又会用光磁盘空间,因为 Bun 的 git 仓库太大,而且最终这些变更还需要一起编译和观察。

所以,我让 Claude 修改 workflow,指示 Claude 永远不要运行 git stashgit reset,也不要运行任何不能一次性 commit 特定文件的 git 命令。也不要跑 cargo。完全不要跑慢命令。

然后 Claude 恢复了 workflow。它开始工作了!只是太慢,所以我把它拆成 4 个 workflow shard,每个 shard 有自己的 worktree(总共 4 个 worktree),每个 shard 运行 16 个 Claude 来提交和 push 文件。

终于开始写代码

得益于所有这些并行化和准备工作,在峰值时 Claude 每分钟写下大约 1300 行代码。每一行代码都由两个独立的 adversarial reviewer(也是 Claude)review,并在提交前经过一轮修复。当时它们完全不能工作。

11 days × 24 hours · PDT

6,502 commits

1695 commits/hour

12am6am12pm6pmMay 4May 4, 7am–8am PDT — 6 commits, +89,278 linesMay 4, 8am–9am PDT — 2 commits, +50,742 linesMay 4, 9am–10am PDT — 1 commit, +28,149 linesMay 4, 11am–12pm PDT — 1 commit, +39,752 linesMay 4, 12pm–1pm PDT — 3 commits, +251,616 linesMay 4, 1pm–2pm PDT — 2 commits, +161,724 linesMay 4, 3pm–4pm PDT — 3 commits, +136,381 linesMay 4, 5pm–6pm PDT — 5 commits, +895 linesMay 4, 6pm–7pm PDT — 5 commits, +17,027 linesMay 4, 7pm–8pm PDT — 1 commit, +106 linesMay 4, 9pm–10pm PDT — 13 commits, +11,661 linesMay 4, 11pm–12am PDT — 6 commits, +8,516 linesMay 5May 5, 12am–1am PDT — 9 commits, +1,381 linesMay 5, 1am–2am PDT — 7 commits, +1,577 linesMay 5, 2am–3am PDT — 4 commits, +2,035 linesMay 5, 3am–4am PDT — 4 commits, +7,808 linesMay 5, 4am–5am PDT — 1 commit, +2,796 linesMay 5, 5am–6am PDT — 2 commits, +29,370 linesMay 5, 8am–9am PDT — 2 commits, +7,076 linesMay 5, 9am–10am PDT — 2 commits, +308 linesMay 5, 11am–12pm PDT — 2 commits, +1,643 linesMay 5, 12pm–1pm PDT — 4 commits, +1,452 linesMay 5, 1pm–2pm PDT — 1 commit, +2,142 linesMay 5, 2pm–3pm PDT — 4 commits, +7,787 linesMay 5, 3pm–4pm PDT — 2 commits, +5,835 linesMay 5, 4pm–5pm PDT — 1 commit, +3,417 linesMay 5, 5pm–6pm PDT — 4 commits, +3,960 linesMay 5, 6pm–7pm PDT — 4 commits, +9,179 linesMay 5, 7pm–8pm PDT — 4 commits, +1,983 linesMay 5, 8pm–9pm PDT — 4 commits, +18,902 linesMay 5, 9pm–10pm PDT — 43 commits, +40,650 linesMay 5, 10pm–11pm PDT — 139 commits, +64,842 linesMay 5, 11pm–12am PDT — 141 commits, +34,814 linesMay 6May 6, 12am–1am PDT — 60 commits, +10,417 linesMay 6, 1am–2am PDT — 296 commits, +38,530 linesMay 6, 2am–3am PDT — 306 commits, +18,836 linesMay 6, 3am–4am PDT — 196 commits, +10,245 linesMay 6, 4am–5am PDT — 86 commits, +2,655 linesMay 6, 5am–6am PDT — 16 commits, +289 linesMay 6, 8am–9am PDT — 5 commits, +264 linesMay 6, 9am–10am PDT — 458 commits, +16,409 linesMay 6, 10am–11am PDT — 695 commits, +44,000 linesMay 6, 11am–12pm PDT — 102 commits, +21,972 linesMay 6, 12pm–1pm PDT — 19 commits, +2,891 linesMay 6, 1pm–2pm PDT — 3 commits, +56 linesMay 6, 3pm–4pm PDT — 64 commits, +3,606 linesMay 6, 4pm–5pm PDT — 264 commits, +60,132 linesMay 6, 5pm–6pm PDT — 268 commits, +40,953 linesMay 6, 6pm–7pm PDT — 281 commits, +16,283 linesMay 6, 7pm–8pm PDT — 258 commits, +26,654 linesMay 6, 8pm–9pm PDT — 327 commits, +16,599 linesMay 6, 9pm–10pm PDT — 74 commits, +8,331 linesMay 6, 10pm–11pm PDT — 17 commits, +2,200 linesMay 6, 11pm–12am PDT — 11 commits, +3,590 linesMay 7May 7, 12am–1am PDT — 17 commits, +6,577 linesMay 7, 1am–2am PDT — 22 commits, +8,718 linesMay 7, 2am–3am PDT — 21 commits, +11,392 linesMay 7, 3am–4am PDT — 53 commits, +6,476 linesMay 7, 4am–5am PDT — 31 commits, +2,356 linesMay 7, 5am–6am PDT — 9 commits, +1,787 linesMay 7, 6am–7am PDT — 4 commits, +580 linesMay 7, 7am–8am PDT — 5 commits, +181 linesMay 7, 11am–12pm PDT — 3 commits, +421 linesMay 7, 12pm–1pm PDT — 1 commit, +13 linesMay 7, 1pm–2pm PDT — 5 commits, +248 linesMay 7, 2pm–3pm PDT — 9 commits, +2,131 linesMay 7, 3pm–4pm PDT — 51 commits, +3,207 linesMay 7, 4pm–5pm PDT — 56 commits, +2,647 linesMay 7, 5pm–6pm PDT — 159 commits, +2,787 linesMay 7, 6pm–7pm PDT — 42 commits, +1,590 linesMay 7, 7pm–8pm PDT — 46 commits, +4,170 linesMay 7, 8pm–9pm PDT — 52 commits, +2,113 linesMay 7, 9pm–10pm PDT — 27 commits, +1,585 linesMay 7, 10pm–11pm PDT — 27 commits, +2,231 linesMay 7, 11pm–12am PDT — 30 commits, +4,987 linesMay 8May 8, 12am–1am PDT — 27 commits, +1,196 linesMay 8, 1am–2am PDT — 14 commits, +904 linesMay 8, 2am–3am PDT — 8 commits, +536 linesMay 8, 3am–4am PDT — 13 commits, +253 linesMay 8, 4am–5am PDT — 3 commits, +771 linesMay 8, 5am–6am PDT — 15 commits, +1,545 linesMay 8, 6am–7am PDT — 12 commits, +1,965 linesMay 8, 7am–8am PDT — 14 commits, +1,866 linesMay 8, 8am–9am PDT — 55 commits, +3,622 linesMay 8, 9am–10am PDT — 35 commits, +4,778 linesMay 8, 10am–11am PDT — 1 commit, +0 linesMay 8, 12pm–1pm PDT — 1 commit, +116 linesMay 8, 1pm–2pm PDT — 2 commits, +66 linesMay 8, 2pm–3pm PDT — 9 commits, +1,071 linesMay 8, 3pm–4pm PDT — 26 commits, +1,691 linesMay 8, 4pm–5pm PDT — 18 commits, +2,751 linesMay 8, 5pm–6pm PDT — 2 commits, +97 linesMay 8, 6pm–7pm PDT — 2 commits, +135 linesMay 8, 7pm–8pm PDT — 11 commits, +1,763 linesMay 8, 8pm–9pm PDT — 20 commits, +5,272 linesMay 8, 9pm–10pm PDT — 12 commits, +952 linesMay 8, 10pm–11pm PDT — 2 commits, +334 linesMay 8, 11pm–12am PDT — 6 commits, +2,033 linesMay 9May 9, 12am–1am PDT — 9 commits, +387 linesMay 9, 1am–2am PDT — 9 commits, +723 linesMay 9, 2am–3am PDT — 8 commits, +98 linesMay 9, 3am–4am PDT — 63 commits, +2,538 linesMay 9, 4am–5am PDT — 11 commits, +8,861 linesMay 9, 5am–6am PDT — 4 commits, +42 linesMay 9, 6am–7am PDT — 3 commits, +2,616 linesMay 9, 7am–8am PDT — 6 commits, +6,993 linesMay 9, 8am–9am PDT — 1 commit, +3,705 linesMay 9, 9am–10am PDT — 11 commits, +199 linesMay 9, 11am–12pm PDT — 1 commit, +23 linesMay 9, 12pm–1pm PDT — 4 commits, +5,012 linesMay 9, 1pm–2pm PDT — 7 commits, +2,080 linesMay 9, 2pm–3pm PDT — 6 commits, +924 linesMay 9, 3pm–4pm PDT — 5 commits, +248 linesMay 9, 4pm–5pm PDT — 17 commits, +508 linesMay 9, 5pm–6pm PDT — 2 commits, +135 linesMay 9, 6pm–7pm PDT — 4 commits, +822 linesMay 9, 7pm–8pm PDT — 1 commit, +7 linesMay 10May 10, 12am–1am PDT — 4 commits, +497 linesMay 10, 1am–2am PDT — 2 commits, +35 linesMay 10, 2am–3am PDT — 1 commit, +131 linesMay 10, 3am–4am PDT — 2 commits, +322 linesMay 10, 4am–5am PDT — 1 commit, +3 linesMay 10, 5am–6am PDT — 1 commit, +26 linesMay 10, 6am–7am PDT — 2 commits, +81 linesMay 10, 7am–8am PDT — 1 commit, +5 linesMay 10, 8am–9am PDT — 4 commits, +78 linesMay 10, 9am–10am PDT — 1 commit, +1 linesMay 10, 10am–11am PDT — 2 commits, +128 linesMay 10, 11am–12pm PDT — 1 commit, +4 linesMay 10, 12pm–1pm PDT — 2 commits, +413 linesMay 10, 1pm–2pm PDT — 1 commit, +25 linesMay 10, 2pm–3pm PDT — 5 commits, +327 linesMay 10, 3pm–4pm PDT — 6 commits, +1,172 linesMay 10, 4pm–5pm PDT — 4 commits, +752 linesMay 10, 5pm–6pm PDT — 3 commits, +227 linesMay 10, 6pm–7pm PDT — 2 commits, +242 linesMay 10, 7pm–8pm PDT — 1 commit, +306 linesMay 10, 8pm–9pm PDT — 1 commit, +54 linesMay 10, 9pm–10pm PDT — 2 commits, +75 linesMay 10, 10pm–11pm PDT — 1 commit, +134 linesMay 10, 11pm–12am PDT — 5 commits, +103 linesMay 11May 11, 12am–1am PDT — 2 commits, +150 linesMay 11, 1am–2am PDT — 4 commits, +398 linesMay 11, 2am–3am PDT — 2 commits, +364 linesMay 11, 3am–4am PDT — 3 commits, +44 linesMay 11, 4am–5am PDT — 7 commits, +9,367 linesMay 11, 6am–7am PDT — 2 commits, +43 linesMay 11, 7am–8am PDT — 2 commits, +149 linesMay 11, 8am–9am PDT — 10 commits, +2,171 linesMay 11, 9am–10am PDT — 16 commits, +2,047 linesMay 11, 10am–11am PDT — 18 commits, +3,356 linesMay 11, 11am–12pm PDT — 9 commits, +861 linesMay 11, 12pm–1pm PDT — 3 commits, +412 linesMay 11, 1pm–2pm PDT — 12 commits, +2,978 linesMay 11, 2pm–3pm PDT — 157 commits, +10,700 linesMay 11, 3pm–4pm PDT — 16 commits, +1,346 linesMay 11, 4pm–5pm PDT — 3 commits, +78 linesMay 11, 5pm–6pm PDT — 41 commits, +2,568 linesMay 11, 6pm–7pm PDT — 55 commits, +4,912 linesMay 11, 7pm–8pm PDT — 53 commits, +3,475 linesMay 11, 8pm–9pm PDT — 32 commits, +1,732 linesMay 11, 9pm–10pm PDT — 46 commits, +4,506 linesMay 11, 10pm–11pm PDT — 45 commits, +1,711 linesMay 11, 11pm–12am PDT — 52 commits, +10,850 linesMay 12May 12, 12am–1am PDT — 30 commits, +3,760 linesMay 12, 1am–2am PDT — 24 commits, +9,443 linesMay 12, 2am–3am PDT — 41 commits, +1,635 linesMay 12, 3am–4am PDT — 39 commits, +788 linesMay 12, 4am–5am PDT — 27 commits, +651 linesMay 12, 5am–6am PDT — 23 commits, +779 linesMay 12, 6am–7am PDT — 1 commit, +137,576 linesMay 12, 7am–8am PDT — 2 commits, +81 linesMay 12, 8am–9am PDT — 2 commits, +75 linesMay 12, 9am–10am PDT — 2 commits, +130 linesMay 12, 10am–11am PDT — 5 commits, +160 linesMay 12, 11am–12pm PDT — 2 commits, +20 linesMay 12, 12pm–1pm PDT — 1 commit, +2 linesMay 12, 1pm–2pm PDT — 30 commits, +2,677 linesMay 12, 2pm–3pm PDT — 41 commits, +7,022 linesMay 12, 3pm–4pm PDT — 4 commits, +200 linesMay 12, 4pm–5pm PDT — 27 commits, +1,423 linesMay 12, 5pm–6pm PDT — 19 commits, +1,055 linesMay 12, 6pm–7pm PDT — 2 commits, +380 linesMay 12, 7pm–8pm PDT — 2 commits, +84 linesMay 12, 9pm–10pm PDT — 7 commits, +273 linesMay 12, 10pm–11pm PDT — 3 commits, +230 linesMay 12, 11pm–12am PDT — 7 commits, +319 linesMay 13May 13, 12am–1am PDT — 2 commits, +133 linesMay 13, 1am–2am PDT — 14 commits, +2,177 linesMay 13, 2am–3am PDT — 12 commits, +685 linesMay 13, 4am–5am PDT — 10 commits, +657 linesMay 13, 5am–6am PDT — 1 commit, +687 linesMay 13, 6am–7am PDT — 11 commits, +380 linesMay 13, 7am–8am PDT — 12 commits, +5,247 linesMay 13, 8am–9am PDT — 14 commits, +1,051 linesMay 13, 9am–10am PDT — 7 commits, +680 linesMay 13, 10am–11am PDT — 10 commits, +412 linesMay 13, 11am–12pm PDT — 6 commits, +314 linesMay 13, 12pm–1pm PDT — 10 commits, +2,980 linesMay 13, 1pm–2pm PDT — 1 commit, +0 linesMay 13, 2pm–3pm PDT — 3 commits, +439 linesMay 13, 5pm–6pm PDT — 7 commits, +114 linesMay 13, 6pm–7pm PDT — 4 commits, +605 linesMay 13, 9pm–10pm PDT — 1 commit, +13 linesMay 13, 10pm–11pm PDT — 1 commit, +48 linesMay 13, 11pm–12am PDT — 1 commit, +8 linesMay 14May 14, 12am–1am PDT — 1 commit, +150 lines

port branch 上的每个 commit(排除 merge)按小时分桶。峰值小时:695 个 commit。

注意时间分布不均吗?我忘了提高运行它的 EC2 instance 的默认 IOPS。一个慢 grep 命令就足以让磁盘读写卡住好几分钟。

把编译错误当作工作队列

写完所有代码后,我让 Claude 写一个 workflow 来修复每个编译错误。我们按 crate 一个一个推进。

✻ claude code · dynamic workflow

≈16,000 errors left

Wed, May 6, 12:40 AM PDT

errors.txt0 fix commits

error: deref *mut EventLoop before field access

error: js_parser/ast/E.rs: port json_stringify for Number/BigInt/RegExp

error: NodeHTTPResponse.rs: wire JSNodeHTTPResponse cached accessors vi

error[E0034]: multiple applicable items in scope

error: test_command.rs: wire coverage façade to bun_sourcemap_jsc::code

error: bundler/ungate_support.rs: un-gate bun_css shim to real ::bun_cs

error: dns.rs: implement pending_cache_for/get_key/get_or_put_into_reso

error: css/css_parser.rs: port DefineShorthand contract, parse_bundler,

error: runtime/crypto/mod.rs: create_crypto_error delegates to boringss

error: bun_core/fmt.rs: implement format_ip reborrow (offset-based slic

error: event_loop/EventLoopTimer.rs: port Timespec::ns from bun.zig

divvied up · 64 claudes

worktree 1

→→

→→

→→

→→

worktree 2

→→

→→

→→

→→

worktree 3

→→

→→

→→

→→

worktree 4

→→

→→

→→

→→

1 fixes2 review1 applies

→ commits land per crate

Phase D 的工作方式,按它真实的 1610 个 commit 回放(5 月 6 日,PDT):cargo check 把约 16000 条错误写入文件,按 crate 分组;workflow 把它们分给 64 个 Claude:4 个 worktree 上共 16 个循环,每个循环 1 个 Claude 修复、2 个 review、1 个应用。每个 chip 都是一批真实 commit:它们落在实际 crate 上,随后计数器才移动。错误行是真实的 commit subject。

最棘手的一类错误是循环依赖。

我们的 Zig 代码库是一个编译单元(实际上就是一个 crate)。我想把新的 Rust 代码库拆成大约 100 个 crate,这样 Rust 编译更快;但这需要避免循环依赖,同时尽量少偏离原 Zig 实现。我在开始 Rust 重写前立刻提交的这个 PR并不充分。我没有推倒重来,而是又跑了一个 workflow,用来分类有循环依赖的代码应该放在哪里,并全部写下来;然后再跑另一个 workflow 执行重构。

修复循环依赖暴露了大约 16000 个编译错误。对 1 个人来说这是巨量;对 64 个 Claude 同时工作来说,并不离谱。

为了最大化并行度,workflow 会按每个 crate 循环:

为了防止 Claude 互相踩脚,cargo check 只在最开始运行;和其他运行一样,直到最后才允许 git

又一次误启动

Claude 把“让所有 crate 编译通过”理解成了“把有编译错误的函数 stub 掉”。Claude 还开始加入可疑地长的解释性注释来说明 workaround,所以我给 adversarial reviewer 加了这条拒绝规则:

如果你需要一整段注释来证明 workaround 没问题,那代码就是错的,修代码。

改了一次 prompt,几个小时后,这些情况就不再发生了。

Smoke test

模型很爱说“smoke tests”。

一旦 cargo check 通过,下一步就是让它能编译并运行 bun --version。它先是有 linker error。然后一启动就 panic。

下一个目标是让它能运行 bun test <file>。一旦这能工作,我们就可以开始跑测试了!于是又来了一个 workflow,围绕 bun CLI 子命令循环:

让测试套件在本地通过

这个 workflow 围绕测试文件循环。

随机运行大约 100 个测试文件,按代码库文件夹分片到 4 个 worktree 中的一个。对每个失败测试,保存 stacktrace 和错误到文件,1 个 implementer 提出修复,2 个 adversarial reviewer,然后 1 个 fixer 应用。

更多误启动

我们的测试套件有很多内存泄漏测试,也有少数集成测试可能运行超过一分钟,例如一个运行 next dev 并检查 hot module reloading 是否能连续 100 次拾取变更的测试。其中一些测试在 debug build 中会 timeout。

我们还有会耗尽机器最大 TCP socket 数的压力测试、读写数 GB 磁盘的测试,以及会 spawn 大约 1 万个进程的测试。

这需要比“拜托了”更强的隔离,所以我们用 systemd-run(cgroups)限制内存和 CPU 使用,并隔离 pid namespace。即便如此,机器还是几次用光磁盘空间并崩溃。

让测试套件在 CI 中通过

第一次 CI 运行两天后,失败列表从 972 个测试文件降到 23 个。又过了一天半,Linux 全绿了。也是第一次,这次 Rust 重写真的让人感觉会成功。

✻ claude code · dynamic workflowbuildkite · the race to green, by platformWindows finished last · May 11, 6:23 AM PDT

6 / 6 platforms green

build #54202 · Thu, May 14, 12:23 AM PDT

macOS x64 · 2 shards

build #52897: shard failuresbuild #52932: shard failuresbuild #52934: shard failuresbuild #52938: shard failuresbuild #52944: shard failuresbuild #52946: shard failuresbuild #52949: shard failuresbuild #52975: shard failuresbuild #52998: shard failuresbuild #53007: shard failuresbuild #53015: shard failuresbuild #53026: shard failuresbuild #53027: shard failuresbuild #53035: shard failuresbuild #53041: shard failuresbuild #53047: shard failuresbuild #53056: shard failuresbuild #53077: shard failuresbuild #53090: shard failuresbuild #53095: shard failuresbuild #53106: shard failuresbuild #53109: shard failuresbuild #53123: shard failuresbuild #53127: shard failuresbuild #53130: shard failuresbuild #53131: shard failuresbuild #53133: shard failuresbuild #53134: shard failuresbuild #53143: shard failuresbuild #53149: all shards passedbuild #53159: all shards passedbuild #53164: all shards passedbuild #53167: all shards passedbuild #53172: all shards passedbuild #53176: all shards passedbuild #53194: all shards passedbuild #53208: all shards passedbuild #53213: shard failuresbuild #53214: shard failuresbuild #53216: no failures (partial run)build #53222: all shards passedbuild #53229: no failures (partial run)build #53241: all shards passedbuild #53265: all shards passedbuild #53271: all shards passedbuild #53304: shard failuresbuild #53327: shard failuresbuild #53340: shard failuresbuild #53401: shard failuresbuild #53431: shard failuresbuild #53491: all shards passedbuild #53503: all shards passedbuild #53748: shard failuresbuild #53753: all shards passedbuild #53787: all shards passedbuild #53811: all shards passedbuild #53933: no failures (partial run)build #53952: all shards passedbuild #53983: shard failuresbuild #53992: shard failuresbuild #53999: shard failuresbuild #54012: all shards passedbuild #54015: all shards passedbuild #54017: all shards passedbuild #54022: shard failuresbuild #54026: shard failuresbuild #54033: all shards passedbuild #54040: all shards passedbuild #54047: shard failuresbuild #54049: shard failuresbuild #54057: shard failuresbuild #54064: all shards passedbuild #54074: all shards passedbuild #54093: all shards passedbuild #54144: no failures (partial run)build #54161: shard failuresbuild #54186: shard failuresbuild #54189: shard failuresbuild #54196: shard failuresbuild #54202: all shards passed

Linux arm64 · 60 shards

build #52934: shard failuresbuild #52938: shard failuresbuild #52944: shard failuresbuild #52969: shard failuresbuild #52975: shard failuresbuild #52980: shard failuresbuild #52988: shard failuresbuild #52996: shard failuresbuild #52998: shard failuresbuild #53007: shard failuresbuild #53013: shard failuresbuild #53014: shard failuresbuild #53015: shard failuresbuild #53026: shard failuresbuild #53027: shard failuresbuild #53031: no failures (partial run)build #53032: shard failuresbuild #53035: shard failuresbuild #53041: shard failuresbuild #53047: shard failuresbuild #53056: shard failuresbuild #53059: shard failuresbuild #53077: shard failuresbuild #53083: shard failuresbuild #53086: shard failuresbuild #53090: shard failuresbuild #53095: shard failuresbuild #53106: shard failuresbuild #53109: shard failuresbuild #53123: shard failuresbuild #53127: shard failuresbuild #53130: shard failuresbuild #53131: shard failuresbuild #53133: shard failuresbuild #53134: shard failuresbuild #53135: shard failuresbuild #53143: shard failuresbuild #53149: shard failuresbuild #53159: shard failuresbuild #53164: shard failuresbuild #53167: all shards passedbuild #53172: shard failuresbuild #53176: all shards passedbuild #53188: shard failuresbuild #53194: shard failuresbuild #53208: all shards passedbuild #53212: shard failuresbuild #53213: shard failuresbuild #53214: shard failuresbuild #53216: all shards passedbuild #53222: all shards passedbuild #53229: all shards passedbuild #53236: no failures (partial run)build #53241: all shards passedbuild #53260: all shards passedbuild #53265: all shards passedbuild #53271: all shards passedbuild #53280: no failures (partial run)build #53298: no failures (partial run)build #53304: shard failuresbuild #53327: all shards passedbuild #53340: all shards passedbuild #53360: no failures (partial run)build #53419: shard failuresbuild #53431: shard failuresbuild #53458: no failures (partial run)build #53485: shard failuresbuild #53491: shard failuresbuild #53503: shard failuresbuild #53514: no failures (partial run)build #53570: shard failuresbuild #53583: shard failuresbuild #53599: shard failuresbuild #53748: shard failuresbuild #53753: all shards passedbuild #53762: no failures (partial run)build #53787: no failures (partial run)build #53811: all shards passedbuild #53852: no failures (partial run)build #53863: shard failuresbuild #53893: shard failuresbuild #53914: all shards passedbuild #53933: all shards passedbuild #53952: all shards passedbuild #53983: shard failuresbuild #53992: shard failuresbuild #53999: shard failuresbuild #54008: no failures (partial run)build #54012: all shards passedbuild #54015: all shards passedbuild #54017: all shards passedbuild #54022: shard failuresbuild #54026: all shards passedbuild #54030: shard failuresbuild #54033: all shards passedbuild #54040: all shards passedbuild #54047: shard failuresbuild #54049: shard failuresbuild #54055: shard failuresbuild #54057: shard failuresbuild #54064: all shards passedbuild #54074: all shards passedbuild #54083: no failures (partial run)build #54093: all shards passedbuild #54144: all shards passedbuild #54161: shard failuresbuild #54186: shard failuresbuild #54189: shard failuresbuild #54196: shard failuresbuild #54202: all shards passed

Linux x64 · 60 shards

build #52934: shard failuresbuild #52938: shard failuresbuild #52944: shard failuresbuild #52969: shard failuresbuild #52975: shard failuresbuild #52988: shard failuresbuild #52996: shard failuresbuild #52998: shard failuresbuild #53007: shard failuresbuild #53013: shard failuresbuild #53014: shard failuresbuild #53015: shard failuresbuild #53026: shard failuresbuild #53027: shard failuresbuild #53032: shard failuresbuild #53033: shard failuresbuild #53035: shard failuresbuild #53041: shard failuresbuild #53047: shard failuresbuild #53056: shard failuresbuild #53059: no failures (partial run)build #53077: shard failuresbuild #53083: no failures (partial run)build #53086: shard failuresbuild #53090: shard failuresbuild #53095: shard failuresbuild #53106: shard failuresbuild #53109: shard failuresbuild #53123: shard failuresbuild #53127: shard failuresbuild #53130: shard failuresbuild #53131: shard failuresbuild #53133: shard failuresbuild #53134: shard failuresbuild #53135: shard failuresbuild #53143: shard failuresbuild #53149: shard failuresbuild #53159: shard failuresbuild #53164: shard failuresbuild #53167: all shards passedbuild #53172: all shards passedbuild #53176: all shards passedbuild #53188: shard failuresbuild #53194: all shards passedbuild #53208: all shards passedbuild #53212: shard failuresbuild #53213: shard failuresbuild #53214: shard failuresbuild #53216: all shards passedbuild #53222: all shards passedbuild #53229: all shards passedbuild #53236: no failures (partial run)build #53241: all shards passedbuild #53260: all shards passedbuild #53265: all shards passedbuild #53271: all shards passedbuild #53280: no failures (partial run)build #53304: shard failuresbuild #53327: all shards passedbuild #53340: all shards passedbuild #53360: no failures (partial run)build #53419: no failures (partial run)build #53431: shard failuresbuild #53458: no failures (partial run)build #53485: shard failuresbuild #53491: all shards passedbuild #53503: all shards passedbuild #53514: no failures (partial run)build #53570: shard failuresbuild #53583: shard failuresbuild #53599: shard failuresbuild #53748: shard failuresbuild #53753: all shards passedbuild #53759: no failures (partial run)build #53781: shard failuresbuild #53787: no failures (partial run)build #53811: all shards passedbuild #53863: shard failuresbuild #53893: shard failuresbuild #53914: all shards passedbuild #53933: shard failuresbuild #53952: all shards passedbuild #53983: shard failuresbuild #53992: shard failuresbuild #53999: shard failuresbuild #54008: no failures (partial run)build #54012: all shards passedbuild #54015: all shards passedbuild #54017: all shards passedbuild #54022: shard failuresbuild #54026: all shards passedbuild #54030: shard failuresbuild #54033: all shards passedbuild #54040: no failures (partial run)build #54047: shard failuresbuild #54049: shard failuresbuild #54055: shard failuresbuild #54057: shard failuresbuild #54064: all shards passedbuild #54074: all shards passedbuild #54083: no failures (partial run)build #54093: all shards passedbuild #54144: all shards passedbuild #54161: shard failuresbuild #54186: shard failuresbuild #54189: shard failuresbuild #54196: shard failuresbuild #54202: all shards passed

macOS arm64 · 4 shards

build #52897: shard failuresbuild #52929: shard failuresbuild #52932: shard failuresbuild #52944: shard failuresbuild #52975: shard failuresbuild #52996: shard failuresbuild #52998: shard failuresbuild #53007: shard failuresbuild #53013: shard failuresbuild #53014: shard failuresbuild #53015: shard failuresbuild #53026: shard failuresbuild #53027: shard failuresbuild #53032: shard failuresbuild #53035: shard failuresbuild #53041: shard failuresbuild #53047: shard failuresbuild #53056: shard failuresbuild #53059: shard failuresbuild #53077: shard failuresbuild #53095: shard failuresbuild #53109: shard failuresbuild #53123: shard failuresbuild #53127: shard failuresbuild #53130: shard failuresbuild #53131: shard failuresbuild #53133: shard failuresbuild #53134: shard failuresbuild #53135: shard failuresbuild #53143: shard failuresbuild #53149: shard failuresbuild #53159: shard failuresbuild #53164: shard failuresbuild #53167: shard failuresbuild #53172: shard failuresbuild #53176: shard failuresbuild #53188: shard failuresbuild #53194: shard failuresbuild #53208: shard failuresbuild #53212: shard failuresbuild #53213: shard failuresbuild #53214: shard failuresbuild #53216: shard failuresbuild #53222: shard failuresbuild #53229: no failures (partial run)build #53236: no failures (partial run)build #53241: all shards passedbuild #53265: all shards passedbuild #53271: no failures (partial run)build #53280: no failures (partial run)build #53304: shard failuresbuild #53327: no failures (partial run)build #53340: no failures (partial run)build #53360: no failures (partial run)build #53368: shard failuresbuild #53379: shard failuresbuild #53383: shard failuresbuild #53401: shard failuresbuild #53431: shard failuresbuild #53458: shard failuresbuild #53491: no failures (partial run)build #53503: shard failuresbuild #53570: shard failuresbuild #53583: shard failuresbuild #53599: shard failuresbuild #53601: shard failuresbuild #53748: shard failuresbuild #53753: all shards passedbuild #53757: no failures (partial run)build #53759: no failures (partial run)build #53787: no failures (partial run)build #53811: all shards passedbuild #53952: no failures (partial run)build #53992: shard failuresbuild #53999: shard failuresbuild #54007: no failures (partial run)build #54012: all shards passedbuild #54015: shard failuresbuild #54017: all shards passedbuild #54022: shard failuresbuild #54026: no failures (partial run)build #54030: shard failuresbuild #54033: all shards passedbuild #54040: shard failuresbuild #54047: shard failuresbuild #54049: shard failuresbuild #54055: shard failuresbuild #54057: shard failuresbuild #54064: shard failuresbuild #54074: shard failuresbuild #54093: shard failuresbuild #54161: shard failuresbuild #54186: shard failuresbuild #54189: shard failuresbuild #54196: shard failuresbuild #54202: all shards passed

Windows x64 · 8 shards

build #53090: shard failuresbuild #53094: shard failuresbuild #53095: shard failuresbuild #53106: shard failuresbuild #53109: shard failuresbuild #53123: shard failuresbuild #53127: shard failuresbuild #53130: shard failuresbuild #53131: shard failuresbuild #53133: shard failuresbuild #53134: shard failuresbuild #53135: shard failuresbuild #53143: shard failuresbuild #53149: shard failuresbuild #53159: shard failuresbuild #53164: shard failuresbuild #53167: shard failuresbuild #53172: shard failuresbuild #53176: shard failuresbuild #53188: shard failuresbuild #53194: shard failuresbuild #53208: shard failuresbuild #53212: shard failuresbuild #53213: shard failuresbuild #53214: shard failuresbuild #53216: shard failuresbuild #53222: shard failuresbuild #53229: shard failuresbuild #53236: shard failuresbuild #53241: shard failuresbuild #53260: shard failuresbuild #53265: shard failuresbuild #53271: shard failuresbuild #53280: shard failuresbuild #53298: shard failuresbuild #53304: shard failuresbuild #53327: all shards passedbuild #53340: all shards passedbuild #53360: no failures (partial run)build #53419: shard failuresbuild #53431: shard failuresbuild #53458: shard failuresbuild #53470: no failures (partial run)build #53485: shard failuresbuild #53491: no failures (partial run)build #53503: shard failuresbuild #53514: no failures (partial run)build #53565: no failures (partial run)build #53570: shard failuresbuild #53599: shard failuresbuild #53745: shard failuresbuild #53748: shard failuresbuild #53753: shard failuresbuild #53757: shard failuresbuild #53759: shard failuresbuild #53762: shard failuresbuild #53769: shard failuresbuild #53781: shard failuresbuild #53787: shard failuresbuild #53808: shard failuresbuild #53811: shard failuresbuild #53852: shard failuresbuild #53863: shard failuresbuild #53883: shard failuresbuild #53893: shard failuresbuild #53914: all shards passedbuild #53933: all shards passedbuild #53952: all shards passedbuild #53973: no failures (partial run)build #53983: shard failuresbuild #53992: shard failuresbuild #53999: shard failuresbuild #54002: no failures (partial run)build #54004: no failures (partial run)build #54007: shard failuresbuild #54008: no failures (partial run)build #54012: all shards passedbuild #54015: all shards passedbuild #54017: all shards passedbuild #54022: shard failuresbuild #54026: all shards passedbuild #54030: all shards passedbuild #54033: all shards passedbuild #54040: all shards passedbuild #54047: shard failuresbuild #54049: shard failuresbuild #54055: shard failuresbuild #54057: shard failuresbuild #54064: all shards passedbuild #54074: all shards passedbuild #54083: all shards passedbuild #54093: all shards passedbuild #54144: shard failuresbuild #54161: shard failuresbuild #54186: shard failuresbuild #54189: shard failuresbuild #54196: shard failuresbuild #54202: all shards passed

Windows arm64 · 8 shards

build #53090: shard failuresbuild #53095: shard failuresbuild #53106: shard failuresbuild #53109: shard failuresbuild #53123: shard failuresbuild #53127: shard failuresbuild #53130: shard failuresbuild #53131: shard failuresbuild #53134: shard failuresbuild #53135: shard failuresbuild #53149: shard failuresbuild #53159: shard failuresbuild #53164: shard failuresbuild #53167: shard failuresbuild #53172: shard failuresbuild #53176: shard failuresbuild #53188: shard failuresbuild #53194: shard failuresbuild #53208: shard failuresbuild #53212: shard failuresbuild #53213: shard failuresbuild #53214: shard failuresbuild #53216: shard failuresbuild #53222: shard failuresbuild #53229: shard failuresbuild #53236: no failures (partial run)build #53241: shard failuresbuild #53260: shard failuresbuild #53265: shard failuresbuild #53271: shard failuresbuild #53304: shard failuresbuild #53327: all shards passedbuild #53340: all shards passedbuild #53360: no failures (partial run)build #53419: no failures (partial run)build #53431: shard failuresbuild #53458: no failures (partial run)build #53485: shard failuresbuild #53491: no failures (partial run)build #53503: no failures (partial run)build #53599: shard failuresbuild #53748: shard failuresbuild #53753: shard failuresbuild #53757: shard failuresbuild #53759: shard failuresbuild #53762: shard failuresbuild #53787: shard failuresbuild #53808: shard failuresbuild #53811: shard failuresbuild #53852: shard failuresbuild #53863: shard failuresbuild #53883: shard failuresbuild #53893: shard failuresbuild #53914: no failures (partial run)build #53933: all shards passedbuild #53952: all shards passedbuild #53983: shard failuresbuild #53992: shard failuresbuild #53999: shard failuresbuild #54007: no failures (partial run)build #54012: all shards passedbuild #54015: all shards passedbuild #54017: all shards passedbuild #54022: shard failuresbuild #54026: all shards passedbuild #54030: no failures (partial run)build #54033: all shards passedbuild #54040: all shards passedbuild #54047: shard failuresbuild #54049: shard failuresbuild #54055: no failures (partial run)build #54057: shard failuresbuild #54064: all shards passedbuild #54074: all shards passedbuild #54083: no failures (partial run)build #54093: all shards passedbuild #54144: shard failuresbuild #54161: shard failuresbuild #54186: shard failuresbuild #54189: shard failuresbuild #54196: shard failuresbuild #54202: all shards passed

跨 135 次实际运行测试的 CI build(从 BuildKite 的 420 个 build 中挖出)中,每个平台的每个 test shard。亮绿色:每个 shard 都通过。暗绿色:没有失败,但运行被提前截断(被 superseded)。红色:至少一个 shard 失败。每条泳道都标注了它的完整套件第一次通过的时间。Linux 的 60 个 shard 比 Windows 早了将近一天全绿。各个平台一直会抖成红色,直到最后几个失败测试消失;最终全绿的 build 是 #54202。

合并前剩下的时间就很直接了。一个 workflow 按平台循环修复 CI 测试失败,直到没有测试失败为止。还有几个 workflow 做 Windows 相关清理、去重代码、减少 unsafe 使用,以及整体清理一些代码。

合并 Rust 重写

当 Bun 的测试套件在 CI 的所有平台上 100% 通过后(我也手动确认测试确实在运行,没有被 skip),我在本地跑了一堆命令测试,然后按下了 merge 按钮。

合并到 main 不是版本化发布。到这个节点,我已经足够有信心继续推进,并承诺采用这次重写;但还没有足够信心发布它。

统计数据

峰值时,我们同时运行 4 个这样的 workflow,每个 workflow 在单独的 worktree 中,每个 workflow 有 16 个 Claude。大约同时运行 64 个 Claude。

git log · claude/phase-a-portpeak: 58 commits in one minute

0

commits

+0

lines written, rewrites included

Mon, May 4, 7:05 AM PDT

first 100-file draft batchPR #30412 openedmerged

全部 6502 个 commit(排除 merge)的回放。粉色柱主要是新增代码;青色柱主要是删除。行数计数会把中间每次重写都算进去,最终落地的 diff 是 +1009272。日志是真实 commit message。

0 个测试被跳过或删除

11 天(5 月 3 日 → 5 月 14 日合并)· 6778 个 commit

Platform

expect() calls

Tests

Files

Debian 13 x64

1,386,826

60,624

4,174

macOS 14 arm64

1,259,953

58,850

4,175

Windows 2019 x64

1,007,544

57,337

4,173

合并前,这花掉了 59 亿 uncached input token、6.9 亿 output token,以及 720 亿 cached input token read,按 API 定价大约是 16.5 万美元。如果手工做,我认为需要 3 个对代码库有完整上下文的工程师大约一年;在那段时间里,我们无法改进 Node.js 兼容性、修 bug、修安全问题或实现新功能。我们绝不会这么做。现实中的替代方案,就是什么都不做,然后永远继续修这篇文章开头列出的那些 bug。

这是今天可能性边界上最前沿的东西。我使用了 Claude Fable 5 的预发布版本,这是一个 Mythos-class 模型。Claude Code 的 dynamic workflow 让 64 个 Claude 连续运行了 11 天(否则我就得自己写 harness 才能做到)。

工作还在继续

自从合并 Rust port 后,我们已经完成了 Claude Code Security 的 11 轮安全 review,并处理了发现的问题。

我们还给 Bun 的每个 parser 加上了 7x24 coverage-guided fuzzing:JavaScript、TypeScript、JSX、CSS、JSON5、JSONC、TOML、YAML、Markdown、INI、Bun Shell scripts、semver ranges、.patch files 和 CSS colors。fuzzer 会自动把发现的 bug 发给 Claude,由它提交复现和修复 PR,然后由人类 review PR。到目前为止,它已经执行我们的 parser 1000 亿次,带来了大约 15 个 PR。

在写作时,Bun 的 Rust 代码大约 4% 位于 unsafe block 中(在约 780000 行里,有约 13000 个 unsafe keyword,分布在约 27000 行),其中 78% 的 block 只有一行:一个来自 C++ 的 pointer,或一次对 C library 的调用。随着我们从忠实的 Zig port(它没有可 grep 的 unsafe keyword)重构为惯用 Rust,我预计这个数字会随时间下降;但我们会继续使用 JavaScriptCore 这样的 C 和 C++ 库,所以它永远会比纯 Rust 项目有更多 unsafe

移植错误

Rust 重写的重点是稳定性,但发布这么大的变更,不引入任何回归是不可能的。

这次重写引入了 19 个已知回归,每个都已经修复。

大多数回归来自两门语言中语法完全相同、语义却不同的代码。

debug_assert! 里的副作用

这两个片段看起来相似,但行为不同。Zig 的 assert 是函数,所以它的参数在每个 build 中都会运行。Rust 的 debug_assert! 是 macro,所以在 release build 中,整个表达式都会被擦除,包括 insert_stale 调用。

// Zig:
if (dev.framework.react_fast_refresh) |rfr| {
    assert(try dev.client_graph.insertStale(rfr.import_source, false) == IncrementalGraph(.client).react_refresh_index);
}

// Rust:
if let Some(rfr) = &dev.framework.react_fast_refresh {
    debug_assert!(dev.client_graph.insert_stale(&rfr.import_source, false)? == react_refresh_index);
}

insert_stale 会把一个文件加入 frontend dev server 的 hot reload graph。在 release build 中它不再运行,于是 HMR 在某些情况下坏掉:带有使用 React 的 HTML route 的项目中,当一个 hot reloaded 文件被 invalidated 时,会出现 Cannot destructure property 'isLikelyComponentType' of 'k'。Debug build 是正常的。#30678

奇数长度的 slice

Bun 的 Zig helper reinterpretSlice(u16, bytes)(早于 builtin cast 支持 slice)使用 @divTrunc,会忽略末尾多出来的奇数字节。bytemuck::cast_slice 则会 panic。Blob.text() 在遇到 UTF-16 byte order mark 后接奇数个字节时,不再返回字符串,而是让进程 panic。我们改回了忽略那个奇数字节:&buf[..buf.len() & !1]#31188

Bounds check

在 macOS 和 Linux 上,我们用 ReleaseFast 编译 Bun 的 Zig 代码,这会移除 bounds check。Rust 的 release build 会保留它们。

Bun 的 module resolver 会把长文件名 intern 到一个全局列表里,溢出后进入 overflow block。原来的 Zig 代码把每个 block 大小设为 count / 4,也就是 2048。移植时留下了一个 placeholder:

/// ... so use a nonzero stand-in until Phase B threads the
/// per-instantiation value through.
pub const BSS_OVERFLOW_BLOCK_SIZE: usize = 64;

这把上限从 840 万个 interned filename 降到了 270272 个,真实项目会撞到;同时也让我们从 Zig 移植过来的 ptrs[4095] off-by-one 变得可达。Rust 没有写越界,而是 panic 了。如果我们使用 ReleaseSafe,Zig 在这种情况下也会 panic(我们只在 Windows 上这么做)。#31503

comptime format string

Output.pretty 会把 <r><d> 颜色 marker 改写成 ANSI escape。在 Zig 中,fmtcomptime,所以 marker 在参数替换前就消失了。Rust function 没有 comptime 参数,所以 Output::pretty 只能看到完成后的字符串,也会把参数里的 marker 一起改写。

// Zig:
pub inline fn pretty(comptime fmt: string, args: anytype) void;
Output.pretty("<r>{f}<r>", .{hyperlink});

// Rust:
pub fn pretty(payload: impl PrettyFmtInput);
Output::pretty(format_args!("<r>{}<r>", hyperlink));

bun update -i 会把 package name 打印成 OSC 8 hyperlink,并用 ESC \ 结束。那个反斜杠正好在尾部 <r>< 前面,marker parser 吃掉了它,于是 r 被作为文本打印出来。

it should say oxfmt, not oxfmtr

在 Rust 中,它必须是 macro:bun_core::pretty!("<r>{}<r>", hyperlink)#30693

Rust 版 Bun 更好了

到目前为止,Bun v1.4.0 修复了 128 个可以在 v1.3.14 中复现的 bug,范围从内存泄漏到崩溃,再到帮助文本颜色错误。

内存使用减少

Rust 有一个强大的语言级内存清理工具:Drop。实现了 Drop 后,每当值离开作用域,drop 函数都会自动调用。

impl Drop for Bytes {
    fn drop(&mut self) {
        if !self.pinned.is_empty() {
            JSC__JSValue__unpinArrayBuffer(self.pinned);
        }
    }
}

在 Zig 中,defer 可用于在作用域结束时运行代码:

const bytes: ArrayBuffer = try .fromPinned(global, value);
defer bytes.unpin();

在 Zig 中,defer 需要添加到每一个可能需要清理的调用点。很容易最后忘了清理(内存泄漏),或者在很少走到的错误处理代码里把清理代码运行两次(double-free)。在 Rust 中,Drop 会在值不再可访问时自动运行,用“隐藏控制流”换掉一个常见 footgun。

Drop 修复了 Bun 中多个和错误处理代码中文件路径有关的内存泄漏。

我们修复了每一个可 instrument 的内存泄漏

我们改进了 Bun 的 LeakSanitizer 集成,用来追踪所有 native code memory allocation

这里有个例子:每次进程内调用 Bun.build() 都会泄漏数 MB 内存,也就是属于这次 build 的 parsed source text 和 AST symbol table,在 build 结束后仍然活着。

// Bundle the same 60-module project 2,000 times in one process
for (let i = 0; i < 2_000; i++) {
  await Bun.build({
    entrypoints: ["./index.js"],
    minify: true,
    sourcemap: "external",
  });
}

在 Bun v1.3.14 中,每次 build 都会永久泄漏大约 3 MB。像 dev server 这种每个 request 都会 bundle 的工具,最终会耗尽内存。在 Bun v1.4.0 中,内存会趋于平稳:

Builds

Bun v1.3.14

Bun v1.4.0

500

1,914 MB

526 MB

1,000

3,506 MB

586 MB

1,500

5,097 MB

608 MB

2,000

6,745 MB

609 MB

此前有一次在 Zig 中做这件事的尝试没有 merge,因为缺少等价于 Drop 的机制,让人更难有信心合并。

二进制体积更小

Rust 重写的初始变更让 Windows 上的二进制体积减少了 3.8 MB,macOS 上减少 5.5 MB,Linux 上减少 6.8 MB。这主要是因为我们在 Zig 代码里用了太多 comptime

初始缩小之后,团队继续探索更多减少二进制体积的机会,例如 Identical Code Folding 这样的 linker optimization、移除 ICU 中未使用的数据,以及用 zstd dictionary 按需懒解压 libicu 的小部分内容。

结合 Rust 重写、ICU 变更和 identical code folding,Bun 的二进制体积在 Linux 和 Windows 上缩小了约 20%

Version

Platform

Size

Bun v1.4.0 (canary)

Windows

76 MB

Bun v1.3.14

Windows

94 MB

Bun v1.4.0 (canary)

Linux

70 MB

Bun v1.3.14

Linux

88 MB

栈空间使用减少

TOML parser,以及 Bun 中所有其他 recursive-descent parser(JSON、YAML、JavaScript、TypeScript 等),现在使用的栈空间都更少了。

这在合并 Rust 重写前导致了一些测试失败:

bun test v1.3.14-canary.1 (e99311e58)
.......

105 | });
106 |
107 | it("Bun.TOML.parse throws on deeply nested inline tables instead of crashing", () => {
108 |   const depth = 25_000;
109 |   const deepToml = "a = " + "{ b = ".repeat(depth) + "1" + " }".repeat(depth);
110 |   expect(() => Bun.TOML.parse(deepToml)).toThrow(RangeError);
                                               ^
error: expect(received).toThrow(expected)

Expected constructor: RangeError

Received function did not throw
Received value: {
  a: {
    b: {
      b: {
        b: {
          b: {
            b: {
              b: {
                b: {
                  b: [Object ...],
                },
              },
            },
          },
        },
      },
    },
  },
}

      at <anonymous> (/var/lib/buildkite-agent/build/test/js/bun/resolve/toml/toml.test.js:110:42)

✗ Bun.TOML.parse throws on deeply nested inline tables instead of crashing [2907.64ms]

Rust 的 LLVM IR codegen 会在 stack variable 不再使用时发出 LLVM 的 llvm.lifetime.startllvm.lifetime.end intrinsic,让 LLVM 可以复用栈空间 slot。这让带有嵌套作用域的大函数显著减少栈空间使用。

此前,我们通过把特别大的函数重构成许多小函数,手动绕过 一个未解决的问题

快 2% 到 5%

Rust 支持 C/C++ 和 Rust 之间的跨语言 link-time optimization,这让跨语言 inline 成为可能(这也太酷了!)。

我们在 Linux x64(EC2,Xeon Platinum 8488C)上对 Bun v1.3.14 和 Bun v1.4.0 做了基准测试。HTTP throughput 用 oha 压测 hello-world server,app workload 用 hyperfine 测量。

HTTP throughput(req/s,3 轮平均)

server

Bun v1.3.14

Bun v1.4.0

Δ

Bun.serve

169.6k

177.7k

+4.8%

node:http

103.8k

108.5k

+4.5%

Elysia

158.9k

163.3k

+2.8%

express

64.5k

66.6k

+3.2%

fastify

91.5k

95.9k

+4.8%

Apps / CLI(hyperfine)

workload

Bun v1.3.14

Bun v1.4.0

Δ

next build

13.62 s

13.03 s

+4.5%

vite build (tsc + vite)

1.69 s

1.65 s

+2.2%

tsc -b --force

0.94 s

0.89 s

+4.7%

Production

Prisma 在 Bun 的 Rust 重写版上发布了 Prisma Compute public beta。

“我们遇到过内存泄漏,以及一个连接池在 VM pause 后 resume 时无法恢复的问题。Rust 重写出现后,我们用同样的 failure mode 测试了它。它处理得非常完美。”——Alexey Orlenko

Claude Code v2.1.181(6 月 17 日发布)及之后的版本使用 Rust port 的 Bun。在 Linux 上启动速度快了 10%,除此之外几乎没人注意到。无聊是好事。

Claude Code startup time from production telemetry (Linux p50): v2.1.179 at 517ms vs v2.1.181, the first release on Rust Bun, at 464ms — 10% faster

发布

Bun v1.3.14 是最后一个用 Zig 写的 Bun 版本。Bun v1.4.0 会是第一个用 Rust 写的 Bun 版本。它现在已经在 canary 中可用,请报告你发现的任何问题:

bun upgrade --canary

可维护性

对我和团队来说,新的 Rust 代码库感觉和旧的 Zig 代码库非常相似。例如,下面是原 Zig 代码和新 Rust 代码的一段:

pub fn canMergeSymbols(
    scope: *Scope,
    existing: Symbol.Kind,
    new: Symbol.Kind,
    comptime is_typescript_enabled: bool,
) SymbolMergeResult {
    if (existing == .unbound) {
        return .replace_with_new;
    }

    if (comptime is_typescript_enabled) {
        // In TypeScript, imports are allowed to silently collide with symbols within
        // the module. Presumably this is because the imports may be type-only:
        //
        //   import {Foo} from 'bar'
        //   class Foo {}
        //
        if (existing == .import) {
            return .replace_with_new;
        }

        // ...
    }

    // ...
}
pub fn can_merge_symbol_kinds<const IS_TYPESCRIPT_ENABLED: bool>(
    scope_kind: Kind,
    existing: symbol::Kind,
    new: symbol::Kind,
) -> SymbolMergeResult {

    if existing == symbol::Kind::Unbound {
        return SymbolMergeResult::ReplaceWithNew;
    }

    if IS_TYPESCRIPT_ENABLED {
        // In TypeScript, imports are allowed to silently collide with symbols within
        // the module. Presumably this is because the imports may be type-only:
        //
        //   import {Foo} from 'bar'
        //   class Foo {}
        //
        if existing == symbol::Kind::Import {
            return SymbolMergeResult::ReplaceWithNew;
        }

        // ...
    }

    // ...
}

任何理解原 Zig 代码的人,也能理解机械翻译出来的 Rust 代码。我 review 原始 Rust 重写 PR 的方式,是检查 adversarial code review agent 是否正确抓到了 Zig 代码和 Rust 代码之间的差异,是否确保 porting guide 和 lifetime guide 被遵循,同时我也自己把很多代码按 Zig vs Rust 并排手动读了一遍。

下一步

Bun v1.4 让 Bun 更快、更小、使用更少内存,并给团队提供了极其强大的工具来系统性地改进未来稳定性:Rust 的 borrow checker、Miri(CI 中正在覆盖越来越多代码)、LeakSanitizer,以及针对 parser 的 7x24 coverage-guided fuzzing。还有更多内容需要重构,但开局很好。

如果由一个对代码库有完整上下文的工程师团队来做,这次 Rust 重写会花一年。借助 Fable,并由 1 个工程师密切监控 Claude Code,我们从开始到所有平台上 100% 测试套件通过,只用了 11 天。

今天,一个工程师能做的事比一年前多得多。