披露:Bun 于 2025 年 12 月被 Anthropic 收购。我和 Bun 团队的其他成员现在都在 Anthropic 工作。在这次 Rust 重写的大部分过程中,我使用了 Claude Fable 5 的预发布版本。
Bun 最初是把 esbuild 的 JavaScript 和 TypeScript 转译器从 Go 逐行移植到 Zig。我在 2021 年 4 月 16 日 写下了第一行 Zig。当时我在 Hacker News 上看到单页的 Zig Language Reference ,被它对底层控制和性能的重视打动,于是押注了 Zig。
从一开始,Bun 的范围就非常大:
JavaScript、TypeScript 和 CSS 的转译器、压缩器和打包器
兼容 npm 的包管理器
类似 Jest 的测试运行器
兼容 Node.js 和 TypeScript 的模块解析
HTTP/1.1 和 WebSocket 客户端
fs、net、tls 以及几十个其他 Node.js API 模块的实现
Bun 的初版是我在一个狭小的 Oakland 公寓里,用 Zig 花 1 年写出来的,那时还没有 LLM。像 Bun 这种范围野心很大的项目,默认结局往往是躺进 GitHub 个人页上一堆死掉 side project 的坟场。Zig 让 Bun 成为了可能。如果没有 Zig,我绝不可能在 1 年里做出这么多东西。
今天,Bun CLI 每月下载量超过 2200 万。Claude Code 和 OpenCode 这样的流行工具选择 Bun 作为运行时。Vercel、Railway、DigitalOcean 等平台也都提供了对 Bun 的一方支持。
Bun 的范围也一直给稳定性带来挑战。下面是我们在 Bun v1.3.14 中修复的一小部分 bug:
在 threadpool 上仍有异步 .write() 进行时,对 zlib、Brotli 或 Zstd stream 调用 .reset(),会导致 node:zlib 中的 heap-use-after-free 崩溃
onerror 回调对 native handle 发起重入的 write() 后又调用 close(),导致 node:zlib 中的 use-after-free 崩溃
node:http2 中,重入的 JS 回调(例如 timeout listener、options getter 或 write callback 里调用 session.request())触发 hashmap rehash,使内部 stream 指针失效,导致 use-after-free 崩溃
UDPSocket.send() 和 sendMany() 中,用户代码在 valueOf() 或 toString() 回调里可以在捕获 payload 和真正发送之间 detach 一个 ArrayBuffer,导致 use-after-free
Buffer#copy 和 Buffer#fill 中,valueOf 回调在参数强制转换期间 detach 或 resize 底层 ArrayBuffer,导致崩溃和越界读取
UDPSocket.sendMany() 中,用户 JS 回调在迭代中途改变 socket 连接状态,导致 heap 越界写入
crypto.scrypt 中,如果输出 buffer 分配失败,callback 和受保护的 password/salt buffer 从未释放,导致内存泄漏
SSLWrapper.init 在错误路径上泄漏了 strdup 出来的 passphrase
tlsSocket.setSession() 中,由于 d2i_SSL_SESSION 后缺少 SSL_SESSION_free,每次调用都会泄漏一个 SSL_SESSION(每次约 6.5 KB)
fs.watch() watcher 在 .close() 后从未被垃圾回收:引用计数下溢让每个 watcher 永久固定为 GC root
CSS parser 在 background-clip 同时有 vendor prefix 和多层背景时发生 double-free 崩溃
DuplexUpgradeContext 从未释放,导致每次 tls.connect({ socket: duplex }) 都完整泄漏一次
MessageEvent 中的竞态崩溃:GC marker thread 在来自 BroadcastChannel 或 MessagePort 的并发访问期间,可能观察到 m_data 中被撕裂的 variant
我们当然可以永远一个一个地修这些 bug,但依赖我们的用户值得更好的结果:我们应该系统性地防止这类 bug 反复出现。
我们 patch 了 Zig 编译器,加入 Address Sanitizer 支持。每次提交都会用 ASAN 跑测试套件。
我们在 Windows 上发布启用 Zig safety check 的 ReleaseSafe 构建
我们用 Fuzzilli 7x24 fuzz Bun 的 runtime API;这是 V8 和 JavaScriptCore 使用的 JavaScript engine fuzzer
我们有大量端到端内存泄漏测试
这已经比很多项目做得更多了。
那份 bugfix 列表让我很难受。我也厌倦了睡前还在担心 Bun 会崩溃。我不怪 Zig,其他 Zig 用户并没有我们遇到的这些 bug,而把 GC 和手动管理内存混在一起,本来就是软件里少见到足以让语言专门为它设计的需求。如果没有 Zig,我们走不到今天,我也会一直感谢它。直到很近之前,对 Bun 这样的项目来说,编程语言选择还是一条单行道。
JavaScript 是一门带垃圾回收的语言,JavaScriptCore(以及 V8)这样的现代 JavaScript 引擎对异常处理和垃圾回收器有严格规则。Zig 和 C 一样,不替你管理内存;对很多项目来说,这种取舍正是使用 Zig 的好理由。Zig 没有构造函数/析构函数,大多数清理逻辑都预期在每个调用点用 defer 显式写出来。
对 Bun 来说,正确处理垃圾回收值和手动管理值的生命周期,一直是稳定性问题的主要来源,最常见的是小内存泄漏,偶尔是崩溃。每一次内存分配都必须仔细 review:这些字节在哪里释放?怎么保证只释放一次?JavaScript 异常有没有正确检查?这个 GC 指针对 conservative stack scanner 可见吗?这块内存是 GC 管理的,还是手动管理的?
对稳定性问题来说,越早知道越好。Fuzzing 发生在代码合并之后。CI 发生在代码 push 之后。运行时 safety check 和 address sanitizer 发生在代码运行时(理想情况下是在开发阶段、CI 之前)。
减少这类问题的常见方法之一,是确保需要清理的代码永远恰好运行一次。Zig 被设计成一门简单、没有隐藏控制流的语言,所以它更偏向用显式的 defer 关键字在作用域结束时运行代码,而不是 C++ 隐式的 ~Destructor 或 Rust 隐式的 Drop。
Language
Cleanup
Zig
defer, errdefer
C++
~Destructor, &&Move
Rust
Drop
对 Zig 代码来说,我们到底应该在什么时候运行清理代码?如果把同一个 *T 传给很多不同函数,怎么知道它已经不再可访问,可以清理了?如果某些函数在被调用后还需要继续引用那段内存,又该怎么办?我们目前的方法是几种东西混在一起:
arena 生命周期:可访问范围很清楚时使用(parser state 不会逃出调用函数,因此 AST node 很适合)
引用计数
非常非常小心
很多项目选择用 style guide 回答这类问题。TigerBeetle 的 TigerStyle 是 Zig 中的一个例子,Google 31000 词的 C++ style guide 是另一个例子。style guide 的挑战在于执行。你怎么确保 style guide 被遵守?历史上,答案是 code review,再辅以 linter 和 static analyzer 的 best-effort 检查。
对 Bun 来说,写一份严格的 style guide,并把清晰的 ownership 预期显式编码进类型系统,确实是一个现实选择。由于 Zig 没有 operator overloading,我们很可能最终会得到大量类似这样的代码:
fn foo(a_ptr: SharedPtr(TCPSocket)) !void {
const a: *TCPSocket = a_ptr.get();
defer a_ptr.deref();
const b = try do_something_with_a(a);
defer b.deref();
// ...
}
这比我们期望的 Zig 写法更不顺手:
fn foo(a: *TCPSocket) !void {
const b = try do_something_with_a(a);
// ...
}
Bun 大约 20% 的代码是 C++,并且 Bun 嵌入了几个 C/C++ 库:
JavaScriptCore,Safari 背后的 JavaScript 引擎
uWebSockets 和 usockets,我们的 HTTP/WebSocket server 和 event loop
lshpack 和 lsquic,HPACK 和 HTTP/3 库
BoringSSL,Google 的 OpenSSL fork
SQLite
对 Bun 来说,用 C++ 代替 Zig 会是一个合理选择。我们会得到构造函数和析构函数,也可以删除大量 extern "C" wrapper 代码。
但是,我们仍然会依赖通过 code review 执行的 style guide。即使用了 ASAN,内存破坏和内存泄漏仍然会发生。
上面那份列表里的大量 bug 都是 use-after-free、double-free,以及错误路径上“忘记 free”。在 safe Rust 里,这些都是编译器错误,并且 Drop 会提供类似 RAII 的自动清理。编译器错误比 style guide 有更好的反馈循环。
历史上,重写通常是个糟糕主意。不算注释,Bun 有 535496 行 Zig。用另一门语言重写,需要一个小工程师团队花整整一年。这意味着在这段时间里冻结 bugfix、安全修复或功能开发。想得到一个可发布版本,风险最低的方法是从 Zig 到 Rust 做机械式移植,尽量少改行为,并使用我们已经用于测试 Bun 的同一套测试套件。
幸运的是,Bun 自己的测试套件是用 TypeScript 写的,这意味着它不依赖 runtime 的实现语言。
一年没有任何用户可见影响不是我们能考虑的现实选项。所以,通过 code style 强制约束来修稳定性问题,是我们最好的选择;这也是我们给 Bun 代码库加入 Rust 风格 smart pointer 时的计划。
但说实话,我不想这么做。自研 smart pointer 的 ergonomics 比 Rust 更差,却没有 Rust 的保证。
如果我换个方向,花一周测试 Anthropic 的新模型能不能把 Bun 重写成 Rust 呢?
一开始,我并不期待它能成功。几天后,测试套件开始有很高比例通过,我也看到新的 Rust 代码和原来的 Zig 代码库高度对应。我的看法从“这值得一试”变成了“我要把它 merge 进去”。
有很多方法可以把这件事做得很糟。比如 prompt Claude:“Rewrite Bun in Rust. Don't make any mistakes.” 然后祈祷它能工作。这不是我做的事。
想想一个人会怎么做。第一个大问题是:
增量重写?还是一次性全部重写?
以我当初在没有 LLM 的情况下,把 esbuild 的 transpiler 从 Go 移植到 Zig 来做 Bun 初版的经验看,一次性全部重写更好。增量重写会加入临时代码,你希望它们最终被删除,但在短期到中期都会很痛苦。
第二个大问题是:怎么做?
我们怎样让 Rust 版 Bun 仍然是以前那个 Bun,保持同样的架构、性能和功能集,同时获得 Rust 的 borrow checker 等语言特性?怎样确保团队在重写后仍然能够维护它?
做一次看起来像把 Zig 代码转译成 Rust 的重写。Bun v1.4 发布后,我们可以逐步重构,减少 unsafe 使用,让它更像惯用 Rust。
大的问题只有这两个。其他都是战术。
软件工程师的大量日常工程工作,都可以被过度简化成循环。
// Pseudocode, not real code:
let task;
while ((task = todoList.pop())) {
const result = task();
const feedback = await Promise.all([review(result), review(result)]);
await apply(feedback, result);
}
一个 task 带着一些上下文(Jira ticket、GitHub issue 等)。result 是你为修它写的代码。code reviewer 调用 review 检查变更是否引入回归、是否正确。然后你处理反馈。
我用 Claude Code 中大约 50 个 dynamic workflow 持续运行了 11 天,把 Bun 重写成了 Rust。
每个 dynamic workflow 都是这样的循环,例如:
生成一份 porting guide,把 Zig pattern 和 type 映射到 Rust pattern 和 type
机械式地把每个 .zig 文件移植成 .rs 文件,遵循 PORTING.md 和 LIFETIMES.tsv
修复每个 crate 的编译错误
让 bun test 或 bun build 这样的子命令工作
让 Bun 的完整测试套件全部通过
几个大型重构和清理 pass
在这 11 天的大部分时间里(以及之后),我都在监控 workflow:手动阅读输出,检查问题和 bug,并 prompt Claude 修改循环来修复问题。
你要怎么 review 一个新增超过 100 万行的 PR?你要如何建立足够信心,负责任地合并大量 LLM 写出的代码?
答案是:一个和语言无关、拥有百万级断言的测试套件;adversarial code review;以及当问题发生时,修复生成代码的流程,而不是手动修代码。
Adversarial review 会要求 Claude(在单独的 context window 中)穷尽式地提出这些变更会产生 bug 或无法工作的理由。
拆分 context window
通常对人来说,review 代码的人不是写代码的人。写代码的人想把代码 merge 进去,这可能让他们倾向于在代码还没准备好时就发出去。
Claude 也是这样。写代码的 Claude 想让代码被接受。review 的 Claude 想找出代码里的问题。
每个 implementer 配 2 个或更多 adversarial reviewer。reviewer 的唯一任务是:找 bug,找代码为什么不能工作的理由。implementer 不 review。reviewer 不 implement。
✻ claude code · dynamic workflowadversarial review3 of the many bugs adversarial review caught before merge
bug 1 of 3 · the async close
✻claudeimplementer
its context: the .zig original, the port plan, its own reasoning
✻claudeadversarial reviewer
its context: only the diff. told to assume the code is wrong.
✻
src/runtime/api/bun/js_bun_spawn_bindings.rs · compiles clean
for stdio in [spawned_stdout, spawned_stderr] {
match stdio {
StdioResult::Buffer(mut pipe) => {
// pipe: Boxuv::Pipe — hand it to libuv to close
pipe.close(Subprocess::on_pipe_close)
}
StdioResult::Fd(fd) => fd.close(),
StdioResult::Unavailable => {}
}
}
✻
uv_close 是异步的:libuv 会一直持有 raw handle pointer,直到下一个 loop tick,然后调用 on_pipe_close 释放这次分配。但 pipe 是一个 Box,它会在这个 match arm 结束时 drop。于是 libuv 手里留下了已经释放的内存,而 close callback 又会再释放一次。先 use-after-free,再 double-free。
✻
Box::leak(pipe).close(Subprocess::on_pipe_close)
f0a454376c7 · win-review: js_bun_spawn_bindings.rs leak Boxuv::Pipe before async uv_close to avoid UAF/double-free in on_pipe_close
这是 adversarial reviewer 真正抓到的三个 bug。每个被引用的 commit subject 里都带着 review attribution。这三个 bug 都能编译,三个看起来都合理。reviewer 是另一个 Claude,运行在自己的 context window 里:它只拿到 diff,除此之外什么都没有,不知道 implementer 的推理;它被要求找出代码错在哪里。这里的代码从引用 commit 中压缩而来,bug 和修复是同一个。
如果你准备做一件又大又贵的事,先降低风险能省时间也省钱。
在写任何代码之前,我花了大约 3 个小时和 Claude 讨论如何把我们 Zig 代码库里的 pattern 尽量贴近地映射到 Rust。Claude 把这次讨论序列化成一份 PORTING.md 文档,后来还上了 Hacker News 。
下一个问题是:怎样给手动管理内存的代码加上 Rust lifetime?
这时我大概这样 prompt Claude:
我:让我们启动一个 dynamic workflow,分析代码库里每个 struct field 的正确 lifetime。这个 workflow 应该读取每个文件里的每个 struct field,并追踪控制流。首先找出那些在 Rust 中难以表达复杂 lifetime 的 struct field,然后为这个 field 提出一个 lifetime,再用 2 个 adversarial review agent review 这个 lifetime,然后应用所有反馈,并序列化成 LIFETIMES.tsv,供其他 Claude 查看。
然后对 PORTING.md 和 LIFETIMES.tsv 一起做一轮 adversarial review,修复所有互相冲突的建议,并再次检查所有内容。我也手动读了一遍。
在要求 Claude 把全部 1448 个 .zig 文件翻译成 .rs 文件之前,我先只试了 3 个文件。对这 3 个文件中的每一个,1 个 implementer 写新的 .rs 文件,2 个 adversarial reviewer 检查这个 .rs 文件是否和 .zig 文件行为一致,且是否遵循 PORTING.md 和 LIFETIMES.tsv。之后,1 个 fixer 应用所有建议。
我让 Claude 在全部 1448 个 .zig 文件上循环跑 workflow。大约 2 分钟后,一个 Claude 在 commit 前运行了 git stash。另一个运行了 git stash pop。然后又来了 git reset HEAD --hard。它们互相踩脚了!如果我把每个 Claude 放进单独的 worktree,又会用光磁盘空间,因为 Bun 的 git 仓库太大,而且最终这些变更还需要一起编译和观察。
所以,我让 Claude 修改 workflow,指示 Claude 永远不要运行 git stash 或 git reset,也不要运行任何不能一次性 commit 特定文件的 git 命令。也不要跑 cargo。完全不要跑慢命令。
然后 Claude 恢复了 workflow。它开始工作了!只是太慢,所以我把它拆成 4 个 workflow shard,每个 shard 有自己的 worktree(总共 4 个 worktree),每个 shard 运行 16 个 Claude 来提交和 push 文件。
得益于所有这些并行化和准备工作,在峰值时 Claude 每分钟写下大约 1300 行代码。每一行代码都由两个独立的 adversarial reviewer(也是 Claude)review,并在提交前经过一轮修复。当时它们完全不能工作。
11 days × 24 hours · PDT
6,502 commits
1695 commits/hour
12am 6am 12pm 6pm May 4 May 4, 7am–8am PDT — 6 commits, +89,278 lines May 4, 8am–9am PDT — 2 commits, +50,742 lines May 4, 9am–10am PDT — 1 commit, +28,149 lines May 4, 11am–12pm PDT — 1 commit, +39,752 lines May 4, 12pm–1pm PDT — 3 commits, +251,616 lines May 4, 1pm–2pm PDT — 2 commits, +161,724 lines May 4, 3pm–4pm PDT — 3 commits, +136,381 lines May 4, 5pm–6pm PDT — 5 commits, +895 lines May 4, 6pm–7pm PDT — 5 commits, +17,027 lines May 4, 7pm–8pm PDT — 1 commit, +106 lines May 4, 9pm–10pm PDT — 13 commits, +11,661 lines May 4, 11pm–12am PDT — 6 commits, +8,516 lines May 5 May 5, 12am–1am PDT — 9 commits, +1,381 lines May 5, 1am–2am PDT — 7 commits, +1,577 lines May 5, 2am–3am PDT — 4 commits, +2,035 lines May 5, 3am–4am PDT — 4 commits, +7,808 lines May 5, 4am–5am PDT — 1 commit, +2,796 lines May 5, 5am–6am PDT — 2 commits, +29,370 lines May 5, 8am–9am PDT — 2 commits, +7,076 lines May 5, 9am–10am PDT — 2 commits, +308 lines May 5, 11am–12pm PDT — 2 commits, +1,643 lines May 5, 12pm–1pm PDT — 4 commits, +1,452 lines May 5, 1pm–2pm PDT — 1 commit, +2,142 lines May 5, 2pm–3pm PDT — 4 commits, +7,787 lines May 5, 3pm–4pm PDT — 2 commits, +5,835 lines May 5, 4pm–5pm PDT — 1 commit, +3,417 lines May 5, 5pm–6pm PDT — 4 commits, +3,960 lines May 5, 6pm–7pm PDT — 4 commits, +9,179 lines May 5, 7pm–8pm PDT — 4 commits, +1,983 lines May 5, 8pm–9pm PDT — 4 commits, +18,902 lines May 5, 9pm–10pm PDT — 43 commits, +40,650 lines May 5, 10pm–11pm PDT — 139 commits, +64,842 lines May 5, 11pm–12am PDT — 141 commits, +34,814 lines May 6 May 6, 12am–1am PDT — 60 commits, +10,417 lines May 6, 1am–2am PDT — 296 commits, +38,530 lines May 6, 2am–3am PDT — 306 commits, +18,836 lines May 6, 3am–4am PDT — 196 commits, +10,245 lines May 6, 4am–5am PDT — 86 commits, +2,655 lines May 6, 5am–6am PDT — 16 commits, +289 lines May 6, 8am–9am PDT — 5 commits, +264 lines May 6, 9am–10am PDT — 458 commits, +16,409 lines May 6, 10am–11am PDT — 695 commits, +44,000 lines May 6, 11am–12pm PDT — 102 commits, +21,972 lines May 6, 12pm–1pm PDT — 19 commits, +2,891 lines May 6, 1pm–2pm PDT — 3 commits, +56 lines May 6, 3pm–4pm PDT — 64 commits, +3,606 lines May 6, 4pm–5pm PDT — 264 commits, +60,132 lines May 6, 5pm–6pm PDT — 268 commits, +40,953 lines May 6, 6pm–7pm PDT — 281 commits, +16,283 lines May 6, 7pm–8pm PDT — 258 commits, +26,654 lines May 6, 8pm–9pm PDT — 327 commits, +16,599 lines May 6, 9pm–10pm PDT — 74 commits, +8,331 lines May 6, 10pm–11pm PDT — 17 commits, +2,200 lines May 6, 11pm–12am PDT — 11 commits, +3,590 lines May 7 May 7, 12am–1am PDT — 17 commits, +6,577 lines May 7, 1am–2am PDT — 22 commits, +8,718 lines May 7, 2am–3am PDT — 21 commits, +11,392 lines May 7, 3am–4am PDT — 53 commits, +6,476 lines May 7, 4am–5am PDT — 31 commits, +2,356 lines May 7, 5am–6am PDT — 9 commits, +1,787 lines May 7, 6am–7am PDT — 4 commits, +580 lines May 7, 7am–8am PDT — 5 commits, +181 lines May 7, 11am–12pm PDT — 3 commits, +421 lines May 7, 12pm–1pm PDT — 1 commit, +13 lines May 7, 1pm–2pm PDT — 5 commits, +248 lines May 7, 2pm–3pm PDT — 9 commits, +2,131 lines May 7, 3pm–4pm PDT — 51 commits, +3,207 lines May 7, 4pm–5pm PDT — 56 commits, +2,647 lines May 7, 5pm–6pm PDT — 159 commits, +2,787 lines May 7, 6pm–7pm PDT — 42 commits, +1,590 lines May 7, 7pm–8pm PDT — 46 commits, +4,170 lines May 7, 8pm–9pm PDT — 52 commits, +2,113 lines May 7, 9pm–10pm PDT — 27 commits, +1,585 lines May 7, 10pm–11pm PDT — 27 commits, +2,231 lines May 7, 11pm–12am PDT — 30 commits, +4,987 lines May 8 May 8, 12am–1am PDT — 27 commits, +1,196 lines May 8, 1am–2am PDT — 14 commits, +904 lines May 8, 2am–3am PDT — 8 commits, +536 lines May 8, 3am–4am PDT — 13 commits, +253 lines May 8, 4am–5am PDT — 3 commits, +771 lines May 8, 5am–6am PDT — 15 commits, +1,545 lines May 8, 6am–7am PDT — 12 commits, +1,965 lines May 8, 7am–8am PDT — 14 commits, +1,866 lines May 8, 8am–9am PDT — 55 commits, +3,622 lines May 8, 9am–10am PDT — 35 commits, +4,778 lines May 8, 10am–11am PDT — 1 commit, +0 lines May 8, 12pm–1pm PDT — 1 commit, +116 lines May 8, 1pm–2pm PDT — 2 commits, +66 lines May 8, 2pm–3pm PDT — 9 commits, +1,071 lines May 8, 3pm–4pm PDT — 26 commits, +1,691 lines May 8, 4pm–5pm PDT — 18 commits, +2,751 lines May 8, 5pm–6pm PDT — 2 commits, +97 lines May 8, 6pm–7pm PDT — 2 commits, +135 lines May 8, 7pm–8pm PDT — 11 commits, +1,763 lines May 8, 8pm–9pm PDT — 20 commits, +5,272 lines May 8, 9pm–10pm PDT — 12 commits, +952 lines May 8, 10pm–11pm PDT — 2 commits, +334 lines May 8, 11pm–12am PDT — 6 commits, +2,033 lines May 9 May 9, 12am–1am PDT — 9 commits, +387 lines May 9, 1am–2am PDT — 9 commits, +723 lines May 9, 2am–3am PDT — 8 commits, +98 lines May 9, 3am–4am PDT — 63 commits, +2,538 lines May 9, 4am–5am PDT — 11 commits, +8,861 lines May 9, 5am–6am PDT — 4 commits, +42 lines May 9, 6am–7am PDT — 3 commits, +2,616 lines May 9, 7am–8am PDT — 6 commits, +6,993 lines May 9, 8am–9am PDT — 1 commit, +3,705 lines May 9, 9am–10am PDT — 11 commits, +199 lines May 9, 11am–12pm PDT — 1 commit, +23 lines May 9, 12pm–1pm PDT — 4 commits, +5,012 lines May 9, 1pm–2pm PDT — 7 commits, +2,080 lines May 9, 2pm–3pm PDT — 6 commits, +924 lines May 9, 3pm–4pm PDT — 5 commits, +248 lines May 9, 4pm–5pm PDT — 17 commits, +508 lines May 9, 5pm–6pm PDT — 2 commits, +135 lines May 9, 6pm–7pm PDT — 4 commits, +822 lines May 9, 7pm–8pm PDT — 1 commit, +7 lines May 10 May 10, 12am–1am PDT — 4 commits, +497 lines May 10, 1am–2am PDT — 2 commits, +35 lines May 10, 2am–3am PDT — 1 commit, +131 lines May 10, 3am–4am PDT — 2 commits, +322 lines May 10, 4am–5am PDT — 1 commit, +3 lines May 10, 5am–6am PDT — 1 commit, +26 lines May 10, 6am–7am PDT — 2 commits, +81 lines May 10, 7am–8am PDT — 1 commit, +5 lines May 10, 8am–9am PDT — 4 commits, +78 lines May 10, 9am–10am PDT — 1 commit, +1 lines May 10, 10am–11am PDT — 2 commits, +128 lines May 10, 11am–12pm PDT — 1 commit, +4 lines May 10, 12pm–1pm PDT — 2 commits, +413 lines May 10, 1pm–2pm PDT — 1 commit, +25 lines May 10, 2pm–3pm PDT — 5 commits, +327 lines May 10, 3pm–4pm PDT — 6 commits, +1,172 lines May 10, 4pm–5pm PDT — 4 commits, +752 lines May 10, 5pm–6pm PDT — 3 commits, +227 lines May 10, 6pm–7pm PDT — 2 commits, +242 lines May 10, 7pm–8pm PDT — 1 commit, +306 lines May 10, 8pm–9pm PDT — 1 commit, +54 lines May 10, 9pm–10pm PDT — 2 commits, +75 lines May 10, 10pm–11pm PDT — 1 commit, +134 lines May 10, 11pm–12am PDT — 5 commits, +103 lines May 11 May 11, 12am–1am PDT — 2 commits, +150 lines May 11, 1am–2am PDT — 4 commits, +398 lines May 11, 2am–3am PDT — 2 commits, +364 lines May 11, 3am–4am PDT — 3 commits, +44 lines May 11, 4am–5am PDT — 7 commits, +9,367 lines May 11, 6am–7am PDT — 2 commits, +43 lines May 11, 7am–8am PDT — 2 commits, +149 lines May 11, 8am–9am PDT — 10 commits, +2,171 lines May 11, 9am–10am PDT — 16 commits, +2,047 lines May 11, 10am–11am PDT — 18 commits, +3,356 lines May 11, 11am–12pm PDT — 9 commits, +861 lines May 11, 12pm–1pm PDT — 3 commits, +412 lines May 11, 1pm–2pm PDT — 12 commits, +2,978 lines May 11, 2pm–3pm PDT — 157 commits, +10,700 lines May 11, 3pm–4pm PDT — 16 commits, +1,346 lines May 11, 4pm–5pm PDT — 3 commits, +78 lines May 11, 5pm–6pm PDT — 41 commits, +2,568 lines May 11, 6pm–7pm PDT — 55 commits, +4,912 lines May 11, 7pm–8pm PDT — 53 commits, +3,475 lines May 11, 8pm–9pm PDT — 32 commits, +1,732 lines May 11, 9pm–10pm PDT — 46 commits, +4,506 lines May 11, 10pm–11pm PDT — 45 commits, +1,711 lines May 11, 11pm–12am PDT — 52 commits, +10,850 lines May 12 May 12, 12am–1am PDT — 30 commits, +3,760 lines May 12, 1am–2am PDT — 24 commits, +9,443 lines May 12, 2am–3am PDT — 41 commits, +1,635 lines May 12, 3am–4am PDT — 39 commits, +788 lines May 12, 4am–5am PDT — 27 commits, +651 lines May 12, 5am–6am PDT — 23 commits, +779 lines May 12, 6am–7am PDT — 1 commit, +137,576 lines May 12, 7am–8am PDT — 2 commits, +81 lines May 12, 8am–9am PDT — 2 commits, +75 lines May 12, 9am–10am PDT — 2 commits, +130 lines May 12, 10am–11am PDT — 5 commits, +160 lines May 12, 11am–12pm PDT — 2 commits, +20 lines May 12, 12pm–1pm PDT — 1 commit, +2 lines May 12, 1pm–2pm PDT — 30 commits, +2,677 lines May 12, 2pm–3pm PDT — 41 commits, +7,022 lines May 12, 3pm–4pm PDT — 4 commits, +200 lines May 12, 4pm–5pm PDT — 27 commits, +1,423 lines May 12, 5pm–6pm PDT — 19 commits, +1,055 lines May 12, 6pm–7pm PDT — 2 commits, +380 lines May 12, 7pm–8pm PDT — 2 commits, +84 lines May 12, 9pm–10pm PDT — 7 commits, +273 lines May 12, 10pm–11pm PDT — 3 commits, +230 lines May 12, 11pm–12am PDT — 7 commits, +319 lines May 13 May 13, 12am–1am PDT — 2 commits, +133 lines May 13, 1am–2am PDT — 14 commits, +2,177 lines May 13, 2am–3am PDT — 12 commits, +685 lines May 13, 4am–5am PDT — 10 commits, +657 lines May 13, 5am–6am PDT — 1 commit, +687 lines May 13, 6am–7am PDT — 11 commits, +380 lines May 13, 7am–8am PDT — 12 commits, +5,247 lines May 13, 8am–9am PDT — 14 commits, +1,051 lines May 13, 9am–10am PDT — 7 commits, +680 lines May 13, 10am–11am PDT — 10 commits, +412 lines May 13, 11am–12pm PDT — 6 commits, +314 lines May 13, 12pm–1pm PDT — 10 commits, +2,980 lines May 13, 1pm–2pm PDT — 1 commit, +0 lines May 13, 2pm–3pm PDT — 3 commits, +439 lines May 13, 5pm–6pm PDT — 7 commits, +114 lines May 13, 6pm–7pm PDT — 4 commits, +605 lines May 13, 9pm–10pm PDT — 1 commit, +13 lines May 13, 10pm–11pm PDT — 1 commit, +48 lines May 13, 11pm–12am PDT — 1 commit, +8 lines May 14 May 14, 12am–1am PDT — 1 commit, +150 lines
port branch 上的每个 commit(排除 merge)按小时分桶。峰值小时:695 个 commit。
注意时间分布不均吗?我忘了提高运行它的 EC2 instance 的默认 IOPS。一个慢 grep 命令就足以让磁盘读写卡住好几分钟。
写完所有代码后,我让 Claude 写一个 workflow 来修复每个编译错误。我们按 crate 一个一个推进。
✻ claude code · dynamic workflow
≈16,000 errors left
Wed, May 6, 12:40 AM PDT
errors.txt0 fix commits
error: deref *mut EventLoop before field access
error: js_parser/ast/E.rs: port json_stringify for Number/BigInt/RegExp
error: NodeHTTPResponse.rs: wire JSNodeHTTPResponse cached accessors vi
error[E0034]: multiple applicable items in scope
error: test_command.rs: wire coverage façade to bun_sourcemap_jsc::code
error: bundler/ungate_support.rs: un-gate bun_css shim to real ::bun_cs
error: dns.rs: implement pending_cache_for/get_key/get_or_put_into_reso
error: css/css_parser.rs: port DefineShorthand contract, parse_bundler,
error: runtime/crypto/mod.rs: create_crypto_error delegates to boringss
error: bun_core/fmt.rs: implement format_ip reborrow (offset-based slic
error: event_loop/EventLoopTimer.rs: port Timespec::ns from bun.zig
divvied up · 64 claudes
worktree 1
→→
→→
→→
→→
worktree 2
→→
→→
→→
→→
worktree 3
→→
→→
→→
→→
worktree 4
→→
→→
→→
→→
1 fixes2 review1 applies
→ commits land per crate
Phase D 的工作方式,按它真实的 1610 个 commit 回放(5 月 6 日,PDT):cargo check 把约 16000 条错误写入文件,按 crate 分组;workflow 把它们分给 64 个 Claude:4 个 worktree 上共 16 个循环,每个循环 1 个 Claude 修复、2 个 review、1 个应用。每个 chip 都是一批真实 commit:它们落在实际 crate 上,随后计数器才移动。错误行是真实的 commit subject。
最棘手的一类错误是循环依赖。
我们的 Zig 代码库是一个编译单元(实际上就是一个 crate)。我想把新的 Rust 代码库拆成大约 100 个 crate,这样 Rust 编译更快;但这需要避免循环依赖,同时尽量少偏离原 Zig 实现。我在开始 Rust 重写前立刻提交的这个 PR 并不充分。我没有推倒重来,而是又跑了一个 workflow,用来分类有循环依赖的代码应该放在哪里,并全部写下来;然后再跑另一个 workflow 执行重构。
修复循环依赖暴露了大约 16000 个编译错误。对 1 个人来说这是巨量;对 64 个 Claude 同时工作来说,并不离谱。
为了最大化并行度,workflow 会按每个 crate 循环:
对每个 crate 运行 cargo check,按文件分组输出并把错误保存到文件
修复这个 crate 内的所有编译错误
2 个 adversarial reviewer review 这个 crate 的变更
1 个 fixer 应用修复
为了防止 Claude 互相踩脚,cargo check 只在最开始运行;和其他运行一样,直到最后才允许 git。
又一次误启动
Claude 把“让所有 crate 编译通过”理解成了“把有编译错误的函数 stub 掉”。Claude 还开始加入可疑地长的解释性注释来说明 workaround,所以我给 adversarial reviewer 加了这条拒绝规则:
如果你需要一整段注释来证明 workaround 没问题,那代码就是错的,修代码。
改了一次 prompt,几个小时后,这些情况就不再发生了。
模型很爱说“smoke tests”。
一旦 cargo check 通过,下一步就是让它能编译并运行 bun --version。它先是有 linker error。然后一启动就 panic。
下一个目标是让它能运行 bun test <file>。一旦这能工作,我们就可以开始跑测试了!于是又来了一个 workflow,围绕 bun CLI 子命令循环:
把每个失败 stacktrace 连同它的子命令保存到文件
对按子命令分组的每个失败 stacktrace,让 1 个 Claude 修复
2 个 adversarial reviewer
1 个 fixer 应用建议
这个 workflow 围绕测试文件循环。
随机运行大约 100 个测试文件,按代码库文件夹分片到 4 个 worktree 中的一个。对每个失败测试,保存 stacktrace 和错误到文件,1 个 implementer 提出修复,2 个 adversarial reviewer,然后 1 个 fixer 应用。
更多误启动
我们的测试套件有很多内存泄漏测试,也有少数集成测试可能运行超过一分钟,例如一个运行 next dev 并检查 hot module reloading 是否能连续 100 次拾取变更的测试。其中一些测试在 debug build 中会 timeout。
我们还有会耗尽机器最大 TCP socket 数的压力测试、读写数 GB 磁盘的测试,以及会 spawn 大约 1 万个进程的测试。
这需要比“拜托了”更强的隔离,所以我们用 systemd-run(cgroups)限制内存和 CPU 使用,并隔离 pid namespace。即便如此,机器还是几次用光磁盘空间并崩溃。
第一次 CI 运行两天后,失败列表从 972 个测试文件降到 23 个。又过了一天半,Linux 全绿了。也是第一次,这次 Rust 重写真的让人感觉会成功。
✻ claude code · dynamic workflowbuildkite · the race to green, by platformWindows finished last · May 11, 6:23 AM PDT
6 / 6 platforms green
build #54202 · Thu, May 14, 12:23 AM PDT
macOS x64 · 2 shards
build #52897: shard failures build #52932: shard failures build #52934: shard failures build #52938: shard failures build #52944: shard failures build #52946: shard failures build #52949: shard failures build #52975: shard failures build #52998: shard failures build #53007: shard failures build #53015: shard failures build #53026: shard failures build #53027: shard failures build #53035: shard failures build #53041: shard failures build #53047: shard failures build #53056: shard failures build #53077: shard failures build #53090: shard failures build #53095: shard failures build #53106: shard failures build #53109: shard failures build #53123: shard failures build #53127: shard failures build #53130: shard failures build #53131: shard failures build #53133: shard failures build #53134: shard failures build #53143: shard failures build #53149: all shards passed build #53159: all shards passed build #53164: all shards passed build #53167: all shards passed build #53172: all shards passed build #53176: all shards passed build #53194: all shards passed build #53208: all shards passed build #53213: shard failures build #53214: shard failures build #53216: no failures (partial run) build #53222: all shards passed build #53229: no failures (partial run) build #53241: all shards passed build #53265: all shards passed build #53271: all shards passed build #53304: shard failures build #53327: shard failures build #53340: shard failures build #53401: shard failures build #53431: shard failures build #53491: all shards passed build #53503: all shards passed build #53748: shard failures build #53753: all shards passed build #53787: all shards passed build #53811: all shards passed build #53933: no failures (partial run) build #53952: all shards passed build #53983: shard failures build #53992: shard failures build #53999: shard failures build #54012: all shards passed build #54015: all shards passed build #54017: all shards passed build #54022: shard failures build #54026: shard failures build #54033: all shards passed build #54040: all shards passed build #54047: shard failures build #54049: shard failures build #54057: shard failures build #54064: all shards passed build #54074: all shards passed build #54093: all shards passed build #54144: no failures (partial run) build #54161: shard failures build #54186: shard failures build #54189: shard failures build #54196: shard failures build #54202: all shards passed
✓
Linux arm64 · 60 shards
build #52934: shard failures build #52938: shard failures build #52944: shard failures build #52969: shard failures build #52975: shard failures build #52980: shard failures build #52988: shard failures build #52996: shard failures build #52998: shard failures build #53007: shard failures build #53013: shard failures build #53014: shard failures build #53015: shard failures build #53026: shard failures build #53027: shard failures build #53031: no failures (partial run) build #53032: shard failures build #53035: shard failures build #53041: shard failures build #53047: shard failures build #53056: shard failures build #53059: shard failures build #53077: shard failures build #53083: shard failures build #53086: shard failures build #53090: shard failures build #53095: shard failures build #53106: shard failures build #53109: shard failures build #53123: shard failures build #53127: shard failures build #53130: shard failures build #53131: shard failures build #53133: shard failures build #53134: shard failures build #53135: shard failures build #53143: shard failures build #53149: shard failures build #53159: shard failures build #53164: shard failures build #53167: all shards passed build #53172: shard failures build #53176: all shards passed build #53188: shard failures build #53194: shard failures build #53208: all shards passed build #53212: shard failures build #53213: shard failures build #53214: shard failures build #53216: all shards passed build #53222: all shards passed build #53229: all shards passed build #53236: no failures (partial run) build #53241: all shards passed build #53260: all shards passed build #53265: all shards passed build #53271: all shards passed build #53280: no failures (partial run) build #53298: no failures (partial run) build #53304: shard failures build #53327: all shards passed build #53340: all shards passed build #53360: no failures (partial run) build #53419: shard failures build #53431: shard failures build #53458: no failures (partial run) build #53485: shard failures build #53491: shard failures build #53503: shard failures build #53514: no failures (partial run) build #53570: shard failures build #53583: shard failures build #53599: shard failures build #53748: shard failures build #53753: all shards passed build #53762: no failures (partial run) build #53787: no failures (partial run) build #53811: all shards passed build #53852: no failures (partial run) build #53863: shard failures build #53893: shard failures build #53914: all shards passed build #53933: all shards passed build #53952: all shards passed build #53983: shard failures build #53992: shard failures build #53999: shard failures build #54008: no failures (partial run) build #54012: all shards passed build #54015: all shards passed build #54017: all shards passed build #54022: shard failures build #54026: all shards passed build #54030: shard failures build #54033: all shards passed build #54040: all shards passed build #54047: shard failures build #54049: shard failures build #54055: shard failures build #54057: shard failures build #54064: all shards passed build #54074: all shards passed build #54083: no failures (partial run) build #54093: all shards passed build #54144: all shards passed build #54161: shard failures build #54186: shard failures build #54189: shard failures build #54196: shard failures build #54202: all shards passed
✓
Linux x64 · 60 shards
build #52934: shard failures build #52938: shard failures build #52944: shard failures build #52969: shard failures build #52975: shard failures build #52988: shard failures build #52996: shard failures build #52998: shard failures build #53007: shard failures build #53013: shard failures build #53014: shard failures build #53015: shard failures build #53026: shard failures build #53027: shard failures build #53032: shard failures build #53033: shard failures build #53035: shard failures build #53041: shard failures build #53047: shard failures build #53056: shard failures build #53059: no failures (partial run) build #53077: shard failures build #53083: no failures (partial run) build #53086: shard failures build #53090: shard failures build #53095: shard failures build #53106: shard failures build #53109: shard failures build #53123: shard failures build #53127: shard failures build #53130: shard failures build #53131: shard failures build #53133: shard failures build #53134: shard failures build #53135: shard failures build #53143: shard failures build #53149: shard failures build #53159: shard failures build #53164: shard failures build #53167: all shards passed build #53172: all shards passed build #53176: all shards passed build #53188: shard failures build #53194: all shards passed build #53208: all shards passed build #53212: shard failures build #53213: shard failures build #53214: shard failures build #53216: all shards passed build #53222: all shards passed build #53229: all shards passed build #53236: no failures (partial run) build #53241: all shards passed build #53260: all shards passed build #53265: all shards passed build #53271: all shards passed build #53280: no failures (partial run) build #53304: shard failures build #53327: all shards passed build #53340: all shards passed build #53360: no failures (partial run) build #53419: no failures (partial run) build #53431: shard failures build #53458: no failures (partial run) build #53485: shard failures build #53491: all shards passed build #53503: all shards passed build #53514: no failures (partial run) build #53570: shard failures build #53583: shard failures build #53599: shard failures build #53748: shard failures build #53753: all shards passed build #53759: no failures (partial run) build #53781: shard failures build #53787: no failures (partial run) build #53811: all shards passed build #53863: shard failures build #53893: shard failures build #53914: all shards passed build #53933: shard failures build #53952: all shards passed build #53983: shard failures build #53992: shard failures build #53999: shard failures build #54008: no failures (partial run) build #54012: all shards passed build #54015: all shards passed build #54017: all shards passed build #54022: shard failures build #54026: all shards passed build #54030: shard failures build #54033: all shards passed build #54040: no failures (partial run) build #54047: shard failures build #54049: shard failures build #54055: shard failures build #54057: shard failures build #54064: all shards passed build #54074: all shards passed build #54083: no failures (partial run) build #54093: all shards passed build #54144: all shards passed build #54161: shard failures build #54186: shard failures build #54189: shard failures build #54196: shard failures build #54202: all shards passed
✓
macOS arm64 · 4 shards
build #52897: shard failures build #52929: shard failures build #52932: shard failures build #52944: shard failures build #52975: shard failures build #52996: shard failures build #52998: shard failures build #53007: shard failures build #53013: shard failures build #53014: shard failures build #53015: shard failures build #53026: shard failures build #53027: shard failures build #53032: shard failures build #53035: shard failures build #53041: shard failures build #53047: shard failures build #53056: shard failures build #53059: shard failures build #53077: shard failures build #53095: shard failures build #53109: shard failures build #53123: shard failures build #53127: shard failures build #53130: shard failures build #53131: shard failures build #53133: shard failures build #53134: shard failures build #53135: shard failures build #53143: shard failures build #53149: shard failures build #53159: shard failures build #53164: shard failures build #53167: shard failures build #53172: shard failures build #53176: shard failures build #53188: shard failures build #53194: shard failures build #53208: shard failures build #53212: shard failures build #53213: shard failures build #53214: shard failures build #53216: shard failures build #53222: shard failures build #53229: no failures (partial run) build #53236: no failures (partial run) build #53241: all shards passed build #53265: all shards passed build #53271: no failures (partial run) build #53280: no failures (partial run) build #53304: shard failures build #53327: no failures (partial run) build #53340: no failures (partial run) build #53360: no failures (partial run) build #53368: shard failures build #53379: shard failures build #53383: shard failures build #53401: shard failures build #53431: shard failures build #53458: shard failures build #53491: no failures (partial run) build #53503: shard failures build #53570: shard failures build #53583: shard failures build #53599: shard failures build #53601: shard failures build #53748: shard failures build #53753: all shards passed build #53757: no failures (partial run) build #53759: no failures (partial run) build #53787: no failures (partial run) build #53811: all shards passed build #53952: no failures (partial run) build #53992: shard failures build #53999: shard failures build #54007: no failures (partial run) build #54012: all shards passed build #54015: shard failures build #54017: all shards passed build #54022: shard failures build #54026: no failures (partial run) build #54030: shard failures build #54033: all shards passed build #54040: shard failures build #54047: shard failures build #54049: shard failures build #54055: shard failures build #54057: shard failures build #54064: shard failures build #54074: shard failures build #54093: shard failures build #54161: shard failures build #54186: shard failures build #54189: shard failures build #54196: shard failures build #54202: all shards passed
✓
Windows x64 · 8 shards
build #53090: shard failures build #53094: shard failures build #53095: shard failures build #53106: shard failures build #53109: shard failures build #53123: shard failures build #53127: shard failures build #53130: shard failures build #53131: shard failures build #53133: shard failures build #53134: shard failures build #53135: shard failures build #53143: shard failures build #53149: shard failures build #53159: shard failures build #53164: shard failures build #53167: shard failures build #53172: shard failures build #53176: shard failures build #53188: shard failures build #53194: shard failures build #53208: shard failures build #53212: shard failures build #53213: shard failures build #53214: shard failures build #53216: shard failures build #53222: shard failures build #53229: shard failures build #53236: shard failures build #53241: shard failures build #53260: shard failures build #53265: shard failures build #53271: shard failures build #53280: shard failures build #53298: shard failures build #53304: shard failures build #53327: all shards passed build #53340: all shards passed build #53360: no failures (partial run) build #53419: shard failures build #53431: shard failures build #53458: shard failures build #53470: no failures (partial run) build #53485: shard failures build #53491: no failures (partial run) build #53503: shard failures build #53514: no failures (partial run) build #53565: no failures (partial run) build #53570: shard failures build #53599: shard failures build #53745: shard failures build #53748: shard failures build #53753: shard failures build #53757: shard failures build #53759: shard failures build #53762: shard failures build #53769: shard failures build #53781: shard failures build #53787: shard failures build #53808: shard failures build #53811: shard failures build #53852: shard failures build #53863: shard failures build #53883: shard failures build #53893: shard failures build #53914: all shards passed build #53933: all shards passed build #53952: all shards passed build #53973: no failures (partial run) build #53983: shard failures build #53992: shard failures build #53999: shard failures build #54002: no failures (partial run) build #54004: no failures (partial run) build #54007: shard failures build #54008: no failures (partial run) build #54012: all shards passed build #54015: all shards passed build #54017: all shards passed build #54022: shard failures build #54026: all shards passed build #54030: all shards passed build #54033: all shards passed build #54040: all shards passed build #54047: shard failures build #54049: shard failures build #54055: shard failures build #54057: shard failures build #54064: all shards passed build #54074: all shards passed build #54083: all shards passed build #54093: all shards passed build #54144: shard failures build #54161: shard failures build #54186: shard failures build #54189: shard failures build #54196: shard failures build #54202: all shards passed
✓
Windows arm64 · 8 shards
build #53090: shard failures build #53095: shard failures build #53106: shard failures build #53109: shard failures build #53123: shard failures build #53127: shard failures build #53130: shard failures build #53131: shard failures build #53134: shard failures build #53135: shard failures build #53149: shard failures build #53159: shard failures build #53164: shard failures build #53167: shard failures build #53172: shard failures build #53176: shard failures build #53188: shard failures build #53194: shard failures build #53208: shard failures build #53212: shard failures build #53213: shard failures build #53214: shard failures build #53216: shard failures build #53222: shard failures build #53229: shard failures build #53236: no failures (partial run) build #53241: shard failures build #53260: shard failures build #53265: shard failures build #53271: shard failures build #53304: shard failures build #53327: all shards passed build #53340: all shards passed build #53360: no failures (partial run) build #53419: no failures (partial run) build #53431: shard failures build #53458: no failures (partial run) build #53485: shard failures build #53491: no failures (partial run) build #53503: no failures (partial run) build #53599: shard failures build #53748: shard failures build #53753: shard failures build #53757: shard failures build #53759: shard failures build #53762: shard failures build #53787: shard failures build #53808: shard failures build #53811: shard failures build #53852: shard failures build #53863: shard failures build #53883: shard failures build #53893: shard failures build #53914: no failures (partial run) build #53933: all shards passed build #53952: all shards passed build #53983: shard failures build #53992: shard failures build #53999: shard failures build #54007: no failures (partial run) build #54012: all shards passed build #54015: all shards passed build #54017: all shards passed build #54022: shard failures build #54026: all shards passed build #54030: no failures (partial run) build #54033: all shards passed build #54040: all shards passed build #54047: shard failures build #54049: shard failures build #54055: no failures (partial run) build #54057: shard failures build #54064: all shards passed build #54074: all shards passed build #54083: no failures (partial run) build #54093: all shards passed build #54144: shard failures build #54161: shard failures build #54186: shard failures build #54189: shard failures build #54196: shard failures build #54202: all shards passed
✓
跨 135 次实际运行测试的 CI build(从 BuildKite 的 420 个 build 中挖出)中,每个平台的每个 test shard。亮绿色:每个 shard 都通过。暗绿色:没有失败,但运行被提前截断(被 superseded)。红色:至少一个 shard 失败。每条泳道都标注了它的完整套件第一次通过的时间。Linux 的 60 个 shard 比 Windows 早了将近一天全绿。各个平台一直会抖成红色,直到最后几个失败测试消失;最终全绿的 build 是 #54202。
合并前剩下的时间就很直接了。一个 workflow 按平台循环修复 CI 测试失败,直到没有测试失败为止。还有几个 workflow 做 Windows 相关清理、去重代码、减少 unsafe 使用,以及整体清理一些代码。
当 Bun 的测试套件在 CI 的所有平台上 100% 通过后(我也手动确认测试确实在运行,没有被 skip),我在本地跑了一堆命令测试,然后按下了 merge 按钮。
合并到 main 不是版本化发布。到这个节点,我已经足够有信心继续推进,并承诺采用这次重写;但还没有足够信心发布它。
峰值时,我们同时运行 4 个这样的 workflow,每个 workflow 在单独的 worktree 中,每个 workflow 有 16 个 Claude。大约同时运行 64 个 Claude。
git log · claude/phase-a-portpeak: 58 commits in one minute
0
commits
+0
lines written, rewrites included
Mon, May 4, 7:05 AM PDT
first 100-file draft batch PR #30412 opened merged
全部 6502 个 commit(排除 merge)的回放。粉色柱主要是新增代码;青色柱主要是删除。行数计数会把中间每次重写都算进去,最终落地的 diff 是 +1009272。日志是真实 commit message。
0 个测试被跳过或删除
11 天(5 月 3 日 → 5 月 14 日合并)· 6778 个 commit
Platform
expect() calls
Tests
Files
Debian 13 x64
1,386,826
60,624
4,174
macOS 14 arm64
1,259,953
58,850
4,175
Windows 2019 x64
1,007,544
57,337
4,173
合并前,这花掉了 59 亿 uncached input token、6.9 亿 output token,以及 720 亿 cached input token read,按 API 定价大约是 16.5 万美元。如果手工做,我认为需要 3 个对代码库有完整上下文的工程师大约一年;在那段时间里,我们无法改进 Node.js 兼容性、修 bug、修安全问题或实现新功能。我们绝不会这么做。现实中的替代方案,就是什么都不做,然后永远继续修这篇文章开头列出的那些 bug。
这是今天可能性边界上最前沿的东西。我使用了 Claude Fable 5 的预发布版本,这是一个 Mythos-class 模型。Claude Code 的 dynamic workflow 让 64 个 Claude 连续运行了 11 天(否则我就得自己写 harness 才能做到)。
自从合并 Rust port 后,我们已经完成了 Claude Code Security 的 11 轮安全 review,并处理了发现的问题。
我们还给 Bun 的每个 parser 加上了 7x24 coverage-guided fuzzing:JavaScript、TypeScript、JSX、CSS、JSON5、JSONC、TOML、YAML、Markdown、INI、Bun Shell scripts、semver ranges、.patch files 和 CSS colors。fuzzer 会自动把发现的 bug 发给 Claude,由它提交复现和修复 PR,然后由人类 review PR。到目前为止,它已经执行我们的 parser 1000 亿次,带来了大约 15 个 PR。
在写作时,Bun 的 Rust 代码大约 4% 位于 unsafe block 中(在约 780000 行里,有约 13000 个 unsafe keyword,分布在约 27000 行),其中 78% 的 block 只有一行:一个来自 C++ 的 pointer,或一次对 C library 的调用。随着我们从忠实的 Zig port(它没有可 grep 的 unsafe keyword)重构为惯用 Rust,我预计这个数字会随时间下降;但我们会继续使用 JavaScriptCore 这样的 C 和 C++ 库,所以它永远会比纯 Rust 项目有更多 unsafe。
Rust 重写的重点是稳定性,但发布这么大的变更,不引入任何回归是不可能的。
这次重写引入了 19 个已知回归,每个都已经修复。
大多数回归来自两门语言中语法完全相同、语义却不同的代码。
debug_assert! 里的副作用
这两个片段看起来相似,但行为不同。Zig 的 assert 是函数,所以它的参数在每个 build 中都会运行。Rust 的 debug_assert! 是 macro,所以在 release build 中,整个表达式都会被擦除,包括 insert_stale 调用。
// Zig:
if (dev.framework.react_fast_refresh) |rfr| {
assert(try dev.client_graph.insertStale(rfr.import_source, false) == IncrementalGraph(.client).react_refresh_index);
}
// Rust:
if let Some(rfr) = &dev.framework.react_fast_refresh {
debug_assert!(dev.client_graph.insert_stale(&rfr.import_source, false)? == react_refresh_index);
}
insert_stale 会把一个文件加入 frontend dev server 的 hot reload graph。在 release build 中它不再运行,于是 HMR 在某些情况下坏掉:带有使用 React 的 HTML route 的项目中,当一个 hot reloaded 文件被 invalidated 时,会出现 Cannot destructure property 'isLikelyComponentType' of 'k'。Debug build 是正常的。#30678
奇数长度的 slice
Bun 的 Zig helper reinterpretSlice(u16, bytes)(早于 builtin cast 支持 slice)使用 @divTrunc,会忽略末尾多出来的奇数字节。bytemuck::cast_slice 则会 panic。Blob.text() 在遇到 UTF-16 byte order mark 后接奇数个字节时,不再返回字符串,而是让进程 panic。我们改回了忽略那个奇数字节:&buf[..buf.len() & !1]。#31188
Bounds check
在 macOS 和 Linux 上,我们用 ReleaseFast 编译 Bun 的 Zig 代码,这会移除 bounds check。Rust 的 release build 会保留它们。
Bun 的 module resolver 会把长文件名 intern 到一个全局列表里,溢出后进入 overflow block。原来的 Zig 代码把每个 block 大小设为 count / 4,也就是 2048。移植时留下了一个 placeholder:
/// ... so use a nonzero stand-in until Phase B threads the
/// per-instantiation value through.
pub const BSS_OVERFLOW_BLOCK_SIZE: usize = 64;
这把上限从 840 万个 interned filename 降到了 270272 个,真实项目会撞到;同时也让我们从 Zig 移植过来的 ptrs[4095] off-by-one 变得可达。Rust 没有写越界,而是 panic 了。如果我们使用 ReleaseSafe,Zig 在这种情况下也会 panic(我们只在 Windows 上这么做)。#31503
comptime format string
Output.pretty 会把 <r> 和 <d> 颜色 marker 改写成 ANSI escape。在 Zig 中,fmt 是 comptime,所以 marker 在参数替换前就消失了。Rust function 没有 comptime 参数,所以 Output::pretty 只能看到完成后的字符串,也会把参数里的 marker 一起改写。
// Zig:
pub inline fn pretty(comptime fmt: string, args: anytype) void;
Output.pretty("<r>{f}<r>", .{hyperlink});
// Rust:
pub fn pretty(payload: impl PrettyFmtInput);
Output::pretty(format_args!("<r>{}<r>", hyperlink));
bun update -i 会把 package name 打印成 OSC 8 hyperlink,并用 ESC \ 结束。那个反斜杠正好在尾部 <r> 的 < 前面,marker parser 吃掉了它,于是 r 被作为文本打印出来。
it should say oxfmt, not oxfmtr
在 Rust 中,它必须是 macro:bun_core::pretty!("<r>{}<r>", hyperlink)。#30693
到目前为止,Bun v1.4.0 修复了 128 个可以在 v1.3.14 中复现的 bug,范围从内存泄漏到崩溃,再到帮助文本颜色错误。
Rust 有一个强大的语言级内存清理工具:Drop。实现了 Drop 后,每当值离开作用域,drop 函数都会自动调用。
impl Drop for Bytes {
fn drop(&mut self) {
if !self.pinned.is_empty() {
JSC__JSValue__unpinArrayBuffer(self.pinned);
}
}
}
在 Zig 中,defer 可用于在作用域结束时运行代码:
const bytes: ArrayBuffer = try .fromPinned(global, value);
defer bytes.unpin();
在 Zig 中,defer 需要添加到每一个可能需要清理的调用点。很容易最后忘了清理(内存泄漏),或者在很少走到的错误处理代码里把清理代码运行两次(double-free)。在 Rust 中,Drop 会在值不再可访问时自动运行,用“隐藏控制流”换掉一个常见 footgun。
Drop 修复了 Bun 中多个和错误处理代码中文件路径有关的内存泄漏。
我们修复了每一个可 instrument 的内存泄漏
我们改进了 Bun 的 LeakSanitizer 集成,用来追踪所有 native code memory allocation 。
这里有个例子:每次进程内调用 Bun.build() 都会泄漏数 MB 内存,也就是属于这次 build 的 parsed source text 和 AST symbol table,在 build 结束后仍然活着。
// Bundle the same 60-module project 2,000 times in one process
for (let i = 0; i < 2_000; i++) {
await Bun.build({
entrypoints: ["./index.js"],
minify: true,
sourcemap: "external",
});
}
在 Bun v1.3.14 中,每次 build 都会永久泄漏大约 3 MB。像 dev server 这种每个 request 都会 bundle 的工具,最终会耗尽内存。在 Bun v1.4.0 中,内存会趋于平稳:
Builds
Bun v1.3.14
Bun v1.4.0
500
1,914 MB
526 MB
1,000
3,506 MB
586 MB
1,500
5,097 MB
608 MB
2,000
6,745 MB
609 MB
此前有一次在 Zig 中做这件事的尝试 没有 merge,因为缺少等价于 Drop 的机制,让人更难有信心合并。
Rust 重写的初始变更让 Windows 上的二进制体积减少了 3.8 MB,macOS 上减少 5.5 MB,Linux 上减少 6.8 MB。这主要是因为我们在 Zig 代码里用了太多 comptime。
初始缩小之后,团队继续探索更多减少二进制体积的机会,例如 Identical Code Folding 这样的 linker optimization、移除 ICU 中未使用的数据,以及用 zstd dictionary 按需懒解压 libicu 的小部分内容。
结合 Rust 重写、ICU 变更和 identical code folding,Bun 的二进制体积在 Linux 和 Windows 上缩小了约 20% 。
Version
Platform
Size
Bun v1.4.0 (canary)
Windows
76 MB
Bun v1.3.14
Windows
94 MB
Bun v1.4.0 (canary)
Linux
70 MB
Bun v1.3.14
Linux
88 MB
TOML parser,以及 Bun 中所有其他 recursive-descent parser(JSON、YAML、JavaScript、TypeScript 等),现在使用的栈空间都更少了。
这在合并 Rust 重写前导致了一些测试失败:
bun test v1.3.14-canary.1 (e99311e58)
.......
105 | });
106 |
107 | it("Bun.TOML.parse throws on deeply nested inline tables instead of crashing", () => {
108 | const depth = 25_000;
109 | const deepToml = "a = " + "{ b = ".repeat(depth) + "1" + " }".repeat(depth);
110 | expect(() => Bun.TOML.parse(deepToml)).toThrow(RangeError);
^
error: expect(received).toThrow(expected)
Expected constructor: RangeError
Received function did not throw
Received value: {
a: {
b: {
b: {
b: {
b: {
b: {
b: {
b: {
b: [Object ...],
},
},
},
},
},
},
},
},
}
at <anonymous> (/var/lib/buildkite-agent/build/test/js/bun/resolve/toml/toml.test.js:110:42)
✗ Bun.TOML.parse throws on deeply nested inline tables instead of crashing [2907.64ms]
Rust 的 LLVM IR codegen 会在 stack variable 不再使用时发出 LLVM 的 llvm.lifetime.start 和 llvm.lifetime.end intrinsic,让 LLVM 可以复用栈空间 slot。这让带有嵌套作用域的大函数显著减少栈空间使用。
此前,我们通过把特别大的函数重构成许多小函数 ,手动绕过 一个未解决的问题 。
Rust 支持 C/C++ 和 Rust 之间的跨语言 link-time optimization,这让跨语言 inline 成为可能(这也太酷了!)。
我们在 Linux x64(EC2,Xeon Platinum 8488C)上对 Bun v1.3.14 和 Bun v1.4.0 做了基准测试。HTTP throughput 用 oha 压测 hello-world server,app workload 用 hyperfine 测量。
HTTP throughput(req/s,3 轮平均)
server
Bun v1.3.14
Bun v1.4.0
Δ
Bun.serve
169.6k
177.7k
+4.8%
node:http
103.8k
108.5k
+4.5%
Elysia
158.9k
163.3k
+2.8%
express
64.5k
66.6k
+3.2%
fastify
91.5k
95.9k
+4.8%
Apps / CLI(hyperfine)
workload
Bun v1.3.14
Bun v1.4.0
Δ
next build
13.62 s
13.03 s
+4.5%
vite build (tsc + vite)
1.69 s
1.65 s
+2.2%
tsc -b --force
0.94 s
0.89 s
+4.7%
Prisma 在 Bun 的 Rust 重写版上发布了 Prisma Compute public beta。
“我们遇到过内存泄漏,以及一个连接池在 VM pause 后 resume 时无法恢复的问题。Rust 重写出现后,我们用同样的 failure mode 测试了它。它处理得非常完美。”——Alexey Orlenko
Claude Code v2.1.181(6 月 17 日发布)及之后的版本使用 Rust port 的 Bun。在 Linux 上启动速度快了 10%,除此之外几乎没人注意到。无聊是好事。
Bun v1.3.14 是最后一个用 Zig 写的 Bun 版本。Bun v1.4.0 会是第一个用 Rust 写的 Bun 版本。它现在已经在 canary 中可用,请报告你发现的任何问题:
bun upgrade --canary
对我和团队来说,新的 Rust 代码库感觉和旧的 Zig 代码库非常相似。例如,下面是原 Zig 代码和新 Rust 代码的一段:
pub fn canMergeSymbols(
scope: *Scope,
existing: Symbol.Kind,
new: Symbol.Kind,
comptime is_typescript_enabled: bool,
) SymbolMergeResult {
if (existing == .unbound) {
return .replace_with_new;
}
if (comptime is_typescript_enabled) {
// In TypeScript, imports are allowed to silently collide with symbols within
// the module. Presumably this is because the imports may be type-only:
//
// import {Foo} from 'bar'
// class Foo {}
//
if (existing == .import) {
return .replace_with_new;
}
// ...
}
// ...
}
pub fn can_merge_symbol_kinds<const IS_TYPESCRIPT_ENABLED: bool>(
scope_kind: Kind,
existing: symbol::Kind,
new: symbol::Kind,
) -> SymbolMergeResult {
if existing == symbol::Kind::Unbound {
return SymbolMergeResult::ReplaceWithNew;
}
if IS_TYPESCRIPT_ENABLED {
// In TypeScript, imports are allowed to silently collide with symbols within
// the module. Presumably this is because the imports may be type-only:
//
// import {Foo} from 'bar'
// class Foo {}
//
if existing == symbol::Kind::Import {
return SymbolMergeResult::ReplaceWithNew;
}
// ...
}
// ...
}
任何理解原 Zig 代码的人,也能理解机械翻译出来的 Rust 代码。我 review 原始 Rust 重写 PR 的方式,是检查 adversarial code review agent 是否正确抓到了 Zig 代码和 Rust 代码之间的差异,是否确保 porting guide 和 lifetime guide 被遵循,同时我也自己把很多代码按 Zig vs Rust 并排手动读了一遍。
Bun v1.4 让 Bun 更快、更小、使用更少内存,并给团队提供了极其强大的工具来系统性地改进未来稳定性:Rust 的 borrow checker、Miri(CI 中正在覆盖越来越多代码)、LeakSanitizer,以及针对 parser 的 7x24 coverage-guided fuzzing。还有更多内容需要重构 ,但开局很好。
如果由一个对代码库有完整上下文的工程师团队来做,这次 Rust 重写会花一年。借助 Fable,并由 1 个工程师密切监控 Claude Code,我们从开始到所有平台上 100% 测试套件通过,只用了 11 天。
今天,一个工程师能做的事比一年前多得多。