[PR #3200] [MERGED] perf(native): skip llvm-nm's dyld-info pass when the symbol table names every export #5653

New issue

Closed

opened 2026-10-31 01:08:17 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 01:08:17 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/3200
Author: @kaoru0822-kitauji
Created: 10/14/2026
Status: ✅ Merged
Merged: 10/14/2026
Merged by: @kaoru0822-kitauji

Base: main ← Head: perf/macho-nm-no-dyldinfo

📝 Commits (2)

4f710d6 perf(native): skip llvm-nm's dyld-info pass when the symbol table names every export
9f8c148 fix(native): fall back to nm's dyld-info pass for unreadable export rows

📊 Changes

5 files changed (+164 additions, -1 deletions)

View changed files

📝 src/native/NativeMachoInspection.ts (+47 -1)
📝 tests/boundary/providers/native/nativeInventory.golden.test.ts (+88 -0)
📝 tests/fixtures/native-macos/dyld-inventory/README.md (+14 -0)
➕ tests/fixtures/native-macos/dyld-inventory/stripped-swift-exports.txt (+9 -0)
➕ tests/fixtures/native-macos/dyld-inventory/stripped-swift-nm.txt (+6 -0)

📄 Description

Summary

inspect_macho spends most of its time on large Apple apps inside one nm call. Apple's llvm-nm runs a dyld-info pass by default to fake up symbols missing from the symbol table, and on Keynote that pass alone takes 5.8 s (arm64) or 10.2 s (x86_64) to print one name. Under -gU the pass can only add export-trie names, and inspect_macho already reads those from dyld_info -exports. This PR runs nm -gjU --no-dyldinfo and falls back to the old nm -gjU only when the symbol table really omits a trie export.

No linked issue: this is a hotspot I measured with a sweep of static tools over /Applications bundles (Keynote inspect-macho took 6.8 s against about 1 s for the others).

Problem and expected behavior

Trigger: rea inspect-macho /Applications/Keynote.app/Contents/MacOS/Keynote (also Pages and GarageBand).
llvm-nm -gjU -arch arm64 Keynote: 5.54 s, 1 line. llvm-nm -gjU --no-dyldinfo: 0.01 s, the same line. Every other slice command (otool, dyld_info ×2, dwarfdump, vtool) takes 0.01 s or less on the same slice, so nm is the whole cost.
Expected: identical inspect_macho results, without the multi-second wait.

Change and scope

src/native/NativeMachoInspection.ts (the owner of the nm/dyld_info merge):
- Run nm -gjU --no-dyldinfo among the parallel slice commands.
- The pass is still needed in one case: with strip -T, the symbol table loses names the export trie keeps. The default pass restores those exact names, and parseDyldSymbols uses them to split ambiguous dyld_info rows. So if any dyld_info -exports row names a symbol the symbol-table run lacks, rerun the default nm -gjU and use its capture. Provenance records the nm run whose output was used.
Not changed: parsers, output schema, the export merge itself, or other tools.
Toolchain: --no-dyldinfo has been in llvm-nm since LLVM 9 (2019). inspect_macho already requires dyld_info, which ships only with Xcode/CLT 14+, so every host that can run it has the flag.

Contract and boundary impact

Semantic owner and earliest changed stage: native Mach-O inspection, command selection only.
Contribution scope fit and linked maintainer decision: existing read-only static inspection; not applicable.
CLI and MCP/tool-catalog contract: unchanged. Only the recorded nm command in provenance gains --no-dyldinfo (+16 bytes).
Provider, bridge, target-format, or platform compatibility: macOS Xcode tools; see the toolchain note above.
Evidence, artifact, provenance, or reconstruction contract: identical normalized_result apart from the nm command; provenance names the capture actually used.
Target selection, authority, upstream behavior, and cleanup details: unchanged; the fallback reuses the same slice runner, signal and failure path.
Generated metadata, package, or installation impact: none.
Read existing files or captured data.
Launch or attach to a process.
Inject instrumentation or run scripts/code.
Change process memory, registers, files, or configuration.
Contact network services.
Recover protected code or work around protections.
Documentation or usability only; no target interaction.
Not sure yet; investigation or maintainer guidance needed.

Evidence and regression coverage

Real CLI, main (ec67aebed) vs this branch, 3 runs each, macOS 26.7.1 arm64, CLT (Apple LLVM 21):

`rea inspect-macho`	wall (main → branch)	CPU user+sys	peak RSS*	output bytes
Keynote (60 MB fat)	7.08 s → 1.25 s	7.79 s → 1.91 s	388 → 374 MB	756,920 → 756,936
Pages	5.01 s → 1.04 s	5.79 s → 1.70 s	395 → 347 MB	576,499 → 576,515
GarageBand	3.58 s → 1.07 s	4.24 s → 1.70 s	383 → 348 MB	540,004 → 540,020
Telegram `libswiftCore.dylib` (13,010 exports)	1.18 s → 1.20 s (noise; 3 runs each)			+16

*/usr/bin/time -l maximum RSS. Agent tokens: the response is unchanged except the 16-byte provenance flag, so the token cost is unchanged; the win is time to an answer.

Correctness:

normalized_result minus provenance is byte-identical (jq -S) between main and the branch for Keynote, Pages, GarageBand, Transmission, libswiftCore, and three built dylibs: a renamed -alias re-export, a -reexported_symbols_list re-export, and a strip -T Swift dylib.
Raw-tool sweep over 268 slices from 190 Mach-O files in /Applications (Contents/MacOS, frameworks, dylibs): default nm -gjU and nm -gjU --no-dyldinfo print the same name set on every slice. Total nm time 32.5 s → 3.2 s; six slices took over 0.5 s (Keynote, Pages and GarageBand, both architectures).
The strip -T Swift dylib is the case the fallback exists for: the symbol table holds 0 names and the trie 6. --no-dyldinfo prints none, so the branch reruns nm -gjU and reports all 6 exports, with that command in provenance.

Tests (tests/boundary/providers/native/nativeInventory.golden.test.ts, through NativeMacOSProvider):

The ordinary inventory runs nm once with --no-dyldinfo, and provenance records it.
The stripped Swift case (new fixtures captured from the real dylib, with the generation script in the fixture README) reruns nm -gjU, keeps the six trie exports, and records only the rerun in provenance.
Both tests fail on main's NativeMachoInspection.ts.

Remaining proof gaps:

The fallback check parses the dyld_info rows with the symbol-table names. It would miss a stripped export whose literal name splits into a different name that the symbol table does hold, like _a [weak-def] stripped while _a remains. I found no such binary; such a row could already be ambiguous on main.
The check parses dyld export rows a second time, about 20 ms or less on libswiftCore's 13k rows.

Validation performed

npm run typecheck — pass
npm run lint — pass
npm run jscpd — pass (3.19%)
npm run test:focused on 51 files (src/native/**, tests/boundary/providers/native/*, every acceptance/MCP/CLI test mentioning inspect_macho) — 557 passed
Real CLI before/after runs and the 268-slice sweep above.

Compatibility, safety, and release

Breaking changes or migration steps: none.
Real Hopper/Ghidra, browser, or OS coverage: real macOS Xcode CLT tools on the apps listed; Hopper/Ghidra not involved.
Package or release metadata impact: none.
Security, privacy, process, or containment review: same tools, the same read-only operand and at most one extra nm launch in the stripped case.

Before merge

This PR addresses a concrete problem or an agreed enhancement.
Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
The PR has one focused outcome and the title follows type(scope): outcome.
Related issue is linked, or the reason for not linking one is stated above.
Tests cover changed observable behavior and meaningful failure paths.
Owning docs, contracts, and generated metadata are updated where needed.
User-visible CLI/MCP changes include representative output.
I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

🤖 Generated with Claude Code

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/3200 **Author:** [@kaoru0822-kitauji](https://github.com/kaoru0822-kitauji) **Created:** 10/14/2026 **Status:** ✅ Merged **Merged:** 10/14/2026 **Merged by:** [@kaoru0822-kitauji](https://github.com/kaoru0822-kitauji) **Base:** `main` ← **Head:** `perf/macho-nm-no-dyldinfo` --- ### 📝 Commits (2) - [`4f710d6`](https://github.com/morluto/rea/commit/4f710d6c0d52ecfbdf97e2c3321a4fa804179a6e) perf(native): skip llvm-nm's dyld-info pass when the symbol table names every export - [`9f8c148`](https://github.com/morluto/rea/commit/9f8c148da011a4bd12c8e159b653b831b0e6687e) fix(native): fall back to nm's dyld-info pass for unreadable export rows ### 📊 Changes **5 files changed** (+164 additions, -1 deletions) <details> <summary>View changed files</summary> 📝 `src/native/NativeMachoInspection.ts` (+47 -1) 📝 `tests/boundary/providers/native/nativeInventory.golden.test.ts` (+88 -0) 📝 `tests/fixtures/native-macos/dyld-inventory/README.md` (+14 -0) ➕ `tests/fixtures/native-macos/dyld-inventory/stripped-swift-exports.txt` (+9 -0) ➕ `tests/fixtures/native-macos/dyld-inventory/stripped-swift-nm.txt` (+6 -0) </details> ### 📄 Description ## Summary `inspect_macho` spends most of its time on large Apple apps inside one `nm` call. Apple's `llvm-nm` runs a dyld-info pass by default to fake up symbols missing from the symbol table, and on Keynote that pass alone takes 5.8 s (arm64) or 10.2 s (x86_64) to print **one** name. Under `-gU` the pass can only add export-trie names, and `inspect_macho` already reads those from `dyld_info -exports`. This PR runs `nm -gjU --no-dyldinfo` and falls back to the old `nm -gjU` only when the symbol table really omits a trie export. No linked issue: this is a hotspot I measured with a sweep of static tools over `/Applications` bundles (Keynote `inspect-macho` took 6.8 s against about 1 s for the others). ## Problem and expected behavior - Trigger: `rea inspect-macho /Applications/Keynote.app/Contents/MacOS/Keynote` (also Pages and GarageBand). - `llvm-nm -gjU -arch arm64 Keynote`: 5.54 s, 1 line. `llvm-nm -gjU --no-dyldinfo`: 0.01 s, the same line. Every other slice command (`otool`, `dyld_info` ×2, `dwarfdump`, `vtool`) takes 0.01 s or less on the same slice, so nm is the whole cost. - Expected: identical `inspect_macho` results, without the multi-second wait. ## Change and scope - `src/native/NativeMachoInspection.ts` (the owner of the nm/dyld_info merge): - Run `nm -gjU --no-dyldinfo` among the parallel slice commands. - The pass is still needed in one case: with `strip -T`, the symbol table loses names the export trie keeps. The default pass restores those exact names, and `parseDyldSymbols` uses them to split ambiguous `dyld_info` rows. So if any `dyld_info -exports` row names a symbol the symbol-table run lacks, rerun the default `nm -gjU` and use its capture. Provenance records the nm run whose output was used. - Not changed: parsers, output schema, the export merge itself, or other tools. - Toolchain: `--no-dyldinfo` has been in llvm-nm since LLVM 9 (2019). `inspect_macho` already requires `dyld_info`, which ships only with Xcode/CLT 14+, so every host that can run it has the flag. ## Contract and boundary impact - Semantic owner and earliest changed stage: native Mach-O inspection, command selection only. - Contribution scope fit and linked maintainer decision: existing read-only static inspection; not applicable. - CLI and MCP/tool-catalog contract: unchanged. Only the recorded nm command in `provenance` gains `--no-dyldinfo` (+16 bytes). - Provider, bridge, target-format, or platform compatibility: macOS Xcode tools; see the toolchain note above. - Evidence, artifact, provenance, or reconstruction contract: identical `normalized_result` apart from the nm command; provenance names the capture actually used. - Target selection, authority, upstream behavior, and cleanup details: unchanged; the fallback reuses the same slice runner, signal and failure path. - Generated metadata, package, or installation impact: none. - [x] Read existing files or captured data. - [ ] Launch or attach to a process. - [ ] Inject instrumentation or run scripts/code. - [ ] Change process memory, registers, files, or configuration. - [ ] Contact network services. - [ ] Recover protected code or work around protections. - [ ] Documentation or usability only; no target interaction. - [ ] Not sure yet; investigation or maintainer guidance needed. ## Evidence and regression coverage **Real CLI, main (`ec67aebed`) vs this branch, 3 runs each, macOS 26.7.1 arm64, CLT (Apple LLVM 21):** | `rea inspect-macho` | wall (main → branch) | CPU user+sys | peak RSS* | output bytes | | --- | ---: | ---: | ---: | ---: | | Keynote (60 MB fat) | 7.08 s → **1.25 s** | 7.79 s → 1.91 s | 388 → 374 MB | 756,920 → 756,936 | | Pages | 5.01 s → **1.04 s** | 5.79 s → 1.70 s | 395 → 347 MB | 576,499 → 576,515 | | GarageBand | 3.58 s → **1.07 s** | 4.24 s → 1.70 s | 383 → 348 MB | 540,004 → 540,020 | | Telegram `libswiftCore.dylib` (13,010 exports) | 1.18 s → 1.20 s (noise; 3 runs each) | | | +16 | \*`/usr/bin/time -l` maximum RSS. Agent tokens: the response is unchanged except the 16-byte provenance flag, so the token cost is unchanged; the win is time to an answer. **Correctness:** - `normalized_result` minus `provenance` is byte-identical (`jq -S`) between main and the branch for Keynote, Pages, GarageBand, Transmission, libswiftCore, and three built dylibs: a renamed `-alias` re-export, a `-reexported_symbols_list` re-export, and a `strip -T` Swift dylib. - Raw-tool sweep over 268 slices from 190 Mach-O files in `/Applications` (`Contents/MacOS`, frameworks, dylibs): default `nm -gjU` and `nm -gjU --no-dyldinfo` print the same name set on every slice. Total nm time 32.5 s → 3.2 s; six slices took over 0.5 s (Keynote, Pages and GarageBand, both architectures). - The `strip -T` Swift dylib is the case the fallback exists for: the symbol table holds 0 names and the trie 6. `--no-dyldinfo` prints none, so the branch reruns `nm -gjU` and reports all 6 exports, with that command in provenance. **Tests** (`tests/boundary/providers/native/nativeInventory.golden.test.ts`, through `NativeMacOSProvider`): - The ordinary inventory runs nm once with `--no-dyldinfo`, and provenance records it. - The stripped Swift case (new fixtures captured from the real dylib, with the generation script in the fixture README) reruns `nm -gjU`, keeps the six trie exports, and records only the rerun in provenance. - Both tests fail on main's `NativeMachoInspection.ts`. **Remaining proof gaps:** - The fallback check parses the `dyld_info` rows with the symbol-table names. It would miss a stripped export whose literal name splits into a different name that the symbol table does hold, like `_a [weak-def]` stripped while `_a` remains. I found no such binary; such a row could already be ambiguous on main. - The check parses dyld export rows a second time, about 20 ms or less on libswiftCore's 13k rows. ## Validation performed - `npm run typecheck` — pass - `npm run lint` — pass - `npm run jscpd` — pass (3.19%) - `npm run test:focused` on 51 files (`src/native/**`, `tests/boundary/providers/native/*`, every acceptance/MCP/CLI test mentioning `inspect_macho`) — 557 passed - Real CLI before/after runs and the 268-slice sweep above. ## Compatibility, safety, and release - Breaking changes or migration steps: none. - Real Hopper/Ghidra, browser, or OS coverage: real macOS Xcode CLT tools on the apps listed; Hopper/Ghidra not involved. - Package or release metadata impact: none. - Security, privacy, process, or containment review: same tools, the same read-only operand and at most one extra `nm` launch in the stripped case. ## Before merge - [x] This PR addresses a concrete problem or an agreed enhancement. - [x] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff. - [x] The PR has one focused outcome and the title follows `type(scope): outcome`. - [x] Related issue is linked, or the reason for not linking one is stated above. - [x] Tests cover changed observable behavior and meaningful failure paths. - [x] Owning docs, contracts, and generated metadata are updated where needed. - [x] User-visible CLI/MCP changes include representative output. - [x] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-31 01:08:17 +00:00