[PR #1768] [MERGED] fix(managed): charge retained member facts instead of decoding bounds #4804

Closed
opened 2026-10-31 00:55:14 +00:00 by bsakaguchi · 0 comments
Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/1768
Author: @flitzrrr
Created: 10/11/2026
Status: ✅ Merged
Merged: 10/11/2026
Merged by: @morluto

Base: main ← Head: fix/managed-member-decode-budget


📝 Commits (4)

  • 70a862f fix(managed): charge retained member facts instead of decoding bounds
  • b82218d fix(managed): bound concurrent decoding and count JSON bytes in place
  • dd64d3c fix(managed): admit derived call and field edges as they are built
  • e376cf2 fix(managed): admit derived caller names before building them

📊 Changes

7 files changed (+315 additions, -36 deletions)

View changed files

➕ src/dotnet/ManagedDecodeBudget.test.ts (+53 -0)
📝 src/dotnet/ManagedDecodeBudget.ts (+53 -7)
📝 src/dotnet/ManagedMemberInspector.test.ts (+127 -0)
📝 src/dotnet/ManagedMemberInspector.ts (+1 -0)
📝 src/dotnet/ManagedMemberInstructionDecoder.ts (+2 -1)
📝 src/dotnet/ManagedMemberRows.ts (+67 -24)
📝 src/dotnet/ManagedMethodBodyReader.ts (+12 -4)

📄 Description

Summary

inspect_managed_members returns no types, methods or call edges for ordinary mid-size assemblies. For example, Newtonsoft.Json 13.0.3 (lib/netstandard2.0, 705,296 bytes) reports coverage.state: "unavailable" with "Managed metadata exceeds the 67108864 byte representation budget". The budget was charged with decoding upper bounds that were never returned, rather than with the facts that are actually retained. After this change the same file yields 496 types, 4,185 methods and 18,846 call edges.

I did not open a separate issue because the reproduction uses a public NuGet package and is included below.

Problem and expected behavior

Before decoding, every method body reserved 256 bytes per IL byte (ManagedMethodBodyReader), every exception section 256 bytes per clause, and every member signature 256 bytes per blob byte (budgetedSignature). These reservations bound transient decoding bookkeeping, but they stayed charged for the whole inspection. The budget therefore scaled with the total IL and signature size, not with the output. Instrumenting the budget on the base revision for the Newtonsoft.Json file above showed 41.0 MiB from method-body bounds and 10.8 MiB from signature bounds, before any projection.

The final projection admission (admitManagedProjection) also charged every decoded string at six bytes per UTF-16 unit, every key at the same rate, and every scalar at 32 bytes. That is roughly eight times its JSON size.

Expected: the budget limits what the inspection retains and projects, so assemblies whose facts fit are inspected completely.

Change and scope

  • ManagedDecodeBudget.admitTransient checks a decoding bound against the remaining capacity without consuming it. reserve now builds on it.
  • IL, exception-clause and signature decoding use admitTransient for their bounds, then reserve the decoded body or signature that is retained, through admitManagedProjection.
  • admitManagedProjection charges the exact UTF-8 JSON bytes of decoded strings, keys and scalars. Escaped control characters and multibyte text still count at their encoded size. Heap-string reservations made before decoding keep their worst-case bound because the content is unknown at that point.

Not included: when the retained facts genuinely exceed the budget, members are still reported as unavailable rather than partial. System.Private.ServiceModel 4.7.0 (2.3 MB) remains such a case. inspect_managed_artifact similarly labels a budget stop as partial without facts. Returning partial member inventories would change what compare_managed_members and the managed application graph can assume, so it deserves its own change.

Contract and boundary impact

  • Semantic owner and earliest changed stage: managed static provider decoding (src/dotnet/ManagedDecodeBudget.ts, ManagedMethodBodyReader.ts, ManagedMemberInstructionDecoder.ts, ManagedMemberRows.ts).
  • CLI and MCP/tool-catalog contract: none; schemas and tools are unchanged. More assemblies return populated results instead of unavailable.
  • Provider, bridge, target-format, or platform compatibility: PE/CLI only; no change to admitted formats.
  • Evidence, artifact, provenance, or reconstruction contract: unchanged shapes; the representation budget and its 64 MiB value are unchanged.
  • Process execution, authorization, cleanup, or containment impact: none; no target code is loaded. Per-item decoding stays bounded before allocation.
  • Generated metadata (docs/public/product-catalog.json), package, or installation impact: none.

Evidence and regression coverage

  • Tests added or updated:
    • src/dotnet/ManagedMemberInspector.test.ts: a fixture method with 200,001 IL bytes plus an 80,003-byte signature. Each bound fits alone, but together they exceeded 64 MiB on the base revision.
    • src/dotnet/ManagedDecodeBudget.test.ts: exact charging of a 12 MiB ASCII string; escaped and multibyte text still exceeds the budget at its encoded size.
  • Base reproduction: both new "charges decoded output" and "charges decoded strings" cases fail on the base revision (the first returns unavailable) and pass with this change.
  • User-visible CLI/MCP output: rea inspect-managed-members ~/.nuget/packages/newtonsoft.json/13.0.3/lib/netstandard2.0/Newtonsoft.Json.dll --format json
    • Before: types: [], methods: [], call_edges: [], coverage: { state: "unavailable", issues: [{ code: "resource-limit", detail: "Managed metadata exceeds the 67108864 byte representation budget" }] }
    • After: 496 types, 4,185 methods, 18,846 call edges, coverage.state: "partial" with no issues. The partial state comes from member signatures or bodies that are not fully decoded, as before.
  • Remaining proof gaps: no Windows or Linux run; no real-provider lane applies.

For evidence-bearing changes:

  • Observed, derived, and inferred claims remain distinguishable.
  • Artifact identity, source provenance, and failed attempts remain preserved.
  • Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

On macOS (Darwin 27.0, Apple Silicon), Node 24.18.0:

  • npm run test:fast — 376 files, 4272 passed, 7 skipped
  • npm run test:focused -- tests/boundary/providers/managed/*.test.ts tests/conformance/providers/managed/managedNativeVerification.test.ts tests/boundary/mcp/managedMcp.test.ts tests/acceptance/managedCliOutput.test.ts — 9 files, 44 passed
  • npm run check — passed
  • npm run verify:test-discovery — 814 test files discovered exactly once
  • npm run build:cached and the CLI run above — passed

Review follow-up

  • b82218de: transient decoding bounds accumulate per method or signature until released; JSON bytes of decoded strings are counted without serializing.
  • dd64d3c5: derived call and field-access records are admitted as they are built.
  • New regressions (IL plus EH over budget, derived edges over budget) fail on the previous head and pass here.
  • npx vitest run src/dotnet — 92 passed; managed boundary, MCP and acceptance — 44 passed; npm run check:fast — passed. Newtonsoft.Json and the 730 KB assembly still return every member.

Compatibility, safety, and release

  • Breaking changes or migration steps: none.
  • Real Hopper/Ghidra, browser, or OS coverage: not applicable; macOS only.
  • Package or release metadata impact: none.
  • Security, privacy, process, or containment review: total retained output stays bounded by the unchanged budget. Transient decoding is still checked before each allocation, and only its capacity is no longer consumed.

Review checklist

  • This PR addresses a concrete problem or an agreed enhancement.
  • The PR has one focused outcome and the title follows type(scope): outcome.
  • Related issue is linked, or the reason for not linking one is stated above.
  • Tests cover changed observable behavior and meaningful failure paths.
  • Owning docs, contracts, and generated metadata are updated where needed.
  • User-visible CLI/MCP changes include representative output.
  • I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

🤖 Generated with Claude Code


🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/1768 **Author:** [@flitzrrr](https://github.com/flitzrrr) **Created:** 10/11/2026 **Status:** ✅ Merged **Merged:** 10/11/2026 **Merged by:** [@morluto](https://github.com/morluto) **Base:** `main` ← **Head:** `fix/managed-member-decode-budget` --- ### 📝 Commits (4) - [`70a862f`](https://github.com/morluto/rea/commit/70a862f39a9efd902f4f848791b21f928bd76f6d) fix(managed): charge retained member facts instead of decoding bounds - [`b82218d`](https://github.com/morluto/rea/commit/b82218de325080b1f5f72265302d3d38e2bb8a3e) fix(managed): bound concurrent decoding and count JSON bytes in place - [`dd64d3c`](https://github.com/morluto/rea/commit/dd64d3c56f2d7cdb94fd677f8002837ae9a21ba4) fix(managed): admit derived call and field edges as they are built - [`e376cf2`](https://github.com/morluto/rea/commit/e376cf2399e5989bd0c94d9f496d60f5e5bf60fc) fix(managed): admit derived caller names before building them ### 📊 Changes **7 files changed** (+315 additions, -36 deletions) <details> <summary>View changed files</summary> ➕ `src/dotnet/ManagedDecodeBudget.test.ts` (+53 -0) 📝 `src/dotnet/ManagedDecodeBudget.ts` (+53 -7) 📝 `src/dotnet/ManagedMemberInspector.test.ts` (+127 -0) 📝 `src/dotnet/ManagedMemberInspector.ts` (+1 -0) 📝 `src/dotnet/ManagedMemberInstructionDecoder.ts` (+2 -1) 📝 `src/dotnet/ManagedMemberRows.ts` (+67 -24) 📝 `src/dotnet/ManagedMethodBodyReader.ts` (+12 -4) </details> ### 📄 Description ## Summary `inspect_managed_members` returns no types, methods or call edges for ordinary mid-size assemblies. For example, `Newtonsoft.Json` 13.0.3 (`lib/netstandard2.0`, 705,296 bytes) reports `coverage.state: "unavailable"` with "Managed metadata exceeds the 67108864 byte representation budget". The budget was charged with decoding upper bounds that were never returned, rather than with the facts that are actually retained. After this change the same file yields 496 types, 4,185 methods and 18,846 call edges. I did not open a separate issue because the reproduction uses a public NuGet package and is included below. ## Problem and expected behavior Before decoding, every method body reserved 256 bytes per IL byte (`ManagedMethodBodyReader`), every exception section 256 bytes per clause, and every member signature 256 bytes per blob byte (`budgetedSignature`). These reservations bound transient decoding bookkeeping, but they stayed charged for the whole inspection. The budget therefore scaled with the total IL and signature size, not with the output. Instrumenting the budget on the base revision for the Newtonsoft.Json file above showed 41.0 MiB from method-body bounds and 10.8 MiB from signature bounds, before any projection. The final projection admission (`admitManagedProjection`) also charged every decoded string at six bytes per UTF-16 unit, every key at the same rate, and every scalar at 32 bytes. That is roughly eight times its JSON size. Expected: the budget limits what the inspection retains and projects, so assemblies whose facts fit are inspected completely. ## Change and scope - `ManagedDecodeBudget.admitTransient` checks a decoding bound against the remaining capacity without consuming it. `reserve` now builds on it. - IL, exception-clause and signature decoding use `admitTransient` for their bounds, then reserve the decoded body or signature that is retained, through `admitManagedProjection`. - `admitManagedProjection` charges the exact UTF-8 JSON bytes of decoded strings, keys and scalars. Escaped control characters and multibyte text still count at their encoded size. Heap-string reservations made before decoding keep their worst-case bound because the content is unknown at that point. Not included: when the retained facts genuinely exceed the budget, members are still reported as unavailable rather than partial. `System.Private.ServiceModel` 4.7.0 (2.3 MB) remains such a case. `inspect_managed_artifact` similarly labels a budget stop as partial without facts. Returning partial member inventories would change what `compare_managed_members` and the managed application graph can assume, so it deserves its own change. ## Contract and boundary impact - Semantic owner and earliest changed stage: managed static provider decoding (`src/dotnet/ManagedDecodeBudget.ts`, `ManagedMethodBodyReader.ts`, `ManagedMemberInstructionDecoder.ts`, `ManagedMemberRows.ts`). - CLI and MCP/tool-catalog contract: none; schemas and tools are unchanged. More assemblies return populated results instead of `unavailable`. - Provider, bridge, target-format, or platform compatibility: PE/CLI only; no change to admitted formats. - Evidence, artifact, provenance, or reconstruction contract: unchanged shapes; the representation budget and its 64 MiB value are unchanged. - Process execution, authorization, cleanup, or containment impact: none; no target code is loaded. Per-item decoding stays bounded before allocation. - Generated metadata (`docs/public/product-catalog.json`), package, or installation impact: none. ## Evidence and regression coverage - Tests added or updated: - `src/dotnet/ManagedMemberInspector.test.ts`: a fixture method with 200,001 IL bytes plus an 80,003-byte signature. Each bound fits alone, but together they exceeded 64 MiB on the base revision. - `src/dotnet/ManagedDecodeBudget.test.ts`: exact charging of a 12 MiB ASCII string; escaped and multibyte text still exceeds the budget at its encoded size. - Base reproduction: both new "charges decoded output" and "charges decoded strings" cases fail on the base revision (the first returns `unavailable`) and pass with this change. - User-visible CLI/MCP output: `rea inspect-managed-members ~/.nuget/packages/newtonsoft.json/13.0.3/lib/netstandard2.0/Newtonsoft.Json.dll --format json` - Before: `types: [], methods: [], call_edges: [], coverage: { state: "unavailable", issues: [{ code: "resource-limit", detail: "Managed metadata exceeds the 67108864 byte representation budget" }] }` - After: 496 types, 4,185 methods, 18,846 call edges, `coverage.state: "partial"` with no issues. The partial state comes from member signatures or bodies that are not fully decoded, as before. - Remaining proof gaps: no Windows or Linux run; no real-provider lane applies. For evidence-bearing changes: - [x] Observed, derived, and inferred claims remain distinguishable. - [x] Artifact identity, source provenance, and failed attempts remain preserved. - [x] Unsupported, incomplete, unavailable, or uncertain outcomes remain visible. ## Validation performed On macOS (Darwin 27.0, Apple Silicon), Node 24.18.0: - `npm run test:fast` — 376 files, 4272 passed, 7 skipped - `npm run test:focused -- tests/boundary/providers/managed/*.test.ts tests/conformance/providers/managed/managedNativeVerification.test.ts tests/boundary/mcp/managedMcp.test.ts tests/acceptance/managedCliOutput.test.ts` — 9 files, 44 passed - `npm run check` — passed - `npm run verify:test-discovery` — 814 test files discovered exactly once - `npm run build:cached` and the CLI run above — passed ### Review follow-up - b82218de: transient decoding bounds accumulate per method or signature until released; JSON bytes of decoded strings are counted without serializing. - dd64d3c5: derived call and field-access records are admitted as they are built. - New regressions (IL plus EH over budget, derived edges over budget) fail on the previous head and pass here. - `npx vitest run src/dotnet` — 92 passed; managed boundary, MCP and acceptance — 44 passed; `npm run check:fast` — passed. Newtonsoft.Json and the 730 KB assembly still return every member. ## Compatibility, safety, and release - Breaking changes or migration steps: none. - Real Hopper/Ghidra, browser, or OS coverage: not applicable; macOS only. - Package or release metadata impact: none. - Security, privacy, process, or containment review: total retained output stays bounded by the unchanged budget. Transient decoding is still checked before each allocation, and only its capacity is no longer consumed. ## Review checklist - [x] This PR addresses a concrete problem or an agreed enhancement. - [x] The PR has one focused outcome and the title follows `type(scope): outcome`. - [x] Related issue is linked, or the reason for not linking one is stated above. - [x] Tests cover changed observable behavior and meaningful failure paths. - [x] Owning docs, contracts, and generated metadata are updated where needed. - [x] User-visible CLI/MCP changes include representative output. - [x] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
bsakaguchi 2026-10-31 00:55:14 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
morluto/rea#4804
No description provided.