[GH-ISSUE #1878] [Bug] Temporary managed row cursors exhaust the retained metadata budget #634

New issue

Closed

opened 2026-10-15 18:26:49 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-15 18:26:49 +00:00

Owner

Originally created by @N0zoM1z0 on GitHub (Oct 11, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/1878

Before submitting

I searched open and closed issues and pull requests for related work.
I removed credentials, private paths, and sensitive target data.

Area

Managed static analysis; Evidence; resource accounting.

Observed behavior

inspect_managed_members still returns coverage.state = unavailable and empty member inventories for valid PE/CLI assemblies after the transient CIL instruction-accounting fix. The reported reason is metadata.representation: the 67,108,864-byte representation budget is exhausted.

One observed inspection fails during custom-attribute metadata inventory, before member/CIL decoding. Another reaches member decoding with most capacity already consumed by earlier metadata lookups.

Reproduction

Select a PE/CLI assembly containing local-method custom-attribute constructors and enough TypeDef/MethodList relationships to require repeated owner lookup.
Run rea inspect-managed-members /path/input.dll --format json.
Inspect normalized_result.coverage and the member arrays.

Observed with two independently selected valid assemblies, about 3.0 MiB and 1.8 MiB. A smaller source-owned regression fixture will be added with the fix.

Expected behavior

Bound retained metadata and its output without permanently accumulating capacity for short-lived row-reading cursors. Preserve the existing representation ceiling, malformed-row checks and meaningful failure reasons.

Root cause and evidence

The source confirms metadataRowCursor permanently calls ManagedDecodeBudget.reserve(256, start) for every cursor construction. declaringTypeForMethod repeatedly reads TypeDef and subsequent MethodList rows while resolving custom attributes. These cursor objects are temporary lookup state, rather than retained facts.

An isolated instrumentation run, preserving the original budget decisions, observed 127,213 cursor charges from the type lookup and 127,212 from the next-row lookup: 65,132,800 bytes of permanent charges for those two paths alone. No member/CIL transient scope had begun at the first failure. The second input incurred 53,411,840 bytes in the same two paths before later decoding exhausted the remaining capacity.

Related: #1801 and its CIL-accounting fixes. This row-cursor ownership problem remains reproducible after #1822. PR #1706 already optimizes ordered attribute-owner lookup and adds caller-selected types; row-budget identity accounting covers repeated row reads independently of that lookup strategy.

REA version

Reproduced with managed sources from main commit bcd350c365030a5d210208c0a8dc238302c43303, which includes #1822.

Runtime and interface

Linux x86_64, Node.js 24.18.0, public CLI, one CPU / 3 GiB. Public MCP parity will be verified with the fix.

Impact

Valid managed targets can lose all usable members before analysis reaches their executable methods. Repeating the unchanged request does not restore capacity.

Originally created by @N0zoM1z0 on GitHub (Oct 11, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/1878 ### Before submitting - [x] I searched open and closed issues and pull requests for related work. - [x] I removed credentials, private paths, and sensitive target data. ### Area Managed static analysis; Evidence; resource accounting. ### Observed behavior `inspect_managed_members` still returns `coverage.state = unavailable` and empty member inventories for valid PE/CLI assemblies after the transient CIL instruction-accounting fix. The reported reason is `metadata.representation`: the 67,108,864-byte representation budget is exhausted. One observed inspection fails during custom-attribute metadata inventory, before member/CIL decoding. Another reaches member decoding with most capacity already consumed by earlier metadata lookups. ### Reproduction 1. Select a PE/CLI assembly containing local-method custom-attribute constructors and enough TypeDef/MethodList relationships to require repeated owner lookup. 2. Run `rea inspect-managed-members /path/input.dll --format json`. 3. Inspect `normalized_result.coverage` and the member arrays. Observed with two independently selected valid assemblies, about 3.0 MiB and 1.8 MiB. A smaller source-owned regression fixture will be added with the fix. ### Expected behavior Bound retained metadata and its output without permanently accumulating capacity for short-lived row-reading cursors. Preserve the existing representation ceiling, malformed-row checks and meaningful failure reasons. ### Root cause and evidence The source confirms `metadataRowCursor` permanently calls `ManagedDecodeBudget.reserve(256, start)` for every cursor construction. `declaringTypeForMethod` repeatedly reads TypeDef and subsequent MethodList rows while resolving custom attributes. These cursor objects are temporary lookup state, rather than retained facts. An isolated instrumentation run, preserving the original budget decisions, observed 127,213 cursor charges from the type lookup and 127,212 from the next-row lookup: 65,132,800 bytes of permanent charges for those two paths alone. No member/CIL transient scope had begun at the first failure. The second input incurred 53,411,840 bytes in the same two paths before later decoding exhausted the remaining capacity. Related: #1801 and its CIL-accounting fixes. This row-cursor ownership problem remains reproducible after #1822. PR #1706 already optimizes ordered attribute-owner lookup and adds caller-selected types; row-budget identity accounting covers repeated row reads independently of that lookup strategy. ### REA version Reproduced with managed sources from main commit `bcd350c365030a5d210208c0a8dc238302c43303`, which includes #1822. ### Runtime and interface Linux x86_64, Node.js 24.18.0, public CLI, one CPU / 3 GiB. Public MCP parity will be verified with the fix. ### Impact Valid managed targets can lose all usable members before analysis reaches their executable methods. Repeating the unchanged request does not restore capacity.

bsakaguchi

2026-10-15 18:26:49 +00:00

closed this issue
added the
performance

P1: high

area: evidence

triage

area: domain

bug
labels

bsakaguchi commented

2026-10-15 18:26:50 +00:00

Author

Owner

@N0zoM1z0 commented on GitHub (Oct 12, 2026):

Follow-up verified in merged PR #1706: inspect_managed_members now supports exact full type-name selection through both CLI and MCP, with the caller's selection and unmatched names preserved in Evidence.

Two normal release DLLs returned complete selected views containing 102 and 16 methods. All 118 CIL bodies were independently checked against bytes mapped from the original PE method headers; CLI/MCP selection and target digests matched. Full artifact metadata and native-boundary metadata remain complete. Full-assembly member extraction still truthfully refuses at the unchanged 64 MiB representation limit; type selection does not guarantee small output because assembly-wide reference metadata is retained.

Validation: 229 focused tests across 21 files, 11 managed conformance controls, real CLI and fresh MCP workflows, static/documentation checks, and all 31 PR CI jobs passed. No local OOM or memory-limit event occurred. Optional decompiler-oracle cases and other providers/clients remain unverified.

Merged commit: a149d183ef. This complements the retained-row accounting correction in #1897 without increasing the representation budget.

@N0zoM1z0 commented on GitHub (Oct 12, 2026): Follow-up verified in merged PR #1706: `inspect_managed_members` now supports exact full type-name selection through both CLI and MCP, with the caller's selection and unmatched names preserved in Evidence. Two normal release DLLs returned complete selected views containing 102 and 16 methods. All 118 CIL bodies were independently checked against bytes mapped from the original PE method headers; CLI/MCP selection and target digests matched. Full artifact metadata and native-boundary metadata remain complete. Full-assembly member extraction still truthfully refuses at the unchanged 64 MiB representation limit; type selection does not guarantee small output because assembly-wide reference metadata is retained. Validation: 229 focused tests across 21 files, 11 managed conformance controls, real CLI and fresh MCP workflows, static/documentation checks, and all 31 PR CI jobs passed. No local OOM or memory-limit event occurred. Optional decompiler-oracle cases and other providers/clients remain unverified. Merged commit: a149d183efd08b0ad7c2a35c99f2394d905621d5. This complements the retained-row accounting correction in #1897 without increasing the representation budget.

bsakaguchi referenced this issue

2026-10-16 21:14:42 +00:00

[PR #634] [MERGED] refactor(contracts): split tool contracts by family #2131

bsakaguchi referenced this issue

2026-10-16 21:14:42 +00:00

[PR #636] [MERGED] refactor(domain): split error taxonomy into thematic modules #2133