morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[PR #5326] [MERGED] feat(il2cpp): read member names, string literals and metadata versions 24 and 27 #6881

New issue

Closed

opened 2026-11-01 21:43:47 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-11-01 21:43:47 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/5326
Author: @8rulerstar
Created: 10/20/2026
Status: ✅ Merged
Merged: 10/20/2026
Merged by: @gensokyo-dev

Base: main ← Head: feat/il2cpp-members

📝 Commits (4)

0c583ba feat(il2cpp): read member names, string literals and metadata versions 24 and 27
5e23301 chore: merge latest main into Unity metadata follow-up
90cab68 fix(il2cpp): bound metadata expansion and query parameter owners
84d1a22 test(il2cpp): cover nonempty property and event tables

📊 Changes

24 files changed (+1707 additions, -315 deletions)

View changed files

📝 .agents/skills/reverse-engineer-anything/SKILL.md (+6 -3)
📝 .agents/skills/reverse-engineer-anything/references/native-and-artifacts.md (+7 -4)
📝 README.md (+1 -1)
📝 docs/cli.md (+6 -5)
📝 docs/il2cpp-metadata.md (+61 -24)
📝 scripts/verify-il2cpp-metadata.mjs (+6 -1)
📝 src/application/il2cpp/Il2cppMetadataService.test.ts (+5 -0)
📝 src/application/il2cpp/Il2cppMetadataService.ts (+7 -1)
📝 src/cli/il2cppCommands.ts (+2 -2)
📝 src/contracts/analysisViewToolContracts.ts (+1 -1)
📝 src/contracts/il2cpp/il2cppToolContracts.ts (+1 -1)
📝 src/domain/analysisView/il2cppMetadataView.test.ts (+122 -4)
📝 src/domain/analysisView/il2cppMetadataView.ts (+125 -17)
📝 src/domain/il2cpp/il2cppMetadata.ts (+148 -20)
➕ src/il2cpp/Il2cppMetadataAdmission.ts (+66 -0)
📝 src/il2cpp/Il2cppMetadataFile.ts (+146 -88)
➕ src/il2cpp/Il2cppMetadataFormat.ts (+53 -0)
➕ src/il2cpp/Il2cppMetadataMembers.ts (+226 -0)
📝 src/il2cpp/Il2cppMetadataProvider.ts (+10 -31)
📝 tests/acceptance/il2cppMetadataCli.test.ts (+114 -5)

...and 4 more files

📄 Description

Summary

Extends inspect_il2cpp_metadata with field, property, event and parameter names and owners, byte-preserving managed string literals, and the observed version24/27 layouts. An analyst can now find a method and request its parameters by method_indices without transferring the entire parameter table. MCP still defaults to a retained summary; full Evidence remains available through CLI/MCP.

Original implementation and cross-version investigation: @8rulerstar, commit 0c583ba61, retained in this branch. Follow-ups synchronize main 1417bf943: runtime changes at 90cab680e complete ownership, resource and compatibility boundaries; 84d1a2244 adds test coverage without changing production code. This is the metadata-only slice of #3784, preceding #5327/#5112 native-address work.

Parse owner ranges once in the owning type/method readers. Member projection consumes those facts; ownership assignment yields to cancellation. Shared format errors/range rules remove the File/Member runtime dependency cycle.
Admit all collection counts before allocation, text/base64 expansion before representation allocation and repeated strings before retaining each row. Decoding caches names/literal spans only within one stable snapshot. Keep exact final64MiB report admission,256MiB input admission, single-reader concurrency, cancellation reasons, failed-source Evidence and cleanup ownership.
Batch parameter-owner filters intersect names/indices and preserve metadata order, paging and mandatory source indices. Newly observed collections carry precise file ranges. A literal has one UTF-8/base64 representation; owner references must resolve within the report.
Historical Evidence remains queryable. Missing member collections have unknown/null counts; requesting them reports unavailable observations, instead of inventing empty inventories. Existing empty arrays remain zero.

Workflow and examples. Static caller-selected metadata only: no target execution, native registration, decryption, mutation or new dependencies/installers. Version24 coverage is the264-byte header layout used by the cited2019.4 sample; other24.x layouts are not guessed.

Executed validation

npm run check passed all6 static tasks and architecture guards at 90cab680e; typecheck/lint passed after the test-only review follow-up. 92 focused tests passed across file/provider/service/domain and public CLI/MCP boundaries, including actual SDK/AJV2020 advertised schemas. Docs generation/check and 5 README tests passed.
Regressions cover repeated long literals, pre-decode/base64 capacity, member-count admission before allocation, owner-loop yields, malformed/empty sentinels, invalid ownership/literal forms, batch filters and historical import after deleting the source. Existing malformed-input, cancellation, retry, cleanup, source identity and path coverage remains. The review follow-up adds nonempty property/event goldens with distinct owners for versions24/27/29/31 and version39 type widths1/2/4, plus public CLI/MCP pages.
Actual merge preview 853f78ede (main 052956af3 + head 84d1a2244, tree 5c7fe1011622014568096abc6c0c11b88071c7ba) passed all92 relevant tests again. Its real Windows SDK/MCP selected-owner task and full Evidence export also passed:6RPC,32,535 estimated used-declaration tokens,256,814 all-input tokens,10.30s wall,12.31s family CPU,1.019GiB sampled family RSS. This integration run includes newer main catalog changes and is separate from the paired measurements below.
Actual Windows x64 and Android arm64 artifacts from MIT-licensed Between Two Worlds1.0.5, Unity2023.2.13f1 / metadata29. Metadata SHA-256: Windows 5691bdd9161d054db69cc842397c7a42c6bf4a544060f05d741e99a6a3a07d29; Android 3814d666a9f83388a71d7ad7ffb67b2cdaf7f2fef5c48c021ddf43373cdad64e.
Final CLI facts equal the original-head facts, input hashes stayed unchanged, and all8 collections agree row-for-row with an independent adapter using Il2CppDumper6.7.46's Metadata API. Literal comparison uses strict UTF-8 or exact base64 bytes from reference-parsed spans. Zero mismatches on either artifact. This closes the previous field/event/parameter comparison gap on these two builds.

Real artifact	Fields	Properties	Events	Parameters	Literals
Windows	30,149	10,399	137	51,080	9,358
Android	29,011	10,198	133	49,921	9,263

Images/types/methods also matched: Windows55/7,261/49,148, Android37/6,838/48,004. Original contributor-reported self-built24/27/31/39 verification remains credited; those real versions/platforms were not rerun here. Generated fixtures exercise supported layouts and malformed inputs, not real-engine qualification. Native-address/Ghidra claims belong to #5327.

Completed-task measurements

Runtime revision90cab680e: context, latency, CPU and memory

Task: read all130 fields of IngameDebugConsole.DebugLogManager and all3 parameters of ReceivedLog (logString, stackTrace, logType), retaining source-scoped row indices and exporting complete Evidence. The old API requires a512-row parameter scan; the new owner selector answers directly. Both delivery variants below use runtime revision 90cab680e, identical files/facts/coverage and Evidence access. The later review commit changes only three test files; production code is identical. Alternating real variants were run in both orders.

Tokenx estimates include used input declarations, arguments, full replies and extra RPCs. The all-declarations counter and whole-wire artifacts include catalog costs; actual model usage/rendering/tool-selection/compaction is unmeasured. Small input is the source-defined fixture, not a real-provider claim.

Task / delivery	RPCs	Used-declaration tokens	All-input-declaration tokens	Wall s	Family CPU s	Sampled family RSS GiB
Small / paged scan	6	26,275	250,554	15.69	14.05	0.880
Small / method filter	6	26,322	250,601	9.78	9.79	0.859
Real / paged scan 1	105	2,152,472	2,376,751	23.31	22.91	2.811
Real / method filter 1	6	32,544	256,823	15.44	13.76	1.051
Real / method filter 2	6	32,544	256,823	11.19	12.23	1.020
Real / paged scan 2	105	2,152,472	2,376,751	18.08	22.07	2.093

Original implementation at 5e233013f: real task105RPC /2,152,408 used-declaration tokens /2,376,687 all-input tokens /14.64s wall /19.77s CPU /3.029GiB sampledRSS. Admission and validation add work; no general decoder speedup is claimed. The primary benefit comes from avoiding unnecessary transfer and repeated queries. Small inputs incur47 extra estimated tokens for explicit selection; latency varies with host load.

Whole harness includes client/catalog/schema validation, initial full decoding, projection, export and teardown. Separate retained-field reads: [24.68, 27.92, 22.11] ms; probes are excluded from primary counters. The complete parent is immutable and reused without re-reading the source.

Linux/Node24.18,2CPU affinity,3GiB V8 heaps,8GiB family stop guard; no guard stops/OOM/retries in measured tasks. RSS sampled20ms across the process family may double-count shared pages or miss short peaks. Runtime caches fresh, OS caches/host load uncontrolled. Engine-only costs unmeasured. The64MiB encoded-report budget is not a total-RSS bound. Reference tools/SDK lived in private verification directories and were not added to REA.

Expected effects:

Read existing files or captured data.
Change files: only caller-requested exclusive CLI Evidence output.

Author readiness

Human owner: @gensokyo-dev

Original contribution: @8rulerstar.

The human owner explicitly authorized this work, delegated technical review and validation to Codex, directed these declarations on their behalf, and requested ready-for-review submission and normal merge. This records that direction; it does not assert an independent human review of the follow-up or that the original contributor reviewed these later commits.

I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
I declare this PR ready for initial maintainer review.

Before merge

This PR addresses a concrete problem or an agreed enhancement.
Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
The PR has one focused outcome and the title follows type(scope): outcome.
Related issue is linked, or the reason for not linking one is stated above.
Tests cover changed observable behavior and meaningful failure paths.
Owning docs, contracts, and generated metadata are updated where needed.
User-visible CLI/MCP changes include representative output.
I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/5326 **Author:** [@8rulerstar](https://github.com/8rulerstar) **Created:** 10/20/2026 **Status:** ✅ Merged **Merged:** 10/20/2026 **Merged by:** [@gensokyo-dev](https://github.com/gensokyo-dev) **Base:** `main` ← **Head:** `feat/il2cpp-members` --- ### 📝 Commits (4) - [`0c583ba`](https://github.com/morluto/rea/commit/0c583ba614da000d41e59e26d18c4753f62c056d) feat(il2cpp): read member names, string literals and metadata versions 24 and 27 - [`5e23301`](https://github.com/morluto/rea/commit/5e233013fc949acef0b180a22fe7063063211926) chore: merge latest main into Unity metadata follow-up - [`90cab68`](https://github.com/morluto/rea/commit/90cab680ef17b96c421afc2a9e2fdc4f2231bced) fix(il2cpp): bound metadata expansion and query parameter owners - [`84d1a22`](https://github.com/morluto/rea/commit/84d1a2244c53031b1a34af2c42c744ce522b67aa) test(il2cpp): cover nonempty property and event tables ### 📊 Changes **24 files changed** (+1707 additions, -315 deletions) <details> <summary>View changed files</summary> 📝 `.agents/skills/reverse-engineer-anything/SKILL.md` (+6 -3) 📝 `.agents/skills/reverse-engineer-anything/references/native-and-artifacts.md` (+7 -4) 📝 `README.md` (+1 -1) 📝 `docs/cli.md` (+6 -5) 📝 `docs/il2cpp-metadata.md` (+61 -24) 📝 `scripts/verify-il2cpp-metadata.mjs` (+6 -1) 📝 `src/application/il2cpp/Il2cppMetadataService.test.ts` (+5 -0) 📝 `src/application/il2cpp/Il2cppMetadataService.ts` (+7 -1) 📝 `src/cli/il2cppCommands.ts` (+2 -2) 📝 `src/contracts/analysisViewToolContracts.ts` (+1 -1) 📝 `src/contracts/il2cpp/il2cppToolContracts.ts` (+1 -1) 📝 `src/domain/analysisView/il2cppMetadataView.test.ts` (+122 -4) 📝 `src/domain/analysisView/il2cppMetadataView.ts` (+125 -17) 📝 `src/domain/il2cpp/il2cppMetadata.ts` (+148 -20) ➕ `src/il2cpp/Il2cppMetadataAdmission.ts` (+66 -0) 📝 `src/il2cpp/Il2cppMetadataFile.ts` (+146 -88) ➕ `src/il2cpp/Il2cppMetadataFormat.ts` (+53 -0) ➕ `src/il2cpp/Il2cppMetadataMembers.ts` (+226 -0) 📝 `src/il2cpp/Il2cppMetadataProvider.ts` (+10 -31) 📝 `tests/acceptance/il2cppMetadataCli.test.ts` (+114 -5) _...and 4 more files_ </details> ### 📄 Description ## Summary Extends `inspect_il2cpp_metadata` with field, property, event and parameter names and owners, byte-preserving managed string literals, and the observed version24/27 layouts. An analyst can now find a method and request its parameters by `method_indices` without transferring the entire parameter table. MCP still defaults to a retained summary; full Evidence remains available through CLI/MCP. Original implementation and cross-version investigation: **@8rulerstar**, commit `0c583ba61`, retained in this branch. Follow-ups synchronize main `1417bf943`: runtime changes at `90cab680e` complete ownership, resource and compatibility boundaries; `84d1a2244` adds test coverage without changing production code. This is the metadata-only slice of #3784, preceding #5327/#5112 native-address work. - Parse owner ranges once in the owning type/method readers. Member projection consumes those facts; ownership assignment yields to cancellation. Shared format errors/range rules remove the File/Member runtime dependency cycle. - Admit all collection counts before allocation, text/base64 expansion before representation allocation and repeated strings before retaining each row. Decoding caches names/literal spans only within one stable snapshot. Keep exact final64MiB report admission,256MiB input admission, single-reader concurrency, cancellation reasons, failed-source Evidence and cleanup ownership. - Batch parameter-owner filters intersect names/indices and preserve metadata order, paging and mandatory source indices. Newly observed collections carry precise file ranges. A literal has one UTF-8/base64 representation; owner references must resolve within the report. - Historical Evidence remains queryable. Missing member collections have unknown/null counts; requesting them reports unavailable observations, instead of inventing empty inventories. Existing empty arrays remain zero. [Workflow and examples](https://github.com/morluto/rea/blob/main/docs/il2cpp-metadata.md). Static caller-selected metadata only: no target execution, native registration, decryption, mutation or new dependencies/installers. Version24 coverage is the264-byte header layout used by the cited2019.4 sample; other24.x layouts are not guessed. ## Executed validation - `npm run check` passed all6 static tasks and architecture guards at `90cab680e`; typecheck/lint passed after the test-only review follow-up. **92 focused tests** passed across file/provider/service/domain and public CLI/MCP boundaries, including actual SDK/AJV2020 advertised schemas. Docs generation/check and **5 README tests** passed. - Regressions cover repeated long literals, pre-decode/base64 capacity, member-count admission before allocation, owner-loop yields, malformed/empty sentinels, invalid ownership/literal forms, batch filters and historical import after deleting the source. Existing malformed-input, cancellation, retry, cleanup, source identity and path coverage remains. The review follow-up adds nonempty property/event goldens with distinct owners for versions24/27/29/31 and version39 type widths1/2/4, plus public CLI/MCP pages. - Actual merge preview `853f78ede` (main `052956af3` + head `84d1a2244`, tree `5c7fe1011622014568096abc6c0c11b88071c7ba`) passed all92 relevant tests again. Its real Windows SDK/MCP selected-owner task and full Evidence export also passed:6RPC,32,535 estimated used-declaration tokens,256,814 all-input tokens,10.30s wall,12.31s family CPU,1.019GiB sampled family RSS. This integration run includes newer main catalog changes and is separate from the paired measurements below. - Actual **Windows x64 and Android arm64** artifacts from MIT-licensed [Between Two Worlds1.0.5](https://github.com/SkS-GameDev/Between-two-worlds/releases/tag/1.0.5), Unity2023.2.13f1 / metadata29. Metadata SHA-256: Windows `5691bdd9161d054db69cc842397c7a42c6bf4a544060f05d741e99a6a3a07d29`; Android `3814d666a9f83388a71d7ad7ffb67b2cdaf7f2fef5c48c021ddf43373cdad64e`. - Final CLI facts equal the original-head facts, input hashes stayed unchanged, and all8 collections agree **row-for-row** with an independent adapter using [Il2CppDumper6.7.46's Metadata API](https://github.com/Perfare/Il2CppDumper/blob/v6.7.46/Il2CppDumper/Il2Cpp/Metadata.cs). Literal comparison uses strict UTF-8 or exact base64 bytes from reference-parsed spans. **Zero mismatches** on either artifact. This closes the previous field/event/parameter comparison gap on these two builds. | Real artifact | Fields | Properties | Events | Parameters | Literals | | --- | ---: | ---: | ---: | ---: | ---: | | Windows |30,149|10,399|137|51,080|9,358| | Android |29,011|10,198|133|49,921|9,263| Images/types/methods also matched: Windows55/7,261/49,148, Android37/6,838/48,004. Original contributor-reported self-built24/27/31/39 verification remains credited; those real versions/platforms were **not rerun here**. Generated fixtures exercise supported layouts and malformed inputs, not real-engine qualification. Native-address/Ghidra claims belong to #5327. ## Completed-task measurements <details> <summary>Runtime revision90cab680e: context, latency, CPU and memory</summary> Task: read all130 fields of `IngameDebugConsole.DebugLogManager` and all3 parameters of `ReceivedLog` (`logString`, `stackTrace`, `logType`), retaining source-scoped row indices and exporting complete Evidence. The old API requires a512-row parameter scan; the new owner selector answers directly. Both delivery variants below use runtime revision `90cab680e`, identical files/facts/coverage and Evidence access. The later review commit changes only three test files; production code is identical. Alternating real variants were run in both orders. Tokenx estimates include used input declarations, arguments, full replies and extra RPCs. The all-declarations counter and whole-wire artifacts include catalog costs; actual model usage/rendering/tool-selection/compaction is unmeasured. Small input is the source-defined fixture, not a real-provider claim. | Task / delivery | RPCs | Used-declaration tokens | All-input-declaration tokens | Wall s | Family CPU s | Sampled family RSS GiB | | --- | ---: | ---: | ---: | ---: | ---: | ---: | | Small / paged scan | 6 | 26,275 | 250,554 | 15.69 | 14.05 | 0.880 | | Small / method filter | 6 | 26,322 | 250,601 | 9.78 | 9.79 | 0.859 | | Real / paged scan 1 | 105 | 2,152,472 | 2,376,751 | 23.31 | 22.91 | 2.811 | | Real / method filter 1 | 6 | 32,544 | 256,823 | 15.44 | 13.76 | 1.051 | | Real / method filter 2 | 6 | 32,544 | 256,823 | 11.19 | 12.23 | 1.020 | | Real / paged scan 2 | 105 | 2,152,472 | 2,376,751 | 18.08 | 22.07 | 2.093 | Original implementation at `5e233013f`: real task105RPC /2,152,408 used-declaration tokens /2,376,687 all-input tokens /14.64s wall /19.77s CPU /3.029GiB sampledRSS. Admission and validation add work; no general decoder speedup is claimed. The primary benefit comes from avoiding unnecessary transfer and repeated queries. Small inputs incur47 extra estimated tokens for explicit selection; latency varies with host load. Whole harness includes client/catalog/schema validation, initial full decoding, projection, export and teardown. Separate retained-field reads: **[24.68, 27.92, 22.11] ms**; probes are excluded from primary counters. The complete parent is immutable and reused without re-reading the source. Linux/Node24.18,2CPU affinity,3GiB V8 heaps,8GiB family stop guard; no guard stops/OOM/retries in measured tasks. RSS sampled20ms across the process family may double-count shared pages or miss short peaks. Runtime caches fresh, OS caches/host load uncontrolled. Engine-only costs unmeasured. The64MiB encoded-report budget is not a total-RSS bound. Reference tools/SDK lived in private verification directories and were not added to REA. </details> Expected effects: - [x] Read existing files or captured data. - [x] Change files: only caller-requested exclusive CLI Evidence output. ## Author readiness Human owner: @gensokyo-dev Original contribution: @8rulerstar. The human owner explicitly authorized this work, delegated technical review and validation to Codex, directed these declarations on their behalf, and requested ready-for-review submission and normal merge. This records that direction; it does not assert an independent human review of the follow-up or that the original contributor reviewed these later commits. - [x] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [x] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [x] I declare this PR ready for initial maintainer review. ## Before merge - [x] This PR addresses a concrete problem or an agreed enhancement. - [x] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff. - [x] The PR has one focused outcome and the title follows `type(scope): outcome`. - [x] Related issue is linked, or the reason for not linking one is stated above. - [x] Tests cover changed observable behavior and meaningful failure paths. - [x] Owning docs, contracts, and generated metadata are updated where needed. - [x] User-visible CLI/MCP changes include representative output. - [x] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-11-01 21:43:47 +00:00

closed this issue
added the
pull-request
label

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#6881

No description provided.

Rows
Columns