morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #6780] [Bug] Nested duplicate byte-string keys evade EVM metadata validation #4134

New issue

Closed

opened 2026-10-31 00:47:03 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:47:03 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 23, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6780

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

The public inspect-evm-interface CLI returns bytecode.metadata.status: "partial" for a terminal CBOR map containing duplicate byte-string keys inside a nested map. It reports only the normal nested-container limitation and retains the solc declaration. Equivalent duplicate text keys in that nested map correctly return not-decoded, clear interpreted entries/compiler declaration, and identify the duplicate.

The current root-level duplicate-byte-key control also correctly returns not-decoded. This is a remaining nested-value case, rather than a reproduction of the resolved root-level behavior in #3991. Each of the four CLI requests below ran once, exited 0, and emitted empty stderr.

Reproduction

Use the built source checkout at the revision below with its locked dependencies and pinned Node toolchain. Set REA_ROOT to that checkout. Save the complete script below as /tmp/nested-duplicate-cli.mjs, then run:

node /tmp/nested-duplicate-cli.mjs "$REA_ROOT"

import {mkdtempSync, mkdirSync, writeFileSync} from 'node:fs';
import {tmpdir} from 'node:os';
import {join, resolve} from 'node:path';
import {spawnSync} from 'node:child_process';
const repo=resolve(process.argv[2]);
const output=resolve(process.argv[3] ?? mkdtempSync(join(tmpdir(),'rea-cbor-')));
const home=join(output,'home');mkdirSync(home,{recursive:true});
const env={...process.env,HOME:home,XDG_CONFIG_HOME:join(home,'config'),XDG_DATA_HOME:join(home,'data'),XDG_CACHE_HOME:join(home,'cache'),XDG_STATE_HOME:join(home,'state'),NO_COLOR:'1'};
const code=Buffer.from('60003560e01c630000000014601057005b00','hex');
const fixtures={
 distinctNestedBytes:'a264736f6c634300081e6178a2410101410202',
 duplicateNestedBytes:'a264736f6c634300081e6178a2410101410102',
 duplicateNestedText:'a264736f6c634300081e6178a2616101616102',
 duplicateRootBytes:'a364736f6c634300081e410101410102',
};
for(const [name,hex] of Object.entries(fixtures)){
 const payload=Buffer.from(hex,'hex'),length=Buffer.alloc(2);length.writeUInt16BE(payload.length);
 const selected=Buffer.concat([code,payload,length]);
 const path=join(output,name+'.hex');writeFileSync(path,selected.toString('hex'));
 const run=spawnSync(process.execPath,[join(repo,'scripts/rea.mjs'),'inspect-evm-interface',path,'hex','--json'],{cwd:output,env,encoding:'utf8',timeout:60000,maxBuffer:4*1024*1024});
 writeFileSync(join(output,name+'.stdout'),run.stdout??'');writeFileSync(join(output,name+'.stderr'),run.stderr??'');
 if(run.status!==0)throw new Error(`${name}: exit=${run.status}: ${run.stderr}`);
 const document=JSON.parse(run.stdout),report=(document.data??document).normalized_result;
 console.log(JSON.stringify({name,exit:run.status,stderr:run.stderr,bytecode_bytes:report.bytecode.bytes,functions:report.functions,metadata:report.bytecode.metadata}));
}

The nested-byte payload represents {"solc": h'00081e', "x": {h'01': 1, h'01': 2}}. The distinct-byte control changes the second byte-string key to 02. The other controls use repeated text keys inside x, and repeated byte-string keys at the root. The terminal lengths are computed from each payload. All artifacts are synthetic flat EVM bytes containing a small dispatcher; no contract execution is involved.

Expected behavior

Return not-decoded with a duplicate-key diagnostic for the invalid nested byte-key map, preserving its candidate offset/length and original bytecode while leaving interpreted entries empty and compiler_version null. Keep the valid distinct-key nested map as partial.

The metadata contract distinguishes partial scalar coverage for nested containers from unsupported/malformed candidates, which must return not-decoded with a diagnostic. RFC 8949 §3.1 identifies duplicate-key maps as invalid; §5.6.1 compares byte-string keys by their bytes. This does not require exposing nested values in REA's scalar view: the demonstrated nested-text control already enforces validity separately from presentation coverage.

Impact and scope

The machine-readable metadata status and diagnostic do not distinguish an invalid nested byte-key map from the valid distinct-key map. A consumer relying on the interpretation status receives no duplicate-key warning, and the invalid trailer still contributes a decoded compiler-version declaration. That declaration remains only a metadata assertion, not verified compiler identity.

Observed scope is the small generated nested-map fixtures through the real public CLI, worker, bundled EVMole/WASM and independent metadata parser. Full original bytecode and the correct metadata range remain available; no evidence loss, wrong compiler authenticity claim, target execution or security effect is asserted. Prevalence in real artifacts and other nesting shapes are unknown.

For this bounded fixture, callers can inspect the retained metadata range and recognize a2 4101 01 4101 02 as two entries with the same byte key. No automatic REA workaround was established.

REA version

rea --version: 6.7.0. Source revision 98ca6c82705bf8c44f814620636852f938d827ca, using that checkout's compiled runtime. The audit coordinator established rea-agents-6.7.0 as the latest published release; the npm release artifact was not separately executed.

Runtime and platform

Node.js v24.22.1; Linux 7.2.8-9-omarchy, x86_64; repository-locked dependencies including bundled EVMole 0.9.3. No blockchain service, wallet, target runtime, Solidity compiler, Hopper or Ghidra was used.

Target and artifact context

Generated ASCII hex carriers for 39-byte flat EVM bytecode in the three nested-map cases, and 36 bytes in the root-duplicate control. Each has an 18-byte dispatcher, a CBOR map, and its two-byte big-endian length suffix. The metadata is synthetic; 0.8.30 is an intentionally supplied declaration.

Interface

CLI

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

Observed selected fields from the four public CLI responses:

name                    exit  metadata.status  entries  compiler_version  duplicate diagnostic
 distinctNestedBytes      0   partial          2        0.8.30            no
 duplicateNestedBytes     0   partial          2        0.8.30            no
 duplicateNestedText      0   not-decoded      0        null              yes
 duplicateRootBytes       0   not-decoded      0        null              yes

Both nested-byte cases return the same interpreted metadata object, including:

{
  "status": "partial",
  "bytecode_offset": 18,
  "cbor_bytes": 19,
  "compiler_version": {"representation":"release-bytes","value":"0.8.30"},
  "diagnostic": "Nested containers are uninterpreted; their exact bytes remain at the recorded range in original bytecode."
}

The nested-text control returns CBOR decode error: found repeat map key "a". The root-byte control returns a repeat-map-key diagnostic for {"kind":"bytes","value":"01"}. All four retain the inferred selector 0x00000000 at decoded offset 16 and have empty stderr.

Expected source: the pinned documentation and RFC sections linked above.
Failure source: executed public CLI output from the complete script above.
Evidence provenance: observed CLI results; the causal explanation below is source-derived.
Local verification: reproduced.
Reproduction completeness: complete for the four reduced offline CLI fixtures. Real deployed or compiler-generated artifacts, other platforms and the published npm artifact were not tested.

Typed outcome or recovery expected

Keep successful interface inspection and original byte identities, but represent the invalid metadata interpretation as not-decoded with its exact candidate range, a duplicate-key diagnostic, no interpreted entries and no compiler-version declaration. Preserve partial for valid nested-container controls.

Analysis or suspected cause

Source-derived: the parser enables duplicate-key rejection, but cborg compares decoded byte-string objects by identity. REA's additional value-based check iterates only the root Map. A nested Map is projected as an uninterpreted container without that check. Compiler-version projection then retains the root solc value. The actual worker puts this independent interpretation into the successful bytecode result.

Additional context

Searched the supplied open-issue snapshot plus GitHub open and closed issue searches for CBOR, EVM metadata, "nested" "CBOR" and "metadata" "duplicate". No existing nested-value byte-key report was found.

Related closed #3991 covers duplicate byte-string keys at the root. Its comments also describe duplicate container-valued root keys. This report concerns scalar byte-string keys inside a nested map value. The current root duplicate-byte control passes the intended rejection, while the nested case does not. The valid nested control establishes that this is validity checking, not a request to expand the documented nested presentation profile.

Restoration checks are proposed, not executed after drafting: preserve the distinct nested-byte control as partial; make the duplicate nested-byte case match the nested-text/root-byte rejection controls; preserve selected bytecode, ranges and interface candidates. No reproducer was rerun after this draft was written. Prepared with AI assistance.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: The public CLI gives the same partial interpretation status and normal limitation for a valid nested map and an invalid duplicate-byte-key map, omitting its promised invalidity diagnostic while retaining interpreted root metadata; original bytes remain intact.
Workaround status: available
Workaround basis: For the demonstrated 19-byte metadata payload, the retained original range exposes both 4101 keys and their values; direct byte inspection identifies the invalid map without changing evidence. A general automatic REA workaround and real-artifact frequency were not established.

Originally created by @coygeek on GitHub (Oct 23, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6780 ## Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ## Area Evidence, artifacts, or reconstruction ## Observed behavior The public `inspect-evm-interface` CLI returns `bytecode.metadata.status: "partial"` for a terminal CBOR map containing duplicate byte-string keys inside a nested map. It reports only the normal nested-container limitation and retains the `solc` declaration. Equivalent duplicate text keys in that nested map correctly return `not-decoded`, clear interpreted entries/compiler declaration, and identify the duplicate. The current root-level duplicate-byte-key control also correctly returns `not-decoded`. This is a remaining nested-value case, rather than a reproduction of the resolved root-level behavior in #3991. Each of the four CLI requests below ran once, exited 0, and emitted empty stderr. ## Reproduction Use the built source checkout at the revision below with its locked dependencies and pinned Node toolchain. Set `REA_ROOT` to that checkout. Save the complete script below as `/tmp/nested-duplicate-cli.mjs`, then run: ```sh node /tmp/nested-duplicate-cli.mjs "$REA_ROOT" ``` ```javascript import {mkdtempSync, mkdirSync, writeFileSync} from 'node:fs'; import {tmpdir} from 'node:os'; import {join, resolve} from 'node:path'; import {spawnSync} from 'node:child_process'; const repo=resolve(process.argv[2]); const output=resolve(process.argv[3] ?? mkdtempSync(join(tmpdir(),'rea-cbor-'))); const home=join(output,'home');mkdirSync(home,{recursive:true}); const env={...process.env,HOME:home,XDG_CONFIG_HOME:join(home,'config'),XDG_DATA_HOME:join(home,'data'),XDG_CACHE_HOME:join(home,'cache'),XDG_STATE_HOME:join(home,'state'),NO_COLOR:'1'}; const code=Buffer.from('60003560e01c630000000014601057005b00','hex'); const fixtures={ distinctNestedBytes:'a264736f6c634300081e6178a2410101410202', duplicateNestedBytes:'a264736f6c634300081e6178a2410101410102', duplicateNestedText:'a264736f6c634300081e6178a2616101616102', duplicateRootBytes:'a364736f6c634300081e410101410102', }; for(const [name,hex] of Object.entries(fixtures)){ const payload=Buffer.from(hex,'hex'),length=Buffer.alloc(2);length.writeUInt16BE(payload.length); const selected=Buffer.concat([code,payload,length]); const path=join(output,name+'.hex');writeFileSync(path,selected.toString('hex')); const run=spawnSync(process.execPath,[join(repo,'scripts/rea.mjs'),'inspect-evm-interface',path,'hex','--json'],{cwd:output,env,encoding:'utf8',timeout:60000,maxBuffer:4*1024*1024}); writeFileSync(join(output,name+'.stdout'),run.stdout??'');writeFileSync(join(output,name+'.stderr'),run.stderr??''); if(run.status!==0)throw new Error(`${name}: exit=${run.status}: ${run.stderr}`); const document=JSON.parse(run.stdout),report=(document.data??document).normalized_result; console.log(JSON.stringify({name,exit:run.status,stderr:run.stderr,bytecode_bytes:report.bytecode.bytes,functions:report.functions,metadata:report.bytecode.metadata})); } ``` The nested-byte payload represents `{"solc": h'00081e', "x": {h'01': 1, h'01': 2}}`. The distinct-byte control changes the second byte-string key to `02`. The other controls use repeated text keys inside `x`, and repeated byte-string keys at the root. The terminal lengths are computed from each payload. All artifacts are synthetic flat EVM bytes containing a small dispatcher; no contract execution is involved. ## Expected behavior Return `not-decoded` with a duplicate-key diagnostic for the invalid nested byte-key map, preserving its candidate offset/length and original bytecode while leaving interpreted entries empty and `compiler_version` null. Keep the valid distinct-key nested map as `partial`. [The metadata contract](https://github.com/morluto/rea/blob/98ca6c82705bf8c44f814620636852f938d827ca/docs/evm-bytecode.md#L44-L50) distinguishes partial scalar coverage for nested containers from unsupported/malformed candidates, which must return `not-decoded` with a diagnostic. [RFC 8949 §3.1](https://www.rfc-editor.org/rfc/rfc8949.html#section-3.1) identifies duplicate-key maps as invalid; [§5.6.1](https://www.rfc-editor.org/rfc/rfc8949.html#section-5.6.1) compares byte-string keys by their bytes. This does not require exposing nested values in REA's scalar view: the demonstrated nested-text control already enforces validity separately from presentation coverage. ## Impact and scope The machine-readable metadata status and diagnostic do not distinguish an invalid nested byte-key map from the valid distinct-key map. A consumer relying on the interpretation status receives no duplicate-key warning, and the invalid trailer still contributes a decoded compiler-version declaration. That declaration remains only a metadata assertion, not verified compiler identity. Observed scope is the small generated nested-map fixtures through the real public CLI, worker, bundled EVMole/WASM and independent metadata parser. Full original bytecode and the correct metadata range remain available; no evidence loss, wrong compiler authenticity claim, target execution or security effect is asserted. Prevalence in real artifacts and other nesting shapes are unknown. For this bounded fixture, callers can inspect the retained metadata range and recognize `a2 4101 01 4101 02` as two entries with the same byte key. No automatic REA workaround was established. ## REA version `rea --version`: `6.7.0`. Source revision `98ca6c82705bf8c44f814620636852f938d827ca`, using that checkout's compiled runtime. The audit coordinator established `rea-agents-6.7.0` as the latest published release; the npm release artifact was not separately executed. ## Runtime and platform Node.js `v24.22.1`; Linux `7.2.8-9-omarchy`, x86_64; repository-locked dependencies including bundled EVMole `0.9.3`. No blockchain service, wallet, target runtime, Solidity compiler, Hopper or Ghidra was used. ## Target and artifact context Generated ASCII hex carriers for 39-byte flat EVM bytecode in the three nested-map cases, and 36 bytes in the root-duplicate control. Each has an 18-byte dispatcher, a CBOR map, and its two-byte big-endian length suffix. The metadata is synthetic; `0.8.30` is an intentionally supplied declaration. ## Interface CLI ## Earliest failure stage Native analysis or reconstruction ## Relevant output or evidence Observed selected fields from the four public CLI responses: ```text name exit metadata.status entries compiler_version duplicate diagnostic distinctNestedBytes 0 partial 2 0.8.30 no duplicateNestedBytes 0 partial 2 0.8.30 no duplicateNestedText 0 not-decoded 0 null yes duplicateRootBytes 0 not-decoded 0 null yes ``` Both nested-byte cases return the same interpreted metadata object, including: ```json { "status": "partial", "bytecode_offset": 18, "cbor_bytes": 19, "compiler_version": {"representation":"release-bytes","value":"0.8.30"}, "diagnostic": "Nested containers are uninterpreted; their exact bytes remain at the recorded range in original bytecode." } ``` The nested-text control returns `CBOR decode error: found repeat map key "a"`. The root-byte control returns a repeat-map-key diagnostic for `{"kind":"bytes","value":"01"}`. All four retain the inferred selector `0x00000000` at decoded offset 16 and have empty stderr. - Expected source: the pinned documentation and RFC sections linked above. - Failure source: executed public CLI output from the complete script above. - Evidence provenance: observed CLI results; the causal explanation below is source-derived. - Local verification: reproduced. - Reproduction completeness: complete for the four reduced offline CLI fixtures. Real deployed or compiler-generated artifacts, other platforms and the published npm artifact were not tested. ## Typed outcome or recovery expected Keep successful interface inspection and original byte identities, but represent the invalid metadata interpretation as `not-decoded` with its exact candidate range, a duplicate-key diagnostic, no interpreted entries and no compiler-version declaration. Preserve `partial` for valid nested-container controls. ## Analysis or suspected cause Source-derived: [the parser enables duplicate-key rejection](https://github.com/morluto/rea/blob/98ca6c82705bf8c44f814620636852f938d827ca/src/evm/SolidityMetadata.ts#L10-L19), but cborg compares decoded byte-string objects by identity. [REA's additional value-based check](https://github.com/morluto/rea/blob/98ca6c82705bf8c44f814620636852f938d827ca/src/evm/SolidityMetadata.ts#L119-L145) iterates only the root `Map`. A nested `Map` is projected as an uninterpreted container without that check. [Compiler-version projection](https://github.com/morluto/rea/blob/98ca6c82705bf8c44f814620636852f938d827ca/src/evm/SolidityMetadata.ts#L146-L160) then retains the root `solc` value. [The actual worker](https://github.com/morluto/rea/blob/98ca6c82705bf8c44f814620636852f938d827ca/src/evm/EvmoleInterfaceWorker.ts#L61-L69) puts this independent interpretation into the successful bytecode result. ## Additional context Searched the supplied open-issue snapshot plus GitHub open and closed issue searches for `CBOR`, `EVM metadata`, `"nested" "CBOR"` and `"metadata" "duplicate"`. No existing nested-value byte-key report was found. Related closed [#3991](https://github.com/morluto/rea/issues/3991) covers duplicate byte-string keys at the root. Its comments also describe duplicate container-valued root keys. This report concerns scalar byte-string keys inside a nested map value. The current root duplicate-byte control passes the intended rejection, while the nested case does not. The valid nested control establishes that this is validity checking, not a request to expand the documented nested presentation profile. Restoration checks are proposed, not executed after drafting: preserve the distinct nested-byte control as `partial`; make the duplicate nested-byte case match the nested-text/root-byte rejection controls; preserve selected bytecode, ranges and interface candidates. No reproducer was rerun after this draft was written. Prepared with AI assistance. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: The public CLI gives the same partial interpretation status and normal limitation for a valid nested map and an invalid duplicate-byte-key map, omitting its promised invalidity diagnostic while retaining interpreted root metadata; original bytes remain intact. - Workaround status: available - Workaround basis: For the demonstrated 19-byte metadata payload, the retained original range exposes both `4101` keys and their values; direct byte inspection identifies the invalid map without changing evidence. A general automatic REA workaround and real-artifact frequency were not established.

bsakaguchi closed this issue

2026-10-31 00:47:03 +00:00

bsakaguchi referenced this issue

2026-10-31 01:11:36 +00:00

[PR #4117] [MERGED] fix(verifiers): align WABT operation timeout #6067

bsakaguchi referenced this issue

2026-10-31 01:11:48 +00:00

[PR #4153] [MERGED] fix(verifiers): launch npm through Node on Windows #6092

bsakaguchi referenced this issue

2026-10-31 01:14:05 +00:00

[PR #4749] [MERGED] fix(verifiers): read packed bridge without extraction #6386