morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #3991] [Bug] Duplicate byte-string CBOR metadata keys receive decoded status #1585

New issue

Closed

opened 2026-10-16 21:07:29 +00:00 by bsakaguchi · 6 comments

bsakaguchi commented

2026-10-16 21:07:29 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 15, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3991

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

inspect_evm_interface accepts a terminal CBOR map with the same byte-string key twice and reports its independent metadata interpretation as decoded, with no diagnostic. Duplicate text keys correctly return not-decoded. This gives callers an incorrect interpretation status for a bounded class of malformed metadata, although the original bytes and both entries remain available.

Observed in the real worker and simulated-provider MCP delivery:

Fixture	Metadata status	Diagnostic	Returned keys
Distinct byte keys	`decoded`	null	bytes `01`, bytes `02`
Repeated text keys	`not-decoded`	`CBOR decode error: found repeat map key "x"`	none
Repeated byte keys	`decoded`	null	bytes `01`, bytes `01`

The repeated-byte metadata was:

{
  "profile": "terminal-cbor-scalars-v1",
  "status": "decoded",
  "bytecode_offset": 1,
  "cbor_bytes": 7,
  "entries": [
    { "key": { "kind": "bytes", "value": "01" }, "value": { "kind": "integer", "value": "1" } },
    { "key": { "kind": "bytes", "value": "01" }, "value": { "kind": "integer", "value": "2" } }
  ],
  "compiler_version": null,
  "diagnostic": null
}

Each worker returned ok: true; each MCP call returned successful Evidence with no inferred functions. The independent test passed its two control assertions and failed on the repeated-byte status assertion with expected 'decoded' to be 'not-decoded'. The existing 19 metadata tests passed in the same focused run. This establishes one independent attempt per fixture and does not measure prevalence in real contract artifacts.

Reproduction

Start from the prepared baseline checkout with its locked dependencies and existing compiled runtime. Use the pinned Node toolchain. Run the following worker reproduction from the repository root. Each input is an ASCII hex carrier for ten EVM bytes. The fixture with repeated byte keys contains a leading STOP, a seven-byte CBOR map and the matching terminal length. The other fixtures control for distinct byte keys and repeated text keys.

node --input-type=module <<'JS'
import { execFileSync } from 'node:child_process';
import { mkdtemp, readFile, rm, writeFile } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join, resolve } from 'node:path';

const directory = await mkdtemp(join(tmpdir(), 'cbor-key-check-'));
try {
  const fixtures = [
    ['distinct byte keys', '00a24101014102020007'],
    ['repeated text keys', '00a26178016178020007'],
    ['repeated byte keys', '00a24101014101020007'],
  ];
  for (const [index, [name, hex]] of fixtures.entries()) {
    const carrier = join(directory, `${index}.hex`);
    const request = join(directory, `${index}.request.json`);
    const reply = join(directory, `${index}.reply.json`);
    await writeFile(carrier, hex);
    await writeFile(request, JSON.stringify({
      snapshot_path: carrier,
      reply_path: reply,
      failure_marker_path: join(directory, `${index}.failure`),
      encoding: 'hex',
    }));
    execFileSync(process.execPath, [
      '--disable-wasm-trap-handler',
      resolve('dist/evm/EvmoleInterfaceWorker.js'),
      request,
    ], { timeout: 30000, env: { ...process.env, HOME: directory } });
    const result = JSON.parse(await readFile(reply, 'utf8'));
    console.log(JSON.stringify({ name, ok: result.ok, metadata: result.bytecode?.metadata }));
  }
} finally {
  await rm(directory, { recursive: true, force: true });
}
JS

Compare the metadata status, diagnostic and entries for the three fixtures. The repeated-byte case returns the incorrect successful status shown below. The worker calls in this command were independently executed through a focused test, once per fixture.
The same focused test bound each actual worker result to a simulated provider execution with the real carrier hash, byte count, encoding and decoded bytecode. It then called the registered MCP tool with { "name": "inspect_evm_interface", "arguments": { "path": "<absolute carrier path>", "encoding": "hex" } }. All three tool calls succeeded and retained the same metadata in structured Evidence. The simulated execution replaced provider admission, launching and result assembly. The parser, worker, shared service and registered MCP handler were real.
For a direct public trigger on a supported Linux x64 host with usable util-linux prlimit, place 00a24101014101020007 in selected.hex and run node scripts/rea.mjs inspect-evm-interface selected.hex hex --format json, using a disposable HOME. This CLI trigger is source-derived and was not executed locally.

Expected behavior

The independent terminal metadata profile must reject repeated map keys for admitted scalar key types. The repeated-byte fixture should report not-decoded with a diagnostic, preserve offset 1 and CBOR length 7, and return no decoded entries or compiler version. The distinct-byte fixture should continue to report decoded and retain both typed keys and values. The repeated-text fixture should remain rejected.

Impact and scope

A selected malformed root map receives an incorrect successful metadata status through the public result contract. This can mislead a consumer relying on the diagnostic/status fields. The observed result retains the original bytecode and both entries; there was no observed target execution, evidence loss or security boundary crossing. Other map shapes and real artifact prevalence were not tested.

Workaround status: available. For the demonstrated scalar root map, the actual response retains each typed key and value. Callers can compare key kind/value pairs, identify the two byte keys 01 and treat the interpretation as invalid while preserving the original evidence. This is a practical manual check for the observed case.

REA version

rea 6.5.0 at source commit 4692bb3330

Runtime and platform

Node 24.18.0; macOS arm64; locked cborg 6.1.3 and bundled EVMole 0.9.3

Target and artifact context

Synthetic ten-byte EVM carriers with STOP and a seven-byte terminal CBOR map. Real compiled worker and WASM engine; MCP provider admission and launching were simulated.

Interface

MCP

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

Expected source: docs/evm-bytecode.md:42-48 promises typed scalar keys and not-decoded with a diagnostic for malformed candidates; src/evm/SolidityMetadata.ts:10-18 explicitly enables duplicate-map-key rejection; src/evm/SolidityMetadata.test.ts:111-129 exercises that rejection for repeated text keys.
Failure source: The independent focused MCP test using the actual compiled worker produced the table and complete repeated-byte result above. src/evm/SolidityMetadata.ts:119-136 projects the decoded map into typed entries and reports decoded; src/evm/EvmoleInterfaceProvider.ts:416-445 carries the worker bytecode into the public result; the simulated-provider MCP check exercised the service and registered handler with that actual result.
Source-derived explanation: node_modules/cborg/lib/decode.js:94-127 checks duplicate keys with JavaScript map membership. Separately decoded byte arrays have different object identities, even when their bytes are equal. Both survive and later become the same typed key in REA's result.
Source-derived reachability: src/composition/evm.ts:7-11, src/cli/evmCommands.ts:32-35 and src/server/registerEvmTools.ts:47-50 bind the operation to the shared inspection service. src/evm/EvmoleInterfaceProvider.ts:204-266 checks host prerequisites and reads the selected carrier; the 20-byte regular hex carrier is within the input budget. src/evm/EvmoleInterfaceWorker.ts:31-90 accepts this encoding and leading STOP, runs the engine and interprets the metadata. src/evm/EvmoleInterfaceProvider.ts:390-445 checks decoded identity and validates the result. src/domain/evm/evmInterface.ts:61-91 checks byte/range consistency without rejecting repeated metadata keys. The remaining host, resource, cancellation and cleanup conditions do not depend on map-key uniqueness.

Typed outcome or recovery expected

Report repeated byte keys as not-decoded with a diagnostic and no decoded entries, while preserving original bytecode and offset/length. Distinct byte keys should remain decoded.

Analysis or suspected cause

Source-derived: locked cborg 6.1.3 node_modules/cborg/lib/decode.js:94-127 tests duplicate keys by JavaScript Map membership, so separate Uint8Array objects compare by identity. src/evm/SolidityMetadata.ts:119-136 projects the two equal byte keys and marks the result decoded.

Additional context

Freshness check: the cited failing implementation is unchanged on origin/main at fc712b3750. The only change to a cited reproduction entry point normalizes global CLI flag syntax. The baseline reproduction was not rerun on that revision.

Restoration check: Run the same three fixtures through the real worker and registered MCP operation. The repeated-byte and repeated-text cases must report not-decoded, a diagnostic, their original offset/length and no decoded entries or compiler version. The distinct-byte control must remain decoded with byte keys 01 and 02 and integer values 1 and 2. Preserve the full bytecode, carrier identity and unrelated interface result. Rejecting all byte-string keys or losing the selected bytes would not restore this contract.

Originally created by @coygeek on GitHub (Oct 15, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3991 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior `inspect_evm_interface` accepts a terminal CBOR map with the same byte-string key twice and reports its independent metadata interpretation as `decoded`, with no diagnostic. Duplicate text keys correctly return `not-decoded`. This gives callers an incorrect interpretation status for a bounded class of malformed metadata, although the original bytes and both entries remain available. Observed in the real worker and simulated-provider MCP delivery: | Fixture | Metadata status | Diagnostic | Returned keys | | --- | --- | --- | --- | | Distinct byte keys | `decoded` | null | bytes `01`, bytes `02` | | Repeated text keys | `not-decoded` | `CBOR decode error: found repeat map key "x"` | none | | Repeated byte keys | `decoded` | null | bytes `01`, bytes `01` | The repeated-byte metadata was: ```json { "profile": "terminal-cbor-scalars-v1", "status": "decoded", "bytecode_offset": 1, "cbor_bytes": 7, "entries": [ { "key": { "kind": "bytes", "value": "01" }, "value": { "kind": "integer", "value": "1" } }, { "key": { "kind": "bytes", "value": "01" }, "value": { "kind": "integer", "value": "2" } } ], "compiler_version": null, "diagnostic": null } ``` Each worker returned `ok: true`; each MCP call returned successful Evidence with no inferred functions. The independent test passed its two control assertions and failed on the repeated-byte status assertion with `expected 'decoded' to be 'not-decoded'`. The existing 19 metadata tests passed in the same focused run. This establishes one independent attempt per fixture and does not measure prevalence in real contract artifacts. ### Reproduction 1. Start from the prepared baseline checkout with its locked dependencies and existing compiled runtime. Use the pinned Node toolchain. Run the following worker reproduction from the repository root. Each input is an ASCII hex carrier for ten EVM bytes. The fixture with repeated byte keys contains a leading STOP, a seven-byte CBOR map and the matching terminal length. The other fixtures control for distinct byte keys and repeated text keys. ```sh node --input-type=module <<'JS' import { execFileSync } from 'node:child_process'; import { mkdtemp, readFile, rm, writeFile } from 'node:fs/promises'; import { tmpdir } from 'node:os'; import { join, resolve } from 'node:path'; const directory = await mkdtemp(join(tmpdir(), 'cbor-key-check-')); try { const fixtures = [ ['distinct byte keys', '00a24101014102020007'], ['repeated text keys', '00a26178016178020007'], ['repeated byte keys', '00a24101014101020007'], ]; for (const [index, [name, hex]] of fixtures.entries()) { const carrier = join(directory, `${index}.hex`); const request = join(directory, `${index}.request.json`); const reply = join(directory, `${index}.reply.json`); await writeFile(carrier, hex); await writeFile(request, JSON.stringify({ snapshot_path: carrier, reply_path: reply, failure_marker_path: join(directory, `${index}.failure`), encoding: 'hex', })); execFileSync(process.execPath, [ '--disable-wasm-trap-handler', resolve('dist/evm/EvmoleInterfaceWorker.js'), request, ], { timeout: 30000, env: { ...process.env, HOME: directory } }); const result = JSON.parse(await readFile(reply, 'utf8')); console.log(JSON.stringify({ name, ok: result.ok, metadata: result.bytecode?.metadata })); } } finally { await rm(directory, { recursive: true, force: true }); } JS ``` 2. Compare the metadata status, diagnostic and entries for the three fixtures. The repeated-byte case returns the incorrect successful status shown below. The worker calls in this command were independently executed through a focused test, once per fixture. 3. The same focused test bound each actual worker result to a simulated provider execution with the real carrier hash, byte count, encoding and decoded bytecode. It then called the registered MCP tool with `{ "name": "inspect_evm_interface", "arguments": { "path": "<absolute carrier path>", "encoding": "hex" } }`. All three tool calls succeeded and retained the same metadata in structured Evidence. The simulated execution replaced provider admission, launching and result assembly. The parser, worker, shared service and registered MCP handler were real. 4. For a direct public trigger on a supported Linux x64 host with usable util-linux `prlimit`, place `00a24101014101020007` in `selected.hex` and run `node scripts/rea.mjs inspect-evm-interface selected.hex hex --format json`, using a disposable HOME. This CLI trigger is source-derived and was not executed locally. ### Expected behavior The independent terminal metadata profile must reject repeated map keys for admitted scalar key types. The repeated-byte fixture should report `not-decoded` with a diagnostic, preserve offset 1 and CBOR length 7, and return no decoded entries or compiler version. The distinct-byte fixture should continue to report `decoded` and retain both typed keys and values. The repeated-text fixture should remain rejected. ### Impact and scope A selected malformed root map receives an incorrect successful metadata status through the public result contract. This can mislead a consumer relying on the diagnostic/status fields. The observed result retains the original bytecode and both entries; there was no observed target execution, evidence loss or security boundary crossing. Other map shapes and real artifact prevalence were not tested. Workaround status: available. For the demonstrated scalar root map, the actual response retains each typed key and value. Callers can compare key kind/value pairs, identify the two byte keys `01` and treat the interpretation as invalid while preserving the original evidence. This is a practical manual check for the observed case. ### REA version rea 6.5.0 at source commit 4692bb33300c7cb77b095b6cc0c33da861c1b860 ### Runtime and platform Node 24.18.0; macOS arm64; locked cborg 6.1.3 and bundled EVMole 0.9.3 ### Target and artifact context Synthetic ten-byte EVM carriers with STOP and a seven-byte terminal CBOR map. Real compiled worker and WASM engine; MCP provider admission and launching were simulated. ### Interface MCP ### Earliest failure stage Native analysis or reconstruction ### Relevant output or evidence - Expected source: `docs/evm-bytecode.md:42-48` promises typed scalar keys and `not-decoded` with a diagnostic for malformed candidates; `src/evm/SolidityMetadata.ts:10-18` explicitly enables duplicate-map-key rejection; `src/evm/SolidityMetadata.test.ts:111-129` exercises that rejection for repeated text keys. - Failure source: The independent focused MCP test using the actual compiled worker produced the table and complete repeated-byte result above. `src/evm/SolidityMetadata.ts:119-136` projects the decoded map into typed entries and reports `decoded`; `src/evm/EvmoleInterfaceProvider.ts:416-445` carries the worker bytecode into the public result; the simulated-provider MCP check exercised the service and registered handler with that actual result. - Source-derived explanation: `node_modules/cborg/lib/decode.js:94-127` checks duplicate keys with JavaScript map membership. Separately decoded byte arrays have different object identities, even when their bytes are equal. Both survive and later become the same typed key in REA's result. - Source-derived reachability: `src/composition/evm.ts:7-11`, `src/cli/evmCommands.ts:32-35` and `src/server/registerEvmTools.ts:47-50` bind the operation to the shared inspection service. `src/evm/EvmoleInterfaceProvider.ts:204-266` checks host prerequisites and reads the selected carrier; the 20-byte regular hex carrier is within the input budget. `src/evm/EvmoleInterfaceWorker.ts:31-90` accepts this encoding and leading STOP, runs the engine and interprets the metadata. `src/evm/EvmoleInterfaceProvider.ts:390-445` checks decoded identity and validates the result. `src/domain/evm/evmInterface.ts:61-91` checks byte/range consistency without rejecting repeated metadata keys. The remaining host, resource, cancellation and cleanup conditions do not depend on map-key uniqueness. ### Typed outcome or recovery expected Report repeated byte keys as not-decoded with a diagnostic and no decoded entries, while preserving original bytecode and offset/length. Distinct byte keys should remain decoded. ### Analysis or suspected cause Source-derived: locked cborg 6.1.3 node_modules/cborg/lib/decode.js:94-127 tests duplicate keys by JavaScript Map membership, so separate Uint8Array objects compare by identity. src/evm/SolidityMetadata.ts:119-136 projects the two equal byte keys and marks the result decoded. ### Additional context Freshness check: the cited failing implementation is unchanged on origin/main at fc712b3750cbe16755c4f78c1f90fdf09fbaca31. The only change to a cited reproduction entry point normalizes global CLI flag syntax. The baseline reproduction was not rerun on that revision. Restoration check: Run the same three fixtures through the real worker and registered MCP operation. The repeated-byte and repeated-text cases must report `not-decoded`, a diagnostic, their original offset/length and no decoded entries or compiler version. The distinct-byte control must remain `decoded` with byte keys `01` and `02` and integer values `1` and `2`. Preserve the full bytecode, carrier identity and unrelated interface result. Rejecting all byte-string keys or losing the selected bytes would not restore this contract.

bsakaguchi commented

2026-10-16 21:07:30 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 15, 2026):

Confirmed: a CBOR map with the same byte-string key twice is malformed just like the text-key case, so reporting it as decoded with no diagnostic is inconsistent. The duplicate-key check should normalize byte keys by their value before accepting the map, matching the not-decoded + diagnostic behavior already applied to repeated text keys.

@XinwangTao commented on GitHub (Oct 15, 2026): Confirmed: a CBOR map with the same byte-string key twice is malformed just like the text-key case, so reporting it as decoded with no diagnostic is inconsistent. The duplicate-key check should normalize byte keys by their value before accepting the map, matching the not-decoded + diagnostic behavior already applied to repeated text keys.

bsakaguchi commented

2026-10-16 21:07:30 +00:00

Author

Owner

@Sriram-PR commented on GitHub (Oct 15, 2026):

I'll take this one. Reproduced with the compiled worker on current main: repeated byte keys come back decoded with no diagnostic, repeated text keys are rejected. cborg's duplicate check compares Map keys by identity, and each byte key decodes to its own Uint8Array, so it never matches. Plan: after decoding, reject the root map when two scalar keys have the same kind and value, so it goes through the existing not-decoded path, plus a test next to the repeated text key one in SolidityMetadata.test.ts. PR shortly.

@Sriram-PR commented on GitHub (Oct 15, 2026): I'll take this one. Reproduced with the compiled worker on current main: repeated byte keys come back `decoded` with no diagnostic, repeated text keys are rejected. cborg's duplicate check compares Map keys by identity, and each byte key decodes to its own `Uint8Array`, so it never matches. Plan: after decoding, reject the root map when two scalar keys have the same kind and value, so it goes through the existing `not-decoded` path, plus a test next to the repeated text key one in `SolidityMetadata.test.ts`. PR shortly.

bsakaguchi commented

2026-10-16 21:07:30 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 15, 2026):

@Sriram-PR The post-decode kind+value comparison approach looks sound. Two test cases worth covering: (1) the two duplicate byte keys carrying different integer values (bytes 01 -> 1, bytes 01 -> 2), since both survive identity-based checks and then collapse into the same typed key during projection; (2) the distinct-byte control (01 -> 1, 02 -> 2) must keep reporting decoded so the fix does not over-reject. That mirrors the table in the report exactly.

@XinwangTao commented on GitHub (Oct 15, 2026): @Sriram-PR The post-decode kind+value comparison approach looks sound. Two test cases worth covering: (1) the two duplicate byte keys carrying different integer values (bytes 01 -> 1, bytes 01 -> 2), since both survive identity-based checks and then collapse into the same typed key during projection; (2) the distinct-byte control (01 -> 1, 02 -> 2) must keep reporting decoded so the fix does not over-reject. That mirrors the table in the report exactly.

bsakaguchi commented

2026-10-16 21:07:31 +00:00

Author

Owner

@Sriram-PR commented on GitHub (Oct 15, 2026):

Both are in #4024: the duplicate case is {h'01': 1, h'01': 2} (a2410101410102), and the distinct control {h'01': 1, h'02': 2} (a2410101410202) still comes back decoded with both keys.

@Sriram-PR commented on GitHub (Oct 15, 2026): Both are in #4024: the duplicate case is `{h'01': 1, h'01': 2}` (`a2410101410102`), and the distinct control `{h'01': 1, h'02': 2}` (`a2410101410202`) still comes back `decoded` with both keys.

bsakaguchi referenced this issue

2026-10-17 21:01:31 +00:00

[GH-ISSUE #2383] [Tracker] Audit analysis and debugging coverage for the TIOBE top 100 languages (October 2026) #794

bsakaguchi referenced this issue

2026-10-17 21:10:47 +00:00

[PR #1369] [CLOSED] fix(javascript): preserve exact static CommonJS export keys #2832

bsakaguchi closed this issue

2026-10-31 00:31:37 +00:00

bsakaguchi commented

2026-10-31 00:31:38 +00:00

Author

Owner

@coygeek commented on GitHub (Oct 21, 2026):

Independently reproduced the same identity-based duplicate-key behavior for root-map keys that are containers on Linux x86_64 with REA 6.7.0: CBOR a280018002 has two identical empty-array keys, and a2a001a002 has two identical empty-map keys. Both return metadata status: "partial" with two entries and only the nested-container diagnostic. The duplicate byte-string control a2410101410102 returns decoded; duplicate text a2617801617802 and integer a201010102 keys return not-decoded. To reproduce, write 608060405200 + either container-key CBOR map + 0005 as ASCII hex to selected.hex, then run node scripts/rea.mjs inspect-evm-interface selected.hex hex --json. PR #4024 at 8787a0251fc398e829ba3cbb61cb2b63ad3125fa leaves these cases uncovered in its diff: the added comparison skips entry.key.kind === "uninterpreted", which includes both array and map keys. This coverage finding is from source inspection; I did not execute the PR build.

@coygeek commented on GitHub (Oct 21, 2026): Independently reproduced the same identity-based duplicate-key behavior for root-map keys that are containers on Linux x86_64 with REA 6.7.0: CBOR `a280018002` has two identical empty-array keys, and `a2a001a002` has two identical empty-map keys. Both return metadata `status: "partial"` with two entries and only the nested-container diagnostic. The duplicate byte-string control `a2410101410102` returns `decoded`; duplicate text `a2617801617802` and integer `a201010102` keys return `not-decoded`. To reproduce, write `608060405200` + either container-key CBOR map + `0005` as ASCII hex to `selected.hex`, then run `node scripts/rea.mjs inspect-evm-interface selected.hex hex --json`. PR #4024 at `8787a0251fc398e829ba3cbb61cb2b63ad3125fa` leaves these cases uncovered in its diff: the added comparison skips `entry.key.kind === "uninterpreted"`, which includes both array and map keys. This coverage finding is from source inspection; I did not execute the PR build.

bsakaguchi commented

2026-10-31 00:31:38 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 21, 2026):

Agreed on #4024's direction, but the diff currently skips entry.key.kind === "uninterpreted", which also covers array/map keys — so duplicate container keys would still slip through as decoded. For parity with the text-key rejection path, the fix should either reject duplicates for those container keys too or explicitly document why they are exempt. Suggest adding the repeated-byte fixture (a2410101410102) as a regression test asserting not-decoded + diagnostic, so this doesn't silently regress later.

@XinwangTao commented on GitHub (Oct 21, 2026): Agreed on #4024's direction, but the diff currently skips `entry.key.kind === "uninterpreted"`, which also covers array/map keys — so duplicate container keys would still slip through as `decoded`. For parity with the text-key rejection path, the fix should either reject duplicates for those container keys too or explicitly document why they are exempt. Suggest adding the repeated-byte fixture (a2410101410102) as a regression test asserting `not-decoded` + diagnostic, so this doesn't silently regress later.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#1585

No description provided.

Rows
Columns