morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #3203] [Bug] Shortest-form empty CBOR text discards decoded terminal metadata #1139

New issue

Closed

opened 2026-10-15 18:31:25 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-15 18:31:25 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 14, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3203

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

A valid terminal CBOR root map containing an empty text key or value encoded as the shortest-form token 0x60 produces bytecode.metadata.status: "not-decoded", no entries and a null compiler version. The diagnostic is CBOR text bytes were unavailable for strict UTF-8 validation. An independently valid solc release-byte declaration in the same map also disappears from the interpreted result.

The actual compiled production worker reproduced this behavior on macOS. All six worker cases exited 0 and returned ok: true without stdout or stderr. The equivalent empty value encoded with the longer 0x78 0x00 length form decoded successfully. The defect concerns shortest-form empty text, rather than every encoding of an empty string.

The public caller command is inspect-evm-interface, with the equivalent MCP tool inspect_evm_interface. Successful public CLI/MCP execution on an admitted Linux x64 host was not run. Source tracing connects both public interfaces to the observed worker. The attempted macOS CLI call correctly exited 1 with capability_unavailable before analysis because the public provider requires Linux x64.

Reproduction

Start with a clean checkout at 58300885207133422af235e0f2b10658cf958b03, use its declared Node toolchain, and install/build from the repository root:

npm ci
npm run build:cached

These are clean-checkout prerequisites. The independent verifier used the prepared build and did not rerun installation or build during verification.

Run this self-contained reproduction from the repository root. It creates synthetic hex carriers and request files, isolates the worker's home and XDG directories, and executes the compiled production worker used by the provider:

node --input-type=module <<'JS'
import { mkdir, writeFile, readFile } from 'node:fs/promises';
import { resolve, join } from 'node:path';
import { execFileSync } from 'node:child_process';
const root = resolve('repro-cbor');
const home = join(root, 'home');
const env = {
  ...process.env,
  HOME: home,
  XDG_CONFIG_HOME: join(home, 'config'),
  XDG_CACHE_HOME: join(home, 'cache'),
  XDG_DATA_HOME: join(home, 'data'),
  TMPDIR: root,
};
for (const key of ['HOME', 'XDG_CONFIG_HOME', 'XDG_CACHE_HOME', 'XDG_DATA_HOME']) {
  await mkdir(env[key], { recursive: true });
}
const cases = [
  ['empty-key', 'a16001'],
  ['empty-value', 'a1616160'],
  ['control', 'a1616101'],
  ['invalid-utf8', 'a1617861ff'],
  ['long-form-empty', 'a161617800'],
  ['empty-and-solc', 'a261616064736f6c634300081e'],
];
for (const [name, payload] of cases) {
  const dir = join(root, name);
  await mkdir(dir, { recursive: true });
  const carrier = join(dir, 'carrier.hex');
  const replyPath = join(dir, 'reply.json');
  const requestPath = join(dir, 'request.json');
  const length = (payload.length / 2).toString(16).padStart(4, '0');
  await writeFile(carrier, '60006000f3' + payload + length);
  await writeFile(requestPath, JSON.stringify({
    snapshot_path: carrier,
    reply_path: replyPath,
    failure_marker_path: join(dir, 'resource.failure'),
    encoding: 'hex',
  }));
  execFileSync(process.execPath, [
    '--max-old-space-size=256',
    '--disable-wasm-trap-handler',
    resolve('dist/evm/EvmoleInterfaceWorker.js'),
    requestPath,
  ], { env, timeout: 30000 });
  const reply = JSON.parse(await readFile(replyPath, 'utf8'));
  console.log(JSON.stringify({ case: name, ok: reply.ok, metadata: reply.bytecode.metadata }));
}
JS

Compare the metadata printed for the six cases with the evidence below. The empty key, empty value and empty value plus solc cases report not-decoded; the non-empty and longer-form empty controls report decoded. Invalid UTF-8 correctly remains not-decoded.
To check the public caller boundary on an admitted Linux x64 host with util-linux prlimit, readable Linux process limits and sufficient inherited resource limits, run:

node scripts/rea.mjs inspect-evm-interface repro-cbor/empty-key/carrier.hex hex --json
node scripts/rea.mjs inspect-evm-interface repro-cbor/empty-value/carrier.hex hex --json

Step 2 was independently executed. Successful Linux CLI calls in step 4 were not executed during verification; their reachability is source-derived. The corresponding MCP request uses inspect_evm_interface with an absolute local carrier path and encoding: "hex".

Expected behavior

docs/specs/solidity/cbor-metadata/spec.md:18-26 requires lossless scalar root-map key/value interpretation and retention through public CLI/MCP. docs/evm-bytecode.md:42-48 documents the independently interpreted terminal-CBOR facet, and src/domain/evm/solidityMetadata.ts:5 admits empty text. A plausible terminal root map does not require proof of Solidity origin.

The empty-key carrier should report status: "decoded", a text key "" with integer value "1", and a null diagnostic. The empty-value carrier should retain text key "a" and text value "". The two-entry compiler carrier should retain both entries and the release-bytes compiler declaration 0.8.30.

Impact and scope

This is a bounded wrong-result defect in the interpreted metadata facet. A valid supported scalar encoding changes the reported status and removes all interpreted entries, including a compiler declaration when one is present. The original bytecode remains available, and the observed interface worker succeeds. No severe harm, contract execution or broader interface-inspection failure was observed.

The verifier reproduced six synthetic worker cases. Production prevalence and successful public Linux CLI/MCP behavior were not measured. Re-encoding a synthetic empty value as 0x7800 succeeds but changes the selected bytes, so it does not restore inspection of the caller's exact artifact. An external lossless-decoder workaround remains unverified.

REA version

6.4.0 (main at 5830088520)

Runtime and platform

Node 26.10.0; macOS 27.0 arm64

Locked dependencies: cborg 6.1.3 and EVMole 0.9.3. Worker execution was observed on macOS. The public provider's documented supported host is Linux x64.

Target and artifact context

Synthetic local hex carriers containing flat legacy EVM bytecode 60006000f3, a definite terminal CBOR root map and its exact two-byte payload length. No network, live chain, contract execution or authentic Solidity-origin assumption is required. The smallest failing carriers are 60006000f3a160010003 for an empty key and 60006000f3a16161600004 for an empty value.

Interface

CLI

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

Observed production-worker metadata for empty-key:
{"profile":"terminal-cbor-scalars-v1","status":"not-decoded","bytecode_offset":5,"cbor_bytes":3,"entries":[],"compiler_version":null,"diagnostic":"CBOR text bytes were unavailable for strict UTF-8 validation."}

All six worker cases: exit 0, ok:true, no stdout/stderr from the worker.
empty-value:     not-decoded; cbor_bytes:4; same diagnostic; entries:[]
empty-and-solc:  not-decoded; cbor_bytes:13; same diagnostic; entries:[]; compiler_version:null
control:         decoded; text "a" -> integer "1"; diagnostic:null
long-form-empty: decoded; text "a" -> text ""; diagnostic:null
invalid-utf8:    not-decoded; diagnostic:"The encoded data was not valid for encoding utf-8"

Existing focused suite: npm run test:local -- src/evm/SolidityMetadata.test.ts
1 file, 15 tests passed. The suite omits shortest-form empty-text coverage.

Typed outcome or recovery expected

Preserve the exact typed empty scalar entries with status: "decoded" and diagnostic: null. Retain both entries and compiler declaration 0.8.30 for the two-entry carrier. Keep the non-empty and longer-form empty controls decoded, and keep malformed UTF-8 not-decoded. Original bytecode and candidate ranges must remain unchanged. Verify the same metadata through the supported CLI or MCP tool on an admitted Linux x64 host.

Analysis or suspected cause

Source-derived at baseline 58300885207133422af235e0f2b10658cf958b03: src/evm/SolidityMetadata.ts:104-118 selects the valid terminal-map candidate. The strict text-validation branch at src/evm/SolidityMetadata.ts:38-43 throws when a string token lacks byteValue; its catch at src/evm/SolidityMetadata.ts:146-155 clears all interpreted entries and the compiler version.

In locked cborg 6.1.3, node_modules/cborg/lib/jump.js:152 supplies the quick token for shortest-form empty text without retained bytes, and node_modules/cborg/lib/decode.js:44-55 returns that token directly. The longer 0x7800 encoding uses the retaining string decoder at node_modules/cborg/lib/3string.js:49-55,77-78. This source distinction agrees with the independently observed control results.

Public-entry reachability is source-derived. src/cli/evmCommands.ts:16-35, src/composition/evm.ts:8-11 and src/application/evm/EvmInterfaceService.ts:37-56 route the selected local carrier to the production provider. After supported-host/resource admission, src/evm/EvmoleInterfaceProvider.ts:139-249 launches the same worker. src/evm/EvmoleInterfaceWorker.ts:31-70 accepts these carriers, runs interface inference and invokes independent metadata interpretation. The provider carries the metadata into validated output at src/evm/EvmoleInterfaceProvider.ts:300-374; MCP reaches the same service through src/server/registerEvmTools.ts:47.

Additional context

Related feature issue #2388 covers broader lossless metadata inspection and deferred supplied-metadata commitment binding; its implementation PR #2420 was superseded by merged PR #2790, while this report concerns the separately reproduced shortest-form empty-text decoding failure in that implemented feature.

P2 is supported because valid input produces consequential metadata loss within a bounded facet while preserving original bytecode and the observed successful worker result. No observed severe harm supports P0/P1. The workaround status remains unverified.

Originally created by @coygeek on GitHub (Oct 14, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3203 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior A valid terminal CBOR root map containing an empty text key or value encoded as the shortest-form token `0x60` produces `bytecode.metadata.status: "not-decoded"`, no entries and a null compiler version. The diagnostic is `CBOR text bytes were unavailable for strict UTF-8 validation.` An independently valid `solc` release-byte declaration in the same map also disappears from the interpreted result. The actual compiled production worker reproduced this behavior on macOS. All six worker cases exited 0 and returned `ok: true` without stdout or stderr. The equivalent empty value encoded with the longer `0x78 0x00` length form decoded successfully. The defect concerns shortest-form empty text, rather than every encoding of an empty string. The public caller command is `inspect-evm-interface`, with the equivalent MCP tool `inspect_evm_interface`. Successful public CLI/MCP execution on an admitted Linux x64 host was not run. Source tracing connects both public interfaces to the observed worker. The attempted macOS CLI call correctly exited 1 with `capability_unavailable` before analysis because the public provider requires Linux x64. ### Reproduction 1. Start with a clean checkout at `58300885207133422af235e0f2b10658cf958b03`, use its declared Node toolchain, and install/build from the repository root: ```sh npm ci npm run build:cached ``` These are clean-checkout prerequisites. The independent verifier used the prepared build and did not rerun installation or build during verification. 2. Run this self-contained reproduction from the repository root. It creates synthetic hex carriers and request files, isolates the worker's home and XDG directories, and executes the compiled production worker used by the provider: ```sh node --input-type=module <<'JS' import { mkdir, writeFile, readFile } from 'node:fs/promises'; import { resolve, join } from 'node:path'; import { execFileSync } from 'node:child_process'; const root = resolve('repro-cbor'); const home = join(root, 'home'); const env = { ...process.env, HOME: home, XDG_CONFIG_HOME: join(home, 'config'), XDG_CACHE_HOME: join(home, 'cache'), XDG_DATA_HOME: join(home, 'data'), TMPDIR: root, }; for (const key of ['HOME', 'XDG_CONFIG_HOME', 'XDG_CACHE_HOME', 'XDG_DATA_HOME']) { await mkdir(env[key], { recursive: true }); } const cases = [ ['empty-key', 'a16001'], ['empty-value', 'a1616160'], ['control', 'a1616101'], ['invalid-utf8', 'a1617861ff'], ['long-form-empty', 'a161617800'], ['empty-and-solc', 'a261616064736f6c634300081e'], ]; for (const [name, payload] of cases) { const dir = join(root, name); await mkdir(dir, { recursive: true }); const carrier = join(dir, 'carrier.hex'); const replyPath = join(dir, 'reply.json'); const requestPath = join(dir, 'request.json'); const length = (payload.length / 2).toString(16).padStart(4, '0'); await writeFile(carrier, '60006000f3' + payload + length); await writeFile(requestPath, JSON.stringify({ snapshot_path: carrier, reply_path: replyPath, failure_marker_path: join(dir, 'resource.failure'), encoding: 'hex', })); execFileSync(process.execPath, [ '--max-old-space-size=256', '--disable-wasm-trap-handler', resolve('dist/evm/EvmoleInterfaceWorker.js'), requestPath, ], { env, timeout: 30000 }); const reply = JSON.parse(await readFile(replyPath, 'utf8')); console.log(JSON.stringify({ case: name, ok: reply.ok, metadata: reply.bytecode.metadata })); } JS ``` 3. Compare the metadata printed for the six cases with the evidence below. The empty key, empty value and empty value plus `solc` cases report `not-decoded`; the non-empty and longer-form empty controls report `decoded`. Invalid UTF-8 correctly remains `not-decoded`. 4. To check the public caller boundary on an admitted Linux x64 host with util-linux `prlimit`, readable Linux process limits and sufficient inherited resource limits, run: ```sh node scripts/rea.mjs inspect-evm-interface repro-cbor/empty-key/carrier.hex hex --json node scripts/rea.mjs inspect-evm-interface repro-cbor/empty-value/carrier.hex hex --json ``` Step 2 was independently executed. Successful Linux CLI calls in step 4 were not executed during verification; their reachability is source-derived. The corresponding MCP request uses `inspect_evm_interface` with an absolute local carrier path and `encoding: "hex"`. ### Expected behavior `docs/specs/solidity/cbor-metadata/spec.md:18-26` requires lossless scalar root-map key/value interpretation and retention through public CLI/MCP. `docs/evm-bytecode.md:42-48` documents the independently interpreted terminal-CBOR facet, and `src/domain/evm/solidityMetadata.ts:5` admits empty text. A plausible terminal root map does not require proof of Solidity origin. The empty-key carrier should report `status: "decoded"`, a text key `""` with integer value `"1"`, and a null diagnostic. The empty-value carrier should retain text key `"a"` and text value `""`. The two-entry compiler carrier should retain both entries and the `release-bytes` compiler declaration `0.8.30`. ### Impact and scope This is a bounded wrong-result defect in the interpreted metadata facet. A valid supported scalar encoding changes the reported status and removes all interpreted entries, including a compiler declaration when one is present. The original bytecode remains available, and the observed interface worker succeeds. No severe harm, contract execution or broader interface-inspection failure was observed. The verifier reproduced six synthetic worker cases. Production prevalence and successful public Linux CLI/MCP behavior were not measured. Re-encoding a synthetic empty value as `0x7800` succeeds but changes the selected bytes, so it does not restore inspection of the caller's exact artifact. An external lossless-decoder workaround remains unverified. ### REA version 6.4.0 (main at 58300885207133422af235e0f2b10658cf958b03) ### Runtime and platform Node 26.10.0; macOS 27.0 arm64 Locked dependencies: cborg 6.1.3 and EVMole 0.9.3. Worker execution was observed on macOS. The public provider's documented supported host is Linux x64. ### Target and artifact context Synthetic local hex carriers containing flat legacy EVM bytecode `60006000f3`, a definite terminal CBOR root map and its exact two-byte payload length. No network, live chain, contract execution or authentic Solidity-origin assumption is required. The smallest failing carriers are `60006000f3a160010003` for an empty key and `60006000f3a16161600004` for an empty value. ### Interface CLI ### Earliest failure stage Native analysis or reconstruction ### Relevant output or evidence ```text Observed production-worker metadata for empty-key: {"profile":"terminal-cbor-scalars-v1","status":"not-decoded","bytecode_offset":5,"cbor_bytes":3,"entries":[],"compiler_version":null,"diagnostic":"CBOR text bytes were unavailable for strict UTF-8 validation."} All six worker cases: exit 0, ok:true, no stdout/stderr from the worker. empty-value: not-decoded; cbor_bytes:4; same diagnostic; entries:[] empty-and-solc: not-decoded; cbor_bytes:13; same diagnostic; entries:[]; compiler_version:null control: decoded; text "a" -> integer "1"; diagnostic:null long-form-empty: decoded; text "a" -> text ""; diagnostic:null invalid-utf8: not-decoded; diagnostic:"The encoded data was not valid for encoding utf-8" Existing focused suite: npm run test:local -- src/evm/SolidityMetadata.test.ts 1 file, 15 tests passed. The suite omits shortest-form empty-text coverage. ``` ### Typed outcome or recovery expected Preserve the exact typed empty scalar entries with `status: "decoded"` and `diagnostic: null`. Retain both entries and compiler declaration `0.8.30` for the two-entry carrier. Keep the non-empty and longer-form empty controls decoded, and keep malformed UTF-8 not-decoded. Original bytecode and candidate ranges must remain unchanged. Verify the same metadata through the supported CLI or MCP tool on an admitted Linux x64 host. ### Analysis or suspected cause Source-derived at baseline `58300885207133422af235e0f2b10658cf958b03`: `src/evm/SolidityMetadata.ts:104-118` selects the valid terminal-map candidate. The strict text-validation branch at `src/evm/SolidityMetadata.ts:38-43` throws when a string token lacks `byteValue`; its catch at `src/evm/SolidityMetadata.ts:146-155` clears all interpreted entries and the compiler version. In locked cborg 6.1.3, `node_modules/cborg/lib/jump.js:152` supplies the quick token for shortest-form empty text without retained bytes, and `node_modules/cborg/lib/decode.js:44-55` returns that token directly. The longer `0x7800` encoding uses the retaining string decoder at `node_modules/cborg/lib/3string.js:49-55,77-78`. This source distinction agrees with the independently observed control results. Public-entry reachability is source-derived. `src/cli/evmCommands.ts:16-35`, `src/composition/evm.ts:8-11` and `src/application/evm/EvmInterfaceService.ts:37-56` route the selected local carrier to the production provider. After supported-host/resource admission, `src/evm/EvmoleInterfaceProvider.ts:139-249` launches the same worker. `src/evm/EvmoleInterfaceWorker.ts:31-70` accepts these carriers, runs interface inference and invokes independent metadata interpretation. The provider carries the metadata into validated output at `src/evm/EvmoleInterfaceProvider.ts:300-374`; MCP reaches the same service through `src/server/registerEvmTools.ts:47`. ### Additional context Related feature issue #2388 covers broader lossless metadata inspection and deferred supplied-metadata commitment binding; its implementation PR #2420 was superseded by merged PR #2790, while this report concerns the separately reproduced shortest-form empty-text decoding failure in that implemented feature. P2 is supported because valid input produces consequential metadata loss within a bounded facet while preserving original bytecode and the observed successful worker result. No observed severe harm supports P0/P1. The workaround status remains unverified.

bsakaguchi

2026-10-15 18:31:25 +00:00