morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #6814] [Bug] Rust dependency inspection silently discards duplicate JSON fields #4161

New issue

Closed

opened 2026-10-31 00:47:17 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:47:17 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 23, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6814

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

Rust dependency inspection silently keeps the final occurrence of duplicate JSON fields in a .dep-v0 section. Duplicate name changes alpha to beta; duplicate dependencies discards the recorded edge; duplicate top-level packages discards the entire first package list. All three synthetic ELF inputs return ok: true, without a duplicate-field diagnostic.

The published auditable-serde 0.9.0 reader rejects the same JSON strings with duplicate field errors. The ordinary control succeeds in both readers. This is an executed static-parser/provider comparison; no binary or target code was executed.

Reproduction

With the source revision below and locked dependencies installed, save this as /tmp/rea-rust-duplicates.mts. Run REA_ROOT="$PWD" node_modules/.bin/tsx /tmp/rea-rust-duplicates.mts from the checkout:

import {pathToFileURL} from 'node:url';
import {deflateSync} from 'node:zlib';
const {readAuditableSection} = await import(pathToFileURL(
  `${process.env.REA_ROOT}/src/rust/RustAuditableReader.ts`).href);
const cases = {
  control: '{"packages":[{"name":"alpha","version":"1.0.0","source":"local","dependencies":[],"root":true}]}',
  'package-name': '{"packages":[{"name":"alpha","name":"beta","version":"1.0.0","source":"local","root":true}]}',
  dependencies: '{"packages":[{"name":"alpha","version":"1.0.0","source":"local","dependencies":[1],"dependencies":[],"root":true},{"name":"beta","version":"1.0.0","source":"local"}]}',
  'top-level': '{"packages":[{"name":"alpha","version":"1.0.0","source":"local","root":true}],"packages":[]}',
};
for (const [name,text] of Object.entries(cases)) {
  const r = readAuditableSection('.dep-v0',0,deflateSync(Buffer.from(text)));
  console.log(JSON.stringify({name,packages:r.packages.map(p=>p.name),edges:r.dependency_edges}));
}

The executed parser probe loaded these exact four strings from files. A separate provider probe wrapped each compressed payload with createRustImage({format:'elf',payload,includeRustc:false}) from tests/fixtures/rust/image.ts, wrote the image, and called new RustBinaryProvider().inspect({path}). Each parser case and each provider case ran once before drafting.

For the independent comparison, a temporary Rust program deserialized each exact string with serde_json::from_str::<auditable_serde::VersionInfo>(&text), using auditable-serde = "=0.9.0" and serde_json = "=1.0.149".

Expected behavior

Surface duplicate-field ambiguity before converting the JSON to ordinary objects, either as a typed malformed-format result or an explicit unresolved observation preserving the conflicting fields. Do not silently report only one package identity, dependency vector, or package list.

This matches the observed rejection by the published format reader. Its VersionInfo and Package deserialization declarations are from the source revision recorded in the 0.9.0 crate. This is a format-reader consistency expectation, not a claim that duplicate member names make every JSON document syntactically invalid.

Impact and scope

Bounded metadata correctness defect for ambiguous or damaged dependency records: REA returns a changed identity or an empty dependency graph without identifying discarded declarations. These fixtures are self-authored; ordinary cargo-auditable output was not observed to contain duplicates. No security impact, binary mutation, or real-world prevalence is asserted.

REA version

rea-agents 6.7.0; source revision e2efd10b70a2b5f15a7ac1b5a457a9a1dbcab4d4. Latest npm version was 6.7.0; the published package artifact was not executed separately.

Runtime and platform

Node.js v24.22.1, Linux x86_64, locked tsx. Independent reader: Rust 1.99.1, auditable-serde 0.9.0, serde_json 1.0.149.

Target and artifact context

Four small synthetic zlib JSON payloads, also embedded in synthetic 4,096-byte ELF64 fixtures with one .dep-v0 section. No private artifacts or executable target runs.

Interface

Other

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

Recorded parser/provider outcomes and independent reader results:

case           REA packages     REA edges  provider  auditable-serde
control        [alpha]          []         ok=true   Ok
package-name   [beta]           []         ok=true   duplicate field `name`
dependencies   [alpha,beta]      []         ok=true   duplicate field `dependencies`
top-level      []               []         ok=true   duplicate field `packages`

The dependency case explicitly contains [1] before the second dependencies: []; the top-level case contains a rooted alpha package before the second packages: []. The complete input strings are above. Provider limitations describe ordinary provenance/resource bounds but contain no duplicate-field warning.

Typed outcome or recovery expected

An explicit format/ambiguity outcome identifying the duplicate field, rather than successful metadata with the earlier occurrence discarded.

Analysis or suspected cause

Source-derived: parseDocument calls JSON.parse before schema validation. The later Zod and graph checks receive only the last occurrence and cannot detect what was lost.

Additional context

Open and closed searches covered Rust duplicate keys, auditable duplicate, and Rust JSON duplicate; no matching report was found before drafting. This is distinct from #6780, which concerns CBOR byte-string keys in EVM metadata.

NOT_ENOUGH_INFO: first affected release, last known-good release, occurrence in actual exporter files, and a verified general workaround. No verification was repeated after this draft was written. Prepared with AI assistance.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: Three reduced ambiguous metadata records return successful changed package identities or dependency inventories through the production parser and ELF provider; the published reference reader rejects all three.
Workaround status: unverified
Workaround basis: The reference reader identifies these duplicate fields, but no complete alternative ELF extraction and analysis workflow was tested.

Originally created by @coygeek on GitHub (Oct 23, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6814 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior Rust dependency inspection silently keeps the final occurrence of duplicate JSON fields in a `.dep-v0` section. Duplicate `name` changes `alpha` to `beta`; duplicate `dependencies` discards the recorded edge; duplicate top-level `packages` discards the entire first package list. All three synthetic ELF inputs return `ok: true`, without a duplicate-field diagnostic. The published `auditable-serde` 0.9.0 reader rejects the same JSON strings with `duplicate field` errors. The ordinary control succeeds in both readers. This is an executed static-parser/provider comparison; no binary or target code was executed. ### Reproduction With the source revision below and locked dependencies installed, save this as `/tmp/rea-rust-duplicates.mts`. Run `REA_ROOT="$PWD" node_modules/.bin/tsx /tmp/rea-rust-duplicates.mts` from the checkout: ```ts import {pathToFileURL} from 'node:url'; import {deflateSync} from 'node:zlib'; const {readAuditableSection} = await import(pathToFileURL( `${process.env.REA_ROOT}/src/rust/RustAuditableReader.ts`).href); const cases = { control: '{"packages":[{"name":"alpha","version":"1.0.0","source":"local","dependencies":[],"root":true}]}', 'package-name': '{"packages":[{"name":"alpha","name":"beta","version":"1.0.0","source":"local","root":true}]}', dependencies: '{"packages":[{"name":"alpha","version":"1.0.0","source":"local","dependencies":[1],"dependencies":[],"root":true},{"name":"beta","version":"1.0.0","source":"local"}]}', 'top-level': '{"packages":[{"name":"alpha","version":"1.0.0","source":"local","root":true}],"packages":[]}', }; for (const [name,text] of Object.entries(cases)) { const r = readAuditableSection('.dep-v0',0,deflateSync(Buffer.from(text))); console.log(JSON.stringify({name,packages:r.packages.map(p=>p.name),edges:r.dependency_edges})); } ``` The executed parser probe loaded these exact four strings from files. A separate provider probe wrapped each compressed payload with `createRustImage({format:'elf',payload,includeRustc:false})` from `tests/fixtures/rust/image.ts`, wrote the image, and called `new RustBinaryProvider().inspect({path})`. Each parser case and each provider case ran once before drafting. For the independent comparison, a temporary Rust program deserialized each exact string with `serde_json::from_str::<auditable_serde::VersionInfo>(&text)`, using `auditable-serde = "=0.9.0"` and `serde_json = "=1.0.149"`. ### Expected behavior Surface duplicate-field ambiguity before converting the JSON to ordinary objects, either as a typed malformed-format result or an explicit unresolved observation preserving the conflicting fields. Do not silently report only one package identity, dependency vector, or package list. This matches the observed rejection by the published format reader. Its [VersionInfo and Package deserialization declarations](https://github.com/rust-secure-code/cargo-auditable/blob/65266184d1f37b3a04db5423108323c6e75fbe01/auditable-serde/src/lib.rs#L34-L102) are from the source revision recorded in the 0.9.0 crate. This is a format-reader consistency expectation, not a claim that duplicate member names make every JSON document syntactically invalid. ### Impact and scope Bounded metadata correctness defect for ambiguous or damaged dependency records: REA returns a changed identity or an empty dependency graph without identifying discarded declarations. These fixtures are self-authored; ordinary cargo-auditable output was not observed to contain duplicates. No security impact, binary mutation, or real-world prevalence is asserted. ### REA version `rea-agents` 6.7.0; source revision `e2efd10b70a2b5f15a7ac1b5a457a9a1dbcab4d4`. Latest npm version was 6.7.0; the published package artifact was not executed separately. ### Runtime and platform Node.js v24.22.1, Linux x86_64, locked tsx. Independent reader: Rust 1.99.1, auditable-serde 0.9.0, serde_json 1.0.149. ### Target and artifact context Four small synthetic zlib JSON payloads, also embedded in synthetic 4,096-byte ELF64 fixtures with one `.dep-v0` section. No private artifacts or executable target runs. ### Interface Other ### Earliest failure stage Native analysis or reconstruction ### Relevant output or evidence Recorded parser/provider outcomes and independent reader results: ```text case REA packages REA edges provider auditable-serde control [alpha] [] ok=true Ok package-name [beta] [] ok=true duplicate field `name` dependencies [alpha,beta] [] ok=true duplicate field `dependencies` top-level [] [] ok=true duplicate field `packages` ``` The dependency case explicitly contains `[1]` before the second `dependencies: []`; the top-level case contains a rooted `alpha` package before the second `packages: []`. The complete input strings are above. Provider limitations describe ordinary provenance/resource bounds but contain no duplicate-field warning. ### Typed outcome or recovery expected An explicit format/ambiguity outcome identifying the duplicate field, rather than successful metadata with the earlier occurrence discarded. ### Analysis or suspected cause Source-derived: [parseDocument calls JSON.parse before schema validation](https://github.com/morluto/rea/blob/e2efd10b70a2b5f15a7ac1b5a457a9a1dbcab4d4/src/rust/RustAuditableReader.ts#L209-L252). The later Zod and graph checks receive only the last occurrence and cannot detect what was lost. ### Additional context Open and closed searches covered `Rust duplicate keys`, `auditable duplicate`, and `Rust JSON duplicate`; no matching report was found before drafting. This is distinct from #6780, which concerns CBOR byte-string keys in EVM metadata. NOT_ENOUGH_INFO: first affected release, last known-good release, occurrence in actual exporter files, and a verified general workaround. No verification was repeated after this draft was written. Prepared with AI assistance. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: Three reduced ambiguous metadata records return successful changed package identities or dependency inventories through the production parser and ELF provider; the published reference reader rejects all three. - Workaround status: unverified - Workaround basis: The reference reader identifies these duplicate fields, but no complete alternative ELF extraction and analysis workflow was tested.

bsakaguchi closed this issue

2026-10-31 00:47:17 +00:00

bsakaguchi referenced this issue

2026-10-31 01:11:51 +00:00

[PR #4161] feat(mcp): support real-time operation progress and ETA notifications #6100

bsakaguchi referenced this issue

2026-11-01 21:11:07 +00:00

[GH-ISSUE #5654] [Tracker] Open PRs that can't merge yet: conflicts and red CI (October 2026) #3425