[GH-ISSUE #551] Inline Evidence omits normalized_result, so tool output cannot be fed to the comparison tools #280

New issue

Closed

opened 2026-10-15 18:23:24 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-15 18:23:24 +00:00

Owner

Originally created by @whft on GitHub (Oct 5, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/551

Remaining scope

compactEvidence still removes normalized_result from the nested evidence object although comparison contracts require it. Preserve the complete record inline so tool output can feed comparison and projection inputs without a bundle re-fetch or hand-built record. Validate the emitted record through its next consumer boundary; align the projection with the canonical Evidence contract.

Verification

Exercise representative valid and malformed producer data through the consumer boundary and assert the resulting observations, unknowns, or actionable failure. Keep CLI/MCP behavior aligned where applicable.

This scope was refreshed after removal of replay and permission plumbing and merge of PR #538; implementation is being addressed in the boundary root-cause cleanup.

Reproduction

Inside one session:

const a = await analyze_function({ procedure: "0x14000125c" });

Object.keys(a.evidence);
// [ 'evidence_id', 'subject', 'provider', 'predicate_type', 'operation',
//   'parameters', 'raw_result', 'confidence', 'authority', 'environment',
//   'limitations', 'locations', 'evidence_links', 'analysis_profile' ]
//  ^ no normalized_result

await compare_functions({ left: a.evidence, right: a.evidence });
// Input validation error: Invalid arguments for tool compare_functions:
//   left.normalized_result: Invalid input,
//   right.normalized_result: Invalid input

The identical records retrieved through the session bundle work:

const bundle = await get_evidence_bundle({});
const d = bundle.result.records.find((r) => r.operation === "analyze_function");

Object.keys(d);
// [ 'evidence_id', 'subject', 'provider', 'predicate_type', 'operation',
//   'parameters', 'raw_result', 'normalized_result', 'confidence',
//   'authority', 'environment', 'limitations', 'locations',
//   'evidence_links', 'analysis_profile' ]
//  ^ normalized_result present

await compare_functions({ left: d, right: d });
// OK

So the same observation yields two differently-shaped Evidence objects depending on how it was obtained. normalized_result on the bundle record carries the full dossier:

procedure, pseudocode, assembly, comments, callers, callees,
incoming_references, outgoing_references, referenced_strings,
referenced_names, basic_blocks, native_api, native_value_flow, limitations

Why this matters

The natural agent behaviour — call a tool, take its evidence, feed it to the next tool — is exactly what fails. The recovery (re-fetch everything through get_evidence_bundle) is not discoverable from the error message, which reports only Invalid input` against a field the caller never received.

It also makes normalized_result look like a field that does not exist, which invites a caller to "fix" it by populating raw_result or hand-building a record — both of which will fail validation for unrelated reasons.

Originally created by @whft on GitHub (Oct 5, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/551 ## Remaining scope compactEvidence still removes normalized_result from the nested evidence object although comparison contracts require it. Preserve the complete record inline so tool output can feed comparison and projection inputs without a bundle re-fetch or hand-built record. Validate the emitted record through its next consumer boundary; align the projection with the canonical Evidence contract. ## Verification Exercise representative valid and malformed producer data through the consumer boundary and assert the resulting observations, unknowns, or actionable failure. Keep CLI/MCP behavior aligned where applicable. This scope was refreshed after removal of replay and permission plumbing and merge of PR #538; implementation is being addressed in the boundary root-cause cleanup. ## Reproduction Inside one session: ```js const a = await analyze_function({ procedure: "0x14000125c" }); Object.keys(a.evidence); // [ 'evidence_id', 'subject', 'provider', 'predicate_type', 'operation', // 'parameters', 'raw_result', 'confidence', 'authority', 'environment', // 'limitations', 'locations', 'evidence_links', 'analysis_profile' ] // ^ no normalized_result await compare_functions({ left: a.evidence, right: a.evidence }); // Input validation error: Invalid arguments for tool compare_functions: // left.normalized_result: Invalid input, // right.normalized_result: Invalid input ``` The identical records retrieved through the session bundle work: ```js const bundle = await get_evidence_bundle({}); const d = bundle.result.records.find((r) => r.operation === "analyze_function"); Object.keys(d); // [ 'evidence_id', 'subject', 'provider', 'predicate_type', 'operation', // 'parameters', 'raw_result', 'normalized_result', 'confidence', // 'authority', 'environment', 'limitations', 'locations', // 'evidence_links', 'analysis_profile' ] // ^ normalized_result present await compare_functions({ left: d, right: d }); // OK ``` So the same observation yields two differently-shaped Evidence objects depending on how it was obtained. `normalized_result` on the bundle record carries the full dossier: ``` procedure, pseudocode, assembly, comments, callers, callees, incoming_references, outgoing_references, referenced_strings, referenced_names, basic_blocks, native_api, native_value_flow, limitations ``` ## Why this matters The natural agent behaviour — call a tool, take its `evidence`, feed it to the next tool — is exactly what fails. The recovery (`re-fetch everything through `get_evidence_bundle`) is not discoverable from the error message, which reports only `Invalid input` against a field the caller never received. It also makes `normalized_result` look like a field that does not exist, which invites a caller to "fix" it by populating `raw_result` or hand-building a record — both of which will fail validation for unrelated reasons.

bsakaguchi

2026-10-15 18:23:24 +00:00