morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #3388] [Bug] Different unnamed reference targets compare as unchanged #1256

New issue

Closed

opened 2026-10-16 21:03:26 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-16 21:03:26 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 14, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3388

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

compare_functions reports its references dimension and overall result as unchanged when two admitted function dossiers reference different addresses in the same artifact but neither target belongs to a named procedure. The result gives equal comparison digests and no limitation about missing target identity. Naming the targets makes the same comparison distinguish them, so the absent names erase the contradictory address observations.

The unnamed case succeeds through the public MCP tool. The different-name control reports changed, and the same-name relocation control reports unchanged. Those controls establish that the loss occurs when endpoint names are absent, rather than from a rejected Evidence shape or a general inability to compare reference records. The reproduction verifies the comparison service's treatment of admitted fixture Evidence only; it does not claim a particular engine produced these dossiers.

Reproduction

Start from a clean checkout of https://github.com/morluto/rea at 8dde6370d0809ad04cda3342d4e6f290f48aa063. Install the locked dependencies and build the runtime from the checkout root. The reported reproduction used an already built source checkout.

npm ci
npm run build:cached

Run the following from the checkout root. It constructs valid fixture Evidence inline using the shipped canonical comparison example, then calls the public compare_functions MCP tool over stdio. These are synthetic dossiers for checking comparison semantics; this reproduction does not require or establish real disassembler output. No target program runs. No input artifact file is required.

case_dir="$(mktemp -d)"
mkdir -p "$case_dir/home"
env HOME="$case_dir/home" REA_LOG_LEVEL=silent node --input-type=module <<'JS'
import { Client } from "@modelcontextprotocol/client";
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
import { createEvidence } from "./dist/domain/evidence.js";
import { FUNCTION_COMPARISON_EXAMPLE } from "./dist/contracts/functionComparisonExample.js";

const kind = {
  available: true, provenance: "ghidra-reference-manager", type: "READ",
  flow: false, call: false, jump: false, data: true, read: true,
  write: false, indirect: false, computed: false, conditional: false,
  terminal: false, primary: true, operand_index: 0, external: false
};
const procedure = (address, name) => ({
  address, name, classification: null,
  body: { available: false, reason: "Not recorded" }
});
function observe(target, name = null) {
  const original = FUNCTION_COMPARISON_EXAMPLE.left;
  const result = structuredClone(original.normalized_result);
  result.limitations = [];
  result.outgoing_references = [{
    source_address: "0x1000", target_address: target,
    source_procedure: procedure("0x1000", "main"),
    target_procedure: name === null ? null : procedure(target, name),
    kind
  }];
  return createEvidence(
    { path: "sample.bin", sha256: "0".repeat(64), format: "mach-o" },
    original.provider,
    {
      operation: "analyze_function", parameters: { procedure: "main" },
      result, confidence: "derived", authority: "shipped-artifact",
      relationshipCoverage: { status: "complete" }
    }
  );
}
const client = new Client({ name: "comparison-reproduction", version: "1" });
const transport = new StdioClientTransport({
  command: process.execPath, args: ["scripts/rea.mjs", "mcp"],
  cwd: process.cwd(), env: { ...process.env }
});
await client.connect(transport);
try {
  for (const [label, left, right] of [
    ["unnamed", observe("0x2000"), observe("0x3000")],
    ["different-names", observe("0x2000", "first"), observe("0x3000", "second")],
    ["same-name-relocation", observe("0x2000", "helper"), observe("0x3000", "helper")]
  ]) {
    const reply = await client.callTool({
      name: "compare_functions", arguments: { left, right }
    });
    if (reply.isError) throw new Error(JSON.stringify(reply.content));
    const result = reply.structuredContent.normalized_result;
    const references = result.dimensions.find(d => d.dimension === "references");
    console.log(JSON.stringify({ label, status: result.status, references }));
  }
} finally {
  await client.close();
}
JS
rm -rf "$case_dir"

Inspect the three output records. Both unnamed inputs identify the same artifact digest, the same selected function, and the same source address. Their only differing reference fact is target_address. Each input declares the reference facet available, retains complete direct-call coverage, and contains the same available READ metadata. The unnamed case reports overall status: "unchanged", equal reference digests, and no reference limitation. The different-name control reports changed; the same-name relocation control reports unchanged.

Expected behavior

The differing unnamed targets must not support an unqualified equality conclusion. For this same-artifact comparison, the recorded addresses identify different endpoints. If the comparison instead needs a stable cross-observation symbolic mapping, that mapping is unavailable and the dimension must preserve the uncertainty. Missing names must not turn two different addresses into an established match. The controls must continue to distinguish first from second and avoid treating a known symbol's relocation alone as a semantic change.

src/contracts/sessionToolContracts.ts:111-114 exposes comparison of explicit function Evidence across references and requires missing facets to remain unknown. AGENTS.md:21 requires unresolved identity to remain unknown rather than aligned. AGENTS.md:29 requires preserving source values and explicit unknowns when normalization loses information.

Impact and scope

Impact level: P2. The assessment is supported. An Evidence comparison can falsely claim reference equality after discarding unnamed endpoint coordinates, and can promote that result into overall function equality. This reproduction establishes a bounded semantic failure with admitted synthetic inputs, not the frequency of these inputs in real engine output.

A workaround is available for the same-artifact case. Callers can compare source_address and target_address in the retained input dossiers directly. The reproduction retains both different target addresses and uses the same digest and source address, so that manual check preserves the observations without relying on the false equality result.

REA version

6.4.0 (main at 8dde6370d0)

Runtime and platform

Node.js v26.10.0; macOS 27.0.1 on arm64; pinned @modelcontextprotocol/client 2.3.1. No external analysis engine was used.

Target and artifact context

Synthetic analyze_function dossiers constructed from the shipped canonical comparison example, with artifact metadata path: "sample.bin", a SHA-256 value of 64 zeroes, and format: "mach-o". The selected function is main at 0x1000; the outgoing READ targets are 0x2000 and 0x3000. The reference-kind provenance is the fixture value ghidra-reference-manager, not evidence of an executed Ghidra session. Both inputs retain the original reference records.

Interface

MCP

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

The observed unnamed case reports overall status: "unchanged" and this reference dimension excerpt:

{
  "dimension": "references",
  "status": "unchanged",
  "left_count": 1,
  "right_count": 1,
  "left_digest": "a52a6996c772498e8d26cbc7c9bdde67415479287655bcb92036d47eb28356ac",
  "right_digest": "a52a6996c772498e8d26cbc7c9bdde67415479287655bcb92036d47eb28356ac",
  "limitations": [],
  "conclusion_kind": "derived_relationship"
}

The observed different-name control reports changed. The observed same-name relocation control reports unchanged.

Typed outcome or recovery expected

Repeat the three MCP calls. The unnamed case must report the differing same-artifact endpoints as changed, or preserve unknown identity with an explanatory limitation when comparability cannot be established. It must not report unqualified unchanged. The different-name control must stay changed, and the same-name relocation control must retain its current behavior. Keep both original reference records and Evidence links accessible; replacing all reference comparisons with unknown is not restoration.

Analysis or suspected cause

Source-derived reasoning: src/domain/functionComparisonNormalization.ts:75-86 and src/domain/functionComparisonNormalization.ts:109-126 project endpoint names while omitting the recorded addresses. src/domain/functionComparisonDimensions.ts:288-290 compares those reduced records as equal once reference-kind metadata is available. The public stdio MCP reproduction returns the quoted result for different unnamed target addresses.

Evidence provenance is observed, local verification is reproduced, and reproduction completeness is complete. The reproduction establishes treatment of admitted synthetic Evidence, not output from a particular disassembler.

Additional context

Related #1947 proposes additive address-normalized provider-text dimensions for relocation noise; this report concerns differing unnamed reference endpoints being discarded from equality comparison.
Related #2222, closed as completed, addressed incomplete call coverage and unresolved call sites; this report retains complete direct-call coverage and available reference kinds but loses endpoint identity in the references dimension.
Related #3195 concerns preservation of complete call-coverage diagnostics while maintaining valid equality; this report concerns an unsupported references equality conclusion after known coordinates are discarded.
Related #2951 and #2980, closed as completed, addressed missing Ghidra availability and relationship declarations that caused unknown self-comparisons; these inputs explicitly declare availability and complete coverage but compare different endpoints as unchanged.

Originally created by @coygeek on GitHub (Oct 14, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3388 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior `compare_functions` reports its `references` dimension and overall result as `unchanged` when two admitted function dossiers reference different addresses in the same artifact but neither target belongs to a named procedure. The result gives equal comparison digests and no limitation about missing target identity. Naming the targets makes the same comparison distinguish them, so the absent names erase the contradictory address observations. The unnamed case succeeds through the public MCP tool. The different-name control reports `changed`, and the same-name relocation control reports `unchanged`. Those controls establish that the loss occurs when endpoint names are absent, rather than from a rejected Evidence shape or a general inability to compare reference records. The reproduction verifies the comparison service's treatment of admitted fixture Evidence only; it does not claim a particular engine produced these dossiers. ### Reproduction 1. Start from a clean checkout of `https://github.com/morluto/rea` at `8dde6370d0809ad04cda3342d4e6f290f48aa063`. Install the locked dependencies and build the runtime from the checkout root. The reported reproduction used an already built source checkout. ```sh npm ci npm run build:cached ``` 2. Run the following from the checkout root. It constructs valid fixture Evidence inline using the shipped canonical comparison example, then calls the public `compare_functions` MCP tool over stdio. These are synthetic dossiers for checking comparison semantics; this reproduction does not require or establish real disassembler output. No target program runs. No input artifact file is required. ```sh case_dir="$(mktemp -d)" mkdir -p "$case_dir/home" env HOME="$case_dir/home" REA_LOG_LEVEL=silent node --input-type=module <<'JS' import { Client } from "@modelcontextprotocol/client"; import { StdioClientTransport } from "@modelcontextprotocol/client/stdio"; import { createEvidence } from "./dist/domain/evidence.js"; import { FUNCTION_COMPARISON_EXAMPLE } from "./dist/contracts/functionComparisonExample.js"; const kind = { available: true, provenance: "ghidra-reference-manager", type: "READ", flow: false, call: false, jump: false, data: true, read: true, write: false, indirect: false, computed: false, conditional: false, terminal: false, primary: true, operand_index: 0, external: false }; const procedure = (address, name) => ({ address, name, classification: null, body: { available: false, reason: "Not recorded" } }); function observe(target, name = null) { const original = FUNCTION_COMPARISON_EXAMPLE.left; const result = structuredClone(original.normalized_result); result.limitations = []; result.outgoing_references = [{ source_address: "0x1000", target_address: target, source_procedure: procedure("0x1000", "main"), target_procedure: name === null ? null : procedure(target, name), kind }]; return createEvidence( { path: "sample.bin", sha256: "0".repeat(64), format: "mach-o" }, original.provider, { operation: "analyze_function", parameters: { procedure: "main" }, result, confidence: "derived", authority: "shipped-artifact", relationshipCoverage: { status: "complete" } } ); } const client = new Client({ name: "comparison-reproduction", version: "1" }); const transport = new StdioClientTransport({ command: process.execPath, args: ["scripts/rea.mjs", "mcp"], cwd: process.cwd(), env: { ...process.env } }); await client.connect(transport); try { for (const [label, left, right] of [ ["unnamed", observe("0x2000"), observe("0x3000")], ["different-names", observe("0x2000", "first"), observe("0x3000", "second")], ["same-name-relocation", observe("0x2000", "helper"), observe("0x3000", "helper")] ]) { const reply = await client.callTool({ name: "compare_functions", arguments: { left, right } }); if (reply.isError) throw new Error(JSON.stringify(reply.content)); const result = reply.structuredContent.normalized_result; const references = result.dimensions.find(d => d.dimension === "references"); console.log(JSON.stringify({ label, status: result.status, references })); } } finally { await client.close(); } JS rm -rf "$case_dir" ``` 3. Inspect the three output records. Both unnamed inputs identify the same artifact digest, the same selected function, and the same source address. Their only differing reference fact is `target_address`. Each input declares the reference facet available, retains complete direct-call coverage, and contains the same available `READ` metadata. The unnamed case reports overall `status: "unchanged"`, equal reference digests, and no reference limitation. The different-name control reports `changed`; the same-name relocation control reports `unchanged`. ### Expected behavior The differing unnamed targets must not support an unqualified equality conclusion. For this same-artifact comparison, the recorded addresses identify different endpoints. If the comparison instead needs a stable cross-observation symbolic mapping, that mapping is unavailable and the dimension must preserve the uncertainty. Missing names must not turn two different addresses into an established match. The controls must continue to distinguish `first` from `second` and avoid treating a known symbol's relocation alone as a semantic change. `src/contracts/sessionToolContracts.ts:111-114` exposes comparison of explicit function Evidence across references and requires missing facets to remain unknown. `AGENTS.md:21` requires unresolved identity to remain unknown rather than aligned. `AGENTS.md:29` requires preserving source values and explicit unknowns when normalization loses information. ### Impact and scope Impact level: P2. The assessment is supported. An Evidence comparison can falsely claim reference equality after discarding unnamed endpoint coordinates, and can promote that result into overall function equality. This reproduction establishes a bounded semantic failure with admitted synthetic inputs, not the frequency of these inputs in real engine output. A workaround is available for the same-artifact case. Callers can compare `source_address` and `target_address` in the retained input dossiers directly. The reproduction retains both different target addresses and uses the same digest and source address, so that manual check preserves the observations without relying on the false equality result. ### REA version 6.4.0 (main at 8dde6370d0809ad04cda3342d4e6f290f48aa063) ### Runtime and platform Node.js v26.10.0; macOS 27.0.1 on arm64; pinned `@modelcontextprotocol/client` 2.3.1. No external analysis engine was used. ### Target and artifact context Synthetic `analyze_function` dossiers constructed from the shipped canonical comparison example, with artifact metadata `path: "sample.bin"`, a SHA-256 value of 64 zeroes, and `format: "mach-o"`. The selected function is `main` at `0x1000`; the outgoing `READ` targets are `0x2000` and `0x3000`. The reference-kind provenance is the fixture value `ghidra-reference-manager`, not evidence of an executed Ghidra session. Both inputs retain the original reference records. ### Interface MCP ### Earliest failure stage Native analysis or reconstruction ### Relevant output or evidence The observed unnamed case reports overall `status: "unchanged"` and this reference dimension excerpt: ```text { "dimension": "references", "status": "unchanged", "left_count": 1, "right_count": 1, "left_digest": "a52a6996c772498e8d26cbc7c9bdde67415479287655bcb92036d47eb28356ac", "right_digest": "a52a6996c772498e8d26cbc7c9bdde67415479287655bcb92036d47eb28356ac", "limitations": [], "conclusion_kind": "derived_relationship" } ``` The observed different-name control reports `changed`. The observed same-name relocation control reports `unchanged`. ### Typed outcome or recovery expected Repeat the three MCP calls. The unnamed case must report the differing same-artifact endpoints as changed, or preserve unknown identity with an explanatory limitation when comparability cannot be established. It must not report unqualified `unchanged`. The different-name control must stay changed, and the same-name relocation control must retain its current behavior. Keep both original reference records and Evidence links accessible; replacing all reference comparisons with unknown is not restoration. ### Analysis or suspected cause Source-derived reasoning: `src/domain/functionComparisonNormalization.ts:75-86` and `src/domain/functionComparisonNormalization.ts:109-126` project endpoint names while omitting the recorded addresses. `src/domain/functionComparisonDimensions.ts:288-290` compares those reduced records as equal once reference-kind metadata is available. The public stdio MCP reproduction returns the quoted result for different unnamed target addresses. Evidence provenance is observed, local verification is reproduced, and reproduction completeness is complete. The reproduction establishes treatment of admitted synthetic Evidence, not output from a particular disassembler. ### Additional context - Related [#1947](https://github.com/morluto/rea/issues/1947) proposes additive address-normalized provider-text dimensions for relocation noise; this report concerns differing unnamed reference endpoints being discarded from equality comparison. - Related [#2222](https://github.com/morluto/rea/issues/2222), closed as completed, addressed incomplete call coverage and unresolved call sites; this report retains complete direct-call coverage and available reference kinds but loses endpoint identity in the references dimension. - Related [#3195](https://github.com/morluto/rea/issues/3195) concerns preservation of complete call-coverage diagnostics while maintaining valid equality; this report concerns an unsupported references equality conclusion after known coordinates are discarded. - Related [#2951](https://github.com/morluto/rea/issues/2951) and [#2980](https://github.com/morluto/rea/issues/2980), closed as completed, addressed missing Ghidra availability and relationship declarations that caused unknown self-comparisons; these inputs explicitly declare availability and complete coverage but compare different endpoints as unchanged.

bsakaguchi closed this issue

2026-10-16 21:03:26 +00:00

bsakaguchi commented

2026-10-16 21:03:27 +00:00

Author

Owner

@pablodiazjorge commented on GitHub (Oct 14, 2026):

I'd like to take this one. I have a fix ready locally: an unnamed endpoint keeps its recorded address when both observations share an artifact, and stays unknown with a limitation across artifacts unless a known fact already differs; named endpoints still match by name, so the relocation control stays unchanged. I'll open the PR as soon as one of my open PRs lands, since I'm at the three-PR limit.

@pablodiazjorge commented on GitHub (Oct 14, 2026): I'd like to take this one. I have a fix ready locally: an unnamed endpoint keeps its recorded address when both observations share an artifact, and stays unknown with a limitation across artifacts unless a known fact already differs; named endpoints still match by name, so the relocation control stays unchanged. I'll open the PR as soon as one of my open PRs lands, since I'm at the three-PR limit.

bsakaguchi referenced this issue

2026-10-17 21:09:44 +00:00

[PR #1256] [MERGED] fix(errors): report caller-selected evidence path failures as invalid input #2744

bsakaguchi referenced this issue

2026-10-31 00:39:00 +00:00

[GH-ISSUE #5449] [Bug] trace-application-feature reports changed Evidence as a retryable input error #3309

bsakaguchi referenced this issue

2026-10-31 00:57:47 +00:00

[PR #2051] [CLOSED] fix(evidence): report a single record selected as a bundle as invalid input #4991