[PR #2088] [CLOSED] fix(comparison): qualify exact function text differences #5017

Closed
opened 2026-10-31 00:58:09 +00:00 by bsakaguchi · 0 comments
Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/2088
Author: @Jax367
Created: 10/12/2026
Status: ❌ Closed

Base: main ← Head: codex/qualify-function-text-differences


📝 Commits (10+)

  • 0e6e63d fix(comparison): qualify exact function text differences
  • ef30383 fix(comparison): preserve dimension caveats in derived Evidence
  • d875434 fix(comparison): keep caveats scoped to selected dimensions
  • 14f241f chore: merge main into function text comparison fixes
  • 8b7f4f4 fix(comparison): validate nested Evidence closure before selecting findings
  • cf77e6d fix(comparison): share closure validation with changed behavior
  • abebf2e chore: merge main into PR 2088
  • 7a8fddf fix(comparison): recheck historical calls-only changes (#2223)
  • 5e78a2d fix(android): make DEX descriptor pattern portable
  • f313a48 fix(ci): isolate Linux Hopper process fixture inventory

📊 Changes

11 files changed (+259 additions, -88 deletions)

View changed files

📝 docs/mcp-contracts.md (+14 -0)
📝 src/contracts/investigationExamples.ts (+1 -0)
📝 src/domain/changedBehavior.ts (+6 -14)
📝 src/domain/comparisonSourceEvidence.ts (+12 -0)
📝 src/domain/functionComparison.test.ts (+40 -0)
📝 src/domain/functionComparisonDimensions.ts (+6 -3)
📝 src/domain/reconstructionVerification.functionSources.test.ts (+21 -18)
📝 src/domain/reconstructionVerification.ts (+23 -20)
📝 src/domain/staticRuntimeCorrelation.test.ts (+71 -1)
📝 src/domain/staticRuntimeCorrelation.ts (+16 -16)
📝 tests/boundary/mcp/functionComparisonMcp.test.ts (+49 -16)

📄 Description

Summary

compare_functions reports exact pseudocode differences without explaining that relocated addresses and generated names can account for them. Qualify both text dimensions, preserve the selected dimension's caveats in reconstruction verification and static/runtime correlation, and document how to read them alongside CFG comparisons. Refs #1947 (the documentation-first option in that design issue).

Problem and expected behavior

For example, FUN_00001010() versus FUN_00002010() produces pseudocode: changed even when only the function's location changed. The result should expose that exact-text limitation inline, so consumers do not treat a textual difference as proof of changed behavior.

Change and scope

  • Qualify pseudocode and assembly comparisons, including unresolved provider comparisons and bounded-diff failures.
  • Carry status and limitations together when selecting a function dimension for reconstruction verification or static/runtime correlation. Keep the Evidence envelope for whole-comparison caveats; an overall function claim includes all dimension limitations. Keep canonical example Evidence consistent with production.
  • Document unchanged CFGs as observed block-size/successor agreement, without claiming behavioral equivalence.
  • Preserve the existing exact statuses, digests, numeric constants, and eight dimensions. Address-normalized dimensions remain a separate design decision; this PR does not close #1947.

Contract and boundary impact

  • Semantic owner and earliest changed stage: domain text comparison in functionComparisonDimensions.ts, and dimension selection in reconstruction verification/static-runtime correlation.
  • CLI and MCP/tool-catalog contract: existing result shape; additional limitation text only. compare_functions is an MCP tool.
  • Provider, bridge, target-format, or platform compatibility: no provider changes.
  • Evidence, artifact, provenance, or reconstruction contract: original input Evidence links, exact text digests and comparison statuses retained; result Evidence identity changes to include the new qualification.
  • Process execution, authorization, cleanup, or containment impact: none.
  • Generated metadata, package, or installation impact: none; generated ignored documentation artifacts were checked locally.

Evidence and regression coverage

  • Tests updated: function comparison and static/runtime correlation domain tests, and the existing MCP reconstruction comparison journey. Verify qualifications on selected claims/correlations, no assembly/CFG caveat leakage into pseudocode, and aggregation only for an overall function claim. The correlation regression uses a producer-generated unknown CFG alongside usable text dimensions.

  • Base evidence: source-derived missing pseudocode qualification; a relocated-label fixture verifies exact text remains changed while CFG remains unchanged. The new inline-limitation assertions fail on the base revision.

  • User-visible MCP output: dimensions[].limitations states that exact provider text has no address/generated-name normalization and layout-only differences do not establish a behavior change.

  • Remaining proof gaps: no real Hopper, Ghidra or IDA run; this change consumes already-authenticated dossiers without executing a native provider.

  • Observed, derived, and inferred claims remain distinguishable.

  • Artifact identity, source provenance, and failed attempts remain preserved.

  • Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

Executed on native Windows with Node 24.18.0 / npm 11.16.0:

  • npm run test:focused -- src/domain/functionComparison.test.ts tests/boundary/mcp/functionComparisonMcp.test.ts src/domain/staticRuntimeCorrelation.test.ts src/domain/reconstructionVerification.test.ts — 49 tests passed.
  • npm run check — all six tasks and architecture guards passed (existing lint warnings, zero errors).
  • npm run docs:check — all six tasks passed.
  • Compiled production stdio MCP probe with canonical function/process Evidence — qualification and source links verified through comparison, reconstruction verification and static/runtime correlation, including the absence of assembly-only caveats from pseudocode claims/correlations; exact statuses retained; connection closed normally.
  • git diff --check — passed.

Compatibility, safety, and release

  • Breaking changes or migration steps: none.
  • Real Hopper/Ghidra, browser, or OS coverage: native Windows domain/MCP checks; no native engine or browser run.
  • Package or release metadata impact: none.
  • Security, privacy, process, or containment review: no new permissions, target execution, or persistent configuration.

Review checklist

  • This PR addresses a concrete problem or an agreed enhancement.
  • The PR has one focused outcome and the title follows type(scope): outcome.
  • Related issue is linked.
  • Tests cover changed observable behavior and meaningful failure paths.
  • Owning docs, contracts, and generated metadata are updated where needed.
  • User-visible MCP changes include representative output.
  • I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/2088 **Author:** [@Jax367](https://github.com/Jax367) **Created:** 10/12/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `codex/qualify-function-text-differences` --- ### 📝 Commits (10+) - [`0e6e63d`](https://github.com/morluto/rea/commit/0e6e63d0310c591f4b129c9e93a01fb701f870b2) fix(comparison): qualify exact function text differences - [`ef30383`](https://github.com/morluto/rea/commit/ef303832d5c2c328f9a446f4151e7fe8667d0e5b) fix(comparison): preserve dimension caveats in derived Evidence - [`d875434`](https://github.com/morluto/rea/commit/d87543485c74541bcc81f816fcaf542d91a6b607) fix(comparison): keep caveats scoped to selected dimensions - [`14f241f`](https://github.com/morluto/rea/commit/14f241fe5b678289929857b81a9daff8dd10a197) chore: merge main into function text comparison fixes - [`8b7f4f4`](https://github.com/morluto/rea/commit/8b7f4f4ea7af2c801d896337457ce01e789da053) fix(comparison): validate nested Evidence closure before selecting findings - [`cf77e6d`](https://github.com/morluto/rea/commit/cf77e6d04987b17986c3e5f2e2587fb7f47818ac) fix(comparison): share closure validation with changed behavior - [`abebf2e`](https://github.com/morluto/rea/commit/abebf2e6b2e9f6ad80d1024441e4286fb7e4c186) chore: merge main into PR 2088 - [`7a8fddf`](https://github.com/morluto/rea/commit/7a8fddf0efe9fcc265fcdc78af5aa8d70fd49f0c) fix(comparison): recheck historical calls-only changes (#2223) - [`5e78a2d`](https://github.com/morluto/rea/commit/5e78a2d7a1b7136057f55d31ffea986c38bcc909) fix(android): make DEX descriptor pattern portable - [`f313a48`](https://github.com/morluto/rea/commit/f313a48b24c89527f8ac493356fc073f80976d73) fix(ci): isolate Linux Hopper process fixture inventory ### 📊 Changes **11 files changed** (+259 additions, -88 deletions) <details> <summary>View changed files</summary> 📝 `docs/mcp-contracts.md` (+14 -0) 📝 `src/contracts/investigationExamples.ts` (+1 -0) 📝 `src/domain/changedBehavior.ts` (+6 -14) 📝 `src/domain/comparisonSourceEvidence.ts` (+12 -0) 📝 `src/domain/functionComparison.test.ts` (+40 -0) 📝 `src/domain/functionComparisonDimensions.ts` (+6 -3) 📝 `src/domain/reconstructionVerification.functionSources.test.ts` (+21 -18) 📝 `src/domain/reconstructionVerification.ts` (+23 -20) 📝 `src/domain/staticRuntimeCorrelation.test.ts` (+71 -1) 📝 `src/domain/staticRuntimeCorrelation.ts` (+16 -16) 📝 `tests/boundary/mcp/functionComparisonMcp.test.ts` (+49 -16) </details> ### 📄 Description ## Summary `compare_functions` reports exact pseudocode differences without explaining that relocated addresses and generated names can account for them. Qualify both text dimensions, preserve the selected dimension's caveats in reconstruction verification and static/runtime correlation, and document how to read them alongside CFG comparisons. Refs #1947 (the documentation-first option in that design issue). ## Problem and expected behavior For example, `FUN_00001010()` versus `FUN_00002010()` produces `pseudocode: changed` even when only the function's location changed. The result should expose that exact-text limitation inline, so consumers do not treat a textual difference as proof of changed behavior. ## Change and scope - Qualify pseudocode and assembly comparisons, including unresolved provider comparisons and bounded-diff failures. - Carry status and limitations together when selecting a function dimension for reconstruction verification or static/runtime correlation. Keep the Evidence envelope for whole-comparison caveats; an overall function claim includes all dimension limitations. Keep canonical example Evidence consistent with production. - Document unchanged CFGs as observed block-size/successor agreement, without claiming behavioral equivalence. - Preserve the existing exact statuses, digests, numeric constants, and eight dimensions. Address-normalized dimensions remain a separate design decision; this PR does not close #1947. ## Contract and boundary impact - Semantic owner and earliest changed stage: domain text comparison in `functionComparisonDimensions.ts`, and dimension selection in reconstruction verification/static-runtime correlation. - CLI and MCP/tool-catalog contract: existing result shape; additional limitation text only. `compare_functions` is an MCP tool. - Provider, bridge, target-format, or platform compatibility: no provider changes. - Evidence, artifact, provenance, or reconstruction contract: original input Evidence links, exact text digests and comparison statuses retained; result Evidence identity changes to include the new qualification. - Process execution, authorization, cleanup, or containment impact: none. - Generated metadata, package, or installation impact: none; generated ignored documentation artifacts were checked locally. ## Evidence and regression coverage - Tests updated: function comparison and static/runtime correlation domain tests, and the existing MCP reconstruction comparison journey. Verify qualifications on selected claims/correlations, no assembly/CFG caveat leakage into pseudocode, and aggregation only for an overall function claim. The correlation regression uses a producer-generated unknown CFG alongside usable text dimensions. - Base evidence: source-derived missing pseudocode qualification; a relocated-label fixture verifies exact text remains changed while CFG remains unchanged. The new inline-limitation assertions fail on the base revision. - User-visible MCP output: `dimensions[].limitations` states that exact provider text has no address/generated-name normalization and layout-only differences do not establish a behavior change. - Remaining proof gaps: no real Hopper, Ghidra or IDA run; this change consumes already-authenticated dossiers without executing a native provider. - [x] Observed, derived, and inferred claims remain distinguishable. - [x] Artifact identity, source provenance, and failed attempts remain preserved. - [x] Unsupported, incomplete, unavailable, or uncertain outcomes remain visible. ## Validation performed Executed on native Windows with Node 24.18.0 / npm 11.16.0: - `npm run test:focused -- src/domain/functionComparison.test.ts tests/boundary/mcp/functionComparisonMcp.test.ts src/domain/staticRuntimeCorrelation.test.ts src/domain/reconstructionVerification.test.ts` — 49 tests passed. - `npm run check` — all six tasks and architecture guards passed (existing lint warnings, zero errors). - `npm run docs:check` — all six tasks passed. - Compiled production stdio MCP probe with canonical function/process Evidence — qualification and source links verified through comparison, reconstruction verification and static/runtime correlation, including the absence of assembly-only caveats from pseudocode claims/correlations; exact statuses retained; connection closed normally. - `git diff --check` — passed. ## Compatibility, safety, and release - Breaking changes or migration steps: none. - Real Hopper/Ghidra, browser, or OS coverage: native Windows domain/MCP checks; no native engine or browser run. - Package or release metadata impact: none. - Security, privacy, process, or containment review: no new permissions, target execution, or persistent configuration. ## Review checklist - [x] This PR addresses a concrete problem or an agreed enhancement. - [x] The PR has one focused outcome and the title follows `type(scope): outcome`. - [x] Related issue is linked. - [x] Tests cover changed observable behavior and meaningful failure paths. - [x] Owning docs, contracts, and generated metadata are updated where needed. - [x] User-visible MCP changes include representative output. - [x] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
bsakaguchi 2026-10-31 00:58:09 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
morluto/rea#5017
No description provided.