morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #5449] [Bug] trace-application-feature reports changed Evidence as a retryable input error #3309

New issue

Closed

opened 2026-10-31 00:39:00 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-31 00:39:00 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 20, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/5449

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

node scripts/rea.mjs trace-application-feature tampered-input.json --json rejects saved application Evidence whose semantic content no longer matches its evidence_id, but reports invalid_request, category: invalid_input, retryable: true, and generic argument-correction guidance. Its diagnostic details correctly identify the semantic-ID mismatch. The unmodified Evidence succeeds with otherwise identical trace arguments.

Reproduction

Independently reproduced through the built CLI. From the baseline checkout with dependencies installed and runtime built, set TMPDIR to an existing scratch directory and run:

(
  REA_ENTRY="$PWD/scripts/rea.mjs"
  CASE_DIR="$(mktemp -d "${TMPDIR:?Set TMPDIR to a scratch directory}/rea-evidence.XXXXXX")"
  export HOME="$CASE_DIR/home"
  export XDG_CONFIG_HOME="$CASE_DIR/config"
  export XDG_DATA_HOME="$CASE_DIR/data"
  export XDG_CACHE_HOME="$CASE_DIR/cache"
  export XDG_STATE_HOME="$CASE_DIR/state"
  mkdir -p "$HOME" "$XDG_CONFIG_HOME" "$XDG_DATA_HOME" "$XDG_CACHE_HOME" "$XDG_STATE_HOME" "$CASE_DIR/app"
  cd "$CASE_DIR" || exit 1
  printf '%s' '{"name":"fixture-app","version":"1.0.0","main":"main.js"}' > app/package.json
  printf '%s\n' 'module.exports = { run(value) { return "search-result:" + value; } };' > app/main.js
  node "$REA_ENTRY" analyze-javascript-application "$CASE_DIR/app" --json > application-evidence.json
  node --input-type=module <<'JS'
import { readFileSync, writeFileSync } from 'node:fs';
const application = JSON.parse(readFileSync('application-evidence.json', 'utf8'));
const request = {
  application,
  seed: { kind: 'string', value: 'search-result' },
  direction: 'both'
};
writeFileSync('control-input.json', JSON.stringify(request));
application.normalized_result.limitations = [];
writeFileSync('tampered-input.json', JSON.stringify(request));
JS
  node "$REA_ENTRY" trace-application-feature "$CASE_DIR/control-input.json" --json > control-result.json
  node "$REA_ENTRY" trace-application-feature "$CASE_DIR/tampered-input.json" --json
)

The producer returned five limitation strings. The mutation replaces only application.normalized_result.limitations with [], retaining the original evidence_id. The control exited 0; the changed record exited 1 with the error below and empty stderr. The verifier also replayed this exact recipe with the same outcome, and separately confirmed that --full-output preserves the error fields inside an ok: false envelope.

Expected behavior

Continue rejecting the changed record with exit 1 and no successful trace, but return the existing Evidence-integrity classification and recovery guidance. Source-derived contract: docs/user-visible-errors.csv:17 describes Evidence validation failures; src/domain/evidenceErrors.ts:13 covers Evidence identity failures; src/domain/analysisErrorProjection.ts:239 and src/domain/analysisErrorPresentation.ts:269 define the integrity error projection. docs/mcp-contracts.md:367 documents inline application Evidence and semantic/identity checks.

Impact and scope

Callers handling a saved Evidence identity failure receive the wrong machine-readable classification and generic recovery guidance. Assessment: P2. The changed record is still rejected safely; no accepted tampered result, security boundary crossing, data loss, or automated retry loop was observed. The verified workaround is to supply the original valid producer output, which succeeds with the same seed and direction. Runtime coverage is limited to trace-application-feature; other Evidence consumers and MCP were not exercised.

REA version

6.7.0 (rea-agents), source commit 074768901f3cdc8205b61773fb1faf6f6c7b3ca8.

Runtime and platform

macOS arm64. NOT_ENOUGH_INFO: the independent verifier's notes do not record the Node.js version or exact macOS release. No external analysis provider was used.

Target and artifact context

Disposable local JavaScript directory fixture: package.json names main.js; main.js exports run(value) returning the literal prefix search-result:. The target is analyzed statically. Input is complete saved application Evidence, with one semantic field changed while preserving its identifier.

Interface

CLI

Earliest failure stage

Request parsing or validation

Relevant output or evidence

Observed stdout from the changed-record request (exit 1, empty stderr):

{"error":"Application workflow failed","code":"invalid_request","category":"invalid_input","message":"Analysis input is invalid. Check the arguments and try again.","retryable":true,"remediation":{"action":"Correct the listed arguments and retry."},"details":{"operation":"trace_application_feature","issues":[{"path":["application","evidence_id"],"reason":"invalid_value","message":"Evidence semantic identifier does not match its record"}]}}

The pristine control returns successful trace Evidence with exit 0. These results establish rejection and incorrect classification, rather than an integrity bypass.

Typed outcome or recovery expected

code: evidence_integrity_mismatch, category: integrity_mismatch, retryable: false, and message Evidence is invalid or has changed. Recreate or re-import it, then try again. Remediation should direct the caller to recreate or re-import valid Evidence. Ordinary request-shape and selector errors should retain their applicable input-error handling.

Analysis or suspected cause

Source-derived explanation: src/domain/evidence.ts:177 detects the semantic identifier mismatch. src/application/javascript/JavaScriptApplicationWorkflowService.ts:60 invokes the application graph parser; src/application/javascript/JavaScriptApplicationEvidenceGraph.ts:350 converts the authentication issue into an input failure, and lines 46–50 create AnalysisInputError. src/cli/applicationCommands.ts:199 projects the resulting error. This conversion explains the observed classification.

Additional context

Prepared with AI assistance and independently reproduced by a separate verifier. Freshness was checked against origin/main at 052956af3416fa01d93837e54fdec3ad2739f6d1: the cited implementation files are unchanged from the tested baseline; the later change in the cited documentation adds unrelated native-function guidance. This is a source freshness check, not a runtime rerun on that revision. Searches covered open and closed issues and PRs by command, integrity code, symptom, and source module; related reports #4436, #3955, and PRs #1256 and #4761 concern different behaviors.

Originally created by @coygeek on GitHub (Oct 20, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/5449 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior `node scripts/rea.mjs trace-application-feature tampered-input.json --json` rejects saved application Evidence whose semantic content no longer matches its `evidence_id`, but reports `invalid_request`, `category: invalid_input`, `retryable: true`, and generic argument-correction guidance. Its diagnostic details correctly identify the semantic-ID mismatch. The unmodified Evidence succeeds with otherwise identical trace arguments. ### Reproduction Independently reproduced through the built CLI. From the baseline checkout with dependencies installed and runtime built, set `TMPDIR` to an existing scratch directory and run: ```sh ( REA_ENTRY="$PWD/scripts/rea.mjs" CASE_DIR="$(mktemp -d "${TMPDIR:?Set TMPDIR to a scratch directory}/rea-evidence.XXXXXX")" export HOME="$CASE_DIR/home" export XDG_CONFIG_HOME="$CASE_DIR/config" export XDG_DATA_HOME="$CASE_DIR/data" export XDG_CACHE_HOME="$CASE_DIR/cache" export XDG_STATE_HOME="$CASE_DIR/state" mkdir -p "$HOME" "$XDG_CONFIG_HOME" "$XDG_DATA_HOME" "$XDG_CACHE_HOME" "$XDG_STATE_HOME" "$CASE_DIR/app" cd "$CASE_DIR" || exit 1 printf '%s' '{"name":"fixture-app","version":"1.0.0","main":"main.js"}' > app/package.json printf '%s\n' 'module.exports = { run(value) { return "search-result:" + value; } };' > app/main.js node "$REA_ENTRY" analyze-javascript-application "$CASE_DIR/app" --json > application-evidence.json node --input-type=module <<'JS' import { readFileSync, writeFileSync } from 'node:fs'; const application = JSON.parse(readFileSync('application-evidence.json', 'utf8')); const request = { application, seed: { kind: 'string', value: 'search-result' }, direction: 'both' }; writeFileSync('control-input.json', JSON.stringify(request)); application.normalized_result.limitations = []; writeFileSync('tampered-input.json', JSON.stringify(request)); JS node "$REA_ENTRY" trace-application-feature "$CASE_DIR/control-input.json" --json > control-result.json node "$REA_ENTRY" trace-application-feature "$CASE_DIR/tampered-input.json" --json ) ``` The producer returned five limitation strings. The mutation replaces only `application.normalized_result.limitations` with `[]`, retaining the original `evidence_id`. The control exited 0; the changed record exited 1 with the error below and empty stderr. The verifier also replayed this exact recipe with the same outcome, and separately confirmed that `--full-output` preserves the error fields inside an `ok: false` envelope. ### Expected behavior Continue rejecting the changed record with exit 1 and no successful trace, but return the existing Evidence-integrity classification and recovery guidance. Source-derived contract: `docs/user-visible-errors.csv:17` describes Evidence validation failures; `src/domain/evidenceErrors.ts:13` covers Evidence identity failures; `src/domain/analysisErrorProjection.ts:239` and `src/domain/analysisErrorPresentation.ts:269` define the integrity error projection. `docs/mcp-contracts.md:367` documents inline application Evidence and semantic/identity checks. ### Impact and scope Callers handling a saved Evidence identity failure receive the wrong machine-readable classification and generic recovery guidance. Assessment: P2. The changed record is still rejected safely; no accepted tampered result, security boundary crossing, data loss, or automated retry loop was observed. The verified workaround is to supply the original valid producer output, which succeeds with the same seed and direction. Runtime coverage is limited to `trace-application-feature`; other Evidence consumers and MCP were not exercised. ### REA version 6.7.0 (`rea-agents`), source commit `074768901f3cdc8205b61773fb1faf6f6c7b3ca8`. ### Runtime and platform macOS arm64. NOT_ENOUGH_INFO: the independent verifier's notes do not record the Node.js version or exact macOS release. No external analysis provider was used. ### Target and artifact context Disposable local JavaScript directory fixture: `package.json` names `main.js`; `main.js` exports `run(value)` returning the literal prefix `search-result:`. The target is analyzed statically. Input is complete saved application Evidence, with one semantic field changed while preserving its identifier. ### Interface CLI ### Earliest failure stage Request parsing or validation ### Relevant output or evidence Observed stdout from the changed-record request (exit 1, empty stderr): ```json {"error":"Application workflow failed","code":"invalid_request","category":"invalid_input","message":"Analysis input is invalid. Check the arguments and try again.","retryable":true,"remediation":{"action":"Correct the listed arguments and retry."},"details":{"operation":"trace_application_feature","issues":[{"path":["application","evidence_id"],"reason":"invalid_value","message":"Evidence semantic identifier does not match its record"}]}} ``` The pristine control returns successful trace Evidence with exit 0. These results establish rejection and incorrect classification, rather than an integrity bypass. ### Typed outcome or recovery expected `code: evidence_integrity_mismatch`, `category: integrity_mismatch`, `retryable: false`, and message `Evidence is invalid or has changed. Recreate or re-import it, then try again.` Remediation should direct the caller to recreate or re-import valid Evidence. Ordinary request-shape and selector errors should retain their applicable input-error handling. ### Analysis or suspected cause Source-derived explanation: `src/domain/evidence.ts:177` detects the semantic identifier mismatch. `src/application/javascript/JavaScriptApplicationWorkflowService.ts:60` invokes the application graph parser; `src/application/javascript/JavaScriptApplicationEvidenceGraph.ts:350` converts the authentication issue into an input failure, and lines 46–50 create `AnalysisInputError`. `src/cli/applicationCommands.ts:199` projects the resulting error. This conversion explains the observed classification. ### Additional context Prepared with AI assistance and independently reproduced by a separate verifier. Freshness was checked against `origin/main` at `052956af3416fa01d93837e54fdec3ad2739f6d1`: the cited implementation files are unchanged from the tested baseline; the later change in the cited documentation adds unrelated native-function guidance. This is a source freshness check, not a runtime rerun on that revision. Searches covered open and closed issues and PRs by command, integrity code, symptom, and source module; related reports #4436, #3955, and PRs #1256 and #4761 concern different behaviors.

bsakaguchi closed this issue

2026-10-31 00:39:00 +00:00

bsakaguchi commented

2026-10-31 00:39:01 +00:00

Author

Owner

@coygeek commented on GitHub (Oct 24, 2026):

Checked the reported fix on main at d87e73f2db.

readEvidence yields EvidenceIntegrityError, captureInputValidation preserves it and the workflow adapter returns it without converting to AnalysisInputError. A production-source probe confirms pristine example success and an unchanged-ID/changed provider field rejection as evidence_integrity_mismatch/integrity_mismatch/retryable:false. Domain regression checks semantic tampering and MCP regression rejects spoofed inline application Evidence; no CLI file replay ran.

Implementation: JavaScriptApplicationEvidenceGraph.ts, JavaScriptApplicationEvidenceGraph.ts.

The relevant regression checks in evidence.test.ts, applicationEvidenceMcp.test.ts passed in the focused test run on this revision. This closure is based on the current source and those checks; provider fixtures do not establish a real-engine run. Closing as completed on main.

@coygeek commented on GitHub (Oct 24, 2026): Checked the reported fix on `main` at d87e73f2db50be72d33a4b3c7199af35518b441c. readEvidence yields EvidenceIntegrityError, captureInputValidation preserves it and the workflow adapter returns it without converting to AnalysisInputError. A production-source probe confirms pristine example success and an unchanged-ID/changed provider field rejection as evidence_integrity_mismatch/integrity_mismatch/retryable:false. Domain regression checks semantic tampering and MCP regression rejects spoofed inline application Evidence; no CLI file replay ran. Implementation: [JavaScriptApplicationEvidenceGraph.ts](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/src/application/javascript/JavaScriptApplicationEvidenceGraph.ts#L44), [JavaScriptApplicationEvidenceGraph.ts](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/src/application/javascript/JavaScriptApplicationEvidenceGraph.ts#L343). The relevant regression checks in [evidence.test.ts](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/src/domain/evidence.test.ts), [applicationEvidenceMcp.test.ts](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/tests/boundary/mcp/applicationEvidenceMcp.test.ts) passed in the focused test run on this revision. This closure is based on the current source and those checks; provider fixtures do not establish a real-engine run. Closing as completed on main.

bsakaguchi referenced this issue

2026-10-31 01:10:19 +00:00

[PR #3797] [MERGED] fix(javascript): keep string Worker paths unresolved without a document base #5905

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#3309

No description provided.

Rows
Columns