morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #6119] [Bug] Environment capture payload errors blame the already-valid operation field #3752

New issue

Closed

opened 2026-10-31 00:43:40 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-31 00:43:40 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 21, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6119

Originally assigned to: @kaoru0822-kitauji on GitHub.

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

Both environment-serve and environment-export blame evidence.operation for authenticated Evidence whose operation is supported but whose normalized_result is malformed. This reproduced for inspect_web_network_capture and capture_browser_scenario. In both cases the supplied operation already equals one of the values listed as expected, while the actual payload-schema issues are discarded.

Reproduction

Executed a focused production-loader test using digest-valid synthetic Evidence and a matching manifest. The Evidence factory computes the IDs; arbitrary edits to existing Evidence would instead stop at authentication. Save this test as tests/boundary/browser/environmentPayload.repro.test.ts, set REA_VERIFY_ARTIFACT_DIR to an existing writable directory for fixture files, and run:

node node_modules/vitest/vitest.mjs run tests/boundary/browser/environmentPayload.repro.test.ts --maxWorkers 1

import { expect, it } from 'vitest';
import { writeFile } from 'node:fs/promises';
import { join } from 'node:path';
import { environmentServeFixture } from '../../fixtures/environmentServe.js';
import { createEvidence, immutableEvidence } from '../../../src/domain/evidence.js';
import { loadEnvironmentSource } from '../../../src/application/environment/environmentSource.js';
import { projectAnalysisError } from '../../../src/domain/analysisErrorProjection.js';
it.each(['inspect_web_network_capture','capture_browser_scenario'])('misattributes a malformed authenticated %s payload',async operation=>{
 const fixture=environmentServeFixture();
 const capture=createEvidence(undefined,fixture.capture.provider,{operation,parameters:{},result:{},confidence:'observed'});
 const original=fixture.manifest.normalized_result as any;
 const manifest=createEvidence(undefined,fixture.manifest.provider,{operation:'inspect_analysis_view',parameters:{},result:{...original,parent_evidence_id:capture.evidence_id},confidence:'derived'});
 expect(immutableEvidence(capture).evidence_id).toBe(capture.evidence_id);
 for(const command of ['environment-serve','environment-export']){
  const result=loadEnvironmentSource(manifest,capture,command);expect(result.ok).toBe(false);if(result.ok)throw Error('expected refusal');
  const projection=projectAnalysisError(result.error);console.log('ENV',operation,command,JSON.stringify(projection));
  expect(projection.details).toMatchObject({operation:command,issues:[{path:['evidence','operation'],reason:'invalid_value'}]});
 }
 await writeFile(join(process.env.REA_VERIFY_ARTIFACT_DIR!,`${operation}.capture.json`),JSON.stringify(capture));
 await writeFile(join(process.env.REA_VERIFY_ARTIFACT_DIR!,`${operation}.manifest.json`),JSON.stringify(manifest));
});

Then execute the generated pairs through the built public commands, as was done for both operation names:

node scripts/rea.mjs environment-serve "$REA_VERIFY_ARTIFACT_DIR/inspect_web_network_capture.manifest.json" --evidence "$REA_VERIFY_ARTIFACT_DIR/inspect_web_network_capture.capture.json" --format json
node scripts/rea.mjs environment-export "$REA_VERIFY_ARTIFACT_DIR/inspect_web_network_capture.manifest.json" "$REA_VERIFY_ARTIFACT_DIR/export" --evidence "$REA_VERIFY_ARTIFACT_DIR/inspect_web_network_capture.capture.json" --format json

Replace the file prefix with capture_browser_scenario to exercise the second branch. All four actual CLI executions exited 1 with the operation-field error before starting a listener or writing a bundle. The focused assertions characterize the incorrect behavior.

Expected behavior

Reserve the operation-field issue for an unsupported operation. For a recognized operation, preserve the actual payload-schema issues beneath evidence.normalized_result. The same loader already prefixes envelope-schema issues by file role, and the shared input issue helper retains schema paths and causes. The capture-kind branches know which schema failed.

Impact and scope

P3: misleading correction detail for malformed, correctly rejected inputs. The evidence remains rejected; no invalid success, replay corruption or failed real-world recovery was observed. Callers can validate the normalized payload against its owning capture schema manually. The fixture deliberately constructs malformed but digest-valid Evidence; no normal REA capture producer emitting such data is established.

REA version

rea-agents 6.7.0; source revision 52415975942a4b9c53a817915b313ccc404f280e.

Runtime and platform

Node.js v26.12.0; macOS 27.0.1 arm64; rea-agents 6.7.0 source checkout with existing build and locked dependencies. The repository Node.js 24.18.0 pin was not separately exercised. Locked Vitest 5.0.3; focused source test plus separately spawned built CLI.

Target and artifact context

Synthetic authenticated Evidence with normalized_result equal to an empty JSON object, paired with a valid environment-manifest Evidence whose parent ID cites it. Both supported capture operation names were exercised.

Interface

CLI

Earliest failure stage

Request parsing or validation

Relevant output or evidence

Actual environment-export output, reduced to the affected fields:

{
  "code": "invalid_request",
  "category": "invalid_input",
  "message": "The evidence file is not web network capture or browser scenario Evidence.",
  "details": {
    "operation": "environment-export",
    "issues": [
      {
        "path": [
          "evidence",
          "operation"
        ],
        "reason": "invalid_value",
        "message": "The evidence file must be complete inspect_web_network_capture or capture_browser_scenario Evidence.",
        "expected": "inspect_web_network_capture or capture_browser_scenario"
      }
    ]
  }
}

The corresponding serve command differs only in command attribution. The browser-scenario pair gives the same issue even though its operation is capture_browser_scenario. Both Evidence envelopes passed authentication before payload validation.

Typed outcome or recovery expected

Keep invalid_request/invalid_input and the existing input-contract recovery. Report the fields that actually fail the selected capture schema, with paths beginning evidence.normalized_result; retain the schema cause internally. Do not blame a supported operation value.

Analysis or suspected cause

The two recognized branches return only on schema success. Both schema failures fall through to a generic error at ["evidence", "operation"]. This is distinct from a bad Evidence digest and from the later parent-ID pairing check. Both commands use this same parser.

Additional context

Fetched main aa51e4343963fdd0b97ec2ebdc3c41579fd9cb38 leaves this code unchanged; no newer runtime was executed. #6044 concerns a separate HAR grouping-view parent parser in WebNetworkProjectionAccumulator. #5974 concerns state counters, #6070 shutdown cleanup, and #6091 retained session cleanup status. PR #5800 changes this file but its patch still falls through on payload validation failure after replacing the HAR parser with safeParseOwnedWebNetworkCapture. Both capture branches belong in this one report. No last known good version was established.

Originally created by @coygeek on GitHub (Oct 21, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6119 Originally assigned to: @kaoru0822-kitauji on GitHub. ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior Both `environment-serve` and `environment-export` blame `evidence.operation` for authenticated Evidence whose operation is supported but whose `normalized_result` is malformed. This reproduced for `inspect_web_network_capture` and `capture_browser_scenario`. In both cases the supplied operation already equals one of the values listed as expected, while the actual payload-schema issues are discarded. ### Reproduction Executed a focused production-loader test using digest-valid synthetic Evidence and a matching manifest. The Evidence factory computes the IDs; arbitrary edits to existing Evidence would instead stop at authentication. Save this test as `tests/boundary/browser/environmentPayload.repro.test.ts`, set `REA_VERIFY_ARTIFACT_DIR` to an existing writable directory for fixture files, and run: ```sh node node_modules/vitest/vitest.mjs run tests/boundary/browser/environmentPayload.repro.test.ts --maxWorkers 1 ``` ```ts import { expect, it } from 'vitest'; import { writeFile } from 'node:fs/promises'; import { join } from 'node:path'; import { environmentServeFixture } from '../../fixtures/environmentServe.js'; import { createEvidence, immutableEvidence } from '../../../src/domain/evidence.js'; import { loadEnvironmentSource } from '../../../src/application/environment/environmentSource.js'; import { projectAnalysisError } from '../../../src/domain/analysisErrorProjection.js'; it.each(['inspect_web_network_capture','capture_browser_scenario'])('misattributes a malformed authenticated %s payload',async operation=>{ const fixture=environmentServeFixture(); const capture=createEvidence(undefined,fixture.capture.provider,{operation,parameters:{},result:{},confidence:'observed'}); const original=fixture.manifest.normalized_result as any; const manifest=createEvidence(undefined,fixture.manifest.provider,{operation:'inspect_analysis_view',parameters:{},result:{...original,parent_evidence_id:capture.evidence_id},confidence:'derived'}); expect(immutableEvidence(capture).evidence_id).toBe(capture.evidence_id); for(const command of ['environment-serve','environment-export']){ const result=loadEnvironmentSource(manifest,capture,command);expect(result.ok).toBe(false);if(result.ok)throw Error('expected refusal'); const projection=projectAnalysisError(result.error);console.log('ENV',operation,command,JSON.stringify(projection)); expect(projection.details).toMatchObject({operation:command,issues:[{path:['evidence','operation'],reason:'invalid_value'}]}); } await writeFile(join(process.env.REA_VERIFY_ARTIFACT_DIR!,`${operation}.capture.json`),JSON.stringify(capture)); await writeFile(join(process.env.REA_VERIFY_ARTIFACT_DIR!,`${operation}.manifest.json`),JSON.stringify(manifest)); }); ``` Then execute the generated pairs through the built public commands, as was done for both operation names: ```sh node scripts/rea.mjs environment-serve "$REA_VERIFY_ARTIFACT_DIR/inspect_web_network_capture.manifest.json" --evidence "$REA_VERIFY_ARTIFACT_DIR/inspect_web_network_capture.capture.json" --format json node scripts/rea.mjs environment-export "$REA_VERIFY_ARTIFACT_DIR/inspect_web_network_capture.manifest.json" "$REA_VERIFY_ARTIFACT_DIR/export" --evidence "$REA_VERIFY_ARTIFACT_DIR/inspect_web_network_capture.capture.json" --format json ``` Replace the file prefix with `capture_browser_scenario` to exercise the second branch. All four actual CLI executions exited 1 with the operation-field error before starting a listener or writing a bundle. The focused assertions characterize the incorrect behavior. ### Expected behavior Reserve the operation-field issue for an unsupported operation. For a recognized operation, preserve the actual payload-schema issues beneath `evidence.normalized_result`. The [same loader already prefixes envelope-schema issues by file role](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/application/environment/environmentSource.ts#L81-L110), and the [shared input issue helper](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/domain/inputIssueProjection.ts#L8-L28) retains schema paths and causes. The capture-kind branches know which schema failed. ### Impact and scope P3: misleading correction detail for malformed, correctly rejected inputs. The evidence remains rejected; no invalid success, replay corruption or failed real-world recovery was observed. Callers can validate the normalized payload against its owning capture schema manually. The fixture deliberately constructs malformed but digest-valid Evidence; no normal REA capture producer emitting such data is established. ### REA version rea-agents 6.7.0; source revision `52415975942a4b9c53a817915b313ccc404f280e`. ### Runtime and platform Node.js v26.12.0; macOS 27.0.1 arm64; rea-agents 6.7.0 source checkout with existing build and locked dependencies. The repository Node.js 24.18.0 pin was not separately exercised. Locked Vitest 5.0.3; focused source test plus separately spawned built CLI. ### Target and artifact context Synthetic authenticated Evidence with normalized_result equal to an empty JSON object, paired with a valid environment-manifest Evidence whose parent ID cites it. Both supported capture operation names were exercised. ### Interface CLI ### Earliest failure stage Request parsing or validation ### Relevant output or evidence Actual `environment-export` output, reduced to the affected fields: ```json { "code": "invalid_request", "category": "invalid_input", "message": "The evidence file is not web network capture or browser scenario Evidence.", "details": { "operation": "environment-export", "issues": [ { "path": [ "evidence", "operation" ], "reason": "invalid_value", "message": "The evidence file must be complete inspect_web_network_capture or capture_browser_scenario Evidence.", "expected": "inspect_web_network_capture or capture_browser_scenario" } ] } } ``` The corresponding serve command differs only in command attribution. The browser-scenario pair gives the same issue even though its operation is capture_browser_scenario. Both Evidence envelopes passed authentication before payload validation. ### Typed outcome or recovery expected Keep invalid_request/invalid_input and the existing input-contract recovery. Report the fields that actually fail the selected capture schema, with paths beginning `evidence.normalized_result`; retain the schema cause internally. Do not blame a supported operation value. ### Analysis or suspected cause The [two recognized branches](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/application/environment/environmentSource.ts#L176-L237) return only on schema success. Both schema failures fall through to a generic error at `["evidence", "operation"]`. This is distinct from a bad Evidence digest and from the later parent-ID pairing check. Both commands use this same parser. ### Additional context Fetched main `aa51e4343963fdd0b97ec2ebdc3c41579fd9cb38` leaves this code unchanged; no newer runtime was executed. #6044 concerns a separate HAR grouping-view parent parser in WebNetworkProjectionAccumulator. #5974 concerns state counters, #6070 shutdown cleanup, and #6091 retained session cleanup status. PR #5800 changes this file but its patch still falls through on payload validation failure after replacing the HAR parser with safeParseOwnedWebNetworkCapture. Both capture branches belong in this one report. No last known good version was established.

bsakaguchi closed this issue

2026-10-31 00:43:40 +00:00

bsakaguchi commented

2026-10-31 00:43:41 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 22, 2026):

确认，归因错位得很典型：用户明明填了受支持的 operation，错误却指向 operation 说它不被接受，真实的 payload schema 问题反而被丢弃——客户端照着错误去改 operation 只会原地打转。这跟 #6122 是同一类病根：错误投影没有做"字段级指向校验"。建议 payload schema 校验失败时优先报告具体字段路径与违反的约束（对齐现有 envelope-schema 按 file role 前缀的做法），operation 仅在其真不支持时才作为错误来源；并加断言：operation 合法时它绝不出现于错误 detail。

@XinwangTao commented on GitHub (Oct 22, 2026): 确认，归因错位得很典型：用户明明填了受支持的 operation，错误却指向 operation 说它不被接受，真实的 payload schema 问题反而被丢弃——客户端照着错误去改 operation 只会原地打转。这跟 #6122 是同一类病根：错误投影没有做"字段级指向校验"。建议 payload schema 校验失败时优先报告具体字段路径与违反的约束（对齐现有 envelope-schema 按 file role 前缀的做法），operation 仅在其真不支持时才作为错误来源；并加断言：operation 合法时它绝不出现于错误 detail。

bsakaguchi commented

2026-10-31 00:43:41 +00:00

Author

Owner

@kaoru0822-kitauji commented on GitHub (Oct 22, 2026):

Reproduced on latest main d77bc0e8a with the issue's production-loader test (run as a source test via npm run test:local). For authenticated Evidence with normalized_result: {} and operation inspect_web_network_capture or capture_browser_scenario, loadEnvironmentSource returns for both environment-serve and environment-export:

{"issues":[{"path":["evidence","operation"],"reason":"invalid_value","expected":"inspect_web_network_capture or capture_browser_scenario"}]}

The capture-schema issues are discarded. Not re-run through the built CLI. Working on a fix in parseCaptureEvidence (src/application/environment/environmentSource.ts).

@kaoru0822-kitauji commented on GitHub (Oct 22, 2026): Reproduced on latest main `d77bc0e8a` with the issue's production-loader test (run as a source test via `npm run test:local`). For authenticated Evidence with `normalized_result: {}` and operation `inspect_web_network_capture` or `capture_browser_scenario`, `loadEnvironmentSource` returns for both `environment-serve` and `environment-export`: ```json {"issues":[{"path":["evidence","operation"],"reason":"invalid_value","expected":"inspect_web_network_capture or capture_browser_scenario"}]} ``` The capture-schema issues are discarded. Not re-run through the built CLI. Working on a fix in `parseCaptureEvidence` (src/application/environment/environmentSource.ts).

bsakaguchi referenced this issue

2026-10-31 01:10:14 +00:00

[PR #3778] [MERGED] fix(browser): release attached emulation overrides #5892

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#3752

No description provided.

Rows
Columns