[GH-ISSUE #6691] [Bug] Environment comparison drops repeated query values and replays the wrong recorded request #4068

New issue

Closed

opened 2026-10-31 00:46:32 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:46:32 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 23, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6691

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

The shared environment comparison workflow drops every occurrence after the first of a repeated query parameter. With recorded responses for both ?tag=a&tag=b and ?tag=a&tag=c, a probe of the second recorded request receives the first request's response. Comparison reports a value disagreement even though the exact probe request and response already exist in the recording. ignored_query is empty.

Observed using the production comparison/replay functions and synthetic HAR Evidence; no live application, browser, or network request was used.

Reproduction

At the source revision below, install locked dependencies with npm ci. Save the following as /tmp/rea-repeated-query.mts, then run ./node_modules/.bin/tsx /tmp/rea-repeated-query.mts from the checkout root:

import {pathToFileURL} from 'node:url';
const root=process.cwd();
const load=(p:string)=>import(pathToFileURL(`${root}/${p}`).href);
const {analysisViewCaptureEvidence,environmentManifestCaptureFixture}=await load('tests/fixtures/analysisView.ts');
const {environmentStateServeFixture,environmentServeSource}=await load('tests/fixtures/environmentServe.ts');
const {compareEnvironment}=await load('src/application/environment/environmentCompare.ts');
const {matchProbeRequest}=await load('src/application/environment/environmentCompareMatch.ts');
const {replayEnvironmentTool}=await load('src/application/environment/environmentReplay.ts');
const row=(second:string)=>({request:{method:'GET',url:'http://app.example/api/items?tag=a&tag='+second},response:{status:200,content:{mimeType:'application/json',text:JSON.stringify({selected:second})}}});
const capture=(values:string[])=>analysisViewCaptureEvidence(environmentManifestCaptureFixture(values.map(row)));
const recorded=capture(['b','c']),probe=capture(['c']);
const source=environmentServeSource(environmentStateServeFixture(recorded));
const call=matchProbeRequest(source.manifest,{method:'GET',path:'/api/items',query:'?tag=a&tag=c',body:undefined});
const replay=replayEnvironmentTool(source,call.tool,call.args);
const result=compareEnvironment({source,captureValue:recorded,probes:[{label:'recorded-second-query',value:probe}],operation:'environment-compare'});
console.log(JSON.stringify({args:call.args,replay:replay.value,summary:result.value?.summary,step:result.value?.probes[0]?.results[0]},null,2));

The focused reproduction ran successfully and demonstrated the failure. It uses the repository's Evidence fixture constructors, then the actual manifest projection, replay, and comparison code.

Expected behavior

Replay and comparison should preserve all query values needed to identify the recorded request, or report that the request cannot be represented. They should not silently drop a value, select a different recording, and count the result as an ordinary response disagreement.

The documented comparison contract says the comparator answers the probe's calls as the served environment would and reports ignored query names.

Impact and scope

Observed for an ordinary repeated tag filter. Different requests become the same tool arguments, making recorded multi-value queries impossible to distinguish and producing a false fidelity failure. The same selection may falsely claim an exact replay. Other query-name patterns and real application captures were not tested.

REA version

Source revision 8c4f96ef4759fd78da66c4b9726825ca71d7ee0c (package.json reports 6.7.0). This is a current-source report; the published 6.7.0 package was not used for this reproduction.

Runtime and platform

Node.js v26.12.1; Linux x86_64.

Target and artifact context

Synthetic HAR Evidence containing two GET requests with distinct second tag values and JSON responses.

Interface

Other — production functions shared by the environment CLI workflow.

Earliest failure stage

Request parsing or validation

Relevant output or evidence

The probe request is ?tag=a&tag=c, but the observed values are:

{"args":{"tag":"a"},"replay":{"body":{"selected":"b"},"replay":{"record_ordinal":0,"exact_match":true,"recorded_query":"?tag=a&tag=b"}},"step":{"equal":false,"differences":[{"category":"value","pointer":"/selected","live":"\"c\"","env":"\"b\""}],"ignored_query":[]}}

Typed outcome or recovery expected

Preserve the complete selected request, or explicitly mark/refuse the unrepresentable query instead of silently discarding its second value.

Analysis or suspected cause

Source-derived: matchProbeRequest writes a query argument only if it is absent, silently skipping subsequent occurrences. scoreExchange then counts any matching value for that name as an exact query match; both recordings tie and the first wins.

Additional context

No fix is included. No real-provider behavior or published-release regression is claimed.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: A valid repeated-query probe selects a different recorded response and produces a false fidelity disagreement in the shared production workflow.
Workaround status: unverified
Workaround basis: The generated tool exposes one scalar argument per query name; no workaround retaining both selected values was tested.

Originally created by @coygeek on GitHub (Oct 23, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6691 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior The shared environment comparison workflow drops every occurrence after the first of a repeated query parameter. With recorded responses for both `?tag=a&tag=b` and `?tag=a&tag=c`, a probe of the second recorded request receives the first request's response. Comparison reports a value disagreement even though the exact probe request and response already exist in the recording. `ignored_query` is empty. Observed using the production comparison/replay functions and synthetic HAR Evidence; no live application, browser, or network request was used. ### Reproduction At the source revision below, install locked dependencies with `npm ci`. Save the following as `/tmp/rea-repeated-query.mts`, then run `./node_modules/.bin/tsx /tmp/rea-repeated-query.mts` from the checkout root: ```typescript import {pathToFileURL} from 'node:url'; const root=process.cwd(); const load=(p:string)=>import(pathToFileURL(`${root}/${p}`).href); const {analysisViewCaptureEvidence,environmentManifestCaptureFixture}=await load('tests/fixtures/analysisView.ts'); const {environmentStateServeFixture,environmentServeSource}=await load('tests/fixtures/environmentServe.ts'); const {compareEnvironment}=await load('src/application/environment/environmentCompare.ts'); const {matchProbeRequest}=await load('src/application/environment/environmentCompareMatch.ts'); const {replayEnvironmentTool}=await load('src/application/environment/environmentReplay.ts'); const row=(second:string)=>({request:{method:'GET',url:'http://app.example/api/items?tag=a&tag='+second},response:{status:200,content:{mimeType:'application/json',text:JSON.stringify({selected:second})}}}); const capture=(values:string[])=>analysisViewCaptureEvidence(environmentManifestCaptureFixture(values.map(row))); const recorded=capture(['b','c']),probe=capture(['c']); const source=environmentServeSource(environmentStateServeFixture(recorded)); const call=matchProbeRequest(source.manifest,{method:'GET',path:'/api/items',query:'?tag=a&tag=c',body:undefined}); const replay=replayEnvironmentTool(source,call.tool,call.args); const result=compareEnvironment({source,captureValue:recorded,probes:[{label:'recorded-second-query',value:probe}],operation:'environment-compare'}); console.log(JSON.stringify({args:call.args,replay:replay.value,summary:result.value?.summary,step:result.value?.probes[0]?.results[0]},null,2)); ``` The focused reproduction ran successfully and demonstrated the failure. It uses the repository's Evidence fixture constructors, then the actual manifest projection, replay, and comparison code. ### Expected behavior Replay and comparison should preserve all query values needed to identify the recorded request, or report that the request cannot be represented. They should not silently drop a value, select a different recording, and count the result as an ordinary response disagreement. The [documented comparison contract](https://github.com/morluto/rea/blob/8c4f96ef4759fd78da66c4b9726825ca71d7ee0c/docs/agent-environments.md#measure-fidelity-against-the-application) says the comparator answers the probe's calls as the served environment would and reports ignored query names. ### Impact and scope Observed for an ordinary repeated `tag` filter. Different requests become the same tool arguments, making recorded multi-value queries impossible to distinguish and producing a false fidelity failure. The same selection may falsely claim an exact replay. Other query-name patterns and real application captures were not tested. ### REA version Source revision `8c4f96ef4759fd78da66c4b9726825ca71d7ee0c` (`package.json` reports 6.7.0). This is a current-source report; the published 6.7.0 package was not used for this reproduction. ### Runtime and platform Node.js v26.12.1; Linux x86_64. ### Target and artifact context Synthetic HAR Evidence containing two GET requests with distinct second `tag` values and JSON responses. ### Interface Other — production functions shared by the environment CLI workflow. ### Earliest failure stage Request parsing or validation ### Relevant output or evidence The probe request is `?tag=a&tag=c`, but the observed values are: ```json {"args":{"tag":"a"},"replay":{"body":{"selected":"b"},"replay":{"record_ordinal":0,"exact_match":true,"recorded_query":"?tag=a&tag=b"}},"step":{"equal":false,"differences":[{"category":"value","pointer":"/selected","live":"\"c\"","env":"\"b\""}],"ignored_query":[]}} ``` ### Typed outcome or recovery expected Preserve the complete selected request, or explicitly mark/refuse the unrepresentable query instead of silently discarding its second value. ### Analysis or suspected cause Source-derived: [`matchProbeRequest`](https://github.com/morluto/rea/blob/8c4f96ef4759fd78da66c4b9726825ca71d7ee0c/src/application/environment/environmentCompareMatch.ts#L166-L174) writes a query argument only if it is absent, silently skipping subsequent occurrences. [`scoreExchange`](https://github.com/morluto/rea/blob/8c4f96ef4759fd78da66c4b9726825ca71d7ee0c/src/application/environment/environmentReplay.ts#L85-L104) then counts any matching value for that name as an exact query match; both recordings tie and the first wins. ### Additional context No fix is included. No real-provider behavior or published-release regression is claimed. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: A valid repeated-query probe selects a different recorded response and produces a false fidelity disagreement in the shared production workflow. - Workaround status: unverified - Workaround basis: The generated tool exposes one scalar argument per query name; no workaround retaining both selected values was tested.

bsakaguchi closed this issue

2026-10-31 00:46:33 +00:00

bsakaguchi referenced this issue

2026-10-31 01:11:20 +00:00

[PR #4068] [MERGED] docs(objdump): show debug-link flag forms that keep their meaning #6031