morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #3386] [Bug] Omitted duplicate requests falsely report a network modification #1253

New issue

Closed

opened 2026-10-16 21:03:23 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-16 21:03:23 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 14, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3386

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

compare-web-captures reports a network modification when one capture contains two identical HTTP observations and the other retains only one while explicitly reporting unavailable request coverage. The retained request facts match; the missing duplicate alone changes the result.

The command exits with status 0 and returns overall_status: "changed". dimensions.network has status: "changed", total_changes: 1, one modified entry for a net_ identity, and reason: null. Comparing the earlier fixture with itself returns dimensions.network.status: "unknown" because its passive attach-window coverage is limited; it reports no modifications.

A separate executed production-provider check used a local synthetic CDP peer that emitted two identical HTTP request/response/completion sequences. In the later capture, one request event supplied an invalid request ID. REA retained the other request and correctly marked network_requests unavailable with an invalid_protocol_value exclusion. The public CLI still reported one modification when comparing those captures. This verifies a provider-produced omission in addition to the normalized JSON reproduction.

Reproduction

Use a clean checkout at the baseline and install the locked dependencies, then build the runtime. These setup commands prepare the runtime; the reported comparisons used the compiled runtime and locked dependencies.

git clone https://github.com/morluto/rea.git rea-network-comparison
cd rea-network-comparison
git checkout 8dde6370d0809ad04cda3342d4e6f290f48aa063
npm ci
npm run build:cached

From the repository root, create two normalized capture fixtures and compare them through the public CLI. The URLs are inert recorded identities; this command does not fetch them. No browser is required. The earlier fixture contains two identical requests; the later fixture retains one and explicitly records the other as unavailable.

REA_ENTRY="$PWD/scripts/rea.mjs"
mkdir capture-comparison-repro
cd capture-comparison-repro
mkdir isolated-home
node --input-type=module <<'JS'
import { writeFileSync } from 'node:fs';
const origin = 'https://example.test';
const url = origin + '/page';
const time = '2026-01-01T00:00:00.000Z';
const completeness = () => ({
  status: 'attach_limited', conditions: ['attach_limited'],
  policy_filtered_sections: [],
  attach_limited_sections: ['network_requests', 'console_events', 'websocket_connections', 'websocket_frames'],
  truncated_sections: [], unavailable_sections: [], excluded: [],
  dropped_events: Object.fromEntries(['scripts', 'network_requests', 'console_events', 'websocket_connections', 'websocket_frames', 'webmcp_tools', 'timeline_events', 'total'].map(key => [key, 0]))
});
const node = (id, parent, role, name) => ({node_id: id, parent_id: parent, role, name, description: null, ignored: false, states: []});
const inspection = () => ({
  browser: {product: 'Fixture/1', protocol_version: '1.3', revision: 'fixture', user_agent: 'Fixture', js_version: '1'},
  target: {target_id: 'page', type: 'page', title: 'Page', url, origin, attached: false},
  capture_window: {started_at: time, ended_at: time, observation_ms: 0},
  completeness: completeness(),
  frames: [{frame_id: 'main', parent_frame_id: null, url, origin}],
  dom: {total_nodes: 0, nodes: []},
  accessibility: {total_nodes: 2, text_capture: {status: 'included', retained_bytes: 10, excluded_fields: 0}, nodes: [node('root', null, 'RootWebArea', 'Page'), node('button', 'root', 'button', 'Submit')]},
  scripts: {total: 0, items: []}, resources: [],
  network: {requests: [], websocket_connections: [], coverage_started_at: time, prior_activity_available: false},
  console: {events: [], coverage_started_at: time, prior_activity_available: false}, workers: [],
  metadata: {responses: [], dom_urls: [], agent_hints: [], excluded_dom_urls: 0, headers_allowlisted: true},
  storage: {origin, usage_bytes: 0, quota_bytes: 1024, local_storage_keys: [], session_storage_keys: [], indexed_db_names: [], cache_names: [], content_fingerprints: [], fingerprint_algorithm: 'sha256', fingerprints_complete: true, values_redacted: true},
  limitations: []
});
const save = (name, value) => writeFileSync(name, JSON.stringify(value));
const request = (request_id) => ({
  request_id, url: origin + '/api', origin, method: 'GET', resource_type: 'Fetch',
  status: 200, mime_type: 'application/json', encoded_data_length: 10, redirects: [],
  initiator: {type: 'script', url: origin + '/app.js', line: 0, column: 0},
  body_shapes: {status: 'not_approved', request: null, response: null}
});
const before = {inspection: inspection()}, after = {inspection: inspection()};
before.inspection.network.requests = [request('one'), request('two')];
after.inspection.network.requests = [request('one')];
after.inspection.completeness.unavailable_sections.push('network_requests');
after.inspection.completeness.excluded.push({section: 'network_requests', reason: 'invalid_protocol_value', count: 1});
save('before.json', before); save('after.json', after);
JS
env HOME="$PWD/isolated-home" node "$REA_ENTRY" compare-web-captures before.json after.json --format json --full-output

Run the self-comparison control with before.json as both inputs, from the same fixture directory.

env HOME="$PWD/isolated-home" node "$REA_ENTRY" compare-web-captures before.json before.json --format json --full-output

Observe the pair's false network modification and overall changed claim. The self-comparison reports network unknown with no modifications because passive attach-window coverage is limited.

Expected behavior

The omitted request should remain unknown rather than establish a network modification. Two copies versus one retained copy do not prove different network activity when the later inventory explicitly reports an omission. Independently comparable response facts should remain eligible to prove a real difference.

docs/tool-design.md:57 says missing coverage remains unknown rather than absence. src/domain/webCaptureDiff.ts:41 promises that a changed status proves an observed difference, while src/domain/webCaptureDiff.ts:42 limits network comparison to observed attachment-window activity.

Impact and scope

P2, supported. A missing duplicate observation produces a false positive network-change result and an overall changed claim. The demonstrated failure concerns incomplete passive captures and does not corrupt the stored evidence.

Workaround status is available. Callers can inspect request completeness and the matching retained request facts before accepting an aggregate modification. Both the unavailable marker and identical retained facts are present in the reproduced inputs; the workaround requires manual review of the captures.

REA version

6.4.0 (main at 8dde6370d0)

Runtime and platform

Node.js v26.10.0 and macOS 27.0.1, with the compiled runtime and locked dependencies. Architecture was not recorded for this reproduction. The CLI and a local synthetic CDP peer were exercised; these observations do not establish a particular real Chrome version's occurrence frequency.

Target and artifact context

Two local normalized passive page-capture JSON files containing an inert https://example.test/page target and GET https://example.test/api HTTP observations. The requests have identical known status 200, MIME type application/json, byte count 10, initiator facts, and empty redirect lists; their capture-specific request IDs differ. The later inventory records one invalid_protocol_value exclusion and unavailable network_requests coverage. Additional production-provider verification used a local synthetic CDP peer with two request/response/completion sequences and one invalid request ID in the later capture. No browser is required for the inline reproduction.

Interface

CLI

Earliest failure stage

Transport or result projection

Relevant output or evidence

Observed CLI result summary:
before.json vs after.json: exit status 0
overall_status: "changed"
dimensions.network.status: "changed"
dimensions.network.total_changes: 1
changes: one "modified" entry for a net_ identity
reason: null

before.json vs before.json:
dimensions.network.status: "unknown"
no modifications; passive attach-window coverage is limited

Production-provider check:
one request retained; network_requests unavailable
invalid_protocol_value exclusion recorded
public CLI comparison still reports one modification

Typed outcome or recovery expected

Rerun the pair above. Network should remain unknown, with no modification inferred solely from the unavailable duplicate. A self-comparison should retain no changes. Also compare matching observed requests with different known response statuses on both sides; a supported response difference must continue to report changed rather than being hidden by a blanket incomplete-coverage rule. The response-status comparison is a restoration check, not a reported executed control for this reproduction.

Analysis or suspected cause

Source-derived reasoning: src/domain/webCaptureDiff.ts:372 hashes the whole grouped request collection, including multiplicity; src/domain/webCaptureDiff.ts:332 determines incomplete inventory coverage, but src/domain/webCaptureDiff.ts:77 still retains the aggregate modification. The executed CLI reproduction marks an unavailable same-identity request collection modified solely because one duplicate was omitted. The production request guard is src/browser/CdpCaptureEventHandlers.ts:215, with unavailable coverage recorded at src/browser/CdpCaptureCompleteness.ts:64.

Additional context

Related open issue #3172 concerns unavailable redirect-hop response fields within retained requests; this report has complete identical retained response fields and no redirects, and concerns omitted request multiplicity.

Related completed issue #1830 concerns unavailable final-response byte counts; this report keeps those fields known and identical while request inventory coverage is unavailable.

Related completed issue #2609 concerns omitted proven additions/removals from complete request inventories when selected body shapes are unavailable; this report concerns a false grouped modification when request inventory itself is unavailable.

Related open issue #3173 concerns lost network-content recovery guidance alongside valid console differences in scenario captures; this report concerns a false passive HTTP network modification.

Related completed issue #2169 concerns false scenario equality from mismatched event-family coverage; this report concerns passive HTTP request multiplicity and false change.

Originally created by @coygeek on GitHub (Oct 14, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3386 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior `compare-web-captures` reports a network modification when one capture contains two identical HTTP observations and the other retains only one while explicitly reporting unavailable request coverage. The retained request facts match; the missing duplicate alone changes the result. The command exits with status 0 and returns `overall_status: "changed"`. `dimensions.network` has `status: "changed"`, `total_changes: 1`, one `modified` entry for a `net_` identity, and `reason: null`. Comparing the earlier fixture with itself returns `dimensions.network.status: "unknown"` because its passive attach-window coverage is limited; it reports no modifications. A separate executed production-provider check used a local synthetic CDP peer that emitted two identical HTTP request/response/completion sequences. In the later capture, one request event supplied an invalid request ID. REA retained the other request and correctly marked `network_requests` unavailable with an `invalid_protocol_value` exclusion. The public CLI still reported one modification when comparing those captures. This verifies a provider-produced omission in addition to the normalized JSON reproduction. ### Reproduction 1. Use a clean checkout at the baseline and install the locked dependencies, then build the runtime. These setup commands prepare the runtime; the reported comparisons used the compiled runtime and locked dependencies. ```sh git clone https://github.com/morluto/rea.git rea-network-comparison cd rea-network-comparison git checkout 8dde6370d0809ad04cda3342d4e6f290f48aa063 npm ci npm run build:cached ``` 2. From the repository root, create two normalized capture fixtures and compare them through the public CLI. The URLs are inert recorded identities; this command does not fetch them. No browser is required. The earlier fixture contains two identical requests; the later fixture retains one and explicitly records the other as unavailable. ```sh REA_ENTRY="$PWD/scripts/rea.mjs" mkdir capture-comparison-repro cd capture-comparison-repro mkdir isolated-home node --input-type=module <<'JS' import { writeFileSync } from 'node:fs'; const origin = 'https://example.test'; const url = origin + '/page'; const time = '2026-01-01T00:00:00.000Z'; const completeness = () => ({ status: 'attach_limited', conditions: ['attach_limited'], policy_filtered_sections: [], attach_limited_sections: ['network_requests', 'console_events', 'websocket_connections', 'websocket_frames'], truncated_sections: [], unavailable_sections: [], excluded: [], dropped_events: Object.fromEntries(['scripts', 'network_requests', 'console_events', 'websocket_connections', 'websocket_frames', 'webmcp_tools', 'timeline_events', 'total'].map(key => [key, 0])) }); const node = (id, parent, role, name) => ({node_id: id, parent_id: parent, role, name, description: null, ignored: false, states: []}); const inspection = () => ({ browser: {product: 'Fixture/1', protocol_version: '1.3', revision: 'fixture', user_agent: 'Fixture', js_version: '1'}, target: {target_id: 'page', type: 'page', title: 'Page', url, origin, attached: false}, capture_window: {started_at: time, ended_at: time, observation_ms: 0}, completeness: completeness(), frames: [{frame_id: 'main', parent_frame_id: null, url, origin}], dom: {total_nodes: 0, nodes: []}, accessibility: {total_nodes: 2, text_capture: {status: 'included', retained_bytes: 10, excluded_fields: 0}, nodes: [node('root', null, 'RootWebArea', 'Page'), node('button', 'root', 'button', 'Submit')]}, scripts: {total: 0, items: []}, resources: [], network: {requests: [], websocket_connections: [], coverage_started_at: time, prior_activity_available: false}, console: {events: [], coverage_started_at: time, prior_activity_available: false}, workers: [], metadata: {responses: [], dom_urls: [], agent_hints: [], excluded_dom_urls: 0, headers_allowlisted: true}, storage: {origin, usage_bytes: 0, quota_bytes: 1024, local_storage_keys: [], session_storage_keys: [], indexed_db_names: [], cache_names: [], content_fingerprints: [], fingerprint_algorithm: 'sha256', fingerprints_complete: true, values_redacted: true}, limitations: [] }); const save = (name, value) => writeFileSync(name, JSON.stringify(value)); const request = (request_id) => ({ request_id, url: origin + '/api', origin, method: 'GET', resource_type: 'Fetch', status: 200, mime_type: 'application/json', encoded_data_length: 10, redirects: [], initiator: {type: 'script', url: origin + '/app.js', line: 0, column: 0}, body_shapes: {status: 'not_approved', request: null, response: null} }); const before = {inspection: inspection()}, after = {inspection: inspection()}; before.inspection.network.requests = [request('one'), request('two')]; after.inspection.network.requests = [request('one')]; after.inspection.completeness.unavailable_sections.push('network_requests'); after.inspection.completeness.excluded.push({section: 'network_requests', reason: 'invalid_protocol_value', count: 1}); save('before.json', before); save('after.json', after); JS env HOME="$PWD/isolated-home" node "$REA_ENTRY" compare-web-captures before.json after.json --format json --full-output ``` 3. Run the self-comparison control with `before.json` as both inputs, from the same fixture directory. ```sh env HOME="$PWD/isolated-home" node "$REA_ENTRY" compare-web-captures before.json before.json --format json --full-output ``` 4. Observe the pair's false network modification and overall changed claim. The self-comparison reports network unknown with no modifications because passive attach-window coverage is limited. ### Expected behavior The omitted request should remain unknown rather than establish a network modification. Two copies versus one retained copy do not prove different network activity when the later inventory explicitly reports an omission. Independently comparable response facts should remain eligible to prove a real difference. `docs/tool-design.md:57` says missing coverage remains unknown rather than absence. `src/domain/webCaptureDiff.ts:41` promises that a changed status proves an observed difference, while `src/domain/webCaptureDiff.ts:42` limits network comparison to observed attachment-window activity. ### Impact and scope P2, supported. A missing duplicate observation produces a false positive network-change result and an overall changed claim. The demonstrated failure concerns incomplete passive captures and does not corrupt the stored evidence. Workaround status is available. Callers can inspect request completeness and the matching retained request facts before accepting an aggregate modification. Both the unavailable marker and identical retained facts are present in the reproduced inputs; the workaround requires manual review of the captures. ### REA version 6.4.0 (main at 8dde6370d0809ad04cda3342d4e6f290f48aa063) ### Runtime and platform Node.js v26.10.0 and macOS 27.0.1, with the compiled runtime and locked dependencies. Architecture was not recorded for this reproduction. The CLI and a local synthetic CDP peer were exercised; these observations do not establish a particular real Chrome version's occurrence frequency. ### Target and artifact context Two local normalized passive page-capture JSON files containing an inert `https://example.test/page` target and `GET https://example.test/api` HTTP observations. The requests have identical known status 200, MIME type `application/json`, byte count 10, initiator facts, and empty redirect lists; their capture-specific request IDs differ. The later inventory records one `invalid_protocol_value` exclusion and unavailable `network_requests` coverage. Additional production-provider verification used a local synthetic CDP peer with two request/response/completion sequences and one invalid request ID in the later capture. No browser is required for the inline reproduction. ### Interface CLI ### Earliest failure stage Transport or result projection ### Relevant output or evidence ```text Observed CLI result summary: before.json vs after.json: exit status 0 overall_status: "changed" dimensions.network.status: "changed" dimensions.network.total_changes: 1 changes: one "modified" entry for a net_ identity reason: null before.json vs before.json: dimensions.network.status: "unknown" no modifications; passive attach-window coverage is limited Production-provider check: one request retained; network_requests unavailable invalid_protocol_value exclusion recorded public CLI comparison still reports one modification ``` ### Typed outcome or recovery expected Rerun the pair above. Network should remain `unknown`, with no modification inferred solely from the unavailable duplicate. A self-comparison should retain no changes. Also compare matching observed requests with different known response statuses on both sides; a supported response difference must continue to report changed rather than being hidden by a blanket incomplete-coverage rule. The response-status comparison is a restoration check, not a reported executed control for this reproduction. ### Analysis or suspected cause Source-derived reasoning: `src/domain/webCaptureDiff.ts:372` hashes the whole grouped request collection, including multiplicity; `src/domain/webCaptureDiff.ts:332` determines incomplete inventory coverage, but `src/domain/webCaptureDiff.ts:77` still retains the aggregate modification. The executed CLI reproduction marks an unavailable same-identity request collection modified solely because one duplicate was omitted. The production request guard is `src/browser/CdpCaptureEventHandlers.ts:215`, with unavailable coverage recorded at `src/browser/CdpCaptureCompleteness.ts:64`. ### Additional context Related open issue [#3172](https://github.com/morluto/rea/issues/3172) concerns unavailable redirect-hop response fields within retained requests; this report has complete identical retained response fields and no redirects, and concerns omitted request multiplicity. Related completed issue [#1830](https://github.com/morluto/rea/issues/1830) concerns unavailable final-response byte counts; this report keeps those fields known and identical while request inventory coverage is unavailable. Related completed issue [#2609](https://github.com/morluto/rea/issues/2609) concerns omitted proven additions/removals from complete request inventories when selected body shapes are unavailable; this report concerns a false grouped modification when request inventory itself is unavailable. Related open issue [#3173](https://github.com/morluto/rea/issues/3173) concerns lost network-content recovery guidance alongside valid console differences in scenario captures; this report concerns a false passive HTTP network modification. Related completed issue [#2169](https://github.com/morluto/rea/issues/2169) concerns false scenario equality from mismatched event-family coverage; this report concerns passive HTTP request multiplicity and false change.

bsakaguchi commented

2026-10-16 21:03:24 +00:00

Author

Owner

@kaoru0822-kitauji commented on GitHub (Oct 15, 2026):

Re-ran the normalized-fixture reproduction from this report through the public CLI (compare-web-captures before.json after.json --format json --full-output) on main at cf0a38ad9:

Pair (two identical requests vs. one retained, with network_requests unavailable and one invalid_protocol_value exclusion): overall_status: "unknown", dimensions.network.status: "unknown", total_changes: 0.
Self-comparison control: the same unknown result with no changes.

This appears resolved by e79e014f2 ("fix(browser): preserve passive network inventory uncertainty"), which keeps sample multiplicity only under full request-inventory coverage (src/domain/webCaptureDiff.ts, networkMap). Not re-verified: the synthetic CDP-peer variant from the report.

@kaoru0822-kitauji commented on GitHub (Oct 15, 2026): Re-ran the normalized-fixture reproduction from this report through the public CLI (`compare-web-captures before.json after.json --format json --full-output`) on main at cf0a38ad9: - Pair (two identical requests vs. one retained, with `network_requests` unavailable and one `invalid_protocol_value` exclusion): `overall_status: "unknown"`, `dimensions.network.status: "unknown"`, `total_changes: 0`. - Self-comparison control: the same `unknown` result with no changes. This appears resolved by e79e014f2 ("fix(browser): preserve passive network inventory uncertainty"), which keeps sample multiplicity only under full request-inventory coverage (`src/domain/webCaptureDiff.ts`, `networkMap`). Not re-verified: the synthetic CDP-peer variant from the report.

bsakaguchi referenced this issue

2026-10-17 21:12:35 +00:00

[PR #1583] [MERGED] feat(process): bound a finalization interval before forced capture timeout #2988

bsakaguchi closed this issue

2026-10-31 00:30:17 +00:00

bsakaguchi commented

2026-10-31 00:30:17 +00:00

Author

Owner

@coygeek commented on GitHub (Oct 18, 2026):

Addressed on main by github.com/morluto/rea@e79e014f23. Passive network comparison now discards duplicate sample multiplicity when request-inventory coverage is incomplete, while retaining observed response-content comparisons. Regression tests explicitly require unknown with no modification for repeated-request count differences under unavailable/dropped/truncated inventory.

I checked the current source and relevant regression coverage; I have not rerun the original runtime reproduction in this review. Closing as completed.

@coygeek commented on GitHub (Oct 18, 2026): Addressed on main by https://github.com/morluto/rea/commit/e79e014f23cf5b5d59aa5e5681897dbcad73b44e. Passive network comparison now discards duplicate sample multiplicity when request-inventory coverage is incomplete, while retaining observed response-content comparisons. Regression tests explicitly require unknown with no modification for repeated-request count differences under unavailable/dropped/truncated inventory. I checked the current source and relevant regression coverage; I have not rerun the original runtime reproduction in this review. Closing as completed.

bsakaguchi referenced this issue

2026-10-31 00:42:35 +00:00

[GH-ISSUE #5966] [Bug] Late process scenario events lose their indexes in validation issues #3636

bsakaguchi referenced this issue

2026-10-31 00:44:43 +00:00

[GH-ISSUE #6445] [Bug] capture-process records exit code zero for a signal-terminated target #3861