[GH-ISSUE #3172] [Bug] Passive capture comparison treats missing redirect response fields as known evidence #1108

Closed
opened 2026-10-15 18:31:08 +00:00 by bsakaguchi · 2 comments
Owner

Originally created by @coygeek on GitHub (Oct 14, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3172

Before submitting

  • I searched the open and closed issues for a related report.
  • I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

compare-web-captures reports a network modification when otherwise identical passive page captures differ only because a redirect hop's status, mime_type, or encoded_data_length is unavailable on one side. Reversing the inputs gives the same false modification. Comparing two captures with the same unavailable hop field instead reports the network dimension unchanged. Both results give callers more certainty about redirect behavior than the observations support.

The independent verification ran 15 offline CLI comparisons at the revision below, and all exited 0. Each missing-versus-known hop-field comparison returned normalized_result.dimensions.network.status: "changed", total_changes: 1, one modified change, and reason: null. Each missing-versus-missing hop-field comparison returned network status: "unchanged", zero changes, and reason: null. Equivalent unavailable fields on the final response correctly returned network unknown with zero changes and an incomplete-response-evidence reason. A known redirect status change from 302 to 307 correctly returned changed.

A focused localhost CDP fake also reproduced the producer-to-comparison path through real inspect-web-page and compare-web-captures CLI calls. Independently omitting status, MIME type, or byte length from the simulated redirect response produced a retained hop null and the same false modification. Missing byte length additionally marked network_requests unavailable; that marker did not prevent the false modification. Missing status and MIME type did not produce that unavailable-section marker. This verifies simulated protocol omissions, not their frequency in real Chrome.

Reproduction

Use a clean checkout at 58300885207133422af235e0f2b10658cf958b03. The comparison commands below were executed against its prepared runtime. Dependency installation and the build are setup instructions and were not rerun during verification.

  1. Install the locked dependencies and build the runtime using the repository's pinned development toolchain.
npm ci
npm run build:cached
  1. Isolate CLI state and create a directory for the local comparison inputs.
mkdir -p repro-home/.config repro-home/.cache repro-home/.local/share repro-data
export HOME="$PWD/repro-home"
export XDG_CONFIG_HOME="$HOME/.config"
export XDG_CACHE_HOME="$HOME/.cache"
export XDG_DATA_HOME="$HOME/.local/share"
  1. Create schema-validated passive snapshots inline. The URLs are observation strings; these commands do not request them. The known snapshot has one final response and one redirect hop. The variants change only the named response field, except the explicitly marked incomplete-byte variant.
cat > make-fixtures.mjs <<'JS'
import { writeFileSync } from 'node:fs';
import { resolve } from 'node:path';
import { pathToFileURL } from 'node:url';
const { captureSnapshotSchema } = await import(pathToFileURL(resolve('dist/domain/webCaptureDiffSchemas.js')));
const origin = 'https://capture.test';
const time = '2026-01-01T00:00:00.000Z';
const capture = captureSnapshotSchema.parse({ inspection: {
  browser: { product: 'Chrome/154.0.0.0', protocol_version: '1.3', revision: 'fixture', user_agent: 'fixture', js_version: '15.4' },
  target: { target_id: 'page', type: 'page', title: 'Fixture', url: origin + '/', origin, attached: true },
  capture_window: { started_at: time, ended_at: time, observation_ms: 0 },
  completeness: {
    status: 'complete_within_window', conditions: ['complete_within_window'],
    policy_filtered_sections: [], attach_limited_sections: [], truncated_sections: [], unavailable_sections: [], excluded: [],
    dropped_events: { scripts: 0, network_requests: 0, console_events: 0, websocket_connections: 0, websocket_frames: 0, webmcp_tools: 0, timeline_events: 0, total: 0 }
  },
  frames: [], dom: { total_nodes: 0, nodes: [] },
  accessibility: { total_nodes: 0, nodes: [], text_capture: { status: 'included', retained_bytes: 0, excluded_fields: 0 } },
  scripts: { total: 0, items: [] }, resources: [],
  network: { coverage_started_at: time, prior_activity_available: false, websocket_connections: [], requests: [{
    request_id: 'request', url: origin + '/final', origin, method: 'GET', resource_type: 'Fetch',
    status: 200, mime_type: 'application/json', encoded_data_length: 20,
    redirects: [{ url: origin + '/start', response_url: origin + '/start', method: 'GET', resource_type: 'Fetch', status: 302, mime_type: 'text/plain', encoded_data_length: 12, request_timestamp: 1, redirect_event_timestamp: 2 }],
    initiator: { type: 'other', url: null, line: null, column: null },
    body_shapes: { status: 'not_approved', request: null, response: null }
  }] },
  console: { events: [], coverage_started_at: time, prior_activity_available: false }, workers: [],
  metadata: { responses: [], dom_urls: [], agent_hints: [], excluded_dom_urls: 0, headers_allowlisted: true },
  storage: { origin, usage_bytes: 0, quota_bytes: 1000, local_storage_keys: [], session_storage_keys: [], indexed_db_names: [], cache_names: [], content_fingerprints: [], fingerprints_complete: true, values_redacted: true },
  limitations: []
}, webmcp: null });
const save = (name, value) => writeFileSync(resolve(process.argv[2] ?? '.', name + '.json'), JSON.stringify(captureSnapshotSchema.parse(value)));
save('known', capture);
for (const field of ['status', 'mime_type', 'encoded_data_length']) {
  for (const level of ['hop', 'request']) {
    const next = structuredClone(capture);
    const request = next.inspection.network.requests[0];
    (level === 'hop' ? request.redirects[0] : request)[field] = null;
    save(`${level}-${field}-missing`, next);
  }
}
const changed = structuredClone(capture);
changed.inspection.network.requests[0].redirects[0].status = 307;
save('status-changed', changed);
const incomplete = structuredClone(capture);
incomplete.inspection.network.requests[0].redirects[0].encoded_data_length = null;
incomplete.inspection.completeness.unavailable_sections = ['network_requests'];
save('hop-bytes-marked-incomplete', incomplete);
JS
node make-fixtures.mjs repro-data
  1. Run a missing-hop-status comparison and its controls.
node scripts/rea.mjs compare-web-captures repro-data/hop-status-missing.json repro-data/known.json --format json
node scripts/rea.mjs compare-web-captures repro-data/hop-status-missing.json repro-data/hop-status-missing.json --format json
node scripts/rea.mjs compare-web-captures repro-data/request-status-missing.json repro-data/known.json --format json
node scripts/rea.mjs compare-web-captures repro-data/known.json repro-data/known.json --format json
node scripts/rea.mjs compare-web-captures repro-data/known.json repro-data/status-changed.json --format json
  1. Run the other missing-hop-field variants, including one with incomplete network coverage recorded.
node scripts/rea.mjs compare-web-captures repro-data/hop-mime_type-missing.json repro-data/known.json --format json
node scripts/rea.mjs compare-web-captures repro-data/hop-encoded_data_length-missing.json repro-data/known.json --format json
node scripts/rea.mjs compare-web-captures repro-data/hop-bytes-marked-incomplete.json repro-data/known.json --format json

Expected behavior

docs/browser-observation.md:22 states that passive comparison does not treat missing or truncated evidence as equivalence. The returned comparison limitation defined at src/domain/webCaptureDiff.ts:40 promises that changed proves an observed difference. The root AGENTS.md, Architecture and Cleanup, also requires preserving omission and uncertainty across boundaries.

When no other difference is proven, a hop response field that is null on either side must remain unknown. The network dimension should return status: "unknown", zero changes, and a reason identifying incomplete response evidence for null-versus-known and null-versus-null observations. Apply this rule to status, MIME type, and byte count while preserving genuine observed differences. Complete identical hops must still compare unchanged, and known status 302 versus 307 must still compare changed.

Impact and scope

P2. This bounded passive-capture comparison defect can mislead an analyst into treating unavailable redirect observations as a proven modification or as equivalence. The verifier observed no data mutation, security boundary crossing, or broader workflow failure. The inputs are schema-valid, and the simulated CDP producer generated the same nullable fields through the public inspection path.

Raw capture inspection exposes the null values, but a complete comparison workflow that preserves the required redirect result was not demonstrated. CLI help provides no passive comparison option to exclude unavailable hop response fields. Deleting redirects from the input would discard relevant evidence. Real Chrome omission frequency was not measured.

REA version

6.4.0 (main at 5830088520)

Runtime and platform

Node.js v26.10.0 on macOS. The verifier did not record the OS version or architecture. Locked dependencies and the built runtime were already prepared; dependency installation and build were not rerun during verification.

Target and artifact context

Local normalized passive page-capture JSON containing one final HTTP response and one ordered redirect hop. Every inline fixture passes captureSnapshotSchema.parse. The example URLs are observation strings and are never requested by the offline reproduction. Additional producer verification used a localhost CDP fake with simulated redirect events.

Interface

CLI

Earliest failure stage

Transport or result projection

Relevant output or evidence

network dimension summary; all comparison commands exited 0:
hop status null vs 302: changed, total_changes=1, change=modified, reason=null
hop status null vs null: unchanged, total_changes=0, reason=null
hop MIME null vs text/plain: changed, total_changes=1, change=modified, reason=null
hop bytes null vs 12: changed, total_changes=1, change=modified, reason=null
hop bytes null with network_requests unavailable vs 12: changed, total_changes=1
final-response status null vs 200: unknown, total_changes=0
  reason: HTTP response status, MIME type, or byte-count evidence is incomplete.
complete identical hops: unchanged, total_changes=0
known hop status 302 vs 307: changed, total_changes=1

The network dimension is the relevant result. Unrecorded WebMCP discovery independently makes the overall result unknown for otherwise identical fixtures, including the complete identical-hop control. Missing-versus-known hop fields still make the overall result changed.

Typed outcome or recovery expected

Return network unknown with zero proven changes and an explanatory reason when unavailable redirect response evidence is the only discrepancy. Preserve that uncertainty for both comparison directions and missing-versus-missing fields, including byte-length omissions with incomplete network coverage marked. Keep any separately proven change visible even if another hop field is unavailable. Complete identical observations must remain unchanged, and known non-null redirect response differences must remain changed.

Analysis or suspected cause

Source-derived at baseline 58300885207133422af235e0f2b10658cf958b03: src/domain/webCaptureDiff.ts:257-271 derives jointly available response fields from final requests only. src/domain/webCaptureDiff.ts:366-373 derives response incompleteness from that field set. src/domain/webCaptureDiff.ts:391-397 includes redirect response objects in the comparison digest, excluding only the two event timestamps. The source therefore does not apply the final-response availability handling to redirect hops. This explanation agrees with the independently executed comparisons.

src/domain/browserObservationSchemas.ts:135-149 accepts nullable redirect response fields. src/browser/CdpCaptureEventHandlers.ts:282-301 retains missing status, MIME type, and byte length as null, and records incomplete network coverage for unavailable or invalid byte length. tests/boundary/browser/webCaptureDiff.test.ts:731-768 requires unavailable final-response status and MIME evidence to remain unknown, including null-versus-null observations.

Additional context

Related to #1830, which fixed unavailable final-response metadata; this report concerns redirect-hop metadata that still produces false changes and false equivalence after that fix. The P2 rating reflects incorrect certainty for bounded comparison inputs without demonstrated broader harm.

Related: #3173 reports a separate failure in the same area; the two have independent fixes.

Originally created by @coygeek on GitHub (Oct 14, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3172 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior `compare-web-captures` reports a network modification when otherwise identical passive page captures differ only because a redirect hop's `status`, `mime_type`, or `encoded_data_length` is unavailable on one side. Reversing the inputs gives the same false modification. Comparing two captures with the same unavailable hop field instead reports the network dimension unchanged. Both results give callers more certainty about redirect behavior than the observations support. The independent verification ran 15 offline CLI comparisons at the revision below, and all exited 0. Each missing-versus-known hop-field comparison returned `normalized_result.dimensions.network.status: "changed"`, `total_changes: 1`, one `modified` change, and `reason: null`. Each missing-versus-missing hop-field comparison returned network `status: "unchanged"`, zero changes, and `reason: null`. Equivalent unavailable fields on the final response correctly returned network unknown with zero changes and an incomplete-response-evidence reason. A known redirect status change from 302 to 307 correctly returned changed. A focused localhost CDP fake also reproduced the producer-to-comparison path through real `inspect-web-page` and `compare-web-captures` CLI calls. Independently omitting status, MIME type, or byte length from the simulated redirect response produced a retained hop null and the same false modification. Missing byte length additionally marked `network_requests` unavailable; that marker did not prevent the false modification. Missing status and MIME type did not produce that unavailable-section marker. This verifies simulated protocol omissions, not their frequency in real Chrome. ### Reproduction Use a clean checkout at `58300885207133422af235e0f2b10658cf958b03`. The comparison commands below were executed against its prepared runtime. Dependency installation and the build are setup instructions and were not rerun during verification. 1. Install the locked dependencies and build the runtime using the repository's pinned development toolchain. ```sh npm ci npm run build:cached ``` 2. Isolate CLI state and create a directory for the local comparison inputs. ```sh mkdir -p repro-home/.config repro-home/.cache repro-home/.local/share repro-data export HOME="$PWD/repro-home" export XDG_CONFIG_HOME="$HOME/.config" export XDG_CACHE_HOME="$HOME/.cache" export XDG_DATA_HOME="$HOME/.local/share" ``` 3. Create schema-validated passive snapshots inline. The URLs are observation strings; these commands do not request them. The known snapshot has one final response and one redirect hop. The variants change only the named response field, except the explicitly marked incomplete-byte variant. ```sh cat > make-fixtures.mjs <<'JS' import { writeFileSync } from 'node:fs'; import { resolve } from 'node:path'; import { pathToFileURL } from 'node:url'; const { captureSnapshotSchema } = await import(pathToFileURL(resolve('dist/domain/webCaptureDiffSchemas.js'))); const origin = 'https://capture.test'; const time = '2026-01-01T00:00:00.000Z'; const capture = captureSnapshotSchema.parse({ inspection: { browser: { product: 'Chrome/154.0.0.0', protocol_version: '1.3', revision: 'fixture', user_agent: 'fixture', js_version: '15.4' }, target: { target_id: 'page', type: 'page', title: 'Fixture', url: origin + '/', origin, attached: true }, capture_window: { started_at: time, ended_at: time, observation_ms: 0 }, completeness: { status: 'complete_within_window', conditions: ['complete_within_window'], policy_filtered_sections: [], attach_limited_sections: [], truncated_sections: [], unavailable_sections: [], excluded: [], dropped_events: { scripts: 0, network_requests: 0, console_events: 0, websocket_connections: 0, websocket_frames: 0, webmcp_tools: 0, timeline_events: 0, total: 0 } }, frames: [], dom: { total_nodes: 0, nodes: [] }, accessibility: { total_nodes: 0, nodes: [], text_capture: { status: 'included', retained_bytes: 0, excluded_fields: 0 } }, scripts: { total: 0, items: [] }, resources: [], network: { coverage_started_at: time, prior_activity_available: false, websocket_connections: [], requests: [{ request_id: 'request', url: origin + '/final', origin, method: 'GET', resource_type: 'Fetch', status: 200, mime_type: 'application/json', encoded_data_length: 20, redirects: [{ url: origin + '/start', response_url: origin + '/start', method: 'GET', resource_type: 'Fetch', status: 302, mime_type: 'text/plain', encoded_data_length: 12, request_timestamp: 1, redirect_event_timestamp: 2 }], initiator: { type: 'other', url: null, line: null, column: null }, body_shapes: { status: 'not_approved', request: null, response: null } }] }, console: { events: [], coverage_started_at: time, prior_activity_available: false }, workers: [], metadata: { responses: [], dom_urls: [], agent_hints: [], excluded_dom_urls: 0, headers_allowlisted: true }, storage: { origin, usage_bytes: 0, quota_bytes: 1000, local_storage_keys: [], session_storage_keys: [], indexed_db_names: [], cache_names: [], content_fingerprints: [], fingerprints_complete: true, values_redacted: true }, limitations: [] }, webmcp: null }); const save = (name, value) => writeFileSync(resolve(process.argv[2] ?? '.', name + '.json'), JSON.stringify(captureSnapshotSchema.parse(value))); save('known', capture); for (const field of ['status', 'mime_type', 'encoded_data_length']) { for (const level of ['hop', 'request']) { const next = structuredClone(capture); const request = next.inspection.network.requests[0]; (level === 'hop' ? request.redirects[0] : request)[field] = null; save(`${level}-${field}-missing`, next); } } const changed = structuredClone(capture); changed.inspection.network.requests[0].redirects[0].status = 307; save('status-changed', changed); const incomplete = structuredClone(capture); incomplete.inspection.network.requests[0].redirects[0].encoded_data_length = null; incomplete.inspection.completeness.unavailable_sections = ['network_requests']; save('hop-bytes-marked-incomplete', incomplete); JS node make-fixtures.mjs repro-data ``` 4. Run a missing-hop-status comparison and its controls. ```sh node scripts/rea.mjs compare-web-captures repro-data/hop-status-missing.json repro-data/known.json --format json node scripts/rea.mjs compare-web-captures repro-data/hop-status-missing.json repro-data/hop-status-missing.json --format json node scripts/rea.mjs compare-web-captures repro-data/request-status-missing.json repro-data/known.json --format json node scripts/rea.mjs compare-web-captures repro-data/known.json repro-data/known.json --format json node scripts/rea.mjs compare-web-captures repro-data/known.json repro-data/status-changed.json --format json ``` 5. Run the other missing-hop-field variants, including one with incomplete network coverage recorded. ```sh node scripts/rea.mjs compare-web-captures repro-data/hop-mime_type-missing.json repro-data/known.json --format json node scripts/rea.mjs compare-web-captures repro-data/hop-encoded_data_length-missing.json repro-data/known.json --format json node scripts/rea.mjs compare-web-captures repro-data/hop-bytes-marked-incomplete.json repro-data/known.json --format json ``` ### Expected behavior `docs/browser-observation.md:22` states that passive comparison does not treat missing or truncated evidence as equivalence. The returned comparison limitation defined at `src/domain/webCaptureDiff.ts:40` promises that changed proves an observed difference. The root `AGENTS.md`, Architecture and Cleanup, also requires preserving omission and uncertainty across boundaries. When no other difference is proven, a hop response field that is null on either side must remain unknown. The network dimension should return `status: "unknown"`, zero changes, and a reason identifying incomplete response evidence for null-versus-known and null-versus-null observations. Apply this rule to status, MIME type, and byte count while preserving genuine observed differences. Complete identical hops must still compare unchanged, and known status 302 versus 307 must still compare changed. ### Impact and scope P2. This bounded passive-capture comparison defect can mislead an analyst into treating unavailable redirect observations as a proven modification or as equivalence. The verifier observed no data mutation, security boundary crossing, or broader workflow failure. The inputs are schema-valid, and the simulated CDP producer generated the same nullable fields through the public inspection path. Raw capture inspection exposes the null values, but a complete comparison workflow that preserves the required redirect result was not demonstrated. CLI help provides no passive comparison option to exclude unavailable hop response fields. Deleting redirects from the input would discard relevant evidence. Real Chrome omission frequency was not measured. ### REA version 6.4.0 (main at 58300885207133422af235e0f2b10658cf958b03) ### Runtime and platform Node.js v26.10.0 on macOS. The verifier did not record the OS version or architecture. Locked dependencies and the built runtime were already prepared; dependency installation and build were not rerun during verification. ### Target and artifact context Local normalized passive page-capture JSON containing one final HTTP response and one ordered redirect hop. Every inline fixture passes `captureSnapshotSchema.parse`. The example URLs are observation strings and are never requested by the offline reproduction. Additional producer verification used a localhost CDP fake with simulated redirect events. ### Interface CLI ### Earliest failure stage Transport or result projection ### Relevant output or evidence ```text network dimension summary; all comparison commands exited 0: hop status null vs 302: changed, total_changes=1, change=modified, reason=null hop status null vs null: unchanged, total_changes=0, reason=null hop MIME null vs text/plain: changed, total_changes=1, change=modified, reason=null hop bytes null vs 12: changed, total_changes=1, change=modified, reason=null hop bytes null with network_requests unavailable vs 12: changed, total_changes=1 final-response status null vs 200: unknown, total_changes=0 reason: HTTP response status, MIME type, or byte-count evidence is incomplete. complete identical hops: unchanged, total_changes=0 known hop status 302 vs 307: changed, total_changes=1 ``` The network dimension is the relevant result. Unrecorded WebMCP discovery independently makes the overall result unknown for otherwise identical fixtures, including the complete identical-hop control. Missing-versus-known hop fields still make the overall result changed. ### Typed outcome or recovery expected Return network unknown with zero proven changes and an explanatory reason when unavailable redirect response evidence is the only discrepancy. Preserve that uncertainty for both comparison directions and missing-versus-missing fields, including byte-length omissions with incomplete network coverage marked. Keep any separately proven change visible even if another hop field is unavailable. Complete identical observations must remain unchanged, and known non-null redirect response differences must remain changed. ### Analysis or suspected cause Source-derived at baseline `58300885207133422af235e0f2b10658cf958b03`: `src/domain/webCaptureDiff.ts:257-271` derives jointly available response fields from final requests only. `src/domain/webCaptureDiff.ts:366-373` derives response incompleteness from that field set. `src/domain/webCaptureDiff.ts:391-397` includes redirect response objects in the comparison digest, excluding only the two event timestamps. The source therefore does not apply the final-response availability handling to redirect hops. This explanation agrees with the independently executed comparisons. `src/domain/browserObservationSchemas.ts:135-149` accepts nullable redirect response fields. `src/browser/CdpCaptureEventHandlers.ts:282-301` retains missing status, MIME type, and byte length as null, and records incomplete network coverage for unavailable or invalid byte length. `tests/boundary/browser/webCaptureDiff.test.ts:731-768` requires unavailable final-response status and MIME evidence to remain unknown, including null-versus-null observations. ### Additional context Related to #1830, which fixed unavailable final-response metadata; this report concerns redirect-hop metadata that still produces false changes and false equivalence after that fix. The P2 rating reflects incorrect certainty for bounded comparison inputs without demonstrated broader harm. Related: #3173 reports a separate failure in the same area; the two have independent fixes.
bsakaguchi 2026-10-15 18:31:08 +00:00
  • closed this issue
  • added the
    bug
    label
Author
Owner

@CassieFox23 commented on GitHub (Oct 14, 2026):

Heads up: this looks fixed on main in 2b60e097f. The change is in src/domain/webCaptureDiff.ts. It is listed on tracker #3582. If it still happens on main, please say so here and I'll take it off the list.

🤖 Generated with Claude Code (Sonnet 5.5, verified by Opus 5.5) 🦊

<!-- gh-comment-id:6174964711 --> @CassieFox23 commented on GitHub (Oct 14, 2026): Heads up: this looks fixed on main in 2b60e097f. The change is in `src/domain/webCaptureDiff.ts`. It is listed on tracker #3582. If it still happens on main, please say so here and I'll take it off the list. 🤖 Generated with Claude Code (Sonnet 5.5, verified by Opus 5.5) 🦊
Author
Owner

@morluto commented on GitHub (Oct 14, 2026):

Resolved on main by 2b60e097f.

Missing redirect response fields remain unknown; they no longer establish false known network evidence.

Rechecked on main b8510ee00 in the current focused run: 20 test files passed, 225 tests passed and one platform-dependent test skipped. Relevant regression coverage: tests/boundary/browser/webCaptureDiff.test.ts.

<!-- gh-comment-id:6174984049 --> @morluto commented on GitHub (Oct 14, 2026): Resolved on main by 2b60e097f. Missing redirect response fields remain unknown; they no longer establish false known network evidence. Rechecked on main b8510ee00 in the current focused run: 20 test files passed, 225 tests passed and one platform-dependent test skipped. Relevant regression coverage: tests/boundary/browser/webCaptureDiff.test.ts.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
morluto/rea#1108
No description provided.