[PR #4476] [MERGED] fix(browser): keep omitted JSON request bodies out of complete shape coverage #6261

Closed
opened 2026-10-31 01:13:07 +00:00 by bsakaguchi · 0 comments
Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/4476
Author: @Adolanium
Created: 10/16/2026
Status: ✅ Merged
Merged: 10/22/2026
Merged by: @morluto

Base: main ← Head: fix/json-body-shape-coverage


📝 Commits (1)

  • bf33c6f fix(browser): keep omitted JSON request bodies out of complete shape coverage

📊 Changes

6 files changed (+108 additions, -14 deletions)

View changed files

📝 docs/browser-observation.md (+1 -1)
📝 src/browser/CdpCaptureEventBodyShapes.ts (+14 -5)
📝 src/browser/CdpCaptureEventHandlers.ts (+1 -1)
📝 src/browser/CdpCaptureEventState.ts (+2 -0)
📝 src/browser/CdpCaptureEvents.ts (+2 -0)
📝 tests/boundary/browser/cdpJsonShapeComparison.test.ts (+88 -7)

📄 Description

Summary

Fixes #4477.

With include_json_body_shapes, a JSON request whose body CDP did not deliver was recorded as complete once its response shape arrived. Two identical captures of such a request compared as unchanged instead of unknown.

Problem and expected behavior

docs/browser-observation.md says unavailable or truncated selected coverage makes equivalence unknown.

CDP sets hasPostData: true and omits postData when a request body is long. requestBodyShape in src/browser/CdpCaptureEventBodyShapes.ts returned status: "unavailable" for that request without marking json_body_shapes unavailable. updateResponseBodyShape then set status: "included" with request: null as soon as a JSON response shape arrived. networkDimension in src/domain/webCaptureDiff.ts treats included plus a complete json_body_shapes section as full coverage, so the comparison reported unchanged. hasPostData was not read anywhere in src/. An unparseable JSON request body had the same upgrade to included, although that case already marked the section unavailable.

Change and scope

requestBodyShape now records a request in missingRequestBodyShapeIds and marks json_body_shapes unavailable when a JSON request-body shape was expected but not obtained:

  • hasPostData: true without postData, and the content type is JSON;
  • postData with a JSON content type that does not parse.

updateResponseBodyShape reports such a request as partial instead of included when the response shape arrives. The set is reset on each requestWillBeSent for the request ID and cleared in beginAuthorizedFrame.

A request without a body (no postData, no hasPostData) keeps today's behavior, including a GET that sends a JSON Content-Type. A request with an omitted body and a non-JSON or absent content type, such as a form post, is also unchanged, because only JSON media types are read. This matches the existing handling of an inline body without a JSON content type.

docs/browser-observation.md now states that such a JSON-typed request is partial and makes json_body_shapes unavailable.

Scope: not applicable. This is a fix within the existing passive capture boundary.

Contract and boundary impact

  • Semantic owner and earliest changed stage: CDP passive capture, request body-shape projection in requestBodyShape.
  • Contribution scope fit and linked maintainer decision: ordinary fix within existing boundaries; not applicable.
  • CLI and MCP/tool-catalog contract: no schema change. body_shapes.status uses the existing partial value for these requests, and completeness.unavailable_sections gains json_body_shapes.
  • Provider, bridge, target-format, or platform compatibility: none.
  • Evidence, artifact, provenance, or reconstruction contract: a capture no longer claims complete JSON body-shape coverage for a request whose body shape was not observed. compare_web_captures reports unknown for such captures.
  • Target selection, authority, upstream behavior, and cleanup details (if applicable): not applicable.
  • Generated metadata (docs/public/product-catalog.json), package, or installation impact: none.

Expected effects (check known effects; use documentation-only on its own).
Unchecked items do not establish absence; leave uncertainties visible.

  • Read existing files or captured data.
  • Launch or attach to a process.
  • Inject instrumentation or run scripts/code.
  • Change process memory, registers, files, or configuration.
  • Contact network services.
  • Recover protected code or work around protections.
  • Documentation or usability only; no target interaction.
  • Not sure yet; investigation or maintainer guidance needed.

Evidence and regression coverage

  • Tests added or updated: tests/boundary/browser/cdpJsonShapeComparison.test.ts feeds CDP events through CdpCaptureEvents and compares the captures through compareWebCaptureEvidence. It covers an omitted JSON body and an unparseable JSON body (both partial, section unavailable, comparison unknown). It also covers a bodiless GET, a bodiless GET with a JSON Content-Type, an omitted body without a media type, and an omitted form body (all included, comparison unchanged).
  • Base reproduction or other evidence: executed on upstream/main (8ab502439) with the same test helper. For hasPostData: true, JSON Content-Type, no postData, and a JSON response, the request had body_shapes.status: "included", unavailable_sections was [], and the network dimension was unchanged. The two new partial cases failed on the base; the four unchanged-behavior cases passed.
  • User-visible CLI/MCP output (if applicable): "body_shapes": { "status": "partial", "request": null, "response": { ... } } and json_body_shapes in completeness.unavailable_sections for such a request.
  • Remaining proof gaps: no live Chromium capture with a long request body was run. The events follow the CDP Network.Request shape (hasPostData, postData).

For evidence-bearing changes:

  • Observed, derived, and inferred claims remain distinguishable.
  • Artifact identity, source provenance, and failed attempts remain preserved.
  • Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

  • npx vitest run tests/boundary/browser/cdpJsonShapeComparison.test.ts src/browser/CdpCaptureEvents.redirectBodyShape.test.ts tests/boundary/browser/cdpBrowserProvider.network.test.ts tests/boundary/browser/cdpBrowserProvider.redirectErrors.test.ts tests/boundary/browser/cdpBrowserProvider.sensitive_data.test.ts src/domain/webCaptureDiff.network.test.ts tests/boundary/mcp/browserComparisonInput.test.ts — 7 files, 119 tests passed.
  • npx vitest run tests/boundary/browser/cdpJsonShapeComparison.test.ts with src/browser restored from 8ab502439 — 2 failed (the omitted and unparseable JSON body cases), 42 passed.
  • npx tsc --noEmit --pretty false — no errors.
  • npx oxlint on the five changed source and test files — 0 errors; 3 existing size and complexity warnings in CdpCaptureEventHandlers.ts.
  • npx oxfmt --check on the changed files and docs/browser-observation.md — all files correctly formatted.

Compatibility, safety, and release

  • Breaking changes or migration steps: none. Captures with an omitted or unparseable JSON request body now report partial and an unavailable json_body_shapes section, so comparisons that reported unchanged for them now report unknown.
  • Real Hopper/Ghidra, browser, or OS coverage: not run.
  • Package or release metadata impact: none.
  • Security, privacy, process, or containment review: no new data is read or retained. hasPostData is a boolean already present in the CDP event.

Author readiness

Human owner: @Adolanium

  • I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
  • I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
  • I declare this PR ready for initial maintainer review.

Before merge

  • This PR addresses a concrete problem or an agreed enhancement.
  • Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
  • The PR has one focused outcome and the title follows type(scope): outcome.
  • Related issue is linked, or the reason for not linking one is stated above.
  • Tests cover changed observable behavior and meaningful failure paths.
  • Owning docs, contracts, and generated metadata are updated where needed.
  • User-visible CLI/MCP changes include representative output.
  • I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/4476 **Author:** [@Adolanium](https://github.com/Adolanium) **Created:** 10/16/2026 **Status:** ✅ Merged **Merged:** 10/22/2026 **Merged by:** [@morluto](https://github.com/morluto) **Base:** `main` ← **Head:** `fix/json-body-shape-coverage` --- ### 📝 Commits (1) - [`bf33c6f`](https://github.com/morluto/rea/commit/bf33c6fbb8155732b4bdfcd433982f1fa36ff559) fix(browser): keep omitted JSON request bodies out of complete shape coverage ### 📊 Changes **6 files changed** (+108 additions, -14 deletions) <details> <summary>View changed files</summary> 📝 `docs/browser-observation.md` (+1 -1) 📝 `src/browser/CdpCaptureEventBodyShapes.ts` (+14 -5) 📝 `src/browser/CdpCaptureEventHandlers.ts` (+1 -1) 📝 `src/browser/CdpCaptureEventState.ts` (+2 -0) 📝 `src/browser/CdpCaptureEvents.ts` (+2 -0) 📝 `tests/boundary/browser/cdpJsonShapeComparison.test.ts` (+88 -7) </details> ### 📄 Description ## Summary Fixes #4477. With `include_json_body_shapes`, a JSON request whose body CDP did not deliver was recorded as complete once its response shape arrived. Two identical captures of such a request compared as `unchanged` instead of `unknown`. ## Problem and expected behavior `docs/browser-observation.md` says unavailable or truncated selected coverage makes equivalence unknown. CDP sets `hasPostData: true` and omits `postData` when a request body is long. `requestBodyShape` in `src/browser/CdpCaptureEventBodyShapes.ts` returned `status: "unavailable"` for that request without marking `json_body_shapes` unavailable. `updateResponseBodyShape` then set `status: "included"` with `request: null` as soon as a JSON response shape arrived. `networkDimension` in `src/domain/webCaptureDiff.ts` treats `included` plus a complete `json_body_shapes` section as full coverage, so the comparison reported `unchanged`. `hasPostData` was not read anywhere in `src/`. An unparseable JSON request body had the same upgrade to `included`, although that case already marked the section unavailable. ## Change and scope `requestBodyShape` now records a request in `missingRequestBodyShapeIds` and marks `json_body_shapes` unavailable when a JSON request-body shape was expected but not obtained: - `hasPostData: true` without `postData`, and the content type is JSON; - `postData` with a JSON content type that does not parse. `updateResponseBodyShape` reports such a request as `partial` instead of `included` when the response shape arrives. The set is reset on each `requestWillBeSent` for the request ID and cleared in `beginAuthorizedFrame`. A request without a body (no `postData`, no `hasPostData`) keeps today's behavior, including a GET that sends a JSON `Content-Type`. A request with an omitted body and a non-JSON or absent content type, such as a form post, is also unchanged, because only JSON media types are read. This matches the existing handling of an inline body without a JSON content type. `docs/browser-observation.md` now states that such a JSON-typed request is `partial` and makes `json_body_shapes` unavailable. Scope: not applicable. This is a fix within the existing passive capture boundary. ## Contract and boundary impact - Semantic owner and earliest changed stage: CDP passive capture, request body-shape projection in `requestBodyShape`. - Contribution scope fit and linked maintainer decision: ordinary fix within existing boundaries; not applicable. - CLI and MCP/tool-catalog contract: no schema change. `body_shapes.status` uses the existing `partial` value for these requests, and `completeness.unavailable_sections` gains `json_body_shapes`. - Provider, bridge, target-format, or platform compatibility: none. - Evidence, artifact, provenance, or reconstruction contract: a capture no longer claims complete JSON body-shape coverage for a request whose body shape was not observed. `compare_web_captures` reports `unknown` for such captures. - Target selection, authority, upstream behavior, and cleanup details (if applicable): not applicable. - Generated metadata (`docs/public/product-catalog.json`), package, or installation impact: none. Expected effects (check known effects; use documentation-only on its own). Unchecked items do not establish absence; leave uncertainties visible. - [x] Read existing files or captured data. - [ ] Launch or attach to a process. - [ ] Inject instrumentation or run scripts/code. - [ ] Change process memory, registers, files, or configuration. - [ ] Contact network services. - [ ] Recover protected code or work around protections. - [ ] Documentation or usability only; no target interaction. - [ ] Not sure yet; investigation or maintainer guidance needed. ## Evidence and regression coverage - Tests added or updated: `tests/boundary/browser/cdpJsonShapeComparison.test.ts` feeds CDP events through `CdpCaptureEvents` and compares the captures through `compareWebCaptureEvidence`. It covers an omitted JSON body and an unparseable JSON body (both `partial`, section unavailable, comparison `unknown`). It also covers a bodiless GET, a bodiless GET with a JSON `Content-Type`, an omitted body without a media type, and an omitted form body (all `included`, comparison `unchanged`). - Base reproduction or other evidence: executed on `upstream/main` (`8ab502439`) with the same test helper. For `hasPostData: true`, JSON `Content-Type`, no `postData`, and a JSON response, the request had `body_shapes.status: "included"`, `unavailable_sections` was `[]`, and the network dimension was `unchanged`. The two new `partial` cases failed on the base; the four unchanged-behavior cases passed. - User-visible CLI/MCP output (if applicable): `"body_shapes": { "status": "partial", "request": null, "response": { ... } }` and `json_body_shapes` in `completeness.unavailable_sections` for such a request. - Remaining proof gaps: no live Chromium capture with a long request body was run. The events follow the CDP `Network.Request` shape (`hasPostData`, `postData`). For evidence-bearing changes: - [x] Observed, derived, and inferred claims remain distinguishable. - [x] Artifact identity, source provenance, and failed attempts remain preserved. - [x] Unsupported, incomplete, unavailable, or uncertain outcomes remain visible. ## Validation performed - `npx vitest run tests/boundary/browser/cdpJsonShapeComparison.test.ts src/browser/CdpCaptureEvents.redirectBodyShape.test.ts tests/boundary/browser/cdpBrowserProvider.network.test.ts tests/boundary/browser/cdpBrowserProvider.redirectErrors.test.ts tests/boundary/browser/cdpBrowserProvider.sensitive_data.test.ts src/domain/webCaptureDiff.network.test.ts tests/boundary/mcp/browserComparisonInput.test.ts` — 7 files, 119 tests passed. - `npx vitest run tests/boundary/browser/cdpJsonShapeComparison.test.ts` with `src/browser` restored from `8ab502439` — 2 failed (the omitted and unparseable JSON body cases), 42 passed. - `npx tsc --noEmit --pretty false` — no errors. - `npx oxlint` on the five changed source and test files — 0 errors; 3 existing size and complexity warnings in `CdpCaptureEventHandlers.ts`. - `npx oxfmt --check` on the changed files and `docs/browser-observation.md` — all files correctly formatted. ## Compatibility, safety, and release - Breaking changes or migration steps: none. Captures with an omitted or unparseable JSON request body now report `partial` and an unavailable `json_body_shapes` section, so comparisons that reported `unchanged` for them now report `unknown`. - Real Hopper/Ghidra, browser, or OS coverage: not run. - Package or release metadata impact: none. - Security, privacy, process, or containment review: no new data is read or retained. `hasPostData` is a boolean already present in the CDP event. ## Author readiness <!-- The human owner completes these declarations before requesting review. The PR author is the owner by default. A different owner must explicitly accept ownership on this PR. AI agents must leave these declarations to a human to dictate to them or there could be catastrophic consequences for many users of REA. Draft PRs may leave them unchecked. These declarations record initial readiness when opening the PR for review; later pushes do not require a new human sign-off. Keep the description and validation evidence current as the work changes. See CONTRIBUTING.md#pull-request-ownership-and-readiness. --> Human owner: @Adolanium - [x] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [x] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [x] I declare this PR ready for initial maintainer review. ## Before merge <!-- Draft PRs may leave pending items unchecked. Resolve material scope and authority questions before integration. --> - [ ] This PR addresses a concrete problem or an agreed enhancement. - [ ] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff. - [ ] The PR has one focused outcome and the title follows `type(scope): outcome`. - [ ] Related issue is linked, or the reason for not linking one is stated above. - [ ] Tests cover changed observable behavior and meaningful failure paths. - [ ] Owning docs, contracts, and generated metadata are updated where needed. - [ ] User-visible CLI/MCP changes include representative output. - [ ] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
bsakaguchi 2026-10-31 01:13:07 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
morluto/rea#6261
No description provided.