[PR #2477] [MERGED] fix(browser): exclude sensitive binary lengths from historical captures #5258

Closed
opened 2026-10-31 01:05:13 +00:00 by bsakaguchi · 0 comments
Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/2477
Author: @chAng-L19
Created: 10/13/2026
Status: ✅ Merged
Merged: 10/13/2026
Merged by: @morluto

Base: main ← Head: fix/capture-sensitive-byte-length


📝 Commits (1)

  • be46325 fix(browser): exclude sensitive binary lengths from historical captures

📊 Changes

4 files changed (+105 additions, -2 deletions)

View changed files

📝 bridge/mitmproxy/capture.py (+1 -1)
📝 scripts/fixtures/generate-mitmproxy-capture.py (+14 -0)
📝 src/browser/history/HarCapture.ts (+2 -1)
📝 tests/boundary/browser/historicalWebCapture.test.ts (+88 -0)

📄 Description

Summary

Honor explicit sensitive-value exclusions for derived binary byte counts in historical HAR and mitmproxy captures. Both producers now apply the same count exclusion already used by SAZ.

Problem and expected behavior

When an excluded literal matches only a decoded body's decimal byte count, the historical HAR and mitmproxy projections retain that count and its binary field. For example, a synthetic 12,345-byte body with sensitive_values: ["12345"] still produces state: "retained", bytes: 12345. The input contract explicitly excludes literals from byte/numeric projections. Substring declarations, such as "2345", must also apply.

The corrected field is state: "redacted" with bytes, content_base64, and sha256 all null, and the corresponding body display is omitted. Unmarked content remains retained; HAR producer-reported content.size remains an independent source fact.

No related public issue was opened; this is a focused correction using synthetic capture data. Before submission, I searched all 1,609 historical PRs, including 207 open PRs, and checked three additional newly opened PRs immediately before submission. I also inspected relevant file diffs: #992 introduced the two affected producers; #2409 already implements count exclusion for SAZ; #2046 concerns session HAR. None implements this correction in the two affected producers.

Change and scope

Check the generated decimal byte count at the two producing boundaries, alongside the existing raw-byte, base64, and digest checks. Reuse existing redacted-field representations and application consumers without adding another downstream filter or dependency.

This follows the contribution scope. Separate maintainer scope decision: not applicable (ordinary fix within an existing offline capture capability).

Contract and boundary impact

  • Semantic owner and earliest changed stage: HAR projectHar and mitmproxy project_record, before binary sidecars are retained as Evidence.
  • Contribution scope fit and linked maintainer decision: ordinary bug fix; not applicable.
  • CLI and MCP/tool-catalog contract: existing sensitive_values semantics; no new fields or operations. Public CLI and MCP results agree.
  • Provider, bridge, target-format, or platform compatibility: historical HAR and the existing mitmproxy 12.2.3 bridge; SAZ unchanged. The branch is based on bb154d106, including the new qlog capability. The subsequent main commits through 172407787 do not change any of the four touched files; a merge-tree check succeeds.
  • Evidence, artifact, provenance, or reconstruction contract: existing explicit-sensitive-value redaction and nullable binary sidecar; retained preview/complete views preserve the exclusion.
  • Target selection, authority, upstream behavior, and cleanup details: caller-selected offline captures; no new target execution, network requests, upstream decoder changes, or cleanup behavior.
  • Generated metadata, package, or installation impact: none.

Expected effects:

  • Read existing files or captured data.
  • Launch or attach to a process.
  • Inject instrumentation or run scripts/code.
  • Change process memory, registers, files, or configuration.
  • Contact network services.
  • Recover protected code or work around protections.
  • Documentation or usability only; no target interaction.
  • Not sure yet; investigation or maintainer guidance needed.

The effects above describe this change; the existing mitmproxy workflow still uses its owned decoder process.

Evidence and regression coverage

  • Tests added or updated: public CLI/MCP HAR regression with an unmarked control, exact and substring count exclusions, independent producer-size preservation, and retained preview/complete views. The existing real-mitmproxy fixture generator gains equivalent control and exclusion assertions.

  • Base reproduction: the new HAR regression fails on a2d8b94dd because the body remains visible. Executing the unchanged production Python projection from that revision retains bytes: 12345; executing the fixed projection redacts both declarations. These Python checks compile the production projection functions from their AST and do not substitute a decoder.

  • Representative public output: binary_fields: [{ pointer: "/response/content/text", representation: "har-base64-content", state: "redacted", bytes: null, content_base64: null, sha256: null }].

  • Remaining proof gaps: the real mitmproxy 12.2.3/Linux verification lane has not run locally on this Windows host. The added generator assertions need that existing lane. Broader source tests are not all green on this host (details below).

  • Observed, derived, and inferred claims remain distinguishable.

  • Artifact identity, source provenance, and failed attempts remain preserved.

  • Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

  • npm run test:focused -- tests/boundary/browser/historicalWebCapture.test.ts src/browser/history/HarCapture.test.ts src/application/WebNetworkCaptureService.test.ts tests/boundary/browser/sazWebCapture.test.ts — 92 passed, 1 skipped, after rebasing onto bb154d106.
  • npm run test:focused -- src/browser/history/QlogCapture.test.ts tests/acceptance/tools/qlogCapture.test.ts — 24 passed, 1 skipped, on the rebased branch.
  • npm run check — passed on the rebased branch (typecheck, lint/module boundaries, formatting, knip, metadata, architecture guards); lint reports 997 warnings, zero errors.
  • npm run verify:test-discovery — passed on the initial base (1,008 test files discovered once).
  • Python AST syntax checks for both changed Python files and production projection control/exact/substring checks — passed. Real mitmproxy process execution is unverified locally.
  • git diff --check origin/main and the repository pre-commit/pre-push hooks — passed.
  • Broader source run before rebase: 424 files, 5,346 passed tests, 58 failed, 95 skipped. Failures include Windows symlink permissions, POSIX path/mode assumptions, provider fixture/readiness mismatches, Windows cleanup, and a cancellation assertion. Those failures are outside this focused diff; their causes have not all been resolved or independently classified. This is not a claim of a passing full suite.
  • npm audit --registry=https://registry.npmjs.org — zero reported advisories for the locked dependency inventory. The Windows native artifact used for broader tests was verified and came from upstream CI at the exact initial base SHA.

Compatibility, safety, and release

  • Breaking changes or migration steps: none; declared count literals now receive the exclusion already documented by the existing input contract.
  • Real Hopper/Ghidra, browser, or OS coverage: Windows x64 Node 24.18.0; real built CLI and stdio MCP against synthetic offline captures. No claim of real Hopper/Ghidra/IDA or Linux mitmproxy verification.
  • Package or release metadata impact: none; no generated outputs committed.
  • Security, privacy, process, or containment review: apply caller declarations at the producers, preserve unmarked evidence, and verify retained capture views. No inference of secrecy from content patterns or field names.

Before merge

  • This PR addresses a concrete problem or an agreed enhancement.
  • Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
  • The PR has one focused outcome and the title follows type(scope): outcome.
  • Related issue is linked, or the reason for not linking one is stated above.
  • Tests cover changed observable behavior and meaningful failure paths.
  • Owning docs, contracts, and generated metadata are updated where needed (existing contract already specifies the corrected behavior).
  • User-visible CLI/MCP changes include representative output.
  • I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/2477 **Author:** [@chAng-L19](https://github.com/chAng-L19) **Created:** 10/13/2026 **Status:** ✅ Merged **Merged:** 10/13/2026 **Merged by:** [@morluto](https://github.com/morluto) **Base:** `main` ← **Head:** `fix/capture-sensitive-byte-length` --- ### 📝 Commits (1) - [`be46325`](https://github.com/morluto/rea/commit/be4632515285e53bfd19116987a128886f4c57cf) fix(browser): exclude sensitive binary lengths from historical captures ### 📊 Changes **4 files changed** (+105 additions, -2 deletions) <details> <summary>View changed files</summary> 📝 `bridge/mitmproxy/capture.py` (+1 -1) 📝 `scripts/fixtures/generate-mitmproxy-capture.py` (+14 -0) 📝 `src/browser/history/HarCapture.ts` (+2 -1) 📝 `tests/boundary/browser/historicalWebCapture.test.ts` (+88 -0) </details> ### 📄 Description ## Summary Honor explicit sensitive-value exclusions for derived binary byte counts in historical HAR and mitmproxy captures. Both producers now apply the same count exclusion already used by SAZ. ## Problem and expected behavior When an excluded literal matches only a decoded body's decimal byte count, the historical HAR and mitmproxy projections retain that count and its binary field. For example, a synthetic 12,345-byte body with `sensitive_values: ["12345"]` still produces `state: "retained", bytes: 12345`. The input contract explicitly excludes literals from byte/numeric projections. Substring declarations, such as `"2345"`, must also apply. The corrected field is `state: "redacted"` with `bytes`, `content_base64`, and `sha256` all `null`, and the corresponding body display is omitted. Unmarked content remains retained; HAR producer-reported `content.size` remains an independent source fact. No related public issue was opened; this is a focused correction using synthetic capture data. Before submission, I searched all 1,609 historical PRs, including 207 open PRs, and checked three additional newly opened PRs immediately before submission. I also inspected relevant file diffs: #992 introduced the two affected producers; #2409 already implements count exclusion for SAZ; #2046 concerns session HAR. None implements this correction in the two affected producers. ## Change and scope Check the generated decimal byte count at the two producing boundaries, alongside the existing raw-byte, base64, and digest checks. Reuse existing redacted-field representations and application consumers without adding another downstream filter or dependency. This follows the [contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope). Separate maintainer scope decision: not applicable (ordinary fix within an existing offline capture capability). ## Contract and boundary impact - Semantic owner and earliest changed stage: HAR `projectHar` and mitmproxy `project_record`, before binary sidecars are retained as Evidence. - Contribution scope fit and linked maintainer decision: ordinary bug fix; not applicable. - CLI and MCP/tool-catalog contract: existing `sensitive_values` semantics; no new fields or operations. Public CLI and MCP results agree. - Provider, bridge, target-format, or platform compatibility: historical HAR and the existing mitmproxy 12.2.3 bridge; SAZ unchanged. The branch is based on `bb154d106`, including the new qlog capability. The subsequent main commits through 172407787 do not change any of the four touched files; a merge-tree check succeeds. - Evidence, artifact, provenance, or reconstruction contract: existing `explicit-sensitive-value` redaction and nullable binary sidecar; retained preview/complete views preserve the exclusion. - Target selection, authority, upstream behavior, and cleanup details: caller-selected offline captures; no new target execution, network requests, upstream decoder changes, or cleanup behavior. - Generated metadata, package, or installation impact: none. Expected effects: - [x] Read existing files or captured data. - [ ] Launch or attach to a process. - [ ] Inject instrumentation or run scripts/code. - [ ] Change process memory, registers, files, or configuration. - [ ] Contact network services. - [ ] Recover protected code or work around protections. - [ ] Documentation or usability only; no target interaction. - [ ] Not sure yet; investigation or maintainer guidance needed. The effects above describe this change; the existing mitmproxy workflow still uses its owned decoder process. ## Evidence and regression coverage - Tests added or updated: public CLI/MCP HAR regression with an unmarked control, exact and substring count exclusions, independent producer-size preservation, and retained preview/complete views. The existing real-mitmproxy fixture generator gains equivalent control and exclusion assertions. - Base reproduction: the new HAR regression fails on `a2d8b94dd` because the body remains visible. Executing the unchanged production Python projection from that revision retains `bytes: 12345`; executing the fixed projection redacts both declarations. These Python checks compile the production projection functions from their AST and do not substitute a decoder. - Representative public output: `binary_fields: [{ pointer: "/response/content/text", representation: "har-base64-content", state: "redacted", bytes: null, content_base64: null, sha256: null }]`. - Remaining proof gaps: the real mitmproxy 12.2.3/Linux verification lane has not run locally on this Windows host. The added generator assertions need that existing lane. Broader source tests are not all green on this host (details below). - [x] Observed, derived, and inferred claims remain distinguishable. - [x] Artifact identity, source provenance, and failed attempts remain preserved. - [x] Unsupported, incomplete, unavailable, or uncertain outcomes remain visible. ## Validation performed - `npm run test:focused -- tests/boundary/browser/historicalWebCapture.test.ts src/browser/history/HarCapture.test.ts src/application/WebNetworkCaptureService.test.ts tests/boundary/browser/sazWebCapture.test.ts` — 92 passed, 1 skipped, after rebasing onto `bb154d106`. - `npm run test:focused -- src/browser/history/QlogCapture.test.ts tests/acceptance/tools/qlogCapture.test.ts` — 24 passed, 1 skipped, on the rebased branch. - `npm run check` — passed on the rebased branch (typecheck, lint/module boundaries, formatting, knip, metadata, architecture guards); lint reports 997 warnings, zero errors. - `npm run verify:test-discovery` — passed on the initial base (1,008 test files discovered once). - Python AST syntax checks for both changed Python files and production projection control/exact/substring checks — passed. Real mitmproxy process execution is unverified locally. - `git diff --check origin/main` and the repository pre-commit/pre-push hooks — passed. - Broader source run before rebase: 424 files, 5,346 passed tests, 58 failed, 95 skipped. Failures include Windows symlink permissions, POSIX path/mode assumptions, provider fixture/readiness mismatches, Windows cleanup, and a cancellation assertion. Those failures are outside this focused diff; their causes have not all been resolved or independently classified. This is not a claim of a passing full suite. - `npm audit --registry=https://registry.npmjs.org` — zero reported advisories for the locked dependency inventory. The Windows native artifact used for broader tests was verified and came from upstream CI at the exact initial base SHA. ## Compatibility, safety, and release - Breaking changes or migration steps: none; declared count literals now receive the exclusion already documented by the existing input contract. - Real Hopper/Ghidra, browser, or OS coverage: Windows x64 Node 24.18.0; real built CLI and stdio MCP against synthetic offline captures. No claim of real Hopper/Ghidra/IDA or Linux mitmproxy verification. - Package or release metadata impact: none; no generated outputs committed. - Security, privacy, process, or containment review: apply caller declarations at the producers, preserve unmarked evidence, and verify retained capture views. No inference of secrecy from content patterns or field names. ## Before merge - [x] This PR addresses a concrete problem or an agreed enhancement. - [x] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff. - [x] The PR has one focused outcome and the title follows `type(scope): outcome`. - [x] Related issue is linked, or the reason for not linking one is stated above. - [x] Tests cover changed observable behavior and meaningful failure paths. - [x] Owning docs, contracts, and generated metadata are updated where needed (existing contract already specifies the corrected behavior). - [x] User-visible CLI/MCP changes include representative output. - [x] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
bsakaguchi 2026-10-31 01:05:13 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
morluto/rea#5258
No description provided.