morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[PR #2046] [CLOSED] feat(browser): redact declared values in session HAR #4990

New issue

Closed

opened 2026-10-31 00:57:44 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:57:44 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/2046
Author: @Hyped01
Created: 10/12/2026
Status: ❌ Closed

Base: main ← Head: feat/session-har-2-redaction

📝 Commits (3)

6b912ce fix(browser): admit browser-serialized HAR request URLs
fba05f5 feat(browser): record observe_web_session traffic as HAR
43a0e2b feat(browser): redact declared values in session HAR

📊 Changes

22 files changed (+4040 additions, -43 deletions)

View changed files

📝 docs/browser-observation.md (+138 -1)
📝 docs/web-network-captures.md (+6 -0)
📝 src/application/BrowserEvidence.ts (+46 -7)
📝 src/browser/CdpObservationSession.ts (+82 -17)
➕ src/browser/CdpSessionHar.ts (+372 -0)
➕ src/browser/CdpSessionNetwork.ts (+480 -0)
➕ src/browser/CdpSessionResult.ts (+122 -0)
➕ src/browser/SessionRedactor.test.ts (+289 -0)
➕ src/browser/SessionRedactor.ts (+372 -0)
📝 src/browser/history/CaptureRedaction.ts (+2 -7)
📝 src/browser/history/HarSchema.ts (+28 -1)
📝 src/browser/network/PlaywrightNetworkContent.ts (+2 -8)
📝 src/browser/network/PlaywrightScenarioNetwork.test.ts (+44 -0)
➕ src/browser/network/TransportCredentials.ts (+7 -0)
📝 src/cli/browserCommands.ts (+12 -1)
📝 src/cliObservationOptions.ts (+24 -0)
📝 src/contracts/browserToolContracts.ts (+12 -1)
📝 src/domain/browserSession.ts (+101 -0)
➕ tests/boundary/browser/cdpObservationSessionNetwork.test.ts (+1493 -0)
📝 tests/boundary/browser/historicalCaptureDecoder.test.ts (+44 -0)

...and 2 more files

📄 Description

Stacked on #2045. Only the top commit, 43a0e2ba0, is new here.

Summary

Refs #774. This PR adds sensitive_values (CLI --sensitive-value) and sensitive_query_parameters (CLI --sensitive-query-parameter) to observe_web_session. Declared values are then replaced, with [REDACTED:value-N] markers, in every field the session result and its HAR retain.

Problem and expected behavior

The recorder from #2045 retains URLs, query values, header values and timeline fields verbatim. A user who knows a token, a device ID or a signature value must be able to keep it out of the evidence, including the spellings in which browsers and servers re-encode it.

Change and scope

What is checked. src/browser/SessionRedactor.ts checks every retained field, both as written and in its UTF-8 percent-decoded, form-decoded and JSON-unescaped forms, nested up to three levels. That covers URLs, query names and values, headers, timeline fields, target IDs, MIME types, methods and browser metadata (browser, har.log.browser, and therefore the Evidence environment). A value that survives replacement in any of those forms empties the field.
URLs are redacted piece by piece, by path segment, query pair and fragment piece:
- Each piece is redacted as written before it is decoded. A raw +, a lowercase escape or an unreserved escape therefore cannot hide a value, and a piece whose result still spells one is emptied.
- Unrelated bytes are unchanged and the URL stays valid.
- A host that spells a value becomes redacted.invalid and keeps its port. A port that spells one loses its digits but keeps its host and the :, a spelling no browser writes, so the removal stays visible. Evidence never derives an origin from such a URL.
- A value that spans pieces fails the whole URL closed. Placeholders follow the order <scheme>//redacted.invalid/, then urn:redacted, then x:y. An entry that none of these can stand in for is omitted and counted under network_requests or websocket_connections/websocket_frames.
- The HAR queryString is derived from the redacted URL, so it never reveals what the URL hides.
Matching is one Aho-Corasick automaton built once per session, so cost stays linear however many literals are declared. Values are matched as spelled: browsers report hosts lowercased and in punycode, and the docs say to declare that spelling. Literals with unpaired surrogates are accepted.
Query parameters. Observed values of named query parameters become declared literals, including values inside retained URL-valued headers.
Evidence parameters keep the parameter names and the count of declared literals, never the literals. A request parameter or parameter name that spells one is left empty, and the result says so.
Counting. redactions.explicit_value_fields counts the changed HAR and network-target fields.
URL-valued headers that are not one absolute URL are redacted as text: relative Location/Content-Location, Link and Refresh. Their markers are not percent-encoded, and a value found only in a decoded spelling empties the header value. The docs say so.
Not detected, as the docs state: percent-encoding in a legacy charset, HTML entities, base64 inside headers, and query parameters inside a fragment (declare those values with sensitive_values).

Contract and boundary impact

Semantic owner and earliest changed stage: src/browser/SessionRedactor.ts, applied in src/browser/CdpSessionResult.ts.
CLI and MCP/tool-catalog contract: additive optional inputs, and an additive redactions.explicit_value_fields output.
Provider, bridge, target-format, or platform compatibility: none.
Evidence, artifact, provenance, or reconstruction contract: the Evidence parameters record names and counts only.
Process execution, authorization, cleanup, or containment impact: none.
Generated metadata (docs/public/product-catalog.json), package, or installation impact: none.

Evidence and regression coverage

Tests added or updated:
- src/browser/SessionRedactor.test.ts:
  - piecewise URL redaction that keeps unrelated bytes;
  - invalid UTF-8, nested and JSON spellings;
  - raw + and lowercase or unreserved escapes;
  - fallbacks and markers;
  - overlapping values;
  - lone surrogates;
  - linear scaling with 3,000 signatures and with nested literals.
- tests/boundary/browser/cdpObservationSessionNetwork.test.ts:
  - exact field counts;
  - a value that spans query pairs;
  - depth-four literals in query values;
  - emptied flag pairs that keep alignment;
  - browser metadata;
  - an observed query value spelled by every URL's scheme;
  - per-section counts for inadmissible HTTP and WebSocket entries;
  - an end-to-end case where each decoded reading is checked against every decoding of the serialized session;
  - one test per redaction call site: retained capture URLs as a query-value source in a navigation-only session, text-redacted URL-valued headers, redirectURL, query names, statusText, header names, and timeline frame and request IDs and details.
- MCP tests: Evidence parameter blanking, including a derived origin and a query parameter name.
Mutation checks: each of these fails a test.
- As-written replacement first.
- The form re-check.
- The re-check of the encoded result.
- Every placeholder fallback.
- Per-section counts.
- The surrogate guards.
- The JSON escape set.
- The Evidence predicate.
Remaining proof gaps: the spellings listed above as not detected.

For evidence-bearing changes:

Observed, derived, and inferred claims remain distinguishable.
Artifact identity, source provenance, and failed attempts remain preserved.
Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

Per-commit gate: pass (see #2045).
npx vitest run on 12 browser, CLI, MCP and redactor test files: 104 passed, 2 skipped.
exposes CLI-equivalent Evidence results and session tools, an existing MCP test, times out only under heavy machine load. It fails the same way on unmodified main.
Cold review with adversarial verification: the final round reported no findings.

Compatibility, safety, and release

Breaking changes or migration steps: none; redaction is opt-in.
Package or release metadata impact: none.
Security, privacy, process, or containment review: redaction fails closed. A field in which any decoding still spells a declared value is emptied, and an unrepresentable entry is omitted and counted.

🤖 Generated with Claude Code

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/2046 **Author:** [@Hyped01](https://github.com/Hyped01) **Created:** 10/12/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `feat/session-har-2-redaction` --- ### 📝 Commits (3) - [`6b912ce`](https://github.com/morluto/rea/commit/6b912ce5b90e9b4a395570af0060c5efee9ab44a) fix(browser): admit browser-serialized HAR request URLs - [`fba05f5`](https://github.com/morluto/rea/commit/fba05f5e123ee1d1893689e2d6b265ea618459ac) feat(browser): record observe_web_session traffic as HAR - [`43a0e2b`](https://github.com/morluto/rea/commit/43a0e2ba067fdd4c811e4a561dc2a09a4ee1da16) feat(browser): redact declared values in session HAR ### 📊 Changes **22 files changed** (+4040 additions, -43 deletions) <details> <summary>View changed files</summary> 📝 `docs/browser-observation.md` (+138 -1) 📝 `docs/web-network-captures.md` (+6 -0) 📝 `src/application/BrowserEvidence.ts` (+46 -7) 📝 `src/browser/CdpObservationSession.ts` (+82 -17) ➕ `src/browser/CdpSessionHar.ts` (+372 -0) ➕ `src/browser/CdpSessionNetwork.ts` (+480 -0) ➕ `src/browser/CdpSessionResult.ts` (+122 -0) ➕ `src/browser/SessionRedactor.test.ts` (+289 -0) ➕ `src/browser/SessionRedactor.ts` (+372 -0) 📝 `src/browser/history/CaptureRedaction.ts` (+2 -7) 📝 `src/browser/history/HarSchema.ts` (+28 -1) 📝 `src/browser/network/PlaywrightNetworkContent.ts` (+2 -8) 📝 `src/browser/network/PlaywrightScenarioNetwork.test.ts` (+44 -0) ➕ `src/browser/network/TransportCredentials.ts` (+7 -0) 📝 `src/cli/browserCommands.ts` (+12 -1) 📝 `src/cliObservationOptions.ts` (+24 -0) 📝 `src/contracts/browserToolContracts.ts` (+12 -1) 📝 `src/domain/browserSession.ts` (+101 -0) ➕ `tests/boundary/browser/cdpObservationSessionNetwork.test.ts` (+1493 -0) 📝 `tests/boundary/browser/historicalCaptureDecoder.test.ts` (+44 -0) _...and 2 more files_ </details> ### 📄 Description > Stacked on #2045. Only the top commit, `43a0e2ba0`, is new here. ## Summary Refs #774. This PR adds `sensitive_values` (CLI `--sensitive-value`) and `sensitive_query_parameters` (CLI `--sensitive-query-parameter`) to `observe_web_session`. Declared values are then replaced, with `[REDACTED:value-N]` markers, in every field the session result and its HAR retain. ## Problem and expected behavior The recorder from #2045 retains URLs, query values, header values and timeline fields verbatim. A user who knows a token, a device ID or a signature value must be able to keep it out of the evidence, including the spellings in which browsers and servers re-encode it. ## Change and scope - **What is checked.** `src/browser/SessionRedactor.ts` checks every retained field, both as written and in its UTF-8 percent-decoded, form-decoded and JSON-unescaped forms, nested up to three levels. That covers URLs, query names and values, headers, timeline fields, target IDs, MIME types, methods and browser metadata (`browser`, `har.log.browser`, and therefore the Evidence environment). A value that survives replacement in any of those forms empties the field. - **URLs are redacted piece by piece**, by path segment, query pair and fragment piece: - Each piece is redacted as written before it is decoded. A raw `+`, a lowercase escape or an unreserved escape therefore cannot hide a value, and a piece whose result still spells one is emptied. - Unrelated bytes are unchanged and the URL stays valid. - A host that spells a value becomes `redacted.invalid` and keeps its port. A port that spells one loses its digits but keeps its host and the `:`, a spelling no browser writes, so the removal stays visible. Evidence never derives an origin from such a URL. - A value that spans pieces fails the whole URL closed. Placeholders follow the order `<scheme>//redacted.invalid/`, then `urn:redacted`, then `x:y`. An entry that none of these can stand in for is omitted and counted under `network_requests` or `websocket_connections`/`websocket_frames`. - The HAR `queryString` is derived from the redacted URL, so it never reveals what the URL hides. - **Matching** is one Aho-Corasick automaton built once per session, so cost stays linear however many literals are declared. Values are matched as spelled: browsers report hosts lowercased and in punycode, and the docs say to declare that spelling. Literals with unpaired surrogates are accepted. - **Query parameters.** Observed values of named query parameters become declared literals, including values inside retained URL-valued headers. - **Evidence parameters** keep the parameter names and the count of declared literals, never the literals. A request parameter or parameter name that spells one is left empty, and the result says so. - **Counting.** `redactions.explicit_value_fields` counts the changed HAR and network-target fields. - **URL-valued headers that are not one absolute URL** are redacted as text: relative `Location`/`Content-Location`, `Link` and `Refresh`. Their markers are not percent-encoded, and a value found only in a decoded spelling empties the header value. The docs say so. - **Not detected, as the docs state:** percent-encoding in a legacy charset, HTML entities, base64 inside headers, and query parameters inside a fragment (declare those values with `sensitive_values`). ## Contract and boundary impact - Semantic owner and earliest changed stage: `src/browser/SessionRedactor.ts`, applied in `src/browser/CdpSessionResult.ts`. - CLI and MCP/tool-catalog contract: additive optional inputs, and an additive `redactions.explicit_value_fields` output. - Provider, bridge, target-format, or platform compatibility: none. - Evidence, artifact, provenance, or reconstruction contract: the Evidence parameters record names and counts only. - Process execution, authorization, cleanup, or containment impact: none. - Generated metadata (`docs/public/product-catalog.json`), package, or installation impact: none. ## Evidence and regression coverage - Tests added or updated: - `src/browser/SessionRedactor.test.ts`: - piecewise URL redaction that keeps unrelated bytes; - invalid UTF-8, nested and JSON spellings; - raw `+` and lowercase or unreserved escapes; - fallbacks and markers; - overlapping values; - lone surrogates; - linear scaling with 3,000 signatures and with nested literals. - `tests/boundary/browser/cdpObservationSessionNetwork.test.ts`: - exact field counts; - a value that spans query pairs; - depth-four literals in query values; - emptied flag pairs that keep alignment; - browser metadata; - an observed query value spelled by every URL's scheme; - per-section counts for inadmissible HTTP and WebSocket entries; - an end-to-end case where each decoded reading is checked against every decoding of the serialized session; - one test per redaction call site: retained capture URLs as a query-value source in a navigation-only session, text-redacted URL-valued headers, `redirectURL`, query names, `statusText`, header names, and timeline frame and request IDs and details. - MCP tests: Evidence parameter blanking, including a derived origin and a query parameter name. - Mutation checks: each of these fails a test. - As-written replacement first. - The form re-check. - The re-check of the encoded result. - Every placeholder fallback. - Per-section counts. - The surrogate guards. - The JSON escape set. - The Evidence predicate. - Remaining proof gaps: the spellings listed above as not detected. For evidence-bearing changes: - [x] Observed, derived, and inferred claims remain distinguishable. - [x] Artifact identity, source provenance, and failed attempts remain preserved. - [x] Unsupported, incomplete, unavailable, or uncertain outcomes remain visible. ## Validation performed - Per-commit gate: pass (see #2045). - `npx vitest run` on 12 browser, CLI, MCP and redactor test files: 104 passed, 2 skipped. - `exposes CLI-equivalent Evidence results and session tools`, an existing MCP test, times out only under heavy machine load. It fails the same way on unmodified `main`. - Cold review with adversarial verification: the final round reported no findings. ## Compatibility, safety, and release - Breaking changes or migration steps: none; redaction is opt-in. - Package or release metadata impact: none. - Security, privacy, process, or containment review: redaction fails closed. A field in which any decoding still spells a declared value is emptied, and an unrepresentable entry is omitted and counted. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-31 00:57:44 +00:00

closed this issue
added the
pull-request
label

bsakaguchi referenced this issue

2026-11-01 21:42:29 +00:00

[PR #4990] [MERGED] perf(apktool): avoid duplicated primary strings on request #6713

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#4990

No description provided.

Rows
Columns