[PR #2044] [CLOSED] fix(browser): admit browser-serialized HAR request URLs #4984

New issue

Closed

opened 2026-10-31 00:57:40 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:57:40 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/2044
Author: @Hyped01
Created: 10/12/2026
Status: ❌ Closed

Base: main ← Head: fix/har-request-url-query-characters

📝 Commits (1)

6b912ce fix(browser): admit browser-serialized HAR request URLs

📊 Changes

7 files changed (+133 additions, -16 deletions)

View changed files

📝 docs/web-network-captures.md (+6 -0)
📝 src/browser/history/CaptureRedaction.ts (+2 -7)
📝 src/browser/history/HarSchema.ts (+28 -1)
📝 src/browser/network/PlaywrightNetworkContent.ts (+2 -8)
📝 src/browser/network/PlaywrightScenarioNetwork.test.ts (+44 -0)
➕ src/browser/network/TransportCredentials.ts (+7 -0)
📝 tests/boundary/browser/historicalCaptureDecoder.test.ts (+44 -0)

📄 Description

Summary

inspect_web_network_capture validates each HAR request.url with ajv-formats' strict RFC 3986 uri format. Browsers serialize request URLs per WHATWG URL and leave several characters unescaped: [ ] { } | ^ and backtick, a % that starts no escape, and a backslash in the query or fragment. A HAR that Chrome itself exports for such a page is therefore rejected whole as a format error.

Problem and expected behavior

On main, decodeHarCapture rejects every one of these browser serializations with HAR schema validation failed: format.:

https://example.test/?q=[1]|{x}
/search?q=100%
/x?cb=%%CACHEBUSTER%%
/collect?e={%22k%22:%22a\%22b%22}

They should decode. Spellings that browsers never produce stay rejected, for example http:\\user:pass@host/ and a tab inside the scheme.

Change and scope

A request URL is admitted when the pinned RFC 3986 check accepts it either as written or after percent-encoding only these characters:
- each [ ] { } | ^ and backtick;
- each % not followed by two hex digits;
- each backslash after the first ? or #.
A backslash before the first ? or #, and a tab, stay rejected.
The original spelling is kept in the decoded output.
The transport-credential header set moves unchanged from history/CaptureRedaction.ts to src/browser/network/TransportCredentials.ts. capture_browser_scenario's network headers use it instead of a private copy, and the session HAR recorder (#2045) uses the same set. URL userinfo stays with src/domain/recordedTransportUrl.ts.

Contract and boundary impact

Semantic owner and earliest changed stage: src/browser/history/HarSchema.ts.
CLI and MCP/tool-catalog contract: none. More HAR files are accepted, and none that were accepted before is rejected.
Provider, bridge, target-format, or platform compatibility: HAR 1.2 input only.
Evidence, artifact, provenance, or reconstruction contract: none. Request URLs are reported as written.
Process execution, authorization, cleanup, or containment impact: none.
Generated metadata (docs/public/product-catalog.json), package, or installation impact: none.

Evidence and regression coverage

Tests added or updated: tests/boundary/browser/historicalCaptureDecoder.test.ts runs the real decodeHarCapture.
- Admitted: an RFC-only URL (http://example.com:99999/), every browser serialization above, a % followed by one hex digit and a non-hex character (?status=100%Done), and a fragment backslash. Each case also checks that the URL is its own WHATWG serialization and that the reported URL keeps its spelling.
- Still rejected: backslash and tab credential spellings, each checked to carry a WHATWG password; a path backslash; and a non-URL.
- PlaywrightScenarioNetwork.test.ts pins every member of the shared credential header set.
Base reproduction: on main every admitted browser serialization above fails with format.
Mutation checks: each of these fails a test.
- Dropping the lone-% rule.
- Dropping the backslash rule.
- Encoding every backslash.
- Ignoring #.
- Encoding backslashes before ?.
- Reverting to the strict check alone.
Remaining proof gaps: none known for the listed spellings.

For evidence-bearing changes:

Observed, derived, and inferred claims remain distinguishable.
Artifact identity, source provenance, and failed attempts remain preserved.
Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

Host: macOS 27.0.1, Node 24.20.0.

turbo run typecheck lint format:check knip metadata:check docs:check:built mcp-catalog:check:built, npm run verify:architecture-guards, npm run verify:test-discovery and npx jscpd src/ tests/ all pass.
npx vitest run on 10 browser, CLI, MCP and process boundary files: 58 passed, 2 skipped.
exposes CLI-equivalent Evidence results and session tools, an existing MCP test, times out only under heavy machine load. It fails the same way on unmodified main.
Cold review with adversarial verification: the final round reported no findings.

Compatibility, safety, and release

Breaking changes or migration steps: none.
Package or release metadata impact: none.
Security, privacy, process, or containment review: the widening is limited to characters that never end a URL authority, and recordedTransportUrl still excludes userinfo from every admitted URL.

This PR stands on its own on main. The session HAR PRs (#2045 onward) build on it.

🤖 Generated with Claude Code

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/2044 **Author:** [@Hyped01](https://github.com/Hyped01) **Created:** 10/12/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `fix/har-request-url-query-characters` --- ### 📝 Commits (1) - [`6b912ce`](https://github.com/morluto/rea/commit/6b912ce5b90e9b4a395570af0060c5efee9ab44a) fix(browser): admit browser-serialized HAR request URLs ### 📊 Changes **7 files changed** (+133 additions, -16 deletions) <details> <summary>View changed files</summary> 📝 `docs/web-network-captures.md` (+6 -0) 📝 `src/browser/history/CaptureRedaction.ts` (+2 -7) 📝 `src/browser/history/HarSchema.ts` (+28 -1) 📝 `src/browser/network/PlaywrightNetworkContent.ts` (+2 -8) 📝 `src/browser/network/PlaywrightScenarioNetwork.test.ts` (+44 -0) ➕ `src/browser/network/TransportCredentials.ts` (+7 -0) 📝 `tests/boundary/browser/historicalCaptureDecoder.test.ts` (+44 -0) </details> ### 📄 Description ## Summary `inspect_web_network_capture` validates each HAR `request.url` with ajv-formats' strict RFC 3986 `uri` format. Browsers serialize request URLs per WHATWG URL and leave several characters unescaped: `[ ] { } | ^` and backtick, a `%` that starts no escape, and a backslash in the query or fragment. A HAR that Chrome itself exports for such a page is therefore rejected whole as a format error. ## Problem and expected behavior On `main`, `decodeHarCapture` rejects every one of these browser serializations with `HAR schema validation failed: format.`: - `https://example.test/?q=[1]|{x}` - `/search?q=100%` - `/x?cb=%%CACHEBUSTER%%` - `/collect?e={%22k%22:%22a\%22b%22}` They should decode. Spellings that browsers never produce stay rejected, for example `http:\\user:pass@host/` and a tab inside the scheme. ## Change and scope - A request URL is admitted when the pinned RFC 3986 check accepts it either as written or after percent-encoding only these characters: - each `[ ] { } | ^` and backtick; - each `%` not followed by two hex digits; - each backslash after the first `?` or `#`. - A backslash before the first `?` or `#`, and a tab, stay rejected. - The original spelling is kept in the decoded output. - The transport-credential header set moves unchanged from `history/CaptureRedaction.ts` to `src/browser/network/TransportCredentials.ts`. `capture_browser_scenario`'s network headers use it instead of a private copy, and the session HAR recorder (#2045) uses the same set. URL userinfo stays with `src/domain/recordedTransportUrl.ts`. ## Contract and boundary impact - Semantic owner and earliest changed stage: `src/browser/history/HarSchema.ts`. - CLI and MCP/tool-catalog contract: none. More HAR files are accepted, and none that were accepted before is rejected. - Provider, bridge, target-format, or platform compatibility: HAR 1.2 input only. - Evidence, artifact, provenance, or reconstruction contract: none. Request URLs are reported as written. - Process execution, authorization, cleanup, or containment impact: none. - Generated metadata (`docs/public/product-catalog.json`), package, or installation impact: none. ## Evidence and regression coverage - Tests added or updated: `tests/boundary/browser/historicalCaptureDecoder.test.ts` runs the real `decodeHarCapture`. - Admitted: an RFC-only URL (`http://example.com:99999/`), every browser serialization above, a `%` followed by one hex digit and a non-hex character (`?status=100%Done`), and a fragment backslash. Each case also checks that the URL is its own WHATWG serialization and that the reported URL keeps its spelling. - Still rejected: backslash and tab credential spellings, each checked to carry a WHATWG password; a path backslash; and a non-URL. - `PlaywrightScenarioNetwork.test.ts` pins every member of the shared credential header set. - Base reproduction: on `main` every admitted browser serialization above fails with `format`. - Mutation checks: each of these fails a test. - Dropping the lone-`%` rule. - Dropping the backslash rule. - Encoding every backslash. - Ignoring `#`. - Encoding backslashes before `?`. - Reverting to the strict check alone. - Remaining proof gaps: none known for the listed spellings. For evidence-bearing changes: - [x] Observed, derived, and inferred claims remain distinguishable. - [x] Artifact identity, source provenance, and failed attempts remain preserved. - [x] Unsupported, incomplete, unavailable, or uncertain outcomes remain visible. ## Validation performed Host: macOS 27.0.1, Node 24.20.0. - `turbo run typecheck lint format:check knip metadata:check docs:check:built mcp-catalog:check:built`, `npm run verify:architecture-guards`, `npm run verify:test-discovery` and `npx jscpd src/ tests/` all pass. - `npx vitest run` on 10 browser, CLI, MCP and process boundary files: 58 passed, 2 skipped. - `exposes CLI-equivalent Evidence results and session tools`, an existing MCP test, times out only under heavy machine load. It fails the same way on unmodified `main`. - Cold review with adversarial verification: the final round reported no findings. ## Compatibility, safety, and release - Breaking changes or migration steps: none. - Package or release metadata impact: none. - Security, privacy, process, or containment review: the widening is limited to characters that never end a URL authority, and `recordedTransportUrl` still excludes userinfo from every admitted URL. This PR stands on its own on `main`. The session HAR PRs (#2045 onward) build on it. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-31 00:57:40 +00:00