morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 07:47:29 +00:00

[GH-ISSUE #3592] [Feature] Report value-free request header and cookie name facets in HAR API surfaces #1335

New issue

Open

opened 2026-10-16 21:04:24 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-16 21:04:24 +00:00

Owner

Originally created by @Y0UW1SH on GitHub (Oct 14, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3592

Before submitting

I searched the open and closed issues and pull requests for a related proposal.
I understand the contribution scope must be reviewed before integration; open questions are welcome here.
This request does not include credentials, private samples, or sensitive target data.

Area

Evidence, artifacts, or reconstruction

Current limitation

Analyst question: "Which endpoints in this capture are authenticated or signed, under which header and cookie names, and which headers change on every request (signature, nonce, timestamp), so I know what code to trace?"

Source-derived (commit 1579b68):

The api-surface view copies no headers: "Query strings, fragments, credentials, headers and bodies are not copied into groups" (docs). Its only opt-in facet is include_query_parameter_names (#2982).
Retained HAR Evidence keeps header names and non-credential values verbatim. Authorization, Proxy-Authorization, Cookie and Set-Cookie values and every cookies[].value are nulled as transport-credential (CaptureRedaction.harCredential), so the scheme token (e.g. Bearer) is gone after inspection.
The workaround is paging capture views with detail: "complete" and comparing values by eye, which copies header values into the agent's context.

Use cases

Web or mobile API clients that sign requests with per-request headers: find which endpoint groups carry them, then trace the producing code with trace_application_feature, trace_javascript_semantics or trace_feature.
Telling cookie-session endpoints from header-token endpoints in one captured session, without values in context.

Desired outcome and success criteria

An opt-in facet on each api-surface group lists header names, cookie names and set-cookie names with value-free variation signals:

"request_header_facets": { "scope": "script-settable", "excluded_names": 9,
  "request_headers": [
    { "name": "authorization", "present_records": 3, "distinct_values": null, "redacted": true, "variation": "unknown" },
    { "name": "x-signature", "present_records": 3, "distinct_values": 3, "value_length": { "min": 64, "max": 64 },
      "alphabets": ["hex"], "variation": "per-record-unique",
      "inferences": [{ "kind": "per-request-token-candidate", "basis": "inferred" }] } ],
  "cookie_names": [{ "name": "session", "record_ordinals": [3, 7, 11] }], "set_cookie_names": [] }

No header or cookie value, prefix, hash or substring appears in CLI or MCP output for a fixture with known values.
Omitting the option leaves api-surface output byte-identical.
Redacted credential headers report distinct_values: null; a cookie header with an empty cookies array reports names as unavailable, not empty.
Role guesses are labelled inferred and never derived from header names.

Expected interaction with targets

Read existing files or captured data

Target and scope details (optional)

Non-goals: no values or hashes emitted; no secrecy guesses from names; no live CDP headers; no SAZ or mitmproxy until #2884 / #2915; no replay, signature verification or key recovery.

Part A (the facet above) only reads retained Evidence. Part B, the auth scheme token, changes producer redaction and can be split out.

Scope question for maintainers (Part B): may inspect_web_network_capture, under an explicit opt-in, keep a fixed-vocabulary auth scheme from a redacted Authorization / Proxy-Authorization value (an IANA-registered name such as Bearer, otherwise unregistered)? RFC 9110 treats the whole field value as credentials, and AGENTS.md requires transport credentials to be redacted. If declined, Part A ships alone and reports only that authorization was present.

Verification available so far

I can suggest a sample or manual check

Verification details (optional)

Proposed, not run: mitmproxy-authored HAR records in the existing real web-network-captures lane carrying Authorization: Bearer …, a per-request X-Signature, an X-Timestamp and a cookie; run the facet through CLI and MCP and assert that no value substring appears. Boundary checks: omitted-option byte identity and the sensitive_values_unretained refusal. Part B: Bearer x, lowercase bearer, and a scheme-less sk-live-123, which must become unregistered.

Alternatives considered

capture views with complete detail: copy values into context and cost tokens.
Live inspect_web_page: keeps an allowlisted header projection only; cookies and authorization headers are never retained.
Reading the HAR outside REA: loses Evidence provenance and exposes values.

Possible direction

A proposal, not a requirement:

An opt-in view field in the merged include_* style (e.g. include_request_header_facets), with a script-settable scope that counts but drops WHATWG forbidden request-header names and HTTP/2 pseudo-headers, and an all scope for non-browser HAR.
Accumulate in WebNetworkApiSurfaceAccumulator, keeping in-memory SHA-256 digests for distinct counts under the 96 MiB projection budget; never serialize them.
Fixed inference rules (per-record-unique hex/base64 of length ≥ 16 across ≥ 3 samples; 10- or 13-digit varying decimals).
Coordinate with open #3425, which changes the same view request and accumulator.

References

https://github.com/morluto/rea/pull/1891; precedent https://github.com/morluto/rea/issues/2913 and https://github.com/morluto/rea/pull/2982 (merged; https://github.com/morluto/rea/pull/2953 closed)
Same view: https://github.com/morluto/rea/pull/3425; related https://github.com/morluto/rea/issues/2914, https://github.com/morluto/rea/pull/2884, https://github.com/morluto/rea/issues/2915
RFC 9110 §11 (https://www.rfc-editor.org/rfc/rfc9110#section-11), IANA HTTP Authentication Schemes (https://www.iana.org/assignments/http-authschemes/), WHATWG Fetch forbidden request-header (https://fetch.spec.whatwg.org/#forbidden-request-header)

Originally created by @Y0UW1SH on GitHub (Oct 14, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3592 ### Before submitting - [x] I searched the open and closed issues and pull requests for a related proposal. - [x] I understand the [contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope) must be reviewed before integration; open questions are welcome here. - [x] This request does not include credentials, private samples, or sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Current limitation Analyst question: "Which endpoints in this capture are authenticated or signed, under which header and cookie names, and which headers change on every request (signature, nonce, timestamp), so I know what code to trace?" Source-derived (commit 1579b68): - The `api-surface` view copies no headers: "Query strings, fragments, credentials, headers and bodies are not copied into groups" ([docs](https://github.com/morluto/rea/blob/1579b68/docs/web-network-captures.md#derived-api-surface)). Its only opt-in facet is `include_query_parameter_names` (#2982). - Retained HAR Evidence keeps header names and non-credential values verbatim. `Authorization`, `Proxy-Authorization`, `Cookie` and `Set-Cookie` values and every `cookies[].value` are nulled as `transport-credential` ([`CaptureRedaction.harCredential`](https://github.com/morluto/rea/blob/1579b68/src/browser/history/CaptureRedaction.ts#L71)), so the scheme token (e.g. `Bearer`) is gone after inspection. - The workaround is paging `capture` views with `detail: "complete"` and comparing values by eye, which copies header values into the agent's context. ### Use cases - Web or mobile API clients that sign requests with per-request headers: find which endpoint groups carry them, then trace the producing code with `trace_application_feature`, `trace_javascript_semantics` or `trace_feature`. - Telling cookie-session endpoints from header-token endpoints in one captured session, without values in context. ### Desired outcome and success criteria An opt-in facet on each `api-surface` group lists header names, cookie names and set-cookie names with value-free variation signals: ```json "request_header_facets": { "scope": "script-settable", "excluded_names": 9, "request_headers": [ { "name": "authorization", "present_records": 3, "distinct_values": null, "redacted": true, "variation": "unknown" }, { "name": "x-signature", "present_records": 3, "distinct_values": 3, "value_length": { "min": 64, "max": 64 }, "alphabets": ["hex"], "variation": "per-record-unique", "inferences": [{ "kind": "per-request-token-candidate", "basis": "inferred" }] } ], "cookie_names": [{ "name": "session", "record_ordinals": [3, 7, 11] }], "set_cookie_names": [] } ``` - No header or cookie value, prefix, hash or substring appears in CLI or MCP output for a fixture with known values. - Omitting the option leaves `api-surface` output byte-identical. - Redacted credential headers report `distinct_values: null`; a `cookie` header with an empty `cookies` array reports names as `unavailable`, not empty. - Role guesses are labelled `inferred` and never derived from header names. ### Expected interaction with targets Read existing files or captured data ### Target and scope details (optional) Non-goals: no values or hashes emitted; no secrecy guesses from names; no live CDP headers; no SAZ or mitmproxy until #2884 / #2915; no replay, signature verification or key recovery. Part A (the facet above) only reads retained Evidence. Part B, the auth scheme token, changes producer redaction and can be split out. **Scope question for maintainers (Part B):** may `inspect_web_network_capture`, under an explicit opt-in, keep a fixed-vocabulary auth scheme from a redacted `Authorization` / `Proxy-Authorization` value (an IANA-registered name such as `Bearer`, otherwise `unregistered`)? RFC 9110 treats the whole field value as credentials, and AGENTS.md requires transport credentials to be redacted. If declined, Part A ships alone and reports only that `authorization` was present. ### Verification available so far I can suggest a sample or manual check ### Verification details (optional) Proposed, not run: mitmproxy-authored HAR records in the existing real web-network-captures lane carrying `Authorization: Bearer …`, a per-request `X-Signature`, an `X-Timestamp` and a cookie; run the facet through CLI and MCP and assert that no value substring appears. Boundary checks: omitted-option byte identity and the `sensitive_values_unretained` refusal. Part B: `Bearer x`, lowercase `bearer`, and a scheme-less `sk-live-123`, which must become `unregistered`. ### Alternatives considered - `capture` views with complete detail: copy values into context and cost tokens. - Live `inspect_web_page`: keeps an allowlisted header projection only; cookies and authorization headers are never retained. - Reading the HAR outside REA: loses Evidence provenance and exposes values. ### Possible direction A proposal, not a requirement: - An opt-in view field in the merged `include_*` style (e.g. `include_request_header_facets`), with a `script-settable` scope that counts but drops WHATWG forbidden request-header names and HTTP/2 pseudo-headers, and an `all` scope for non-browser HAR. - Accumulate in `WebNetworkApiSurfaceAccumulator`, keeping in-memory SHA-256 digests for distinct counts under the 96 MiB projection budget; never serialize them. - Fixed inference rules (per-record-unique hex/base64 of length ≥ 16 across ≥ 3 samples; 10- or 13-digit varying decimals). - Coordinate with open #3425, which changes the same view request and accumulator. ### References - https://github.com/morluto/rea/pull/1891; precedent https://github.com/morluto/rea/issues/2913 and https://github.com/morluto/rea/pull/2982 (merged; https://github.com/morluto/rea/pull/2953 closed) - Same view: https://github.com/morluto/rea/pull/3425; related https://github.com/morluto/rea/issues/2914, https://github.com/morluto/rea/pull/2884, https://github.com/morluto/rea/issues/2915 - RFC 9110 §11 (https://www.rfc-editor.org/rfc/rfc9110#section-11), IANA HTTP Authentication Schemes (https://www.iana.org/assignments/http-authschemes/), WHATWG Fetch forbidden request-header (https://fetch.spec.whatwg.org/#forbidden-request-header)

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#1335

No description provided.

Rows
Columns