morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #4398] [Feature] Audit an exported Evidence bundle for secrets, local paths, and target content before sharing #1750

New issue

Open

opened 2026-10-16 21:09:49 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-16 21:09:49 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 16, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/4398

Before submitting

I searched the open and closed issues and pull requests for a related proposal.
I understand the contribution scope must be reviewed before integration; open questions are welcome here.
This request does not include credentials, private samples, or sensitive target data.

Area

Evidence, artifacts, or reconstruction

Current limitation

export_evidence_bundle takes only path and overwrite and writes the session's sealed records unchanged (src/contracts/sessionToolContracts.ts, src/application/EvidenceBundleCommands.ts). Records can contain artifact-path locations with absolute local paths (src/domain/evidence.ts), environment details, and free-form raw_result and normalized_result payloads that may include target strings, pseudocode, or bytes.

REA does redact some sources at capture time: browser and HAR captures drop authorization, proxy-authorization, cookie, and set-cookie headers and caller-declared sensitive values (src/browser/history/CaptureRedaction.ts, docs/web-network-captures.md), and process scenarios redact inputs marked sensitive. But docs/web-network-captures.md also states that "Unknown extension names and token-looking query parameters do not establish sensitivity", and there is no pattern-based secret detection, no path check, and no check for verbatim target content anywhere in the export path.

Guidance puts the burden on the user without tooling: DISCLAIMER.md says users are "responsible for protecting credentials, personal data, captured traffic, and investigation artifacts", and lists "copying proprietary software or reproducing its protected code and assets" as unsupported; SECURITY.md asks reporters to remove "socket capability tokens, and unrelated local paths"; website/AGENTS.md keeps "personal paths, account data, credentials and raw captures outside the public assets" as a manual rule.

Use cases

Attaching a bundle to a REA bug report, where the issue templates ask reporters to remove credentials, private paths, and sensitive target data.
Publishing a website evidence note or showcase derived from a bundle.
Handing a bundle to a colleague outside the machine where it was captured.

Desired outcome and success criteria

Add a read-only audit for an exported Evidence bundle that reports likely credentials, absolute local paths and host identifiers, and verbatim target content (decompiled code, embedded target bytes) by Evidence ID and JSON location, without printing the matched values, so a user can decide what is safe to attach to an issue, report, or website before sharing it.

A read-only CLI command and MCP tool take a bundle path and return findings, each with Evidence ID, JSON pointer, rule ID, and severity. Matched values are never printed; at most a length and a short hash prefix.
Failures: credential patterns (cloud and API keys, bearer tokens, JWTs, PEM private keys), authorization or cookie header values that escaped capture-time redaction, and caller-declared sensitive values.
Warnings: absolute home-directory paths on macOS, Linux, and Windows (including artifact-path locations), local host and user names, environment values in process records, and decompiled or pseudocode payloads.
Optionally, given the original artifact, flag embedded runs of target bytes above a size threshold.
Exit status is nonzero when failures exist; output ordering is deterministic; bundle size and scan time are bounded and limitations are reported when a bound is hit.
The audit never modifies the bundle. If a later redacted-export mode is added, it writes a new bundle with recomputed Evidence IDs and an old-to-new ID map, because evidence_id is a content digest and in-place redaction would break integrity and evidence_links.
Documentation describes running the audit before sharing a bundle.

Expected interaction with targets

Read existing files or captured data

Target and scope details (optional)

Reads an existing bundle file and, optionally, the original artifact; no target execution, network access, or writes. Non-goals: guaranteeing that a clean audit means a bundle is safe to publish; automatic redaction in this first slice.

Verification available so far

I can suggest a sample or manual check

Verification details (optional)

Possible fixture: a bundle containing a synthetic PEM key in a raw_result, a /home/<user>/ artifact path, and a pseudocode result. Expected: one failure and two warnings at the right JSON pointers, with no matched value in the output.

Alternatives considered

Running a general secret scanner (for example gitleaks) over the JSON: finds some credentials but cannot map findings to Evidence IDs, does not know which fields hold paths or pseudocode, and ignores REA's record structure.
Extending capture-time redaction: still needed per source, but cannot catch values from providers that do not redact or from caller-supplied inputs.

Possible direction

No response

References

Checked against main at commit 8ab5024 (rea-agents 6.5.0).
Adjacent capture-time work: #3592, PR #2872, PR #3994.

Originally created by @coygeek on GitHub (Oct 16, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/4398 ### Before submitting - [x] I searched the open and closed issues and pull requests for a related proposal. - [x] I understand the [contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope) must be reviewed before integration; open questions are welcome here. - [x] This request does not include credentials, private samples, or sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Current limitation `export_evidence_bundle` takes only `path` and `overwrite` and writes the session's sealed records unchanged (`src/contracts/sessionToolContracts.ts`, `src/application/EvidenceBundleCommands.ts`). Records can contain `artifact-path` locations with absolute local paths (`src/domain/evidence.ts`), environment details, and free-form `raw_result` and `normalized_result` payloads that may include target strings, pseudocode, or bytes. REA does redact some sources at capture time: browser and HAR captures drop `authorization`, `proxy-authorization`, `cookie`, and `set-cookie` headers and caller-declared sensitive values (`src/browser/history/CaptureRedaction.ts`, `docs/web-network-captures.md`), and process scenarios redact inputs marked sensitive. But `docs/web-network-captures.md` also states that "Unknown extension names and token-looking query parameters do not establish sensitivity", and there is no pattern-based secret detection, no path check, and no check for verbatim target content anywhere in the export path. Guidance puts the burden on the user without tooling: `DISCLAIMER.md` says users are "responsible for protecting credentials, personal data, captured traffic, and investigation artifacts", and lists "copying proprietary software or reproducing its protected code and assets" as unsupported; `SECURITY.md` asks reporters to remove "socket capability tokens, and unrelated local paths"; `website/AGENTS.md` keeps "personal paths, account data, credentials and raw captures outside the public assets" as a manual rule. ### Use cases - Attaching a bundle to a REA bug report, where the issue templates ask reporters to remove credentials, private paths, and sensitive target data. - Publishing a website evidence note or showcase derived from a bundle. - Handing a bundle to a colleague outside the machine where it was captured. ### Desired outcome and success criteria Add a read-only audit for an exported Evidence bundle that reports likely credentials, absolute local paths and host identifiers, and verbatim target content (decompiled code, embedded target bytes) by Evidence ID and JSON location, without printing the matched values, so a user can decide what is safe to attach to an issue, report, or website before sharing it. - A read-only CLI command and MCP tool take a bundle path and return findings, each with Evidence ID, JSON pointer, rule ID, and severity. Matched values are never printed; at most a length and a short hash prefix. - Failures: credential patterns (cloud and API keys, bearer tokens, JWTs, PEM private keys), authorization or cookie header values that escaped capture-time redaction, and caller-declared sensitive values. - Warnings: absolute home-directory paths on macOS, Linux, and Windows (including `artifact-path` locations), local host and user names, environment values in process records, and decompiled or pseudocode payloads. - Optionally, given the original artifact, flag embedded runs of target bytes above a size threshold. - Exit status is nonzero when failures exist; output ordering is deterministic; bundle size and scan time are bounded and limitations are reported when a bound is hit. - The audit never modifies the bundle. If a later redacted-export mode is added, it writes a new bundle with recomputed Evidence IDs and an old-to-new ID map, because `evidence_id` is a content digest and in-place redaction would break integrity and `evidence_links`. - Documentation describes running the audit before sharing a bundle. ### Expected interaction with targets Read existing files or captured data ### Target and scope details (optional) Reads an existing bundle file and, optionally, the original artifact; no target execution, network access, or writes. Non-goals: guaranteeing that a clean audit means a bundle is safe to publish; automatic redaction in this first slice. ### Verification available so far I can suggest a sample or manual check ### Verification details (optional) Possible fixture: a bundle containing a synthetic PEM key in a `raw_result`, a `/home/<user>/` artifact path, and a pseudocode result. Expected: one failure and two warnings at the right JSON pointers, with no matched value in the output. ### Alternatives considered - Running a general secret scanner (for example gitleaks) over the JSON: finds some credentials but cannot map findings to Evidence IDs, does not know which fields hold paths or pseudocode, and ignores REA's record structure. - Extending capture-time redaction: still needed per source, but cannot catch values from providers that do not redact or from caller-supplied inputs. ### Possible direction _No response_ ### References - Checked against `main` at commit 8ab5024 (`rea-agents` 6.5.0). - Adjacent capture-time work: #3592, PR #2872, PR #3994.

bsakaguchi commented

2026-10-16 21:09:50 +00:00

Author

Owner

@Jackzigen commented on GitHub (Oct 16, 2026):

I'd like to take this one. Planned shape, following the existing evidence-import / evidence-export / compare family:

New read-only command rea evidence-audit <bundle> (+ MCP tool) that validates and parses the bundle through the existing readEvidenceBundle path, then runs a deterministic rule scan over every record and revision.
Domain module with the rules split exactly along the severity line from the issue: failures = credential patterns (cloud/API keys, bearer tokens, JWTs, PEM private keys), authorization/cookie header values that escaped capture-time redaction, caller-declared sensitive values; warnings = absolute home-directory paths (macOS/Linux/Windows, including artifact-path locations), environment values in process records, decompiled/pseudocode payloads.
Each finding reports evidence ID, JSON pointer, rule ID, severity, matched length and a short hash prefix — never the matched value itself.
Deterministic ordering (evidence ID, pointer, rule ID); nonzero exit when failures exist; bounded scan with an explicit limitation note when a bound is hit; the bundle is never modified.

One scope question before I write code: should the optional original-artifact target-bytes check be part of this first slice, or a follow-up? The issue marks it optional, and it adds an artifact read + embed search to an otherwise bundle-only operation. My default is to leave it out of the first PR and note it in the docs, keeping the audit purely bundle-local.

@Jackzigen commented on GitHub (Oct 16, 2026): I'd like to take this one. Planned shape, following the existing `evidence-import` / `evidence-export` / `compare` family: - New read-only command `rea evidence-audit <bundle>` (+ MCP tool) that validates and parses the bundle through the existing `readEvidenceBundle` path, then runs a deterministic rule scan over every record and revision. - Domain module with the rules split exactly along the severity line from the issue: **failures** = credential patterns (cloud/API keys, bearer tokens, JWTs, PEM private keys), authorization/cookie header values that escaped capture-time redaction, caller-declared sensitive values; **warnings** = absolute home-directory paths (macOS/Linux/Windows, including `artifact-path` locations), environment values in process records, decompiled/pseudocode payloads. - Each finding reports evidence ID, JSON pointer, rule ID, severity, matched length and a short hash prefix — never the matched value itself. - Deterministic ordering (evidence ID, pointer, rule ID); nonzero exit when failures exist; bounded scan with an explicit limitation note when a bound is hit; the bundle is never modified. One scope question before I write code: should the optional original-artifact target-bytes check be part of this first slice, or a follow-up? The issue marks it optional, and it adds an artifact read + embed search to an otherwise bundle-only operation. My default is to leave it out of the first PR and note it in the docs, keeping the audit purely bundle-local.

bsakaguchi commented

2026-10-16 21:09:50 +00:00

Author

Owner

@Jackzigen commented on GitHub (Oct 16, 2026):

PR is up from my fork: https://github.com/morluto/rea/compare/main...Jackzigen:rea:evidence-audit-command

Covers CLI (rea evidence-audit) + MCP (audit_evidence_bundle), reports only pointers/lengths/hash-prefixes so the output is safe to paste publicly. Details in the PR description.

@Jackzigen commented on GitHub (Oct 16, 2026): PR is up from my fork: https://github.com/morluto/rea/compare/main...Jackzigen:rea:evidence-audit-command Covers CLI (`rea evidence-audit`) + MCP (`audit_evidence_bundle`), reports only pointers/lengths/hash-prefixes so the output is safe to paste publicly. Details in the PR description.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#1750

No description provided.

Rows
Columns