morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #2915] [Feature] Derive API surfaces from retained native mitmproxy flows #1026

New issue

Open

opened 2026-10-15 18:30:27 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-15 18:30:27 +00:00

Owner

Originally created by @zhangguiping-xydt on GitHub (Oct 13, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/2915

Problem

REA already inspects native mitmproxy saved flows, but its retained api-surface view rejects them. Analysts must export/convert to HAR or aggregate native flow facts manually to answer which HTTP endpoints were recorded.

Desired outcome and success criteria

Admit supported retained native mitmproxy HTTP flow Evidence to the existing API-surface workflow.
Interpret the existing producer-owned request/response facts and exact byte sidecars at a format-aware boundary, preserving missing or unreadable fields as unknown/excluded.
Reuse the existing method/origin/conservative-path grouping, response status/MIME summaries, record references, budgets and paging.
Exclude non-HTTP flows and unsupported request targets explicitly; do not infer HTTP URLs from arbitrary TCP/UDP messages or guess origins for ambiguous requests.
Preserve transport authentication exclusions and explicit-sensitive-value refusals.
Work through retained and portable CLI/MCP evidence after the selected saved-flow file is removed, without invoking mitmdump again.

Expected interaction with targets

The added projection reads retained captured data only. The existing initial inspection invokes the caller-supplied pinned mitmdump decoder offline with its existing private workspace and cleanup lifecycle. This feature adds no live proxy, trust-store change, target traffic or replay.

Verification available so far

Source inspection of main 1b6c899: format admission accepts HAR only. SAZ extension #2857 / #2884 is separate. Existing native-preview PR #2876 changes headline delivery, not API-surface support.

Proposed checks: real mitmproxy 12.2.3 recorded flow artifact and compiled CLI/stdio MCP projection; HTTP URL components and exact text/byte identity; missing responses; duplicate Content-Type; mixed flow types; credential/sensitive exclusions; paging and source deletion; existing HAR behavior and resource limits.

Alternatives considered

HAR conversion can discard native fields and requires an extra step even though REA has retained the decoded flow evidence. This proposal reuses the current decoder and grouping semantics rather than adding a new provider.

Implementation available

Implementation: github.com/zhangguiping-xydt/rea@75b2f3d6e3

The branch has passed focused tests, static checks, documentation checks and exact-commit compiled CLI / production stdio MCP verification using real locally recorded traffic. Original working capture files were removed before retained-Evidence projection/comparison; provenance and paging were checked.

Originally created by @zhangguiping-xydt on GitHub (Oct 13, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/2915 ## Problem REA already inspects native mitmproxy saved flows, but its retained `api-surface` view rejects them. Analysts must export/convert to HAR or aggregate native flow facts manually to answer which HTTP endpoints were recorded. ## Desired outcome and success criteria - Admit supported retained native mitmproxy HTTP flow Evidence to the existing API-surface workflow. - Interpret the existing producer-owned request/response facts and exact byte sidecars at a format-aware boundary, preserving missing or unreadable fields as unknown/excluded. - Reuse the existing method/origin/conservative-path grouping, response status/MIME summaries, record references, budgets and paging. - Exclude non-HTTP flows and unsupported request targets explicitly; do not infer HTTP URLs from arbitrary TCP/UDP messages or guess origins for ambiguous requests. - Preserve transport authentication exclusions and explicit-sensitive-value refusals. - Work through retained and portable CLI/MCP evidence after the selected saved-flow file is removed, without invoking mitmdump again. ## Expected interaction with targets The added projection reads retained captured data only. The existing initial inspection invokes the caller-supplied pinned mitmdump decoder offline with its existing private workspace and cleanup lifecycle. This feature adds no live proxy, trust-store change, target traffic or replay. ## Verification available so far Source inspection of main `1b6c899`: [format admission](https://github.com/morluto/rea/blob/1b6c899b031421dcf2d725a63e67d99375028d6e/src/domain/analysisView/webNetworkApiSurfaceView.ts#L100) accepts HAR only. SAZ extension #2857 / #2884 is separate. Existing native-preview PR #2876 changes headline delivery, not API-surface support. Proposed checks: real mitmproxy 12.2.3 recorded flow artifact and compiled CLI/stdio MCP projection; HTTP URL components and exact text/byte identity; missing responses; duplicate Content-Type; mixed flow types; credential/sensitive exclusions; paging and source deletion; existing HAR behavior and resource limits. ## Alternatives considered HAR conversion can discard native fields and requires an extra step even though REA has retained the decoded flow evidence. This proposal reuses the current decoder and grouping semantics rather than adding a new provider. ### Implementation available Implementation: https://github.com/zhangguiping-xydt/rea/commit/75b2f3d6e34e669df56560aa3e1c86a9a9625cea The branch has passed focused tests, static checks, documentation checks and exact-commit compiled CLI / production stdio MCP verification using real locally recorded traffic. Original working capture files were removed before retained-Evidence projection/comparison; provenance and paging were checked.

bsakaguchi added the

area: native

enhancement

labels

2026-10-15 18:30:27 +00:00

bsakaguchi referenced this issue

2026-10-16 21:10:15 +00:00

[GH-ISSUE #4504] [Bug] Windows snapshot disposal failure replaces the original native failure #1787

bsakaguchi referenced this issue

2026-10-17 21:07:57 +00:00

[PR #1027] [MERGED] fix(windows): allow Java to read retained target snapshots #2591

bsakaguchi referenced this issue

2026-10-17 21:07:58 +00:00

[PR #1030] [MERGED] feat(ghidra): support native x86 PE applications on Windows #2594

bsakaguchi added

enhancement

and removed

enhancement

labels

2026-10-31 00:29:51 +00:00

bsakaguchi commented

2026-10-31 00:29:51 +00:00

Author

Owner

@matbrgz commented on GitHub (Oct 19, 2026):

Cross-referencing with sibling proposal #2857 ([Feature] Derive API surfaces from retained Fiddler SAZ captures). Both proposals target proxy/capture format ingestion for API surface and schema reconstruction.

Consolidated under Cluster AA (HTTP & Proxy Capture API Surface Reconstruction: Fiddler SAZ & mitmproxy) in tracker #4330.

@matbrgz commented on GitHub (Oct 19, 2026): Cross-referencing with sibling proposal #2857 (`[Feature] Derive API surfaces from retained Fiddler SAZ captures`). Both proposals target proxy/capture format ingestion for API surface and schema reconstruction. Consolidated under **Cluster AA (HTTP & Proxy Capture API Surface Reconstruction: Fiddler SAZ & mitmproxy)** in tracker #4330.

bsakaguchi referenced this issue

2026-10-31 00:48:17 +00:00

[GH-ISSUE #7075] Windows: Ghidra provider fails to start - LaunchSupport cannot write java_home.save (access denied) #4264

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#1026

No description provided.

Rows
Columns