morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[PR #2340] [MERGED] feat(pcap): add bounded capture summaries through CLI and MCP #5174

New issue

Closed

opened 2026-10-31 01:04:32 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 01:04:32 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/2340
Author: @3QL
Created: 10/12/2026
Status: ✅ Merged
Merged: 10/13/2026
Merged by: @N0zoM1z0

Base: main ← Head: feat/pcap-analysis

📝 Commits (9)

3f2874b feat(pcap): inspect pcap/pcapng captures through caller-supplied tshark
b8f8341 Merge branch 'morluto:main' into feat/pcap-analysis
1530978 Merge branch 'morluto:main' into feat/pcap-analysis
66952e0 Merge branch 'morluto:main' into feat/pcap-analysis
31061c8 fix(pcap): bind analysis to owned captures and validate evidence
dd760ce fix(pcap): verify native platform startup and cleanup
3e8a438 chore(pcap): sync upstream main and retain native startup bounds
df63a4c fix(pcap): bound native analysis and page complete retained facts
ae88bdb chore: reconcile packet and Dart AOT retained views

📊 Changes

69 files changed (+4082 additions, -28 deletions)

View changed files

📝 .agents/skills/reverse-engineer-anything/SKILL.md (+6 -0)
📝 .github/workflows/real-packet-capture.yml (+8 -0)
📝 README.md (+1 -0)
📝 README_ar.md (+1 -0)
📝 README_de.md (+1 -0)
📝 README_es.md (+1 -0)
📝 README_fa.md (+1 -0)
📝 README_fr.md (+1 -0)
📝 README_id.md (+1 -0)
📝 README_it.md (+1 -0)
📝 README_ja.md (+1 -0)
📝 README_ko.md (+1 -0)
📝 README_pl.md (+1 -0)
📝 README_pt-BR.md (+1 -0)
📝 README_ru.md (+1 -0)
📝 README_th.md (+1 -0)
📝 README_tr.md (+1 -0)
📝 README_uk.md (+1 -0)
📝 README_vi.md (+1 -0)
📝 README_zh-TW.md (+1 -0)

...and 49 more files

📄 Description

REA now summarizes saved pcap/pcapng files through the existing local CLI/MCP model, using caller-selected Wireshark 4.x tshark and capinfos. It derives address/protocol counts and same-response DNS associations, preserving recorded SNI/URI values and unresolved attribution. Existing inspect_packet_capture remains the field/filter/decode-as workflow.

The follow-up enforces native memory limits (Linux prlimit with stricter inherited ceilings; Windows owned Job Objects), serial admission and close draining. Commands use an empty private personal profile and owned temporary paths. Both tools read one streamed stable snapshot, hashed again after analysis. Cleanup failure now returns a typed error with completed Evidence where available; resources stay owned for retry. One domain owner creates both successful/partial historical Evidence.

Arbitrary 200/100/50-item truncation was removed. Complete inventories remain bounded by real output/graph/retained-memory budgets. Opt-in detail summary retains complete Evidence once, and kind pcap views page endpoints, DNS questions/addresses, SNI, protocol stacks, URIs, unattributed addresses or diagnostics. Pages preserve source coverage, work after source deletion and run no additional decoder. URI credentials remain excluded; metadata and stderr remain useful retained evidence.

Validation on ae88bdbcf8:

Full check, generated docs, duplication gate (2.98%) and test discovery passed; all 1,018 tests were discovered once. 64 focused checks passed with eight explicit platform skips. After current-main conflict resolution, 26 interface/native regression checks passed, including Dart AOT summaries.
Production-boundary CLI/stdio SDK tests cover all facets, late DNS/SNI rows, strict schemas, portable views after source deletion, original-source/private-snapshot mutation, cancellation, concurrent admission/close draining and failed cleanup retry/partial Evidence.
Real data: independent Scapy raw-frame/IP parsing plus Python aggregation of separately invoked TShark fields matched six PCAP/PCAPNG corpora totaling 13,026 packets. Includes actual curl HTTP traffic (12,023 packets/1,000 distinct URIs), 60 certificate-verified HTTPS requests/SNI names and one actual 132-address DNS response. All source hashes, CLI/MCP Evidence IDs/results, SDK schemas and retained pages passed on the final head.
The existing real TShark CI workflow now also exercises this family. Its 17 native verifier cases pass locally, including PCAP summary/retained protocol pages. These CI format fixtures are synthetic and do not establish capture provenance.

Performance: same final binary, three alternating pairs, fresh MCP sessions and warm filesystem. Task: final ten HTTP URIs plus exact packet/URI/endpoint counts from the 12,023-packet/1,000-URI corpus. Complete replies and summary/retained pages returned identical task facts. Estimated o200k task request/result text fell from 20,856 to 2,544 tokens; including the full advertised input catalog, 154,364 to 136,052. Median wall time was 10.228s versus 10.532s; CPU 11.898s versus 11.941s; sampled family peak 917.21 versus 894.64 MiB. Retained pages took 61–82ms without another decoder. This demonstrates smaller returned context, not a general speedup or actual agent billing; host concurrency and source-cache state remain limits of this local comparison.

Scope: offline caller-selected recordings only; no live acquisition, replay, automatic target requests, decrypted-body recovery, process attribution or network-completeness claim. Capture metadata/names are producer observations; aggregation is derived and Evidence has historical-reference authority. Current real proof is Linux/Node 24.18.0/Wireshark 4.0.17. Actual Windows/Bun Wireshark execution is unverified here; macOS capture inspection returns a capability gap until an enforceable native memory boundary is verified. Version probes remain available. Original other-platform reports predate these changes.

All heavy local work used two CPU cores and an 8 GiB family guard; supplementary privileged loopback data acquisition additionally used a 512 MiB native address-space limit. No OOM or guard stop occurred. Captures, certificates, keys, binaries and private benchmark artifacts are not committed.

Original implementation by @3QL; follow-up review, fixes and real-data verification by @N0zoM1z0. CI remains the final merge gate.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/2340 **Author:** [@3QL](https://github.com/3QL) **Created:** 10/12/2026 **Status:** ✅ Merged **Merged:** 10/13/2026 **Merged by:** [@N0zoM1z0](https://github.com/N0zoM1z0) **Base:** `main` ← **Head:** `feat/pcap-analysis` --- ### 📝 Commits (9) - [`3f2874b`](https://github.com/morluto/rea/commit/3f2874b9659176dab24e486372d03eaf0be5bd33) feat(pcap): inspect pcap/pcapng captures through caller-supplied tshark - [`b8f8341`](https://github.com/morluto/rea/commit/b8f83410347aa94cbce1e910daa82280f13cd23f) Merge branch 'morluto:main' into feat/pcap-analysis - [`1530978`](https://github.com/morluto/rea/commit/1530978b4c8deea7e53eb286266976b4f3522755) Merge branch 'morluto:main' into feat/pcap-analysis - [`66952e0`](https://github.com/morluto/rea/commit/66952e010ed05b5f93803e2f81356362fede18bf) Merge branch 'morluto:main' into feat/pcap-analysis - [`31061c8`](https://github.com/morluto/rea/commit/31061c83378e5dab9c7b499304a28df09f3f2b98) fix(pcap): bind analysis to owned captures and validate evidence - [`dd760ce`](https://github.com/morluto/rea/commit/dd760ce33676dc3a5a40102159a39ad7d63680ae) fix(pcap): verify native platform startup and cleanup - [`3e8a438`](https://github.com/morluto/rea/commit/3e8a43881c887cd021868d2e0db906b190405482) chore(pcap): sync upstream main and retain native startup bounds - [`df63a4c`](https://github.com/morluto/rea/commit/df63a4cb25c7dff716196fc6df3ff99aa88065dd) fix(pcap): bound native analysis and page complete retained facts - [`ae88bdb`](https://github.com/morluto/rea/commit/ae88bdbcf86919be7bba996934b31151649b2c5e) chore: reconcile packet and Dart AOT retained views ### 📊 Changes **69 files changed** (+4082 additions, -28 deletions) <details> <summary>View changed files</summary> 📝 `.agents/skills/reverse-engineer-anything/SKILL.md` (+6 -0) 📝 `.github/workflows/real-packet-capture.yml` (+8 -0) 📝 `README.md` (+1 -0) 📝 `README_ar.md` (+1 -0) 📝 `README_de.md` (+1 -0) 📝 `README_es.md` (+1 -0) 📝 `README_fa.md` (+1 -0) 📝 `README_fr.md` (+1 -0) 📝 `README_id.md` (+1 -0) 📝 `README_it.md` (+1 -0) 📝 `README_ja.md` (+1 -0) 📝 `README_ko.md` (+1 -0) 📝 `README_pl.md` (+1 -0) 📝 `README_pt-BR.md` (+1 -0) 📝 `README_ru.md` (+1 -0) 📝 `README_th.md` (+1 -0) 📝 `README_tr.md` (+1 -0) 📝 `README_uk.md` (+1 -0) 📝 `README_vi.md` (+1 -0) 📝 `README_zh-TW.md` (+1 -0) _...and 49 more files_ </details> ### 📄 Description REA now summarizes saved pcap/pcapng files through the existing local CLI/MCP model, using caller-selected Wireshark 4.x tshark and capinfos. It derives address/protocol counts and same-response DNS associations, preserving recorded SNI/URI values and unresolved attribution. Existing inspect_packet_capture remains the field/filter/decode-as workflow. The follow-up enforces native memory limits (Linux prlimit with stricter inherited ceilings; Windows owned Job Objects), serial admission and close draining. Commands use an empty private personal profile and owned temporary paths. Both tools read one streamed stable snapshot, hashed again after analysis. Cleanup failure now returns a typed error with completed Evidence where available; resources stay owned for retry. One domain owner creates both successful/partial historical Evidence. Arbitrary 200/100/50-item truncation was removed. Complete inventories remain bounded by real output/graph/retained-memory budgets. Opt-in detail summary retains complete Evidence once, and kind pcap views page endpoints, DNS questions/addresses, SNI, protocol stacks, URIs, unattributed addresses or diagnostics. Pages preserve source coverage, work after source deletion and run no additional decoder. URI credentials remain excluded; metadata and stderr remain useful retained evidence. Validation on ae88bdbcf86919be7bba996934b31151649b2c5e: - Full check, generated docs, duplication gate (2.98%) and test discovery passed; all 1,018 tests were discovered once. 64 focused checks passed with eight explicit platform skips. After current-main conflict resolution, 26 interface/native regression checks passed, including Dart AOT summaries. - Production-boundary CLI/stdio SDK tests cover all facets, late DNS/SNI rows, strict schemas, portable views after source deletion, original-source/private-snapshot mutation, cancellation, concurrent admission/close draining and failed cleanup retry/partial Evidence. - Real data: independent Scapy raw-frame/IP parsing plus Python aggregation of separately invoked TShark fields matched six PCAP/PCAPNG corpora totaling 13,026 packets. Includes actual curl HTTP traffic (12,023 packets/1,000 distinct URIs), 60 certificate-verified HTTPS requests/SNI names and one actual 132-address DNS response. All source hashes, CLI/MCP Evidence IDs/results, SDK schemas and retained pages passed on the final head. - The existing real TShark CI workflow now also exercises this family. Its 17 native verifier cases pass locally, including PCAP summary/retained protocol pages. These CI format fixtures are synthetic and do not establish capture provenance. Performance: same final binary, three alternating pairs, fresh MCP sessions and warm filesystem. Task: final ten HTTP URIs plus exact packet/URI/endpoint counts from the 12,023-packet/1,000-URI corpus. Complete replies and summary/retained pages returned identical task facts. Estimated o200k task request/result text fell from 20,856 to 2,544 tokens; including the full advertised input catalog, 154,364 to 136,052. Median wall time was 10.228s versus 10.532s; CPU 11.898s versus 11.941s; sampled family peak 917.21 versus 894.64 MiB. Retained pages took 61–82ms without another decoder. This demonstrates smaller returned context, not a general speedup or actual agent billing; host concurrency and source-cache state remain limits of this local comparison. Scope: offline caller-selected recordings only; no live acquisition, replay, automatic target requests, decrypted-body recovery, process attribution or network-completeness claim. Capture metadata/names are producer observations; aggregation is derived and Evidence has historical-reference authority. Current real proof is Linux/Node 24.18.0/Wireshark 4.0.17. Actual Windows/Bun Wireshark execution is unverified here; macOS capture inspection returns a capability gap until an enforceable native memory boundary is verified. Version probes remain available. Original other-platform reports predate these changes. All heavy local work used two CPU cores and an 8 GiB family guard; supplementary privileged loopback data acquisition additionally used a 512 MiB native address-space limit. No OOM or guard stop occurred. Captures, certificates, keys, binaries and private benchmark artifacts are not committed. Original implementation by @3QL; follow-up review, fixes and real-data verification by @N0zoM1z0. CI remains the final merge gate. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-31 01:04:32 +00:00

closed this issue
added the
pull-request
label

bsakaguchi referenced this issue

2026-11-01 21:43:08 +00:00

[PR #5174] fix(browser): enforce web bundle result acceptance deadlines #6802

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#5174

No description provided.

Rows
Columns