[PR #2496] [MERGED] feat(captures): discover installed protocol and field definitions #5275

Closed
opened 2026-10-31 01:05:22 +00:00 by bsakaguchi · 0 comments
Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/2496
Author: @Olorinm
Created: 10/13/2026
Status: ✅ Merged
Merged: 10/13/2026
Merged by: @baron

Base: main ← Head: feat/packet-field-discovery


📝 Commits (8)

  • 69682f3 feat(captures): discover installed protocol and field definitions
  • f67ca1a test(captures): isolate the direct glossary oracle profile
  • 730158a test(captures): accept padded kernel limit reports
  • 44899da fix(captures): detach every retained glossary column
  • df821f8 fix(captures): describe every field discovery CLI option
  • f5d9724 test(captures): verify complete field discovery coverage
  • 9ca3c87 chore: synchronize field discovery with CI fixture fixes
  • b2aff43 fix(captures): retain discovery operation across cleanup retries

📊 Changes

19 files changed (+1095 additions, -219 deletions)

View changed files

📝 .agents/skills/reverse-engineer-anything/SKILL.md (+3 -0)
📝 .github/workflows/real-packet-capture.yml (+1 -1)
📝 docs/packet-captures.md (+46 -0)
📝 scripts/verify-packet-capture.mjs (+150 -3)
📝 src/application/CapabilityInventory.ts (+1 -1)
📝 src/application/captures/PacketCaptureService.ts (+62 -8)
📝 src/captures/TsharkCaptureProvider.ts (+217 -125)
➕ src/captures/parseFieldDiscovery.ts (+92 -0)
📝 src/cli/captureCommands.ts (+67 -26)
📝 src/cliCommandNames.ts (+1 -0)
📝 src/contracts/captures/captureToolContracts.ts (+20 -0)
📝 src/contracts/toolEffects.ts (+5 -0)
➕ src/domain/captures/packetFields.ts (+100 -0)
📝 src/process/ArtifactAnalysisResources.ts (+27 -18)
📝 src/server/registerCaptureTools.ts (+13 -0)
📝 tests/boundary/captures/captureProvider.test.ts (+162 -2)
➕ tests/boundary/captures/fieldDiscovery.test.ts (+67 -0)
📝 tests/boundary/cli/captureCleanup.test.ts (+9 -0)
📝 tests/boundary/mcp/packetCaptureMcp.test.ts (+52 -35)

📄 Description

Summary

Add inspect_packet_fields / inspect-packet-fields so analysts can find the installed TShark protocol and field definitions before selecting fields for inspect_packet_capture. For example, { "kind": "fields", "protocol": "usb", "query": "idVendor" } returns usb.idVendor, its FT_UINT16 type, parent usb, display base and bitmask.

Problem and expected behavior

Capture inspection requires exact field names but currently offers no installation-specific discovery. Search protocol registrations or field definitions by literal substring, optionally selecting an exact protocol. Return 50 matches by default, offset/limit paging, explicit full output within resource limits, and examined/matching/returned counts. Duplicate abbreviations remain separate records with source line numbers. No linked issue: this is a focused follow-up to #2318.

Change and scope

Extend the existing TShark adapter and its shared CLI/MCP Evidence workflow. Reuse the same admission slot, owned process execution, executable verification, inherited Linux limits, private empty personal profile and retryable cleanup. The common resource owner accepts a per-request operation so discovery failures have the correct identity; it does not create a second queue. This small common-owner change is identical to #2487; both PRs are independently based on main.

Contract and boundary impact

  • Semantic owner: strict discovery schemas and one adapter parser for the native -G fields P/F grammar. Preserve names, types, parents, display bases, masks, descriptions and duplicate registrations.
  • Scope decision: not applicable; this extends an integrated provider's installation inspection, with no new provider or target execution/mutation authority.
  • CLI/MCP: additive tool/command; kind, protocol, query, offset, limit. CLI --all conflicts with --limit. Catalog registrations describe tool capabilities, not observed traffic.
  • Compatibility: existing Linux/TShark 4.x/util-linux requirements. Discovery reads no capture file and selects no interface.
  • Evidence: executable and limiter paths/digests, version/warnings, argv, selection and coverage. Global configuration, libraries and plugins remain trusted and are not fully fingerprinted; each request probes again, without caching or frozen-page claims.
  • Resources: the measured TShark 4.6.9 glossary is 22,303,197 bytes, so discovery alone allows 32 MiB producer output; capture analysis retains its 4 MiB cap. Parse incrementally, allocate only selected records, conservatively budget selected representation at 8 MiB, and copy every retained column out of the producer buffer. Over-budget/malformed output fails with captured diagnostics.
  • Installation/generated metadata: contracts, authored skill, guide and existing real-provider lane updated; no new dependencies or committed generated files.

Expected effects:

  • Read existing tool files and installation metadata.
  • Launch an owned local analysis process.
  • Create and clean a private temporary workspace/profile.

Evidence and regression coverage

Parser/provider tests cover duplicate registrations, malformed off-page records, paging/filtering/no matches, representation and command-output budgets, failed version probes, shared admission, executable changes, shutdown, cancellation and cleanup retry. MCP tests validate SDK-converted schemas and actionable unavailable results. Existing signal-provider regressions pass after the common-owner change.

The real verifier compares source ordinals and native field columns against direct TShark output under a separate empty profile; checks USB vendor-ID discovery, protocol paging, full UDP field selection, negative queries and CLI/MCP parity; and retains the existing packet-capture workflow and cleanup checks.

  • Observed facts and interpretation remain distinguishable.
  • Provenance and failed attempts remain preserved.
  • Coverage and unavailable outcomes remain explicit.

Validation performed

Cleanup follow-up on b2aff43e8: retained workspaces now keep the operation that acquired them alongside process ownership. Shutdown and pre-request cleanup retries use that identity, rather than the provider default or the next request's operation. Extended the existing field discovery boundary regression to reject a different operation while cleanup is pending, verify two failed shutdown attempts keep the original operation, and then complete cleanup. No public contract or execution scope changed.

  • Latest local validation: 62 focused regressions, npm run check, git diff --check, and a Gitleaks diff scan passed.
  • Native TShark verification on b2aff43e8 passed: 60 focused tests and 23 real CLI/MCP cases. Full CI is still running; the historical runs below apply to the preceding heads.

Earlier feature validation:

  • npm run check, npm run docs:check, npm run verify:test-discovery — passed; 1,023 test files discovered exactly once.
  • npm run jscpd — passed, 2.95% duplicated lines.
  • Focused local regression on the rebased head — 56 passed, including the product catalog, CLI help and portable advertised-pattern checks.
  • Linux verification on df821f83d — 60 tests and 23 real TShark CLI/MCP cases passed.
  • Full upstream CI on df821f83d — passed.
  • Final diff Gitleaks scan and git diff --check — passed.

Six discovery cases measured 651–761 ms per MCP call and 7.5–20.3 KB response envelopes. tokenx estimates including one tool definition, request and response were 6,634–11,253 tokens. Warm MCP server, fresh native processes, no filesystem-cache control; these are estimates, not billed usage or evidence of improved agent task completion.

Compatibility, safety, and release

No breaking call/configuration changes or migrations. macOS/Windows integration remains unsupported. The private profile does not sandbox installed plugins or libraries. Limits are per process/file, not aggregate quotas. Ambient environment is not serialized. Model-driven task selection, CPU time and peak process-family memory remain unmeasured; no cache or speedup claim.

Before merge

  • Concrete problem and one focused outcome; no material authority expansion.
  • Related upstream work and no-linked-issue reason stated above.
  • Observable behavior, meaningful failures and real public workflows covered.
  • Owning contracts/docs updated; ignored metadata regenerated and checked.
  • Final diff checked for secrets, unrelated cleanup and unsupported claims.

Summary by CodeRabbit

  • New Features
    • Added packet-field discovery through the command-line and MCP interfaces. Search installed protocol and field registrations, filter results, and page through matching records.
  • Improvements
    • Source-map comparisons now recognize Unicode-equivalent paths as possible matches while keeping ambiguous file identities unconfirmed.
    • Ghidra installation checks now account for declared Java version requirements and report unresolved or malformed installation metadata.
  • Bug Fixes
    • Invalid UTF-8 in APK metadata is now reported as unreadable output rather than silently replaced.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/2496 **Author:** [@Olorinm](https://github.com/Olorinm) **Created:** 10/13/2026 **Status:** ✅ Merged **Merged:** 10/13/2026 **Merged by:** [@baron](https://github.com/baron) **Base:** `main` ← **Head:** `feat/packet-field-discovery` --- ### 📝 Commits (8) - [`69682f3`](https://github.com/morluto/rea/commit/69682f3a82611abdd005df20097e8c038425afbd) feat(captures): discover installed protocol and field definitions - [`f67ca1a`](https://github.com/morluto/rea/commit/f67ca1a4365cc6dcd6f4fb15912c9bb699176a5d) test(captures): isolate the direct glossary oracle profile - [`730158a`](https://github.com/morluto/rea/commit/730158a27fb38d3a298f80f462b5747e91e4cd81) test(captures): accept padded kernel limit reports - [`44899da`](https://github.com/morluto/rea/commit/44899dadc3f00f1f9069ce79d6b6f2ff0436a22d) fix(captures): detach every retained glossary column - [`df821f8`](https://github.com/morluto/rea/commit/df821f83d7575be07fc359039d2d379a1ce56080) fix(captures): describe every field discovery CLI option - [`f5d9724`](https://github.com/morluto/rea/commit/f5d9724066501701b2fa254d12208476d24323b8) test(captures): verify complete field discovery coverage - [`9ca3c87`](https://github.com/morluto/rea/commit/9ca3c87fc9148db7e20ca0a05501077259f381b0) chore: synchronize field discovery with CI fixture fixes - [`b2aff43`](https://github.com/morluto/rea/commit/b2aff43e8e83719e3b8672922f62ae038146b7ae) fix(captures): retain discovery operation across cleanup retries ### 📊 Changes **19 files changed** (+1095 additions, -219 deletions) <details> <summary>View changed files</summary> 📝 `.agents/skills/reverse-engineer-anything/SKILL.md` (+3 -0) 📝 `.github/workflows/real-packet-capture.yml` (+1 -1) 📝 `docs/packet-captures.md` (+46 -0) 📝 `scripts/verify-packet-capture.mjs` (+150 -3) 📝 `src/application/CapabilityInventory.ts` (+1 -1) 📝 `src/application/captures/PacketCaptureService.ts` (+62 -8) 📝 `src/captures/TsharkCaptureProvider.ts` (+217 -125) ➕ `src/captures/parseFieldDiscovery.ts` (+92 -0) 📝 `src/cli/captureCommands.ts` (+67 -26) 📝 `src/cliCommandNames.ts` (+1 -0) 📝 `src/contracts/captures/captureToolContracts.ts` (+20 -0) 📝 `src/contracts/toolEffects.ts` (+5 -0) ➕ `src/domain/captures/packetFields.ts` (+100 -0) 📝 `src/process/ArtifactAnalysisResources.ts` (+27 -18) 📝 `src/server/registerCaptureTools.ts` (+13 -0) 📝 `tests/boundary/captures/captureProvider.test.ts` (+162 -2) ➕ `tests/boundary/captures/fieldDiscovery.test.ts` (+67 -0) 📝 `tests/boundary/cli/captureCleanup.test.ts` (+9 -0) 📝 `tests/boundary/mcp/packetCaptureMcp.test.ts` (+52 -35) </details> ### 📄 Description ## Summary Add `inspect_packet_fields` / `inspect-packet-fields` so analysts can find the installed TShark protocol and field definitions before selecting fields for `inspect_packet_capture`. For example, `{ "kind": "fields", "protocol": "usb", "query": "idVendor" }` returns `usb.idVendor`, its `FT_UINT16` type, parent `usb`, display base and bitmask. ## Problem and expected behavior Capture inspection requires exact field names but currently offers no installation-specific discovery. Search protocol registrations or field definitions by literal substring, optionally selecting an exact protocol. Return 50 matches by default, offset/limit paging, explicit full output within resource limits, and examined/matching/returned counts. Duplicate abbreviations remain separate records with source line numbers. No linked issue: this is a focused follow-up to #2318. ## Change and scope Extend the existing TShark adapter and its shared CLI/MCP Evidence workflow. Reuse the same admission slot, owned process execution, executable verification, inherited Linux limits, private empty personal profile and retryable cleanup. The common resource owner accepts a per-request operation so discovery failures have the correct identity; it does not create a second queue. This small common-owner change is identical to #2487; both PRs are independently based on main. ## Contract and boundary impact - Semantic owner: strict discovery schemas and one adapter parser for the native `-G fields` P/F grammar. Preserve names, types, parents, display bases, masks, descriptions and duplicate registrations. - Scope decision: not applicable; this extends an integrated provider's installation inspection, with no new provider or target execution/mutation authority. - CLI/MCP: additive tool/command; `kind`, `protocol`, `query`, `offset`, `limit`. CLI `--all` conflicts with `--limit`. Catalog registrations describe tool capabilities, not observed traffic. - Compatibility: existing Linux/TShark 4.x/util-linux requirements. Discovery reads no capture file and selects no interface. - Evidence: executable and limiter paths/digests, version/warnings, argv, selection and coverage. Global configuration, libraries and plugins remain trusted and are not fully fingerprinted; each request probes again, without caching or frozen-page claims. - Resources: the measured TShark 4.6.9 glossary is 22,303,197 bytes, so discovery alone allows 32 MiB producer output; capture analysis retains its 4 MiB cap. Parse incrementally, allocate only selected records, conservatively budget selected representation at 8 MiB, and copy every retained column out of the producer buffer. Over-budget/malformed output fails with captured diagnostics. - Installation/generated metadata: contracts, authored skill, guide and existing real-provider lane updated; no new dependencies or committed generated files. Expected effects: - [x] Read existing tool files and installation metadata. - [x] Launch an owned local analysis process. - [x] Create and clean a private temporary workspace/profile. ## Evidence and regression coverage Parser/provider tests cover duplicate registrations, malformed off-page records, paging/filtering/no matches, representation and command-output budgets, failed version probes, shared admission, executable changes, shutdown, cancellation and cleanup retry. MCP tests validate SDK-converted schemas and actionable unavailable results. Existing signal-provider regressions pass after the common-owner change. The real verifier compares source ordinals and native field columns against direct TShark output under a separate empty profile; checks USB vendor-ID discovery, protocol paging, full UDP field selection, negative queries and CLI/MCP parity; and retains the existing packet-capture workflow and cleanup checks. - [x] Observed facts and interpretation remain distinguishable. - [x] Provenance and failed attempts remain preserved. - [x] Coverage and unavailable outcomes remain explicit. ## Validation performed Cleanup follow-up on `b2aff43e8`: retained workspaces now keep the operation that acquired them alongside process ownership. Shutdown and pre-request cleanup retries use that identity, rather than the provider default or the next request's operation. Extended the existing field discovery boundary regression to reject a different operation while cleanup is pending, verify two failed shutdown attempts keep the original operation, and then complete cleanup. No public contract or execution scope changed. - Latest local validation: 62 focused regressions, `npm run check`, `git diff --check`, and a Gitleaks diff scan passed. - [Native TShark verification on b2aff43e8](https://github.com/morluto/rea/actions/runs/38356702503) passed: 60 focused tests and 23 real CLI/MCP cases. Full CI is still running; the historical runs below apply to the preceding heads. Earlier feature validation: - `npm run check`, `npm run docs:check`, `npm run verify:test-discovery` — passed; 1,023 test files discovered exactly once. - `npm run jscpd` — passed, 2.95% duplicated lines. - Focused local regression on the rebased head — 56 passed, including the product catalog, CLI help and portable advertised-pattern checks. - [Linux verification on df821f83d](https://github.com/morluto/rea/actions/runs/38351726972) — 60 tests and 23 real TShark CLI/MCP cases passed. - [Full upstream CI on df821f83d](https://github.com/morluto/rea/actions/runs/38351727290) — passed. - Final diff Gitleaks scan and `git diff --check` — passed. Six discovery cases measured 651–761 ms per MCP call and 7.5–20.3 KB response envelopes. `tokenx` estimates including one tool definition, request and response were 6,634–11,253 tokens. Warm MCP server, fresh native processes, no filesystem-cache control; these are estimates, not billed usage or evidence of improved agent task completion. ## Compatibility, safety, and release No breaking call/configuration changes or migrations. macOS/Windows integration remains unsupported. The private profile does not sandbox installed plugins or libraries. Limits are per process/file, not aggregate quotas. Ambient environment is not serialized. Model-driven task selection, CPU time and peak process-family memory remain unmeasured; no cache or speedup claim. ## Before merge - [x] Concrete problem and one focused outcome; no material authority expansion. - [x] Related upstream work and no-linked-issue reason stated above. - [x] Observable behavior, meaningful failures and real public workflows covered. - [x] Owning contracts/docs updated; ignored metadata regenerated and checked. - [x] Final diff checked for secrets, unrelated cleanup and unsupported claims. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added packet-field discovery through the command-line and MCP interfaces. Search installed protocol and field registrations, filter results, and page through matching records. * **Improvements** * Source-map comparisons now recognize Unicode-equivalent paths as possible matches while keeping ambiguous file identities unconfirmed. * Ghidra installation checks now account for declared Java version requirements and report unresolved or malformed installation metadata. * **Bug Fixes** * Invalid UTF-8 in APK metadata is now reported as unreadable output rather than silently replaced. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
bsakaguchi 2026-10-31 01:05:22 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
morluto/rea#5275
No description provided.