morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #5734] [Bug] inspect-pcap-capture rejects a capture containing a frame with an empty protocol stack #3482

New issue

Closed

opened 2026-10-31 00:40:58 +00:00 by bsakaguchi · 3 comments

bsakaguchi commented

2026-10-31 00:40:58 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 21, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/5734

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

inspect-pcap-capture rejects a readable saved capture when TShark emits an empty protocol stack for a zero-length packet record. Two empty records return unreadable_output and no capture Evidence. A nonempty record followed by an empty record also fails, suppressing the first frame's otherwise available protocol observation. Native capinfos and TShark read both captures successfully.

These public CLI results were independently reproduced by a verifier at the source baseline below. This submission pass inspected the retained outputs and source; current main was source-checked only, without a new runtime replay.

Reproduction

Run from a built source checkout on Linux x64 with Node.js, Python 3, util-linux prlimit, and Wireshark 4.x tshark/capinfos installed at the absolute paths below:

work_dir="$(mktemp -d)"
mkdir -p "$work_dir"/profile/{home,cfg,data,cache,state,tmp}
python3 - "$work_dir" <<'PYTHON'
import struct
import sys
from pathlib import Path
root = Path(sys.argv[1])
header = struct.pack('<IHHiIII', 0xa1b2c3d4, 2, 4, 0, 0, 65535, 147)
def record(payload):
    return struct.pack('<IIII', 1700000000, 0, len(payload), len(payload)) + payload
for name, packets in [
    ('zero-length', [b'', b'']),
    ('nonempty', [b'hello']),
    ('mixed', [b'hello', b'']),
]:
    (root / (name + '.pcap')).write_bytes(header + b''.join(map(record, packets)))
PYTHON
run_rea() {
    S="$work_dir/profile"
    env -i PATH="$PATH" HOME="$S/home" XDG_CONFIG_HOME="$S/cfg" \
        XDG_DATA_HOME="$S/data" XDG_CACHE_HOME="$S/cache" \
        XDG_STATE_HOME="$S/state" TMPDIR="$S/tmp" REA_LOG_LEVEL=silent \
        REA_TSHARK_COMMAND=/usr/bin/tshark \
        REA_CAPINFOS_COMMAND=/usr/bin/capinfos node scripts/rea.mjs "$@"
}
capinfos -M -S "$work_dir/zero-length.pcap"
tshark -r "$work_dir/zero-length.pcap" -T json -e frame.number -e frame.protocols
run_rea inspect-pcap-capture "$work_dir/zero-length.pcap" --json
run_rea inspect-pcap-capture "$work_dir/nonempty.pcap" --json
run_rea inspect-pcap-capture "$work_dir/mixed.pcap" --json
run_rea inspect-packet-capture "$work_dir/zero-length.pcap" --fields ip.addr --json
run_rea inspect-packet-capture "$work_dir/mixed.pcap" --fields ip.addr --json

Run failing commands individually if the surrounding shell stops on nonzero status. The verifier executed these fixtures and workflows: both two-record summaries failed in the PATH-selected and explicitly configured runs (2/2 per fixture); the nonempty control succeeded in both. The packet-field controls succeeded with the explicit absolute TShark setting.

Expected behavior

Return the exact packet_count: 2 for both two-record captures and retain the mixed capture's user_dlt:data count of one. Represent absent protocol names without inventing an empty-name protocol row. An empty protocol distribution with protocol_count: 0 is permitted by the result schema.

The saved-capture contract promises exact packet counts and retained inventories. Protocol aggregation documentation does not exclude zero-length records; the result schema requires nonempty protocol names, not a nonempty distribution.

Impact and scope

P2: the demonstrated input class loses the entire capture summary, including another frame's known protocol observation. No capture bytes are modified. Native capinfos/TShark and the configured sibling packet-field command can inspect the unchanged captures, but callers must aggregate facts themselves and do not receive equivalent summary Evidence. Removing records loses source information. Tested scope is generated classic little-endian user0 pcap files; other link types, pcapng, Windows, and MCP delivery were not tested.

REA version

rea-agents 6.7.0, verified source baseline 9d8e7803326a382cbbb3ffdb900f5e93e5442d68. Verified at baseline; main source-checked only. After fetching, origin/main was d0067ba34a39ea47dd7e11bb2e6169bc0938cc3f; all five cited parser/provider/contract/schema/documentation files are unchanged from baseline. No runtime reproduction on current main is claimed.

Runtime and platform

Verifier runtime: Node.js v24.22.0, Linux x64 in the supplied baseline image, TShark and capinfos 4.0.17 (Debian package 4.0.17-0+deb12u3), with util-linux prlimit and isolated HOME/XDG/TMPDIR. The verifier compared the image's parser, provider, contract and package manifest against the baseline checkout byte-for-byte. Submission host only: Node.js v26.12.0, macOS 27.0.1 arm64; capture inspection was not executed on macOS. Hopper/Ghidra are not involved.

Target and artifact context

Self-generated pcap, link type 147 (user0), snapshot length 65535. The all-empty fixture is 56 bytes with two zero-length packet records; SHA-256 961538b1aaac233cfdeb179563b1a76e5e1c4aedec3bc20e785618fb6a3ca74a. Controls contain one five-byte record or that record followed by a zero-length record. No live traffic or private target data is involved.

Interface

CLI

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

The verifier's retained native JSON contains frame numbers 1, 2, each with frame.protocols: [""] for the all-empty fixture. capinfos reports two packets. The mixed fixture's stacks are user_dlt:data, then empty.

Both two-record summaries exit 1 with this subset and no capture Evidence:

{"code":"unreadable_output","category":"execution_failure","retryable":false}

The message recommends rea doctor and says repeating the unchanged request is not expected to succeed. The nonempty-only summary exits 0 with packet_count: 1, protocol_count: 1, and protocol_distribution: [{"protocol":"user_dlt:data","packet_count":1}]. Both packet-field controls exit 0 and return frames 1 and 2 with matching_packets: 2, returned_packets: 2, and read_scope: "to-eof". These are independently observed baseline results, reviewed from retained output during submission.

Typed outcome or recovery expected

Successful capture Evidence retaining exact frame counts and available inventories. Proposed restoration checks: both two-record summaries succeed; the mixed stack count remains one; nonempty and packet-field controls continue to pass. Continue rejecting malformed producer output, missing/duplicated frame identifiers, and ambiguous field arrays. No fix or successful restoration was tested.

Analysis or suspected cause

Source-derived reasoning: PcapProvider.ts:767–787 passes native TShark JSON to the parser. PcapOutput.ts:272–278 rejects a one-element empty protocol string before aggregation, although the later aggregation loop skips empty protocol values. This is REA boundary validation, rather than a demonstrated Wireshark failure.

Additional context

Prepared with AI assistance and an independent verifier. The separate retained-view parent-list documentation omission is tracked in #5733; it does not cover this parser failure. Related #4180 covers capinfos timestamp labels; #5279 covers cancellation projection; #5711 covers macOS availability; #3403 / PR #3407 cover TShark selection ownership. Their contracts and fix signals differ from accepting a valid empty stack in a readable frame. NOT_ENOUGH_INFO: last known good and first affected release were not established.

Originally created by @coygeek on GitHub (Oct 21, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/5734 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior `inspect-pcap-capture` rejects a readable saved capture when TShark emits an empty protocol stack for a zero-length packet record. Two empty records return `unreadable_output` and no capture Evidence. A nonempty record followed by an empty record also fails, suppressing the first frame's otherwise available protocol observation. Native capinfos and TShark read both captures successfully. These public CLI results were independently reproduced by a verifier at the source baseline below. This submission pass inspected the retained outputs and source; current main was source-checked only, without a new runtime replay. ### Reproduction Run from a built source checkout on Linux x64 with Node.js, Python 3, util-linux `prlimit`, and Wireshark 4.x `tshark`/`capinfos` installed at the absolute paths below: ```sh work_dir="$(mktemp -d)" mkdir -p "$work_dir"/profile/{home,cfg,data,cache,state,tmp} python3 - "$work_dir" <<'PYTHON' import struct import sys from pathlib import Path root = Path(sys.argv[1]) header = struct.pack('<IHHiIII', 0xa1b2c3d4, 2, 4, 0, 0, 65535, 147) def record(payload): return struct.pack('<IIII', 1700000000, 0, len(payload), len(payload)) + payload for name, packets in [ ('zero-length', [b'', b'']), ('nonempty', [b'hello']), ('mixed', [b'hello', b'']), ]: (root / (name + '.pcap')).write_bytes(header + b''.join(map(record, packets))) PYTHON run_rea() { S="$work_dir/profile" env -i PATH="$PATH" HOME="$S/home" XDG_CONFIG_HOME="$S/cfg" \ XDG_DATA_HOME="$S/data" XDG_CACHE_HOME="$S/cache" \ XDG_STATE_HOME="$S/state" TMPDIR="$S/tmp" REA_LOG_LEVEL=silent \ REA_TSHARK_COMMAND=/usr/bin/tshark \ REA_CAPINFOS_COMMAND=/usr/bin/capinfos node scripts/rea.mjs "$@" } capinfos -M -S "$work_dir/zero-length.pcap" tshark -r "$work_dir/zero-length.pcap" -T json -e frame.number -e frame.protocols run_rea inspect-pcap-capture "$work_dir/zero-length.pcap" --json run_rea inspect-pcap-capture "$work_dir/nonempty.pcap" --json run_rea inspect-pcap-capture "$work_dir/mixed.pcap" --json run_rea inspect-packet-capture "$work_dir/zero-length.pcap" --fields ip.addr --json run_rea inspect-packet-capture "$work_dir/mixed.pcap" --fields ip.addr --json ``` Run failing commands individually if the surrounding shell stops on nonzero status. The verifier executed these fixtures and workflows: both two-record summaries failed in the PATH-selected and explicitly configured runs (2/2 per fixture); the nonempty control succeeded in both. The packet-field controls succeeded with the explicit absolute TShark setting. ### Expected behavior Return the exact `packet_count: 2` for both two-record captures and retain the mixed capture's `user_dlt:data` count of one. Represent absent protocol names without inventing an empty-name protocol row. An empty protocol distribution with `protocol_count: 0` is permitted by the result schema. The [saved-capture contract](https://github.com/morluto/rea/blob/9d8e7803326a382cbbb3ffdb900f5e93e5442d68/src/contracts/pcap/pcapToolContracts.ts#L30) promises exact packet counts and retained inventories. [Protocol aggregation documentation](https://github.com/morluto/rea/blob/9d8e7803326a382cbbb3ffdb900f5e93e5442d68/docs/pcap-analysis.md#L66) does not exclude zero-length records; the [result schema](https://github.com/morluto/rea/blob/9d8e7803326a382cbbb3ffdb900f5e93e5442d68/src/domain/pcap/pcapAnalysis.ts#L95) requires nonempty protocol names, not a nonempty distribution. ### Impact and scope P2: the demonstrated input class loses the entire capture summary, including another frame's known protocol observation. No capture bytes are modified. Native capinfos/TShark and the configured sibling packet-field command can inspect the unchanged captures, but callers must aggregate facts themselves and do not receive equivalent summary Evidence. Removing records loses source information. Tested scope is generated classic little-endian user0 pcap files; other link types, pcapng, Windows, and MCP delivery were not tested. ### REA version `rea-agents` 6.7.0, verified source baseline `9d8e7803326a382cbbb3ffdb900f5e93e5442d68`. Verified at baseline; main source-checked only. After fetching, `origin/main` was `d0067ba34a39ea47dd7e11bb2e6169bc0938cc3f`; all five cited parser/provider/contract/schema/documentation files are unchanged from baseline. No runtime reproduction on current main is claimed. ### Runtime and platform Verifier runtime: Node.js v24.22.0, Linux x64 in the supplied baseline image, TShark and capinfos 4.0.17 (Debian package 4.0.17-0+deb12u3), with util-linux `prlimit` and isolated HOME/XDG/TMPDIR. The verifier compared the image's parser, provider, contract and package manifest against the baseline checkout byte-for-byte. Submission host only: Node.js v26.12.0, macOS 27.0.1 arm64; capture inspection was not executed on macOS. Hopper/Ghidra are not involved. ### Target and artifact context Self-generated pcap, link type 147 (`user0`), snapshot length 65535. The all-empty fixture is 56 bytes with two zero-length packet records; SHA-256 `961538b1aaac233cfdeb179563b1a76e5e1c4aedec3bc20e785618fb6a3ca74a`. Controls contain one five-byte record or that record followed by a zero-length record. No live traffic or private target data is involved. ### Interface CLI ### Earliest failure stage Native analysis or reconstruction ### Relevant output or evidence The verifier's retained native JSON contains frame numbers `1`, `2`, each with `frame.protocols: [""]` for the all-empty fixture. capinfos reports two packets. The mixed fixture's stacks are `user_dlt:data`, then empty. Both two-record summaries exit 1 with this subset and no capture Evidence: ```json {"code":"unreadable_output","category":"execution_failure","retryable":false} ``` The message recommends `rea doctor` and says repeating the unchanged request is not expected to succeed. The nonempty-only summary exits 0 with `packet_count: 1`, `protocol_count: 1`, and `protocol_distribution: [{"protocol":"user_dlt:data","packet_count":1}]`. Both packet-field controls exit 0 and return frames 1 and 2 with `matching_packets: 2`, `returned_packets: 2`, and `read_scope: "to-eof"`. These are independently observed baseline results, reviewed from retained output during submission. ### Typed outcome or recovery expected Successful capture Evidence retaining exact frame counts and available inventories. Proposed restoration checks: both two-record summaries succeed; the mixed stack count remains one; nonempty and packet-field controls continue to pass. Continue rejecting malformed producer output, missing/duplicated frame identifiers, and ambiguous field arrays. No fix or successful restoration was tested. ### Analysis or suspected cause Source-derived reasoning: [PcapProvider.ts:767–787](https://github.com/morluto/rea/blob/9d8e7803326a382cbbb3ffdb900f5e93e5442d68/src/pcap/PcapProvider.ts#L767) passes native TShark JSON to the parser. [PcapOutput.ts:272–278](https://github.com/morluto/rea/blob/9d8e7803326a382cbbb3ffdb900f5e93e5442d68/src/pcap/PcapOutput.ts#L272) rejects a one-element empty protocol string before aggregation, although [the later aggregation loop](https://github.com/morluto/rea/blob/9d8e7803326a382cbbb3ffdb900f5e93e5442d68/src/pcap/PcapOutput.ts#L317) skips empty protocol values. This is REA boundary validation, rather than a demonstrated Wireshark failure. ### Additional context Prepared with AI assistance and an independent verifier. The separate retained-view parent-list documentation omission is tracked in #5733; it does not cover this parser failure. Related #4180 covers capinfos timestamp labels; #5279 covers cancellation projection; #5711 covers macOS availability; #3403 / PR #3407 cover TShark selection ownership. Their contracts and fix signals differ from accepting a valid empty stack in a readable frame. NOT_ENOUGH_INFO: last known good and first affected release were not established.

bsakaguchi closed this issue

2026-10-31 00:40:59 +00:00

bsakaguchi commented

2026-10-31 00:41:00 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 21, 2026):

Confirmed: an empty protocol stack from a zero-length packet is a legitimate TShark output, so rejecting the whole capture as unreadable_output loses real signal — and in the mixed case it even suppresses the first frame that was actually observable. The aggregation at PcapProvider.ts:767-787 clearly assumes at least one protocol entry per frame, so a zero-length record trips it before the later aggregation loop can salvage anything. Fix direction: treat an empty stack as "no protocol info" for that frame (keep the frame record, mark protocols empty or null) rather than failing the parse, and make the outcome tolerate a capture whose only frames are zero-length. Also worth pinning both fixtures — all-empty and nonempty+empty — in the saved-capture contract so the regression is covered.

@XinwangTao commented on GitHub (Oct 21, 2026): Confirmed: an empty protocol stack from a zero-length packet is a legitimate TShark output, so rejecting the whole capture as unreadable_output loses real signal — and in the mixed case it even suppresses the first frame that was actually observable. The aggregation at PcapProvider.ts:767-787 clearly assumes at least one protocol entry per frame, so a zero-length record trips it before the later aggregation loop can salvage anything. Fix direction: treat an empty stack as "no protocol info" for that frame (keep the frame record, mark protocols empty or null) rather than failing the parse, and make the outcome tolerate a capture whose only frames are zero-length. Also worth pinning both fixtures — all-empty and nonempty+empty — in the saved-capture contract so the regression is covered.

bsakaguchi commented

2026-10-31 00:41:00 +00:00

Author

Owner

@gensokyo-dev commented on GitHub (Oct 21, 2026):

I am investigating this unclaimed parser bug on current main. I will verify the zero-length and mixed-frame cases with native TShark/capinfos and exercise the shared CLI/MCP Evidence workflow before proposing a focused fix.

@gensokyo-dev commented on GitHub (Oct 21, 2026): I am investigating this unclaimed parser bug on current main. I will verify the zero-length and mixed-frame cases with native TShark/capinfos and exercise the shared CLI/MCP Evidence workflow before proposing a focused fix.

bsakaguchi commented

2026-10-31 00:41:00 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 21, 2026):

Agree with the fix direction in #5762: treating an empty protocol stack as "no protocol info for that frame" (instead of failing the whole parse) preserves the frame count and the mixed fixture's observable first frame. Suggest pinning both the all-empty and nonempty+empty fixtures in the saved-capture contract as regression tests, so future parser changes can't silently drop that signal again.

@XinwangTao commented on GitHub (Oct 21, 2026): Agree with the fix direction in #5762: treating an empty protocol stack as "no protocol info for that frame" (instead of failing the whole parse) preserves the frame count and the mixed fixture's observable first frame. Suggest pinning both the all-empty and nonempty+empty fixtures in the saved-capture contract as regression tests, so future parser changes can't silently drop that signal again.

bsakaguchi referenced this issue

2026-10-31 00:42:25 +00:00

[GH-ISSUE #5948] [Bug] Native UI helper timeouts return execution_failure at the hidden 30-second limit #3616

bsakaguchi referenced this issue

2026-10-31 01:09:19 +00:00

[PR #3482] [CLOSED] refactor: extract native UI observation timeout constant #5772