[GH-ISSUE #6735] [Bug] TShark field glossary search drops exact Greek sigma substrings #4098

New issue

Closed

opened 2026-10-31 00:46:48 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:46:48 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 23, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6735

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

The production parser used by inspect_packet_fields drops a literal substring that appears exactly in a protocol's display name. For a valid tshark -G fields record P\tΟΣ\texample, querying Σ returns no matches, while an empty query or the full name ΟΣ returns the record. The same failure was observed by directly calling the current source parser and the parser shipped in npm rea-agents@6.7.0. This is a parser-boundary reproduction with a synthetic glossary, not a claim that a stock installed TShark glossary contains this particular name or that a live provider was exercised.

Reproduction

From a source checkout with locked dependencies installed, run:

node --import tsx --input-type=module <<'JS'
import { parseFieldDiscovery } from './src/captures/parseFieldDiscovery.ts';
const glossary = 'P\tΟΣ\texample\n';
for (const query of ['', 'Σ', 'ΟΣ']) {
  console.log(query, parseFieldDiscovery(glossary, {
    kind: 'protocols', protocol: null, query, offset: 0, limit: 50,
  }).output_window);
}
JS

This direct production-parser probe was run once against main and once against the published package's equivalent dist/captures/parseFieldDiscovery.js before this draft was written.

Expected behavior

A case-insensitive literal substring search must include an exact literal substring. The public input contract describes query as a “Case-insensitive literal substring in name, abbreviation or field description”: packetFields.ts.

Impact and scope

Protocol/field names or descriptions containing a word-final Greek sigma can disappear from discovery results, including an exact substring query copied from the displayed producer text. The wrong matching, returned, and has_more values make the omitted records appear absent. An empty query or a full-name query returned the row in the probe. The frequency in real built-in or user-provided dissector glossaries is NOT_ENOUGH_INFO; no real TShark glossary was surveyed.

REA version

Published rea-agents@6.7.0 (rea --version: 6.7.0), and main source bc411c24befceced5a9ea1ca1c1ce123fd02bf48.

Runtime and platform

Node.js v26.12.1; Linux x64. No external TShark process was used for this parser-boundary reproduction.

Target and artifact context

One synthetic TShark -G fields protocol-registration line with a Unicode display name and ASCII abbreviation.

Interface

Other

Earliest failure stage

Transport or result projection

Relevant output or evidence

Observed output on both source and release:

query=""    examined=1 matching=1 returned=1 has_more=false
query="Σ"   examined=1 matching=0 returned=0 has_more=false
query="ΟΣ"  examined=1 matching=1 returned=1 has_more=false

'ΟΣ'.includes('Σ') is true. The parser independently lowercases the whole producer value and the query: 'ΟΣ'.toLowerCase() is ος (final sigma), while 'Σ'.toLowerCase() is σ. See parseFieldDiscovery.ts.

Typed outcome or recovery expected

Return the matching glossary record with matching: 1 and returned: 1; keep exact producer text in the record.

Analysis or suspected cause

Source-derived cause: context-sensitive Unicode lowercasing is not a substring-preserving case-folding operation. This parser applies it independently to the haystack and needle.

Additional context

The related reports #6634 and #6714 concern native search and sigrok decoder search respectively. This report concerns the separate TShark glossary parser and inspect_packet_fields contract; neither of those fixes alone changes this parser.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: The production parser demonstrably omits an exact matching glossary row and reports an empty result for a valid Unicode query; real-glossary frequency remains unmeasured.
Workaround status: available
Workaround basis: The same probe returned the record with an empty query or a full-name query, allowing manual discovery without modifying the glossary.

Originally created by @coygeek on GitHub (Oct 23, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6735 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior The production parser used by `inspect_packet_fields` drops a literal substring that appears exactly in a protocol's display name. For a valid `tshark -G fields` record `P\tΟΣ\texample`, querying `Σ` returns no matches, while an empty query or the full name `ΟΣ` returns the record. The same failure was observed by directly calling the current source parser and the parser shipped in npm `rea-agents@6.7.0`. This is a parser-boundary reproduction with a synthetic glossary, not a claim that a stock installed TShark glossary contains this particular name or that a live provider was exercised. ### Reproduction From a source checkout with locked dependencies installed, run: ```sh node --import tsx --input-type=module <<'JS' import { parseFieldDiscovery } from './src/captures/parseFieldDiscovery.ts'; const glossary = 'P\tΟΣ\texample\n'; for (const query of ['', 'Σ', 'ΟΣ']) { console.log(query, parseFieldDiscovery(glossary, { kind: 'protocols', protocol: null, query, offset: 0, limit: 50, }).output_window); } JS ``` This direct production-parser probe was run once against main and once against the published package's equivalent `dist/captures/parseFieldDiscovery.js` before this draft was written. ### Expected behavior A case-insensitive literal substring search must include an exact literal substring. The public input contract describes `query` as a “Case-insensitive literal substring in name, abbreviation or field description”: [packetFields.ts](https://github.com/morluto/rea/blob/bc411c24befceced5a9ea1ca1c1ce123fd02bf48/src/domain/captures/packetFields.ts). ### Impact and scope Protocol/field names or descriptions containing a word-final Greek sigma can disappear from discovery results, including an exact substring query copied from the displayed producer text. The wrong `matching`, `returned`, and `has_more` values make the omitted records appear absent. An empty query or a full-name query returned the row in the probe. The frequency in real built-in or user-provided dissector glossaries is NOT_ENOUGH_INFO; no real TShark glossary was surveyed. ### REA version Published `rea-agents@6.7.0` (`rea --version`: `6.7.0`), and main source `bc411c24befceced5a9ea1ca1c1ce123fd02bf48`. ### Runtime and platform Node.js v26.12.1; Linux x64. No external TShark process was used for this parser-boundary reproduction. ### Target and artifact context One synthetic TShark `-G fields` protocol-registration line with a Unicode display name and ASCII abbreviation. ### Interface Other ### Earliest failure stage Transport or result projection ### Relevant output or evidence Observed output on both source and release: ```text query="" examined=1 matching=1 returned=1 has_more=false query="Σ" examined=1 matching=0 returned=0 has_more=false query="ΟΣ" examined=1 matching=1 returned=1 has_more=false ``` `'ΟΣ'.includes('Σ')` is true. The parser independently lowercases the whole producer value and the query: `'ΟΣ'.toLowerCase()` is `ος` (final sigma), while `'Σ'.toLowerCase()` is `σ`. See [parseFieldDiscovery.ts](https://github.com/morluto/rea/blob/bc411c24befceced5a9ea1ca1c1ce123fd02bf48/src/captures/parseFieldDiscovery.ts#L51). ### Typed outcome or recovery expected Return the matching glossary record with `matching: 1` and `returned: 1`; keep exact producer text in the record. ### Analysis or suspected cause Source-derived cause: context-sensitive Unicode lowercasing is not a substring-preserving case-folding operation. This parser applies it independently to the haystack and needle. ### Additional context The related reports #6634 and #6714 concern native search and sigrok decoder search respectively. This report concerns the separate TShark glossary parser and `inspect_packet_fields` contract; neither of those fixes alone changes this parser. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: The production parser demonstrably omits an exact matching glossary row and reports an empty result for a valid Unicode query; real-glossary frequency remains unmeasured. - Workaround status: available - Workaround basis: The same probe returned the record with an empty query or a full-name query, allowing manual discovery without modifying the glossary.

bsakaguchi closed this issue

2026-10-31 00:46:48 +00:00

bsakaguchi referenced this issue

2026-10-31 01:11:31 +00:00

[PR #4098] [MERGED] fix(kuna): report output beyond REA's buffer as deterministic and final #6057

bsakaguchi referenced this issue

2026-10-31 01:11:32 +00:00

[PR #4104] [MERGED] fix(kuna): suggest no restart when a fixed buffer stopped the export #6060