[GH-ISSUE #6598] [Bug] Qlog incomplete-record views omit STREAM frames with unknown FIN #3984

New issue

Closed

opened 2026-10-31 00:45:47 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:45:47 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 23, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6598

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

The production qlog decoder accepts a STREAM frame whose fin is omitted or null and retains its contribution as fin: null, but leaves that record out of incomplete_fact_records. Consequently the qlog summary reports zero incomplete records and the incomplete-records facet returns an empty page. This contradicts the documented inventory of omitted interpretable frame metadata. Byte ranges remain correct; no fabricated final size was observed.

Reproduction

Executed one source-boundary probe with five synthetic cases against the revision below. This is a decoder/schema/projector reproduction, not an end-to-end CLI or MCP execution. With the checkout's locked dependencies installed and its pinned Node runtime selected, the following self-contained probe runs outside the checkout; set REA_CHECKOUT to that checkout's absolute path. Inputs remain in memory.

export REA_CHECKOUT=/absolute/path/to/rea
qlog_probe=$(mktemp /tmp/qlog-fin-XXXXXX.mts)
trap 'rm -f "$qlog_probe"' EXIT
cat > "$qlog_probe" <<'TS'
import { pathToFileURL } from 'node:url';
const root = process.argv[2];
const { decodeQlogCapture } = await import(pathToFileURL(`${root}/src/browser/history/QlogCapture.ts`).href);
const { inspectWebNetworkCaptureInputSchema, webNetworkCapturePayloadSchema } = await import(pathToFileURL(`${root}/src/domain/webNetworkCapture.ts`).href);
const { qlogViewRequestSchema, projectQlogCapturePage, summarizeQlogCapture } = await import(pathToFileURL(`${root}/src/domain/analysisView/qlogCaptureView.ts`).href);
inspectWebNetworkCaptureInputSchema.parse({capture_path:'/tmp/synthetic.qlog',format:'qlog'});
for (const name of ['missing-fin', 'null-fin', 'false-fin', 'true-fin', 'missing-offset']) {
  const frame: Record<string, unknown> = {frame_type:'stream',stream_id:0,offset:0,length:10};
  if (name === 'null-fin') frame.fin = null;
  if (name === 'false-fin' || name === 'missing-offset') frame.fin = false;
  if (name === 'true-fin') frame.fin = true;
  if (name === 'missing-offset') delete frame.offset;
  const text = JSON.stringify({qlog_format:'JSON',qlog_version:'0.3',traces:[{events:[{name:'transport:packet_sent',time:1,data:{frames:[frame]}}]}]});
  const out = webNetworkCapturePayloadSchema.parse(decodeQlogCapture(text, []));
  if (!('qlog' in out)) throw Error('qlog result required');
  const trace = out.qlog.traces[0];
  const view = qlogViewRequestSchema.parse({kind:'qlog',facet:'incomplete-records',trace_index:0});
  const page = projectQlogCapturePage(out.qlog, view);
  console.log(JSON.stringify({name,contribution:trace.streams[0]?.contributions[0]??null,unique_bytes:trace.streams[0]?.unique_bytes??null,final_sizes:trace.streams[0]?.reported_final_sizes??null,incomplete:trace.incomplete_fact_records,summary_count:summarizeQlogCapture(out.qlog).incomplete_fact_record_count,page:page.ok?page.value.page:null}));
}
console.log(JSON.stringify({runtime:process.version,platform:process.platform,arch:process.arch}));
TS
node --import "$REA_CHECKOUT/node_modules/tsx/dist/loader.mjs" "$qlog_probe" "$REA_CHECKOUT"

The /tmp/synthetic.qlog path is used only to validate the public request shape; no artifact at that path is read or created. No browser, recorded URL, HTTP/3 session, process capture, live configuration, or instrumentation is used.

Expected behavior

Missing frame metadata is supported input. FIN is interpreted to derive final sizes, so an omitted/null FIN belongs in the documented incomplete-metadata inventory. Both unknown-FIN cases should retain the valid ten-byte contribution, keep fin: null, include ordinal 0 in incomplete_fact_records, report summary count 1, and return [0] in the incomplete-record page. Explicit false/true FIN must remain distinguishable.

Impact and scope

The inventory and summary underreport partial transport evidence for supported event subsets with unknown FIN. Consumers discovering missing facts through those views cannot find the affected records. In the bounded probe both unknown-FIN cases exhibit the discrepancy; the controls behave as described below. Detailed contribution output preserves fin: null, and byte totals remain valid. Actual frequency in retained captures and previous-version behavior are unknown.

Workaround: inspect the complete contributions for nullable FIN instead of relying on the incomplete-record inventory. The probe directly shows those nullable fields surviving.

REA version

Source package 6.7.0 at immutable main commit 642f6571ea9d333ea50114c25e648dbb5b8a3c46. The npm registry latest release was checked before the audit; this reproduction used source modules.

Runtime and platform

Node.js v24.22.1; Linux x86_64; tsx 4.23.15 from the locked dependency installation. No optional analysis provider was used.

Target and artifact context

Synthetic JSON qlog 0.3 event subsets within the selected aioquic-1.5.0-json-0.3 interpretation profile. Each case contains one trace, one packet-sent event, and one STREAM frame. These fixtures establish parser semantics, not aioquic producer provenance.

Interface

Other — production source boundary shared by CLI/MCP historical-capture inspection and retained qlog views.

Earliest failure stage

Capture or observation — transport-fact extraction in decodeQlogCapture, before service or view projection.

Relevant output or evidence

Observed selected values from the single successful probe invocation (exit 0):

missing-fin: contribution.fin=null, unique_bytes="10", final_sizes=[], incomplete=[], summary_count=0, page.items=[]
null-fin: contribution.fin=null, unique_bytes="10", final_sizes=[], incomplete=[], summary_count=0, page.items=[]
false-fin: contribution.fin=false, unique_bytes="10", final_sizes=[], incomplete=[], summary_count=0, page.items=[]
true-fin: contribution.fin=true, unique_bytes="10", final_sizes=["10"], incomplete=[], summary_count=0, page.items=[]
missing-offset: contribution=null, incomplete=[0], summary_count=1, page.items=[0]
runtime=v24.22.1, platform=linux, arch=x64

Expected source: supported missing metadata, FIN interpretation and incomplete-record definition.
Failure source: the executed probe above; the collector's missing-field test and nullable FIN contribution explain its observed result.
Evidence provenance: observed — direct execution of the production decoder, payload schema, summary and page projector; public call-path reachability is source-derived.
Local verification: reproduced — source boundary only; CLI/MCP end-to-end execution was not run.
Reproduction completeness: complete for the stated source-boundary discrepancy; no shipped transport or real-producer claim is made.

Supported reachability and guards: format: "qlog" is a public capture format, and the public request schema accepts the probe request. The normal Node capture decoder invokes this exact production function. The fixture supplies the required JSON container, trace/events array, string name, finite numeric time, object data, and valid QUIC integers. STREAM validation expressly permits absent/null FIN. All outputs passed the production payload schema, and the view request passed its selector schema. The summary count and incomplete-record page expose the same producer-owned inventory rather than independently losing FIN.

Typed outcome or recovery expected

Restoration check: repeat the five cases. Missing/null FIN should each produce incomplete ordinal [0], summary count 1, and page item [0] while retaining fin: null and ten-byte coverage. False FIN should retain no final size and no incomplete entry; true FIN should retain final size "10" and no incomplete entry. The missing-offset control must remain incomplete. Passing must not come from rejecting supported partial frames or discarding their usable byte ranges.

Analysis or suspected cause

Source-derived: the incomplete-record branch covers missing stream ID, offset and length, while the subsequent contribution maps unknown FIN to null without adding the ordinal. These are one issue: the summary and focused page correctly propagate an underfilled inventory. A narrower intended definition limited to byte-range inputs would need an explicit contract clarification; the current documentation covers omitted interpretable frame metadata, and FIN has a documented interpretation.

Additional context

Targeted GitHub issue searches were run separately for open and closed issues using qlog, qlog "fin", "incomplete_fact_records", qlog "incomplete", and qlog "final". No matching FIN/inventory report was returned. Related #3133 owns performance measurement; closed #3960 concerns the shared nesting limit; closed #2016 concerns broader analysis-efficiency work. None describes this trigger or failed inventory contract. Submission-time duplicate searches are performed separately immediately before filing.

Last-known-good version: NOT_ENOUGH_INFO; no regression boundary was established. Unknowns: full CLI/MCP behavior was not executed, no live aioquic capture was produced, prevalence and regression range are unknown. The documented empty-inventory caveat about complete logging/network capture does not assert completeness here; the observed failure is omission of metadata already known to be absent in a recorded frame. An independent pre-draft source review agreed that the documented inventory includes FIN. No fix, build, full suite, commit, or issue submission was performed.

Privacy scrub: every input is synthetic; the report contains no credentials, private home paths, real target data, or raw capture. AI assistance: investigation, source-boundary probe and draft preparation were AI-assisted; no human submission or review declaration is made. All source/probe verification finished before this draft was written; no later re-verification is part of this report.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: Supported partial STREAM records are absent from the promised incomplete-metadata inventory and its summary/page, impairing discovery of unknown FIN; byte totals and the detailed nullable field remain correct.
Workaround status: available
Workaround basis: The executed probe preserves contribution.fin as null, allowing inspection of complete contributions instead of relying on the underreported inventory.

Originally created by @coygeek on GitHub (Oct 23, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6598 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior The production qlog decoder accepts a STREAM frame whose `fin` is omitted or null and retains its contribution as `fin: null`, but leaves that record out of `incomplete_fact_records`. Consequently the qlog summary reports zero incomplete records and the `incomplete-records` facet returns an empty page. This contradicts the documented inventory of omitted interpretable frame metadata. Byte ranges remain correct; no fabricated final size was observed. ### Reproduction Executed one source-boundary probe with five synthetic cases against the revision below. This is a decoder/schema/projector reproduction, not an end-to-end CLI or MCP execution. With the checkout's locked dependencies installed and its pinned Node runtime selected, the following self-contained probe runs outside the checkout; set `REA_CHECKOUT` to that checkout's absolute path. Inputs remain in memory. ```sh export REA_CHECKOUT=/absolute/path/to/rea qlog_probe=$(mktemp /tmp/qlog-fin-XXXXXX.mts) trap 'rm -f "$qlog_probe"' EXIT cat > "$qlog_probe" <<'TS' import { pathToFileURL } from 'node:url'; const root = process.argv[2]; const { decodeQlogCapture } = await import(pathToFileURL(`${root}/src/browser/history/QlogCapture.ts`).href); const { inspectWebNetworkCaptureInputSchema, webNetworkCapturePayloadSchema } = await import(pathToFileURL(`${root}/src/domain/webNetworkCapture.ts`).href); const { qlogViewRequestSchema, projectQlogCapturePage, summarizeQlogCapture } = await import(pathToFileURL(`${root}/src/domain/analysisView/qlogCaptureView.ts`).href); inspectWebNetworkCaptureInputSchema.parse({capture_path:'/tmp/synthetic.qlog',format:'qlog'}); for (const name of ['missing-fin', 'null-fin', 'false-fin', 'true-fin', 'missing-offset']) { const frame: Record<string, unknown> = {frame_type:'stream',stream_id:0,offset:0,length:10}; if (name === 'null-fin') frame.fin = null; if (name === 'false-fin' || name === 'missing-offset') frame.fin = false; if (name === 'true-fin') frame.fin = true; if (name === 'missing-offset') delete frame.offset; const text = JSON.stringify({qlog_format:'JSON',qlog_version:'0.3',traces:[{events:[{name:'transport:packet_sent',time:1,data:{frames:[frame]}}]}]}); const out = webNetworkCapturePayloadSchema.parse(decodeQlogCapture(text, [])); if (!('qlog' in out)) throw Error('qlog result required'); const trace = out.qlog.traces[0]; const view = qlogViewRequestSchema.parse({kind:'qlog',facet:'incomplete-records',trace_index:0}); const page = projectQlogCapturePage(out.qlog, view); console.log(JSON.stringify({name,contribution:trace.streams[0]?.contributions[0]??null,unique_bytes:trace.streams[0]?.unique_bytes??null,final_sizes:trace.streams[0]?.reported_final_sizes??null,incomplete:trace.incomplete_fact_records,summary_count:summarizeQlogCapture(out.qlog).incomplete_fact_record_count,page:page.ok?page.value.page:null})); } console.log(JSON.stringify({runtime:process.version,platform:process.platform,arch:process.arch})); TS node --import "$REA_CHECKOUT/node_modules/tsx/dist/loader.mjs" "$qlog_probe" "$REA_CHECKOUT" ``` The `/tmp/synthetic.qlog` path is used only to validate the public request shape; no artifact at that path is read or created. No browser, recorded URL, HTTP/3 session, process capture, live configuration, or instrumentation is used. ### Expected behavior Missing frame metadata is supported input. FIN is interpreted to derive final sizes, so an omitted/null FIN belongs in the documented incomplete-metadata inventory. Both unknown-FIN cases should retain the valid ten-byte contribution, keep `fin: null`, include ordinal `0` in `incomplete_fact_records`, report summary count `1`, and return `[0]` in the incomplete-record page. Explicit false/true FIN must remain distinguishable. ### Impact and scope The inventory and summary underreport partial transport evidence for supported event subsets with unknown FIN. Consumers discovering missing facts through those views cannot find the affected records. In the bounded probe both unknown-FIN cases exhibit the discrepancy; the controls behave as described below. Detailed contribution output preserves `fin: null`, and byte totals remain valid. Actual frequency in retained captures and previous-version behavior are unknown. Workaround: inspect the complete contributions for nullable FIN instead of relying on the incomplete-record inventory. The probe directly shows those nullable fields surviving. ### REA version Source package 6.7.0 at immutable main commit `642f6571ea9d333ea50114c25e648dbb5b8a3c46`. The npm registry latest release was checked before the audit; this reproduction used source modules. ### Runtime and platform Node.js v24.22.1; Linux x86_64; tsx 4.23.15 from the locked dependency installation. No optional analysis provider was used. ### Target and artifact context Synthetic JSON qlog 0.3 event subsets within the selected `aioquic-1.5.0-json-0.3` interpretation profile. Each case contains one trace, one packet-sent event, and one STREAM frame. These fixtures establish parser semantics, not aioquic producer provenance. ### Interface Other — production source boundary shared by CLI/MCP historical-capture inspection and retained qlog views. ### Earliest failure stage Capture or observation — transport-fact extraction in `decodeQlogCapture`, before service or view projection. ### Relevant output or evidence Observed selected values from the single successful probe invocation (exit 0): ```text missing-fin: contribution.fin=null, unique_bytes="10", final_sizes=[], incomplete=[], summary_count=0, page.items=[] null-fin: contribution.fin=null, unique_bytes="10", final_sizes=[], incomplete=[], summary_count=0, page.items=[] false-fin: contribution.fin=false, unique_bytes="10", final_sizes=[], incomplete=[], summary_count=0, page.items=[] true-fin: contribution.fin=true, unique_bytes="10", final_sizes=["10"], incomplete=[], summary_count=0, page.items=[] missing-offset: contribution=null, incomplete=[0], summary_count=1, page.items=[0] runtime=v24.22.1, platform=linux, arch=x64 ``` - Expected source: [supported missing metadata](https://github.com/morluto/rea/blob/642f6571ea9d333ea50114c25e648dbb5b8a3c46/docs/web-network-captures.md#L788-L795), [FIN interpretation and incomplete-record definition](https://github.com/morluto/rea/blob/642f6571ea9d333ea50114c25e648dbb5b8a3c46/docs/web-network-captures.md#L818-L824). - Failure source: the executed probe above; [the collector's missing-field test](https://github.com/morluto/rea/blob/642f6571ea9d333ea50114c25e648dbb5b8a3c46/src/browser/history/QlogCapture.ts#L280-L287) and [nullable FIN contribution](https://github.com/morluto/rea/blob/642f6571ea9d333ea50114c25e648dbb5b8a3c46/src/browser/history/QlogCapture.ts#L308-L314) explain its observed result. - Evidence provenance: observed — direct execution of the production decoder, payload schema, summary and page projector; public call-path reachability is source-derived. - Local verification: reproduced — source boundary only; CLI/MCP end-to-end execution was not run. - Reproduction completeness: complete for the stated source-boundary discrepancy; no shipped transport or real-producer claim is made. Supported reachability and guards: `format: "qlog"` is a [public capture format](https://github.com/morluto/rea/blob/642f6571ea9d333ea50114c25e648dbb5b8a3c46/src/domain/webNetworkCapture.ts#L43-L60), and the public request schema accepts the probe request. The normal [Node capture decoder](https://github.com/morluto/rea/blob/642f6571ea9d333ea50114c25e648dbb5b8a3c46/src/browser/history/NodeCaptureProcess.ts#L22-L35) invokes this exact production function. The fixture supplies the required JSON container, trace/events array, string name, finite numeric time, object data, and valid QUIC integers. [STREAM validation](https://github.com/morluto/rea/blob/642f6571ea9d333ea50114c25e648dbb5b8a3c46/src/browser/history/QlogCapture.ts#L207-L221) expressly permits absent/null FIN. All outputs passed the production payload schema, and the view request passed its selector schema. The [summary count](https://github.com/morluto/rea/blob/642f6571ea9d333ea50114c25e648dbb5b8a3c46/src/domain/analysisView/qlogCaptureView.ts#L125-L142) and [incomplete-record page](https://github.com/morluto/rea/blob/642f6571ea9d333ea50114c25e648dbb5b8a3c46/src/domain/analysisView/qlogCaptureView.ts#L212-L217) expose the same producer-owned inventory rather than independently losing FIN. ### Typed outcome or recovery expected Restoration check: repeat the five cases. Missing/null FIN should each produce incomplete ordinal `[0]`, summary count `1`, and page item `[0]` while retaining `fin: null` and ten-byte coverage. False FIN should retain no final size and no incomplete entry; true FIN should retain final size `"10"` and no incomplete entry. The missing-offset control must remain incomplete. Passing must not come from rejecting supported partial frames or discarding their usable byte ranges. ### Analysis or suspected cause Source-derived: the incomplete-record branch covers missing stream ID, offset and length, while the subsequent contribution maps unknown FIN to null without adding the ordinal. These are one issue: the summary and focused page correctly propagate an underfilled inventory. A narrower intended definition limited to byte-range inputs would need an explicit contract clarification; the current documentation covers omitted interpretable frame metadata, and FIN has a documented interpretation. ### Additional context Targeted GitHub issue searches were run separately for open and closed issues using `qlog`, `qlog "fin"`, `"incomplete_fact_records"`, `qlog "incomplete"`, and `qlog "final"`. No matching FIN/inventory report was returned. Related [#3133](https://github.com/morluto/rea/issues/3133) owns performance measurement; closed [#3960](https://github.com/morluto/rea/issues/3960) concerns the shared nesting limit; closed [#2016](https://github.com/morluto/rea/issues/2016) concerns broader analysis-efficiency work. None describes this trigger or failed inventory contract. Submission-time duplicate searches are performed separately immediately before filing. Last-known-good version: `NOT_ENOUGH_INFO`; no regression boundary was established. Unknowns: full CLI/MCP behavior was not executed, no live aioquic capture was produced, prevalence and regression range are unknown. The documented empty-inventory caveat about complete logging/network capture does not assert completeness here; the observed failure is omission of metadata already known to be absent in a recorded frame. An independent pre-draft source review agreed that the documented inventory includes FIN. No fix, build, full suite, commit, or issue submission was performed. Privacy scrub: every input is synthetic; the report contains no credentials, private home paths, real target data, or raw capture. AI assistance: investigation, source-boundary probe and draft preparation were AI-assisted; no human submission or review declaration is made. All source/probe verification finished before this draft was written; no later re-verification is part of this report. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: Supported partial STREAM records are absent from the promised incomplete-metadata inventory and its summary/page, impairing discovery of unknown FIN; byte totals and the detailed nullable field remain correct. - Workaround status: available - Workaround basis: The executed probe preserves contribution.fin as null, allowing inspection of complete contributions instead of relying on the underreported inventory.

bsakaguchi closed this issue

2026-10-31 00:45:47 +00:00

bsakaguchi referenced this issue

2026-10-31 01:11:41 +00:00

[PR #4137] [CLOSED] fix(reconstruction): accept artifact comparisons sourced from inspect_artifact #6078