[GH-ISSUE #6836] [Bug] Automatic protobuf decoding ignores illegal MIME encoding and version parameters #4177

New issue

Closed

opened 2026-10-31 00:47:27 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:47:27 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 23, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6836

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

Automatic protobuf interpretation treats application/protobuf;encoding=json, application/protobuf;charset=iso-8859-1, and application/protobuf;version=99 exactly like unparameterized binary protobuf. A two-byte synthetic HAR response is decoded with framing: "raw", status: "complete", error: null, and complete aggregate protobuf coverage in all three cases.

These are reserved parameters with illegal or unsupported values under the Protobuf MIME rules. The source metadata is retained, but no diagnostic says its declared profile was disregarded. This is observed source-boundary execution through the production HAR/protobuf functions, not a public CLI/MCP reproduction.

Reproduction

From the repository root with locked dependencies installed, save this executed script outside the checkout as /tmp/rea-protobuf-mime-repro.mjs, then run node --import tsx /tmp/rea-protobuf-mime-repro.mjs. It uses only an in-memory synthetic HAR and inert bytes 08 01; no application or recorded destination is executed/contacted.

import {pathToFileURL} from 'node:url';
import {createHash} from 'node:crypto';
const repo=process.cwd(), load=p=>import(pathToFileURL(repo+'/'+p));
const {decodeHarCapture}=await load('src/browser/history/HarCapture.ts');
const {prepareCaptureProtobuf,decodeCaptureProtobuf}=await load('src/browser/history/protobuf/ProtobufCapture.ts');
const {ArtifactResourceScope}=await load('src/artifacts/ArtifactResourceScope.ts');
const {inspectWebNetworkCaptureInputSchema,webNetworkCaptureSchema}=await load('src/domain/webNetworkCapture.ts');
const {protobufHar}=await load('tests/fixtures/protobufCapture.ts');
for(const contentType of [
 'application/protobuf',
 'application/protobuf;encoding=binary',
 'application/protobuf;encoding=json',
 'application/protobuf;charset=iso-8859-1',
 'application/protobuf;version=99',
]){
 const text=protobufHar(Buffer.from([8,1]),contentType);
 const capture={...decodeHarCapture(text,[]),artifact:{path:'/tmp/synthetic.har',bytes:Buffer.byteLength(text),sha256:createHash('sha256').update(text).digest('hex')},format:'har',runtime_attribution:'unknown',limitations:[]};
 const input=inspectWebNetworkCaptureInputSchema.parse({capture_path:capture.artifact.path,format:'har',record_ordinals:[0]});
 const resources=new ArtifactResourceScope();
 try{const prepared=await prepareCaptureProtobuf(capture,input);await decodeCaptureProtobuf(capture,input,resources,prepared);
 const checked=webNetworkCaptureSchema.parse(capture),body=checked.records[0].protobuf?.[0];
 console.log(JSON.stringify({contentType,coverage:checked.protobuf_coverage,framing:body?.framing,status:body?.status,fields:body?.fields.map(({number,value})=>({number,value})),error:body?.error??null}));
 }finally{await resources.close();}
}

The five-case probe ran once. The first two cases are controls for a valid implicit/explicit binary encoding. All five final reports passed webNetworkCaptureSchema validation.

Expected behavior

Keep the capture source, but report that automatic interpretation cannot admit the declared MIME profile, or return a typed format/admission failure. Do not silently discard reserved format parameters and report complete successful decoding as the selected profile.

The official Protobuf MIME rules prohibit encoding=json on application/protobuf, disallow a charset on binary encoding, reserve version until a wire format is versioned, and require parsers to fail on unknown or illegal values of those parameters. This concerns reserved encoding parameters; ordinary non-encoding MIME parameters need not change wire framing.

Impact and scope

Recorded unsupported or invalid MIME declarations produce apparently complete binary interpretations instead of explicit format uncertainty. A caller cannot distinguish accepted binary MIME from a discarded encoding/version declaration using the returned decoding status. Raw source bytes and the original content-type string remain available.

Observed only with synthetic HAR base64 response content, automatic framing, and the three declarations above. The public CLI/MCP, published npm runtime, other platforms/capture formats, valid +json content, explicit framing overrides, and real recordings were not tested for this issue.

REA version

rea-agents 6.7.0; source main 9066444f6b8e9ffe1607a9b02cde3895a37db431. The coordinating investigation checked npm latest as 6.7.0; this worker executed source modules.

Runtime and platform

Node.js v26.12.1; Linux 7.2.8-9-omarchy x64; locked tsx 4.23.15. The repository's Node pin is 24.22.1; that pinned runtime was not separately exercised.

Target and artifact context

HAR 1.2 fixture with one base64 response containing 08 01 (field 1 = 1). Only response.content.mimeType varies. No sensitive values or real devices.

Interface

Other

Earliest failure stage

Request parsing or validation

Relevant output or evidence

Evidence provenance: executed production source-boundary probe and source review, compared with the official MIME contract. Local verification: reproduced. Reproduction completeness: complete for these HAR fixtures; public CLI/MCP untested. Probe stderr is empty.

Actual stdout:

{"contentType":"application/protobuf","coverage":{"status":"complete","selected_records":1,"inspected_records":1},"framing":"raw","status":"complete","fields":[{"number":1,"value":"1"}],"error":null}
{"contentType":"application/protobuf;encoding=binary","coverage":{"status":"complete","selected_records":1,"inspected_records":1},"framing":"raw","status":"complete","fields":[{"number":1,"value":"1"}],"error":null}
{"contentType":"application/protobuf;encoding=json","coverage":{"status":"complete","selected_records":1,"inspected_records":1},"framing":"raw","status":"complete","fields":[{"number":1,"value":"1"}],"error":null}
{"contentType":"application/protobuf;charset=iso-8859-1","coverage":{"status":"complete","selected_records":1,"inspected_records":1},"framing":"raw","status":"complete","fields":[{"number":1,"value":"1"}],"error":null}
{"contentType":"application/protobuf;version=99","coverage":{"status":"complete","selected_records":1,"inspected_records":1},"framing":"raw","status":"complete","fields":[{"number":1,"value":"1"}],"error":null}

Workaround control: running the same source-boundary inputs with record_ordinals omitted leaves retained_body_bytes: 2 and has_protobuf: false for every case. The source remains inspectable without publishing the automatic binary interpretation. That control also passed final capture-schema validation, with empty stderr.

Typed outcome or recovery expected

Preserve the recorded body and explain the unsupported/invalid MIME declaration before automatic decoding. A partial/unknown interpretation or an applicable typed format failure should identify the problematic parameter; an explicit caller-requested interpretation can be considered separately.

Analysis or suspected cause

Source-derived: mediaTypeEssence discards everything after the first semicolon. automaticFraming then selects raw from the remaining application/protobuf essence, and body preparation sees a known, usable declaration. No owner checks the reserved parameter values.

Additional context

The supplied open issue index and all-state searches for protobuf MIME parameter and protobuf encoding=json found no matching report. Related #5926 concerns excluded/conflicting Content-Type metadata; this case uses fully visible MIME values with reserved encoding parameters. #5942 concerns base64 failure accounting.

Prepared with Codex from synthetic source probes and the official Protobuf MIME reference. Verification finished before drafting.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: Three executed invalid/unsupported MIME declarations produce complete binary protobuf facts and aggregate coverage, identically to valid binary controls. The documented MIME rules require rejecting their reserved parameter values; the source remains intact.
Workaround status: available
Workaround basis: An executed control omitted record_ordinals, retained both source bytes in each binary sidecar, and produced no protobuf interpretation. Users can inspect the retained source without automatic decoding until the recorded format declaration is resolved.

Originally created by @coygeek on GitHub (Oct 23, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6836 ## Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ## Area Evidence, artifacts, or reconstruction ## Observed behavior Automatic protobuf interpretation treats `application/protobuf;encoding=json`, `application/protobuf;charset=iso-8859-1`, and `application/protobuf;version=99` exactly like unparameterized binary protobuf. A two-byte synthetic HAR response is decoded with `framing: "raw"`, `status: "complete"`, `error: null`, and complete aggregate protobuf coverage in all three cases. These are reserved parameters with illegal or unsupported values under the Protobuf MIME rules. The source metadata is retained, but no diagnostic says its declared profile was disregarded. This is observed source-boundary execution through the production HAR/protobuf functions, not a public CLI/MCP reproduction. ## Reproduction From the repository root with locked dependencies installed, save this executed script outside the checkout as `/tmp/rea-protobuf-mime-repro.mjs`, then run `node --import tsx /tmp/rea-protobuf-mime-repro.mjs`. It uses only an in-memory synthetic HAR and inert bytes `08 01`; no application or recorded destination is executed/contacted. ```javascript import {pathToFileURL} from 'node:url'; import {createHash} from 'node:crypto'; const repo=process.cwd(), load=p=>import(pathToFileURL(repo+'/'+p)); const {decodeHarCapture}=await load('src/browser/history/HarCapture.ts'); const {prepareCaptureProtobuf,decodeCaptureProtobuf}=await load('src/browser/history/protobuf/ProtobufCapture.ts'); const {ArtifactResourceScope}=await load('src/artifacts/ArtifactResourceScope.ts'); const {inspectWebNetworkCaptureInputSchema,webNetworkCaptureSchema}=await load('src/domain/webNetworkCapture.ts'); const {protobufHar}=await load('tests/fixtures/protobufCapture.ts'); for(const contentType of [ 'application/protobuf', 'application/protobuf;encoding=binary', 'application/protobuf;encoding=json', 'application/protobuf;charset=iso-8859-1', 'application/protobuf;version=99', ]){ const text=protobufHar(Buffer.from([8,1]),contentType); const capture={...decodeHarCapture(text,[]),artifact:{path:'/tmp/synthetic.har',bytes:Buffer.byteLength(text),sha256:createHash('sha256').update(text).digest('hex')},format:'har',runtime_attribution:'unknown',limitations:[]}; const input=inspectWebNetworkCaptureInputSchema.parse({capture_path:capture.artifact.path,format:'har',record_ordinals:[0]}); const resources=new ArtifactResourceScope(); try{const prepared=await prepareCaptureProtobuf(capture,input);await decodeCaptureProtobuf(capture,input,resources,prepared); const checked=webNetworkCaptureSchema.parse(capture),body=checked.records[0].protobuf?.[0]; console.log(JSON.stringify({contentType,coverage:checked.protobuf_coverage,framing:body?.framing,status:body?.status,fields:body?.fields.map(({number,value})=>({number,value})),error:body?.error??null})); }finally{await resources.close();} } ``` The five-case probe ran once. The first two cases are controls for a valid implicit/explicit binary encoding. All five final reports passed `webNetworkCaptureSchema` validation. ## Expected behavior Keep the capture source, but report that automatic interpretation cannot admit the declared MIME profile, or return a typed format/admission failure. Do not silently discard reserved format parameters and report complete successful decoding as the selected profile. The official [Protobuf MIME rules](https://protobuf.dev/reference/protobuf/mime-types/) prohibit `encoding=json` on `application/protobuf`, disallow a charset on binary encoding, reserve `version` until a wire format is versioned, and require parsers to fail on unknown or illegal values of those parameters. This concerns reserved encoding parameters; ordinary non-encoding MIME parameters need not change wire framing. ## Impact and scope Recorded unsupported or invalid MIME declarations produce apparently complete binary interpretations instead of explicit format uncertainty. A caller cannot distinguish accepted binary MIME from a discarded encoding/version declaration using the returned decoding status. Raw source bytes and the original content-type string remain available. Observed only with synthetic HAR base64 response content, automatic framing, and the three declarations above. The public CLI/MCP, published npm runtime, other platforms/capture formats, valid `+json` content, explicit framing overrides, and real recordings were not tested for this issue. ## REA version `rea-agents` 6.7.0; source main `9066444f6b8e9ffe1607a9b02cde3895a37db431`. The coordinating investigation checked npm latest as 6.7.0; this worker executed source modules. ## Runtime and platform Node.js v26.12.1; Linux 7.2.8-9-omarchy x64; locked `tsx` 4.23.15. The repository's Node pin is 24.22.1; that pinned runtime was not separately exercised. ## Target and artifact context HAR 1.2 fixture with one base64 response containing `08 01` (`field 1 = 1`). Only `response.content.mimeType` varies. No sensitive values or real devices. ## Interface Other ## Earliest failure stage Request parsing or validation ## Relevant output or evidence Evidence provenance: executed production source-boundary probe and source review, compared with the official MIME contract. Local verification: reproduced. Reproduction completeness: complete for these HAR fixtures; public CLI/MCP untested. Probe stderr is empty. Actual stdout: ```jsonl {"contentType":"application/protobuf","coverage":{"status":"complete","selected_records":1,"inspected_records":1},"framing":"raw","status":"complete","fields":[{"number":1,"value":"1"}],"error":null} {"contentType":"application/protobuf;encoding=binary","coverage":{"status":"complete","selected_records":1,"inspected_records":1},"framing":"raw","status":"complete","fields":[{"number":1,"value":"1"}],"error":null} {"contentType":"application/protobuf;encoding=json","coverage":{"status":"complete","selected_records":1,"inspected_records":1},"framing":"raw","status":"complete","fields":[{"number":1,"value":"1"}],"error":null} {"contentType":"application/protobuf;charset=iso-8859-1","coverage":{"status":"complete","selected_records":1,"inspected_records":1},"framing":"raw","status":"complete","fields":[{"number":1,"value":"1"}],"error":null} {"contentType":"application/protobuf;version=99","coverage":{"status":"complete","selected_records":1,"inspected_records":1},"framing":"raw","status":"complete","fields":[{"number":1,"value":"1"}],"error":null} ``` Workaround control: running the same source-boundary inputs with `record_ordinals` omitted leaves `retained_body_bytes: 2` and `has_protobuf: false` for every case. The source remains inspectable without publishing the automatic binary interpretation. That control also passed final capture-schema validation, with empty stderr. ## Typed outcome or recovery expected Preserve the recorded body and explain the unsupported/invalid MIME declaration before automatic decoding. A partial/unknown interpretation or an applicable typed format failure should identify the problematic parameter; an explicit caller-requested interpretation can be considered separately. ## Analysis or suspected cause Source-derived: [`mediaTypeEssence`](https://github.com/morluto/rea/blob/9066444f6b8e9ffe1607a9b02cde3895a37db431/src/browser/history/protobuf/ProtobufCapture.ts#L96-L98) discards everything after the first semicolon. [`automaticFraming`](https://github.com/morluto/rea/blob/9066444f6b8e9ffe1607a9b02cde3895a37db431/src/browser/history/protobuf/ProtobufCapture.ts#L170-L193) then selects `raw` from the remaining `application/protobuf` essence, and [body preparation](https://github.com/morluto/rea/blob/9066444f6b8e9ffe1607a9b02cde3895a37db431/src/browser/history/protobuf/ProtobufCapture.ts#L287-L300) sees a known, usable declaration. No owner checks the reserved parameter values. ## Additional context The supplied open issue index and all-state searches for `protobuf MIME parameter` and `protobuf encoding=json` found no matching report. Related #5926 concerns excluded/conflicting Content-Type metadata; this case uses fully visible MIME values with reserved encoding parameters. #5942 concerns base64 failure accounting. Prepared with Codex from synthetic source probes and the official Protobuf MIME reference. Verification finished before drafting. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: Three executed invalid/unsupported MIME declarations produce complete binary protobuf facts and aggregate coverage, identically to valid binary controls. The documented MIME rules require rejecting their reserved parameter values; the source remains intact. - Workaround status: available - Workaround basis: An executed control omitted record_ordinals, retained both source bytes in each binary sidecar, and produced no protobuf interpretation. Users can inspect the retained source without automatic decoding until the recorded format declaration is resolved.

bsakaguchi closed this issue

2026-10-31 00:47:28 +00:00

bsakaguchi referenced this issue

2026-10-31 01:12:00 +00:00

[PR #4198] fix(ghidra): prevent loader relocations from constant folding #6120