morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #6889] [Bug] Unsupported HAR body encodings are treated as decoded response text #4224

New issue

Closed

opened 2026-10-31 00:47:50 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-31 00:47:50 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 23, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6889

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

A successfully imported HAR with response.content.encoding: "hex" or "base64url" loses that distinction at the response-body reader. readRecordedResponseBody returns the encoded representation as ordinary decoded bodyText, and the production replay classifier treats it as the response text. For the same five-byte hello resource, the two outputs are 68656c6c6f and aGVsbG8, respectively. Plain-text and supported base64 controls both return hello.

This is observed production source-boundary execution, including HAR import, final capture-schema validation, body reading, and replay classification. No replay server, public CLI, or MCP session was started.

Reproduction

Use the source checkout and locked dependencies with the runtime below. Save this script as /tmp/har-body-encoding.mjs, then run it from the checkout:

FIXTURE_TIME=$(TZ=America/Los_Angeles date '+%Y-%m-%dT%H:%M:%S%:z') node --import ./node_modules/tsx/dist/loader.mjs /tmp/har-body-encoding.mjs

import { pathToFileURL } from 'node:url';
import { createHash } from 'node:crypto';
const load = p => import(pathToFileURL(`${process.cwd()}/${p}`));
const { decodeHarCapture } = await load('src/browser/history/HarCapture.ts');
const { webNetworkCaptureSchema } = await load('src/domain/webNetworkCapture.ts');
const { readRecordedResponseBody } = await load('src/domain/analysisView/recordedResponseBody.ts');
const { recordedExchange, replayedBody } = await load('src/application/environment/environmentSource.ts');
for (const [name, text, encoding] of [
 ['plain-control', 'hello', undefined],
 ['base64-control', 'aGVsbG8=', 'base64'],
 ['hex-encoding', '68656c6c6f', 'hex'],
 ['base64url-encoding', 'aGVsbG8', 'base64url'],
]) {
 const har = {log:{version:'1.2',creator:{name:'local fixture',version:'1'},entries:[{
  startedDateTime:process.env.FIXTURE_TIME,time:0,
  request:{method:'GET',url:'https://example.invalid/message',httpVersion:'HTTP/1.1',cookies:[],headers:[],queryString:[],headersSize:-1,bodySize:0},
  response:{status:200,statusText:'OK',httpVersion:'HTTP/1.1',cookies:[],headers:[],redirectURL:'',headersSize:-1,bodySize:5,content:{size:5,mimeType:'text/plain',text,...(encoding===undefined?{}:{encoding})}},
  cache:{},timings:{send:0,wait:0,receive:0}
 }]}};
 const serialized=JSON.stringify(har);
 const capture=webNetworkCaptureSchema.parse({...decodeHarCapture(serialized,[]),artifact:{path:'/tmp/fixture.har',bytes:Buffer.byteLength(serialized),sha256:createHash('sha256').update(serialized).digest('hex')},format:'har',runtime_attribution:'unknown',limitations:[]});
 const record=capture.records[0];
 console.log(JSON.stringify({name,encoding:record.reported.response.content.encoding??null,read:readRecordedResponseBody(record,262144),replayed:replayedBody(recordedExchange(record))}));
}

Each case ran in an independent fixture in the retained probe. The listing keeps those four cases and omits unrelated timing-validation cases from the executed probe. Executable and temporary paths are expressed generically for portability. Both unsupported-encoding cases reproduced in the initial four-case probe and the final combined probe before drafting.

Expected behavior

Preserve the original encoded text and its encoding in imported evidence. When the response reader does not support a declared encoding, keep its decoded body unavailable or explicitly opaque instead of presenting the encoding's characters as decoded response text. Adding decoders for additional encodings is not required to restore this distinction.

The HAR 1.2 content definition distinguishes HTTP-decoded text from a representation with an explicit encoding. The checked-in RecordedResponseBody.bodyText contract also describes producer-decoded response text. A schema-admitted arbitrary encoding string does not establish that no decoding is needed.

Impact and scope

P2: downstream body consumers receive the wrong text for HAR encodings other than the supported base64 value, without an unsupported-encoding signal. The original imported fields remain intact. The observed fixtures are small synthetic HAR responses with text/plain content; prevalence in real producer captures is unknown. This report does not assert that REA must implement hex or base64url decoding.

The independently constructed plain-text and supported base64 controls produce the expected hello body. Re-exporting or converting an affected capture into one of those representations is a possible workaround, but neither conversion nor re-export was exercised, so practical result-preserving availability remains unverified. Public environment serving, state extraction, published packages, and other platforms were not exercised.

REA version

rea-agents package metadata 6.7.0; sole tested source revision d0432d63c9fd359ee722c232993523765fe84cf9. No released-package reproduction is claimed.

Runtime and platform

Node.js v24.22.1; Linux x86_64; locked clone-local tsx loader and dependencies. No external capture provider or recorded destination ran.

Target and artifact context

Four independent benign HAR 1.2 fixtures representing the same hello resource. The only response-content changes are text and optional encoding. URLs use example.invalid and are never contacted.

Interface

Other

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

Evidence provenance: observed production source-boundary execution. Local verification: reproduced. Reproduction completeness: complete for the four source-boundary fixtures; public CLI/MCP and an actual serving session are untested. The final probe exited 0 with empty stderr; every imported capture passed webNetworkCaptureSchema.

Actual retained output:

{"name":"plain-control","encoding":null,"read":{"bodyText":"hello"},"replayed":{"kind":"text","text":"hello"}}
{"name":"base64-control","encoding":"base64","read":{"bodyText":"hello"},"replayed":{"kind":"text","text":"hello"}}
{"name":"hex-encoding","encoding":"hex","read":{"bodyText":"68656c6c6f"},"replayed":{"kind":"text","text":"68656c6c6f"}}
{"name":"base64url-encoding","encoding":"base64url","read":{"bodyText":"aGVsbG8"},"replayed":{"kind":"text","text":"aGVsbG8"}}

Exact source boundaries: recordedResponseBody.ts reads response.content.encoding, then line 40 treats every value other than base64 as plain decoded text. src/application/environment/environmentSource.ts:173-185 passes that result into recordedExchange; lines 77-100 classify it as ordinary text. The HAR import retains the original encoding at src/browser/history/HarCapture.ts:146-197; the failure begins in the consumer, not the stored source fields. The HAR-only admission path at environmentSource.ts:317-352 accepts the same validated capture format; a full public call was not executed.

Typed outcome or recovery expected

An unsupported declared body encoding must remain distinguishable from decoded text. Preserve the producer's source facts and identify unavailable decoding rather than quietly treating its encoded characters as the body.

Analysis or suspected cause

Source-derived: the body reader equates encoding !== "base64" with an absent encoding. That branch combines a known plain-text case with unsupported declared encodings, so the result has neither an unavailable body nor a retained encoding marker.

Additional context

The supplied local open-issue titles and full bodies were searched for HAR/body/base64 terms, the reader's name, content.encoding, base64url, and unknown/unsupported HAR encodings. No matching report was found. Open #6836 concerns reserved parameters inside protobuf MIME values; this finding concerns HAR's separate response.content.encoding field. The submitting agent also searched current open and closed issues before publication. All bug reproduction finished before drafting, with no post-draft rerun.

Restoration check: unsupported declared encodings must not emerge as ordinary decoded body text, while the existing plain-text and canonical base64 controls must continue to return hello; original imported fields must remain preserved.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: Two executed schema-admitted encoding declarations produce the representation's characters as decoded text and ordinary replay content, while both supported controls produce the intended resource. This is bounded incorrect interpretation; original evidence is preserved and real-capture prevalence is unknown.
Workaround status: unverified
Workaround basis: Independent plain-text and canonical base64 fixture controls return hello, but no affected capture was converted or re-exported. The controls do not establish a practical result-preserving workaround for affected users.

Originally created by @coygeek on GitHub (Oct 23, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6889 ## Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ## Area Evidence, artifacts, or reconstruction ## Observed behavior A successfully imported HAR with `response.content.encoding: "hex"` or `"base64url"` loses that distinction at the response-body reader. `readRecordedResponseBody` returns the encoded representation as ordinary decoded `bodyText`, and the production replay classifier treats it as the response text. For the same five-byte `hello` resource, the two outputs are `68656c6c6f` and `aGVsbG8`, respectively. Plain-text and supported base64 controls both return `hello`. This is observed production source-boundary execution, including HAR import, final capture-schema validation, body reading, and replay classification. No replay server, public CLI, or MCP session was started. ## Reproduction Use the source checkout and locked dependencies with the runtime below. Save this script as `/tmp/har-body-encoding.mjs`, then run it from the checkout: ```sh FIXTURE_TIME=$(TZ=America/Los_Angeles date '+%Y-%m-%dT%H:%M:%S%:z') node --import ./node_modules/tsx/dist/loader.mjs /tmp/har-body-encoding.mjs ``` ```javascript import { pathToFileURL } from 'node:url'; import { createHash } from 'node:crypto'; const load = p => import(pathToFileURL(`${process.cwd()}/${p}`)); const { decodeHarCapture } = await load('src/browser/history/HarCapture.ts'); const { webNetworkCaptureSchema } = await load('src/domain/webNetworkCapture.ts'); const { readRecordedResponseBody } = await load('src/domain/analysisView/recordedResponseBody.ts'); const { recordedExchange, replayedBody } = await load('src/application/environment/environmentSource.ts'); for (const [name, text, encoding] of [ ['plain-control', 'hello', undefined], ['base64-control', 'aGVsbG8=', 'base64'], ['hex-encoding', '68656c6c6f', 'hex'], ['base64url-encoding', 'aGVsbG8', 'base64url'], ]) { const har = {log:{version:'1.2',creator:{name:'local fixture',version:'1'},entries:[{ startedDateTime:process.env.FIXTURE_TIME,time:0, request:{method:'GET',url:'https://example.invalid/message',httpVersion:'HTTP/1.1',cookies:[],headers:[],queryString:[],headersSize:-1,bodySize:0}, response:{status:200,statusText:'OK',httpVersion:'HTTP/1.1',cookies:[],headers:[],redirectURL:'',headersSize:-1,bodySize:5,content:{size:5,mimeType:'text/plain',text,...(encoding===undefined?{}:{encoding})}}, cache:{},timings:{send:0,wait:0,receive:0} }]}}; const serialized=JSON.stringify(har); const capture=webNetworkCaptureSchema.parse({...decodeHarCapture(serialized,[]),artifact:{path:'/tmp/fixture.har',bytes:Buffer.byteLength(serialized),sha256:createHash('sha256').update(serialized).digest('hex')},format:'har',runtime_attribution:'unknown',limitations:[]}); const record=capture.records[0]; console.log(JSON.stringify({name,encoding:record.reported.response.content.encoding??null,read:readRecordedResponseBody(record,262144),replayed:replayedBody(recordedExchange(record))})); } ``` Each case ran in an independent fixture in the retained probe. The listing keeps those four cases and omits unrelated timing-validation cases from the executed probe. Executable and temporary paths are expressed generically for portability. Both unsupported-encoding cases reproduced in the initial four-case probe and the final combined probe before drafting. ## Expected behavior Preserve the original encoded text and its encoding in imported evidence. When the response reader does not support a declared encoding, keep its decoded body unavailable or explicitly opaque instead of presenting the encoding's characters as decoded response text. Adding decoders for additional encodings is not required to restore this distinction. The [HAR 1.2 content definition](https://github.com/ahmadnassri/har-spec/blob/master/versions/1.2.md#content) distinguishes HTTP-decoded text from a representation with an explicit `encoding`. The checked-in `RecordedResponseBody.bodyText` contract also describes producer-decoded response text. A schema-admitted arbitrary encoding string does not establish that no decoding is needed. ## Impact and scope P2: downstream body consumers receive the wrong text for HAR encodings other than the supported `base64` value, without an unsupported-encoding signal. The original imported fields remain intact. The observed fixtures are small synthetic HAR responses with text/plain content; prevalence in real producer captures is unknown. This report does not assert that REA must implement hex or base64url decoding. The independently constructed plain-text and supported base64 controls produce the expected `hello` body. Re-exporting or converting an affected capture into one of those representations is a possible workaround, but neither conversion nor re-export was exercised, so practical result-preserving availability remains unverified. Public environment serving, state extraction, published packages, and other platforms were not exercised. ## REA version `rea-agents` package metadata 6.7.0; sole tested source revision `d0432d63c9fd359ee722c232993523765fe84cf9`. No released-package reproduction is claimed. ## Runtime and platform Node.js v24.22.1; Linux x86_64; locked clone-local tsx loader and dependencies. No external capture provider or recorded destination ran. ## Target and artifact context Four independent benign HAR 1.2 fixtures representing the same `hello` resource. The only response-content changes are `text` and optional `encoding`. URLs use `example.invalid` and are never contacted. ## Interface Other ## Earliest failure stage Native analysis or reconstruction ## Relevant output or evidence Evidence provenance: observed production source-boundary execution. Local verification: reproduced. Reproduction completeness: complete for the four source-boundary fixtures; public CLI/MCP and an actual serving session are untested. The final probe exited 0 with empty stderr; every imported capture passed `webNetworkCaptureSchema`. Actual retained output: ```jsonl {"name":"plain-control","encoding":null,"read":{"bodyText":"hello"},"replayed":{"kind":"text","text":"hello"}} {"name":"base64-control","encoding":"base64","read":{"bodyText":"hello"},"replayed":{"kind":"text","text":"hello"}} {"name":"hex-encoding","encoding":"hex","read":{"bodyText":"68656c6c6f"},"replayed":{"kind":"text","text":"68656c6c6f"}} {"name":"base64url-encoding","encoding":"base64url","read":{"bodyText":"aGVsbG8"},"replayed":{"kind":"text","text":"aGVsbG8"}} ``` Exact source boundaries: [`recordedResponseBody.ts`](https://github.com/morluto/rea/blob/d0432d63c9fd359ee722c232993523765fe84cf9/src/domain/analysisView/recordedResponseBody.ts#L32-L61) reads `response.content.encoding`, then line 40 treats every value other than `base64` as plain decoded text. `src/application/environment/environmentSource.ts:173-185` passes that result into `recordedExchange`; lines 77-100 classify it as ordinary text. The HAR import retains the original encoding at `src/browser/history/HarCapture.ts:146-197`; the failure begins in the consumer, not the stored source fields. The HAR-only admission path at `environmentSource.ts:317-352` accepts the same validated capture format; a full public call was not executed. ## Typed outcome or recovery expected An unsupported declared body encoding must remain distinguishable from decoded text. Preserve the producer's source facts and identify unavailable decoding rather than quietly treating its encoded characters as the body. ## Analysis or suspected cause Source-derived: the body reader equates `encoding !== "base64"` with an absent encoding. That branch combines a known plain-text case with unsupported declared encodings, so the result has neither an unavailable body nor a retained encoding marker. ## Additional context The supplied local open-issue titles and full bodies were searched for HAR/body/base64 terms, the reader's name, `content.encoding`, `base64url`, and unknown/unsupported HAR encodings. No matching report was found. Open #6836 concerns reserved parameters inside protobuf MIME values; this finding concerns HAR's separate `response.content.encoding` field. The submitting agent also searched current open and closed issues before publication. All bug reproduction finished before drafting, with no post-draft rerun. Restoration check: unsupported declared encodings must not emerge as ordinary decoded body text, while the existing plain-text and canonical base64 controls must continue to return `hello`; original imported fields must remain preserved. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: Two executed schema-admitted encoding declarations produce the representation's characters as decoded text and ordinary replay content, while both supported controls produce the intended resource. This is bounded incorrect interpretation; original evidence is preserved and real-capture prevalence is unknown. - Workaround status: unverified - Workaround basis: Independent plain-text and canonical base64 fixture controls return `hello`, but no affected capture was converted or re-exported. The controls do not establish a practical result-preserving workaround for affected users.

bsakaguchi closed this issue

2026-10-31 00:47:51 +00:00

bsakaguchi commented

2026-10-31 00:47:51 +00:00

Author

Owner

@gongfpp commented on GitHub (Oct 23, 2026):

I am testing a narrow fix at the shared retained-response-body reader: only absent content.encoding is interpreted as plain text; unsupported encodings remain unavailable to replay/state extraction while the original HAR Evidence is retained. The existing base64 decoder and its binary/truncation behavior stay unchanged. I have reproduced the bad replay through the built CLI and MCP client, and am checking supported/unsupported controls and the state projection. If someone is already covering this scope, please let me know so we can coordinate; this is not a claim of assignment or maintainer approval.

@gongfpp commented on GitHub (Oct 23, 2026): I am testing a narrow fix at the shared retained-response-body reader: only absent content.encoding is interpreted as plain text; unsupported encodings remain unavailable to replay/state extraction while the original HAR Evidence is retained. The existing base64 decoder and its binary/truncation behavior stay unchanged. I have reproduced the bad replay through the built CLI and MCP client, and am checking supported/unsupported controls and the state projection. If someone is already covering this scope, please let me know so we can coordinate; this is not a claim of assignment or maintainer approval.

bsakaguchi commented

2026-10-31 00:47:51 +00:00

Author

Owner

@gongfpp commented on GitHub (Oct 23, 2026):

Main now fixes the shared response-body reader in 254a8e7018, so I will not open another production fix for #6889. I am retaining the local supported/unsupported HAR and real served-MCP regression work rather than competing with the completed fix. The earlier local validation was not a test of the new main commit.

@gongfpp commented on GitHub (Oct 23, 2026): Main now fixes the shared response-body reader in 254a8e7018e6b984d1cbc607e11dbab0866c9ff7, so I will not open another production fix for #6889. I am retaining the local supported/unsupported HAR and real served-MCP regression work rather than competing with the completed fix. The earlier local validation was not a test of the new main commit.

bsakaguchi referenced this issue

2026-10-31 01:12:09 +00:00

[PR #4224] fix(browser): preserve uncertainty for omitted initiator URLs (#4211) #6136

bsakaguchi referenced this issue

2026-11-01 21:10:53 +00:00

[GH-ISSUE #3582] [Tracker] Ready to close or merge: issues and pull requests (October 2026) #1330

bsakaguchi referenced this issue

2026-11-01 21:11:07 +00:00

[GH-ISSUE #5654] [Tracker] Open PRs that can't merge yet: conflicts and red CI (October 2026) #3425