morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #3959] [Bug] Discarded Cookie values exhaust the retained-header budget and hide ordinary headers #1566

New issue

Closed

opened 2026-10-16 21:07:13 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-16 21:07:13 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 15, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3959

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Process capture or runtime observation

Observed behavior

A browser scenario that requests network header values can lose the entire header set because a Cookie value exceeds a limit for retained header allocation, even though REA would retain that credential value only as null. In a simulated-browser CLI reproduction, a 2000-character Cookie caused an ordinary X-Ordinary value to disappear and made the capture incomplete. Emptying the Cookie at the same budget, or raising the budget with the original Cookie, restored both header items with credential redaction intact.

Reproduction

From the baseline checkout with its dependencies and built runtime available, use the pinned Node.js version. Create a disposable fixture directory and save this browser substitute there. The import hook replaces browser opening and directs bundled composition to the existing production modules. It does not replace header capture or synthesize the final Evidence.

fixture_dir="$(mktemp -d)"
mkdir "$fixture_dir/home"
cat > "$fixture_dir/browser-fixture.mjs" <<'BROWSER_FIXTURE'
import { EventEmitter } from 'node:events';
import { registerHooks } from 'node:module';
import { resolve } from 'node:path';
import { pathToFileURL } from 'node:url';
const root = process.cwd();
const urlFor = path => pathToFileURL(resolve(root, path)).href;
const composition = urlFor('dist/composition/browserScenario.js');
const bundledComposition = urlFor('dist/cli-runtime/composition__browserScenario-');
const browserModule = urlFor('dist/browser/PlaywrightScenarioBrowser.js');
export async function openPlaywrightScenarioBrowser() {
  const context = Object.assign(new EventEmitter(), {
    setDefaultTimeout() {}, setDefaultNavigationTimeout() {},
    async addCookies() {},
  });
  let currentUrl = 'about:blank';
  const cookieLength = Number(process.env.FIXTURE_COOKIE_LENGTH ?? 2000);
  const headers = [{name:'Cookie', value:'c'.repeat(cookieLength)}, {name:'X-Ordinary', value:'keep-me'}];
  const request = {
    method:()=> 'GET', url:()=> currentUrl, resourceType:()=> 'document',
    headers:()=> Object.fromEntries(headers.map(({name,value})=>[name.toLowerCase(),value])),
    headersArray:async()=>headers, redirectedFrom:()=>null,
    postDataBuffer:()=>null, failure:()=>null,
  };
  const page = Object.assign(new EventEmitter(), {
    url:()=>currentUrl, context:()=>context,
    async goto(url) { currentUrl=url; context.emit('request',request); context.emit('requestfinished',request); },
    async waitForTimeout() {},
  });
  const browser = { version:()=> 'Controlled browser fixture' };
  const cleanup = { browserClosed:false, async close(finish) { await finish?.(); this.browserClosed=true; } };
  return {page, context, browser, cleanup};
}
registerHooks({ load(url, context, nextLoad) {
  if (url.startsWith(bundledComposition)) return {format:'module',shortCircuit:true,source:`export {createBrowserScenarioProvider} from ${JSON.stringify(composition)};`};
  if (url === browserModule) return {format:'module',shortCircuit:true,source:`export * from ${JSON.stringify(browserModule+'?actual')}; export {openPlaywrightScenarioBrowser} from ${JSON.stringify(import.meta.url)};`};
  return nextLoad(url,context);
}});
BROWSER_FIXTURE

Run this driver from the same checkout. It invokes node scripts/rea.mjs capture-browser-scenario <inputJson> --json three times with an isolated HOME, a declared fixture secret, header values selected, and the indicated Cookie length and header budget. Each fake navigation emits a request and requestfinished through the production event listeners. The first case is the failure; the other cases are controls. The driver writes the complete returned JSON into the disposable directory and prints the relevant header and completeness states.

FIXTURE_DIR="$fixture_dir" node --input-type=module <<'CLI_DRIVER'
import { execFileSync } from 'node:child_process';
import { writeFileSync } from 'node:fs';
const scratch = process.env.FIXTURE_DIR;
for (const [label,cookieLength,budget] of [['large-cookie',2000,4096],['empty-cookie',0,4096],['raised-budget',2000,1048576]]) {
  const scenario = {browser:{mode:'launch',executable_path:'/controlled/chromium'},start_url:{url:'https://app.example.test/'},actions:[{step_id:'wait',action:'wait_for_timeout',duration_ms:1}],secrets:[{secret_id:'declared',environment_variable:'FIXTURE_SECRET'}],capture:{after_each_step:['url'],at_end:[],events:[],network:{header_values:true},limits:{max_network_header_allocation_bytes:budget}}};
  const args=['--import',scratch+'/browser-fixture.mjs','scripts/rea.mjs','capture-browser-scenario',JSON.stringify(scenario),'--json'];
  try {
    const stdout=execFileSync(process.execPath,args,{env:{...process.env,HOME:scratch+'/home',FIXTURE_COOKIE_LENGTH:String(cookieLength),FIXTURE_SECRET:'fixture-secret-value'},encoding:'utf8',timeout:30000});
    writeFileSync(scratch+'/'+label+'.json',stdout);
    const doc=JSON.parse(stdout);const result=doc.normalized_result ?? doc;
    console.log(JSON.stringify({label,operation:doc.operation,headers:result.events?.items?.filter(e=>e.kind==='network-content').map(e=>e.headers),completeness:result.completeness,error:doc.error}));
  } catch(error) { console.log(JSON.stringify({label,status:error.status,stdout:String(error.stdout),stderr:String(error.stderr)})); }
}
CLI_DRIVER

After inspecting the output, remove the disposable directory with rm -r "$fixture_dir". These steps use no network and do not modify repository files. Each of the three cases was executed twice, including a replay of these exact draft commands, and every CLI invocation completed with exit code 0.

Expected behavior

The caller-selected header allocation limit covers retained header output, including the names and ordinary values that survive redaction. Credential values become null with redacted: true, while ordinary values remain intact. Increasing only a credential value that REA discards should not make an otherwise fitting retained header set unavailable. At the tested 4096-byte budget, both items should remain captured: Cookie with a null value and redacted true, and X-Ordinary with keep-me and redacted false.

Impact and scope

Selected captures lose requested ordinary header evidence and become incomplete because of a discarded credential value. The failure is bounded to affected captures and explicitly reported as unavailable; no credential exposure or security-boundary crossing was observed.

Workaround: The executed identical 2000-character Cookie case at a 1048576-byte header budget retained the ordinary value, preserved null credential redaction, and returned a complete capture.

Assessment: P2.

REA version

rea 6.5.0 at source commit 4692bb3330

Runtime and platform

Node 24.18.0; macOS arm64

Target and artifact context

Simulated browser at the browser-opening boundary, with synthetic Cookie and X-Ordinary headers. Production CLI validation, session, collector, header decoder and Evidence projection run. No real Chromium or external service was used.

Interface

CLI

Earliest failure stage

Capture or observation

Relevant output or evidence

Observed with the 2000-character Cookie and a 4096-byte budget:

{"state":"unavailable","message":"Headers exceed the 4096-byte allocation budget."}

The request network-content event had no header items. The capture reported status: "incomplete", equality_eligible: false, and missing_sections: ["events"]. With an empty Cookie at the same budget, and with the original Cookie at a raised 1048576-byte budget, the observed header result was:

{"state":"captured","items":[{"name":"Cookie","value":null,"redacted":true},{"name":"X-Ordinary","value":"keep-me","redacted":false}]}

Both controls reported complete captures eligible for equality. Source-derived reasoning explains the difference: the allocation preflight includes every original header value before credential handling replaces that value with null. The Cookie contents therefore consume the budget without becoming retained evidence. This explanation is corroborated by the control that changes only the discarded value's length.

Expected source: docs/browser-scenario-contract.md:184-205 permits caller-selected positive retention limits and calls the header limit retained-header allocation; docs/browser-scenario-contract.md:314-319 requires credential values to be null/redacted while ordinary values remain intact.
Failure source: the simulated-browser CLI outputs above; src/browser/network/PlaywrightNetworkContent.ts:81-103 accounts for original values before credential replacement; src/browser/network/PlaywrightScenarioNetwork.ts:129-140 and :610-616 carry the selected limit into the header decoder.

The CLI passed request validation and returned operation capture_browser_scenario. Source reachability also follows src/cliBrowserScenarioCommands.ts:39-58, src/composition/browserScenario.ts:9-15, src/browser/PlaywrightScenarioSession.ts:243-251, and src/browser/PlaywrightScenarioEvents.ts:111-156. Selecting header values enables network collection even when the caller's explicit event-family list is empty. Launch ownership admits the fixture request; the default metadata, event, and transaction limits admitted this observation. The ordered Request header API is the production input boundary exercised here. Real-browser behavior and other credential-header names were not executed.

Typed outcome or recovery expected

Repeat the 4096-byte CLI case with the 2000-character Cookie. It must return the same captured header items as the empty-Cookie control, preserve keep-me, and keep the Cookie value null/redacted. The header capture must no longer make these otherwise complete fixture captures ineligible for equality. Keep the fitting empty-Cookie control and check that genuinely excessive ordinary retained header output still respects the selected allocation limit. Raising the limit is an observed workaround, not a verified code fix.

Additional context

Freshness checked against origin/main at 22ec8bdb2f. The cited failing implementation files are unchanged from the reproduced baseline; this is a source freshness check, not a runtime rerun on that revision.

Originally created by @coygeek on GitHub (Oct 15, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3959 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Process capture or runtime observation ### Observed behavior A browser scenario that requests network header values can lose the entire header set because a Cookie value exceeds a limit for retained header allocation, even though REA would retain that credential value only as null. In a simulated-browser CLI reproduction, a 2000-character Cookie caused an ordinary X-Ordinary value to disappear and made the capture incomplete. Emptying the Cookie at the same budget, or raising the budget with the original Cookie, restored both header items with credential redaction intact. ### Reproduction From the baseline checkout with its dependencies and built runtime available, use the pinned Node.js version. Create a disposable fixture directory and save this browser substitute there. The import hook replaces browser opening and directs bundled composition to the existing production modules. It does not replace header capture or synthesize the final Evidence. ```sh fixture_dir="$(mktemp -d)" mkdir "$fixture_dir/home" cat > "$fixture_dir/browser-fixture.mjs" <<'BROWSER_FIXTURE' import { EventEmitter } from 'node:events'; import { registerHooks } from 'node:module'; import { resolve } from 'node:path'; import { pathToFileURL } from 'node:url'; const root = process.cwd(); const urlFor = path => pathToFileURL(resolve(root, path)).href; const composition = urlFor('dist/composition/browserScenario.js'); const bundledComposition = urlFor('dist/cli-runtime/composition__browserScenario-'); const browserModule = urlFor('dist/browser/PlaywrightScenarioBrowser.js'); export async function openPlaywrightScenarioBrowser() { const context = Object.assign(new EventEmitter(), { setDefaultTimeout() {}, setDefaultNavigationTimeout() {}, async addCookies() {}, }); let currentUrl = 'about:blank'; const cookieLength = Number(process.env.FIXTURE_COOKIE_LENGTH ?? 2000); const headers = [{name:'Cookie', value:'c'.repeat(cookieLength)}, {name:'X-Ordinary', value:'keep-me'}]; const request = { method:()=> 'GET', url:()=> currentUrl, resourceType:()=> 'document', headers:()=> Object.fromEntries(headers.map(({name,value})=>[name.toLowerCase(),value])), headersArray:async()=>headers, redirectedFrom:()=>null, postDataBuffer:()=>null, failure:()=>null, }; const page = Object.assign(new EventEmitter(), { url:()=>currentUrl, context:()=>context, async goto(url) { currentUrl=url; context.emit('request',request); context.emit('requestfinished',request); }, async waitForTimeout() {}, }); const browser = { version:()=> 'Controlled browser fixture' }; const cleanup = { browserClosed:false, async close(finish) { await finish?.(); this.browserClosed=true; } }; return {page, context, browser, cleanup}; } registerHooks({ load(url, context, nextLoad) { if (url.startsWith(bundledComposition)) return {format:'module',shortCircuit:true,source:`export {createBrowserScenarioProvider} from ${JSON.stringify(composition)};`}; if (url === browserModule) return {format:'module',shortCircuit:true,source:`export * from ${JSON.stringify(browserModule+'?actual')}; export {openPlaywrightScenarioBrowser} from ${JSON.stringify(import.meta.url)};`}; return nextLoad(url,context); }}); BROWSER_FIXTURE ``` Run this driver from the same checkout. It invokes `node scripts/rea.mjs capture-browser-scenario <inputJson> --json` three times with an isolated HOME, a declared fixture secret, header values selected, and the indicated Cookie length and header budget. Each fake navigation emits a request and requestfinished through the production event listeners. The first case is the failure; the other cases are controls. The driver writes the complete returned JSON into the disposable directory and prints the relevant header and completeness states. ```sh FIXTURE_DIR="$fixture_dir" node --input-type=module <<'CLI_DRIVER' import { execFileSync } from 'node:child_process'; import { writeFileSync } from 'node:fs'; const scratch = process.env.FIXTURE_DIR; for (const [label,cookieLength,budget] of [['large-cookie',2000,4096],['empty-cookie',0,4096],['raised-budget',2000,1048576]]) { const scenario = {browser:{mode:'launch',executable_path:'/controlled/chromium'},start_url:{url:'https://app.example.test/'},actions:[{step_id:'wait',action:'wait_for_timeout',duration_ms:1}],secrets:[{secret_id:'declared',environment_variable:'FIXTURE_SECRET'}],capture:{after_each_step:['url'],at_end:[],events:[],network:{header_values:true},limits:{max_network_header_allocation_bytes:budget}}}; const args=['--import',scratch+'/browser-fixture.mjs','scripts/rea.mjs','capture-browser-scenario',JSON.stringify(scenario),'--json']; try { const stdout=execFileSync(process.execPath,args,{env:{...process.env,HOME:scratch+'/home',FIXTURE_COOKIE_LENGTH:String(cookieLength),FIXTURE_SECRET:'fixture-secret-value'},encoding:'utf8',timeout:30000}); writeFileSync(scratch+'/'+label+'.json',stdout); const doc=JSON.parse(stdout);const result=doc.normalized_result ?? doc; console.log(JSON.stringify({label,operation:doc.operation,headers:result.events?.items?.filter(e=>e.kind==='network-content').map(e=>e.headers),completeness:result.completeness,error:doc.error})); } catch(error) { console.log(JSON.stringify({label,status:error.status,stdout:String(error.stdout),stderr:String(error.stderr)})); } } CLI_DRIVER ``` After inspecting the output, remove the disposable directory with `rm -r "$fixture_dir"`. These steps use no network and do not modify repository files. Each of the three cases was executed twice, including a replay of these exact draft commands, and every CLI invocation completed with exit code 0. ### Expected behavior The caller-selected header allocation limit covers retained header output, including the names and ordinary values that survive redaction. Credential values become null with `redacted: true`, while ordinary values remain intact. Increasing only a credential value that REA discards should not make an otherwise fitting retained header set unavailable. At the tested 4096-byte budget, both items should remain captured: Cookie with a null value and redacted true, and X-Ordinary with keep-me and redacted false. ### Impact and scope Selected captures lose requested ordinary header evidence and become incomplete because of a discarded credential value. The failure is bounded to affected captures and explicitly reported as unavailable; no credential exposure or security-boundary crossing was observed. Workaround: The executed identical 2000-character Cookie case at a 1048576-byte header budget retained the ordinary value, preserved null credential redaction, and returned a complete capture. Assessment: P2. ### REA version rea 6.5.0 at source commit 4692bb33300c7cb77b095b6cc0c33da861c1b860 ### Runtime and platform Node 24.18.0; macOS arm64 ### Target and artifact context Simulated browser at the browser-opening boundary, with synthetic Cookie and X-Ordinary headers. Production CLI validation, session, collector, header decoder and Evidence projection run. No real Chromium or external service was used. ### Interface CLI ### Earliest failure stage Capture or observation ### Relevant output or evidence Observed with the 2000-character Cookie and a 4096-byte budget: ```json {"state":"unavailable","message":"Headers exceed the 4096-byte allocation budget."} ``` The request network-content event had no header items. The capture reported `status: "incomplete"`, `equality_eligible: false`, and `missing_sections: ["events"]`. With an empty Cookie at the same budget, and with the original Cookie at a raised 1048576-byte budget, the observed header result was: ```json {"state":"captured","items":[{"name":"Cookie","value":null,"redacted":true},{"name":"X-Ordinary","value":"keep-me","redacted":false}]} ``` Both controls reported complete captures eligible for equality. Source-derived reasoning explains the difference: the allocation preflight includes every original header value before credential handling replaces that value with null. The Cookie contents therefore consume the budget without becoming retained evidence. This explanation is corroborated by the control that changes only the discarded value's length. - Expected source: [docs/browser-scenario-contract.md:184-205](https://github.com/morluto/rea/blob/4692bb33300c7cb77b095b6cc0c33da861c1b860/docs/browser-scenario-contract.md#L184) permits caller-selected positive retention limits and calls the header limit retained-header allocation; [docs/browser-scenario-contract.md:314-319](https://github.com/morluto/rea/blob/4692bb33300c7cb77b095b6cc0c33da861c1b860/docs/browser-scenario-contract.md#L314) requires credential values to be null/redacted while ordinary values remain intact. - Failure source: the simulated-browser CLI outputs above; [src/browser/network/PlaywrightNetworkContent.ts:81-103](https://github.com/morluto/rea/blob/4692bb33300c7cb77b095b6cc0c33da861c1b860/src/browser/network/PlaywrightNetworkContent.ts#L81) accounts for original values before credential replacement; [src/browser/network/PlaywrightScenarioNetwork.ts:129-140](https://github.com/morluto/rea/blob/4692bb33300c7cb77b095b6cc0c33da861c1b860/src/browser/network/PlaywrightScenarioNetwork.ts#L129) and `:610-616` carry the selected limit into the header decoder. The CLI passed request validation and returned operation `capture_browser_scenario`. Source reachability also follows [src/cliBrowserScenarioCommands.ts:39-58](https://github.com/morluto/rea/blob/4692bb33300c7cb77b095b6cc0c33da861c1b860/src/cliBrowserScenarioCommands.ts#L39), [src/composition/browserScenario.ts:9-15](https://github.com/morluto/rea/blob/4692bb33300c7cb77b095b6cc0c33da861c1b860/src/composition/browserScenario.ts#L9), [src/browser/PlaywrightScenarioSession.ts:243-251](https://github.com/morluto/rea/blob/4692bb33300c7cb77b095b6cc0c33da861c1b860/src/browser/PlaywrightScenarioSession.ts#L243), and [src/browser/PlaywrightScenarioEvents.ts:111-156](https://github.com/morluto/rea/blob/4692bb33300c7cb77b095b6cc0c33da861c1b860/src/browser/PlaywrightScenarioEvents.ts#L111). Selecting header values enables network collection even when the caller's explicit event-family list is empty. Launch ownership admits the fixture request; the default metadata, event, and transaction limits admitted this observation. The ordered Request header API is the production input boundary exercised here. Real-browser behavior and other credential-header names were not executed. ### Typed outcome or recovery expected Repeat the 4096-byte CLI case with the 2000-character Cookie. It must return the same captured header items as the empty-Cookie control, preserve keep-me, and keep the Cookie value null/redacted. The header capture must no longer make these otherwise complete fixture captures ineligible for equality. Keep the fitting empty-Cookie control and check that genuinely excessive ordinary retained header output still respects the selected allocation limit. Raising the limit is an observed workaround, not a verified code fix. ### Additional context Freshness checked against origin/main at 22ec8bdb2f7d4626587f23b1eed4a207df600386. The cited failing implementation files are unchanged from the reproduced baseline; this is a source freshness check, not a runtime rerun on that revision.

bsakaguchi commented

2026-10-16 21:07:14 +00:00

Author

Owner

@Sravan1011 commented on GitHub (Oct 15, 2026):

@coygeek I'll take this one

@Sravan1011 commented on GitHub (Oct 15, 2026): @coygeek I'll take this one

bsakaguchi referenced this issue

2026-10-17 21:12:27 +00:00

[PR #1566] [CLOSED] fix(reference): preserve cancellation at traversal completion #2975

bsakaguchi closed this issue

2026-10-31 00:31:29 +00:00

bsakaguchi commented

2026-10-31 00:31:30 +00:00

Author

Owner

@coygeek commented on GitHub (Oct 24, 2026):

Checked the reported fix on main at d87e73f2db.

The retained allocation cost now counts credential header names but zero discarded credential-value characters. A production-source probe repeats 2000-character Cookie at 4096 bytes and empty-Cookie control, obtaining identical captured Cookie(null,redacted) and X-Ordinary keep-me; a large ordinary retained value still refuses. Regression uses a 50000-character Cookie plus ordinary control.

Implementation: PlaywrightNetworkContent.ts.

The relevant regression checks in PlaywrightNetworkContent.test.ts passed in the focused test run on this revision. This closure is based on the current source and those checks; provider fixtures do not establish a real-engine run. Closing as completed on main.

@coygeek commented on GitHub (Oct 24, 2026): Checked the reported fix on `main` at d87e73f2db50be72d33a4b3c7199af35518b441c. The retained allocation cost now counts credential header names but zero discarded credential-value characters. A production-source probe repeats 2000-character Cookie at 4096 bytes and empty-Cookie control, obtaining identical captured Cookie(null,redacted) and X-Ordinary keep-me; a large ordinary retained value still refuses. Regression uses a 50000-character Cookie plus ordinary control. Implementation: [PlaywrightNetworkContent.ts](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/src/browser/network/PlaywrightNetworkContent.ts#L86). The relevant regression checks in [PlaywrightNetworkContent.test.ts](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/src/browser/network/PlaywrightNetworkContent.test.ts) passed in the focused test run on this revision. This closure is based on the current source and those checks; provider fixtures do not establish a real-engine run. Closing as completed on main.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#1566

No description provided.

Rows
Columns