morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #6474] [Bug] observe-web-execution counts discarded POST bodies against its retained metadata budget #3885

New issue

Closed

opened 2026-10-31 00:44:55 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-31 00:44:55 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 23, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6474

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Process capture or runtime observation

Observed behavior

During an armed observe-web-execution window, a same-origin 9 MiB POST makes the entire CLI call exit 1 with unreadable_output and “Request initiator evidence exceeds its complete 8 MiB event budget.” The observed initiator was only 394 serialized bytes. A 1 MiB POST succeeded and returned its request metadata with captured coverage. Neither body nor headers appear in the retained request record.

Reproduction

From a fresh checkout at the baseline, install/build the locked source. Set REA_BROWSER_EXECUTABLE to an existing Chrome or chrome-headless-shell executable. This driver uses the checkout's locked playwright-core, serves a synthetic page on loopback, starts a disposable headless browser, discovers the page through the public CLI, and triggers each POST after actual CLI arming. It closes its browser and server in finally.

git checkout 9eb829aca34be36ff112826337996f6139f82eff
npm ci && npm run build:cached
export WORK="$(mktemp -d)"
mkdir -p "$WORK/home"
# Export REA_BROWSER_EXECUTABLE pointing to an existing browser executable.
cat > "$WORK/repro.mjs" <<'JS'
import {createServer} from 'node:http';
import {createServer as netServer} from 'node:net';
import {spawn,execFileSync} from 'node:child_process';
import {once} from 'node:events';
import {writeFileSync,readFileSync} from 'node:fs';
import {createRequire} from 'node:module';
const root=process.cwd(), work=process.env.WORK;
const require=createRequire(root+'/package.json');
const {chromium}=require('playwright-core');
writeFileSync(work+'/page.html', '<!doctype html><title>Request body fixture</title><script>async function upload(n){return fetch("/upload",{method:"POST",body:"x".repeat(n)}).then(r=>r.text())}</script>');
const server=createServer((req,res)=>{if(req.url==='/upload'){req.resume();req.on('end',()=>res.end('ok'));}else{res.setHeader('content-type','text/html');res.end(readFileSync(work+'/page.html'));}});
await new Promise(r=>server.listen(0,'127.0.0.1',r));
const origin='http://127.0.0.1:'+server.address().port;
const portServer=netServer();await new Promise(r=>portServer.listen(0,'127.0.0.1',r));const port=portServer.address().port;await new Promise(r=>portServer.close(r));
let context;
try{
context=await chromium.launchPersistentContext(work+'/profile',{headless:true,executablePath:process.env.REA_BROWSER_EXECUTABLE,args:['--remote-debugging-port='+port]});
const page=context.pages()[0];await page.goto(origin);
const endpoint='http://127.0.0.1:'+port;
console.log('BROWSER',await context.browser().version());
const env={...process.env,HOME:work+'/home',REA_LOG_LEVEL:'silent'};
const targetsRaw=execFileSync(process.execPath,['scripts/rea.mjs','list-browser-targets',endpoint,'--json'],{env,encoding:'utf8',timeout:20000});
writeFileSync(work+'/targets.json',targetsRaw);
const discovery=await (await fetch(endpoint+'/json/list')).json();
const target=discovery.find(x=>x.type==='page'&&x.url.startsWith(origin));
const session=await context.newCDPSession(page);await session.send('Network.enable');
let eventEvidence=[];
session.on('Network.requestWillBeSent',p=>{if(p.request.url.endsWith('/upload')) eventEvidence.push({method:p.request.method,bodyBytes:p.request.postData?.length??0,eventBytes:Buffer.byteLength(JSON.stringify(p)),initiatorBytes:Buffer.byteLength(JSON.stringify(p.initiator))});});
for(const [label,size] of [['control',1048576],['large',9437184],['large-repeat',9437184]]){
  const child=spawn(process.execPath,['scripts/rea.mjs','observe-web-execution',endpoint,target.id,'--observation-ms','1500','--json'],{env,stdio:['ignore','pipe','pipe']});
  let out='',err='',armed=false,action;
  const timer=setTimeout(()=>child.kill('SIGKILL'),20000);
  child.stdout.on('data',b=>out+=b);child.stderr.on('data',b=>{err+=b;if(!armed&&err.includes('"completed":1')&&err.includes('browser_execution')){armed=true;action=page.evaluate(n=>upload(n),size);action.catch(()=>{});}});
  const [code,signal]=await once(child,'close');clearTimeout(timer);await action;
  writeFileSync(work+'/'+label+'.stdout.json',out);writeFileSync(work+'/'+label+'.stderr.txt',err);
  const doc=JSON.parse(out),r=doc.normalized_result??doc;
  console.log(JSON.stringify({label,size,armed,code,signal,requests:r.requests?.map(x=>({method:x.method,keys:Object.keys(x)})),end_reason:r.window?.end_reason,coverage:r.coverage?.state,codeName:r.code,category:r.category,details:r.details,event:eventEvidence.at(-1)}));
}
await session.detach();
} finally {await context?.close();await new Promise(r=>server.close(r));}
JS
HOME="$WORK/home" node "$WORK/repro.mjs"
# Inspect the JSON/stdout/stderr files, then remove the disposable fixture:
rm -r "$WORK"

The driver was run twice. Both 9 MiB trials failed on each run (4/4 overall); both 1 MiB controls succeeded (2/2 overall). No provider substitution or source test was used.

Expected behavior

Return the admitted POST's request metadata and the completed execution observation when retained metadata fits its budget and the protocol message fits its separate transport budget.

The runtime guide distinguishes a 64 MiB protocol-message allowance from an 8 MiB retained request-metadata allowance. The request protocol contract explicitly excludes bodies and headers. Discarded body fields should not consume the retained-metadata budget. Genuinely oversized retained evidence should still fail without partial success.

Impact and scope

P2: an otherwise fitting observation is lost when its armed window contains the tested large upload. Both large cases returned only an error, including no completed coverage result. No corruption or security-boundary crossing was observed. No workaround preserving this same observation was demonstrated. Natural frequency, other browsers and other Node versions were not measured.

REA version

6.7.0 (source 9eb829aca3)

Runtime and platform

Node.js v26.12.1; macOS 27.0.1; arm64; HeadlessChrome 145.0.7632.6; locked playwright-core 1.58.2. The repository's Node 24.18.0 pin was not exercised.

Target and artifact context

Synthetic HTTP page served only on 127.0.0.1, an externally owned disposable headless page target, and same-origin POST bodies consisting of repeated ASCII x. No external service or sensitive target data.

Interface

CLI

Earliest failure stage

Capture or observation

Relevant output or evidence

The driver invoked node scripts/rea.mjs observe-web-execution ENDPOINT TARGET_ID --observation-ms 1500 --json for each trial. Reduced observed results:

{"body_bytes":1048576,"exit":0,"retained_requests":1,"method":"POST","end_reason":"window_elapsed","coverage":"captured"}
{"body_bytes":9437184,"exit":1,"code":"unreadable_output","category":"execution_failure","details":{"operation":"observe_web_execution","reason":"Request initiator evidence exceeds its complete 8 MiB event budget."}}

The second 9 MiB trial returned the same failure. Independent native CDP monitoring of these same requests measured 394-byte initiators in all cases. Its complete serialized event parameters measured 2,447,998 bytes for the small request and 22,021,414 bytes for each large request, below the 64 MiB transport limit. Those measurements came from a separate monitoring connection. The full producer representation is larger than the upload, so this does not establish an exact upload-size threshold.

Typed outcome or recovery expected

Success with the POST metadata, retained initiator/callsites and completed coverage, subject to the existing scope and genuine evidence limits. Changing only the error category would leave this incorrect rejection in place.

Analysis or suspected cause

Source-derived: CdpRuntimeRequests.ts:30-44 validates and scope-filters the request, then adds Buffer.byteLength(JSON.stringify(event.params)) to the retained request budget. This counts the original body fields even though the retained request construction omits them. The active-window collector propagates the stored failure and ends the observation.

Additional context

Related #5954 concerns the classification/recovery of genuine retention-budget refusals, including oversized retained initiators, and explicitly preserves their rejection. This report concerns small retained initiators rejected because discarded request bodies consume that budget. NOT_ENOUGH_INFO: the first affected release and a known-good version are unknown. No fix was applied; restoration and genuine retained-overflow controls remain proposed checks.

Originally created by @coygeek on GitHub (Oct 23, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6474 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Process capture or runtime observation ### Observed behavior During an armed `observe-web-execution` window, a same-origin 9 MiB POST makes the entire CLI call exit 1 with `unreadable_output` and “Request initiator evidence exceeds its complete 8 MiB event budget.” The observed initiator was only 394 serialized bytes. A 1 MiB POST succeeded and returned its request metadata with captured coverage. Neither body nor headers appear in the retained request record. ### Reproduction From a fresh checkout at the baseline, install/build the locked source. Set `REA_BROWSER_EXECUTABLE` to an existing Chrome or chrome-headless-shell executable. This driver uses the checkout's locked `playwright-core`, serves a synthetic page on loopback, starts a disposable headless browser, discovers the page through the public CLI, and triggers each POST after actual CLI arming. It closes its browser and server in `finally`. ```sh git checkout 9eb829aca34be36ff112826337996f6139f82eff npm ci && npm run build:cached export WORK="$(mktemp -d)" mkdir -p "$WORK/home" # Export REA_BROWSER_EXECUTABLE pointing to an existing browser executable. cat > "$WORK/repro.mjs" <<'JS' import {createServer} from 'node:http'; import {createServer as netServer} from 'node:net'; import {spawn,execFileSync} from 'node:child_process'; import {once} from 'node:events'; import {writeFileSync,readFileSync} from 'node:fs'; import {createRequire} from 'node:module'; const root=process.cwd(), work=process.env.WORK; const require=createRequire(root+'/package.json'); const {chromium}=require('playwright-core'); writeFileSync(work+'/page.html', '<!doctype html><title>Request body fixture</title><script>async function upload(n){return fetch("/upload",{method:"POST",body:"x".repeat(n)}).then(r=>r.text())}</script>'); const server=createServer((req,res)=>{if(req.url==='/upload'){req.resume();req.on('end',()=>res.end('ok'));}else{res.setHeader('content-type','text/html');res.end(readFileSync(work+'/page.html'));}}); await new Promise(r=>server.listen(0,'127.0.0.1',r)); const origin='http://127.0.0.1:'+server.address().port; const portServer=netServer();await new Promise(r=>portServer.listen(0,'127.0.0.1',r));const port=portServer.address().port;await new Promise(r=>portServer.close(r)); let context; try{ context=await chromium.launchPersistentContext(work+'/profile',{headless:true,executablePath:process.env.REA_BROWSER_EXECUTABLE,args:['--remote-debugging-port='+port]}); const page=context.pages()[0];await page.goto(origin); const endpoint='http://127.0.0.1:'+port; console.log('BROWSER',await context.browser().version()); const env={...process.env,HOME:work+'/home',REA_LOG_LEVEL:'silent'}; const targetsRaw=execFileSync(process.execPath,['scripts/rea.mjs','list-browser-targets',endpoint,'--json'],{env,encoding:'utf8',timeout:20000}); writeFileSync(work+'/targets.json',targetsRaw); const discovery=await (await fetch(endpoint+'/json/list')).json(); const target=discovery.find(x=>x.type==='page'&&x.url.startsWith(origin)); const session=await context.newCDPSession(page);await session.send('Network.enable'); let eventEvidence=[]; session.on('Network.requestWillBeSent',p=>{if(p.request.url.endsWith('/upload')) eventEvidence.push({method:p.request.method,bodyBytes:p.request.postData?.length??0,eventBytes:Buffer.byteLength(JSON.stringify(p)),initiatorBytes:Buffer.byteLength(JSON.stringify(p.initiator))});}); for(const [label,size] of [['control',1048576],['large',9437184],['large-repeat',9437184]]){ const child=spawn(process.execPath,['scripts/rea.mjs','observe-web-execution',endpoint,target.id,'--observation-ms','1500','--json'],{env,stdio:['ignore','pipe','pipe']}); let out='',err='',armed=false,action; const timer=setTimeout(()=>child.kill('SIGKILL'),20000); child.stdout.on('data',b=>out+=b);child.stderr.on('data',b=>{err+=b;if(!armed&&err.includes('"completed":1')&&err.includes('browser_execution')){armed=true;action=page.evaluate(n=>upload(n),size);action.catch(()=>{});}}); const [code,signal]=await once(child,'close');clearTimeout(timer);await action; writeFileSync(work+'/'+label+'.stdout.json',out);writeFileSync(work+'/'+label+'.stderr.txt',err); const doc=JSON.parse(out),r=doc.normalized_result??doc; console.log(JSON.stringify({label,size,armed,code,signal,requests:r.requests?.map(x=>({method:x.method,keys:Object.keys(x)})),end_reason:r.window?.end_reason,coverage:r.coverage?.state,codeName:r.code,category:r.category,details:r.details,event:eventEvidence.at(-1)})); } await session.detach(); } finally {await context?.close();await new Promise(r=>server.close(r));} JS HOME="$WORK/home" node "$WORK/repro.mjs" # Inspect the JSON/stdout/stderr files, then remove the disposable fixture: rm -r "$WORK" ``` The driver was run twice. Both 9 MiB trials failed on each run (4/4 overall); both 1 MiB controls succeeded (2/2 overall). No provider substitution or source test was used. ### Expected behavior Return the admitted POST's request metadata and the completed execution observation when retained metadata fits its budget and the protocol message fits its separate transport budget. The [runtime guide](https://github.com/morluto/rea/blob/9eb829aca34be36ff112826337996f6139f82eff/docs/web-runtime.md#L55) distinguishes a 64 MiB protocol-message allowance from an 8 MiB retained request-metadata allowance. The [request protocol contract](https://github.com/morluto/rea/blob/9eb829aca34be36ff112826337996f6139f82eff/src/browser/execution/CdpRuntimeProtocol.ts#L76) explicitly excludes bodies and headers. Discarded body fields should not consume the retained-metadata budget. Genuinely oversized retained evidence should still fail without partial success. ### Impact and scope P2: an otherwise fitting observation is lost when its armed window contains the tested large upload. Both large cases returned only an error, including no completed coverage result. No corruption or security-boundary crossing was observed. No workaround preserving this same observation was demonstrated. Natural frequency, other browsers and other Node versions were not measured. ### REA version 6.7.0 (source 9eb829aca34be36ff112826337996f6139f82eff) ### Runtime and platform Node.js v26.12.1; macOS 27.0.1; arm64; HeadlessChrome 145.0.7632.6; locked playwright-core 1.58.2. The repository's Node 24.18.0 pin was not exercised. ### Target and artifact context Synthetic HTTP page served only on 127.0.0.1, an externally owned disposable headless page target, and same-origin POST bodies consisting of repeated ASCII `x`. No external service or sensitive target data. ### Interface CLI ### Earliest failure stage Capture or observation ### Relevant output or evidence The driver invoked `node scripts/rea.mjs observe-web-execution ENDPOINT TARGET_ID --observation-ms 1500 --json` for each trial. Reduced observed results: ```json {"body_bytes":1048576,"exit":0,"retained_requests":1,"method":"POST","end_reason":"window_elapsed","coverage":"captured"} {"body_bytes":9437184,"exit":1,"code":"unreadable_output","category":"execution_failure","details":{"operation":"observe_web_execution","reason":"Request initiator evidence exceeds its complete 8 MiB event budget."}} ``` The second 9 MiB trial returned the same failure. Independent native CDP monitoring of these same requests measured 394-byte initiators in all cases. Its complete serialized event parameters measured 2,447,998 bytes for the small request and 22,021,414 bytes for each large request, below the 64 MiB transport limit. Those measurements came from a separate monitoring connection. The full producer representation is larger than the upload, so this does not establish an exact upload-size threshold. ### Typed outcome or recovery expected Success with the POST metadata, retained initiator/callsites and completed coverage, subject to the existing scope and genuine evidence limits. Changing only the error category would leave this incorrect rejection in place. ### Analysis or suspected cause Source-derived: [CdpRuntimeRequests.ts:30-44](https://github.com/morluto/rea/blob/9eb829aca34be36ff112826337996f6139f82eff/src/browser/execution/CdpRuntimeRequests.ts#L30) validates and scope-filters the request, then adds `Buffer.byteLength(JSON.stringify(event.params))` to the retained request budget. This counts the original body fields even though the [retained request construction](https://github.com/morluto/rea/blob/9eb829aca34be36ff112826337996f6139f82eff/src/browser/execution/CdpRuntimeRequests.ts#L79) omits them. The [active-window collector](https://github.com/morluto/rea/blob/9eb829aca34be36ff112826337996f6139f82eff/src/browser/execution/CdpExecutionCollection.ts#L31) propagates the stored failure and ends the observation. ### Additional context Related [#5954](https://github.com/morluto/rea/issues/5954) concerns the classification/recovery of genuine retention-budget refusals, including oversized retained initiators, and explicitly preserves their rejection. This report concerns small retained initiators rejected because discarded request bodies consume that budget. NOT_ENOUGH_INFO: the first affected release and a known-good version are unknown. No fix was applied; restoration and genuine retained-overflow controls remain proposed checks.

bsakaguchi closed this issue

2026-10-31 00:44:55 +00:00

bsakaguchi commented

2026-10-31 00:44:56 +00:00

Author

Owner

@mvanhorn commented on GitHub (Oct 23, 2026):

Implemented in https://github.com/morluto/rea/pull/6761.

@mvanhorn commented on GitHub (Oct 23, 2026): Implemented in https://github.com/morluto/rea/pull/6761.

bsakaguchi referenced this issue

2026-10-31 01:12:12 +00:00

[PR #4259] [CLOSED] fix(javascript): preserve optional native-addon bindings #6144