morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #4685] [Bug] Runtime observation counts a malformed context event as retained without reporting its loss #2443

New issue

Closed

opened 2026-10-17 21:04:45 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-17 21:04:45 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 17, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/4685

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Process capture or runtime observation

Observed behavior

observe-javascript-runtime returns successful Evidence after receiving two Runtime.executionContextCreated events, one with numeric context ID 1 and one with string context ID "2". It reports events_observed: 2, events_retained: 2, events_dropped: 0, and truncated: false, but returns only context 1. Neither unknowns nor limitations identifies the malformed event or its omission.

This reproduced twice on each tested build. An all-valid control changes only "2" to numeric 2 and returns both contexts, twice on each build. A single-valid-event run returns the same context metadata and metadata byte count as the mixed run, once on each build.

Reproduction

Prepare a source checkout with its pinned Node/npm toolchain, locked dependencies (npm ci), and built runtime (npm run build:cached). For the published-package comparison, install rea-agents@6.5.0 into a separate empty directory using npm install --ignore-scripts --no-audit --no-fund rea-agents@6.5.0; use its node_modules/rea-agents directory as the package root. Both preparation routes were executed.
Save the following as repro.mjs in an external scratch directory. From either the source root or installed package root, run node /path/to/repro.mjs. The fixture loads that package's ws dependency, starts a loopback Inspector stand-in, invokes the public CLI five times with separate HOME directories, saves complete JSON results beside itself, and closes its sockets and server after each call. /path/to/repro.mjs represents the chosen fixture path.

import { createServer } from 'node:http';
import { createRequire } from 'node:module';
import { join, dirname } from 'node:path';
import { fileURLToPath } from 'node:url';
import { mkdir, writeFile } from 'node:fs/promises';
import { spawn } from 'node:child_process';
import assert from 'node:assert/strict';
const require = createRequire(join(process.cwd(), 'package.json'));
const { WebSocketServer } = require('ws');
const scratch = dirname(fileURLToPath(import.meta.url));
const reports = [];
for (const mode of ['mixed', 'control', 'single', 'mixed', 'control']) {
  const index = reports.length + 1;
  const home = join(scratch, `home-${index}`);
  await mkdir(home, {recursive:true});
  const events = [{method:'Runtime.executionContextCreated',params:{context:{id:1,name:'main',origin:''}}}];
  if (mode !== 'single') events.push({method:'Runtime.executionContextCreated',params:{context:{id:mode === 'mixed' ? '2' : 2,name:'worker',origin:''}}});
  const commands = [];
  const sent = [];
  let endpoint;
  const server = createServer((req,res) => {
    res.setHeader('Content-Type','application/json');
    res.end(JSON.stringify(req.url === '/json/version' ? {Browser:'node.js/v24.18.0','Protocol-Version':'1.3','V8-Version':'13.6'} : [{id:'target',type:'node',url:'https://example.test/main.js',attached:false,webSocketDebuggerUrl:endpoint.replace('http:','ws:')+'/target'}]));
  });
  const wss = new WebSocketServer({server});
  wss.on('connection', socket => socket.on('message', raw => {
    const command = JSON.parse(raw);
    commands.push(command);
    socket.send(JSON.stringify({id:command.id,result:{}}));
    if (command.method === 'Debugger.enable') setTimeout(() => {
      for (const event of events) {socket.send(JSON.stringify(event)); sent.push(event);}
    },30);
  }));
  await new Promise(resolve => server.listen(0,'127.0.0.1',resolve));
  endpoint = `http://127.0.0.1:${server.address().port}`;
  const env = { ...process.env, HOME:home, PATH:dirname(process.execPath)+':'+process.env.PATH, NO_COLOR:'1' };
  for (const key of ['CLAUDE_CONFIG_DIR','CODEX_HOME','COPILOT_HOME','OPENCODE_CONFIG','OPENCODE_CONFIG_DIR','XDG_CONFIG_HOME','XDG_DATA_HOME','XDG_CACHE_HOME','XDG_STATE_HOME','GEMINI_CLI_HOME','QWEN_HOME','GROK_HOME','HERMES_HOME','HERMES_DATA_DIR_SUFFIX','PI_CONFIG_DIR','PI_CODING_AGENT_DIR','OMP_PROFILE','PI_PROFILE','SAND_DATA_ROOT','KIMI_CODE_HOME','QODER_CONFIG_DIR','DSH_HOME','DSH_AGENTS_HOME']) delete env[key];
  const args=['scripts/rea.mjs','observe-javascript-runtime',endpoint,'target','--runtime-kind','node','--observation-ms','200','--json'];
  const child=spawn(process.execPath,args,{env});
  let stdout='', stderr='';
  child.stdout.on('data',data=>stdout+=data);
  child.stderr.on('data',data=>stderr+=data);
  const code=await new Promise(resolve=>child.on('close',resolve));
  for (const socket of wss.clients) socket.terminate();
  await new Promise(resolve=>wss.close(resolve));
  await new Promise(resolve=>server.close(resolve));
  const evidence=JSON.parse(stdout);
  const result=evidence.normalized_result;
  await writeFile(join(scratch,`run-${index}-${mode}.json`),JSON.stringify({args,code,stderr,sent,commands,evidence},null,2)+'\n');
  assert.equal(code,0);
  assert.equal(stderr,'');
  assert.deepEqual(commands.map(c=>c.method),['Runtime.enable','Debugger.enable']);
  assert.equal(sent.length,events.length);
  assert.equal(result.capture.events_observed,events.length);
  assert.equal(result.execution_contexts.length,mode==='control'?2:1);
  reports.push({mode,code,capture:result.capture,execution_contexts:result.execution_contexts,scripts:result.scripts,unknowns:result.unknowns,limitations:result.limitations});
  console.log(JSON.stringify(reports.at(-1)));
}
assert.equal(reports[0].capture.metadata_bytes_retained,reports[2].capture.metadata_bytes_retained);
await writeFile(join(scratch,'results.json'),JSON.stringify(reports,null,2)+'\n');
console.log('PASS: mixed reproduced twice; all-valid control passed twice; single-event baseline passed once.');

Compare the saved mixed, control, and single results. Both mixed runs exit zero and count two retained events while returning only context 1, with no malformed-context diagnostic. Both controls return contexts 1 and 2. The single-event run and mixed runs retain 522 metadata bytes; the controls retain 1048. The script's assertions confirm the observed behavior, not restoration of the contract.

The underlying public command is node scripts/rea.mjs observe-javascript-runtime ENDPOINT target --runtime-kind node --observation-ms 200 --json; the fixture supplies an ephemeral literal-loopback endpoint. Both enable commands receive successful replies before the events are sent during the observation window.

Expected behavior

The runtime-observation guide lists execution-context lifecycle events as direct observations and says capture retains valid context metadata. The tool-design contract requires coverage and limitations, says missing coverage is unknown rather than absence, and requires partial failures to be reported. Repository guidance also requires preserving omissions and meaningful failure reasons at unknown boundaries.

When an observed event cannot be represented because its context ID is malformed, retain the valid sibling observations and explicitly account for the invalid event, or return a typed protocol failure with partial observations. Counting the event as retained is reasonable only if an explicit record of its invalidity is also retained. The documented script-only invalid_protocol_value counter does not mandate a particular context-counter field, and an invalid string ID need not become a valid context.

Impact and scope

The successful result hides a known rejected protocol event and overstates retained observations for this bounded malformed-producer input. The all-valid workflow succeeds. No real Node/Electron producer emitting this input was established; deployment frequency, destroyed-context events, and downstream reconciliation effects were not tested. The fixture executes no target JavaScript and demonstrates no security boundary crossing.

Impact level: P2
Assessment status: supported
Impact basis: The public CLI consistently discards an observed malformed context event without a corresponding invalid-event record or diagnostic while counting it as retained. This misleads callers about the captured evidence; broader real-producer impact is unknown.
Workaround status: unverified
Workaround basis: No caller option was established that detects or recovers the discarded event. Making the stand-in emit valid IDs passes the control, but does not establish a practical recovery for an affected real producer.

REA version

Current main source commit 4d980d017622d3175d86aaf3b0dcfbdf4ca3d091, built from locked dependencies; independently installed published npm package rea-agents@6.5.0. Both public CLI entry points report 6.5.0. GitHub's latest published release and npm's latest version both identify that release.

Runtime and platform

Node v24.18.0 (repository pin), npm 11.16.0 (repository pin), Darwin arm64. No real Node or Electron application was attached.

Target and artifact context

A loopback HTTP/WebSocket stand-in advertises one unattached node target at https://example.test/main.js and a same-port WebSocket. Valid JSON/CDP envelopes carry the context events; the context ID type is the only difference between the mixed and two-event control inputs. No scripts, source reads, or external services are involved.

Interface

CLI

Earliest failure stage

Capture or observation

Relevant output or evidence

Expected source: docs/javascript-runtime-observation.md:103-112,135-146; docs/tool-design.md:51-68; AGENTS.md:19,31 in the tested source checkout. The release-tag runtime guide has the same lifecycle and retention text at lines 98-107 and 130-141; its tool-design contract has the same relevant text at lines 51-68.
Failure source: Public CLI mixed/control/single results below; source-derived explanation in src/inspector/V8InspectorProvider.ts:429-448,487-499,520-523,589-594.
Evidence provenance: observed
Local verification: reproduced
Reproduction completeness: complete

Both tested builds produced the following selected output. All ten calls exited zero with empty stderr. The mixed and control rows each represent two agreeing calls per build; the single row represents one call per build.

{"mode":"mixed","events_observed":2,"events_retained":2,"events_dropped":0,"metadata_bytes_retained":522,"truncated":false,"truncation_reasons":[],"context_keys":["1"]}
{"mode":"control","events_observed":2,"events_retained":2,"events_dropped":0,"metadata_bytes_retained":1048,"truncated":false,"truncation_reasons":[],"context_keys":["1","2"]}
{"mode":"single","events_observed":1,"events_retained":1,"events_dropped":0,"metadata_bytes_retained":522,"truncated":false,"truncation_reasons":[],"context_keys":["1"]}

The mixed result's execution contexts are [{"context_key":"1","state":"created","name":"main","origin":""}]. Its unknowns and limitations are identical to the controls: they describe scripts collected before attachment, bounded observation, unverified process role, passive authority, location/process identity, and lack of runtime graph edges. They contain no malformed-context diagnostic. The script-specific invalid-value counter is zero throughout because no script events are sent. The stand-in receives only Runtime.enable and Debugger.enable.

Typed outcome or recovery expected

With the same mixed input, preserve context 1 and distinguish the malformed event through explicit invalid-event accounting or a typed failure carrying the valid partial observations. If it remains counted as retained, preserve a record of its invalidity. Do not invent context 2 from a string ID. The all-valid control should continue succeeding with both contexts, and the single-event input should continue returning one retained context. Generic attach-window caveats do not identify this known rejection.

Analysis or suspected cause

Source-derived reasoning: ingestEvent counts the recognized context event before dispatch. contextKey rejects nonnumeric IDs; ingestContext then calls retainEvent(state, 0) and returns without adding a context or invalid-event record. The helper increments eventsRetained. The final projection returns the surviving context map and unchanged capture counters. The result schema in src/domain/javascript/javascriptRuntimeObservation.ts:152-173 has script-invalid accounting but no invalid-context ledger. These source checks explain the separately observed public CLI output.

Additional context

Related closed #2538 concerns fatal malformed message envelopes during observation; this fixture sends accepted envelopes with an invalid event-specific ID. Closed #3379 concerns malformed Electron DOM/resource/worker collection replies. Open #3910 concerns frames added during passive Electron observation. Closed #4376 concerns reconciliation of an already retained invalid-script counter; here the V8 producer never records invalid contexts. Open #4040 covers browser/Electron capture changes, including frame inventory, rather than this V8 context-event rejection. None covers this event-accounting behavior.

Originally created by @coygeek on GitHub (Oct 17, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/4685 ### Before submitting - [X] I searched the open and closed issues for a related report. - [X] I removed credentials, tokens, private paths, and sensitive target data. ### Area Process capture or runtime observation ### Observed behavior `observe-javascript-runtime` returns successful Evidence after receiving two `Runtime.executionContextCreated` events, one with numeric context ID `1` and one with string context ID `"2"`. It reports `events_observed: 2`, `events_retained: 2`, `events_dropped: 0`, and `truncated: false`, but returns only context `1`. Neither unknowns nor limitations identifies the malformed event or its omission. This reproduced twice on each tested build. An all-valid control changes only `"2"` to numeric `2` and returns both contexts, twice on each build. A single-valid-event run returns the same context metadata and metadata byte count as the mixed run, once on each build. ### Reproduction 1. Prepare a source checkout with its pinned Node/npm toolchain, locked dependencies (`npm ci`), and built runtime (`npm run build:cached`). For the published-package comparison, install `rea-agents@6.5.0` into a separate empty directory using `npm install --ignore-scripts --no-audit --no-fund rea-agents@6.5.0`; use its `node_modules/rea-agents` directory as the package root. Both preparation routes were executed. 2. Save the following as `repro.mjs` in an external scratch directory. From either the source root or installed package root, run `node /path/to/repro.mjs`. The fixture loads that package's `ws` dependency, starts a loopback Inspector stand-in, invokes the public CLI five times with separate HOME directories, saves complete JSON results beside itself, and closes its sockets and server after each call. `/path/to/repro.mjs` represents the chosen fixture path. ```js import { createServer } from 'node:http'; import { createRequire } from 'node:module'; import { join, dirname } from 'node:path'; import { fileURLToPath } from 'node:url'; import { mkdir, writeFile } from 'node:fs/promises'; import { spawn } from 'node:child_process'; import assert from 'node:assert/strict'; const require = createRequire(join(process.cwd(), 'package.json')); const { WebSocketServer } = require('ws'); const scratch = dirname(fileURLToPath(import.meta.url)); const reports = []; for (const mode of ['mixed', 'control', 'single', 'mixed', 'control']) { const index = reports.length + 1; const home = join(scratch, `home-${index}`); await mkdir(home, {recursive:true}); const events = [{method:'Runtime.executionContextCreated',params:{context:{id:1,name:'main',origin:''}}}]; if (mode !== 'single') events.push({method:'Runtime.executionContextCreated',params:{context:{id:mode === 'mixed' ? '2' : 2,name:'worker',origin:''}}}); const commands = []; const sent = []; let endpoint; const server = createServer((req,res) => { res.setHeader('Content-Type','application/json'); res.end(JSON.stringify(req.url === '/json/version' ? {Browser:'node.js/v24.18.0','Protocol-Version':'1.3','V8-Version':'13.6'} : [{id:'target',type:'node',url:'https://example.test/main.js',attached:false,webSocketDebuggerUrl:endpoint.replace('http:','ws:')+'/target'}])); }); const wss = new WebSocketServer({server}); wss.on('connection', socket => socket.on('message', raw => { const command = JSON.parse(raw); commands.push(command); socket.send(JSON.stringify({id:command.id,result:{}})); if (command.method === 'Debugger.enable') setTimeout(() => { for (const event of events) {socket.send(JSON.stringify(event)); sent.push(event);} },30); })); await new Promise(resolve => server.listen(0,'127.0.0.1',resolve)); endpoint = `http://127.0.0.1:${server.address().port}`; const env = { ...process.env, HOME:home, PATH:dirname(process.execPath)+':'+process.env.PATH, NO_COLOR:'1' }; for (const key of ['CLAUDE_CONFIG_DIR','CODEX_HOME','COPILOT_HOME','OPENCODE_CONFIG','OPENCODE_CONFIG_DIR','XDG_CONFIG_HOME','XDG_DATA_HOME','XDG_CACHE_HOME','XDG_STATE_HOME','GEMINI_CLI_HOME','QWEN_HOME','GROK_HOME','HERMES_HOME','HERMES_DATA_DIR_SUFFIX','PI_CONFIG_DIR','PI_CODING_AGENT_DIR','OMP_PROFILE','PI_PROFILE','SAND_DATA_ROOT','KIMI_CODE_HOME','QODER_CONFIG_DIR','DSH_HOME','DSH_AGENTS_HOME']) delete env[key]; const args=['scripts/rea.mjs','observe-javascript-runtime',endpoint,'target','--runtime-kind','node','--observation-ms','200','--json']; const child=spawn(process.execPath,args,{env}); let stdout='', stderr=''; child.stdout.on('data',data=>stdout+=data); child.stderr.on('data',data=>stderr+=data); const code=await new Promise(resolve=>child.on('close',resolve)); for (const socket of wss.clients) socket.terminate(); await new Promise(resolve=>wss.close(resolve)); await new Promise(resolve=>server.close(resolve)); const evidence=JSON.parse(stdout); const result=evidence.normalized_result; await writeFile(join(scratch,`run-${index}-${mode}.json`),JSON.stringify({args,code,stderr,sent,commands,evidence},null,2)+'\n'); assert.equal(code,0); assert.equal(stderr,''); assert.deepEqual(commands.map(c=>c.method),['Runtime.enable','Debugger.enable']); assert.equal(sent.length,events.length); assert.equal(result.capture.events_observed,events.length); assert.equal(result.execution_contexts.length,mode==='control'?2:1); reports.push({mode,code,capture:result.capture,execution_contexts:result.execution_contexts,scripts:result.scripts,unknowns:result.unknowns,limitations:result.limitations}); console.log(JSON.stringify(reports.at(-1))); } assert.equal(reports[0].capture.metadata_bytes_retained,reports[2].capture.metadata_bytes_retained); await writeFile(join(scratch,'results.json'),JSON.stringify(reports,null,2)+'\n'); console.log('PASS: mixed reproduced twice; all-valid control passed twice; single-event baseline passed once.'); ``` 3. Compare the saved mixed, control, and single results. Both mixed runs exit zero and count two retained events while returning only context `1`, with no malformed-context diagnostic. Both controls return contexts `1` and `2`. The single-event run and mixed runs retain 522 metadata bytes; the controls retain 1048. The script's assertions confirm the observed behavior, not restoration of the contract. The underlying public command is `node scripts/rea.mjs observe-javascript-runtime ENDPOINT target --runtime-kind node --observation-ms 200 --json`; the fixture supplies an ephemeral literal-loopback endpoint. Both enable commands receive successful replies before the events are sent during the observation window. ### Expected behavior The runtime-observation guide lists execution-context lifecycle events as direct observations and says capture retains valid context metadata. The tool-design contract requires coverage and limitations, says missing coverage is unknown rather than absence, and requires partial failures to be reported. Repository guidance also requires preserving omissions and meaningful failure reasons at unknown boundaries. When an observed event cannot be represented because its context ID is malformed, retain the valid sibling observations and explicitly account for the invalid event, or return a typed protocol failure with partial observations. Counting the event as retained is reasonable only if an explicit record of its invalidity is also retained. The documented script-only `invalid_protocol_value` counter does not mandate a particular context-counter field, and an invalid string ID need not become a valid context. ### Impact and scope The successful result hides a known rejected protocol event and overstates retained observations for this bounded malformed-producer input. The all-valid workflow succeeds. No real Node/Electron producer emitting this input was established; deployment frequency, destroyed-context events, and downstream reconciliation effects were not tested. The fixture executes no target JavaScript and demonstrates no security boundary crossing. - Impact level: P2 - Assessment status: supported - Impact basis: The public CLI consistently discards an observed malformed context event without a corresponding invalid-event record or diagnostic while counting it as retained. This misleads callers about the captured evidence; broader real-producer impact is unknown. - Workaround status: unverified - Workaround basis: No caller option was established that detects or recovers the discarded event. Making the stand-in emit valid IDs passes the control, but does not establish a practical recovery for an affected real producer. ### REA version Current main source commit `4d980d017622d3175d86aaf3b0dcfbdf4ca3d091`, built from locked dependencies; independently installed published npm package `rea-agents@6.5.0`. Both public CLI entry points report `6.5.0`. GitHub's latest published release and npm's latest version both identify that release. ### Runtime and platform Node v24.18.0 (repository pin), npm 11.16.0 (repository pin), Darwin arm64. No real Node or Electron application was attached. ### Target and artifact context A loopback HTTP/WebSocket stand-in advertises one unattached `node` target at `https://example.test/main.js` and a same-port WebSocket. Valid JSON/CDP envelopes carry the context events; the context ID type is the only difference between the mixed and two-event control inputs. No scripts, source reads, or external services are involved. ### Interface CLI ### Earliest failure stage Capture or observation ### Relevant output or evidence - Expected source: `docs/javascript-runtime-observation.md:103-112,135-146`; `docs/tool-design.md:51-68`; `AGENTS.md:19,31` in the tested source checkout. The release-tag runtime guide has the same lifecycle and retention text at lines 98-107 and 130-141; its tool-design contract has the same relevant text at lines 51-68. - Failure source: Public CLI mixed/control/single results below; source-derived explanation in `src/inspector/V8InspectorProvider.ts:429-448,487-499,520-523,589-594`. - Evidence provenance: observed - Local verification: reproduced - Reproduction completeness: complete Both tested builds produced the following selected output. All ten calls exited zero with empty stderr. The mixed and control rows each represent two agreeing calls per build; the single row represents one call per build. ```json {"mode":"mixed","events_observed":2,"events_retained":2,"events_dropped":0,"metadata_bytes_retained":522,"truncated":false,"truncation_reasons":[],"context_keys":["1"]} {"mode":"control","events_observed":2,"events_retained":2,"events_dropped":0,"metadata_bytes_retained":1048,"truncated":false,"truncation_reasons":[],"context_keys":["1","2"]} {"mode":"single","events_observed":1,"events_retained":1,"events_dropped":0,"metadata_bytes_retained":522,"truncated":false,"truncation_reasons":[],"context_keys":["1"]} ``` The mixed result's execution contexts are `[{"context_key":"1","state":"created","name":"main","origin":""}]`. Its unknowns and limitations are identical to the controls: they describe scripts collected before attachment, bounded observation, unverified process role, passive authority, location/process identity, and lack of runtime graph edges. They contain no malformed-context diagnostic. The script-specific invalid-value counter is zero throughout because no script events are sent. The stand-in receives only `Runtime.enable` and `Debugger.enable`. ### Typed outcome or recovery expected With the same mixed input, preserve context `1` and distinguish the malformed event through explicit invalid-event accounting or a typed failure carrying the valid partial observations. If it remains counted as retained, preserve a record of its invalidity. Do not invent context `2` from a string ID. The all-valid control should continue succeeding with both contexts, and the single-event input should continue returning one retained context. Generic attach-window caveats do not identify this known rejection. ### Analysis or suspected cause Source-derived reasoning: `ingestEvent` counts the recognized context event before dispatch. `contextKey` rejects nonnumeric IDs; `ingestContext` then calls `retainEvent(state, 0)` and returns without adding a context or invalid-event record. The helper increments `eventsRetained`. The final projection returns the surviving context map and unchanged capture counters. The result schema in `src/domain/javascript/javascriptRuntimeObservation.ts:152-173` has script-invalid accounting but no invalid-context ledger. These source checks explain the separately observed public CLI output. ### Additional context Related closed [#2538](https://github.com/morluto/rea/issues/2538) concerns fatal malformed message envelopes during observation; this fixture sends accepted envelopes with an invalid event-specific ID. Closed [#3379](https://github.com/morluto/rea/issues/3379) concerns malformed Electron DOM/resource/worker collection replies. Open [#3910](https://github.com/morluto/rea/issues/3910) concerns frames added during passive Electron observation. Closed [#4376](https://github.com/morluto/rea/issues/4376) concerns reconciliation of an already retained invalid-script counter; here the V8 producer never records invalid contexts. Open [#4040](https://github.com/morluto/rea/pull/4040) covers browser/Electron capture changes, including frame inventory, rather than this V8 context-event rejection. None covers this event-accounting behavior.

bsakaguchi closed this issue

2026-10-17 21:04:45 +00:00

bsakaguchi referenced this issue

2026-10-31 00:30:55 +00:00

[GH-ISSUE #3796] [Tracker] Open feature requests by area (October 2026) #1471

bsakaguchi referenced this issue

2026-10-31 00:50:14 +00:00

[GH-ISSUE #7551] [Bug] Linked-worktree imports hash Git administration files into source identities #4490

bsakaguchi referenced this issue

2026-10-31 01:05:30 +00:00

[PR #2630] feat(cli): expose caller-selected reference-source path exclusions #5293

bsakaguchi referenced this issue