morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #3957] [Bug] Omitted failed or unfinished responses undercount selected response-content events #1565

New issue

Closed

opened 2026-10-16 21:07:13 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-16 21:07:13 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 15, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3957

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Process capture or runtime observation

Observed behavior

A capture-browser-scenario caller who selects response-body capture and reaches capture.limits.max_network_transactions receives an incomplete omission inventory when an omitted transaction observes a response and then fails or remains unfinished at capture cutoff. events.omissions.by_kind has no network-content row for that response. The same transaction contributes a response-content omission when it finishes normally, and retains a response-content event with an explicitly unavailable body when admitted under a higher transaction limit. The omitted transaction range and aggregate incomplete/equality-ineligible flags remain present, so the demonstrated defect concerns event accounting rather than an observed false equality claim.

Reproduction

Use the baseline and installed dependencies described above. From the repository root, create the following reproduction fixture. It supplies a fake browser at the browser-opening boundary and calls the registered capture-browser-scenario CLI handler; the production parser, application service, provider, session, network collection and Evidence projection run unchanged. The fixture prints six calls for the three terminal states at transaction limits of one and two. Its assertions describe the observed baseline behavior, including the defect.

cat > src/browser/omittedResponseVerification.test.ts <<'TEST'
import { EventEmitter } from "node:events";
import { Cli } from "incur";
import { expect, it, vi } from "vitest";
import { registerBrowserScenarioCommands } from "../cliBrowserScenarioCommands.js";
import { silentLogger } from "../logger.js";

const state = vi.hoisted(() => ({ opened: undefined as any }));
vi.mock("./PlaywrightScenarioBrowser.js", async (importOriginal) => ({
  ...await importOriginal<typeof import("./PlaywrightScenarioBrowser.js")>(),
  openPlaywrightScenarioBrowser: async () => state.opened,
}));

it("compares omission inventory through the public CLI handler", async () => {
  const results: any[] = [];
  for (const terminal of ["finished", "failed", "unfinished"] as const) {
    for (const limit of [1, 2]) {
      let closed = false;
      const context = Object.assign(new EventEmitter(), {
        setDefaultTimeout() {}, setDefaultNavigationTimeout() {}, addCookies: async () => {},
      });
      const request = (id: number) => ({
        method: () => "GET", url: () => `https://app.example.test/${id}`,
        resourceType: () => "fetch", headers: () => ({}),
        headersArray: async () => [], postDataBuffer: () => null,
        redirectedFrom: () => null,
        failure: () => ({ errorText: "connection reset" }),
      });
      const first = request(1), second = request(2);
      const response = {
        request: () => second, url: second.url, status: () => 200,
        headers: () => ({}), headersArray: async () => [],
        body: async () => Buffer.from("ok"),
      };
      const page = Object.assign(new EventEmitter(), {
        url: () => "https://app.example.test/",
        context: () => context, waitForTimeout: async () => {},
        goto: async () => {
          context.emit("request", first);
          context.emit("requestfinished", first);
          context.emit("request", second);
          context.emit("response", response);
          if (terminal !== "unfinished") context.emit(`request${terminal}`, second);
        },
      });
      state.opened = {
        page, context, browser: { version: () => "fixture" },
        cleanup: {
          get browserClosed() { return closed; },
          async close(finish?: () => Promise<void>) { await finish?.(); closed = true; },
        },
      };
      const scenario = {
        browser: { mode: "launch", executable_path: process.execPath },
        start_url: { url: "https://app.example.test/" },
        actions: [{ step_id: "wait", action: "wait_for_timeout", duration_ms: 1 }],
        storage: {},
        capture: {
          after_each_step: [], at_end: [], events: ["network"],
          network: { request_body: false, response_body: true, header_values: false },
          limits: { max_network_transactions: limit },
        },
      };
      const cli = Cli.create("repro", { sync: false });
      registerBrowserScenarioCommands(cli, silentLogger, {});
      let stdout = "", exitCode = 0;
      await cli.serve(["capture-browser-scenario", JSON.stringify(scenario), "--json"], {
        env: {}, stdout: (text) => { stdout += text; }, exit: (code) => { exitCode = code; },
      });
      const output = JSON.parse(stdout);
      expect(exitCode).toBe(0);
      expect(output.normalized_result).toBeDefined();
      const events = output.normalized_result.events;
      const content = events.items.filter((item: any) => item.kind === "network-content");
      results.push({ terminal, limit, dropped: events.dropped,
        completeness: output.normalized_result.completeness, omissions: events.omissions, content: content.map((item: any) => ({ phase: item.phase, body: item.body })) });
    }
  }
  console.log(JSON.stringify(results, null, 2));
  expect(results.find(r => r.terminal === "finished" && r.limit === 1).omissions.by_kind)
    .toContainEqual({ kind: "network-content", count: 1, reasons: ["max-network-transactions"] });
  for (const terminal of ["failed", "unfinished"]) {
    expect(results.find(r => r.terminal === terminal && r.limit === 1).omissions.by_kind
      .find((row: any) => row.kind === "network-content")).toBeUndefined();
    expect(results.find(r => r.terminal === terminal && r.limit === 2).content)
      .toHaveLength(1);
  }
});
TEST
npm run test:local -- src/browser/omittedResponseVerification.test.ts
rm src/browser/omittedResponseVerification.test.ts

Inspect the printed rows for limit: 1. The second request is beyond the selected transaction limit and observes a 200 response. The finished variant includes one omitted network-content event. The failed and unfinished variants omit that kind entirely.
Compare the limit: 2 controls. Each retains one response-phase network-content event. The failed body is unavailable with reason request-failed; the unfinished body is unavailable with reason response-unfinished. These controls show that the selected content event exists even when its body cannot be read.

Expected behavior

Omitted selected events must be counted by kind and reason. An observed response whose selected response-content event cannot be retained because its transaction exceeds the limit must contribute one network-content omission for that response phase, including when the request later fails or remains unfinished. An unavailable body is still represented by a response-content event when the transaction is admitted. The count must not require reading the body of a failed or unfinished response.

Impact and scope

Callers receive an incorrect inventory of omitted selected evidence and an understated dropped-event count when a response belongs to a transaction beyond the selected capacity. The demonstrated scope is bounded metadata accounting; the capture still reports incomplete coverage and disallows equality, and no false comparison result was observed.

Workaround: In the executed two-request workload, raising max_network_transactions from one to two retains the selected response-content event for the failed and unfinished cases with its explicit unavailable body state. A workload that fits the selected transaction budget avoids this omission-accounting defect while preserving the requested evidence.

Assessment: P2.

REA version

rea 6.5.0 at source commit 4692bb3330

Runtime and platform

Node 24.18.0; macOS arm64

Target and artifact context

Simulated browser at the browser-opening boundary; the registered public CLI handler, parser, service, provider, session, network collection and Evidence projection run. No Chromium process or network request was used.

Interface

CLI

Earliest failure stage

Capture or observation

Relevant output or evidence

The focused test passed once after completing the simulated context's required methods, and the six CLI-handler calls returned Evidence with exit callback code zero. With only response bodies selected, the limit-one observations were:

Second transaction state	Omitted kinds and counts	Total dropped
Finished	request: 1, response: 1, request-finished: 1, network-content: 1	4
Failed	request: 1, response: 1, request-failed: 1	3
Unfinished	request: 1, response: 1	2

Each row identifies request-2 in network_transaction_range. Every limit-one result has completeness.status: "incomplete", equality_eligible: false and missing_sections: ["events"]. The limit-two controls retain the selected response-content event in all three cases and report no budget omissions. Therefore response-body selection alone reproduces the missing omission count; request-body and header-value selection are unnecessary.

Source-derived diagnosis: the omitted response path remembers that a response occurred, while only the finished terminal path accounts for its selected response-content omission. The omitted failed path returns after recording the failed event, and cutoff finalization visits admitted transactions only. This diagnosis is separate from the executed CLI-handler observations.

Expected source: docs/browser-scenario-contract.md:182-193 promises omitted-event counts by kind and reason; docs/browser-scenario-contract.md:318-322 specifies unavailable response bodies at cutoff; src/contracts/browserScenarioToolContracts.ts:25-35 advertises independent content selection and explicit unavailable states.
Failure source: the executed simulated-browser CLI fixture above produces the omission matrix. The producing branches are src/browser/network/PlaywrightScenarioNetwork.ts:209-212, src/browser/network/PlaywrightScenarioNetwork.ts:245-254, src/browser/network/PlaywrightScenarioNetwork.ts:277-283 and src/browser/network/PlaywrightScenarioNetwork.ts:316-360.

The fixture reproduces the public CLI handler with a simulated provider boundary, not real-browser behavior. Supported-entry-point reachability also follows from src/cliBrowserScenarioCommands.ts:39-60, src/browser/PlaywrightScenarioSession.ts:242-252 and src/browser/PlaywrightScenarioEvents.ts:125-163. Launch mode admits these context requests through its ownership guard, and the CLI schema accepts the positive transaction limit and independent response-body selection.

Typed outcome or recovery expected

With the same fixture and transaction limit of one, each second request that observes a response must contribute exactly one omitted response-phase network-content event regardless of whether it finishes, fails, or remains unfinished at cutoff. The total dropped count must include that omission. Keep the finished control at one response-content omission, without double counting. With the limit raised to two, keep the selected response-content event and the failed/unfinished body's explicit unavailable reason. Preserve the incomplete/equality-ineligible status of budget-limited captures and avoid body reads for failed or unfinished responses. No fix or restoration run has been performed.

Additional context

Freshness checked against origin/main at 22ec8bdb2f. The cited failing implementation files are unchanged from the reproduced baseline; this is a source freshness check, not a runtime rerun on that revision.

Originally created by @coygeek on GitHub (Oct 15, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3957 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Process capture or runtime observation ### Observed behavior A `capture-browser-scenario` caller who selects response-body capture and reaches `capture.limits.max_network_transactions` receives an incomplete omission inventory when an omitted transaction observes a response and then fails or remains unfinished at capture cutoff. `events.omissions.by_kind` has no `network-content` row for that response. The same transaction contributes a response-content omission when it finishes normally, and retains a response-content event with an explicitly unavailable body when admitted under a higher transaction limit. The omitted transaction range and aggregate incomplete/equality-ineligible flags remain present, so the demonstrated defect concerns event accounting rather than an observed false equality claim. ### Reproduction 1. Use the baseline and installed dependencies described above. From the repository root, create the following reproduction fixture. It supplies a fake browser at the browser-opening boundary and calls the registered `capture-browser-scenario` CLI handler; the production parser, application service, provider, session, network collection and Evidence projection run unchanged. The fixture prints six calls for the three terminal states at transaction limits of one and two. Its assertions describe the observed baseline behavior, including the defect. ```sh cat > src/browser/omittedResponseVerification.test.ts <<'TEST' import { EventEmitter } from "node:events"; import { Cli } from "incur"; import { expect, it, vi } from "vitest"; import { registerBrowserScenarioCommands } from "../cliBrowserScenarioCommands.js"; import { silentLogger } from "../logger.js"; const state = vi.hoisted(() => ({ opened: undefined as any })); vi.mock("./PlaywrightScenarioBrowser.js", async (importOriginal) => ({ ...await importOriginal<typeof import("./PlaywrightScenarioBrowser.js")>(), openPlaywrightScenarioBrowser: async () => state.opened, })); it("compares omission inventory through the public CLI handler", async () => { const results: any[] = []; for (const terminal of ["finished", "failed", "unfinished"] as const) { for (const limit of [1, 2]) { let closed = false; const context = Object.assign(new EventEmitter(), { setDefaultTimeout() {}, setDefaultNavigationTimeout() {}, addCookies: async () => {}, }); const request = (id: number) => ({ method: () => "GET", url: () => `https://app.example.test/${id}`, resourceType: () => "fetch", headers: () => ({}), headersArray: async () => [], postDataBuffer: () => null, redirectedFrom: () => null, failure: () => ({ errorText: "connection reset" }), }); const first = request(1), second = request(2); const response = { request: () => second, url: second.url, status: () => 200, headers: () => ({}), headersArray: async () => [], body: async () => Buffer.from("ok"), }; const page = Object.assign(new EventEmitter(), { url: () => "https://app.example.test/", context: () => context, waitForTimeout: async () => {}, goto: async () => { context.emit("request", first); context.emit("requestfinished", first); context.emit("request", second); context.emit("response", response); if (terminal !== "unfinished") context.emit(`request${terminal}`, second); }, }); state.opened = { page, context, browser: { version: () => "fixture" }, cleanup: { get browserClosed() { return closed; }, async close(finish?: () => Promise<void>) { await finish?.(); closed = true; }, }, }; const scenario = { browser: { mode: "launch", executable_path: process.execPath }, start_url: { url: "https://app.example.test/" }, actions: [{ step_id: "wait", action: "wait_for_timeout", duration_ms: 1 }], storage: {}, capture: { after_each_step: [], at_end: [], events: ["network"], network: { request_body: false, response_body: true, header_values: false }, limits: { max_network_transactions: limit }, }, }; const cli = Cli.create("repro", { sync: false }); registerBrowserScenarioCommands(cli, silentLogger, {}); let stdout = "", exitCode = 0; await cli.serve(["capture-browser-scenario", JSON.stringify(scenario), "--json"], { env: {}, stdout: (text) => { stdout += text; }, exit: (code) => { exitCode = code; }, }); const output = JSON.parse(stdout); expect(exitCode).toBe(0); expect(output.normalized_result).toBeDefined(); const events = output.normalized_result.events; const content = events.items.filter((item: any) => item.kind === "network-content"); results.push({ terminal, limit, dropped: events.dropped, completeness: output.normalized_result.completeness, omissions: events.omissions, content: content.map((item: any) => ({ phase: item.phase, body: item.body })) }); } } console.log(JSON.stringify(results, null, 2)); expect(results.find(r => r.terminal === "finished" && r.limit === 1).omissions.by_kind) .toContainEqual({ kind: "network-content", count: 1, reasons: ["max-network-transactions"] }); for (const terminal of ["failed", "unfinished"]) { expect(results.find(r => r.terminal === terminal && r.limit === 1).omissions.by_kind .find((row: any) => row.kind === "network-content")).toBeUndefined(); expect(results.find(r => r.terminal === terminal && r.limit === 2).content) .toHaveLength(1); } }); TEST npm run test:local -- src/browser/omittedResponseVerification.test.ts rm src/browser/omittedResponseVerification.test.ts ``` 2. Inspect the printed rows for `limit: 1`. The second request is beyond the selected transaction limit and observes a 200 response. The finished variant includes one omitted `network-content` event. The failed and unfinished variants omit that kind entirely. 3. Compare the `limit: 2` controls. Each retains one response-phase `network-content` event. The failed body is unavailable with reason `request-failed`; the unfinished body is unavailable with reason `response-unfinished`. These controls show that the selected content event exists even when its body cannot be read. ### Expected behavior Omitted selected events must be counted by kind and reason. An observed response whose selected response-content event cannot be retained because its transaction exceeds the limit must contribute one `network-content` omission for that response phase, including when the request later fails or remains unfinished. An unavailable body is still represented by a response-content event when the transaction is admitted. The count must not require reading the body of a failed or unfinished response. ### Impact and scope Callers receive an incorrect inventory of omitted selected evidence and an understated dropped-event count when a response belongs to a transaction beyond the selected capacity. The demonstrated scope is bounded metadata accounting; the capture still reports incomplete coverage and disallows equality, and no false comparison result was observed. Workaround: In the executed two-request workload, raising max_network_transactions from one to two retains the selected response-content event for the failed and unfinished cases with its explicit unavailable body state. A workload that fits the selected transaction budget avoids this omission-accounting defect while preserving the requested evidence. Assessment: P2. ### REA version rea 6.5.0 at source commit 4692bb33300c7cb77b095b6cc0c33da861c1b860 ### Runtime and platform Node 24.18.0; macOS arm64 ### Target and artifact context Simulated browser at the browser-opening boundary; the registered public CLI handler, parser, service, provider, session, network collection and Evidence projection run. No Chromium process or network request was used. ### Interface CLI ### Earliest failure stage Capture or observation ### Relevant output or evidence The focused test passed once after completing the simulated context's required methods, and the six CLI-handler calls returned Evidence with exit callback code zero. With only response bodies selected, the limit-one observations were: | Second transaction state | Omitted kinds and counts | Total dropped | | --- | --- | --- | | Finished | request: 1, response: 1, request-finished: 1, network-content: 1 | 4 | | Failed | request: 1, response: 1, request-failed: 1 | 3 | | Unfinished | request: 1, response: 1 | 2 | Each row identifies `request-2` in `network_transaction_range`. Every limit-one result has `completeness.status: "incomplete"`, `equality_eligible: false` and `missing_sections: ["events"]`. The limit-two controls retain the selected response-content event in all three cases and report no budget omissions. Therefore response-body selection alone reproduces the missing omission count; request-body and header-value selection are unnecessary. Source-derived diagnosis: the omitted response path remembers that a response occurred, while only the finished terminal path accounts for its selected response-content omission. The omitted failed path returns after recording the failed event, and cutoff finalization visits admitted transactions only. This diagnosis is separate from the executed CLI-handler observations. - Expected source: [docs/browser-scenario-contract.md:182-193](https://github.com/morluto/rea/blob/4692bb33300c7cb77b095b6cc0c33da861c1b860/docs/browser-scenario-contract.md#L182) promises omitted-event counts by kind and reason; [docs/browser-scenario-contract.md:318-322](https://github.com/morluto/rea/blob/4692bb33300c7cb77b095b6cc0c33da861c1b860/docs/browser-scenario-contract.md#L318) specifies unavailable response bodies at cutoff; [src/contracts/browserScenarioToolContracts.ts:25-35](https://github.com/morluto/rea/blob/4692bb33300c7cb77b095b6cc0c33da861c1b860/src/contracts/browserScenarioToolContracts.ts#L25) advertises independent content selection and explicit unavailable states. - Failure source: the executed simulated-browser CLI fixture above produces the omission matrix. The producing branches are [src/browser/network/PlaywrightScenarioNetwork.ts:209-212](https://github.com/morluto/rea/blob/4692bb33300c7cb77b095b6cc0c33da861c1b860/src/browser/network/PlaywrightScenarioNetwork.ts#L209), [src/browser/network/PlaywrightScenarioNetwork.ts:245-254](https://github.com/morluto/rea/blob/4692bb33300c7cb77b095b6cc0c33da861c1b860/src/browser/network/PlaywrightScenarioNetwork.ts#L245), [src/browser/network/PlaywrightScenarioNetwork.ts:277-283](https://github.com/morluto/rea/blob/4692bb33300c7cb77b095b6cc0c33da861c1b860/src/browser/network/PlaywrightScenarioNetwork.ts#L277) and [src/browser/network/PlaywrightScenarioNetwork.ts:316-360](https://github.com/morluto/rea/blob/4692bb33300c7cb77b095b6cc0c33da861c1b860/src/browser/network/PlaywrightScenarioNetwork.ts#L316). The fixture reproduces the public CLI handler with a simulated provider boundary, not real-browser behavior. Supported-entry-point reachability also follows from [src/cliBrowserScenarioCommands.ts:39-60](https://github.com/morluto/rea/blob/4692bb33300c7cb77b095b6cc0c33da861c1b860/src/cliBrowserScenarioCommands.ts#L39), [src/browser/PlaywrightScenarioSession.ts:242-252](https://github.com/morluto/rea/blob/4692bb33300c7cb77b095b6cc0c33da861c1b860/src/browser/PlaywrightScenarioSession.ts#L242) and [src/browser/PlaywrightScenarioEvents.ts:125-163](https://github.com/morluto/rea/blob/4692bb33300c7cb77b095b6cc0c33da861c1b860/src/browser/PlaywrightScenarioEvents.ts#L125). Launch mode admits these context requests through its ownership guard, and the CLI schema accepts the positive transaction limit and independent response-body selection. ### Typed outcome or recovery expected With the same fixture and transaction limit of one, each second request that observes a response must contribute exactly one omitted response-phase `network-content` event regardless of whether it finishes, fails, or remains unfinished at cutoff. The total dropped count must include that omission. Keep the finished control at one response-content omission, without double counting. With the limit raised to two, keep the selected response-content event and the failed/unfinished body's explicit unavailable reason. Preserve the incomplete/equality-ineligible status of budget-limited captures and avoid body reads for failed or unfinished responses. No fix or restoration run has been performed. ### Additional context Freshness checked against origin/main at 22ec8bdb2f7d4626587f23b1eed4a207df600386. The cited failing implementation files are unchanged from the reproduced baseline; this is a source freshness check, not a runtime rerun on that revision.

bsakaguchi referenced this issue

2026-10-17 21:01:31 +00:00

[GH-ISSUE #2383] [Tracker] Audit analysis and debugging coverage for the TIOBE top 100 languages (October 2026) #794

bsakaguchi referenced this issue

2026-10-17 21:12:47 +00:00

[PR #1610] [CLOSED] fix(javascript): bound module source payload before deep semantic analysis #3002

bsakaguchi closed this issue

2026-10-31 00:31:28 +00:00

bsakaguchi referenced this issue

2026-10-31 00:50:04 +00:00

[GH-ISSUE #7510] [Bug] analyze-javascript-application never completes on a 20 MB Expo Router web bundle, even with semantic limits at 1 #4466

bsakaguchi referenced this issue