[GH-ISSUE #4033] [Bug] Browser environment verifier accepts unrelated errors as cancellation #1605

New issue

Closed

opened 2026-10-16 21:07:49 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-16 21:07:49 +00:00

Owner

Originally created by @N0zoM1z0 on GitHub (Oct 15, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/4033

Area

Other — real-browser CI verifier

Observed behavior

The attached-browser environment cancellation verifier accepts every Result failure as successful cancellation. Controlled AnalysisProtocolError, AnalysisCancelledError with incomplete cleanup, and a cancelled-step observation with browser.cleanup: "incomplete" all pass its existing oracle. It also leaves its directly created provider unclosed.

Reproduction

Baseline: 337d2e33c5aae40f20c64584c341a3593420b8e0 (the affected verifier is identical at 6688ed6d, the initially inspected main).

Executed the existing verifyCancelledEnvironment function with a recording CDP connection that supplies the two required emulation snapshots and a provider that settles only when its actual cancellation signal aborts. Six controls: typed cancellation and a cancelled step pass; completed actions fail; unrelated protocol failure, incomplete cancellation cleanup and incomplete-step cleanup incorrectly pass.

This is controlled verifier-boundary evidence, not a reproduced browser defect. The complete unmodified npm run verify:browser separately passed on Linux x64 with real Chromium 153.0.8010.12, Node 24.18.0 and npm 11.16.0.

Expected behavior

Require the typed cancellation outcome or a genuinely cancelled step, require confirmed cleanup, close the owned provider, and preserve unrelated failures. The subsequent external-target and restored-environment checks must remain.

Impact and scope

A real-browser CI run can report cancellation_cleanup: true for an unrelated operation failure or unconfirmed cleanup. This is independent of the storage/service-worker cleanup repair already covered by #4010. No claim that this caused the main CI failure in run 38702522552.

Version / interface / stage

Source SHA above; Linux x64; Node 24.18.0; npm 11.16.0. Interface: verifier calling the production provider. Earliest stage: verification of cancellation output.

Supporting evidence and limits

The original condition is !cancelled.ok || cancelled.value.steps.some(({ status }) => status === "cancelled") in scripts/lib/browser-scenario-environment-e2e.mjs. All six controls were executed; real engine injection and Windows/macOS runs are unavailable for this finding.

Originally created by @N0zoM1z0 on GitHub (Oct 15, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/4033 ### Area Other — real-browser CI verifier ### Observed behavior The attached-browser environment cancellation verifier accepts every `Result` failure as successful cancellation. Controlled `AnalysisProtocolError`, `AnalysisCancelledError` with incomplete cleanup, and a cancelled-step observation with `browser.cleanup: "incomplete"` all pass its existing oracle. It also leaves its directly created provider unclosed. ### Reproduction Baseline: `337d2e33c5aae40f20c64584c341a3593420b8e0` (the affected verifier is identical at `6688ed6d`, the initially inspected main). Executed the existing `verifyCancelledEnvironment` function with a recording CDP connection that supplies the two required emulation snapshots and a provider that settles only when its actual cancellation signal aborts. Six controls: typed cancellation and a cancelled step pass; completed actions fail; **unrelated protocol failure, incomplete cancellation cleanup and incomplete-step cleanup incorrectly pass**. This is controlled verifier-boundary evidence, not a reproduced browser defect. The complete unmodified `npm run verify:browser` separately passed on Linux x64 with real Chromium 153.0.8010.12, Node 24.18.0 and npm 11.16.0. ### Expected behavior Require the typed cancellation outcome or a genuinely cancelled step, require confirmed cleanup, close the owned provider, and preserve unrelated failures. The subsequent external-target and restored-environment checks must remain. ### Impact and scope A real-browser CI run can report `cancellation_cleanup: true` for an unrelated operation failure or unconfirmed cleanup. This is independent of the storage/service-worker cleanup repair already covered by #4010. No claim that this caused the main CI failure in run 38702522552. ### Version / interface / stage Source SHA above; Linux x64; Node 24.18.0; npm 11.16.0. Interface: verifier calling the production provider. Earliest stage: verification of cancellation output. ### Supporting evidence and limits The original condition is `!cancelled.ok || cancelled.value.steps.some(({ status }) => status === "cancelled")` in `scripts/lib/browser-scenario-environment-e2e.mjs`. All six controls were executed; real engine injection and Windows/macOS runs are unavailable for this finding.

bsakaguchi

2026-10-16 21:07:49 +00:00