[GH-ISSUE #6166] [Bug] capture_browser_scenario reports cleanup_incomplete and drops Evidence after all actions completed #3766

New issue

Open

opened 2026-10-31 00:43:50 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-31 00:43:50 +00:00

Owner

Originally created by @debavelaerel on GitHub (Oct 21, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6166

Before submitting

I searched the open and closed issues for a related report. Related but distinct: #4659 (profile left behind after process termination), #5989 (CDP discovery/attach cleanup erases the primary failure), #4227 (lifecycle audit).
I removed credentials, tokens, private paths, and sensitive target data.

Area

Process capture or runtime observation

Observed behavior

capture_browser_scenario (launch mode) intermittently ends with cleanup_incomplete even though every declared action completed:

{"error":"Browser scenario capture failed","code":"cleanup_incomplete","category":"execution_failure",
 "message":"Browser scenario cleanup could not be confirmed: Analysis operation timed out after 30000ms: capture_browser_scenario. Collected steps and events are retained in partial_observation. Review the reported resources before retrying.",
 "retryable":false,
 "details":{"provider_id":"rea-playwright-browser-scenario","operation":"capture_browser_scenario","cleanup":"incomplete","resources":["browser_transport","browser_profile"], ...}}

In the second case the retained partial_observation shows all steps completed, including the final wait_for_timeout; the timeout therefore happened after the last action, during final capture or teardown. Consequences:

The completed observation has no evidence_id, so it cannot be referenced, exported, or compared like a normal result even though nothing in the declared scenario failed.
retryable: false plus "review the reported resources" gives the caller no way to tell whether the owned Chrome process and temporary profile still exist.
Wall time grew from ~45 s to >240 s for the same prefix on the failing runs.

Seen in two situations on a public Next.js funnel (https://hint.app/pt-BR/...):

A. After a click that makes the page call getUserMedia (camera) in headless Chrome: the page shows its own "camera denied" dialog; a following click on that dialog's button timed out and the run ended with cleanup_incomplete.
B. No camera involved: a ~45-action click-through of https://hint.app/pt-BR/marriage-reading/welcome whose later screens run several Rive (.riv + rive.wasm) canvas animations. All actions completed; the result was still cleanup_incomplete. Re-running the identical scenario one action longer completed normally, so this is intermittent.

Reproduction

CLI, rea capture-browser-scenario <input.json> --format json, with:

{"browser":{"mode":"launch","executable_path":"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome","headless":true},
 "start_url":{"url":"https://hint.app/pt-BR/palmistry/welcome"},
 "environment":{"locale":"pt-BR","timezone":"America/Sao_Paulo"},
 "capture":{"after_each_step":["url","accessibility"],"at_end":["url","accessibility"]},
 "actions":[ "...click through the quiz (role=button locators) until the 'Iniciar escaneamento' button...",
   {"step_id":"scan","action":"click","locator":{"kind":"role","role":"button","name":"Iniciar escaneamento"},"timeout_ms":12000},
   {"step_id":"w","action":"wait_for_timeout","duration_ms":5000},
   {"step_id":"dismiss","action":"click","locator":{"kind":"css","selector":"[role=dialog] button >> nth=1"},"timeout_ms":12000},
   {"step_id":"settle","action":"wait_for_timeout","duration_ms":2500}]}

I can attach the full action lists for both cases if useful. The third-party site can change at any time, so a local fixture page that calls getUserMedia and runs a continuous canvas/WASM animation is probably a better regression target.

Expected behavior

When all actions complete, a slow final capture or teardown should not discard the result. Return Evidence with an evidence_id, mark the missing final sections as incomplete, and report cleanup status separately.
If cleanup really is incomplete, say which owned resources (process identity, profile path) are still present, so the caller can act on them. #4659 covers the profile half of this.

Impact and scope

Agents driving multi-step web funnels lose completed observations and are told not to retry. In my run this happened 2 out of ~35 scenario captures.

REA version

rea-agents 6.7.0 (npm). Not yet re-verified on current main.

Runtime and platform

macOS 26.6 arm64, Node 24.14.0, Google Chrome 155.0.8059.85 (headless launch)

Target and artifact context

Public website, https://hint.app/pt-BR/palmistry/welcome and /pt-BR/marriage-reading/welcome (Next.js on Cloudflare, Rive animations)

Interface

CLI (the MCP tool showed the same behavior in this session)

Earliest failure stage

Capture or observation

Originally created by @debavelaerel on GitHub (Oct 21, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6166 ### Before submitting - [x] I searched the open and closed issues for a related report. Related but distinct: #4659 (profile left behind after process termination), #5989 (CDP discovery/attach cleanup erases the primary failure), #4227 (lifecycle audit). - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Process capture or runtime observation ### Observed behavior `capture_browser_scenario` (launch mode) intermittently ends with `cleanup_incomplete` even though every declared action completed: ```json {"error":"Browser scenario capture failed","code":"cleanup_incomplete","category":"execution_failure", "message":"Browser scenario cleanup could not be confirmed: Analysis operation timed out after 30000ms: capture_browser_scenario. Collected steps and events are retained in partial_observation. Review the reported resources before retrying.", "retryable":false, "details":{"provider_id":"rea-playwright-browser-scenario","operation":"capture_browser_scenario","cleanup":"incomplete","resources":["browser_transport","browser_profile"], ...}} ``` In the second case the retained `partial_observation` shows all steps `completed`, including the final `wait_for_timeout`; the timeout therefore happened after the last action, during final capture or teardown. Consequences: 1. The completed observation has no `evidence_id`, so it cannot be referenced, exported, or compared like a normal result even though nothing in the declared scenario failed. 2. `retryable: false` plus "review the reported resources" gives the caller no way to tell whether the owned Chrome process and temporary profile still exist. 3. Wall time grew from ~45 s to >240 s for the same prefix on the failing runs. Seen in two situations on a public Next.js funnel (`https://hint.app/pt-BR/...`): - **A.** After a click that makes the page call `getUserMedia` (camera) in headless Chrome: the page shows its own "camera denied" dialog; a following click on that dialog's button timed out and the run ended with `cleanup_incomplete`. - **B.** No camera involved: a ~45-action click-through of `https://hint.app/pt-BR/marriage-reading/welcome` whose later screens run several Rive (`.riv` + `rive.wasm`) canvas animations. All actions completed; the result was still `cleanup_incomplete`. Re-running the identical scenario one action longer completed normally, so this is intermittent. ### Reproduction CLI, `rea capture-browser-scenario <input.json> --format json`, with: ```json {"browser":{"mode":"launch","executable_path":"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome","headless":true}, "start_url":{"url":"https://hint.app/pt-BR/palmistry/welcome"}, "environment":{"locale":"pt-BR","timezone":"America/Sao_Paulo"}, "capture":{"after_each_step":["url","accessibility"],"at_end":["url","accessibility"]}, "actions":[ "...click through the quiz (role=button locators) until the 'Iniciar escaneamento' button...", {"step_id":"scan","action":"click","locator":{"kind":"role","role":"button","name":"Iniciar escaneamento"},"timeout_ms":12000}, {"step_id":"w","action":"wait_for_timeout","duration_ms":5000}, {"step_id":"dismiss","action":"click","locator":{"kind":"css","selector":"[role=dialog] button >> nth=1"},"timeout_ms":12000}, {"step_id":"settle","action":"wait_for_timeout","duration_ms":2500}]} ``` I can attach the full action lists for both cases if useful. The third-party site can change at any time, so a local fixture page that calls `getUserMedia` and runs a continuous canvas/WASM animation is probably a better regression target. ### Expected behavior - When all actions complete, a slow final capture or teardown should not discard the result. Return Evidence with an `evidence_id`, mark the missing final sections as incomplete, and report cleanup status separately. - If cleanup really is incomplete, say which owned resources (process identity, profile path) are still present, so the caller can act on them. #4659 covers the profile half of this. ### Impact and scope Agents driving multi-step web funnels lose completed observations and are told not to retry. In my run this happened 2 out of ~35 scenario captures. ### REA version rea-agents 6.7.0 (npm). Not yet re-verified on current `main`. ### Runtime and platform macOS 26.6 arm64, Node 24.14.0, Google Chrome 155.0.8059.85 (headless launch) ### Target and artifact context Public website, https://hint.app/pt-BR/palmistry/welcome and /pt-BR/marriage-reading/welcome (Next.js on Cloudflare, Rive animations) ### Interface CLI (the MCP tool showed the same behavior in this session) ### Earliest failure stage Capture or observation

bsakaguchi commented

2026-10-31 00:43:50 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 22, 2026):

确认这是失败语义被污染的问题：所有 action 已完成、Evidence 已收集，却因为 cleanup 环节 30s 超时把整个 capture 判成 retryable=false 的失败，用户反而拿不到本该拿到的完整结果——这跟 #5989 里"cleanup 抹掉主失败信息"是同一类反模式。建议把 capture 与 cleanup 的成败分开建模：cleanup 超时/失败只降级为警告字段，已收集的 steps/events 照常作为结果返回；若担心资源泄漏，可以让 cleanup 异步后台重试而非阻塞在主结果路径上，同时把 profile 遗留问题挂到 #4659 一起解决。

@XinwangTao commented on GitHub (Oct 22, 2026): 确认这是失败语义被污染的问题：所有 action 已完成、Evidence 已收集，却因为 cleanup 环节 30s 超时把整个 capture 判成 retryable=false 的失败，用户反而拿不到本该拿到的完整结果——这跟 #5989 里"cleanup 抹掉主失败信息"是同一类反模式。建议把 capture 与 cleanup 的成败分开建模：cleanup 超时/失败只降级为警告字段，已收集的 steps/events 照常作为结果返回；若担心资源泄漏，可以让 cleanup 异步后台重试而非阻塞在主结果路径上，同时把 profile 遗留问题挂到 #4659 一起解决。

bsakaguchi referenced this issue

2026-10-31 01:10:51 +00:00

[PR #3967] feat(procmon): read saved Process Monitor XML and CSV exports as Evidence #5972

bsakaguchi referenced this issue

2026-11-01 21:10:57 +00:00

[GH-ISSUE #3796] [Tracker] Open feature requests by area (October 2026) #1471