[GH-ISSUE #6168] [Feature] Declared permission grants and synthetic camera/microphone for capture_browser_scenario #3767

Open
opened 2026-10-31 00:43:50 +00:00 by bsakaguchi · 1 comment
Owner

Originally created by @debavelaerel on GitHub (Oct 21, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6168

Before submitting

  • I searched the open and closed issues and pull requests for a related proposal. I found none about browser permissions or media devices in capture_browser_scenario.
  • I understand the contribution scope must be reviewed before integration; open questions are welcome here.
  • This request does not include credentials, private samples, or sensitive target data.

Area

Process capture or runtime observation

Current limitation

capture_browser_scenario has no way to declare browser permission grants or synthetic media devices. A flow that calls getUserMedia (camera or microphone), geolocation, or notifications cannot be exercised. In launch mode the page simply sees "permission denied" and the scenario stops there.

Observed case: the public palmistry funnel https://hint.app/pt-BR/palmistry/welcome requires a live camera "palm scan" (the page says recognition runs on-device) before it continues to the results and paywall. Headless REA gets the site's "camera access denied" dialog and cannot observe anything after it.

Use cases

  • Reverse-engineering on-device ML flows such as hand, face, or document scanning: which models and WASM load, what is sent over the network, and what happens after recognition.
  • Flows gated by geolocation or notification prompts.

Desired outcome and success criteria

An explicit, caller-declared environment.permissions list (for example camera, microphone, geolocation) and an optional synthetic media source (Chromium's fake device, optionally fed from a caller-selected local Y4M or WAV file), recorded in the Evidence parameters. The default stays as it is today (nothing granted).

Expected interaction with targets

This lets the target page receive synthetic sensor input. That is new target-facing authority, which is why I'm filing it as a proposal for a scope decision rather than a PR.

Verification available so far

Only the negative observation above, on rea-agents 6.7.0 with Chrome 155 headless.

Alternatives considered

Launching Chrome myself with --use-fake-ui-for-media-stream --use-fake-device-for-media-stream and connecting by CDP. Connect mode works for passive observation, but the scenario provenance then no longer records the grant.

Possible direction

Playwright browserContext.grantPermissions plus Chromium fake-media launch arguments, owned by the scenario provider.

Originally created by @debavelaerel on GitHub (Oct 21, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6168 ### Before submitting - [x] I searched the open and closed issues and pull requests for a related proposal. I found none about browser permissions or media devices in `capture_browser_scenario`. - [x] I understand the [contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope) must be reviewed before integration; open questions are welcome here. - [x] This request does not include credentials, private samples, or sensitive target data. ### Area Process capture or runtime observation ### Current limitation `capture_browser_scenario` has no way to declare browser permission grants or synthetic media devices. A flow that calls `getUserMedia` (camera or microphone), geolocation, or notifications cannot be exercised. In launch mode the page simply sees "permission denied" and the scenario stops there. Observed case: the public palmistry funnel `https://hint.app/pt-BR/palmistry/welcome` requires a live camera "palm scan" (the page says recognition runs on-device) before it continues to the results and paywall. Headless REA gets the site's "camera access denied" dialog and cannot observe anything after it. ### Use cases - Reverse-engineering on-device ML flows such as hand, face, or document scanning: which models and WASM load, what is sent over the network, and what happens after recognition. - Flows gated by geolocation or notification prompts. ### Desired outcome and success criteria An explicit, caller-declared `environment.permissions` list (for example `camera`, `microphone`, `geolocation`) and an optional synthetic media source (Chromium's fake device, optionally fed from a caller-selected local Y4M or WAV file), recorded in the Evidence parameters. The default stays as it is today (nothing granted). ### Expected interaction with targets This lets the target page receive synthetic sensor input. That is new target-facing authority, which is why I'm filing it as a proposal for a scope decision rather than a PR. ### Verification available so far Only the negative observation above, on rea-agents 6.7.0 with Chrome 155 headless. ### Alternatives considered Launching Chrome myself with `--use-fake-ui-for-media-stream --use-fake-device-for-media-stream` and connecting by CDP. Connect mode works for passive observation, but the scenario provenance then no longer records the grant. ### Possible direction Playwright `browserContext.grantPermissions` plus Chromium fake-media launch arguments, owned by the scenario provider.
Author
Owner

@XinwangTao commented on GitHub (Oct 22, 2026):

赞同这个方向,对逆向 on-device ML 流程(手/脸/文档扫描)尤其有价值——没有合成媒体,这类站点的"识别前"阶段根本走不过去,观察不到后续的模型/WASM 加载和网络行为。两个实现层面的建议:一是权限授予走 CDP Browser.grantPermissions 并明确只在 capture 会话上下文生效,别污染用户真实的浏览器权限状态;二是合成摄像头最好支持注入可配置的帧源(图片或视频文件),这样能模拟不同的手掌/文档输入来覆盖更多识别分支,而不仅是固定色块。另外建议把 geolocation 的 mock 值和通知授权也纳入同一个声明式权限结构里,保持 capture 场景描述自包含。

<!-- gh-comment-id:6330365837 --> @XinwangTao commented on GitHub (Oct 22, 2026): 赞同这个方向,对逆向 on-device ML 流程(手/脸/文档扫描)尤其有价值——没有合成媒体,这类站点的"识别前"阶段根本走不过去,观察不到后续的模型/WASM 加载和网络行为。两个实现层面的建议:一是权限授予走 CDP Browser.grantPermissions 并明确只在 capture 会话上下文生效,别污染用户真实的浏览器权限状态;二是合成摄像头最好支持注入可配置的帧源(图片或视频文件),这样能模拟不同的手掌/文档输入来覆盖更多识别分支,而不仅是固定色块。另外建议把 geolocation 的 mock 值和通知授权也纳入同一个声明式权限结构里,保持 capture 场景描述自包含。
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
morluto/rea#3767
No description provided.