morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #5610] [Bug] capture-web-screenshot reports complete coverage for an image cropped below the visual viewport #3402

New issue

Closed

opened 2026-10-31 00:39:58 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-31 00:39:58 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 20, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/5610

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

capture-web-screenshot can return a cropped image as complete visible-viewport Evidence when another CDP client explicitly enlarges the visual viewport while preserving a smaller compositor surface. With Emulation.setDeviceMetricsOverride using dontSetVisibleSize: true, width 1280, height 720 and device scale 1, two independently verified production CLI captures returned a 1280×463 PNG, reported viewport: {width: 1280, height: 463} and completeness.status: "complete_within_window", and omitted the visible marker at CSS rows 650–699. This requires the explicit non-default viewport/surface mismatch; ordinary Playwright viewport controls succeeded.

Reproduction

Use the source revision and browser listed below, with the built CLI and installed dependencies. This needs a disposable browser with a loopback debugging endpoint and a second CDP client that explicitly preserves a smaller compositor surface while changing the visual viewport. Set CHROMIUM to the installed Chrome for Testing executable before the browser command.

Create screenshot-fixture/index.html containing the following page and serve that directory on loopback with python3 -m http.server 8000 --bind 127.0.0.1 --directory screenshot-fixture.

<!doctype html><html><head><meta charset="utf-8"><title>screenshot crop</title></head><body style="margin:0">
<div style="height:600px;background:#eee">top</div>
<div id="marker" style="position:absolute;top:650px;left:0;width:200px;height:50px;background:#c00">marker</div>
</body></html>

Start the disposable browser from the checkout directory. The browser's native content area must be shorter than 650 CSS pixels; the actual native content height depends on browser chrome and window constraints.

"$CHROMIUM" --remote-debugging-port=9333 --user-data-dir=screenshot-profile \
  --window-size=1280,720 --force-device-scale-factor=1 \
  --no-first-run --no-default-browser-check --disable-background-networking \
  --disable-component-update --disable-sync --metrics-recording-only \
  --host-resolver-rules='MAP * ~NOTFOUND, EXCLUDE 127.0.0.1' about:blank

From the baseline checkout, run the following second-client script. It uses the installed Playwright dependency to configure the viewport, invokes the production REA CLI, writes both PNGs, and closes the disposable browser in its cleanup block. Keep the server running until the script exits.

node --input-type=module <<'JS'
import assert from 'node:assert/strict';
import { chromium } from 'playwright-core';
import { execFile } from 'node:child_process';
import { promisify } from 'node:util';
import { writeFile } from 'node:fs/promises';
const exec = promisify(execFile);
const endpoint = 'http://127.0.0.1:9333';
const origin = 'http://127.0.0.1:8000';
const browser = await chromium.connectOverCDP(endpoint);
const dimensions = bytes => ({width: bytes.readUInt32BE(16), height: bytes.readUInt32BE(20)});
try {
  const context = browser.contexts()[0];
  const page = context.pages()[0] ?? await context.newPage();
  await page.goto(origin, {waitUntil: 'domcontentloaded'});
  const session = await context.newCDPSession(page);
  const native = (await session.send('Page.getLayoutMetrics')).cssVisualViewport;
  assert.ok(native.clientHeight < 650, 'Use a smaller native browser window');
  await session.send('Emulation.setDeviceMetricsOverride', {
    width: 1280, height: 720, deviceScaleFactor: 1,
    mobile: false, dontSetVisibleSize: true
  });
  const before = (await session.send('Page.getLayoutMetrics')).cssVisualViewport;
  const marker = await page.locator('#marker').boundingBox();
  assert.equal(before.clientHeight, 720);
  assert.equal(marker.y, 650);
  const options = {maxBuffer: 10000000, timeout: 60000};
  await exec(process.execPath, ['scripts/rea.mjs', 'list-browser-targets', endpoint, '--json'], options);
  const {targetInfo} = await session.send('Target.getTargetInfo');
  const {stdout} = await exec(process.execPath, [
    'scripts/rea.mjs', 'capture-web-screenshot', endpoint, targetInfo.targetId,
    '--allowed-origins', origin, '--json'
  ], options);
  const result = JSON.parse(stdout).normalized_result;
  const actual = Buffer.from(result.artifact.data_base64, 'base64');
  await writeFile('screenshot-fixture/rea.png', actual);
  const control = await session.send('Page.captureScreenshot', {
    format: 'png', fromSurface: true, captureBeyondViewport: true,
    clip: {x: before.pageX, y: before.pageY,
      width: before.clientWidth, height: before.clientHeight, scale: 1}
  });
  const controlPng = Buffer.from(control.data, 'base64');
  await writeFile('screenshot-fixture/control.png', controlPng);
  console.log(JSON.stringify({before, marker, png: dimensions(actual),
    viewport: result.viewport, completeness: result.completeness,
    limitations: result.limitations, control: dimensions(controlPng)}, null, 2));
} finally {
  const session = await browser.newBrowserCDPSession();
  await session.send('Browser.close').catch(() => {});
  await browser.close();
}
JS

Compare the reported visual viewport, PNG dimensions and completeness, and inspect both images. The failing image stops before the marker while the clipped control contains it. Stop the fixture server and remove the task-created fixture and profile directories after inspection. The exact failing PNG height is determined by the native surface; 463 was observed locally, rather than a universal expected value.

Expected behavior

docs/browser-observation.md:34 promises a visible-viewport PNG and line 260 promises the complete visible viewport at the tested revision. With device scale 1 and a 1280×720 visual viewport, complete coverage must include the marker within that rectangle. If capture cannot cover it, return an explicit failure or truthful incomplete coverage. The external-emulation/disconnect warning at lines 77–85 concerns state preservation and does not exclude these pages from screenshot coverage.

Impact and scope

P2: silent loss of visible screenshot evidence in this demonstrated non-default configuration, while reporting complete coverage. Two explicit mismatch runs reproduced it. Native-window and ordinary Playwright newContext({viewport: {width: 1280, height: 720}}) controls succeeded; broader DevTools/Playwright incidence is unknown. No origin boundary was crossed or security consequence demonstrated.

A verified alternate capture from the owning CDP client used captureBeyondViewport: true and an explicit visual-viewport clip, returning all 720 rows and the marker. Browser resizing was not verified as a workaround. The owning client may need to reapply emulation after REA disconnects.

REA version

6.7.0 (rea-agents); source commit 074768901f3cdc8205b61773fb1faf6f6c7b3ca8.

Runtime and platform

Node v26.12.0; macOS arm64; Chrome for Testing 156.0.8078.4; Playwright-core 1.58.2. The decisive browser launch forced device scale 1.

Target and artifact context

Disposable Chromium/profile and an owned HTML fixture served only on 127.0.0.1; inline PNG screenshot Evidence. The second CDP client remains connected with its explicit metrics override during capture. No personal browser or external page was used.

Interface

CLI

Earliest failure stage

Capture or observation

Relevant output or evidence

Observed production CLI summary (both explicit mismatch runs exited 0):

{
  "visual_viewport_before": {"width": 1280, "height": 720},
  "png": {"width": 1280, "height": 463},
  "viewport": {"width": 1280, "height": 463},
  "completeness": {
    "status": "complete_within_window",
    "excluded": [],
    "truncated_sections": [],
    "unavailable_sections": []
  },
  "clipped_control_png": {"width": 1280, "height": 720}
}

This is a reduced summary, not a verbatim full CLI result. The REA PNG had no marker pixels; the clipped control contained the marker at rows 650–699. The limitations claimed the screenshot contained the visible viewport.

A separate direct page-CDP control replayed REA's Page.enable / Page.getFrameTree / Page.captureScreenshot sequence and read layout metrics immediately before and after capture: both stayed 1280×720 while the PNG was 1280×463. This establishes capture-time cropping independently of the emulation reset observed after disconnect; it is not a claim that the CLI reports layout metrics.

Typed outcome or recovery expected

Repeat the explicit mismatch: successful complete Evidence must cover the full visual viewport and contain the marker; otherwise expose the missing coverage or return an explicit failure. Preserve native-window and ordinary emulated-context captures, device-scale handling, origin authorization and document-identity checks.

Analysis or suspected cause

Source-derived: src/browser/CdpScreenshot.ts:76–83 requests {format: "png", fromSurface: true, captureBeyondViewport: false} without an explicit viewport clip. Lines 125–130 derive viewport from the PNG dimensions and return complete coverage without checking actual viewport metrics. The production CLI and independent capture-time control establish the observable failure; this source reading explains why a shorter surface can be described as complete.

Additional context

Prepared with AI assistance and independently reproduced by a separate verifier. The submission check fetched current origin/main at c3d7d0aafc4943c82a5a8513cd3da7f4a7cfe643: CdpScreenshot.ts, the screenshot domain and screenshot e2e source are unchanged from the tested baseline; the only intervening change to the cited guide documents long passive-observation waits. No browser rerun was performed during submission.

Related #4089 concerns caller emulation state on disconnect; this report concerns incomplete pixels while the visual viewport is still 720 rows high. Searches covered open/closed issues and PRs, including merged PR #1321 (scenario emulation cleanup), #2988 (comparison bounds), #4040 (PNG validation/navigation), and closed #4956 (geometry drift during capture); none reports this stable visual-viewport/compositor-surface mismatch. Ports in the recipe are fixed examples replacing dynamically selected loopback ports used during verification. Verification closed its servers and browsers and removed disposable profiles.

Originally created by @coygeek on GitHub (Oct 20, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/5610 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior `capture-web-screenshot` can return a cropped image as complete visible-viewport Evidence when another CDP client explicitly enlarges the visual viewport while preserving a smaller compositor surface. With `Emulation.setDeviceMetricsOverride` using `dontSetVisibleSize: true`, width 1280, height 720 and device scale 1, two independently verified production CLI captures returned a 1280×463 PNG, reported `viewport: {width: 1280, height: 463}` and `completeness.status: "complete_within_window"`, and omitted the visible marker at CSS rows 650–699. This requires the explicit non-default viewport/surface mismatch; ordinary Playwright viewport controls succeeded. ### Reproduction Use the source revision and browser listed below, with the built CLI and installed dependencies. This needs a disposable browser with a loopback debugging endpoint and a second CDP client that explicitly preserves a smaller compositor surface while changing the visual viewport. Set `CHROMIUM` to the installed Chrome for Testing executable before the browser command. 1. Create `screenshot-fixture/index.html` containing the following page and serve that directory on loopback with `python3 -m http.server 8000 --bind 127.0.0.1 --directory screenshot-fixture`. ```html <!doctype html><html><head><meta charset="utf-8"><title>screenshot crop</title></head><body style="margin:0"> <div style="height:600px;background:#eee">top</div> <div id="marker" style="position:absolute;top:650px;left:0;width:200px;height:50px;background:#c00">marker</div> </body></html> ``` 2. Start the disposable browser from the checkout directory. The browser's native content area must be shorter than 650 CSS pixels; the actual native content height depends on browser chrome and window constraints. ```sh "$CHROMIUM" --remote-debugging-port=9333 --user-data-dir=screenshot-profile \ --window-size=1280,720 --force-device-scale-factor=1 \ --no-first-run --no-default-browser-check --disable-background-networking \ --disable-component-update --disable-sync --metrics-recording-only \ --host-resolver-rules='MAP * ~NOTFOUND, EXCLUDE 127.0.0.1' about:blank ``` 3. From the baseline checkout, run the following second-client script. It uses the installed Playwright dependency to configure the viewport, invokes the production REA CLI, writes both PNGs, and closes the disposable browser in its cleanup block. Keep the server running until the script exits. ```sh node --input-type=module <<'JS' import assert from 'node:assert/strict'; import { chromium } from 'playwright-core'; import { execFile } from 'node:child_process'; import { promisify } from 'node:util'; import { writeFile } from 'node:fs/promises'; const exec = promisify(execFile); const endpoint = 'http://127.0.0.1:9333'; const origin = 'http://127.0.0.1:8000'; const browser = await chromium.connectOverCDP(endpoint); const dimensions = bytes => ({width: bytes.readUInt32BE(16), height: bytes.readUInt32BE(20)}); try { const context = browser.contexts()[0]; const page = context.pages()[0] ?? await context.newPage(); await page.goto(origin, {waitUntil: 'domcontentloaded'}); const session = await context.newCDPSession(page); const native = (await session.send('Page.getLayoutMetrics')).cssVisualViewport; assert.ok(native.clientHeight < 650, 'Use a smaller native browser window'); await session.send('Emulation.setDeviceMetricsOverride', { width: 1280, height: 720, deviceScaleFactor: 1, mobile: false, dontSetVisibleSize: true }); const before = (await session.send('Page.getLayoutMetrics')).cssVisualViewport; const marker = await page.locator('#marker').boundingBox(); assert.equal(before.clientHeight, 720); assert.equal(marker.y, 650); const options = {maxBuffer: 10000000, timeout: 60000}; await exec(process.execPath, ['scripts/rea.mjs', 'list-browser-targets', endpoint, '--json'], options); const {targetInfo} = await session.send('Target.getTargetInfo'); const {stdout} = await exec(process.execPath, [ 'scripts/rea.mjs', 'capture-web-screenshot', endpoint, targetInfo.targetId, '--allowed-origins', origin, '--json' ], options); const result = JSON.parse(stdout).normalized_result; const actual = Buffer.from(result.artifact.data_base64, 'base64'); await writeFile('screenshot-fixture/rea.png', actual); const control = await session.send('Page.captureScreenshot', { format: 'png', fromSurface: true, captureBeyondViewport: true, clip: {x: before.pageX, y: before.pageY, width: before.clientWidth, height: before.clientHeight, scale: 1} }); const controlPng = Buffer.from(control.data, 'base64'); await writeFile('screenshot-fixture/control.png', controlPng); console.log(JSON.stringify({before, marker, png: dimensions(actual), viewport: result.viewport, completeness: result.completeness, limitations: result.limitations, control: dimensions(controlPng)}, null, 2)); } finally { const session = await browser.newBrowserCDPSession(); await session.send('Browser.close').catch(() => {}); await browser.close(); } JS ``` 4. Compare the reported visual viewport, PNG dimensions and completeness, and inspect both images. The failing image stops before the marker while the clipped control contains it. Stop the fixture server and remove the task-created fixture and profile directories after inspection. The exact failing PNG height is determined by the native surface; 463 was observed locally, rather than a universal expected value. ### Expected behavior `docs/browser-observation.md:34` promises a visible-viewport PNG and line 260 promises the complete visible viewport at the tested revision. With device scale 1 and a 1280×720 visual viewport, complete coverage must include the marker within that rectangle. If capture cannot cover it, return an explicit failure or truthful incomplete coverage. The external-emulation/disconnect warning at lines 77–85 concerns state preservation and does not exclude these pages from screenshot coverage. ### Impact and scope P2: silent loss of visible screenshot evidence in this demonstrated non-default configuration, while reporting complete coverage. Two explicit mismatch runs reproduced it. Native-window and ordinary Playwright `newContext({viewport: {width: 1280, height: 720}})` controls succeeded; broader DevTools/Playwright incidence is unknown. No origin boundary was crossed or security consequence demonstrated. A verified alternate capture from the owning CDP client used `captureBeyondViewport: true` and an explicit visual-viewport clip, returning all 720 rows and the marker. Browser resizing was not verified as a workaround. The owning client may need to reapply emulation after REA disconnects. ### REA version 6.7.0 (rea-agents); source commit `074768901f3cdc8205b61773fb1faf6f6c7b3ca8`. ### Runtime and platform Node v26.12.0; macOS arm64; Chrome for Testing 156.0.8078.4; Playwright-core 1.58.2. The decisive browser launch forced device scale 1. ### Target and artifact context Disposable Chromium/profile and an owned HTML fixture served only on 127.0.0.1; inline PNG screenshot Evidence. The second CDP client remains connected with its explicit metrics override during capture. No personal browser or external page was used. ### Interface CLI ### Earliest failure stage Capture or observation ### Relevant output or evidence Observed production CLI summary (both explicit mismatch runs exited 0): ```json { "visual_viewport_before": {"width": 1280, "height": 720}, "png": {"width": 1280, "height": 463}, "viewport": {"width": 1280, "height": 463}, "completeness": { "status": "complete_within_window", "excluded": [], "truncated_sections": [], "unavailable_sections": [] }, "clipped_control_png": {"width": 1280, "height": 720} } ``` This is a reduced summary, not a verbatim full CLI result. The REA PNG had no marker pixels; the clipped control contained the marker at rows 650–699. The limitations claimed the screenshot contained the visible viewport. A separate direct page-CDP control replayed REA's `Page.enable` / `Page.getFrameTree` / `Page.captureScreenshot` sequence and read layout metrics immediately before and after capture: both stayed 1280×720 while the PNG was 1280×463. This establishes capture-time cropping independently of the emulation reset observed after disconnect; it is not a claim that the CLI reports layout metrics. ### Typed outcome or recovery expected Repeat the explicit mismatch: successful complete Evidence must cover the full visual viewport and contain the marker; otherwise expose the missing coverage or return an explicit failure. Preserve native-window and ordinary emulated-context captures, device-scale handling, origin authorization and document-identity checks. ### Analysis or suspected cause Source-derived: `src/browser/CdpScreenshot.ts:76–83` requests `{format: "png", fromSurface: true, captureBeyondViewport: false}` without an explicit viewport clip. Lines 125–130 derive `viewport` from the PNG dimensions and return complete coverage without checking actual viewport metrics. The production CLI and independent capture-time control establish the observable failure; this source reading explains why a shorter surface can be described as complete. ### Additional context Prepared with AI assistance and independently reproduced by a separate verifier. The submission check fetched current `origin/main` at `c3d7d0aafc4943c82a5a8513cd3da7f4a7cfe643`: `CdpScreenshot.ts`, the screenshot domain and screenshot e2e source are unchanged from the tested baseline; the only intervening change to the cited guide documents long passive-observation waits. No browser rerun was performed during submission. Related #4089 concerns caller emulation state on disconnect; this report concerns incomplete pixels while the visual viewport is still 720 rows high. Searches covered open/closed issues and PRs, including merged PR #1321 (scenario emulation cleanup), #2988 (comparison bounds), #4040 (PNG validation/navigation), and closed #4956 (geometry drift during capture); none reports this stable visual-viewport/compositor-surface mismatch. Ports in the recipe are fixed examples replacing dynamically selected loopback ports used during verification. Verification closed its servers and browsers and removed disposable profiles.

bsakaguchi closed this issue

2026-10-31 00:39:58 +00:00

bsakaguchi commented

2026-10-31 00:39:59 +00:00

Author

Owner

@matbrgz commented on GitHub (Oct 20, 2026):

Cross-referencing with canonical issue #5609 ([Bug] observe-web-session returns pre-window lifecycle milestones as observed activity). Both address lifecycle milestone boundaries, viewport clipping, and accurate coverage reporting in browser observation tools (observe-web-session and capture-web-screenshot). Cataloged under Cluster BI in consolidation tracker #4330.

@matbrgz commented on GitHub (Oct 20, 2026): Cross-referencing with canonical issue #5609 (`[Bug] observe-web-session returns pre-window lifecycle milestones as observed activity`). Both address lifecycle milestone boundaries, viewport clipping, and accurate coverage reporting in browser observation tools (`observe-web-session` and `capture-web-screenshot`). Cataloged under Cluster BI in consolidation tracker #4330.

bsakaguchi referenced this issue

2026-10-31 00:46:08 +00:00

[GH-ISSUE #6639] [Bug] WASM glue drops valid URL literals with line separators in queries or fragments #4021

bsakaguchi referenced this issue

2026-10-31 01:09:08 +00:00

[PR #3455] [CLOSED] fix(wasm): preserve local matches for dotted selected paths #5746

bsakaguchi referenced this issue

2026-11-01 21:46:14 +00:00

[PR #6237] [MERGED] fix(wasm): test glue literals for a .wasm suffix in linear time #7217