morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #3868] [Bug] Process capture rendered lines gain an extra blank cell after wide characters #1488

New issue

Closed

opened 2026-10-16 21:06:05 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-16 21:06:05 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 15, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3868

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Process capture or runtime observation

Observed behavior

Process capture pads each rendered row to the terminal width using JavaScript string length, which counts a two-cell character such as 界 as one. The padding invents a blank cell that the terminal row never had. With a normalization rule that replaces spaces, the invented cell becomes visible: a four-column row "界 " is stored as "界___" instead of "界__".

The capture succeeds with columns: 4 and cursor_x: 2 but stores lines: ["界___"], which contains a third underscore for a cell that does not exist. The raw PTY output remains "界".

Reproduction

Run from the root of a built source checkout; the control uses the checkout's pinned @xterm/headless. The script creates and removes its own temporary directory.

node --input-type=module - <<'JS'
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import {spawnSync} from 'node:child_process';
import {createRequire} from 'node:module';
const cli=path.resolve('scripts/rea.mjs');
const work=fs.mkdtempSync(path.join(os.tmpdir(),'rea-wide-probe-'));
try {
  const scenario={executable:'/usr/bin/printf',arguments:['界'],terminal:{columns:4,rows:1,scrollback:0},normalization:{patterns:[{pattern:' ',replacement:'_'}]},timeout_ms:1000};
  fs.writeFileSync(path.join(work,'scenario.json'),JSON.stringify(scenario));
  const run=spawnSync(process.execPath,[cli,'capture-process','scenario.json','--json'],{cwd:work,encoding:'utf8',timeout:30000,maxBuffer:1024*1024});
  if(run.status!==0) throw new Error(run.stderr||run.stdout);
  const c=JSON.parse(run.stdout).normalized_result;
  const require=createRequire(path.resolve('package.json'));
  const {Terminal}=require('@xterm/headless');
  const t=new Terminal({allowProposedApi:true,cols:4,rows:1,scrollback:0});
  await new Promise(resolve=>t.write('界',resolve));
  const actualRow=t.buffer.active.getLine(0).translateToString(false,0,4);
  console.log(JSON.stringify({cli_status:run.status,raw_output:c.frames.map(f=>f.raw_data??f.data).join(''),columns:c.rendered_frames[0].columns,cursor_x:c.rendered_frames[0].cursor_x,lines:c.rendered_frames[0].lines,xterm_row:actualRow,expected_normalized_row:actualRow.replaceAll(' ','_')},null,2));
  t.dispose();
} finally {fs.rmSync(work,{recursive:true,force:true});}
JS

Expected behavior

The rendered line is "界__": the terminal row is "界" followed by two blank cells (as the pinned xterm reports), and normalization replaces each space with _. docs/process-capture.md:252-255 ("Scenario and result details") says rendered lines hold the visible rows after normalization and that columns records terminal cell width. AGENTS.md:19 requires consumers to use the producing owner's parsed facts.

Impact and scope

Rows containing wide characters gain an invented cell in retained rendered Evidence when normalization rewrites spaces, which comparison and trace consumers then treat as terminal output.

Workaround: available. The raw PTY output and serialized terminal state were correct in the reproduction and can be read instead; avoiding space-replacing normalization rules also avoids the trigger, though broader Unicode coverage was not tested.

Reporter's impact estimate: P2 (supported).

REA version

6.5.0, built from source at main commit 034342ba28 (latest npm release is 6.5.0)

Runtime and platform

Node.js v26.10.0; macOS 27.0.1 arm64

Target and artifact context

PTY process capture of /usr/bin/printf with a two-cell CJK character

Interface

CLI

Earliest failure stage

Capture or observation

Relevant output or evidence

Expected source: docs/process-capture.md:252-255 (rendered lines are the visible rows after normalization; columns is cell width)
Failure source: observed rea capture-process result below; .padEnd(this.#terminal.cols, " ") at src/process/capture/TerminalRenderer.ts:134-136
Evidence provenance: observed
Local verification: reproduced
Reproduction completeness: complete

{
  "cli_status": 0,
  "raw_output": "界",
  "columns": 4,
  "cursor_x": 2,
  "lines": [
    "界___"
  ],
  "xterm_row": "界  ",
  "expected_normalized_row": "界__"
}

Source-derived analysis: at src/process/capture/TerminalRenderer.ts:134-143, the renderer takes xterm's row (already selected by cell width via translateToString(false, 0, cols)) and calls .padEnd(cols, " "). Because "界 ".length is 3, padEnd(4) adds a fourth character before normalization runs. The serialized terminal state is not affected.

Typed outcome or recovery expected

Rendered lines should contain exactly the terminal row's cells after normalization ("界__" for the example), with no padding beyond the cell width.

Restoration check:

With the script above, lines is currently ["界___"]; after a fix it should equal expected_normalized_row, "界__". As a control, the same scenario with an ASCII argument such as ab should continue to produce "ab__" (proposed control; not run).

Analysis or suspected cause

Source-derived: src/process/capture/TerminalRenderer.ts:134-143 pads xterm's cell-width row with String.prototype.padEnd to the column count, which counts UTF-16 code units rather than cells.

Additional context

Closed #1464 corrected documentation that reconstructed trimmed rows by padding to JavaScript string length and explicitly separated that from producer Evidence loss; this report is about the producer itself. Closed #3168 concerns normalization budget accounting. No open issue reports this failure.

This report came from AI-assisted source review with local reproduction where stated above; the evidence labels distinguish observed runs from source-only reasoning.

Originally created by @coygeek on GitHub (Oct 15, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3868 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Process capture or runtime observation ### Observed behavior Process capture pads each rendered row to the terminal width using JavaScript string length, which counts a two-cell character such as `界` as one. The padding invents a blank cell that the terminal row never had. With a normalization rule that replaces spaces, the invented cell becomes visible: a four-column row `"界 "` is stored as `"界___"` instead of `"界__"`. The capture succeeds with `columns: 4` and `cursor_x: 2` but stores `lines: ["界___"]`, which contains a third underscore for a cell that does not exist. The raw PTY output remains `"界"`. ### Reproduction Run from the root of a built source checkout; the control uses the checkout's pinned `@xterm/headless`. The script creates and removes its own temporary directory. ```sh node --input-type=module - <<'JS' import fs from 'node:fs'; import os from 'node:os'; import path from 'node:path'; import {spawnSync} from 'node:child_process'; import {createRequire} from 'node:module'; const cli=path.resolve('scripts/rea.mjs'); const work=fs.mkdtempSync(path.join(os.tmpdir(),'rea-wide-probe-')); try { const scenario={executable:'/usr/bin/printf',arguments:['界'],terminal:{columns:4,rows:1,scrollback:0},normalization:{patterns:[{pattern:' ',replacement:'_'}]},timeout_ms:1000}; fs.writeFileSync(path.join(work,'scenario.json'),JSON.stringify(scenario)); const run=spawnSync(process.execPath,[cli,'capture-process','scenario.json','--json'],{cwd:work,encoding:'utf8',timeout:30000,maxBuffer:1024*1024}); if(run.status!==0) throw new Error(run.stderr||run.stdout); const c=JSON.parse(run.stdout).normalized_result; const require=createRequire(path.resolve('package.json')); const {Terminal}=require('@xterm/headless'); const t=new Terminal({allowProposedApi:true,cols:4,rows:1,scrollback:0}); await new Promise(resolve=>t.write('界',resolve)); const actualRow=t.buffer.active.getLine(0).translateToString(false,0,4); console.log(JSON.stringify({cli_status:run.status,raw_output:c.frames.map(f=>f.raw_data??f.data).join(''),columns:c.rendered_frames[0].columns,cursor_x:c.rendered_frames[0].cursor_x,lines:c.rendered_frames[0].lines,xterm_row:actualRow,expected_normalized_row:actualRow.replaceAll(' ','_')},null,2)); t.dispose(); } finally {fs.rmSync(work,{recursive:true,force:true});} JS ``` ### Expected behavior The rendered line is `"界__"`: the terminal row is `"界"` followed by two blank cells (as the pinned xterm reports), and normalization replaces each space with `_`. `docs/process-capture.md:252-255` ("Scenario and result details") says rendered `lines` hold the visible rows after normalization and that `columns` records terminal cell width. `AGENTS.md:19` requires consumers to use the producing owner's parsed facts. ### Impact and scope Rows containing wide characters gain an invented cell in retained rendered Evidence when normalization rewrites spaces, which comparison and trace consumers then treat as terminal output. Workaround: available. The raw PTY output and serialized terminal state were correct in the reproduction and can be read instead; avoiding space-replacing normalization rules also avoids the trigger, though broader Unicode coverage was not tested. Reporter's impact estimate: P2 (supported). ### REA version 6.5.0, built from source at main commit 034342ba28c8e56c3746be92b78aa7299afa02bd (latest npm release is 6.5.0) ### Runtime and platform Node.js v26.10.0; macOS 27.0.1 arm64 ### Target and artifact context PTY process capture of /usr/bin/printf with a two-cell CJK character ### Interface CLI ### Earliest failure stage Capture or observation ### Relevant output or evidence - Expected source: `docs/process-capture.md:252-255` (rendered `lines` are the visible rows after normalization; `columns` is cell width) - Failure source: observed `rea capture-process` result below; `.padEnd(this.#terminal.cols, " ")` at `src/process/capture/TerminalRenderer.ts:134-136` - Evidence provenance: observed - Local verification: reproduced - Reproduction completeness: complete ```json { "cli_status": 0, "raw_output": "界", "columns": 4, "cursor_x": 2, "lines": [ "界___" ], "xterm_row": "界 ", "expected_normalized_row": "界__" } ``` Source-derived analysis: at `src/process/capture/TerminalRenderer.ts:134-143`, the renderer takes xterm's row (already selected by cell width via `translateToString(false, 0, cols)`) and calls `.padEnd(cols, " ")`. Because `"界 ".length` is 3, `padEnd(4)` adds a fourth character before normalization runs. The serialized terminal state is not affected. ### Typed outcome or recovery expected Rendered lines should contain exactly the terminal row's cells after normalization ("界__" for the example), with no padding beyond the cell width. Restoration check: With the script above, `lines` is currently `["界___"]`; after a fix it should equal `expected_normalized_row`, `"界__"`. As a control, the same scenario with an ASCII argument such as `ab` should continue to produce `"ab__"` (proposed control; not run). ### Analysis or suspected cause Source-derived: src/process/capture/TerminalRenderer.ts:134-143 pads xterm's cell-width row with String.prototype.padEnd to the column count, which counts UTF-16 code units rather than cells. ### Additional context Closed #1464 corrected documentation that reconstructed trimmed rows by padding to JavaScript string length and explicitly separated that from producer Evidence loss; this report is about the producer itself. Closed #3168 concerns normalization budget accounting. No open issue reports this failure. This report came from AI-assisted source review with local reproduction where stated above; the evidence labels distinguish observed runs from source-only reasoning.

bsakaguchi commented

2026-10-16 21:06:07 +00:00

Author

Owner

@hichipli commented on GitHub (Oct 15, 2026):

Thanks for the precise reproduction. PR #4142 is now ready for review. I reproduced the extra cell on main a40ee741 through the real process-capture boundary, then corrected the renderer to consume xterm’s cell-width row directly.

Validation on macOS arm64: 34 focused tests passed (2 Linux-only skips), npm run check passed, and actual compiled CLI calls retain 界__ and the ASCII control ab__ in a four-column terminal. Raw output is preserved; owned process cleanup and temporary-root removal are reported. The PR includes Unicode controls, the failing-base regression and remaining platform/sampling limitations.

@hichipli commented on GitHub (Oct 15, 2026): Thanks for the precise reproduction. PR #4142 is now ready for review. I reproduced the extra cell on main `a40ee741` through the real process-capture boundary, then corrected the renderer to consume xterm’s cell-width row directly. Validation on macOS arm64: 34 focused tests passed (2 Linux-only skips), `npm run check` passed, and actual compiled CLI calls retain `界__` and the ASCII control `ab__` in a four-column terminal. Raw output is preserved; owned process cleanup and temporary-root removal are reported. The PR includes Unicode controls, the failing-base regression and remaining platform/sampling limitations.

bsakaguchi referenced this issue

2026-10-16 21:09:05 +00:00

[GH-ISSUE #4271] [Bug] Colliding ZIP paths receive a false integrity diagnosis and ineffective retry advice #1687

bsakaguchi referenced this issue

2026-10-17 21:11:59 +00:00

[PR #1513] [MERGED] fix: bind identity and coverage to the producing rule #2933

bsakaguchi referenced this issue

2026-10-17 21:12:11 +00:00

[PR #1535] [MERGED] fix(reference): join internal symlink targets with the inventory NFC normalizer #2952

bsakaguchi closed this issue

2026-10-31 00:31:00 +00:00