[GH-ISSUE #6665] [Bug] Desktop inspection duplicates a sibling after lossy POSIX filename decoding #4043

New issue

Closed

opened 2026-10-31 00:46:18 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:46:18 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 23, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6665

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

The production desktop-inspection service returns successful Evidence with two identical bad�.txt entries when the directory actually contains two distinct POSIX byte filenames. Both entries have the replacement-character file's size and file_id; the other file's identity and size disappear. Coverage is complete-within-scan, with no omissions or issues.

Reproduction

Executed once on an ordinary local filesystem. From a source checkout with its locked dependencies installed, save this as /tmp/rea-desktop-names.mts and run REA_CHECKOUT="$PWD" node --import ./node_modules/tsx/dist/loader.mjs /tmp/rea-desktop-names.mts. Use the Node version stated below. No runtime build or native analysis provider is needed.

import { mkdtemp, writeFile, readdir, lstat, rm } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join, resolve } from 'node:path';
import { pathToFileURL } from 'node:url';
const checkout = process.env.REA_CHECKOUT!;
const { createDesktopApplicationService } = await import(pathToFileURL(resolve(checkout, 'src/composition/desktop.ts')).href);
const root = await mkdtemp(join(tmpdir(), 'rea-filename-repro-'));
try {
  const prefix = Buffer.from(root + '/');
  await writeFile(Buffer.concat([prefix, Buffer.from('626164ff2e747874','hex')]), 'RAW');
  await writeFile(join(root, 'bad\uFFFD.txt'), 'UNICODE');
  const actual = await Promise.all((await readdir(root, {encoding:'buffer'})).map(async name => ({name_hex:name.toString('hex'),bytes:(await lstat(Buffer.concat([prefix,name]))).size})));
  const service = createDesktopApplicationService();
  try {
    const result = await service.inspect({path:root});
    if (!result.ok) console.log(JSON.stringify({actual,ok:false,error:result.error}));
    else {
      const report = result.value.normalized_result;
      console.log(JSON.stringify({actual,ok:true,files:report.files.map(f=>({path:f.path,bytes:f.bytes,file_id:f.file_id,read_status:f.read_status,issues:f.issues})),coverage:report.coverage,omissions:report.omissions}));
    }
  } finally { await service.close(); }
} finally { await rm(root, {recursive:true,force:true}); }

Expected behavior

Preserve the identity and metadata of each observed entry. If a raw filename cannot be represented faithfully, return an explicit limitation or typed failure rather than substitute a different sibling and claim complete coverage. The desktop identity contract binds file_id, local_path, metadata and snapshot identity to observed files. The repository's identity rule requires preserving source identity and uncertainty across boundaries.

Impact and scope

Observed a bounded metadata-integrity failure for two local files: a three-byte file is omitted and a seven-byte sibling is represented twice under one ID. The source files remain unchanged. This probe selected ordinary text files, so it demonstrates incorrect directory metadata and identity, not wrong executable analysis or content decoding. Other platforms, filename patterns, transport adapters and production frequency were not tested. A safe operational workaround was not tested.

REA version

Source revision 5a5bcb6b549cfdfba6dd3f02dd681e505ffd2a26, package version 6.7.0. This main revision is newer than the published package; no packaged-release reproduction is claimed.

Runtime and platform

Node.js v24.22.1; Linux x86_64; locked checkout dependencies. The fixture uses local POSIX byte filenames, with no external service or provider.

Target and artifact context

A temporary directory containing 626164ff2e747874 (bad<FF>.txt, three bytes) and 626164efbfbd2e747874 (bad�.txt, seven bytes). Raw-byte readdir and lstat independently establish the two distinct names and sizes. The script removes its fixture directory.

Interface

Other

Earliest failure stage

Capture or observation

Relevant output or evidence

The executed service probe exited zero and printed the following facts. The identical root-dependent digest is abbreviated here; both original rows contained the same complete file_id.

{
  "actual": [
    {"name_hex":"626164efbfbd2e747874","bytes":7},
    {"name_hex":"626164ff2e747874","bytes":3}
  ],
  "ok": true,
  "files": [
    {"path":"bad�.txt","bytes":7,"file_id":"dsf_<same digest>","read_status":"not-selected","issues":[]},
    {"path":"bad�.txt","bytes":7,"file_id":"dsf_<same digest>","read_status":"not-selected","issues":[]}
  ],
  "coverage": {"status":"complete-within-scan","entries_examined":2,"files_probed":0,"bytes_read":0},
  "omissions": []
}

This executed the service shared by the CLI/MCP workflows, including its output-schema validation and Evidence construction. A complete CLI/MCP transport round trip was not run.

Typed outcome or recovery expected

The same fixture must not produce two successful facts for the replacement-character sibling while silently losing the raw-byte sibling. Preserve distinct identities where supported, or explicitly disclose the unrepresentable entry without claiming complete coverage. A literal, valid U+FFFD filename must remain distinguishable from a filename containing an invalid UTF-8 byte.

Analysis or suspected cause

Source-derived: DirectoryArtifactReader opens the directory with default string decoding, then constructs and stats a path from the decoded child.name. Byte FF becomes U+FFFD, so the subsequent lstat admits the actual U+FFFD sibling's identity. Later file-ID construction receives that already-substituted path and metadata.

Additional context

Related #6653 reports substitution in the separate reference-source reader. This report concerns DirectoryArtifactReader and desktop Evidence; fixing the reference reader alone does not change this independent traversal owner. No source edits, full-suite run or fixed-build verification was performed.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: The executed production-service probe silently omits one file's distinct metadata and duplicates its sibling while reporting complete directory coverage. The observed failure is bounded to filename identity and metadata.
Workaround status: unverified
Workaround basis: No operational workaround was executed. A corrected implementation or explicit unsupported-name outcome remains to be verified by maintainers.

Originally created by @coygeek on GitHub (Oct 23, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6665 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior The production desktop-inspection service returns successful Evidence with two identical `bad�.txt` entries when the directory actually contains two distinct POSIX byte filenames. Both entries have the replacement-character file's size and `file_id`; the other file's identity and size disappear. Coverage is `complete-within-scan`, with no omissions or issues. ### Reproduction Executed once on an ordinary local filesystem. From a source checkout with its locked dependencies installed, save this as `/tmp/rea-desktop-names.mts` and run `REA_CHECKOUT="$PWD" node --import ./node_modules/tsx/dist/loader.mjs /tmp/rea-desktop-names.mts`. Use the Node version stated below. No runtime build or native analysis provider is needed. ```ts import { mkdtemp, writeFile, readdir, lstat, rm } from 'node:fs/promises'; import { tmpdir } from 'node:os'; import { join, resolve } from 'node:path'; import { pathToFileURL } from 'node:url'; const checkout = process.env.REA_CHECKOUT!; const { createDesktopApplicationService } = await import(pathToFileURL(resolve(checkout, 'src/composition/desktop.ts')).href); const root = await mkdtemp(join(tmpdir(), 'rea-filename-repro-')); try { const prefix = Buffer.from(root + '/'); await writeFile(Buffer.concat([prefix, Buffer.from('626164ff2e747874','hex')]), 'RAW'); await writeFile(join(root, 'bad\uFFFD.txt'), 'UNICODE'); const actual = await Promise.all((await readdir(root, {encoding:'buffer'})).map(async name => ({name_hex:name.toString('hex'),bytes:(await lstat(Buffer.concat([prefix,name]))).size}))); const service = createDesktopApplicationService(); try { const result = await service.inspect({path:root}); if (!result.ok) console.log(JSON.stringify({actual,ok:false,error:result.error})); else { const report = result.value.normalized_result; console.log(JSON.stringify({actual,ok:true,files:report.files.map(f=>({path:f.path,bytes:f.bytes,file_id:f.file_id,read_status:f.read_status,issues:f.issues})),coverage:report.coverage,omissions:report.omissions})); } } finally { await service.close(); } } finally { await rm(root, {recursive:true,force:true}); } ``` ### Expected behavior Preserve the identity and metadata of each observed entry. If a raw filename cannot be represented faithfully, return an explicit limitation or typed failure rather than substitute a different sibling and claim complete coverage. The [desktop identity contract](https://github.com/morluto/rea/blob/5a5bcb6b549cfdfba6dd3f02dd681e505ffd2a26/docs/desktop-applications.md#L116-L128) binds `file_id`, `local_path`, metadata and snapshot identity to observed files. The repository's [identity rule](https://github.com/morluto/rea/blob/5a5bcb6b549cfdfba6dd3f02dd681e505ffd2a26/AGENTS.md#L17-L21) requires preserving source identity and uncertainty across boundaries. ### Impact and scope Observed a bounded metadata-integrity failure for two local files: a three-byte file is omitted and a seven-byte sibling is represented twice under one ID. The source files remain unchanged. This probe selected ordinary text files, so it demonstrates incorrect directory metadata and identity, not wrong executable analysis or content decoding. Other platforms, filename patterns, transport adapters and production frequency were not tested. A safe operational workaround was not tested. ### REA version Source revision `5a5bcb6b549cfdfba6dd3f02dd681e505ffd2a26`, package version `6.7.0`. This main revision is newer than the published package; no packaged-release reproduction is claimed. ### Runtime and platform Node.js `v24.22.1`; Linux x86_64; locked checkout dependencies. The fixture uses local POSIX byte filenames, with no external service or provider. ### Target and artifact context A temporary directory containing `626164ff2e747874` (`bad<FF>.txt`, three bytes) and `626164efbfbd2e747874` (`bad�.txt`, seven bytes). Raw-byte `readdir` and `lstat` independently establish the two distinct names and sizes. The script removes its fixture directory. ### Interface Other ### Earliest failure stage Capture or observation ### Relevant output or evidence The executed service probe exited zero and printed the following facts. The identical root-dependent digest is abbreviated here; both original rows contained the same complete `file_id`. ```json { "actual": [ {"name_hex":"626164efbfbd2e747874","bytes":7}, {"name_hex":"626164ff2e747874","bytes":3} ], "ok": true, "files": [ {"path":"bad�.txt","bytes":7,"file_id":"dsf_<same digest>","read_status":"not-selected","issues":[]}, {"path":"bad�.txt","bytes":7,"file_id":"dsf_<same digest>","read_status":"not-selected","issues":[]} ], "coverage": {"status":"complete-within-scan","entries_examined":2,"files_probed":0,"bytes_read":0}, "omissions": [] } ``` This executed the service shared by the CLI/MCP workflows, including its [output-schema validation and Evidence construction](https://github.com/morluto/rea/blob/5a5bcb6b549cfdfba6dd3f02dd681e505ffd2a26/src/application/artifacts/DesktopApplicationService.ts#L36-L60). A complete CLI/MCP transport round trip was not run. ### Typed outcome or recovery expected The same fixture must not produce two successful facts for the replacement-character sibling while silently losing the raw-byte sibling. Preserve distinct identities where supported, or explicitly disclose the unrepresentable entry without claiming complete coverage. A literal, valid U+FFFD filename must remain distinguishable from a filename containing an invalid UTF-8 byte. ### Analysis or suspected cause Source-derived: [DirectoryArtifactReader](https://github.com/morluto/rea/blob/5a5bcb6b549cfdfba6dd3f02dd681e505ffd2a26/src/artifacts/DirectoryArtifactReader.ts#L170-L205) opens the directory with default string decoding, then constructs and stats a path from the decoded `child.name`. Byte `FF` becomes U+FFFD, so the subsequent `lstat` admits the actual U+FFFD sibling's identity. Later file-ID construction receives that already-substituted path and metadata. ### Additional context Related [#6653](https://github.com/morluto/rea/issues/6653) reports substitution in the separate reference-source reader. This report concerns `DirectoryArtifactReader` and desktop Evidence; fixing the reference reader alone does not change this independent traversal owner. No source edits, full-suite run or fixed-build verification was performed. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: The executed production-service probe silently omits one file's distinct metadata and duplicates its sibling while reporting complete directory coverage. The observed failure is bounded to filename identity and metadata. - Workaround status: unverified - Workaround basis: No operational workaround was executed. A corrected implementation or explicit unsupported-name outcome remains to be verified by maintainers.

bsakaguchi closed this issue

2026-10-31 00:46:18 +00:00

bsakaguchi referenced this issue

2026-10-31 01:11:10 +00:00

[PR #4043] fix(setup): write the recommended tool call timeout into dsh registrations #6012

bsakaguchi referenced this issue

2026-11-01 21:11:07 +00:00

[GH-ISSUE #5654] [Tracker] Open PRs that can't merge yet: conflicts and red CI (October 2026) #3425