morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #6765] [Bug] Historical-source symlink targets alias replacement-character filenames after lossy UTF-8 decoding #4122

New issue

Closed

opened 2026-10-31 00:46:58 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:46:58 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 23, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6765

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

import-reference-source records alias.ts as an internal symlink to bad�.ts, although its target's actual basename bytes are 626164ff2e7473 (bad<FF>.ts). A different existing file has the UTF-8 basename 626164efbfbd2e7473 (bad�.ts). The import substitutes that second filename in the symlink's retained identity.

The CLI exits 0. Root enumeration correctly reports that the raw-byte filename cannot be represented as UTF-8, but the alias entry still has target_state: "internal" and limitations: [], identifying a different file.

Reproduction

One generated Linux fixture was executed through the built public CLI. From a prepared source checkout (npm ci, npm run build:cached), the probe created the following inputs with Node's filesystem API:

const raw = Buffer.from([0x62,0x61,0x64,0xff,0x2e,0x74,0x73]);
await writeFile(Buffer.concat([Buffer.from(root + "/"), raw]),
  "export const raw = 1;\n");
await writeFile(join(root, "bad\ufffd.ts"), "export const unicode = 1;\n");
await symlink(raw, join(root, "alias.ts"));

Here root is a freshly created temporary directory, and join, writeFile, and symlink are the ordinary node:path / node:fs/promises APIs. The probe ran the equivalent public invocation below, with isolated HOME and XDG configuration directories:

node scripts/rea.mjs import-reference-source "$ROOT" --json

It also read the target with readlink(alias, {encoding:"buffer"}) and resolved it with realpath(alias, {encoding:"buffer"}). Both preserve the raw FF byte. Compare those values with the graph entry for alias.ts. No source code in the fixture is executed.

Expected behavior

Keep the target's filesystem identity, or mark the target unsupported/unknown with an explicit limitation when a text field cannot represent its bytes. Do not report the separate replacement-character filename as a resolved target. The repository's input/evidence preservation contract requires preserving paths and keeping identity separate from display formatting. The historical-source CLI guidance describes explicit unsupported-name coverage without replacement-name access; the existing raw-filename regression test protects directory enumeration.

Impact and scope

The observed historical-source graph falsely identifies another existing file as the alias target. The affected directory already carries an unsupported-name limitation, but that limitation does not qualify or correct this resolved alias entry. This is a bounded evidence-integrity failure with non-UTF-8 POSIX target names; Linux was tested. Other hosts, missing replacement siblings, downstream comparisons, and installed npm-package execution were not tested.

REA version

Package 6.7.0; source revision 2bcd7e6d13f0a8118d6619718254255e84f75d86. The latest published npm version was checked before investigation.

Runtime and platform

Node.js v26.12.1, Linux x64, repository-locked dependencies and a successfully built source CLI. No native analysis provider was involved.

Target and artifact context

Two self-authored local TypeScript files with distinct raw basename bytes and different ASCII contents, plus one local symlink named alias.ts. Every path stays inside the same fixture. The selected root is canonical and no path contains parent components.

Interface

CLI

Earliest failure stage

Capture or observation

Relevant output or evidence

The command exited 0 with empty stderr. Observed target controls:

{"readlink_hex":"626164ff2e7473","realpath_basename_hex":"626164ff2e7473"}

Relevant graph entry:

{"path":"alias.ts","kind":"symlink","target":"bad�.ts","target_state":"internal","limitations":[]}

The graph has inventory_state: "partial" and a root limitation reporting exactly one non-UTF-8 basename, with raw hex 626164ff2e7473. This shows that directory enumeration rejects the unrepresentable filename while the symlink observation still substitutes its replacement-character sibling.

Expected source: the preservation contract linked above and tests/boundary/filesystem/referenceSourceFilenameBytes.test.ts, which protects raw directory-entry identity.
Failure source: the executed public CLI output and buffer-preserving filesystem controls; target observation reads the link target using default string decoding before resolving and projecting it.
Evidence provenance: observed.
Local verification: reproduced.
Reproduction completeness: complete for the retained target-identity failure.

Typed outcome or recovery expected

An unrepresentable target should not become a different resolved internal path. Restoration check: the raw-byte alias is represented truthfully or explicitly unsupported/unknown, the ordinary bad�.ts file remains independently inventoried, and the root still records its unsupported raw-name coverage. Do not hide the problem by dropping the valid alias entry or treating its substituted text as identity.

Analysis or suspected cause

Source-derived: readlink(absolute) returns a replacement-decoded string. The subsequent path lookup uses those substituted bytes and finds the valid Unicode sibling. The graph projection then persists that different target without a per-entry limitation. Filesystem identity checks around the symlink do not restore the lost target bytes.

Additional context

Residual follow-up to closed #6653: the current checkout includes its byte-preserving directory enumeration fix and regression test. This report concerns the still-separate symlink-target observation, and the complete built CLI graph demonstrates its contrary result. Existing PR #3562 concerns parent-component resolution; this fixture has no parent components, and its target string-decoding boundary is independent. First affected release, real-world frequency, and a result-preserving REA workaround are unknown.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: The executed CLI commits a resolved internal symlink target naming a different existing file; the observed harm is bounded incorrect historical-source evidence.
Workaround status: unverified
Workaround basis: Renaming source files or rewriting symlink targets changes the recorded input identity, and no equivalent REA import preserving those original identities was demonstrated.

Originally created by @coygeek on GitHub (Oct 23, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6765 ## Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ## Area Evidence, artifacts, or reconstruction ## Observed behavior `import-reference-source` records `alias.ts` as an internal symlink to `bad�.ts`, although its target's actual basename bytes are `626164ff2e7473` (`bad<FF>.ts`). A different existing file has the UTF-8 basename `626164efbfbd2e7473` (`bad�.ts`). The import substitutes that second filename in the symlink's retained identity. The CLI exits 0. Root enumeration correctly reports that the raw-byte filename cannot be represented as UTF-8, but the alias entry still has `target_state: "internal"` and `limitations: []`, identifying a different file. ## Reproduction One generated Linux fixture was executed through the built public CLI. From a prepared source checkout (`npm ci`, `npm run build:cached`), the probe created the following inputs with Node's filesystem API: ```javascript const raw = Buffer.from([0x62,0x61,0x64,0xff,0x2e,0x74,0x73]); await writeFile(Buffer.concat([Buffer.from(root + "/"), raw]), "export const raw = 1;\n"); await writeFile(join(root, "bad\ufffd.ts"), "export const unicode = 1;\n"); await symlink(raw, join(root, "alias.ts")); ``` Here `root` is a freshly created temporary directory, and `join`, `writeFile`, and `symlink` are the ordinary `node:path` / `node:fs/promises` APIs. The probe ran the equivalent public invocation below, with isolated HOME and XDG configuration directories: ```sh node scripts/rea.mjs import-reference-source "$ROOT" --json ``` It also read the target with `readlink(alias, {encoding:"buffer"})` and resolved it with `realpath(alias, {encoding:"buffer"})`. Both preserve the raw `FF` byte. Compare those values with the graph entry for `alias.ts`. No source code in the fixture is executed. ## Expected behavior Keep the target's filesystem identity, or mark the target unsupported/unknown with an explicit limitation when a text field cannot represent its bytes. Do not report the separate replacement-character filename as a resolved target. The repository's [input/evidence preservation contract](https://github.com/morluto/rea/blob/2bcd7e6d13f0a8118d6619718254255e84f75d86/AGENTS.md#L13-L21) requires preserving paths and keeping identity separate from display formatting. The [historical-source CLI guidance](https://github.com/morluto/rea/blob/2bcd7e6d13f0a8118d6619718254255e84f75d86/docs/cli.md#L397-L404) describes explicit unsupported-name coverage without replacement-name access; the existing raw-filename regression test protects directory enumeration. ## Impact and scope The observed historical-source graph falsely identifies another existing file as the alias target. The affected directory already carries an unsupported-name limitation, but that limitation does not qualify or correct this resolved alias entry. This is a bounded evidence-integrity failure with non-UTF-8 POSIX target names; Linux was tested. Other hosts, missing replacement siblings, downstream comparisons, and installed npm-package execution were not tested. ## REA version Package 6.7.0; source revision `2bcd7e6d13f0a8118d6619718254255e84f75d86`. The latest published npm version was checked before investigation. ## Runtime and platform Node.js v26.12.1, Linux x64, repository-locked dependencies and a successfully built source CLI. No native analysis provider was involved. ## Target and artifact context Two self-authored local TypeScript files with distinct raw basename bytes and different ASCII contents, plus one local symlink named `alias.ts`. Every path stays inside the same fixture. The selected root is canonical and no path contains parent components. ## Interface CLI ## Earliest failure stage Capture or observation ## Relevant output or evidence The command exited 0 with empty stderr. Observed target controls: ```json {"readlink_hex":"626164ff2e7473","realpath_basename_hex":"626164ff2e7473"} ``` Relevant graph entry: ```json {"path":"alias.ts","kind":"symlink","target":"bad�.ts","target_state":"internal","limitations":[]} ``` The graph has `inventory_state: "partial"` and a root limitation reporting exactly one non-UTF-8 basename, with raw hex `626164ff2e7473`. This shows that directory enumeration rejects the unrepresentable filename while the symlink observation still substitutes its replacement-character sibling. - Expected source: the preservation contract linked above and `tests/boundary/filesystem/referenceSourceFilenameBytes.test.ts`, which protects raw directory-entry identity. - Failure source: the executed public CLI output and buffer-preserving filesystem controls; [target observation](https://github.com/morluto/rea/blob/2bcd7e6d13f0a8118d6619718254255e84f75d86/src/reference/ReferenceSourceReaderEntries.ts#L375-L421) reads the link target using default string decoding before resolving and projecting it. - Evidence provenance: observed. - Local verification: reproduced. - Reproduction completeness: complete for the retained target-identity failure. ## Typed outcome or recovery expected An unrepresentable target should not become a different resolved internal path. Restoration check: the raw-byte alias is represented truthfully or explicitly unsupported/unknown, the ordinary `bad�.ts` file remains independently inventoried, and the root still records its unsupported raw-name coverage. Do not hide the problem by dropping the valid alias entry or treating its substituted text as identity. ## Analysis or suspected cause Source-derived: `readlink(absolute)` returns a replacement-decoded string. The subsequent path lookup uses those substituted bytes and finds the valid Unicode sibling. The [graph projection](https://github.com/morluto/rea/blob/2bcd7e6d13f0a8118d6619718254255e84f75d86/src/application/ReferenceSourceImportEntries.ts#L401-L415) then persists that different target without a per-entry limitation. Filesystem identity checks around the symlink do not restore the lost target bytes. ## Additional context Residual follow-up to closed [#6653](https://github.com/morluto/rea/issues/6653): the current checkout includes its byte-preserving directory enumeration fix and regression test. This report concerns the still-separate symlink-target observation, and the complete built CLI graph demonstrates its contrary result. Existing [PR #3562](https://github.com/morluto/rea/pull/3562) concerns parent-component resolution; this fixture has no parent components, and its target string-decoding boundary is independent. First affected release, real-world frequency, and a result-preserving REA workaround are unknown. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: The executed CLI commits a resolved internal symlink target naming a different existing file; the observed harm is bounded incorrect historical-source evidence. - Workaround status: unverified - Workaround basis: Renaming source files or rewriting symlink targets changes the recorded input identity, and no equivalent REA import preserving those original identities was demonstrated.

bsakaguchi closed this issue

2026-10-31 00:46:58 +00:00

bsakaguchi referenced this issue

2026-10-31 01:11:39 +00:00

[PR #4122] [CLOSED] test(kuna): wait for cold concurrent publication beyond the poll default #6073

bsakaguchi referenced this issue

2026-10-31 01:12:32 +00:00

[PR #4343] [MERGED] fix(kuna): name the setting that owns a timed-out Kuna run #6188