morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #2652] fix(reference): deep source paths bypass the import I/O result #952

New issue

Closed

opened 2026-10-15 18:29:51 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-15 18:29:51 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 13, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/2652

Summary

Importing an existing local source directory on macOS can reject with native ENAMETOOLONG when traversal assembles a child pathname beyond the filesystem's absolute-path limit. The selected root is valid and readable, and every component is within the filename limit. The importer returns neither its typed I/O failure nor a graph identifying the unreadable entry. Excluding an accessible ancestor lets the same tree import, while deliberately omitting that subtree.

Runtime or environment

Source checkout a2d8b94dd5, tested with Node 24.18.0 and npm 11.16.0 on macOS. The package manifest and supplied latest published package are rea-agents 6.4.0; the published package was not run. The fixture filesystem reported PATH_MAX 1024 and NAME_MAX 255. Verification exercised the source reader and application importer without a build, provider, or installed CLI invocation. Current source recheck at 3ae826ceba found the relevant traversal and error handling unchanged. This recheck used source inspection without a new runtime test.

Steps to reproduce

Use the source checkout with its locked dependencies installed. This is the neutral runnable form of the executed fixed-depth fixture and characterization test. It creates two one-byte files per tree, twelve 90-byte directory components in the failing tree, and two in the short control. Relative directory creation permits the existing tree to exceed the absolute-path limit. This is a bounded observed case, not a claim of globally minimal depth.

fixture="$(mktemp -d)"
export REFERENCE_FIXTURE_ROOT="$fixture"
python3 - "$fixture" <<'PY'
from pathlib import Path
import os, sys
base = Path(sys.argv[1]).resolve()
for kind, depth in [('deep', 12), ('short', 2)]:
    root = base / kind
    root.mkdir()
    (root / 'kept.txt').write_text('k')
    os.chdir(root)
    os.mkdir('nested')
    os.chdir('nested')
    for index in range(depth):
        name = 'd' + str(index).zfill(2) + 'x' * 87
        os.mkdir(name)
        os.chdir(name)
    Path('deep.txt').write_text('d')
    print(kind, depth, len(os.fsencode(os.getcwd() + '/deep.txt')))
    os.chdir(base)
print('PATH_MAX', os.pathconf(base, 'PC_PATH_MAX'))
print('NAME_MAX', os.pathconf(base, 'PC_NAME_MAX'))
PY

Save this test at an unused src/reference/referencePathLength.test.ts path. It characterizes the current raw errors and verifies the short-path and same-tree exclusion controls:

import { it, expect } from "vitest";
import { lstat, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js";
import { readReferenceSource } from "./ReferenceSourceReader.js";
import { importReferenceSource } from "../application/ReferenceSourceImport.js";

const base = process.env.REFERENCE_FIXTURE_ROOT;
if (!base) throw new Error("Set REFERENCE_FIXTURE_ROOT to the fixture directory");
async function owned<T>(operation: (scope: ArtifactResourceScope) => Promise<T>) {
  const scope = new ArtifactResourceScope();
  try { return await operation(scope); }
  finally { await scope.close(); }
}
const importTree = (root: string) => owned(scope => importReferenceSource({
  root, caller: "reference-path-check", policy: { secretPatterns: [] },
}, scope));

it("checks the native deep-path result and short/excluded controls", async () => {
  console.log("runtime", process.version, process.platform);
  const root = join(base, "deep");
  expect((await lstat(root)).isDirectory()).toBe(true);
  for (const [label, call] of [
    ["reader", () => owned(scope => readReferenceSource(root, scope))],
    ["import", () => importTree(root)],
  ] as const) {
    let rejection: unknown;
    try { await call(); } catch (cause) { rejection = cause; }
    expect(rejection).toBeInstanceOf(Error);
    const failure = rejection as NodeJS.ErrnoException;
    expect(failure.code).toBe("ENAMETOOLONG");
    expect(failure.syscall).toBe("lstat");
    console.log(label, JSON.stringify({code: failure.code, syscall: failure.syscall,
      pathBytes: Buffer.byteLength(failure.path ?? ""), typedTag: Reflect.get(failure, "tag") ?? null}));
  }
  const short = await importTree(join(base, "short"));
  expect(short.ok).toBe(true);
  if (!short.ok) throw short.error;
  expect(short.value.entries).toContainEqual(expect.objectContaining({
    kind: "file", path: "kept.txt", content_state: "hashed",
  }));
  console.log("short", JSON.stringify({ok: short.ok, entries: short.value.entries.length}));
  await writeFile(join(root, ".gitignore"), "nested/\n");
  const excluded = await importTree(root);
  expect(excluded.ok).toBe(true);
  if (!excluded.ok) throw excluded.error;
  expect(excluded.value.entries).toContainEqual(expect.objectContaining({
    kind: "file", path: "kept.txt", content_state: "hashed",
  }));
  expect(excluded.value.exclusions).toContainEqual(expect.objectContaining({
    path: "nested", reason: "project-ignored", pattern: "nested/",
  }));
  console.log("excluded", JSON.stringify({ok: excluded.ok, exclusions: excluded.value.exclusions}));
});

npm run test:local -- src/reference/referencePathLength.test.ts

The local run used the focused source runner on the reader and importer. The installed command was not executed. Remove the temporary test and owned fixture after use. Relative cleanup avoids opening the overlong absolute child path:

python3 - "$fixture" <<'PY'
import os, sys
root = os.path.realpath(sys.argv[1])
os.chdir(root)
def remove_children():
    for name in os.listdir('.'):
        if os.path.isdir(name):
            os.chdir(name)
            remove_children()
            os.chdir('..')
            os.rmdir(name)
        else:
            os.unlink(name)
remove_children()
os.chdir(os.path.dirname(root))
os.rmdir(os.path.basename(root))
PY
rm src/reference/referencePathLength.test.ts
unset REFERENCE_FIXTURE_ROOT

Expected behavior

The selected directory is an admitted input to the documented historical-source import. An OS failure inspecting an entry should use the existing I/O representation. The reader models failed metadata as an entry with code io, and the application exposes a typed ReferenceSourceImportError with an execution_failure projection. This does not require successful access beyond the OS limit. It requires the ordinary filesystem failure to remain an explicit result rather than escape as a native rejection.

Preserve the actionable cause and any retained partial observations. Either an explicit unreadable entry in a partial graph or an applicable typed I/O failure can represent the failure without asserting successful coverage.

Actual behavior

The reader and importer both rejected with native ENAMETOOLONG from lstat. Neither error carried a typed tag. In the executed fixture the first failing pathname was 1088 bytes and the deepest file pathname was 1188 bytes. The selected root resolved normally. The short control imported five entries. Adding nested/ to the deep fixture's root .gitignore made that same tree import with the readable sibling file retained and a project exclusion recorded.

runtime v24.18.0 darwin
reader {"code":"ENAMETOOLONG","syscall":"lstat","pathBytes":1088,"typedTag":null}
import {"code":"ENAMETOOLONG","syscall":"lstat","pathBytes":1088,"typedTag":null}
short {"ok":true,"entries":5}
excluded {"ok":true,"exclusions":[{"path":"nested","reason":"project-ignored","pattern":"nested/"}]}

The characterization test passed once, asserting both defect observations and both controls; its body took 30 ms. The bounded chains and both file payloads were inspected afterward and remained unchanged. Only the fixture's ignore file was intentionally added for the control. The test's pass confirms the defect, not a correction.

Source inspection establishes that the native import rejection bypasses the CLI's handled import-error result. No installed CLI output, process exit status, or published-runtime reproduction is claimed.

Evidence

Expected source: docs/cli.md:209 documents the historical-source import, registered at src/cli/utilityCommands.ts:210; AGENTS.md, Architecture and Cleanup, requires expected boundary failures to use the tagged error algebra and Result; src/application/ReferenceSourceImportTypes.ts:5 identifies the typed expected failure, with io projection at line 31; src/reference/ReferenceSourceReaderEntries.ts:203 represents failed entry metadata as io.
Failure source: The executed native fixture and source test above produced both ENAMETOOLONG rejections. Relevant error-handling boundaries are src/reference/ReferenceSourceReaderErrors.ts:24, src/reference/ReferenceSourceReaderEntries.ts:174, src/reference/ReferenceSourceReader.ts:29 and src/application/ReferenceSourceImport.ts:194. The source-derived CLI outcome is located at src/cli/utilityCommands.ts:275 and src/cliLogging.ts:100.
Evidence provenance: observed
Local verification: reproduced
Reproduction completeness: complete

The fixture has no symlinks or cancellation, and its directory names match no default exclusions. Each resource scope was open, and cleanup succeeded. The test uses no error mocks and executes no files from the imported tree.

Related: #2551 covers an overlong selected root failing during admission. This report covers child-path failure after an existing readable root has been admitted.

Restoration check

With the same unexcluded deep tree, both source entry points settle to an explicit existing I/O representation instead of rejecting with native ENAMETOOLONG. Any graph identifies unreadable coverage and retains readable sibling observations; a typed operation failure preserves the OS reason and applicable partial observations. Do not silently skip the subtree or report it fully enumerated.

Keep the short control importing its files normally, and the root .gitignore control succeeding with the project exclusion recorded. Verify source bytes remain unchanged. When an operation-level I/O failure is returned, the documented CLI route should use its existing handled import-failure representation. No particular implementation is prescribed.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: An admitted source tree produces an untyped import rejection and bypasses the handled I/O outcome. The observed failure is contained to that import and did not modify source bytes. The bounded sample establishes the pathname condition, not its prevalence in projects.
Workaround status: available
Workaround basis: A root .gitignore entry nested/ was observed to let the same fixture import while retaining its readable sibling file. It deliberately omits the subtree and is practical when that subtree is unnecessary for the reference inventory. Imports requiring that subtree still need an explicit represented failure.

Originally created by @coygeek on GitHub (Oct 13, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/2652 ## Summary Importing an existing local source directory on macOS can reject with native ENAMETOOLONG when traversal assembles a child pathname beyond the filesystem's absolute-path limit. The selected root is valid and readable, and every component is within the filename limit. The importer returns neither its typed I/O failure nor a graph identifying the unreadable entry. Excluding an accessible ancestor lets the same tree import, while deliberately omitting that subtree. ## Runtime or environment Source checkout a2d8b94dd50dab8d966239dd63ab84718aa8b4b4, tested with Node 24.18.0 and npm 11.16.0 on macOS. The package manifest and supplied latest published package are rea-agents 6.4.0; the published package was not run. The fixture filesystem reported PATH_MAX 1024 and NAME_MAX 255. Verification exercised the source reader and application importer without a build, provider, or installed CLI invocation. Current source recheck at 3ae826ceba6f25b08bce50bcb51001209bc16b57 found the relevant traversal and error handling unchanged. This recheck used source inspection without a new runtime test. ## Steps to reproduce Use the source checkout with its locked dependencies installed. This is the neutral runnable form of the executed fixed-depth fixture and characterization test. It creates two one-byte files per tree, twelve 90-byte directory components in the failing tree, and two in the short control. Relative directory creation permits the existing tree to exceed the absolute-path limit. This is a bounded observed case, not a claim of globally minimal depth. ```bash fixture="$(mktemp -d)" export REFERENCE_FIXTURE_ROOT="$fixture" python3 - "$fixture" <<'PY' from pathlib import Path import os, sys base = Path(sys.argv[1]).resolve() for kind, depth in [('deep', 12), ('short', 2)]: root = base / kind root.mkdir() (root / 'kept.txt').write_text('k') os.chdir(root) os.mkdir('nested') os.chdir('nested') for index in range(depth): name = 'd' + str(index).zfill(2) + 'x' * 87 os.mkdir(name) os.chdir(name) Path('deep.txt').write_text('d') print(kind, depth, len(os.fsencode(os.getcwd() + '/deep.txt'))) os.chdir(base) print('PATH_MAX', os.pathconf(base, 'PC_PATH_MAX')) print('NAME_MAX', os.pathconf(base, 'PC_NAME_MAX')) PY ``` Save this test at an unused `src/reference/referencePathLength.test.ts` path. It characterizes the current raw errors and verifies the short-path and same-tree exclusion controls: ```typescript import { it, expect } from "vitest"; import { lstat, writeFile } from "node:fs/promises"; import { join } from "node:path"; import { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js"; import { readReferenceSource } from "./ReferenceSourceReader.js"; import { importReferenceSource } from "../application/ReferenceSourceImport.js"; const base = process.env.REFERENCE_FIXTURE_ROOT; if (!base) throw new Error("Set REFERENCE_FIXTURE_ROOT to the fixture directory"); async function owned<T>(operation: (scope: ArtifactResourceScope) => Promise<T>) { const scope = new ArtifactResourceScope(); try { return await operation(scope); } finally { await scope.close(); } } const importTree = (root: string) => owned(scope => importReferenceSource({ root, caller: "reference-path-check", policy: { secretPatterns: [] }, }, scope)); it("checks the native deep-path result and short/excluded controls", async () => { console.log("runtime", process.version, process.platform); const root = join(base, "deep"); expect((await lstat(root)).isDirectory()).toBe(true); for (const [label, call] of [ ["reader", () => owned(scope => readReferenceSource(root, scope))], ["import", () => importTree(root)], ] as const) { let rejection: unknown; try { await call(); } catch (cause) { rejection = cause; } expect(rejection).toBeInstanceOf(Error); const failure = rejection as NodeJS.ErrnoException; expect(failure.code).toBe("ENAMETOOLONG"); expect(failure.syscall).toBe("lstat"); console.log(label, JSON.stringify({code: failure.code, syscall: failure.syscall, pathBytes: Buffer.byteLength(failure.path ?? ""), typedTag: Reflect.get(failure, "tag") ?? null})); } const short = await importTree(join(base, "short")); expect(short.ok).toBe(true); if (!short.ok) throw short.error; expect(short.value.entries).toContainEqual(expect.objectContaining({ kind: "file", path: "kept.txt", content_state: "hashed", })); console.log("short", JSON.stringify({ok: short.ok, entries: short.value.entries.length})); await writeFile(join(root, ".gitignore"), "nested/\n"); const excluded = await importTree(root); expect(excluded.ok).toBe(true); if (!excluded.ok) throw excluded.error; expect(excluded.value.entries).toContainEqual(expect.objectContaining({ kind: "file", path: "kept.txt", content_state: "hashed", })); expect(excluded.value.exclusions).toContainEqual(expect.objectContaining({ path: "nested", reason: "project-ignored", pattern: "nested/", })); console.log("excluded", JSON.stringify({ok: excluded.ok, exclusions: excluded.value.exclusions})); }); ``` ```bash npm run test:local -- src/reference/referencePathLength.test.ts ``` The local run used the focused source runner on the reader and importer. The installed command was not executed. Remove the temporary test and owned fixture after use. Relative cleanup avoids opening the overlong absolute child path: ```bash python3 - "$fixture" <<'PY' import os, sys root = os.path.realpath(sys.argv[1]) os.chdir(root) def remove_children(): for name in os.listdir('.'): if os.path.isdir(name): os.chdir(name) remove_children() os.chdir('..') os.rmdir(name) else: os.unlink(name) remove_children() os.chdir(os.path.dirname(root)) os.rmdir(os.path.basename(root)) PY rm src/reference/referencePathLength.test.ts unset REFERENCE_FIXTURE_ROOT ``` ## Expected behavior The selected directory is an admitted input to the documented historical-source import. An OS failure inspecting an entry should use the existing I/O representation. The reader models failed metadata as an entry with code io, and the application exposes a typed ReferenceSourceImportError with an execution_failure projection. This does not require successful access beyond the OS limit. It requires the ordinary filesystem failure to remain an explicit result rather than escape as a native rejection. Preserve the actionable cause and any retained partial observations. Either an explicit unreadable entry in a partial graph or an applicable typed I/O failure can represent the failure without asserting successful coverage. ## Actual behavior The reader and importer both rejected with native ENAMETOOLONG from lstat. Neither error carried a typed tag. In the executed fixture the first failing pathname was 1088 bytes and the deepest file pathname was 1188 bytes. The selected root resolved normally. The short control imported five entries. Adding nested/ to the deep fixture's root .gitignore made that same tree import with the readable sibling file retained and a project exclusion recorded. ```text runtime v24.18.0 darwin reader {"code":"ENAMETOOLONG","syscall":"lstat","pathBytes":1088,"typedTag":null} import {"code":"ENAMETOOLONG","syscall":"lstat","pathBytes":1088,"typedTag":null} short {"ok":true,"entries":5} excluded {"ok":true,"exclusions":[{"path":"nested","reason":"project-ignored","pattern":"nested/"}]} ``` The characterization test passed once, asserting both defect observations and both controls; its body took 30 ms. The bounded chains and both file payloads were inspected afterward and remained unchanged. Only the fixture's ignore file was intentionally added for the control. The test's pass confirms the defect, not a correction. Source inspection establishes that the native import rejection bypasses the CLI's handled import-error result. No installed CLI output, process exit status, or published-runtime reproduction is claimed. ## Evidence - Expected source: `docs/cli.md:209` documents the historical-source import, registered at `src/cli/utilityCommands.ts:210`; `AGENTS.md`, Architecture and Cleanup, requires expected boundary failures to use the tagged error algebra and Result; `src/application/ReferenceSourceImportTypes.ts:5` identifies the typed expected failure, with io projection at line 31; `src/reference/ReferenceSourceReaderEntries.ts:203` represents failed entry metadata as io. - Failure source: The executed native fixture and source test above produced both ENAMETOOLONG rejections. Relevant error-handling boundaries are `src/reference/ReferenceSourceReaderErrors.ts:24`, `src/reference/ReferenceSourceReaderEntries.ts:174`, `src/reference/ReferenceSourceReader.ts:29` and `src/application/ReferenceSourceImport.ts:194`. The source-derived CLI outcome is located at `src/cli/utilityCommands.ts:275` and `src/cliLogging.ts:100`. - Evidence provenance: observed - Local verification: reproduced - Reproduction completeness: complete The fixture has no symlinks or cancellation, and its directory names match no default exclusions. Each resource scope was open, and cleanup succeeded. The test uses no error mocks and executes no files from the imported tree. Related: #2551 covers an overlong selected root failing during admission. This report covers child-path failure after an existing readable root has been admitted. ## Restoration check With the same unexcluded deep tree, both source entry points settle to an explicit existing I/O representation instead of rejecting with native ENAMETOOLONG. Any graph identifies unreadable coverage and retains readable sibling observations; a typed operation failure preserves the OS reason and applicable partial observations. Do not silently skip the subtree or report it fully enumerated. Keep the short control importing its files normally, and the root .gitignore control succeeding with the project exclusion recorded. Verify source bytes remain unchanged. When an operation-level I/O failure is returned, the documented CLI route should use its existing handled import-failure representation. No particular implementation is prescribed. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: An admitted source tree produces an untyped import rejection and bypasses the handled I/O outcome. The observed failure is contained to that import and did not modify source bytes. The bounded sample establishes the pathname condition, not its prevalence in projects. - Workaround status: available - Workaround basis: A root .gitignore entry nested/ was observed to let the same fixture import while retaining its readable sibling file. It deliberately omits the subtree and is practical when that subtree is unnecessary for the reference inventory. Imports requiring that subtree still need an explicit represented failure.

bsakaguchi

2026-10-15 18:29:51 +00:00

closed this issue
added the
area: domain

bug
labels

bsakaguchi referenced this issue

2026-10-17 21:07:11 +00:00

[PR #952] [MERGED] refactor(apple): group artifact producers and verification #2528

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#952

No description provided.

Rows
Columns