morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #6736] [Bug] PyInstaller inventory collapses distinct POSIX backslash and slash paths #4099

New issue

Closed

opened 2026-10-31 00:46:49 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:46:49 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 23, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6736

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

On a real Linux onefile executable built with PyInstaller 6.22.3, inspect-artifact maps both the literal filename a\b.txt and the separate destination a/b.txt to logical path a/b.txt. Both occurrences retain distinct source names, IDs and verified content, but report no limitation and complete inventory coverage. extract-artifact then exits 1 with artifact_operation_failed because those projected paths collide.

PyInstaller's own CArchiveReader returns both original names and their different payloads from the unchanged executable. The executable was built for this check and never launched.

Reproduction

Executed from the requested source checkout using the runtimes below, with paths parameterized here. Locked dependencies and dist were already built. python3.12 denotes the tested CPython 3.12.15 interpreter and node the tested Node 24.22.1 executable.

work=$(mktemp -d /tmp/rea-backslash.XXXXXX)
mkdir -p "$work/input/a"
python3.12 - "$work" <<'PY'
from pathlib import Path
import sys
p = Path(sys.argv[1]) / 'input'
(p / 'main.py').write_text('pass\n')
(p / 'a\\b.txt').write_text('literal backslash\n')
(p / 'a' / 'b.txt').write_text('nested slash\n')
PY
uv pip install --python python3.12 --target "$work/site" 'pyinstaller==6.22.3'
PYTHONPATH="$work/site" PYINSTALLER_CONFIG_DIR="$work/config" \
python3.12 -m PyInstaller --onefile --noconfirm --log-level ERROR \
  --name backslash-demo --distpath "$work/dist" --workpath "$work/build" \
  --specpath "$work/input" \
  --add-data "$work/input/a\b.txt:." --add-data "$work/input/a/b.txt:a" \
  "$work/input/main.py"
node scripts/rea.mjs inspect-artifact "$work/dist/backslash-demo" --json
node scripts/rea.mjs extract-artifact "$work/dist/backslash-demo" --json

The producer build succeeded. Inspection and extraction each ran once, yielding the results described above. An independent read used CArchiveReader on that executable and called extract(r'a\b.txt') and extract('a/b.txt'); both succeeded with the payloads below.

Expected behavior

Preserve the distinct POSIX destination names in inventory and permit both files to be extracted on Linux. A literal backslash is part of a POSIX filename, while / separates components.

The released PyInstaller writer uses the producer OS's path normalization and applies Windows separator conversion only on Windows. Its bootloader extraction path uses each data row's stored name. REA documents inventory and extraction of CArchive members using archive path rules. The verified producer TOC and returned bytes establish that these are separate valid entries.

Impact and scope

This bounded namespace error makes a valid Linux artifact appear to contain conflicting destinations and blocks its extraction through REA. The inventory preserves both content identities and original source names, so content loss is not claimed. No target modification occurred. Frequency outside this fixture is unknown; Windows, macOS and MCP were not exercised.

A demonstrated external workaround is PyInstaller's CArchiveReader, which preserves these names and returns both payloads for this artifact when the matching Python environment is available.

REA version

6.7.0; source revision 3c5c887a2cc7371a94fe0fd8b6c478dbf2af60db (main checkout; the published npm artifact was not separately executed).

Runtime and platform

Node.js v24.22.1; CPython 3.12.15; PyInstaller 6.22.3 with hooks-contrib 2026.9; Linux 7.2.8-9-omarchy x86_64, glibc 2.44. No native analysis engine was required. Current PyPI metadata identified PyInstaller 6.22.3 before its scratch-only installation.

Target and artifact context

Self-authored 12,050,976-byte Linux ELF onefile executable containing an appended CArchive, a pass entry point and the two ordinary text resources. Carrier SHA-256: 2d0176146e91d2ea36b7cf8a77e2c61392d283da62705d9e2817c49cff64e970. Byte-for-byte reproducibility of the producer build across environments was not tested.

Interface

CLI

Earliest failure stage

Artifact or evidence registration

Relevant output or evidence

Observed producer-reader output, with Python string escaping:

'a\\b.txt' -> b'literal backslash\n'
'a/b.txt'  -> b'nested slash\n'

Projection of the observed successful CLI inventory:

{
  "coverage": {"scope":"container-entries","status":"complete"},
  "occurrences": [
    {"logical_path":"a/b.txt","source_name":"a/b.txt","declared_size":13,"hash_status":"verified","limitations":[]},
    {"logical_path":"a/b.txt","source_name":"a\\b.txt","declared_size":18,"hash_status":"verified","limitations":[]}
  ]
}

Observed extraction error prefix:

Artifact extraction path collision: "a/b.txt". Multiple selected occurrences share this normalized logical path; they cannot be materialized at the same destination path, even when the archive is valid.

Existing contract: retain valid producer member identities and use the archive's path semantics, as linked above.
Failure provenance: real producer build, independent producer-reader bytes, and public REA CLI output; no mocks.
Local verification: reproduced.
Reproduction completeness: complete for the Linux fixture; target execution was neither needed nor performed.

Typed outcome or recovery expected

Return distinct logical destinations and successful extraction for both resources on POSIX. Preserve Windows archive separator handling. If producer path semantics cannot be established, expose that uncertainty instead of silently assigning a false shared destination.

Analysis or suspected cause

Source-derived and consistent with the observed output: projectStoredMember passes every source name through canonicalMemberPath, which unconditionally replaces backslashes with slashes. That interpretation fits Windows producer paths but changes the meaning of the observed POSIX filename. No repair was implemented.

Additional context

No known-good REA revision was established. Open and closed searches for PyInstaller backslashes, CArchive paths, canonicalMemberPath, and PyInstaller collisions found no matching report. #6539 concerns a PYZ namespace directory colliding with a data destination; this fixture instead has two ordinary data members whose source names are changed by separator conversion. All runtime and source verification was completed before writing this draft; no reproduction was rerun afterward.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: Public CLI inventory assigns a false shared path to two valid producer entries, and public CLI extraction fails because of that projection. The defect is bounded to affected path spellings; source names and both content identities remain available.
Workaround status: available
Workaround basis: On the demonstrated artifact, PyInstaller's CArchiveReader retained both original names and returned both distinct text payloads using the tested matching Python environment. This bypasses REA's path projection without modifying the carrier.

Originally created by @coygeek on GitHub (Oct 23, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6736 ## Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ## Area Evidence, artifacts, or reconstruction ## Observed behavior On a real Linux onefile executable built with PyInstaller 6.22.3, `inspect-artifact` maps both the literal filename `a\b.txt` and the separate destination `a/b.txt` to logical path `a/b.txt`. Both occurrences retain distinct source names, IDs and verified content, but report no limitation and complete inventory coverage. `extract-artifact` then exits 1 with `artifact_operation_failed` because those projected paths collide. PyInstaller's own `CArchiveReader` returns both original names and their different payloads from the unchanged executable. The executable was built for this check and never launched. ## Reproduction Executed from the requested source checkout using the runtimes below, with paths parameterized here. Locked dependencies and `dist` were already built. `python3.12` denotes the tested CPython 3.12.15 interpreter and `node` the tested Node 24.22.1 executable. ```sh work=$(mktemp -d /tmp/rea-backslash.XXXXXX) mkdir -p "$work/input/a" python3.12 - "$work" <<'PY' from pathlib import Path import sys p = Path(sys.argv[1]) / 'input' (p / 'main.py').write_text('pass\n') (p / 'a\\b.txt').write_text('literal backslash\n') (p / 'a' / 'b.txt').write_text('nested slash\n') PY uv pip install --python python3.12 --target "$work/site" 'pyinstaller==6.22.3' PYTHONPATH="$work/site" PYINSTALLER_CONFIG_DIR="$work/config" \ python3.12 -m PyInstaller --onefile --noconfirm --log-level ERROR \ --name backslash-demo --distpath "$work/dist" --workpath "$work/build" \ --specpath "$work/input" \ --add-data "$work/input/a\b.txt:." --add-data "$work/input/a/b.txt:a" \ "$work/input/main.py" node scripts/rea.mjs inspect-artifact "$work/dist/backslash-demo" --json node scripts/rea.mjs extract-artifact "$work/dist/backslash-demo" --json ``` The producer build succeeded. Inspection and extraction each ran once, yielding the results described above. An independent read used `CArchiveReader` on that executable and called `extract(r'a\b.txt')` and `extract('a/b.txt')`; both succeeded with the payloads below. ## Expected behavior Preserve the distinct POSIX destination names in inventory and permit both files to be extracted on Linux. A literal backslash is part of a POSIX filename, while `/` separates components. The released [PyInstaller writer](https://github.com/pyinstaller/pyinstaller/blob/v6.22.3/PyInstaller/archive/writers.py#L177-L196) uses the producer OS's path normalization and applies Windows separator conversion only on Windows. Its [bootloader extraction path](https://github.com/pyinstaller/pyinstaller/blob/v6.22.3/bootloader/src/pyi_launch.c#L63-L100) uses each data row's stored name. REA documents [inventory and extraction of CArchive members using archive path rules](https://github.com/morluto/rea/blob/3c5c887a2cc7371a94fe0fd8b6c478dbf2af60db/docs/python-packaged-applications.md#L3-L16). The verified producer TOC and returned bytes establish that these are separate valid entries. ## Impact and scope This bounded namespace error makes a valid Linux artifact appear to contain conflicting destinations and blocks its extraction through REA. The inventory preserves both content identities and original source names, so content loss is not claimed. No target modification occurred. Frequency outside this fixture is unknown; Windows, macOS and MCP were not exercised. A demonstrated external workaround is PyInstaller's `CArchiveReader`, which preserves these names and returns both payloads for this artifact when the matching Python environment is available. ## REA version `6.7.0`; source revision `3c5c887a2cc7371a94fe0fd8b6c478dbf2af60db` (main checkout; the published npm artifact was not separately executed). ## Runtime and platform Node.js v24.22.1; CPython 3.12.15; PyInstaller 6.22.3 with hooks-contrib 2026.9; Linux 7.2.8-9-omarchy x86_64, glibc 2.44. No native analysis engine was required. Current PyPI metadata identified PyInstaller 6.22.3 before its scratch-only installation. ## Target and artifact context Self-authored 12,050,976-byte Linux ELF onefile executable containing an appended CArchive, a `pass` entry point and the two ordinary text resources. Carrier SHA-256: `2d0176146e91d2ea36b7cf8a77e2c61392d283da62705d9e2817c49cff64e970`. Byte-for-byte reproducibility of the producer build across environments was not tested. ## Interface CLI ## Earliest failure stage Artifact or evidence registration ## Relevant output or evidence Observed producer-reader output, with Python string escaping: ```text 'a\\b.txt' -> b'literal backslash\n' 'a/b.txt' -> b'nested slash\n' ``` Projection of the observed successful CLI inventory: ```json { "coverage": {"scope":"container-entries","status":"complete"}, "occurrences": [ {"logical_path":"a/b.txt","source_name":"a/b.txt","declared_size":13,"hash_status":"verified","limitations":[]}, {"logical_path":"a/b.txt","source_name":"a\\b.txt","declared_size":18,"hash_status":"verified","limitations":[]} ] } ``` Observed extraction error prefix: ```text Artifact extraction path collision: "a/b.txt". Multiple selected occurrences share this normalized logical path; they cannot be materialized at the same destination path, even when the archive is valid. ``` - Existing contract: retain valid producer member identities and use the archive's path semantics, as linked above. - Failure provenance: real producer build, independent producer-reader bytes, and public REA CLI output; no mocks. - Local verification: reproduced. - Reproduction completeness: complete for the Linux fixture; target execution was neither needed nor performed. ## Typed outcome or recovery expected Return distinct logical destinations and successful extraction for both resources on POSIX. Preserve Windows archive separator handling. If producer path semantics cannot be established, expose that uncertainty instead of silently assigning a false shared destination. ## Analysis or suspected cause Source-derived and consistent with the observed output: [projectStoredMember](https://github.com/morluto/rea/blob/3c5c887a2cc7371a94fe0fd8b6c478dbf2af60db/src/artifacts/pyinstaller/CArchiveMembers.ts#L82-L96) passes every source name through [canonicalMemberPath](https://github.com/morluto/rea/blob/3c5c887a2cc7371a94fe0fd8b6c478dbf2af60db/src/artifacts/pyinstaller/CArchiveMembers.ts#L283-L291), which unconditionally replaces backslashes with slashes. That interpretation fits Windows producer paths but changes the meaning of the observed POSIX filename. No repair was implemented. ## Additional context No known-good REA revision was established. Open and closed searches for PyInstaller backslashes, CArchive paths, canonicalMemberPath, and PyInstaller collisions found no matching report. #6539 concerns a PYZ namespace directory colliding with a data destination; this fixture instead has two ordinary data members whose source names are changed by separator conversion. All runtime and source verification was completed before writing this draft; no reproduction was rerun afterward. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: Public CLI inventory assigns a false shared path to two valid producer entries, and public CLI extraction fails because of that projection. The defect is bounded to affected path spellings; source names and both content identities remain available. - Workaround status: available - Workaround basis: On the demonstrated artifact, PyInstaller's CArchiveReader retained both original names and returned both distinct text payloads using the tested matching Python environment. This bypasses REA's path projection without modifying the carrier.

bsakaguchi closed this issue

2026-10-31 00:46:49 +00:00

bsakaguchi referenced this issue

2026-11-01 21:10:57 +00:00

[GH-ISSUE #3796] [Tracker] Open feature requests by area (October 2026) #1471