[GH-ISSUE #1574] [Bug] Windows artifact extraction reports regular files as extracted when colon names become NTFS alternate streams #531

New issue

Closed

opened 2026-10-15 18:25:55 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-15 18:25:55 +00:00

Owner

Originally created by @dakjdakd on GitHub (Oct 10, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/1574

Before submitting

I searched open and closed issues and related extraction PRs for an existing report of this behavior.
The fixture contains only synthetic text; commands and output below omit private paths and credentials.

Area

Evidence, artifacts, or reconstruction.

Observed behavior

On Windows with an NTFS destination, extract-artifact reports a ZIP's ordinary file entry hello.txt:hidden as successfully materialized. The archive entry contains 14 bytes, and the returned extraction result records that exact logical name, created: true, bytes_written: 14, and containment_verified: true.

The filesystem result is different: directory enumeration shows a zero-byte hello.txt. Its 14-byte payload is in the NTFS alternate data stream named hidden, rather than in a distinct ordinary file corresponding to the archive entry. Reading the colon path retrieves the stream, which makes the apparent success easy to miss if verification only opens the same path again.

The archive itself is valid. A colon is a literal character in its logical entry name, and inventory correctly retains that name. The incorrect transition happens when the logical entry is passed to Windows file creation, where the colon has a different meaning.

Reproduction

From a built REA checkout on Windows, create this reduced ZIP using Python's standard library. No target application code is needed or executed:

from zipfile import ZipFile, ZIP_STORED

with ZipFile("ads.zip", "w", compression=ZIP_STORED) as archive:
    archive.writestr("hello.txt:hidden", b"secret payload")

Run the public CLI from PowerShell and retain its JSON result:

npm run build:cached
$extraction = node scripts/rea.mjs extract-artifact .\ads.zip --format json |
    ConvertFrom-Json
$result = $extraction.normalized_result
$result.artifacts | Format-List
$output = $result.output_root
Get-ChildItem -LiteralPath $output | Select-Object Name, Length
Get-Item -LiteralPath (Join-Path $output 'hello.txt') -Stream * |
    Select-Object FileName, Stream, Length
Get-Content -LiteralPath (Join-Path $output 'hello.txt') -Stream hidden

On the affected implementation, the CLI succeeds. Its relevant extraction fields are:

{
  "artifacts": [
    {
      "relative_path": "hello.txt:hidden",
      "bytes_written": 14,
      "created": true
    }
  ],
  "containment_verified": true,
  "cleanup": {
    "attempted": false,
    "verified": true,
    "residual_paths": []
  },
  "limitations": [
    "All regular files in the active artifact were materialized; nested archive contents remain represented by their containing file."
  ]
}

The independent filesystem observations are:

Directory listing:
Name       Length
hello.txt       0

Streams on hello.txt:
Stream    Length
:$DATA         0
hidden        14

Contents of hidden:
secret payload

The command creates an operation-owned extraction directory and returns its path in output_root.

Expected behavior

Extraction should preserve an ordinary archive file's identity at the destination, or refuse the operation if the destination cannot represent that identity. Windows should not silently interpret an admitted logical filename as an alternate data stream and then report ordinary-file materialization as complete.

For a Windows destination, a path containing stream syntax should produce a typed path failure before creating its base file or parent directories. The extraction should roll back its uncommitted output tree, including any ordinary entries successfully written earlier in the operation. The diagnostic should identify the offending logical path and explain the destination limitation.

Inventory should continue preserving the original logical name. On a POSIX destination that supports the literal name, extraction should continue creating hello.txt:hidden with the original bytes. Stripping the colon or renaming the entry would change identity and could introduce collisions with other archive entries.

Impact and scope

The reported extraction manifest and the ordinary filesystem tree disagree. A user inspecting the extracted directory sees an empty base file, while common enumeration, copying, repackaging, or later analysis workflows may omit the payload in its alternate stream. The returned byte count and completeness statement can therefore overstate what those workflows actually receive.

This is a reproducible correctness and evidence-integrity issue for Windows artifact materialization. The reproduction establishes neither escape from the output directory nor code execution. It also does not establish every possible Windows filename alias or reserved-name behavior; those require their own evidence.

Version, runtime, and target

Affected REA source revision: 0e19db9fce74af2081144102f5787cf90a450244; package version 6.3.0. The prepared fix has also been rebased onto c19e15a2b3f09b3f693d7eab5263d83f4d2d1cd4 and revalidated there.
Runtime: Node.js 24.18.0 and npm 11.16.0.
Platform: Windows x64 with NTFS output.
Target: a standard Python-produced ZIP containing one ordinary file entry.
Interface: CLI, using the REA artifact graph provider.
Earliest affected stage: artifact materialization, before successful extraction Evidence is produced.

Source-derived analysis

normalizeArtifactPath checks logical archive paths without applying destination-specific filename rules. Retaining a colon there is appropriate for inventory and POSIX extraction.

SafeOutputTree.write subsequently passes the normalized name to Windows open. Both writing and durable readback use that same colon path. Windows resolves both operations to the alternate stream, so its byte count and digest agree even though the ordinary-file identity was lost. A successful hash comparison therefore does not detect this representation mismatch.

The appropriate boundary is the shared output-tree materializer: reject Windows stream syntax there, retain the typed failure through the provider's existing translation, and use existing rollback ownership. A focused fix and regression coverage are ready, and The corresponding implementation is available in https://github.com/morluto/rea/pull/1575.

Originally created by @dakjdakd on GitHub (Oct 10, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/1574 ## Before submitting - [x] I searched open and closed issues and related extraction PRs for an existing report of this behavior. - [x] The fixture contains only synthetic text; commands and output below omit private paths and credentials. ## Area Evidence, artifacts, or reconstruction. ## Observed behavior On Windows with an NTFS destination, `extract-artifact` reports a ZIP's ordinary file entry `hello.txt:hidden` as successfully materialized. The archive entry contains 14 bytes, and the returned extraction result records that exact logical name, `created: true`, `bytes_written: 14`, and `containment_verified: true`. The filesystem result is different: directory enumeration shows a zero-byte `hello.txt`. Its 14-byte payload is in the NTFS alternate data stream named `hidden`, rather than in a distinct ordinary file corresponding to the archive entry. Reading the colon path retrieves the stream, which makes the apparent success easy to miss if verification only opens the same path again. The archive itself is valid. A colon is a literal character in its logical entry name, and inventory correctly retains that name. The incorrect transition happens when the logical entry is passed to Windows file creation, where the colon has a different meaning. ## Reproduction From a built REA checkout on Windows, create this reduced ZIP using Python's standard library. No target application code is needed or executed: ```python from zipfile import ZipFile, ZIP_STORED with ZipFile("ads.zip", "w", compression=ZIP_STORED) as archive: archive.writestr("hello.txt:hidden", b"secret payload") ``` Run the public CLI from PowerShell and retain its JSON result: ```powershell npm run build:cached $extraction = node scripts/rea.mjs extract-artifact .\ads.zip --format json | ConvertFrom-Json $result = $extraction.normalized_result $result.artifacts | Format-List $output = $result.output_root Get-ChildItem -LiteralPath $output | Select-Object Name, Length Get-Item -LiteralPath (Join-Path $output 'hello.txt') -Stream * | Select-Object FileName, Stream, Length Get-Content -LiteralPath (Join-Path $output 'hello.txt') -Stream hidden ``` On the affected implementation, the CLI succeeds. Its relevant extraction fields are: ```json { "artifacts": [ { "relative_path": "hello.txt:hidden", "bytes_written": 14, "created": true } ], "containment_verified": true, "cleanup": { "attempted": false, "verified": true, "residual_paths": [] }, "limitations": [ "All regular files in the active artifact were materialized; nested archive contents remain represented by their containing file." ] } ``` The independent filesystem observations are: ```text Directory listing: Name Length hello.txt 0 Streams on hello.txt: Stream Length :$DATA 0 hidden 14 Contents of hidden: secret payload ``` The command creates an operation-owned extraction directory and returns its path in `output_root`. ## Expected behavior Extraction should preserve an ordinary archive file's identity at the destination, or refuse the operation if the destination cannot represent that identity. Windows should not silently interpret an admitted logical filename as an alternate data stream and then report ordinary-file materialization as complete. For a Windows destination, a path containing stream syntax should produce a typed path failure before creating its base file or parent directories. The extraction should roll back its uncommitted output tree, including any ordinary entries successfully written earlier in the operation. The diagnostic should identify the offending logical path and explain the destination limitation. Inventory should continue preserving the original logical name. On a POSIX destination that supports the literal name, extraction should continue creating `hello.txt:hidden` with the original bytes. Stripping the colon or renaming the entry would change identity and could introduce collisions with other archive entries. ## Impact and scope The reported extraction manifest and the ordinary filesystem tree disagree. A user inspecting the extracted directory sees an empty base file, while common enumeration, copying, repackaging, or later analysis workflows may omit the payload in its alternate stream. The returned byte count and completeness statement can therefore overstate what those workflows actually receive. This is a reproducible correctness and evidence-integrity issue for Windows artifact materialization. The reproduction establishes neither escape from the output directory nor code execution. It also does not establish every possible Windows filename alias or reserved-name behavior; those require their own evidence. ## Version, runtime, and target - Affected REA source revision: `0e19db9fce74af2081144102f5787cf90a450244`; package version 6.3.0. The prepared fix has also been rebased onto `c19e15a2b3f09b3f693d7eab5263d83f4d2d1cd4` and revalidated there. - Runtime: Node.js 24.18.0 and npm 11.16.0. - Platform: Windows x64 with NTFS output. - Target: a standard Python-produced ZIP containing one ordinary file entry. - Interface: CLI, using the REA artifact graph provider. - Earliest affected stage: artifact materialization, before successful extraction Evidence is produced. ## Source-derived analysis [`normalizeArtifactPath`](https://github.com/morluto/rea/blob/main/src/artifacts/ArtifactPaths.ts) checks logical archive paths without applying destination-specific filename rules. Retaining a colon there is appropriate for inventory and POSIX extraction. [`SafeOutputTree.write`](https://github.com/morluto/rea/blob/main/src/artifacts/SafeOutputTree.ts) subsequently passes the normalized name to Windows `open`. Both writing and durable readback use that same colon path. Windows resolves both operations to the alternate stream, so its byte count and digest agree even though the ordinary-file identity was lost. A successful hash comparison therefore does not detect this representation mismatch. The appropriate boundary is the shared output-tree materializer: reject Windows stream syntax there, retain the typed failure through the provider's existing translation, and use existing rollback ownership. A focused fix and regression coverage are ready, and The corresponding implementation is available in https://github.com/morluto/rea/pull/1575.

bsakaguchi

2026-10-15 18:25:55 +00:00