morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #6864] [Bug] extract-artifact rescans its own output and rejects an unchanged application bundle #4202

New issue

Closed

opened 2026-10-31 00:47:40 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-31 00:47:40 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 23, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6864

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

With TMPDIR inside the selected application bundle's Contents directory, extract-artifact copies a source file, traverses its newly created rea-extracted-* directory, and fails on the copy it just wrote: Regular artifact entry is missing from inventory: Contents/rea-extracted-<uuid>/Contents/Info.plist.

Inspection of the same unchanged source succeeds with complete coverage and no integrity contradictions. Extraction reports integrity_mismatch and recommends record-and-continue; that retry fails identically. Both failed output trees were removed and the original files remained unchanged. Moving TMPDIR outside the source makes the same extraction succeed with both original files and matching digests.

Reproduction

Use the stated built checkout. Save this executed probe as /tmp/nested-output-repro.py, then run it from the repository root with Node v24.22.1 on PATH:

REA_EVIDENCE_ROOT=/tmp python3 /tmp/nested-output-repro.py

from pathlib import Path
import hashlib, json, os, plistlib, struct, subprocess, tempfile
launcher = str(Path('scripts/rea.mjs').resolve())
r = Path(tempfile.mkdtemp(prefix='nested-output-', dir=os.environ['REA_EVIDENCE_ROOT']))
app = r/'Fixture.app'
(app/'Contents/MacOS').mkdir(parents=True)
(app/'Contents/Info.plist').write_bytes(plistlib.dumps({'CFBundlePackageType':'APPL', 'CFBundleExecutable':'fixture'}))
(app/'Contents/MacOS/fixture').write_bytes(struct.pack('<8I', 0xfeedfacf, 0x0100000c, 0, 2, 0, 0, 0, 0))
(app/'Contents/MacOS/fixture').chmod(0o755)
(r/'home').mkdir(); (r/'outside').mkdir()
env = {**os.environ, 'HOME':str(r/'home'), 'XDG_CONFIG_HOME':str(r/'home/config'), 'XDG_CACHE_HOME':str(r/'home/cache')}
before = {str(p.relative_to(app)):hashlib.sha256(p.read_bytes()).hexdigest() for p in app.rglob('*') if p.is_file()}
for name, command, temporary, extra in [
    ('inspect', 'inspect-artifact', app/'Contents', []),
    ('extract', 'extract-artifact', app/'Contents', []),
    ('continue', 'extract-artifact', app/'Contents', ['--integrity-policy', 'record-and-continue']),
    ('outside-control', 'extract-artifact', r/'outside', [])]:
    p = subprocess.run(['node', launcher, command, str(app), '--json', *extra], env={**env, 'TMPDIR':str(temporary)}, capture_output=True, text=True, timeout=45)
    (r/(name+'.json')).write_text(p.stdout); (r/(name+'.stderr')).write_text(p.stderr)
    x = json.loads(p.stdout); result = x.get('normalized_result', x)
    print(name, 'exit', p.returncode)
    if name == 'inspect':
        inventory = result['substeps'][0]['evidence']['normalized_result']
        print('coverage', inventory['coverage'], 'contradictions', inventory['integrity_contradictions'])
    elif p.returncode:
        print(result['category'], result['details'])
    else:
        print('paths', [a['relative_path'] for a in result['artifacts']])
        assert {a['relative_path'] for a in result['artifacts']} == set(before)
        assert all(hashlib.sha256((Path(result['output_root'])/name).read_bytes()).hexdigest() == digest for name,digest in before.items())
after = {str(p.relative_to(app)):hashlib.sha256(p.read_bytes()).hexdigest() for p in app.rglob('*') if p.is_file()}
assert before == after
assert not list((app/'Contents').glob('rea-extracted-*'))
print('original files unchanged; failed output trees absent; control bytes exact')
print('evidence', r)

The final probe ran one inspection, one default extraction, one record-and-continue extraction, and one outside-directory control. An earlier probe independently reproduced the two failed policy calls. The bundle contains only its required plist and an inert 32-byte Mach-O admission fixture; no external process changes the source during extraction.

Expected behavior

The extraction's own destination must not become a source member. A valid, unchanged bundle should materialize its original regular files without a false source-integrity failure. If an overlapping temporary parent is refused, it should be identified before copying, with an applicable destination explanation rather than a checksum retry.

The extraction contract materializes regular files exposed by the active source reader into REA's fresh temporary directory. Internal output files are not members of the source snapshot. The successful outside-directory control establishes that this source is extractable.

Impact and scope

P2: a valid extraction fails for a temporary directory beneath the inspected tree. The verified workaround is to set TMPDIR outside that tree; both original files then extract exactly. No source data loss or residual failed output was observed. This is specifically output entering the subsequent traversal: an earlier reduced fixture with TMPDIR at the bundle root and no root-level regular files succeeded, so the report does not claim every overlapping layout fails. macOS, Windows and MCP were not exercised.

REA version

Built main source 4526862728af01f1135714079c0d8a8c754eb364; public launcher reports 6.7.0. Latest published release: rea-agents-6.7.0, tag commit 27dc42d10249f0410904a75c0997842eb03b931c. The release tag was compared in source; its packaged runtime was not executed.

Runtime and platform

Node.js v24.22.1; Linux x86_64; repository-locked dependencies and prepared build. Python 3 standard library generated the fixtures and checked the filesystem. No native provider or target executable was launched.

Target and artifact context

Self-authored Fixture.app/Contents/Info.plist and Fixture.app/Contents/MacOS/fixture; TMPDIR=Fixture.app/Contents. The executable fixture is static parser input, not a runnable-app claim.

Interface

CLI

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

Final public CLI result, with only the random destination suffix shortened:

inspect: exit 0; coverage={scope:container-entries,status:complete}; contradictions=[]
extract: exit 1; category=integrity_mismatch; reason=integrity
  Regular artifact entry is missing from inventory: Contents/rea-extracted-<uuid>/Contents/Info.plist
record-and-continue: exit 1; same category, reason and self-generated path pattern
outside control: exit 0; paths=[Contents/Info.plist, Contents/MacOS/fixture]
original files unchanged; failed output trees absent; control bytes exact

Evidence provenance: observed public CLI output, stored JSON, source/output digest comparisons and output-directory enumeration. Local verification: reproduced. Reproduction completeness: complete for this fixture. No external modification, malformed archive, race injection or full test suite was used.

Typed outcome or recovery expected

Avoid consuming the owned output as source input, or return an applicable overlap/destination refusal before writing. Preserve the existing refusal for genuinely new unrelated source files; weakening inventory completeness would not restore this contract.

Analysis or suspected cause

Source-derived: inventory runs first, then the output tree is created before the second traversal. That traversal has no exclusion for its owned destination and rejects the copied entry as a new regular source file. These relevant operations also appear in the latest release tag; release runtime behavior remains untested.

Additional context

Open/closed GitHub searches and the 1,069-entry open-issue snapshot found no matching output-self-inclusion report. Closed #1318 concerns a file disappearing due to an external source change; this fixture retains every original. Closed #4271 concerns colliding ZIP member paths; this fixture has no duplicate paths. Open #6536 concerns observer-created process-capture effects, an independent workflow. #3490 requests destination/subset options; this report requires neither new interface.

No fix or last-known-good runtime was established. Prepared with AI assistance. Verification finished before this draft was written.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: The supported public extraction command fails on an unchanged admitted bundle because it reads its own newly created copies; inspection and the outside-directory control succeed.
Workaround status: available
Workaround basis: Setting TMPDIR to the executed sibling outside directory produced both original files with matching SHA-256 values.

Originally created by @coygeek on GitHub (Oct 23, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6864 ## Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ## Area Evidence, artifacts, or reconstruction ## Observed behavior With `TMPDIR` inside the selected application bundle's `Contents` directory, `extract-artifact` copies a source file, traverses its newly created `rea-extracted-*` directory, and fails on the copy it just wrote: `Regular artifact entry is missing from inventory: Contents/rea-extracted-<uuid>/Contents/Info.plist`. Inspection of the same unchanged source succeeds with complete coverage and no integrity contradictions. Extraction reports `integrity_mismatch` and recommends `record-and-continue`; that retry fails identically. Both failed output trees were removed and the original files remained unchanged. Moving `TMPDIR` outside the source makes the same extraction succeed with both original files and matching digests. ## Reproduction Use the stated built checkout. Save this executed probe as `/tmp/nested-output-repro.py`, then run it from the repository root with Node v24.22.1 on `PATH`: ```sh REA_EVIDENCE_ROOT=/tmp python3 /tmp/nested-output-repro.py ``` ```python from pathlib import Path import hashlib, json, os, plistlib, struct, subprocess, tempfile launcher = str(Path('scripts/rea.mjs').resolve()) r = Path(tempfile.mkdtemp(prefix='nested-output-', dir=os.environ['REA_EVIDENCE_ROOT'])) app = r/'Fixture.app' (app/'Contents/MacOS').mkdir(parents=True) (app/'Contents/Info.plist').write_bytes(plistlib.dumps({'CFBundlePackageType':'APPL', 'CFBundleExecutable':'fixture'})) (app/'Contents/MacOS/fixture').write_bytes(struct.pack('<8I', 0xfeedfacf, 0x0100000c, 0, 2, 0, 0, 0, 0)) (app/'Contents/MacOS/fixture').chmod(0o755) (r/'home').mkdir(); (r/'outside').mkdir() env = {**os.environ, 'HOME':str(r/'home'), 'XDG_CONFIG_HOME':str(r/'home/config'), 'XDG_CACHE_HOME':str(r/'home/cache')} before = {str(p.relative_to(app)):hashlib.sha256(p.read_bytes()).hexdigest() for p in app.rglob('*') if p.is_file()} for name, command, temporary, extra in [ ('inspect', 'inspect-artifact', app/'Contents', []), ('extract', 'extract-artifact', app/'Contents', []), ('continue', 'extract-artifact', app/'Contents', ['--integrity-policy', 'record-and-continue']), ('outside-control', 'extract-artifact', r/'outside', [])]: p = subprocess.run(['node', launcher, command, str(app), '--json', *extra], env={**env, 'TMPDIR':str(temporary)}, capture_output=True, text=True, timeout=45) (r/(name+'.json')).write_text(p.stdout); (r/(name+'.stderr')).write_text(p.stderr) x = json.loads(p.stdout); result = x.get('normalized_result', x) print(name, 'exit', p.returncode) if name == 'inspect': inventory = result['substeps'][0]['evidence']['normalized_result'] print('coverage', inventory['coverage'], 'contradictions', inventory['integrity_contradictions']) elif p.returncode: print(result['category'], result['details']) else: print('paths', [a['relative_path'] for a in result['artifacts']]) assert {a['relative_path'] for a in result['artifacts']} == set(before) assert all(hashlib.sha256((Path(result['output_root'])/name).read_bytes()).hexdigest() == digest for name,digest in before.items()) after = {str(p.relative_to(app)):hashlib.sha256(p.read_bytes()).hexdigest() for p in app.rglob('*') if p.is_file()} assert before == after assert not list((app/'Contents').glob('rea-extracted-*')) print('original files unchanged; failed output trees absent; control bytes exact') print('evidence', r) ``` The final probe ran one inspection, one default extraction, one `record-and-continue` extraction, and one outside-directory control. An earlier probe independently reproduced the two failed policy calls. The bundle contains only its required plist and an inert 32-byte Mach-O admission fixture; no external process changes the source during extraction. ## Expected behavior The extraction's own destination must not become a source member. A valid, unchanged bundle should materialize its original regular files without a false source-integrity failure. If an overlapping temporary parent is refused, it should be identified before copying, with an applicable destination explanation rather than a checksum retry. The [extraction contract](https://github.com/morluto/rea/blob/4526862728af01f1135714079c0d8a8c754eb364/src/contracts/artifactToolContracts.ts#L86-L90) materializes regular files exposed by the active source reader into REA's fresh temporary directory. Internal output files are not members of the source snapshot. The successful outside-directory control establishes that this source is extractable. ## Impact and scope P2: a valid extraction fails for a temporary directory beneath the inspected tree. The verified workaround is to set `TMPDIR` outside that tree; both original files then extract exactly. No source data loss or residual failed output was observed. This is specifically output entering the subsequent traversal: an earlier reduced fixture with `TMPDIR` at the bundle root and no root-level regular files succeeded, so the report does not claim every overlapping layout fails. macOS, Windows and MCP were not exercised. ## REA version Built main source `4526862728af01f1135714079c0d8a8c754eb364`; public launcher reports `6.7.0`. Latest published release: `rea-agents-6.7.0`, tag commit `27dc42d10249f0410904a75c0997842eb03b931c`. The release tag was compared in source; its packaged runtime was not executed. ## Runtime and platform Node.js v24.22.1; Linux x86_64; repository-locked dependencies and prepared build. Python 3 standard library generated the fixtures and checked the filesystem. No native provider or target executable was launched. ## Target and artifact context Self-authored `Fixture.app/Contents/Info.plist` and `Fixture.app/Contents/MacOS/fixture`; `TMPDIR=Fixture.app/Contents`. The executable fixture is static parser input, not a runnable-app claim. ## Interface CLI ## Earliest failure stage Native analysis or reconstruction ## Relevant output or evidence Final public CLI result, with only the random destination suffix shortened: ```text inspect: exit 0; coverage={scope:container-entries,status:complete}; contradictions=[] extract: exit 1; category=integrity_mismatch; reason=integrity Regular artifact entry is missing from inventory: Contents/rea-extracted-<uuid>/Contents/Info.plist record-and-continue: exit 1; same category, reason and self-generated path pattern outside control: exit 0; paths=[Contents/Info.plist, Contents/MacOS/fixture] original files unchanged; failed output trees absent; control bytes exact ``` Evidence provenance: observed public CLI output, stored JSON, source/output digest comparisons and output-directory enumeration. Local verification: reproduced. Reproduction completeness: complete for this fixture. No external modification, malformed archive, race injection or full test suite was used. ## Typed outcome or recovery expected Avoid consuming the owned output as source input, or return an applicable overlap/destination refusal before writing. Preserve the existing refusal for genuinely new unrelated source files; weakening inventory completeness would not restore this contract. ## Analysis or suspected cause Source-derived: [inventory runs first](https://github.com/morluto/rea/blob/4526862728af01f1135714079c0d8a8c754eb364/src/artifacts/extraction/ArtifactExtraction.ts#L70-L83), then [the output tree is created before the second traversal](https://github.com/morluto/rea/blob/4526862728af01f1135714079c0d8a8c754eb364/src/artifacts/extraction/ArtifactExtraction.ts#L264-L272). [That traversal](https://github.com/morluto/rea/blob/4526862728af01f1135714079c0d8a8c754eb364/src/artifacts/extraction/ArtifactExtraction.ts#L320-L332) has no exclusion for its owned destination and rejects the copied entry as a new regular source file. These relevant operations also appear in the latest release tag; release runtime behavior remains untested. ## Additional context Open/closed GitHub searches and the 1,069-entry open-issue snapshot found no matching output-self-inclusion report. Closed [#1318](https://github.com/morluto/rea/issues/1318) concerns a file disappearing due to an external source change; this fixture retains every original. Closed [#4271](https://github.com/morluto/rea/issues/4271) concerns colliding ZIP member paths; this fixture has no duplicate paths. Open [#6536](https://github.com/morluto/rea/issues/6536) concerns observer-created process-capture effects, an independent workflow. [#3490](https://github.com/morluto/rea/issues/3490) requests destination/subset options; this report requires neither new interface. No fix or last-known-good runtime was established. Prepared with AI assistance. Verification finished before this draft was written. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: The supported public extraction command fails on an unchanged admitted bundle because it reads its own newly created copies; inspection and the outside-directory control succeed. - Workaround status: available - Workaround basis: Setting TMPDIR to the executed sibling outside directory produced both original files with matching SHA-256 values.

bsakaguchi closed this issue

2026-10-31 00:47:40 +00:00

bsakaguchi commented

2026-10-31 00:47:41 +00:00

Author

Owner

@morluto commented on GitHub (Oct 23, 2026):

Resolved on main in 5e8f8b316. Directory extraction rejects an output parent inside its canonical source before creating output, and the control extracts exact bytes outside the source. This is an explicitly accepted outcome in the issue. Validation: src/application/artifacts/ArtifactCallerPaths.test.ts — refuses directory extraction output inside the source before copying and keeps outside extraction exact; recorded execution at a732bd065. Real filesystem/shared extraction boundary, not public CLI; later reader/materialization changes do not remove the pre-write overlap guard.

@morluto commented on GitHub (Oct 23, 2026): Resolved on main in [5e8f8b316](https://github.com/morluto/rea/commit/5e8f8b3163e4ff731b2c391a897e5b7e6a0f1a90). Directory extraction rejects an output parent inside its canonical source before creating output, and the control extracts exact bytes outside the source. This is an explicitly accepted outcome in the issue. Validation: `src/application/artifacts/ArtifactCallerPaths.test.ts` — refuses directory extraction output inside the source before copying and keeps outside extraction exact; recorded execution at `a732bd065`. Real filesystem/shared extraction boundary, not public CLI; later reader/materialization changes do not remove the pre-write overlap guard.

bsakaguchi referenced this issue

2026-10-31 01:12:30 +00:00

[PR #4338] [CLOSED] fix(javascript): verify loader origins, assignment operators, and bootstrap integrity in static analysis #6182

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#4202

No description provided.

Rows
Columns