morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #6702] [Bug] PyInstaller binary members are always marked non-executable in artifact inventory #4073

New issue

Closed

opened 2026-10-31 00:46:36 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:46:36 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 23, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6702

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

inspect-artifact successfully inventories a valid CArchive containing an executable shell resource (helper.sh, type b) and a data control (healthy.txt, type x), but reports executable: false for both. The type b declaration survives in format_metadata.type_code, so the public generic flag contradicts the executable marking encoded by this archive type. The same false flag is present in the exported inventory Evidence.

Reproduction

Run from the built source checkout described below. This creates a tiny inert ELF carrier with an ordinary CArchive 2.1 directory and two stored members. It follows the repository's existing synthetic ELF/CArchive fixture convention; it is not a runnable PyInstaller application. Nothing in the carrier or its members is executed.

python3 - <<'PYTHON'
"""Generate inert CArchive bytes; never execute the embedded shell text."""
import pathlib
import struct

root = pathlib.Path('/tmp/rea-repro')
root.mkdir(parents=True, exist_ok=True)
prefix = bytearray(64)
struct.pack_into('<I', prefix, 0, 0x464c457f)
prefix[4:7] = b'\x02\x01\x01'
struct.pack_into('<HHI', prefix, 16, 2, 62, 1)
struct.pack_into('<H', prefix, 52, 64)
payloads, entries, offset = [], [], 0
for name, typ, data in [
    ('helper.sh', 'b', b'#!/bin/sh\nexit 0\n'),
    ('healthy.txt', 'x', b'healthy\n'),
]:
    name_bytes = name.encode() + b'\0'
    size = 18 + len(name_bytes)
    padded = (size + 15) // 16 * 16
    entries.append(
        struct.pack('!IIIIBB', padded, offset, len(data), len(data), 0, ord(typ))
        + name_bytes + b'\0' * (padded - size)
    )
    payloads.append(data)
    offset += len(data)
toc = b''.join(entries)
cookie = struct.pack(
    '!8sIIII64s', b'MEI\x0c\x0b\x0a\x0b\x0e',
    offset + len(toc) + 88, offset, len(toc), 311, b'libpython3.11.so',
)
(root / 'inert-onefile.bin').write_bytes(bytes(prefix) + b''.join(payloads) + toc + cookie)
PYTHON
node scripts/rea.mjs inspect-artifact /tmp/rea-repro/inert-onefile.bin --json \
  > /tmp/rea-repro/inspection.json
python3 - <<'PYTHON'
import json
with open('/tmp/rea-repro/inspection.json') as source:
    report = json.load(source)
inventory = report['normalized_result']['substeps'][0]['evidence']['normalized_result']
print('root_format:', inventory['manifest']['root_format'])
for item in inventory['occurrences']:
    if item['logical_path'] != '.':
        print(item['logical_path'], item['format_metadata']['type_code'],
              'executable=' + str(item['executable']))
PYTHON

The researcher and a separate reviewing invocation each observed the same result before this draft was written. This is not a claim about population frequency or behavior of an executed application.

Expected behavior

Retain the executable marking of CArchive type b members in the generic inventory flag, while keeping the type x data control non-executable. This does not require executing or disassembling the member or changing extraction permissions.

The released PyInstaller producer explicitly maps executable DATA files, including shell scripts, to type b so that their executable bit is restored: PyInstaller 6.0.0 building/api.py, lines 309–315. Its POSIX bootloader grants S_IXUSR to ARCHIVE_ITEM_BINARY members and omits that bit for other extracted files: pyi_archive.c, lines 343–348. pyi_archive.h, line 26 defines that type as b. These sources establish the archive marking independently of the .sh suffix or payload text.

Impact and scope

The artifact inventory gives consumers a false negative when they use executable to identify packaged executable resources. Source inspection shows that every regular PyInstaller member goes through the same hardcoded false, including type b; the runtime reproduction covers one such member and a type x control on Linux. A practical workaround is to inspect the preserved format_metadata.type_code and recognize CArchive type b manually. No member-byte loss, target execution, or defect in REA's deliberately private extraction permissions was observed or claimed.

REA version

Source build 6.7.0 at 8c4f96ef4759fd78da66c4b9726825ca71d7ee0c; the latest npm release checked for the investigation was also 6.7.0. The source launcher reported 6.7.0. Published-package execution was not tested.

Runtime and platform

Node.js v26.12.1; Linux x86_64; Python 3 standard library for fixture construction. No Hopper, Ghidra, PyInstaller installation, or native engine is required.

Target and artifact context

Self-authored 241-byte inert ELF/CArchive carrier, CArchive layout 2.1, Python cookie version 311, library name libpython3.11.so. Its two stored members contain 17 bytes of inert shell text and 8 bytes of plain text. Member records use ordinary 16-byte alignment. The synthetic carrier proves static reader and CLI reachability; a real bootloader-generated application was not tested.

Interface

CLI

Earliest failure stage

Artifact or evidence registration

Relevant output or evidence

The CLI exited 0 with empty stderr. Its compact inventory projection was:

root_format: pyinstaller
healthy.txt x executable=False
helper.sh b executable=False

Both members have hash_status: verified; the original type declarations remain intact. The reader-level check and the retained exported Evidence agree with this public CLI result.

Evidence provenance: observed, with source-derived explanation.
Local verification: reproduced through the reader and public source CLI.
Reproduction completeness: complete for the synthetic static carrier; no real application or repaired build was executed.

Typed outcome or recovery expected

Successful inspection should report helper.sh as executable: true while preserving its type, content identity and the data control's false flag. No error, decryption, target execution or changed output-file mode is necessary. A restoration check using this same fixture is proposed, not executed against a fix.

Analysis or suspected cause

The common fileEntry builder hardcodes executable: false, ignoring its formatMetadata.type_code. The shared occurrence projection copies that value directly. This explains the observed false flag without relying on suffix-based executable detection.

The existing synthetic CArchive fixture builder uses the same inert ELF header convention. The inspected PyInstaller reader and CLI/MCP acceptance tests do not assert executable member metadata; no full suite was run for this report.

Additional context

Open and closed searches for PyInstaller, CArchive and executable metadata found no matching report. #1819 concerns Windows host-file executable detection, rather than packaged CArchive declarations. #6674 concerns a data member named PYZ.pyz; #6539 concerns namespace/path collisions; #6441 concerns extraction derivation notices; #6485 concerns comparison of metadata declarations. Their triggers and expected restoration differ from this generic executable flag.

NOT_ENOUGH_INFO: behavior on other platforms, MCP transport, a real PyInstaller-produced application, the published npm package, and a fixed build were not tested. PyInstaller producer and bootloader semantics above are source observations, not a claim that either was run.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: The successful public inventory publishes an incorrect executable flag for an ordinary supported CArchive member type, which can omit packaged helpers from consumers' executable-resource inventories. The observed failure is bounded to metadata; no file-content loss or execution failure was established.
Workaround status: available
Workaround basis: The same observed output preserves format_metadata.container: carchive and type_code: b, allowing consumers to recover the executable marking without changing or executing the archive.

Originally created by @coygeek on GitHub (Oct 23, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6702 ## Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ## Area Evidence, artifacts, or reconstruction ## Observed behavior `inspect-artifact` successfully inventories a valid CArchive containing an executable shell resource (`helper.sh`, type `b`) and a data control (`healthy.txt`, type `x`), but reports `executable: false` for both. The type `b` declaration survives in `format_metadata.type_code`, so the public generic flag contradicts the executable marking encoded by this archive type. The same false flag is present in the exported inventory Evidence. ## Reproduction Run from the built source checkout described below. This creates a tiny inert ELF carrier with an ordinary CArchive 2.1 directory and two stored members. It follows the repository's existing synthetic ELF/CArchive fixture convention; it is not a runnable PyInstaller application. Nothing in the carrier or its members is executed. ```sh python3 - <<'PYTHON' """Generate inert CArchive bytes; never execute the embedded shell text.""" import pathlib import struct root = pathlib.Path('/tmp/rea-repro') root.mkdir(parents=True, exist_ok=True) prefix = bytearray(64) struct.pack_into('<I', prefix, 0, 0x464c457f) prefix[4:7] = b'\x02\x01\x01' struct.pack_into('<HHI', prefix, 16, 2, 62, 1) struct.pack_into('<H', prefix, 52, 64) payloads, entries, offset = [], [], 0 for name, typ, data in [ ('helper.sh', 'b', b'#!/bin/sh\nexit 0\n'), ('healthy.txt', 'x', b'healthy\n'), ]: name_bytes = name.encode() + b'\0' size = 18 + len(name_bytes) padded = (size + 15) // 16 * 16 entries.append( struct.pack('!IIIIBB', padded, offset, len(data), len(data), 0, ord(typ)) + name_bytes + b'\0' * (padded - size) ) payloads.append(data) offset += len(data) toc = b''.join(entries) cookie = struct.pack( '!8sIIII64s', b'MEI\x0c\x0b\x0a\x0b\x0e', offset + len(toc) + 88, offset, len(toc), 311, b'libpython3.11.so', ) (root / 'inert-onefile.bin').write_bytes(bytes(prefix) + b''.join(payloads) + toc + cookie) PYTHON node scripts/rea.mjs inspect-artifact /tmp/rea-repro/inert-onefile.bin --json \ > /tmp/rea-repro/inspection.json python3 - <<'PYTHON' import json with open('/tmp/rea-repro/inspection.json') as source: report = json.load(source) inventory = report['normalized_result']['substeps'][0]['evidence']['normalized_result'] print('root_format:', inventory['manifest']['root_format']) for item in inventory['occurrences']: if item['logical_path'] != '.': print(item['logical_path'], item['format_metadata']['type_code'], 'executable=' + str(item['executable'])) PYTHON ``` The researcher and a separate reviewing invocation each observed the same result before this draft was written. This is not a claim about population frequency or behavior of an executed application. ## Expected behavior Retain the executable marking of CArchive type `b` members in the generic inventory flag, while keeping the type `x` data control non-executable. This does not require executing or disassembling the member or changing extraction permissions. The released PyInstaller producer explicitly maps executable DATA files, including shell scripts, to type `b` so that their executable bit is restored: [PyInstaller 6.0.0 building/api.py, lines 309–315](https://github.com/pyinstaller/pyinstaller/blob/v6.0.0/PyInstaller/building/api.py#L309-L315). Its POSIX bootloader grants `S_IXUSR` to `ARCHIVE_ITEM_BINARY` members and omits that bit for other extracted files: [pyi_archive.c, lines 343–348](https://github.com/pyinstaller/pyinstaller/blob/v6.0.0/bootloader/src/pyi_archive.c#L343-L348). [pyi_archive.h, line 26](https://github.com/pyinstaller/pyinstaller/blob/v6.0.0/bootloader/src/pyi_archive.h#L26) defines that type as `b`. These sources establish the archive marking independently of the `.sh` suffix or payload text. ## Impact and scope The artifact inventory gives consumers a false negative when they use `executable` to identify packaged executable resources. Source inspection shows that every regular PyInstaller member goes through the same hardcoded `false`, including type `b`; the runtime reproduction covers one such member and a type `x` control on Linux. A practical workaround is to inspect the preserved `format_metadata.type_code` and recognize CArchive type `b` manually. No member-byte loss, target execution, or defect in REA's deliberately private extraction permissions was observed or claimed. ## REA version Source build `6.7.0` at `8c4f96ef4759fd78da66c4b9726825ca71d7ee0c`; the latest npm release checked for the investigation was also `6.7.0`. The source launcher reported `6.7.0`. Published-package execution was not tested. ## Runtime and platform Node.js `v26.12.1`; Linux x86_64; Python 3 standard library for fixture construction. No Hopper, Ghidra, PyInstaller installation, or native engine is required. ## Target and artifact context Self-authored 241-byte inert ELF/CArchive carrier, CArchive layout 2.1, Python cookie version 311, library name `libpython3.11.so`. Its two stored members contain 17 bytes of inert shell text and 8 bytes of plain text. Member records use ordinary 16-byte alignment. The synthetic carrier proves static reader and CLI reachability; a real bootloader-generated application was not tested. ## Interface CLI ## Earliest failure stage Artifact or evidence registration ## Relevant output or evidence The CLI exited 0 with empty stderr. Its compact inventory projection was: ```text root_format: pyinstaller healthy.txt x executable=False helper.sh b executable=False ``` Both members have `hash_status: verified`; the original type declarations remain intact. The reader-level check and the retained exported Evidence agree with this public CLI result. - Evidence provenance: observed, with source-derived explanation. - Local verification: reproduced through the reader and public source CLI. - Reproduction completeness: complete for the synthetic static carrier; no real application or repaired build was executed. ## Typed outcome or recovery expected Successful inspection should report `helper.sh` as `executable: true` while preserving its type, content identity and the data control's false flag. No error, decryption, target execution or changed output-file mode is necessary. A restoration check using this same fixture is proposed, not executed against a fix. ## Analysis or suspected cause The common [`fileEntry` builder hardcodes `executable: false`](https://github.com/morluto/rea/blob/8c4f96ef4759fd78da66c4b9726825ca71d7ee0c/src/artifacts/pyinstaller/CArchiveMembers.ts#L311-L333), ignoring its `formatMetadata.type_code`. The shared [occurrence projection copies that value directly](https://github.com/morluto/rea/blob/8c4f96ef4759fd78da66c4b9726825ca71d7ee0c/src/artifacts/inventory/ArtifactGraphConstruction.ts#L103-L113). This explains the observed false flag without relying on suffix-based executable detection. The existing [synthetic CArchive fixture builder](https://github.com/morluto/rea/blob/8c4f96ef4759fd78da66c4b9726825ca71d7ee0c/src/artifacts/pyinstaller/CArchive.fixture.ts#L8-L23) uses the same inert ELF header convention. The inspected PyInstaller reader and CLI/MCP acceptance tests do not assert executable member metadata; no full suite was run for this report. ## Additional context Open and closed searches for PyInstaller, CArchive and executable metadata found no matching report. #1819 concerns Windows host-file executable detection, rather than packaged CArchive declarations. #6674 concerns a data member named PYZ.pyz; #6539 concerns namespace/path collisions; #6441 concerns extraction derivation notices; #6485 concerns comparison of metadata declarations. Their triggers and expected restoration differ from this generic executable flag. NOT_ENOUGH_INFO: behavior on other platforms, MCP transport, a real PyInstaller-produced application, the published npm package, and a fixed build were not tested. PyInstaller producer and bootloader semantics above are source observations, not a claim that either was run. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: The successful public inventory publishes an incorrect executable flag for an ordinary supported CArchive member type, which can omit packaged helpers from consumers' executable-resource inventories. The observed failure is bounded to metadata; no file-content loss or execution failure was established. - Workaround status: available - Workaround basis: The same observed output preserves `format_metadata.container: carchive` and `type_code: b`, allowing consumers to recover the executable marking without changing or executing the archive.

bsakaguchi closed this issue

2026-10-31 00:46:36 +00:00

bsakaguchi referenced this issue

2026-10-31 01:11:22 +00:00

[PR #4073] [MERGED] fix(adb): normalize absent installer package sentinels #6037

bsakaguchi referenced this issue

2026-11-01 21:11:07 +00:00

[GH-ISSUE #5654] [Tracker] Open PRs that can't merge yet: conflicts and red CI (October 2026) #3425