morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #1491] fix(artifacts): classify container family from bytes before the filename suffix #503

New issue

Closed

opened 2026-10-15 18:25:39 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-15 18:25:39 +00:00

Owner

Originally created by @cursor[bot] on GitHub (Oct 10, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/1491

TL;DR: Inventory names a container family from the filename suffix and never checks the bytes. A non-ZIP file named .apk is opened as a ZIP, and a text file named .dmg or .pkg is recorded as that format.

Related to the boundary split in #1470: one rule (the suffix) is used as format identity before the boundary that can actually see the bytes.

Area

Evidence, artifacts, or reconstruction

Observed behavior

classifyAndHashRoot in src/artifacts/inventory/classify.ts calls classifyContainerExtension before classifyOpenedRoot. The extension helper returns apk, ipa, msix, appx, zip, asar, dmg, or pkg from the lower-cased path alone.

zipPackageFormatForPath documents the opposite rule: the suffix is not package identity until ZIP magic has been checked. BinaryTargetResolver.detectArtifactFormat follows that rule. It reads the header first, and only then uses the suffix to choose among ZIP families. It also requires xar! for .pkg and the koly trailer for .dmg.

Inventory does not. createReader then selects ZipArtifactReader for every apk / ipa / msix / appx / zip format, AsarArtifactReader for asar, and the native DMG reader for dmg. There is no pkg reader. inventoryLimitations still says the PKG or DMG root hash was observed and that child inventory needs a macOS adapter, including when the file is not that format and including PKG on macOS.

projectAndroidApplication accepts only root_format === "apk". A ZIP that is not an APK, renamed to .apk, takes that path. A real APK renamed to .zip does not.

Nested members have the same first guess. classifyArtifactPath assigns apk / ipa / asar from the suffix, and byte classification replaces that only after a digest. Encrypted entries and symlinks never get that replacement.

Expected behavior

The manifest root_format should be a byte observation, with the suffix used only to distinguish ZIP families after hasZipSignature succeeds. A file that is not a DMG or PKG should not be recorded as one, and the macOS-adapter limitation should be reserved for bytes that actually are that format.

Where

src/artifacts/inventory/classify.ts — classifyContainerExtension, classifyAndHashRoot
src/domain/zipPackageFormat.ts — zipPackageFormatForPath
src/application/BinaryTargetResolver.ts — detectArtifactFormat
src/artifacts/inventory/reader.ts — createReader, inventoryLimitations
src/artifacts/inventory/ArtifactGraphConstruction.ts — classifyArtifactPath, artifactRoleForFormat
src/domain/android/androidApplication.ts — projectAndroidApplication requires root_format === "apk"

Originally created by @cursor[bot] on GitHub (Oct 10, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/1491 **TL;DR:** Inventory names a container family from the filename suffix and never checks the bytes. A non-ZIP file named `.apk` is opened as a ZIP, and a text file named `.dmg` or `.pkg` is recorded as that format. Related to the boundary split in #1470: one rule (the suffix) is used as format identity before the boundary that can actually see the bytes. ### Area Evidence, artifacts, or reconstruction ### Observed behavior `classifyAndHashRoot` in `src/artifacts/inventory/classify.ts` calls `classifyContainerExtension` before `classifyOpenedRoot`. The extension helper returns `apk`, `ipa`, `msix`, `appx`, `zip`, `asar`, `dmg`, or `pkg` from the lower-cased path alone. `zipPackageFormatForPath` documents the opposite rule: the suffix is not package identity until ZIP magic has been checked. `BinaryTargetResolver.detectArtifactFormat` follows that rule. It reads the header first, and only then uses the suffix to choose among ZIP families. It also requires `xar!` for `.pkg` and the `koly` trailer for `.dmg`. Inventory does not. `createReader` then selects `ZipArtifactReader` for every `apk` / `ipa` / `msix` / `appx` / `zip` format, `AsarArtifactReader` for `asar`, and the native DMG reader for `dmg`. There is no `pkg` reader. `inventoryLimitations` still says the PKG or DMG root hash was observed and that child inventory needs a macOS adapter, including when the file is not that format and including PKG on macOS. `projectAndroidApplication` accepts only `root_format === "apk"`. A ZIP that is not an APK, renamed to `.apk`, takes that path. A real APK renamed to `.zip` does not. Nested members have the same first guess. `classifyArtifactPath` assigns `apk` / `ipa` / `asar` from the suffix, and byte classification replaces that only after a digest. Encrypted entries and symlinks never get that replacement. ### Expected behavior The manifest `root_format` should be a byte observation, with the suffix used only to distinguish ZIP families after `hasZipSignature` succeeds. A file that is not a DMG or PKG should not be recorded as one, and the macOS-adapter limitation should be reserved for bytes that actually are that format. ### Where - `src/artifacts/inventory/classify.ts` — `classifyContainerExtension`, `classifyAndHashRoot` - `src/domain/zipPackageFormat.ts` — `zipPackageFormatForPath` - `src/application/BinaryTargetResolver.ts` — `detectArtifactFormat` - `src/artifacts/inventory/reader.ts` — `createReader`, `inventoryLimitations` - `src/artifacts/inventory/ArtifactGraphConstruction.ts` — `classifyArtifactPath`, `artifactRoleForFormat` - `src/domain/android/androidApplication.ts` — `projectAndroidApplication` requires `root_format === "apk"`

bsakaguchi

2026-10-15 18:25:39 +00:00

closed this issue
added the
area: evidence

triage

bug
labels

bsakaguchi commented

2026-10-15 18:25:41 +00:00

Author

Owner

@CassieFox23 commented on GitHub (Oct 10, 2026):

TL;DR: Most of this reproduces on 983b070d4. APK or ELF bytes named .dmg or .pkg are recorded as that format, and a plain ZIP named .apk is projected as an APK. A plain text file never gets that far, because the CLI front door rejects it first.

I used WordleSolver (GPL-3.0-only) from F-Droid, a small Linux ELF executable, and a 43-byte text file. Each ran through rea inspect-artifact <file> --format json on Node 24.

file	result	as in issue?
APK bytes named `.dmg`	`root_format "dmg"`, 1 occurrence, macOS note (below)	yes
APK bytes named `.pkg`	`root_format "pkg"`, same macOS note	yes
ELF named `.dmg` / `.pkg`	`root_format "dmg"` / `"pkg"`, same note	yes
ELF named `.bin` (control)	`root_format "elf"`	control
ELF named `.apk`	exit 1, reason `io`: "Artifact could not be read or written. Check file access and try again."	yes, see below
APK bytes named `.asar`	exit 1, "Malformed ASAR during inventory"	yes
text named `.apk` / `.dmg` / `.pkg`	exit 1, `target_unavailable`, "unsupported binary format"	no
real APK named `.zip`	`root_format "zip"`; graph says "requires APK Evidence"	yes

The macOS note reads:

DMG root hash is observed; child inventory requires a native macOS adapter.

The ELF named .apk gets the ZIP reader. On the same file, zip.js (with REA's options) says "End of central directory not found". I did not trace that this is the error REA reports as io. Still, the message points at file access, not at the format.

A plain ZIP named `.apk`

The ZIP holds one text file and no manifest or dex.

{"root_format":"apk","coverage":{"status":"complete-within-inventory","inventory_complete":true},
 "counts":{"manifests":0,"resources":0,"dex":0,"jvm_classes":0,"native_libraries":0,"javascript":0,"signing":0}}

inspect-android-package on the same file does fail: "AndroidManifest.xml not found".

Nested members

I built a ZIP holding a text file named inner-text.apk and a symlink entry named link.apk.

entry	occurrence `artifact_kind` / `artifact_format`	node `format`
`inner-text.apk` (22 B of text)	`container` / `apk`	`file`
`inner-text.asar` (text)	`container` / `asar`	`file`
`link.apk` (symlink)	`container` / `apk`, `not-hashed`	no node
`inner-real.apk` (the APK)	`container` / `apk`	`zip`

So the node gets a byte check, but the occurrence keeps the suffix guess. The symlink never gets a byte check.

Not run

Encrypted ZIP entries.
macOS, where the DMG reader exists.
Real DMG and PKG files.
.ipa, .msix and .appx suffixes.

Edit: The sources for WordleSolver do not agree on its licence. F-Droid says GPL-3.0-only, the repo's LICENSE file says Apache-2.0, and the source headers say GPL-3.0-or-later.

Test setup

Box: Nobara 44 (Fedora 44 base), Linux 7.2.9
CPU: AMD Ryzen 9 5900X, 12 cores / 24 threads
Node: 24.18.0
Head: 983b070d4
App: WordleSolver 1.13 from F-Droid, which is GPL-3.0-only
APK hash: same as the F-Droid index

🤖 Generated with Claude Code (Opus 5.5) 🦊

@CassieFox23 commented on GitHub (Oct 10, 2026): **TL;DR:** Most of this reproduces on `983b070d4`. APK or ELF bytes named `.dmg` or `.pkg` are recorded as that format, and a plain ZIP named `.apk` is projected as an APK. A plain text file never gets that far, because the CLI front door rejects it first. I used [WordleSolver](https://f-droid.org/packages/org.billthefarmer.solver/) ([GPL-3.0-only](https://gitlab.com/fdroid/fdroiddata/-/blob/93455203833a2fe27d499d6f7870319a23712004/metadata/org.billthefarmer.solver.yml#L4)) from F-Droid, a small Linux ELF executable, and a 43-byte text file. Each ran through `rea inspect-artifact <file> --format json` on Node 24. | file | result | as in issue? | |---|---|---| | APK bytes named `.dmg` | `root_format "dmg"`, 1 occurrence, macOS note (below) | yes | | APK bytes named `.pkg` | `root_format "pkg"`, same macOS note | yes | | ELF named `.dmg` / `.pkg` | `root_format "dmg"` / `"pkg"`, same note | yes | | ELF named `.bin` (control) | `root_format "elf"` | control | | ELF named `.apk` | exit 1, reason `io`: "Artifact could not be read or written. Check file access and try again." | yes, see below | | APK bytes named `.asar` | exit 1, "Malformed ASAR during inventory" | yes | | text named `.apk` / `.dmg` / `.pkg` | exit 1, `target_unavailable`, "unsupported binary format" | no | | real APK named `.zip` | `root_format "zip"`; graph says "requires APK Evidence" | yes | The macOS note reads: ```text DMG root hash is observed; child inventory requires a native macOS adapter. ``` The ELF named `.apk` gets the ZIP reader. On the same file, zip.js (with REA's options) says "End of central directory not found". I did not trace that this is the error REA reports as `io`. Still, the message points at file access, not at the format. ### A plain ZIP named `.apk` The ZIP holds one text file and no manifest or dex. ```json {"root_format":"apk","coverage":{"status":"complete-within-inventory","inventory_complete":true}, "counts":{"manifests":0,"resources":0,"dex":0,"jvm_classes":0,"native_libraries":0,"javascript":0,"signing":0}} ``` `inspect-android-package` on the same file does fail: "AndroidManifest.xml not found". ### Nested members I built a ZIP holding a text file named `inner-text.apk` and a symlink entry named `link.apk`. | entry | occurrence `artifact_kind` / `artifact_format` | node `format` | |---|---|---| | `inner-text.apk` (22 B of text) | `container` / `apk` | `file` | | `inner-text.asar` (text) | `container` / `asar` | `file` | | `link.apk` (symlink) | `container` / `apk`, `not-hashed` | no node | | `inner-real.apk` (the APK) | `container` / `apk` | `zip` | So the node gets a byte check, but the occurrence keeps the suffix guess. The symlink never gets a byte check. ### Not run - Encrypted ZIP entries. - macOS, where the DMG reader exists. - Real DMG and PKG files. - `.ipa`, `.msix` and `.appx` suffixes. Edit: The sources for WordleSolver do not agree on its licence. F-Droid says `GPL-3.0-only`, the repo's [LICENSE](https://github.com/billthefarmer/wordlesolver/blob/59180002e66c3befaebdbbf0a41073baf9f0cef4/LICENSE) file says `Apache-2.0`, and the [source headers](https://github.com/billthefarmer/wordlesolver/blob/59180002e66c3befaebdbbf0a41073baf9f0cef4/src/main/java/org/billthefarmer/solver/Main.java#L8-L10) say `GPL-3.0-or-later`. <details> <summary>Test setup</summary> - Box: Nobara 44 (Fedora 44 base), Linux 7.2.9 - CPU: AMD Ryzen 9 5900X, 12 cores / 24 threads - Node: 24.18.0 - Head: `983b070d4` - App: [WordleSolver 1.13](https://f-droid.org/packages/org.billthefarmer.solver/) from F-Droid, which is [GPL-3.0-only](https://gitlab.com/fdroid/fdroiddata/-/blob/93455203833a2fe27d499d6f7870319a23712004/metadata/org.billthefarmer.solver.yml#L4) - APK hash: same as the F-Droid index </details> 🤖 Generated with Claude Code (Opus 5.5) 🦊

bsakaguchi commented

2026-10-15 18:25:41 +00:00

Author

Owner

@kaoru0822-kitauji commented on GitHub (Oct 10, 2026):

Root classification was made byte-first in #1513 (ELF/APK bytes named .dmg, .pkg, .apk, .asar now report their byte format). The remaining parts are fixed in #1553: nested members take ZIP/ASAR families from bytes, unread members (encrypted entries, symlinks) named like containers report unknown with a limitation, and the Android projection says when no root AndroidManifest.xml was observed and names the root format when it rejects non-APK inventory.

@kaoru0822-kitauji commented on GitHub (Oct 10, 2026): Root classification was made byte-first in #1513 (ELF/APK bytes named `.dmg`, `.pkg`, `.apk`, `.asar` now report their byte format). The remaining parts are fixed in #1553: nested members take ZIP/ASAR families from bytes, unread members (encrypted entries, symlinks) named like containers report `unknown` with a limitation, and the Android projection says when no root `AndroidManifest.xml` was observed and names the root format when it rejects non-APK inventory.

bsakaguchi referenced this issue

2026-10-15 18:28:32 +00:00

[GH-ISSUE #2383] [Tracker] Audit analysis and debugging coverage for the TIOBE top 100 languages (October 2026) #794

bsakaguchi referenced this issue

2026-10-16 21:13:32 +00:00

[PR #503] [MERGED] fix: correct native inventories and bridge error classification #2030