[PR #1553] [MERGED] fix(artifacts): take nested container families from bytes #2967

New issue

Closed

opened 2026-10-17 21:12:21 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-17 21:12:21 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/1553
Author: @kaoru0822-kitauji
Created: 10/10/2026
Status: ✅ Merged
Merged: 10/10/2026
Merged by: @kaoru0822-kitauji

Base: main ← Head: fix/inventory-root-format-bytes

📝 Commits (1)

77604d3 fix(artifacts): take nested container families from bytes

📊 Changes

8 files changed (+263 additions, -67 deletions)

View changed files

📝 docs/android-analysis.md (+4 -0)
📝 src/artifacts/inventory/ArtifactGraphConstruction.ts (+83 -31)
📝 src/artifacts/inventory/classify.ts (+4 -27)
📝 src/artifacts/inventory/scanReader.ts (+9 -2)
📝 src/domain/android/androidApplication.ts (+8 -1)
📝 tests/boundary/filesystem/asarInventoryContainment.test.ts (+39 -4)
📝 tests/boundary/filesystem/zipDetection.test.ts (+78 -1)
📝 tests/composition/android/application.test.ts (+38 -1)

📄 Description

Summary

Fixes #1491.

#1513 made root classification byte-first, so the root cases in the issue (ELF or APK bytes named .dmg, .pkg, .apk or .asar) already report the byte format on main. This PR covers what was still open: nested members, and the Android projection of a ZIP that only has an .apk name.

Problem and expected behavior

On 2d99e1e3, a ZIP holding inner-text.apk (text), inner-text.asar (text) and a symlink entry link.apk gave these occurrences:

entry	before	after
`inner-text.apk`	`container` / `apk`	`resource` / `file`
`inner-text.asar`	`container` / `asar`	`resource` / `file`
`link.apk` (symlink)	`container` / `apk`, `not-hashed`	`unknown` / `unknown`, with a limitation
`inner-real.apk` (ZIP bytes)	`container` / `apk`	unchanged

classifyArtifactPath returned a container family from the suffix, and byte classification only replaced it when the bytes were ZIP. Encrypted entries and symlinks were never read, so they kept the guess. A filesystem .asar without an ASAR header also failed the whole inventory as a malformed archive, while the same bytes as a root were already classified as file.

A plain ZIP named .apk was projected as an Android application with no sign that no manifest was present.

Change and scope

classifyArtifactPath no longer returns container families. Nested ZIP families follow ZIP magic (zipPackageFormatForPath only picks among them after magic). ASAR follows the header pickle check, now shared by root and nested classification (hasAsarHeader moved from classify.ts to ArtifactGraphConstruction.ts).
An unread member named like a container (encrypted entry, symlink, unavailable unpacked companion) gets unknown / unknown and the limitation The path suffix suggests APK, but these bytes were not read, so the container family is unknown. Other suffix role hints (.js, .plist, .dylib, ...) are unchanged.
Filesystem ASAR expansion requires the header. A .asar with a valid header but a malformed body still fails with format.
project_android_application_graph adds No root AndroidManifest.xml occurrence was observed. The APK family comes from the .apk suffix of a ZIP archive, so these bytes may not be an APK. when the root has no manifest. Its rejection names the observed root format (... the inventory root format is zip. APK is chosen for a ZIP archive whose name ends in .apk.).

Not included: accepting a real APK named .zip in the Android projection. Its result contract has root_format: "apk", and inferring APK from contents would be a separate contract decision. The error now says what to do.

Contract and boundary impact

Semantic owner and earliest changed stage: artifact inventory occurrence classification (src/artifacts/inventory), then the Android domain projection.
CLI and MCP/tool-catalog contract: no schema change. artifact_kind / artifact_format values for affected nested members change as in the table above; existing enum values only.
Provider, bridge, target-format, or platform compatibility: none.
Evidence, artifact, provenance, or reconstruction contract: nested container family is now a byte observation; unread members report unknown with an explicit limitation.
Process execution, authorization, cleanup, or containment impact: a non-ASAR .asar file in an app directory is hashed as a file instead of being opened by the ASAR reader.
Generated metadata, package, or installation impact: none (npm run docs:check passes).

Evidence and regression coverage

Tests added or updated:
- tests/boundary/filesystem/zipDetection.test.ts: nested ZIP members by bytes (text .apk, ZIP .apk, ZIP .bin, encrypted .ipa) and an unfollowed filesystem symlink link.apk.
- tests/boundary/filesystem/asarInventoryContainment.test.ts: real/ordinary/encrypted .asar members in a ZIP; a headerless filesystem .asar is a file; a header with malformed JSON is still rejected. The old test "keeps malformed filesystem archives rejected" pinned the headerless case as a failure and is replaced.
- tests/composition/android/application.test.ts: the missing-manifest limitation and the rejection message.
Base reproduction: the issue comment's nested ZIP run through rea inspect-artifact on main and on this branch (table above).
Remaining proof gaps: no real APK/IPA/MSIX files were inventoried; fixtures are zip.js-built archives.

For evidence-bearing changes:

Observed, derived, and inferred claims remain distinguishable.
Artifact identity, source provenance, and failed attempts remain preserved.
Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

npm run check:fast — passed.
npm run check:changed — 44 files, 444 tests passed.
npm run test:focused -- all of tests/boundary/filesystem, tests/boundary/providers/native, tests/composition, artifactMcp, mobileApplicationGraphMcp, cliOutput, nonBinaryComposition — 1446 passed, 8 failed. The same 8 fail on unchanged main on this Mac (no ibtool with Command Line Tools only; the real hdiutil verify-refusal test).
npm run docs:check — passed.
Real app: rea inspect-artifact /Applications/Discord.app gives the same 580 occurrences (kind, format, hash status, limitations) on main and on this branch; Contents/Resources/app.asar is still container / asar and expanded.
Real Hopper/Ghidra verification: not applicable, not run.

Compatibility, safety, and release

No breaking schema change. Callers that relied on a suffix-only apk / ipa / asar format for unread or non-container nested members now see unknown or file.

🤖 Generated with Claude Code

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/1553 **Author:** [@kaoru0822-kitauji](https://github.com/kaoru0822-kitauji) **Created:** 10/10/2026 **Status:** ✅ Merged **Merged:** 10/10/2026 **Merged by:** [@kaoru0822-kitauji](https://github.com/kaoru0822-kitauji) **Base:** `main` ← **Head:** `fix/inventory-root-format-bytes` --- ### 📝 Commits (1) - [`77604d3`](https://github.com/morluto/rea/commit/77604d35dfcbf6c2c23373bc9918b486edf4561b) fix(artifacts): take nested container families from bytes ### 📊 Changes **8 files changed** (+263 additions, -67 deletions) <details> <summary>View changed files</summary> 📝 `docs/android-analysis.md` (+4 -0) 📝 `src/artifacts/inventory/ArtifactGraphConstruction.ts` (+83 -31) 📝 `src/artifacts/inventory/classify.ts` (+4 -27) 📝 `src/artifacts/inventory/scanReader.ts` (+9 -2) 📝 `src/domain/android/androidApplication.ts` (+8 -1) 📝 `tests/boundary/filesystem/asarInventoryContainment.test.ts` (+39 -4) 📝 `tests/boundary/filesystem/zipDetection.test.ts` (+78 -1) 📝 `tests/composition/android/application.test.ts` (+38 -1) </details> ### 📄 Description ## Summary Fixes #1491. #1513 made root classification byte-first, so the root cases in the issue (ELF or APK bytes named `.dmg`, `.pkg`, `.apk` or `.asar`) already report the byte format on main. This PR covers what was still open: nested members, and the Android projection of a ZIP that only has an `.apk` name. ## Problem and expected behavior On `2d99e1e3`, a ZIP holding `inner-text.apk` (text), `inner-text.asar` (text) and a symlink entry `link.apk` gave these occurrences: | entry | before | after | |---|---|---| | `inner-text.apk` | `container` / `apk` | `resource` / `file` | | `inner-text.asar` | `container` / `asar` | `resource` / `file` | | `link.apk` (symlink) | `container` / `apk`, `not-hashed` | `unknown` / `unknown`, with a limitation | | `inner-real.apk` (ZIP bytes) | `container` / `apk` | unchanged | `classifyArtifactPath` returned a container family from the suffix, and byte classification only replaced it when the bytes were ZIP. Encrypted entries and symlinks were never read, so they kept the guess. A filesystem `.asar` without an ASAR header also failed the whole inventory as a malformed archive, while the same bytes as a root were already classified as `file`. A plain ZIP named `.apk` was projected as an Android application with no sign that no manifest was present. ## Change and scope - `classifyArtifactPath` no longer returns container families. Nested ZIP families follow ZIP magic (`zipPackageFormatForPath` only picks among them after magic). ASAR follows the header pickle check, now shared by root and nested classification (`hasAsarHeader` moved from `classify.ts` to `ArtifactGraphConstruction.ts`). - An unread member named like a container (encrypted entry, symlink, unavailable unpacked companion) gets `unknown` / `unknown` and the limitation `The path suffix suggests APK, but these bytes were not read, so the container family is unknown.` Other suffix role hints (`.js`, `.plist`, `.dylib`, ...) are unchanged. - Filesystem ASAR expansion requires the header. A `.asar` with a valid header but a malformed body still fails with `format`. - `project_android_application_graph` adds `No root AndroidManifest.xml occurrence was observed. The APK family comes from the .apk suffix of a ZIP archive, so these bytes may not be an APK.` when the root has no manifest. Its rejection names the observed root format (`... the inventory root format is zip. APK is chosen for a ZIP archive whose name ends in .apk.`). Not included: accepting a real APK named `.zip` in the Android projection. Its result contract has `root_format: "apk"`, and inferring APK from contents would be a separate contract decision. The error now says what to do. ## Contract and boundary impact - Semantic owner and earliest changed stage: artifact inventory occurrence classification (`src/artifacts/inventory`), then the Android domain projection. - CLI and MCP/tool-catalog contract: no schema change. `artifact_kind` / `artifact_format` values for affected nested members change as in the table above; existing enum values only. - Provider, bridge, target-format, or platform compatibility: none. - Evidence, artifact, provenance, or reconstruction contract: nested container family is now a byte observation; unread members report unknown with an explicit limitation. - Process execution, authorization, cleanup, or containment impact: a non-ASAR `.asar` file in an app directory is hashed as a file instead of being opened by the ASAR reader. - Generated metadata, package, or installation impact: none (`npm run docs:check` passes). ## Evidence and regression coverage - Tests added or updated: - `tests/boundary/filesystem/zipDetection.test.ts`: nested ZIP members by bytes (text `.apk`, ZIP `.apk`, ZIP `.bin`, encrypted `.ipa`) and an unfollowed filesystem symlink `link.apk`. - `tests/boundary/filesystem/asarInventoryContainment.test.ts`: real/ordinary/encrypted `.asar` members in a ZIP; a headerless filesystem `.asar` is a file; a header with malformed JSON is still rejected. The old test "keeps malformed filesystem archives rejected" pinned the headerless case as a failure and is replaced. - `tests/composition/android/application.test.ts`: the missing-manifest limitation and the rejection message. - Base reproduction: the issue comment's nested ZIP run through `rea inspect-artifact` on main and on this branch (table above). - Remaining proof gaps: no real APK/IPA/MSIX files were inventoried; fixtures are zip.js-built archives. For evidence-bearing changes: - [x] Observed, derived, and inferred claims remain distinguishable. - [x] Artifact identity, source provenance, and failed attempts remain preserved. - [x] Unsupported, incomplete, unavailable, or uncertain outcomes remain visible. ## Validation performed - `npm run check:fast` — passed. - `npm run check:changed` — 44 files, 444 tests passed. - `npm run test:focused --` all of `tests/boundary/filesystem`, `tests/boundary/providers/native`, `tests/composition`, `artifactMcp`, `mobileApplicationGraphMcp`, `cliOutput`, `nonBinaryComposition` — 1446 passed, 8 failed. The same 8 fail on unchanged main on this Mac (no `ibtool` with Command Line Tools only; the real `hdiutil` verify-refusal test). - `npm run docs:check` — passed. - Real app: `rea inspect-artifact /Applications/Discord.app` gives the same 580 occurrences (kind, format, hash status, limitations) on main and on this branch; `Contents/Resources/app.asar` is still `container` / `asar` and expanded. - Real Hopper/Ghidra verification: not applicable, not run. ## Compatibility, safety, and release No breaking schema change. Callers that relied on a suffix-only `apk` / `ipa` / `asar` format for unread or non-container nested members now see `unknown` or `file`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-17 21:12:21 +00:00