[PR #7106] [MERGED] fix(artifacts): admit raw UDRW disk images #7646

Closed
opened 2026-11-03 13:39:20 +00:00 by bsakaguchi · 0 comments
Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/7106
Author: @kaoru0822-kitauji
Created: 10/25/2026
Status: ✅ Merged
Merged: 10/25/2026
Merged by: @kaoru0822-kitauji

Base: main ← Head: fix/5687-udrw-dmg


📝 Commits (3)

  • 1c5559b fix(artifacts): admit raw UDRW disk images
  • 366b718 Merge remote-tracking branch 'origin/main' into fix/5687-udrw-dmg
  • 4f3aa4e fix(artifacts): read disk image signature windows at the recognizer

📊 Changes

8 files changed (+496 additions, -64 deletions)

View changed files

📝 src/application/BinaryTargetResolver.ts (+32 -28)
📝 src/artifacts/NativeDmgArtifactReader.ts (+20 -0)
📝 src/artifacts/inventory/classify.ts (+21 -31)
➕ src/domain/apple/diskImageFormat.ts (+86 -0)
➕ tests/boundary/cli/rawDiskImageCli.test.ts (+133 -0)
📝 tests/boundary/filesystem/artifactInventoryDescriptorCleanup.test.ts (+26 -5)
📝 tests/boundary/filesystem/zipDetection.test.ts (+126 -0)
📝 tests/boundary/providers/native/nativeDmgArtifactReader.test.ts (+52 -0)

📄 Description

What broke: inspect-artifact refused native-mountable raw (UDRW/UDTO) .dmg images as "unsupported binary format", because both target admission and root inventory required the UDIF koly trailer, and the reader treated hdiutil's "has no checksum" as a verify failure.
Fix: One domain owner (src/domain/apple/diskImageFormat.ts) recognizes disk images by structure for both boundaries. The reader treats the untranslated "has no checksum" status line as an explicit absent checksum, attaches read-only, and adds a limitation saying integrity was not verified.
Risk: Low to moderate. .dmg-suffixed files with a valid GPT/APM/MBR/HFS+ structure now reach hdiutil attach rather than being refused up front. Zero-filled/structureless blobs, checksum mismatches and unrecognized images are still refused.

Summary

Fixes #5687. A detached HFS+ UDRW image (CRawDiskImage, no koly trailer, no stored checksum) now gets a DMG root plus contained-file inventory, a recorded nonzero hdiutil verify with the read-only attach, an explicit "integrity was not verified" limitation, and its owned mount released. UDZO/UDRO behavior is unchanged.

Problem and expected behavior

  • Trigger: hdiutil create -format UDRW -fs HFS+ -srcfolder … fixture.dmg, then rea inspect-artifact fixture.dmg --json → target_unavailable / file_format_rejected before any reader runs.
  • Violated invariant: a recognized Apple disk image should not get a generic unsupported-binary refusal. A missing checksum must not be reported as a verified one or as a mismatch.
  • Recognition alone would not have been enough. hdiutil verify exits 1 with hdiutil: verify: "<path>" has no checksum., and the reader turned that into a failure before attaching.

Change and scope

  • hasDiskImageSignature(size, read) is the single owner of DMG structure recognition. It reads its own trailer and leading 1028-byte windows through the caller's filling positional read, so both boundaries observe the same bytes. It accepts the UDIF koly trailer or a raw device layout as written by hdiutil (GPT header rev 1.0 at LBA 1, Apple partition map ER/PM, MBR boot signature with a well-formed used table, or an unpartitioned HFS+ H+/v4 or HFSX HX/v5 volume header). BinaryTargetResolver and inventory/classify.ts both call it, and their duplicate koly probes are removed. The existing .dmg suffix requirement is kept.
  • Content gating is needed because hdiutil itself treats any file as a raw image with "no checksum" (observed for a 1 MiB zero file and a random file). Mountability is not used as evidence.
  • NativeDmgArtifactReader: the untranslated has no checksum. status line (a literal in /usr/bin/hdiutil, not localized) becomes checksumAbsent. Verify is recorded with its exit code, there is no retry, attach goes ahead, and entries() reports the limitation through onCoverage. An is INVALID line still takes precedence and stays an integrity failure.
  • The positional range reads that feed these probes (readHandleRange in the resolver, the classify read callback shared with the trailing-ZIP locator) now fill until EOF, as the PositionalRead contract expects. Target admission also fills its 4 KiB format-probe prefix. An existing controlled short-read regression caught that the old single-read callback could miss the trailer. A review finding showed that an unfilled admission prefix could hide raw signatures.
  • Not included: .cdr/.iso suffix admission, raw APFS containers without a partition map (hdiutil cannot create one), ISO 9660 hybrids, 4K-sector GPT, and FAT/exFAT without a partition map. Raw UDTO bytes saved as .dmg are covered by the GPT rule.
  • Contribution scope: not applicable. This is an ordinary fix inside the existing read-only DMG boundary.

Contract and boundary impact

  • Semantic owner and earliest changed stage: src/domain/apple/diskImageFormat.ts (format recognition), consumed by target admission and root inventory classification. Checksum-absent policy is owned by NativeDmgArtifactReader. The superseded hasKolyTrailer and the resolver's inline trailer probe are removed.

  • Contribution scope fit and linked maintainer decision: not applicable (fix within the existing boundary).

  • CLI and MCP/tool-catalog contract: no schema change. The CLI was exercised. MCP shares the same resolver and inventory layers but was not run.

  • Provider, bridge, target-format, or platform compatibility: more .dmg variants are admitted on macOS. On other hosts the root is classified dmg with the existing "requires a native macOS adapter" limitation.

  • Evidence, artifact, provenance, or reconstruction contract: provenance keeps the failed verify (exit 1) before attach. The new limitation states the absent checksum, and the root SHA-256 remains the byte identity.

  • Target selection, authority, upstream behavior, and cleanup details: same read-only -nobrowse attach and owned detach. hdiutil info showed no images left attached after the runs below.

  • Generated metadata, package, or installation impact: none.

  • Read existing files or captured data.

Evidence and regression coverage

  • Tests added or updated:

    • tests/boundary/filesystem/zipDetection.test.ts: GPT, APM, MBR, HFS+ and HFSX layouts admitted as dmg at both boundaries; the same bytes named .img are not. Zero-filled, empty-table MBR, bad MBR status, unknown GPT revision, short GPT header size, ER without PM, H+ with version 5, truncated HFS+ signature and text blobs are refused at both boundaries.
    • tests/boundary/providers/native/nativeDmgArtifactReader.test.ts: simulated host. "has no checksum" means one verify with no retry, then attach and entries, an onCoverage limitation, provenance [verify 1, attach 0], and detach on close.
    • tests/boundary/filesystem/artifactInventoryDescriptorCleanup.test.ts: the controlled 3-byte short-read regression now includes a raw GPT .dmg and checks target admission (parseBinaryTarget) as well as inventory. Before the review fix it failed at admission.
    • tests/boundary/cli/rawDiskImageCli.test.ts (darwin, real hdiutil): detached UDRW inventory with the limitation and provenance, the UDZO control without the limitation, a zero-filled .dmg refused with file_format_rejected, and no image left attached.
  • Base reproduction: on origin/main 160d821ac, the built CLI returned target_unavailable/file_format_rejected for the UDRW fixture. With the source changes stashed, the 5 new admission cases and the reader case fail (6 failed / 84 passed).

  • User-visible CLI output for the UDRW fixture after the fix (summarized from --json): root_format: dmg, coverage container-entries/complete, occurrences fixture.dmg/RawVerifierFixture/hello.txt, provenance verify exit 1 → attach exit 0, limitation: hdiutil verify found no stored checksum in this disk image, so its integrity was not verified; the root SHA-256 identifies the inventoried bytes.

  • Manual real-hdiutil matrix (built CLI): UDRW GPT/HFS+, UDRW -layout NONE HFS+ and HFSX, UDRW APM (SPUD), UDRW MBR/FAT, UDRW GPT/APFS and UDTO bytes as .dmg all inventoried their contents. UDZO and UDRO controls were unchanged (verify exit 0, no limitation). A GPT partition map with a zeroed filesystem was admitted, attach failed with "no mountable file systems", and the existing retained-root partial observation was returned with nothing left attached. Zero-filled and random .dmg files were refused as before.

  • Remaining proof gaps: MCP not run. The checksum-absent limitation is not carried into the retained-root snapshot when attach fails after an absent checksum; that snapshot's provenance still shows verify exit 1. The DMG reader still does not forward the inner directory reader's onCoverage; this is pre-existing and unchanged here. Encrypted and sparse images were not tested.

  • Observed, derived, and inferred claims remain distinguishable.

  • Artifact identity, source provenance, and failed attempts remain preserved.

  • Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

  • npm run test:focused -- tests/boundary/filesystem/zipDetection.test.ts tests/boundary/providers/native/nativeDmgArtifactReader.test.ts tests/boundary/cli/rawDiskImageCli.test.ts: 92 passed (macOS, real hdiutil)
  • npm run test:focused -- 23 related files (DMG reader/protocol/root observation/cancellation, descriptor cleanup, target admission/diagnostics/cancellation/raw-binary/godot/macho/DOL/DOS COM/dispatch strings, archive inventory/safety/failure detail, ZIP integrity, extraction format, Apple application, raw disk image CLI): 349 passed, after merging origin/main
  • Same new tests with source changes stashed: 6 failed (expected)
  • npm run check:fast: passed
  • npm run format:check: passed
  • Manual node scripts/rea.mjs inspect-artifact <image> --json matrix above, plus hdiutil info showing no images left attached

Compatibility, safety, and release

  • Breaking changes or migration steps: none.
  • Real Hopper/Ghidra, browser, or OS coverage: macOS (Darwin 25.6, arm64) with system /usr/bin/hdiutil. Hopper/Ghidra not involved.
  • Package or release metadata impact: none.
  • Security, privacy, process, or containment review: no new commands or authority. The same read-only, -nobrowse, owned-mount-root attach is used. Structureless blobs are still refused before any hdiutil call.

Author readiness

Human owner: @kaoru0822-kitauji

  • I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
  • I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
  • I declare this PR ready for initial maintainer review.

Before merge

  • This PR addresses a concrete problem or an agreed enhancement.
  • Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
  • The PR has one focused outcome and the title follows type(scope): outcome.
  • Related issue is linked, or the reason for not linking one is stated above.
  • Tests cover changed observable behavior and meaningful failure paths.
  • Owning docs, contracts, and generated metadata are updated where needed.
  • User-visible CLI/MCP changes include representative output.
  • I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

🤖 Generated with Claude Code


🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/7106 **Author:** [@kaoru0822-kitauji](https://github.com/kaoru0822-kitauji) **Created:** 10/25/2026 **Status:** ✅ Merged **Merged:** 10/25/2026 **Merged by:** [@kaoru0822-kitauji](https://github.com/kaoru0822-kitauji) **Base:** `main` ← **Head:** `fix/5687-udrw-dmg` --- ### 📝 Commits (3) - [`1c5559b`](https://github.com/morluto/rea/commit/1c5559b8bfa3ecf3c747ba1ad0fe6c32f9437d85) fix(artifacts): admit raw UDRW disk images - [`366b718`](https://github.com/morluto/rea/commit/366b71868c4f3681a864fd947e6767a46a330521) Merge remote-tracking branch 'origin/main' into fix/5687-udrw-dmg - [`4f3aa4e`](https://github.com/morluto/rea/commit/4f3aa4ea517745e3d6fecfc94eb1c2f45866f621) fix(artifacts): read disk image signature windows at the recognizer ### 📊 Changes **8 files changed** (+496 additions, -64 deletions) <details> <summary>View changed files</summary> 📝 `src/application/BinaryTargetResolver.ts` (+32 -28) 📝 `src/artifacts/NativeDmgArtifactReader.ts` (+20 -0) 📝 `src/artifacts/inventory/classify.ts` (+21 -31) ➕ `src/domain/apple/diskImageFormat.ts` (+86 -0) ➕ `tests/boundary/cli/rawDiskImageCli.test.ts` (+133 -0) 📝 `tests/boundary/filesystem/artifactInventoryDescriptorCleanup.test.ts` (+26 -5) 📝 `tests/boundary/filesystem/zipDetection.test.ts` (+126 -0) 📝 `tests/boundary/providers/native/nativeDmgArtifactReader.test.ts` (+52 -0) </details> ### 📄 Description **What broke:** `inspect-artifact` refused native-mountable raw (UDRW/UDTO) `.dmg` images as "unsupported binary format", because both target admission and root inventory required the UDIF `koly` trailer, and the reader treated hdiutil's "has no checksum" as a verify failure. **Fix:** One domain owner (`src/domain/apple/diskImageFormat.ts`) recognizes disk images by structure for both boundaries. The reader treats the untranslated "has no checksum" status line as an explicit absent checksum, attaches read-only, and adds a limitation saying integrity was not verified. **Risk:** Low to moderate. `.dmg`-suffixed files with a valid GPT/APM/MBR/HFS+ structure now reach `hdiutil attach` rather than being refused up front. Zero-filled/structureless blobs, checksum mismatches and unrecognized images are still refused. ## Summary Fixes #5687. A detached HFS+ UDRW image (`CRawDiskImage`, no `koly` trailer, no stored checksum) now gets a DMG root plus contained-file inventory, a recorded nonzero `hdiutil verify` with the read-only `attach`, an explicit "integrity was not verified" limitation, and its owned mount released. UDZO/UDRO behavior is unchanged. ## Problem and expected behavior - Trigger: `hdiutil create -format UDRW -fs HFS+ -srcfolder … fixture.dmg`, then `rea inspect-artifact fixture.dmg --json` → `target_unavailable` / `file_format_rejected` before any reader runs. - Violated invariant: a recognized Apple disk image should not get a generic unsupported-binary refusal. A missing checksum must not be reported as a verified one or as a mismatch. - Recognition alone would not have been enough. `hdiutil verify` exits 1 with `hdiutil: verify: "<path>" has no checksum.`, and the reader turned that into a failure before attaching. ## Change and scope - `hasDiskImageSignature(size, read)` is the single owner of DMG structure recognition. It reads its own trailer and leading 1028-byte windows through the caller's filling positional read, so both boundaries observe the same bytes. It accepts the UDIF `koly` trailer or a raw device layout as written by hdiutil (GPT header rev 1.0 at LBA 1, Apple partition map `ER`/`PM`, MBR boot signature with a well-formed used table, or an unpartitioned HFS+ `H+`/v4 or HFSX `HX`/v5 volume header). `BinaryTargetResolver` and `inventory/classify.ts` both call it, and their duplicate `koly` probes are removed. The existing `.dmg` suffix requirement is kept. - Content gating is needed because hdiutil itself treats any file as a raw image with "no checksum" (observed for a 1 MiB zero file and a random file). Mountability is not used as evidence. - `NativeDmgArtifactReader`: the untranslated `has no checksum.` status line (a literal in `/usr/bin/hdiutil`, not localized) becomes `checksumAbsent`. Verify is recorded with its exit code, there is no retry, attach goes ahead, and `entries()` reports the limitation through `onCoverage`. An `is INVALID` line still takes precedence and stays an integrity failure. - The positional range reads that feed these probes (`readHandleRange` in the resolver, the classify read callback shared with the trailing-ZIP locator) now fill until EOF, as the `PositionalRead` contract expects. Target admission also fills its 4 KiB format-probe prefix. An existing controlled short-read regression caught that the old single-read callback could miss the trailer. A review finding showed that an unfilled admission prefix could hide raw signatures. - Not included: `.cdr`/`.iso` suffix admission, raw APFS containers without a partition map (hdiutil cannot create one), ISO 9660 hybrids, 4K-sector GPT, and FAT/exFAT without a partition map. Raw UDTO bytes saved as `.dmg` are covered by the GPT rule. - Contribution scope: not applicable. This is an ordinary fix inside the existing read-only DMG boundary. ## Contract and boundary impact - Semantic owner and earliest changed stage: `src/domain/apple/diskImageFormat.ts` (format recognition), consumed by target admission and root inventory classification. Checksum-absent policy is owned by `NativeDmgArtifactReader`. The superseded `hasKolyTrailer` and the resolver's inline trailer probe are removed. - Contribution scope fit and linked maintainer decision: not applicable (fix within the existing boundary). - CLI and MCP/tool-catalog contract: no schema change. The CLI was exercised. MCP shares the same resolver and inventory layers but was not run. - Provider, bridge, target-format, or platform compatibility: more `.dmg` variants are admitted on macOS. On other hosts the root is classified `dmg` with the existing "requires a native macOS adapter" limitation. - Evidence, artifact, provenance, or reconstruction contract: provenance keeps the failed `verify` (exit 1) before `attach`. The new limitation states the absent checksum, and the root SHA-256 remains the byte identity. - Target selection, authority, upstream behavior, and cleanup details: same read-only `-nobrowse` attach and owned detach. `hdiutil info` showed no images left attached after the runs below. - Generated metadata, package, or installation impact: none. - [x] Read existing files or captured data. ## Evidence and regression coverage - Tests added or updated: - `tests/boundary/filesystem/zipDetection.test.ts`: GPT, APM, MBR, HFS+ and HFSX layouts admitted as `dmg` at both boundaries; the same bytes named `.img` are not. Zero-filled, empty-table MBR, bad MBR status, unknown GPT revision, short GPT header size, `ER` without `PM`, `H+` with version 5, truncated HFS+ signature and text blobs are refused at both boundaries. - `tests/boundary/providers/native/nativeDmgArtifactReader.test.ts`: simulated host. "has no checksum" means one verify with no retry, then attach and entries, an `onCoverage` limitation, provenance `[verify 1, attach 0]`, and detach on close. - `tests/boundary/filesystem/artifactInventoryDescriptorCleanup.test.ts`: the controlled 3-byte short-read regression now includes a raw GPT `.dmg` and checks target admission (`parseBinaryTarget`) as well as inventory. Before the review fix it failed at admission. - `tests/boundary/cli/rawDiskImageCli.test.ts` (darwin, real hdiutil): detached UDRW inventory with the limitation and provenance, the UDZO control without the limitation, a zero-filled `.dmg` refused with `file_format_rejected`, and no image left attached. - Base reproduction: on `origin/main` 160d821ac, the built CLI returned `target_unavailable`/`file_format_rejected` for the UDRW fixture. With the source changes stashed, the 5 new admission cases and the reader case fail (6 failed / 84 passed). - User-visible CLI output for the UDRW fixture after the fix (summarized from `--json`): `root_format: dmg`, coverage `container-entries/complete`, occurrences `fixture.dmg/RawVerifierFixture/hello.txt`, provenance `verify` exit 1 → `attach` exit 0, limitation: `hdiutil verify found no stored checksum in this disk image, so its integrity was not verified; the root SHA-256 identifies the inventoried bytes.` - Manual real-hdiutil matrix (built CLI): UDRW GPT/HFS+, UDRW `-layout NONE` HFS+ and HFSX, UDRW APM (`SPUD`), UDRW MBR/FAT, UDRW GPT/APFS and UDTO bytes as `.dmg` all inventoried their contents. UDZO and UDRO controls were unchanged (verify exit 0, no limitation). A GPT partition map with a zeroed filesystem was admitted, attach failed with "no mountable file systems", and the existing retained-root partial observation was returned with nothing left attached. Zero-filled and random `.dmg` files were refused as before. - Remaining proof gaps: MCP not run. The checksum-absent limitation is not carried into the retained-root snapshot when attach fails after an absent checksum; that snapshot's provenance still shows verify exit 1. The DMG reader still does not forward the inner directory reader's `onCoverage`; this is pre-existing and unchanged here. Encrypted and sparse images were not tested. - [x] Observed, derived, and inferred claims remain distinguishable. - [x] Artifact identity, source provenance, and failed attempts remain preserved. - [x] Unsupported, incomplete, unavailable, or uncertain outcomes remain visible. ## Validation performed - `npm run test:focused -- tests/boundary/filesystem/zipDetection.test.ts tests/boundary/providers/native/nativeDmgArtifactReader.test.ts tests/boundary/cli/rawDiskImageCli.test.ts`: 92 passed (macOS, real hdiutil) - `npm run test:focused --` 23 related files (DMG reader/protocol/root observation/cancellation, descriptor cleanup, target admission/diagnostics/cancellation/raw-binary/godot/macho/DOL/DOS COM/dispatch strings, archive inventory/safety/failure detail, ZIP integrity, extraction format, Apple application, raw disk image CLI): 349 passed, after merging origin/main - Same new tests with source changes stashed: 6 failed (expected) - `npm run check:fast`: passed - `npm run format:check`: passed - Manual `node scripts/rea.mjs inspect-artifact <image> --json` matrix above, plus `hdiutil info` showing no images left attached ## Compatibility, safety, and release - Breaking changes or migration steps: none. - Real Hopper/Ghidra, browser, or OS coverage: macOS (Darwin 25.6, arm64) with system `/usr/bin/hdiutil`. Hopper/Ghidra not involved. - Package or release metadata impact: none. - Security, privacy, process, or containment review: no new commands or authority. The same read-only, `-nobrowse`, owned-mount-root attach is used. Structureless blobs are still refused before any hdiutil call. ## Author readiness Human owner: @kaoru0822-kitauji - [x] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [x] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [x] I declare this PR ready for initial maintainer review. ## Before merge - [x] This PR addresses a concrete problem or an agreed enhancement. - [x] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff. - [x] The PR has one focused outcome and the title follows `type(scope): outcome`. - [x] Related issue is linked, or the reason for not linking one is stated above. - [x] Tests cover changed observable behavior and meaningful failure paths. - [x] Owning docs, contracts, and generated metadata are updated where needed. - [x] User-visible CLI/MCP changes include representative output. - [x] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
bsakaguchi 2026-11-03 13:39:20 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
morluto/rea#7646
No description provided.