morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #4688] [Bug] Apple application graph silently omits an unread file while reporting complete coverage #2446

New issue

Closed

opened 2026-10-17 21:04:47 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-17 21:04:47 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 17, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/4688

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

The public project-apple-application-graph CLI accepts genuine inventory Evidence for a generated IPA with an encrypted Payload/Fixture.app/libentry.dylib member. The inventory retains this file with artifact_id: null, hash_status: unavailable, and encrypted: true. The Apple graph omits the path entirely, returns an empty components.native_libraries array, and reports coverage.status: complete-within-inventory with inventory_complete: true. No limitation discloses the omitted member or says component absence is unknown.

An otherwise identical readable-member control retains the path in components.native_libraries. Both fixtures contain only harmless text; no application or member is executed, and the encrypted member is never decrypted.

Reproduction

Use the repository's pinned Node.js and npm toolchain. In a fresh checkout, run npm ci and npm run build:cached; both preparation commands were executed for the source build tested here. The same comparison was also executed from the installed npm package directory, which already contains the built CLI runtime.
From the repository root, run this inline fixture and public CLI comparison. It writes all generated files beneath a disposable directory and gives every REA CLI process only PATH and an isolated HOME, so client configuration overrides are absent.

work=$(mktemp -d)
mkdir "$work/home"
HOME="$work/home" node --input-type=module - "$work" <<'JS'
import { createRequire } from 'node:module';
import { writeFileSync } from 'node:fs';
import { join } from 'node:path';
import { spawnSync } from 'node:child_process';
const require = createRequire(join(process.cwd(), 'package.json'));
const { ZipWriter, Uint8ArrayWriter, TextReader } = require('@zip.js/zip.js');
const work = process.argv[2];
const member = 'Payload/Fixture.app/libentry.dylib';
function cli(...args) {
  const r = spawnSync(process.execPath,
    ['scripts/rea.mjs', ...args, '--format', 'json'], {
      encoding: 'utf8', env: { PATH: process.env.PATH, HOME: join(work, 'home') }
    });
  if (r.status !== 0) throw new Error(r.stderr + r.stdout);
  return JSON.parse(r.stdout);
}
for (const encrypted of [false, true]) {
  const label = encrypted ? 'encrypted' : 'readable';
  const writer = new ZipWriter(new Uint8ArrayWriter());
  await writer.add('Payload/Fixture.app/Info.plist',
    new TextReader('<plist><dict/></plist>'));
  await writer.add(member, new TextReader('harmless fixture text'),
    encrypted ? { password: 'fixture-only' } : {});
  const archive = join(work, `${label}.ipa`);
  writeFileSync(archive, await writer.close());
  for (let run = 1; run <= 2; run++) {
    const inspection = cli('inspect-artifact', archive);
    const inventory = inspection.normalized_result.substeps[0].evidence;
    const input = join(work, `${label}-input-${run}.json`);
    writeFileSync(input, JSON.stringify({ inventory_evidence: [inventory] }));
    const output = cli('project-apple-application-graph', input);
    writeFileSync(join(work, `${label}-inspection-${run}.json`),
      JSON.stringify(inspection, null, 2));
    writeFileSync(join(work, `${label}-projection-${run}.json`),
      JSON.stringify(output, null, 2));
    const graph = output.normalized_result;
    const occurrence = inventory.normalized_result.occurrences.find(
      item => item.logical_path === member);
    console.log(JSON.stringify({ encrypted, run,
      hash_status: occurrence.hash_status,
      null_identity: occurrence.artifact_id === null,
      inventory_coverage: inventory.normalized_result.coverage,
      native_paths: graph.components.native_libraries.map(item => item.path),
      path_in_result: JSON.stringify(graph).includes(member),
      coverage: graph.coverage,
      unread_notice: graph.limitations.some(text =>
        /unread|unavailable|omitt|absence is unknown/i.test(text)) }));
  }
}
JS

Compare the readable and encrypted lines and the saved inspection/projection results. On each tested build, the failure reproduced in both end-to-end repetitions of the encrypted case, and the readable control retained the member in both repetitions.

Expected behavior

The Apple guide promises every component in each inventory category and complete component arrays from supplied inventory pages. The repository also requires preserving omission and uncertainty across boundaries and retaining unknown identity as unknown rather than treating it as absence. These Apple and repository contracts apply to the rooted, non-sidecar file in this fixture independently of Android or Harmony behavior.

The projection should preserve the known file path and unavailable identity, or explicitly disclose its omission and unknown component absence. While it silently loses this supplied component fact, overall coverage should not imply a complete projection. Complete inventory member accounting can still legitimately remain inventory_complete: true. No hash or native-byte interpretation should be invented for unread content.

Impact and scope

The graph loses a known file's presence without warning for an inventory that the public inspector produces and the public projection accepts. On each tested build, this reproduced in 2/2 encrypted-case repetitions. The readable control retained the same path in 2/2 repetitions. The demonstrated scope is a generated ZIP-format IPA containing a rooted encrypted file; other unavailable-identity causes and runtime MCP behavior were not exercised.

Impact level: P2
Assessment status: supported
Impact basis: A bounded class of accepted inventories produces missing component data and misleading projection completeness, while the original inventory retains the facts and ordinary readable inputs continue to project.
Workaround status: available
Workaround basis: The public inspect-artifact output retains the exact member path, null identity, encryption flag, and unavailable hash status, so a caller can cross-check its inventory occurrences to preserve presence and uncertainty. This does not repair the graph output.

REA version

Fresh main at 5853089885 (built CLI reports 6.5.0), and latest published npm package rea-agents@6.5.0 installed separately.

Runtime and platform

Node v24.18.0; npm 11.16.0, matching the pinned toolchain; macOS 27.0.1; uname -sm: Darwin arm64. No optional native analysis provider is involved.

Target and artifact context

Two generated ZIP-format IPA fixtures each contain Payload/Fixture.app/Info.plist and Payload/Fixture.app/libentry.dylib. Both members contain inert text. Only the dylib member's archive encryption differs between variants. The producer reports complete container-entry coverage for both. The dylib suffix establishes the projection category in the readable control; this report makes no claim that the text is a valid Mach-O or executable library.

Interface

CLI

Earliest failure stage

Transport or result projection

Relevant output or evidence

Expected source: docs/apple-application-analysis.md:33 and :57-61; AGENTS.md:19 and :21.
Failure source: src/domain/apple/appleApplication.ts:123-146 and :170-210.
Evidence provenance: observed
Local verification: reproduced
Reproduction completeness: complete

The inline reproduction ran node scripts/rea.mjs inspect-artifact <fixture.ipa> --format json, extracted its inventory_artifact Evidence, and ran node scripts/rea.mjs project-apple-application-graph <input.json> --format json. All calls exited successfully. The smallest distinguishing output for the first repetition is:

{"encrypted":false,"run":1,"hash_status":"verified","null_identity":false,"inventory_coverage":{"scope":"container-entries","status":"complete"},"native_paths":["Payload/Fixture.app/libentry.dylib"],"path_in_result":true,"coverage":{"status":"complete-within-inventory","inventory_complete":true},"unread_notice":false}
{"encrypted":true,"run":1,"hash_status":"unavailable","null_identity":true,"inventory_coverage":{"scope":"container-entries","status":"complete"},"native_paths":[],"path_in_result":false,"coverage":{"status":"complete-within-inventory","inventory_complete":true},"unread_notice":false}

The second repetition and both repetitions on the separately installed release returned the same distinguishing facts. The encrypted inventory also directly reports entry_kind: file and encrypted: true. The complete projection limitations contain only the ordinary ZIP central-directory, plist, runtime, and bridge notices; none identifies the unread member or its omission.

Typed outcome or recovery expected

Repeat the same public inspector and projection calls for both variants. The encrypted file must remain visible as a known path with unavailable identity, or its omission and unknown component absence must be explicit. Coverage must reflect any remaining loss of supplied component information. The readable control must continue to retain its native-library category entry with complete coverage. A passing result must not invent a hash, assert valid native bytes, decrypt content, or require target execution.

Analysis or suspected cause

Source-derived reasoning: the Apple projection maps a null artifact identity to an undefined component and removes undefined components from its component input. Its completeness calculation checks inventory coverage and the DMG special case, while its limitations do not account for an unread file. The successful public CLI comparison demonstrates that the real producer's null-identity file passes the input guards and reaches this projection path.

Additional context

Issue #2603 covers the analogous Android omission and disclosure behavior. It is related precedent, not the Apple contract used here. Issue #3174 concerns authenticated extensionless Mach-O components outside application roots. This reproduction uses a rooted .dylib member with unavailable identity, so it exercises a different failure. Targeted searches of open and closed issues and PRs for Apple/IPA, unread or encrypted components, inventory, and projection found no direct duplicate. PR #3300 fixes the extensionless case, while open PRs #3504 and #3505 concern launchd plist classification and bundle-directory identity. This is a read-only data projection correctness report with no security boundary crossed.

Originally created by @coygeek on GitHub (Oct 17, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/4688 ### Before submitting - [X] I searched the open and closed issues for a related report. - [X] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior The public `project-apple-application-graph` CLI accepts genuine inventory Evidence for a generated IPA with an encrypted `Payload/Fixture.app/libentry.dylib` member. The inventory retains this file with `artifact_id: null`, `hash_status: unavailable`, and `encrypted: true`. The Apple graph omits the path entirely, returns an empty `components.native_libraries` array, and reports `coverage.status: complete-within-inventory` with `inventory_complete: true`. No limitation discloses the omitted member or says component absence is unknown. An otherwise identical readable-member control retains the path in `components.native_libraries`. Both fixtures contain only harmless text; no application or member is executed, and the encrypted member is never decrypted. ### Reproduction 1. Use the repository's pinned Node.js and npm toolchain. In a fresh checkout, run `npm ci` and `npm run build:cached`; both preparation commands were executed for the source build tested here. The same comparison was also executed from the installed npm package directory, which already contains the built CLI runtime. 2. From the repository root, run this inline fixture and public CLI comparison. It writes all generated files beneath a disposable directory and gives every REA CLI process only PATH and an isolated HOME, so client configuration overrides are absent. ```sh work=$(mktemp -d) mkdir "$work/home" HOME="$work/home" node --input-type=module - "$work" <<'JS' import { createRequire } from 'node:module'; import { writeFileSync } from 'node:fs'; import { join } from 'node:path'; import { spawnSync } from 'node:child_process'; const require = createRequire(join(process.cwd(), 'package.json')); const { ZipWriter, Uint8ArrayWriter, TextReader } = require('@zip.js/zip.js'); const work = process.argv[2]; const member = 'Payload/Fixture.app/libentry.dylib'; function cli(...args) { const r = spawnSync(process.execPath, ['scripts/rea.mjs', ...args, '--format', 'json'], { encoding: 'utf8', env: { PATH: process.env.PATH, HOME: join(work, 'home') } }); if (r.status !== 0) throw new Error(r.stderr + r.stdout); return JSON.parse(r.stdout); } for (const encrypted of [false, true]) { const label = encrypted ? 'encrypted' : 'readable'; const writer = new ZipWriter(new Uint8ArrayWriter()); await writer.add('Payload/Fixture.app/Info.plist', new TextReader('<plist><dict/></plist>')); await writer.add(member, new TextReader('harmless fixture text'), encrypted ? { password: 'fixture-only' } : {}); const archive = join(work, `${label}.ipa`); writeFileSync(archive, await writer.close()); for (let run = 1; run <= 2; run++) { const inspection = cli('inspect-artifact', archive); const inventory = inspection.normalized_result.substeps[0].evidence; const input = join(work, `${label}-input-${run}.json`); writeFileSync(input, JSON.stringify({ inventory_evidence: [inventory] })); const output = cli('project-apple-application-graph', input); writeFileSync(join(work, `${label}-inspection-${run}.json`), JSON.stringify(inspection, null, 2)); writeFileSync(join(work, `${label}-projection-${run}.json`), JSON.stringify(output, null, 2)); const graph = output.normalized_result; const occurrence = inventory.normalized_result.occurrences.find( item => item.logical_path === member); console.log(JSON.stringify({ encrypted, run, hash_status: occurrence.hash_status, null_identity: occurrence.artifact_id === null, inventory_coverage: inventory.normalized_result.coverage, native_paths: graph.components.native_libraries.map(item => item.path), path_in_result: JSON.stringify(graph).includes(member), coverage: graph.coverage, unread_notice: graph.limitations.some(text => /unread|unavailable|omitt|absence is unknown/i.test(text)) })); } } JS ``` 3. Compare the readable and encrypted lines and the saved inspection/projection results. On each tested build, the failure reproduced in both end-to-end repetitions of the encrypted case, and the readable control retained the member in both repetitions. ### Expected behavior The Apple guide promises every component in each inventory category and complete component arrays from supplied inventory pages. The repository also requires preserving omission and uncertainty across boundaries and retaining unknown identity as unknown rather than treating it as absence. These Apple and repository contracts apply to the rooted, non-sidecar file in this fixture independently of Android or Harmony behavior. The projection should preserve the known file path and unavailable identity, or explicitly disclose its omission and unknown component absence. While it silently loses this supplied component fact, overall coverage should not imply a complete projection. Complete inventory member accounting can still legitimately remain `inventory_complete: true`. No hash or native-byte interpretation should be invented for unread content. ### Impact and scope The graph loses a known file's presence without warning for an inventory that the public inspector produces and the public projection accepts. On each tested build, this reproduced in 2/2 encrypted-case repetitions. The readable control retained the same path in 2/2 repetitions. The demonstrated scope is a generated ZIP-format IPA containing a rooted encrypted file; other unavailable-identity causes and runtime MCP behavior were not exercised. - Impact level: P2 - Assessment status: supported - Impact basis: A bounded class of accepted inventories produces missing component data and misleading projection completeness, while the original inventory retains the facts and ordinary readable inputs continue to project. - Workaround status: available - Workaround basis: The public inspect-artifact output retains the exact member path, null identity, encryption flag, and unavailable hash status, so a caller can cross-check its inventory occurrences to preserve presence and uncertainty. This does not repair the graph output. ### REA version Fresh main at 58530898852d3dc5c7415afac1b92d3fec2f8fb5 (built CLI reports 6.5.0), and latest published npm package rea-agents@6.5.0 installed separately. ### Runtime and platform Node v24.18.0; npm 11.16.0, matching the pinned toolchain; macOS 27.0.1; `uname -sm`: Darwin arm64. No optional native analysis provider is involved. ### Target and artifact context Two generated ZIP-format IPA fixtures each contain `Payload/Fixture.app/Info.plist` and `Payload/Fixture.app/libentry.dylib`. Both members contain inert text. Only the dylib member's archive encryption differs between variants. The producer reports complete container-entry coverage for both. The dylib suffix establishes the projection category in the readable control; this report makes no claim that the text is a valid Mach-O or executable library. ### Interface CLI ### Earliest failure stage Transport or result projection ### Relevant output or evidence - Expected source: docs/apple-application-analysis.md:33 and :57-61; AGENTS.md:19 and :21. - Failure source: src/domain/apple/appleApplication.ts:123-146 and :170-210. - Evidence provenance: observed - Local verification: reproduced - Reproduction completeness: complete The inline reproduction ran `node scripts/rea.mjs inspect-artifact <fixture.ipa> --format json`, extracted its inventory_artifact Evidence, and ran `node scripts/rea.mjs project-apple-application-graph <input.json> --format json`. All calls exited successfully. The smallest distinguishing output for the first repetition is: ```json {"encrypted":false,"run":1,"hash_status":"verified","null_identity":false,"inventory_coverage":{"scope":"container-entries","status":"complete"},"native_paths":["Payload/Fixture.app/libentry.dylib"],"path_in_result":true,"coverage":{"status":"complete-within-inventory","inventory_complete":true},"unread_notice":false} {"encrypted":true,"run":1,"hash_status":"unavailable","null_identity":true,"inventory_coverage":{"scope":"container-entries","status":"complete"},"native_paths":[],"path_in_result":false,"coverage":{"status":"complete-within-inventory","inventory_complete":true},"unread_notice":false} ``` The second repetition and both repetitions on the separately installed release returned the same distinguishing facts. The encrypted inventory also directly reports `entry_kind: file` and `encrypted: true`. The complete projection limitations contain only the ordinary ZIP central-directory, plist, runtime, and bridge notices; none identifies the unread member or its omission. ### Typed outcome or recovery expected Repeat the same public inspector and projection calls for both variants. The encrypted file must remain visible as a known path with unavailable identity, or its omission and unknown component absence must be explicit. Coverage must reflect any remaining loss of supplied component information. The readable control must continue to retain its native-library category entry with complete coverage. A passing result must not invent a hash, assert valid native bytes, decrypt content, or require target execution. ### Analysis or suspected cause Source-derived reasoning: the Apple projection maps a null artifact identity to an undefined component and removes undefined components from its component input. Its completeness calculation checks inventory coverage and the DMG special case, while its limitations do not account for an unread file. The successful public CLI comparison demonstrates that the real producer's null-identity file passes the input guards and reaches this projection path. ### Additional context [Issue #2603](https://github.com/morluto/rea/issues/2603) covers the analogous Android omission and disclosure behavior. It is related precedent, not the Apple contract used here. [Issue #3174](https://github.com/morluto/rea/issues/3174) concerns authenticated extensionless Mach-O components outside application roots. This reproduction uses a rooted .dylib member with unavailable identity, so it exercises a different failure. Targeted searches of open and closed issues and PRs for Apple/IPA, unread or encrypted components, inventory, and projection found no direct duplicate. PR #3300 fixes the extensionless case, while open PRs #3504 and #3505 concern launchd plist classification and bundle-directory identity. This is a read-only data projection correctness report with no security boundary crossed.

bsakaguchi closed this issue

2026-10-17 21:04:48 +00:00

bsakaguchi commented

2026-10-17 21:04:48 +00:00

Author

Owner

@morluto commented on GitHub (Oct 17, 2026):

Fixed on main in c192c5881 (adapted from @rodricksz4h5's proposal and authored ZIP regressions in #4692; this is not a PR merge or readiness attestation).

Root cause: Apple discarded null-identity occurrence facts before classification and calculated completeness without accounting for unread files. The shared mobile occurrence owner now retains authenticated or unavailable facts once. Apple applies its existing root/AppleDouble attribution to unread files and slices, preserving path, format, encryption and limitations with null identity. Coverage becomes partial while inventory_complete still reports manifest count completeness. Directories and symlinks keep their separate semantics. Duplicate component schemas and old producer helpers were removed; Android/Harmony controls preserve complete replies and projection IDs byte for byte.

Validation at the pushed commit:

77 focused checks across seven files pass, including real ZIP inventory, built public inspect-artifact → project-apple-application-graph readable/encrypted controls, production MCP for all three mobile families, Evidence retention and converted catalog schema validity.
Non-encrypted unavailable file/slice, explicit directory, symlink, root attribution and AppleDouble controls pass. Four new ZIP assertions fail on the prior source.
Build, check:fast/typecheck/lint, docs:check (five README checks), and pre-push checks pass after rebasing onto e5ee3ac65. Two scoped source reviews found no actionable defects.

18 tiny completed source-projection cost controls separated first processes from two alternating repeats. Repeated Apple task medians: elapsed 48.30→47.85 ms, CPU 61.62→60.93 ms, one-process family peak RSS 75.81→75.51 MB, full reply 554→611 cl100k_base estimated tokens. These timings do not establish a performance improvement. Android/Harmony reply estimates remain 510/581 and their reply bytes are unchanged. Full public task/agent/catalog, large-input, failure and true-cold costs were not measured.

The additive unavailable_components field participates in Apple projection IDs, including when empty. Only authored harmless fixtures were inspected: no target execution, decryption, real deep provider, published npm verification or heavy profiling. Context7/DeepWiki zip.js research was checked against installed declarations and actual ZIP inventory controls.

@morluto commented on GitHub (Oct 17, 2026): Fixed on main in c192c5881 (adapted from @rodricksz4h5's proposal and authored ZIP regressions in #4692; this is not a PR merge or readiness attestation). Root cause: Apple discarded null-identity occurrence facts before classification and calculated completeness without accounting for unread files. The shared mobile occurrence owner now retains authenticated or unavailable facts once. Apple applies its existing root/AppleDouble attribution to unread files and slices, preserving path, format, encryption and limitations with null identity. Coverage becomes partial while inventory_complete still reports manifest count completeness. Directories and symlinks keep their separate semantics. Duplicate component schemas and old producer helpers were removed; Android/Harmony controls preserve complete replies and projection IDs byte for byte. Validation at the pushed commit: - 77 focused checks across seven files pass, including real ZIP inventory, built public inspect-artifact → project-apple-application-graph readable/encrypted controls, production MCP for all three mobile families, Evidence retention and converted catalog schema validity. - Non-encrypted unavailable file/slice, explicit directory, symlink, root attribution and AppleDouble controls pass. Four new ZIP assertions fail on the prior source. - Build, check:fast/typecheck/lint, docs:check (five README checks), and pre-push checks pass after rebasing onto e5ee3ac65. Two scoped source reviews found no actionable defects. 18 tiny completed source-projection cost controls separated first processes from two alternating repeats. Repeated Apple task medians: elapsed 48.30→47.85 ms, CPU 61.62→60.93 ms, one-process family peak RSS 75.81→75.51 MB, full reply 554→611 cl100k_base estimated tokens. These timings do not establish a performance improvement. Android/Harmony reply estimates remain 510/581 and their reply bytes are unchanged. Full public task/agent/catalog, large-input, failure and true-cold costs were not measured. The additive unavailable_components field participates in Apple projection IDs, including when empty. Only authored harmless fixtures were inspected: no target execution, decryption, real deep provider, published npm verification or heavy profiling. Context7/DeepWiki zip.js research was checked against installed declarations and actual ZIP inventory controls.

bsakaguchi referenced this issue

2026-10-31 00:40:47 +00:00

[GH-ISSUE #5718] [Bug] inspect-binary-layout allocates the declared BSS and fails on a small valid ELF #3466

bsakaguchi referenced this issue

2026-10-31 01:05:16 +00:00

[PR #2486] [MERGED] fix(pwntools): let callers widen the artifact input bound #5265

bsakaguchi referenced this issue

2026-10-31 01:05:35 +00:00

[PR #2676] [CLOSED] fix(native): support configurable REA_PWNTOOLS_MAX_INPUT_BYTES for large artifacts #5305