morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #2548] fix(ida): headless input includes bytes beyond the selected Mach-O slice #866

New issue

Closed

opened 2026-10-15 18:29:07 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-15 18:29:07 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 13, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/2548

Summary

Opening an admitted universal Mach-O for headless IDA can copy bytes beyond the selected architecture's slice into the private analysis input. A 200000-byte fixture with an admitted 1000-byte arm64 slice produced a 2100-byte private input. REA hashes the longer copy and accepts it through the headless session adapter, despite its promise to give the provider exactly the selected slice. The original container remains unchanged.

Steps to reproduce

Use the source checkout and installed dependencies described under Runtime or environment. No IDA installation is required to reproduce the filesystem-copy failure.
Create src/ida/universalCopyRegression.test.ts with the fixture below. It declares two in-bounds, nonoverlapping FAT entries with matching Mach-O headers, selects arm64 through the normal resolver, and checks the resulting private input.

import { createHash } from "node:crypto";
import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { expect, it } from "vitest";
import { parseBinaryTarget } from "../application/BinaryTargetResolver.js";
import { thinMach } from "../domain/binaryTarget.fixture.js";
import { IdaWorkspace } from "./IdaWorkspace.js";

it("copies exactly the admitted arm64 slice", async () => {
  const root = await mkdtemp(join(tmpdir(), "universal-copy-"));
  try {
    const bytes = Buffer.alloc(200000);
    for (let i = 0; i < bytes.length; i++) bytes[i] = i % 251;
    bytes.fill(0, 0, 48);
    bytes.writeUInt32BE(0xcafebabe, 0);
    bytes.writeUInt32BE(2, 4);
    const slices = [
      { cpu: 0x0100000c, offset: 100, size: 1000 },
      { cpu: 0x01000007, offset: 131072, size: 68928 },
    ];
    slices.forEach(({ cpu, offset, size }, index) => {
      const entry = 8 + index * 20;
      bytes.writeUInt32BE(cpu, entry);
      bytes.writeUInt32BE(offset, entry + 8);
      bytes.writeUInt32BE(size, entry + 12);
      const header = thinMach(0xfeedfacf, cpu);
      header.writeUInt32BE(2, 12);
      header.copy(bytes, offset);
    });
    const path = join(root, "universal");
    await writeFile(path, bytes);
    const admitted = await parseBinaryTarget(path, {
      hostArchitecture: "arm64",
    });
    if (!admitted.ok) throw admitted.error;
    expect(admitted.value).toMatchObject({
      format: "mach-o-universal",
      architecture: "arm64",
      universalSlice: { offset: 100, size: 1000 },
    });
    const workspace = await IdaWorkspace.allocate(admitted.value, root);
    await workspace.copyInput();
    const input = await readFile(workspace.inputPath);
    const expected = bytes.subarray(100, 1100);
    console.log({ copiedLength: input.length, expectedLength: expected.length });
    expect(input.equals(expected)).toBe(true);
    expect(workspace.inputSha256).toBe(
      createHash("sha256").update(expected).digest("hex"),
    );
  } finally {
    await rm(root, { recursive: true, force: true });
  }
});

Run the focused test. The equality assertion should detect the extra bytes on the tested source baseline.

npm run test:focused -- src/ida/universalCopyRegression.test.ts

The verifier ran this fixture through the same admission and copy operations in observation tests that asserted the current incorrect output. The inline restoration assertion above is a maintainer reproduction recipe; that exact assertion was not run by the verifier.

Expected behavior

The private headless input contains exactly the admitted arm64 slice, source range [100, 1100), with a length of 1000 bytes. Its SHA-256 equals the digest of those selected bytes. REA separately verifies the complete source container against the digest recorded during admission.

Actual behavior

The verifier observed a 2100-byte private input. The first 1000 bytes equal the selected slice, and the following 1100 bytes equal source range [65536, 66636). The private-input digest matches the 2100-byte file and differs from the slice digest. The unchanged source passes the whole-container digest check.

The normal headless session startup also passed the 2100-byte input to a recording connection's database-open operation and accepted its health reply. This establishes adapter reachability; it does not establish real IDA analysis results. The verifier launched no real IDA engine and observed no disassembly error or provider crash.

Evidence

Expected source: src/ida/IdaWorkspace.ts:58-62 promises selected-slice thinning; src/ida/IdaWorkspaceThinning.test.ts:48-59 requires exact slice bytes, size and digest.
Failure source: src/ida/IdaWorkspace.ts:82-91 computes a negative subarray end for a read beginning after the selected slice; the observed copy contains 1100 bytes from the next read. src/ida/IdaSessionClient.ts:437-445 uses that copy for headless startup, and its identity check at lines 528-535 accepts the digest of the same longer file.
Evidence provenance: observed
Local verification: reproduced
Reproduction completeness: complete

The focused verifier probe passed two observation tests. It confirmed actual admission, the precise extra byte range, digest behavior, session startup through the existing recording connection, an exact read-boundary control, and a separately admitted thin-input control. All five existing workspace-thinning tests also passed. Those existing universal fixtures fit inside one read and do not exercise the reproduced trigger.

The copy loop reads 64 KiB chunks. For the reproduced slice, the read starting at 65536 passes an end of -64436 to Buffer.subarray, which interprets it relative to that read's length and yields a 1100-byte prefix. A subsequent nonempty-length check allows those bytes to be written. The failure depends on the later read's length and the selected slice end; being a non-final slice alone does not guarantee extra output.

Related report: morluto/rea#2285, an open report by another author of the same copy defect. Its proposed fix, morluto/rea#2291, was open and unmerged when checked. Both are references; the local observation above independently establishes this baseline's failure.

Restoration check

For the admitted fixture above, require a 1000-byte private input equal to source range [100, 1100) and a digest equal to those bytes. Keep whole-source change rejection intact. Also check a slice crossing a read boundary, a slice ending exactly at a read boundary, a slice ending in the final read, and a separately admitted thin target. The controls must preserve the same exact-byte contract rather than merely accept the longer copy's digest.

Runtime or environment

Failure verified at a2d8b94dd5 and still present at 8b181716ac.
Node.js v26.10.0; npm 11.19.1; Vitest 5.0.3; macOS arm64.
Synthetic two-architecture Mach-O container; filesystem-copy and session-adapter verification with the existing recording connection. No real IDA engine was used.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: An admitted universal input produces an incorrect private analysis file and selected-slice digest. The failure is consequential but bounded to particular read and slice layouts; the verifier did not establish incorrect analysis results, a crash, or source-data loss.
Workaround status: available
Workaround basis: The verifier extracted the selected bytes into a separate thin Mach-O, admitted that file through the normal resolver, and confirmed that its private input and digest exactly preserve those bytes. This preparation avoids the failing universal-copy path for the tested fixture.

Originally created by @coygeek on GitHub (Oct 13, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/2548 ## Summary Opening an admitted universal Mach-O for headless IDA can copy bytes beyond the selected architecture's slice into the private analysis input. A 200000-byte fixture with an admitted 1000-byte arm64 slice produced a 2100-byte private input. REA hashes the longer copy and accepts it through the headless session adapter, despite its promise to give the provider exactly the selected slice. The original container remains unchanged. ## Steps to reproduce 1. Use the source checkout and installed dependencies described under Runtime or environment. No IDA installation is required to reproduce the filesystem-copy failure. 2. Create `src/ida/universalCopyRegression.test.ts` with the fixture below. It declares two in-bounds, nonoverlapping FAT entries with matching Mach-O headers, selects arm64 through the normal resolver, and checks the resulting private input. ```ts import { createHash } from "node:crypto"; import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { expect, it } from "vitest"; import { parseBinaryTarget } from "../application/BinaryTargetResolver.js"; import { thinMach } from "../domain/binaryTarget.fixture.js"; import { IdaWorkspace } from "./IdaWorkspace.js"; it("copies exactly the admitted arm64 slice", async () => { const root = await mkdtemp(join(tmpdir(), "universal-copy-")); try { const bytes = Buffer.alloc(200000); for (let i = 0; i < bytes.length; i++) bytes[i] = i % 251; bytes.fill(0, 0, 48); bytes.writeUInt32BE(0xcafebabe, 0); bytes.writeUInt32BE(2, 4); const slices = [ { cpu: 0x0100000c, offset: 100, size: 1000 }, { cpu: 0x01000007, offset: 131072, size: 68928 }, ]; slices.forEach(({ cpu, offset, size }, index) => { const entry = 8 + index * 20; bytes.writeUInt32BE(cpu, entry); bytes.writeUInt32BE(offset, entry + 8); bytes.writeUInt32BE(size, entry + 12); const header = thinMach(0xfeedfacf, cpu); header.writeUInt32BE(2, 12); header.copy(bytes, offset); }); const path = join(root, "universal"); await writeFile(path, bytes); const admitted = await parseBinaryTarget(path, { hostArchitecture: "arm64", }); if (!admitted.ok) throw admitted.error; expect(admitted.value).toMatchObject({ format: "mach-o-universal", architecture: "arm64", universalSlice: { offset: 100, size: 1000 }, }); const workspace = await IdaWorkspace.allocate(admitted.value, root); await workspace.copyInput(); const input = await readFile(workspace.inputPath); const expected = bytes.subarray(100, 1100); console.log({ copiedLength: input.length, expectedLength: expected.length }); expect(input.equals(expected)).toBe(true); expect(workspace.inputSha256).toBe( createHash("sha256").update(expected).digest("hex"), ); } finally { await rm(root, { recursive: true, force: true }); } }); ``` 3. Run the focused test. The equality assertion should detect the extra bytes on the tested source baseline. ```sh npm run test:focused -- src/ida/universalCopyRegression.test.ts ``` The verifier ran this fixture through the same admission and copy operations in observation tests that asserted the current incorrect output. The inline restoration assertion above is a maintainer reproduction recipe; that exact assertion was not run by the verifier. ## Expected behavior The private headless input contains exactly the admitted arm64 slice, source range [100, 1100), with a length of 1000 bytes. Its SHA-256 equals the digest of those selected bytes. REA separately verifies the complete source container against the digest recorded during admission. ## Actual behavior The verifier observed a 2100-byte private input. The first 1000 bytes equal the selected slice, and the following 1100 bytes equal source range [65536, 66636). The private-input digest matches the 2100-byte file and differs from the slice digest. The unchanged source passes the whole-container digest check. The normal headless session startup also passed the 2100-byte input to a recording connection's database-open operation and accepted its health reply. This establishes adapter reachability; it does not establish real IDA analysis results. The verifier launched no real IDA engine and observed no disassembly error or provider crash. ## Evidence - Expected source: `src/ida/IdaWorkspace.ts:58-62` promises selected-slice thinning; `src/ida/IdaWorkspaceThinning.test.ts:48-59` requires exact slice bytes, size and digest. - Failure source: `src/ida/IdaWorkspace.ts:82-91` computes a negative subarray end for a read beginning after the selected slice; the observed copy contains 1100 bytes from the next read. `src/ida/IdaSessionClient.ts:437-445` uses that copy for headless startup, and its identity check at lines 528-535 accepts the digest of the same longer file. - Evidence provenance: observed - Local verification: reproduced - Reproduction completeness: complete The focused verifier probe passed two observation tests. It confirmed actual admission, the precise extra byte range, digest behavior, session startup through the existing recording connection, an exact read-boundary control, and a separately admitted thin-input control. All five existing workspace-thinning tests also passed. Those existing universal fixtures fit inside one read and do not exercise the reproduced trigger. The copy loop reads 64 KiB chunks. For the reproduced slice, the read starting at 65536 passes an end of -64436 to Buffer.subarray, which interprets it relative to that read's length and yields a 1100-byte prefix. A subsequent nonempty-length check allows those bytes to be written. The failure depends on the later read's length and the selected slice end; being a non-final slice alone does not guarantee extra output. Related report: [morluto/rea#2285](https://github.com/morluto/rea/issues/2285), an open report by another author of the same copy defect. Its proposed fix, [morluto/rea#2291](https://github.com/morluto/rea/pull/2291), was open and unmerged when checked. Both are references; the local observation above independently establishes this baseline's failure. ## Restoration check For the admitted fixture above, require a 1000-byte private input equal to source range [100, 1100) and a digest equal to those bytes. Keep whole-source change rejection intact. Also check a slice crossing a read boundary, a slice ending exactly at a read boundary, a slice ending in the final read, and a separately admitted thin target. The controls must preserve the same exact-byte contract rather than merely accept the longer copy's digest. ## Runtime or environment - Failure verified at a2d8b94dd50dab8d966239dd63ab84718aa8b4b4 and still present at 8b181716aca2cb467be499cba226abfb5ae19141. - Node.js v26.10.0; npm 11.19.1; Vitest 5.0.3; macOS arm64. - Synthetic two-architecture Mach-O container; filesystem-copy and session-adapter verification with the existing recording connection. No real IDA engine was used. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: An admitted universal input produces an incorrect private analysis file and selected-slice digest. The failure is consequential but bounded to particular read and slice layouts; the verifier did not establish incorrect analysis results, a crash, or source-data loss. - Workaround status: available - Workaround basis: The verifier extracted the selected bytes into a separate thin Mach-O, admitted that file through the normal resolver, and confirmed that its private input and digest exactly preserve those bytes. This preparation avoids the failing universal-copy path for the tested fixture.

bsakaguchi

2026-10-15 18:29:07 +00:00

closed this issue
added the
area: domain

bug
labels

bsakaguchi commented

2026-10-15 18:29:08 +00:00

Author

Owner

@nikolay-dementiev commented on GitHub (Oct 13, 2026):

This is the same defect as #2285. It was fixed by #2291, merged into main as c6acd19b6: IdaWorkspace.copyInput() now stops the private input at the end of the selected slice. On real IDA 9.1, a universal file whose selected slice is followed by another slice now produces a private input byte-identical to the selected slice. The regression test from #2291 covers a layout like the one in this report.

@nikolay-dementiev commented on GitHub (Oct 13, 2026): This is the same defect as #2285. It was fixed by #2291, merged into `main` as c6acd19b6: `IdaWorkspace.copyInput()` now stops the private input at the end of the selected slice. On real IDA 9.1, a universal file whose selected slice is followed by another slice now produces a private input byte-identical to the selected slice. The regression test from #2291 covers a layout like the one in this report.

bsakaguchi commented

2026-10-15 18:29:08 +00:00

Author

Owner

@morluto commented on GitHub (Oct 14, 2026):

Fixed on main in c6acd19b6d. The private IDA input now contains exactly the intersection of each read chunk with the admitted slice, while the complete original container is verified separately. Bytes outside the selected slice are excluded.

Validation: src/ida/IdaWorkspaceThinning.test.ts checks exact bytes, length and SHA-256 across multi-read FAT slices and passes in CI run 38514789942 at 500e32399. This resolves the filesystem-copy defect; no attached IDA GUI or real IDA execution is claimed.

@morluto commented on GitHub (Oct 14, 2026): Fixed on main in c6acd19b6d842699ab807d9526dc762c743686f4. The private IDA input now contains exactly the intersection of each read chunk with the admitted slice, while the complete original container is verified separately. Bytes outside the selected slice are excluded. Validation: `src/ida/IdaWorkspaceThinning.test.ts` checks exact bytes, length and SHA-256 across multi-read FAT slices and passes in CI run 38514789942 at 500e32399. This resolves the filesystem-copy defect; no attached IDA GUI or real IDA execution is claimed.

bsakaguchi referenced this issue

2026-10-16 21:17:00 +00:00

[PR #866] [MERGED] fix(setup): read an OpenCode V2 server named type as a table entry #2302

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#866

No description provided.

Rows
Columns