morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #6657] [Bug] Embedded Clang bundles borrow missing header bytes from adjacent native sections #4037

New issue

Closed

opened 2026-10-31 00:46:15 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:46:15 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 23, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6657

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Native analysis or decompilation

Observed behavior

A 24-byte .hip_fatbin section containing only the Clang bundle magic is reported completely inspected when its neighboring .data section begins with eight zero bytes. The bundle's required eight-byte entry count is absent from the GPU section; those unrelated neighboring bytes supply it. The same 24-byte bundle is correctly rejected standalone. Changing only the first neighboring .data byte to 1 changes the embedded result from complete Evidence to ProviderAdapterError, although the GPU section bytes and extent are unchanged.

Reproduction

Run from the source checkout below with its locked dependencies installed and pinned Node on PATH. This is the executed production shared-service probe, not a claimed CLI or MCP execution. It writes inert synthetic files and result JSON under a temporary directory. The optional GPU_EVIDENCE_DIRECTORY selects the parent of that directory; omit it to use /tmp.

node --import tsx --input-type=module <<'NODE'
import { mkdtemp, writeFile } from 'node:fs/promises';
import { join, resolve } from 'node:path';
import { pathToFileURL } from 'node:url';
import { tmpdir } from 'node:os';
const load = file => import(pathToFileURL(resolve(file)).href);
const { GpuCodeObjectsProvider } = await load('src/gpu/GpuCodeObjectsProvider.ts');
const { GpuCodeObjectsService } = await load('src/application/gpu/GpuCodeObjectsService.ts');
const work = await mkdtemp(join(process.env.GPU_EVIDENCE_DIRECTORY ?? tmpdir(), 'embedded-bundle-'));
function hostElf(gpu, following) {
  const names = Buffer.from('\0.shstrtab\0.hip_fatbin\0.data\0');
  const rows = [{ name: '', type: 0, bytes: Buffer.alloc(0) }, { name: '.shstrtab', type: 3, bytes: names }, { name: '.hip_fatbin', type: 1, bytes: gpu }, { name: '.data', type: 1, bytes: following }];
  let end = 64;
  const offsets = rows.map(row => { end = Math.ceil(end / 8) * 8; const offset = end; end += row.bytes.length; return offset; });
  const table = Math.ceil(end / 8) * 8;
  const result = Buffer.alloc(table + 64 * rows.length);
  result.set([0x7f, 69, 76, 70, 2, 1, 1]);
  result.writeUInt16LE(2, 16); result.writeUInt16LE(62, 18); result.writeUInt32LE(1, 20);
  result.writeBigUInt64LE(BigInt(table), 40); result.writeUInt16LE(64, 52); result.writeUInt16LE(64, 58); result.writeUInt16LE(rows.length, 60); result.writeUInt16LE(1, 62);
  rows.forEach((row, index) => {
    if (index === 0) return;
    const h = table + index * 64;
    result.writeUInt32LE(names.indexOf(Buffer.from(row.name + '\0')), h);
    result.writeUInt32LE(row.type, h + 4);
    result.writeBigUInt64LE(BigInt(offsets[index]), h + 24);
    result.writeBigUInt64LE(BigInt(row.bytes.length), h + 32);
    row.bytes.copy(result, offsets[index]);
  });
  return result;
}
const magic = Buffer.from('__CLANG_OFFLOAD_BUNDLE__');
const header = Buffer.concat([magic, Buffer.alloc(8)]);
const one = Buffer.alloc(8); one.writeBigUInt64LE(1n);
const cases = {
  standaloneTruncated: magic,
  embeddedTruncatedZero: hostElf(magic, Buffer.alloc(8)),
  embeddedTruncatedOne: hostElf(magic, one),
  standaloneControl: header,
  embeddedControl: hostElf(header, Buffer.alloc(8)),
};
for (const [name, bytes] of Object.entries(cases)) {
  const path = join(work, name + '.bin');
  await writeFile(path, bytes);
  const service = new GpuCodeObjectsService(new GpuCodeObjectsProvider());
  try {
    const result = await service.inspect({ path, analysis: 'kernels', detail: 'complete' });
    await writeFile(join(work, name + '.json'), JSON.stringify(result, null, 2));
    const report = result.ok ? result.value.normalized_result : null;
    console.log(JSON.stringify({ name, ok: result.ok, ...(report ? { discovery: report.coverage.discovery, payloads: report.coverage.payloads, objects: report.objects.map(o => ({ kind: o.kind, size: o.location.size, inspection: o.inspection })), kernels: report.kernels.length } : { error: result.error._tag, message: result.error.message }) }));
  } finally { await service.close(); }
}
NODE

Each of these five cases was inspected once in this probe; a delegated probe also observed the same outcomes in both inventory and kernel modes. No fixture code was executed. The process exited 0, including cases where the service returned an error result.

Expected behavior

Treat the 24-byte embedded bundle as truncated without using .data bytes to complete its header. It may return a format error or retain the native section with a malformed/partial result, but it must not certify discovery and payload coverage as complete. A complete 32-byte zero-entry bundle must continue to work both standalone and embedded.

LLVM's bundled binary layout requires the 24-byte magic followed by an eight-byte entry count. REA's GPU provenance contract gives each contained object's locations its own decoded coordinate space. The range validator rejects metadata outside that space; the standalone result demonstrates the existing truncation behavior for the same bytes.

Impact and scope

Static inventory can incorrectly claim a complete empty GPU bundle for a truncated section, with its answer depending on unrelated neighboring section contents. Both inventory and kernel analysis reproduced the issue on the synthetic ELF fixtures. The observed passing result has no GPU payload or kernel facts, so this report does not claim that a real kernel was omitted. Copying the exact GPU-section bytes to a standalone file demonstrably detects this truncation and is a practical diagnostic workaround. No security-boundary crossing, target execution, real-producer prevalence, or other native formats were investigated.

REA version

Source revision 23def04096869b85c4d3c14a73ed09168483c9f6; package metadata 6.7.0. The source service at that revision was executed. The installed published-package runtime was not tested.

Runtime and platform

Node.js v24.22.1; Linux 7.2.8-9-omarchy x86_64. No CUDA/ROCm compiler, driver, GPU, Hopper, or Ghidra was used.

Target and artifact context

Synthetic little-endian ELF64 x86-64 host with .shstrtab, .hip_fatbin, and .data sections. The truncated GPU section begins at file offset 96 and has size 24. The paired truncated host files differ at exactly byte 120, immediately after that section; their GPU-section bytes and sizes are identical. The complete controls have 32-byte bundles with zero entries. All inputs are generated by the reproduction.

Interface

Other

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

The executed probe printed:

{"name":"standaloneTruncated","ok":false,"error":"BinaryTargetError","message":"Cannot open artifact: GPU byte range is outside its containing object"}
{"name":"embeddedTruncatedZero","ok":true,"discovery":"complete","payloads":"complete","objects":[{"kind":"native-section","size":24,"inspection":"complete"}],"kernels":0}
{"name":"embeddedTruncatedOne","ok":false,"error":"ProviderAdapterError","message":"Provider gpu-code-objects adapter failed during inspect_gpu_code_objects"}
{"name":"standaloneControl","ok":true,"discovery":"complete","payloads":"complete","objects":[],"kernels":0}
{"name":"embeddedControl","ok":true,"discovery":"complete","payloads":"complete","objects":[{"kind":"native-section","size":32,"inspection":"complete"}],"kernels":0}

The error's recorded diagnostics in the one-byte-neighbor variant identify objects[1].metadata_location with GPU source range is outside its declared coordinate space. A delegated independent fixture comparison confirmed that the GPU sections are identical and only the neighboring byte changes.

Expected source: LLVM's binary layout, REA's coordinate-space documentation and range validator linked above, plus the observed standalone truncation control.
Failure source: executed GpuCodeObjectsService.inspect results for the inline fixtures; source-derived explanation below.
Evidence provenance: observed.
Local verification: reproduced.
Reproduction completeness: complete.

Typed outcome or recovery expected

Return a precise truncated/malformed bundle result, or retain partial section discovery with the truncation reason. Unrelated .data contents must not turn a missing bundle header into a complete inventory.

Analysis or suspected cause

Source-derived: the native-section source adapter translates source.read(offset + start, length) without checking the section's own size. The Clang directory reader requests 32 bytes for its header. The file-level source can satisfy that request by reading eight bytes beyond the declared 24-byte section. The zero count produces no child metadata range for the final schema to reject, and the section is marked complete.

Additional context

Restoration check: the same two truncated embedded fixtures should report the same section truncation regardless of the neighboring byte; neither may report complete discovery/payload coverage. The truncated standalone input must remain rejected, and the complete 32-byte standalone and embedded controls must remain accepted. Preserve the distinction between malformed section data and unexpected adapter failure.

Read-only all-state searches for "hip_fatbin" "truncated" and "offload" "truncated" returned no matches. Broader "GPU" "section" and "ClangOffloadBundle" results were inspected; the latter found only #6613, about HIP target suffixes. NOT_ENOUGH_INFO: first affected release, real-producer prevalence, CLI/MCP transport, and non-ELF embedding were not tested. Prepared with AI assistance.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: A bounded static-analysis result falsely marks a truncated embedded GPU bundle completely inspected, and unrelated neighboring data changes that result. The reproduced consequence is incorrect coverage and diagnosis, with no observed target execution or lost real kernel.
Workaround status: available
Workaround basis: The identical 24-byte section inspected standalone returns the expected truncation error. Extracting that section to a disposable standalone file is an observed practical way to diagnose this fixture without changing the original artifact.

Originally created by @coygeek on GitHub (Oct 23, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6657 ## Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ## Area Native analysis or decompilation ## Observed behavior A 24-byte `.hip_fatbin` section containing only the Clang bundle magic is reported completely inspected when its neighboring `.data` section begins with eight zero bytes. The bundle's required eight-byte entry count is absent from the GPU section; those unrelated neighboring bytes supply it. The same 24-byte bundle is correctly rejected standalone. Changing only the first neighboring `.data` byte to 1 changes the embedded result from complete Evidence to `ProviderAdapterError`, although the GPU section bytes and extent are unchanged. ## Reproduction Run from the source checkout below with its locked dependencies installed and pinned Node on `PATH`. This is the executed production shared-service probe, not a claimed CLI or MCP execution. It writes inert synthetic files and result JSON under a temporary directory. The optional `GPU_EVIDENCE_DIRECTORY` selects the parent of that directory; omit it to use `/tmp`. ```sh node --import tsx --input-type=module <<'NODE' import { mkdtemp, writeFile } from 'node:fs/promises'; import { join, resolve } from 'node:path'; import { pathToFileURL } from 'node:url'; import { tmpdir } from 'node:os'; const load = file => import(pathToFileURL(resolve(file)).href); const { GpuCodeObjectsProvider } = await load('src/gpu/GpuCodeObjectsProvider.ts'); const { GpuCodeObjectsService } = await load('src/application/gpu/GpuCodeObjectsService.ts'); const work = await mkdtemp(join(process.env.GPU_EVIDENCE_DIRECTORY ?? tmpdir(), 'embedded-bundle-')); function hostElf(gpu, following) { const names = Buffer.from('\0.shstrtab\0.hip_fatbin\0.data\0'); const rows = [{ name: '', type: 0, bytes: Buffer.alloc(0) }, { name: '.shstrtab', type: 3, bytes: names }, { name: '.hip_fatbin', type: 1, bytes: gpu }, { name: '.data', type: 1, bytes: following }]; let end = 64; const offsets = rows.map(row => { end = Math.ceil(end / 8) * 8; const offset = end; end += row.bytes.length; return offset; }); const table = Math.ceil(end / 8) * 8; const result = Buffer.alloc(table + 64 * rows.length); result.set([0x7f, 69, 76, 70, 2, 1, 1]); result.writeUInt16LE(2, 16); result.writeUInt16LE(62, 18); result.writeUInt32LE(1, 20); result.writeBigUInt64LE(BigInt(table), 40); result.writeUInt16LE(64, 52); result.writeUInt16LE(64, 58); result.writeUInt16LE(rows.length, 60); result.writeUInt16LE(1, 62); rows.forEach((row, index) => { if (index === 0) return; const h = table + index * 64; result.writeUInt32LE(names.indexOf(Buffer.from(row.name + '\0')), h); result.writeUInt32LE(row.type, h + 4); result.writeBigUInt64LE(BigInt(offsets[index]), h + 24); result.writeBigUInt64LE(BigInt(row.bytes.length), h + 32); row.bytes.copy(result, offsets[index]); }); return result; } const magic = Buffer.from('__CLANG_OFFLOAD_BUNDLE__'); const header = Buffer.concat([magic, Buffer.alloc(8)]); const one = Buffer.alloc(8); one.writeBigUInt64LE(1n); const cases = { standaloneTruncated: magic, embeddedTruncatedZero: hostElf(magic, Buffer.alloc(8)), embeddedTruncatedOne: hostElf(magic, one), standaloneControl: header, embeddedControl: hostElf(header, Buffer.alloc(8)), }; for (const [name, bytes] of Object.entries(cases)) { const path = join(work, name + '.bin'); await writeFile(path, bytes); const service = new GpuCodeObjectsService(new GpuCodeObjectsProvider()); try { const result = await service.inspect({ path, analysis: 'kernels', detail: 'complete' }); await writeFile(join(work, name + '.json'), JSON.stringify(result, null, 2)); const report = result.ok ? result.value.normalized_result : null; console.log(JSON.stringify({ name, ok: result.ok, ...(report ? { discovery: report.coverage.discovery, payloads: report.coverage.payloads, objects: report.objects.map(o => ({ kind: o.kind, size: o.location.size, inspection: o.inspection })), kernels: report.kernels.length } : { error: result.error._tag, message: result.error.message }) })); } finally { await service.close(); } } NODE ``` Each of these five cases was inspected once in this probe; a delegated probe also observed the same outcomes in both inventory and kernel modes. No fixture code was executed. The process exited 0, including cases where the service returned an error result. ## Expected behavior Treat the 24-byte embedded bundle as truncated without using `.data` bytes to complete its header. It may return a format error or retain the native section with a malformed/partial result, but it must not certify discovery and payload coverage as complete. A complete 32-byte zero-entry bundle must continue to work both standalone and embedded. LLVM's [bundled binary layout](https://clang.llvm.org/docs/ClangOffloadBundler.html#bundled-binary-file-layout) requires the 24-byte magic followed by an eight-byte entry count. REA's [GPU provenance contract](https://github.com/morluto/rea/blob/23def04096869b85c4d3c14a73ed09168483c9f6/docs/gpu-code-objects.md#L121-L125) gives each contained object's locations its own decoded coordinate space. The [range validator](https://github.com/morluto/rea/blob/23def04096869b85c4d3c14a73ed09168483c9f6/src/domain/gpu/gpuCodeObjects.ts#L192-L218) rejects metadata outside that space; the standalone result demonstrates the existing truncation behavior for the same bytes. ## Impact and scope Static inventory can incorrectly claim a complete empty GPU bundle for a truncated section, with its answer depending on unrelated neighboring section contents. Both inventory and kernel analysis reproduced the issue on the synthetic ELF fixtures. The observed passing result has no GPU payload or kernel facts, so this report does not claim that a real kernel was omitted. Copying the exact GPU-section bytes to a standalone file demonstrably detects this truncation and is a practical diagnostic workaround. No security-boundary crossing, target execution, real-producer prevalence, or other native formats were investigated. ## REA version Source revision `23def04096869b85c4d3c14a73ed09168483c9f6`; package metadata 6.7.0. The source service at that revision was executed. The installed published-package runtime was not tested. ## Runtime and platform Node.js v24.22.1; Linux 7.2.8-9-omarchy x86_64. No CUDA/ROCm compiler, driver, GPU, Hopper, or Ghidra was used. ## Target and artifact context Synthetic little-endian ELF64 x86-64 host with `.shstrtab`, `.hip_fatbin`, and `.data` sections. The truncated GPU section begins at file offset 96 and has size 24. The paired truncated host files differ at exactly byte 120, immediately after that section; their GPU-section bytes and sizes are identical. The complete controls have 32-byte bundles with zero entries. All inputs are generated by the reproduction. ## Interface Other ## Earliest failure stage Native analysis or reconstruction ## Relevant output or evidence The executed probe printed: ```json {"name":"standaloneTruncated","ok":false,"error":"BinaryTargetError","message":"Cannot open artifact: GPU byte range is outside its containing object"} {"name":"embeddedTruncatedZero","ok":true,"discovery":"complete","payloads":"complete","objects":[{"kind":"native-section","size":24,"inspection":"complete"}],"kernels":0} {"name":"embeddedTruncatedOne","ok":false,"error":"ProviderAdapterError","message":"Provider gpu-code-objects adapter failed during inspect_gpu_code_objects"} {"name":"standaloneControl","ok":true,"discovery":"complete","payloads":"complete","objects":[],"kernels":0} {"name":"embeddedControl","ok":true,"discovery":"complete","payloads":"complete","objects":[{"kind":"native-section","size":32,"inspection":"complete"}],"kernels":0} ``` The error's recorded diagnostics in the one-byte-neighbor variant identify `objects[1].metadata_location` with `GPU source range is outside its declared coordinate space`. A delegated independent fixture comparison confirmed that the GPU sections are identical and only the neighboring byte changes. - Expected source: LLVM's binary layout, REA's coordinate-space documentation and range validator linked above, plus the observed standalone truncation control. - Failure source: executed `GpuCodeObjectsService.inspect` results for the inline fixtures; source-derived explanation below. - Evidence provenance: observed. - Local verification: reproduced. - Reproduction completeness: complete. ## Typed outcome or recovery expected Return a precise truncated/malformed bundle result, or retain partial section discovery with the truncation reason. Unrelated `.data` contents must not turn a missing bundle header into a complete inventory. ## Analysis or suspected cause Source-derived: [the native-section source adapter](https://github.com/morluto/rea/blob/23def04096869b85c4d3c14a73ed09168483c9f6/src/gpu/GpuInventory.ts#L189-L220) translates `source.read(offset + start, length)` without checking the section's own size. [The Clang directory reader](https://github.com/morluto/rea/blob/23def04096869b85c4d3c14a73ed09168483c9f6/src/gpu/ClangOffloadBundle.ts#L15-L24) requests 32 bytes for its header. The file-level source can satisfy that request by reading eight bytes beyond the declared 24-byte section. The zero count produces no child metadata range for the final schema to reject, and the section is marked complete. ## Additional context Restoration check: the same two truncated embedded fixtures should report the same section truncation regardless of the neighboring byte; neither may report complete discovery/payload coverage. The truncated standalone input must remain rejected, and the complete 32-byte standalone and embedded controls must remain accepted. Preserve the distinction between malformed section data and unexpected adapter failure. Read-only all-state searches for `"hip_fatbin" "truncated"` and `"offload" "truncated"` returned no matches. Broader `"GPU" "section"` and `"ClangOffloadBundle"` results were inspected; the latter found only [#6613](https://github.com/morluto/rea/issues/6613), about HIP target suffixes. NOT_ENOUGH_INFO: first affected release, real-producer prevalence, CLI/MCP transport, and non-ELF embedding were not tested. Prepared with AI assistance. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: A bounded static-analysis result falsely marks a truncated embedded GPU bundle completely inspected, and unrelated neighboring data changes that result. The reproduced consequence is incorrect coverage and diagnosis, with no observed target execution or lost real kernel. - Workaround status: available - Workaround basis: The identical 24-byte section inspected standalone returns the expected truncation error. Extracting that section to a disposable standalone file is an observed practical way to diagnose this fixture without changing the original artifact.

bsakaguchi closed this issue

2026-10-31 00:46:15 +00:00

bsakaguchi referenced this issue

2026-10-31 01:11:08 +00:00

[PR #4037] [CLOSED] fix(contracts): restore canonical mobile graph examples #6007

bsakaguchi referenced this issue

2026-11-01 21:11:07 +00:00

[GH-ISSUE #5654] [Tracker] Open PRs that can't merge yet: conflicts and red CI (October 2026) #3425