[GH-ISSUE #6703] [Bug] CCOB v1 inventory consumes later bundles and loses all kernels #4077

New issue

Closed

opened 2026-10-31 00:46:38 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:46:38 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 23, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6703

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Native analysis or decompilation

Observed behavior

An ELF .hip_fatbin containing an aligned CCOB v1 bundle followed by a v2 bundle yields zero kernels and a malformed first bundle. The v1 bundle alone yields one kernel; replacing the first bundle with v2 yields both kernels. This directly observed provider failure concerns container boundaries, not GPU execution.

Reproduction

From the source checkout with locked dependencies and Node 24.22.1, run this inert synthetic-fixture probe. Each final combination was inspected once; it writes and removes only a temporary directory.

node --import tsx --input-type=module <<'JS'
import { mkdtemp, writeFile, rm } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { GpuCodeObjectsProvider } from './src/gpu/GpuCodeObjectsProvider.ts';
import { ccobFixture, elfFixture } from './tests/fixtures/gpu/codeObjects.ts';
const dir = await mkdtemp(join(tmpdir(), 'rea-ccob-'));
for (const version of [1, 2]) {
  const first = ccobFixture(version, 'zlib');
  for (const mode of ['single', 'linked']) {
    const bundles = mode === 'single' ? first : Buffer.concat([
      first, Buffer.alloc((8 - first.length % 8) % 8), ccobFixture(2, 'zlib')
    ]);
    const path = join(dir, 'host.elf');
    await writeFile(path, elfFixture(62, [{ name: '.hip_fatbin', bytes: bundles }]));
    const provider = new GpuCodeObjectsProvider();
    const result = await provider.inspect({ path, analysis: 'kernels' });
    if (!result.ok) throw result.error;
    console.log(JSON.stringify({ version, mode, kernels: result.value.kernels.length,
      payloads: result.value.coverage.payloads,
      errors: result.value.objects.flatMap(x => x.limitations) }));
    await provider.close();
  }
}
await rm(dir, { recursive: true });
JS

Expected behavior

Decode the individual compressed bundles in the ELF section and retain both kernels, as with the v2 control. REA advertises CCOB v1/v2/v3 and already tests flat mixed bundle sequences. LLVM's published layout distinguishes v1 from versions carrying an explicit total length.

Impact and scope

The tested flat sequence loses all kernel facts, including the independently readable later bundle, while correctly signaling partial coverage. A separate initial matrix also observed v1 failures with trailing zero padding and both zlib/Zstd; v2/v3 controls retained their kernels. These were synthetic artifacts; frequency in real linked HIP binaries and real compiler/runtime behavior are untested. Exact standalone bundle inspection succeeded and provides a bounded workaround when individual byte ranges are known.

REA version

rea-agents 6.7.0 source revision 8c4f96ef4759fd78da66c4b9726825ca71d7ee0c.

Runtime and platform

Node v24.22.1; Linux x86_64; locked source dependencies. No GPU or external analysis provider.

Target and artifact context

Synthetic ELF64 little-endian x86_64 host with .hip_fatbin; repository fixture builders supply MessagePack AMDGPU declarations and aligned CCOB records. Fixtures are inert declarations, not claimed to be complete executable GPU objects.

Interface

Other — production GpuCodeObjectsProvider.inspect source API. CLI and MCP transports were not run.

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

Observed output from the inline probe:

{"version":1,"mode":"single","kernels":1,"payloads":"complete","errors":[]}
{"version":1,"mode":"linked","kernels":0,"payloads":"partial","errors":["GPU compressed payload contains bytes outside its complete codec stream"]}
{"version":2,"mode":"single","kernels":1,"payloads":"complete","errors":[]}
{"version":2,"mode":"linked","kernels":2,"payloads":"complete","errors":[]}

Typed outcome or recovery expected

A successful report retaining both bundles and kernels, with the first object's encoded range confined to its codec stream.

Analysis or suspected cause

Source-derived: the v1 header reader assigns total = remaining. The HIP loop reads that whole range and advances by total, so it includes padding and every later bundle in the first compressed payload. The codec owner rejects unconsumed input. The observed error matches this path; the codec's strict accounting itself need not be weakened.

Additional context

Open/closed searches for CCOB, CCOB concatenated, and GPU padding found the original capability request #5345 but no report of this boundary failure. No source files were changed. Real compiler-produced linked v1 samples remain untested.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: The production provider loses both kernels for the bounded synthetic linked-container case; the same declarations decode individually and in the v2 control. No input mutation or execution failure occurred.
Workaround status: available
Workaround basis: Inspecting the exact standalone first bundle succeeded; individually known bundle ranges can be analyzed separately. Automatic boundary extraction from real linked artifacts was not verified.

Originally created by @coygeek on GitHub (Oct 23, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6703 ## Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ## Area Native analysis or decompilation ## Observed behavior An ELF `.hip_fatbin` containing an aligned CCOB v1 bundle followed by a v2 bundle yields zero kernels and a malformed first bundle. The v1 bundle alone yields one kernel; replacing the first bundle with v2 yields both kernels. This directly observed provider failure concerns container boundaries, not GPU execution. ## Reproduction From the source checkout with locked dependencies and Node 24.22.1, run this inert synthetic-fixture probe. Each final combination was inspected once; it writes and removes only a temporary directory. ```sh node --import tsx --input-type=module <<'JS' import { mkdtemp, writeFile, rm } from 'node:fs/promises'; import { tmpdir } from 'node:os'; import { join } from 'node:path'; import { GpuCodeObjectsProvider } from './src/gpu/GpuCodeObjectsProvider.ts'; import { ccobFixture, elfFixture } from './tests/fixtures/gpu/codeObjects.ts'; const dir = await mkdtemp(join(tmpdir(), 'rea-ccob-')); for (const version of [1, 2]) { const first = ccobFixture(version, 'zlib'); for (const mode of ['single', 'linked']) { const bundles = mode === 'single' ? first : Buffer.concat([ first, Buffer.alloc((8 - first.length % 8) % 8), ccobFixture(2, 'zlib') ]); const path = join(dir, 'host.elf'); await writeFile(path, elfFixture(62, [{ name: '.hip_fatbin', bytes: bundles }])); const provider = new GpuCodeObjectsProvider(); const result = await provider.inspect({ path, analysis: 'kernels' }); if (!result.ok) throw result.error; console.log(JSON.stringify({ version, mode, kernels: result.value.kernels.length, payloads: result.value.coverage.payloads, errors: result.value.objects.flatMap(x => x.limitations) })); await provider.close(); } } await rm(dir, { recursive: true }); JS ``` ## Expected behavior Decode the individual compressed bundles in the ELF section and retain both kernels, as with the v2 control. REA advertises CCOB v1/v2/v3 and already [tests flat mixed bundle sequences](https://github.com/morluto/rea/blob/8c4f96ef4759fd78da66c4b9726825ca71d7ee0c/tests/boundary/gpu/codeObjects.test.ts#L611-L618). LLVM's [published layout](https://releases.llvm.org/21.1.2/tools/clang/docs/ClangOffloadBundler.html#compression-and-decompression) distinguishes v1 from versions carrying an explicit total length. ## Impact and scope The tested flat sequence loses all kernel facts, including the independently readable later bundle, while correctly signaling partial coverage. A separate initial matrix also observed v1 failures with trailing zero padding and both zlib/Zstd; v2/v3 controls retained their kernels. These were synthetic artifacts; frequency in real linked HIP binaries and real compiler/runtime behavior are untested. Exact standalone bundle inspection succeeded and provides a bounded workaround when individual byte ranges are known. ## REA version rea-agents 6.7.0 source revision `8c4f96ef4759fd78da66c4b9726825ca71d7ee0c`. ## Runtime and platform Node v24.22.1; Linux x86_64; locked source dependencies. No GPU or external analysis provider. ## Target and artifact context Synthetic ELF64 little-endian x86_64 host with `.hip_fatbin`; repository fixture builders supply MessagePack AMDGPU declarations and aligned CCOB records. Fixtures are inert declarations, not claimed to be complete executable GPU objects. ## Interface Other — production `GpuCodeObjectsProvider.inspect` source API. CLI and MCP transports were not run. ## Earliest failure stage Native analysis or reconstruction ## Relevant output or evidence Observed output from the inline probe: ```json {"version":1,"mode":"single","kernels":1,"payloads":"complete","errors":[]} {"version":1,"mode":"linked","kernels":0,"payloads":"partial","errors":["GPU compressed payload contains bytes outside its complete codec stream"]} {"version":2,"mode":"single","kernels":1,"payloads":"complete","errors":[]} {"version":2,"mode":"linked","kernels":2,"payloads":"complete","errors":[]} ``` ## Typed outcome or recovery expected A successful report retaining both bundles and kernels, with the first object's encoded range confined to its codec stream. ## Analysis or suspected cause Source-derived: [the v1 header reader assigns `total = remaining`](https://github.com/morluto/rea/blob/8c4f96ef4759fd78da66c4b9726825ca71d7ee0c/src/gpu/ClangOffloadBundle.ts#L87-L107). [The HIP loop reads that whole range and advances by `total`](https://github.com/morluto/rea/blob/8c4f96ef4759fd78da66c4b9726825ca71d7ee0c/src/gpu/GpuInventory.ts#L536-L573), so it includes padding and every later bundle in the first compressed payload. [The codec owner rejects unconsumed input](https://github.com/morluto/rea/blob/8c4f96ef4759fd78da66c4b9726825ca71d7ee0c/src/gpu/GpuPayloadCodec.ts#L47-L53). The observed error matches this path; the codec's strict accounting itself need not be weakened. ## Additional context Open/closed searches for `CCOB`, `CCOB concatenated`, and `GPU padding` found the original capability request #5345 but no report of this boundary failure. No source files were changed. Real compiler-produced linked v1 samples remain untested. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: The production provider loses both kernels for the bounded synthetic linked-container case; the same declarations decode individually and in the v2 control. No input mutation or execution failure occurred. - Workaround status: available - Workaround basis: Inspecting the exact standalone first bundle succeeded; individually known bundle ranges can be analyzed separately. Automatic boundary extraction from real linked artifacts was not verified.

bsakaguchi closed this issue

2026-10-31 00:46:38 +00:00

bsakaguchi referenced this issue

2026-10-31 01:11:25 +00:00

[PR #4077] [MERGED] fix(ghidra): consume inspected configuration directory identity #6042