[GH-ISSUE #6887] [Bug] CCOB v1 inventory splits a valid zlib payload at embedded CCOB bytes #4223

New issue

Closed

opened 2026-10-31 00:47:50 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:47:50 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 23, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6887

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Native analysis or decompilation

Observed behavior

The production GpuCodeObjectsProvider.inspect({ path, analysis: "inventory" }) rejects a standalone 132-byte CCOB v1 artifact whose valid zlib payload contains the ordinary text ordinary CCOB marker. It returns BinaryTargetError: Cannot open artifact: CCOB header is truncated and retains a malformed wrapper with an incorrectly shortened encoded size. A v2 wrapper around the identical compressed bytes succeeds and retains the host bundle entry.

Reproduction

From the source clone with its locked dependencies and Node 24.22.1, save the following as ../evidence/recordings/ccob-marker-probe.mjs. The fixture contains only one host bundle entry and inert text; it executes no artifact content.

import { createHash } from 'node:crypto';
import { writeFile } from 'node:fs/promises';
import { resolve, join } from 'node:path';
import { pathToFileURL } from 'node:url';
import { deflateSync, inflateSync } from 'node:zlib';
const clone = resolve(process.argv[2] ?? process.cwd());
const output = resolve(process.argv[3] ?? '.');
const { GpuCodeObjectsProvider } = await import(pathToFileURL(join(clone, 'src/gpu/GpuCodeObjectsProvider.ts')));
const id = Buffer.from('host-x86_64-unknown-linux');
const payload = Buffer.from('ordinary CCOB marker');
const directory = Buffer.alloc(32 + 24 + id.length);
directory.write('__CLANG_OFFLOAD_BUNDLE__');
directory.writeBigUInt64LE(1n, 24);
directory.writeBigUInt64LE(BigInt(directory.length), 32);
directory.writeBigUInt64LE(BigInt(payload.length), 40);
directory.writeBigUInt64LE(BigInt(id.length), 48);
id.copy(directory, 56);
const decoded = Buffer.concat([directory, payload]);
const compressed = deflateSync(decoded, { level: 0 });
console.log(JSON.stringify({ node: process.version, platform: process.platform, arch: process.arch,
  decoded_bytes: decoded.length, compressed_bytes: compressed.length,
  codec_round_trip: inflateSync(compressed).equals(decoded),
  marker_offset_in_compressed_data: compressed.indexOf('CCOB') }));
for (const version of [1, 2]) {
  const width = version === 1 ? 20 : 24;
  const header = Buffer.alloc(width);
  header.write('CCOB');
  header.writeUInt16LE(version, 4);
  header.writeUInt16LE(0, 6);
  if (version === 1) header.writeUInt32LE(decoded.length, 8);
  else {
    header.writeUInt32LE(width + compressed.length, 8);
    header.writeUInt32LE(decoded.length, 12);
  }
  createHash('md5').update(decoded).digest().copy(header, width - 8, 0, 8);
  const file = Buffer.concat([header, compressed]);
  const path = join(output, `marker-v${version}.ccob`);
  await writeFile(path, file);
  const provider = new GpuCodeObjectsProvider();
  try {
    const result = await provider.inspect({ path, analysis: 'inventory' });
    const report = result.ok ? result.value : result.error.partialObservation?.normalized_result;
    console.log(JSON.stringify({ version, file_bytes: file.length, sha256: createHash('sha256').update(file).digest('hex'),
      ok: result.ok,
      ...(result.ok ? {} : { error_tag: result.error._tag, message: result.error.message,
        cause: result.error.cause?.message }),
      coverage: report?.coverage,
      objects: report?.objects?.map(({kind, inspection, encoded_size, decoded_size, bundle_id, limitations}) =>
        ({kind, inspection, encoded_size, decoded_size, bundle_id, limitations})) }));
  } finally { await provider.close(); }
}

Run from the clone root after creating ../evidence/recordings:

mise exec node@24.22.1 -- node --import ./node_modules/tsx/dist/loader.mjs ../evidence/recordings/ccob-marker-probe.mjs . ../evidence/recordings

Each version was inspected once in the final file-backed probe. No CLI or MCP transport, GPU, or compiler-produced artifact was tested.

Expected behavior

The standalone v1 wrapper should decode and retain the host entry, as the v2 control does. REA advertises CCOB v1/v2/v3 support. An occurrence of the magic inside compressed data is not another container. The published LLVM 21.1.0 layout has no restriction on compressed payload bytes; its standalone decompressor passes the payload to the codec. These LLVM references are source evidence, not an observed LLVM runtime comparison.

Impact and scope

The tested valid v1 input cannot be inventoried, and its host entry is absent even from the retained partial observation. The same 112 compressed bytes round-trip to the exact 101-byte plain bundle and work inside v2. The fixture uses a legal zlib level-0 stream so the payload marker is deterministic; other compression levels, Zstd, real compiler-produced bundles, and frequency in real artifacts remain untested.

REA version

rea-agents 6.7.0 source revision 2443b949a48c3b2eba687e7381682e49e909d5b9.

Runtime and platform

Node v24.22.1; Linux x86_64; locked source dependencies. No external analysis provider or GPU.

Target and artifact context

Synthetic standalone CCOB v1/v2, zlib method 0, correct declared decoded length and truncated MD5, wrapping a Clang offload bundle with one host-x86_64-unknown-linux entry containing ordinary CCOB marker. The two files differ only in wrapper header layout/version.

Interface

Other — production source API GpuCodeObjectsProvider.inspect.

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

The probe observed:

Node v24.22.1; linux x64
Decoded bytes: 101; compressed bytes: 112; zlib round trip: true
The payload's CCOB marker is at compressed-data offset 97 (file offset 117 in v1).

v1: file_bytes=132; ok=false; error_tag=BinaryTargetError
message: Cannot open artifact: CCOB header is truncated
partial coverage: discovery=partial; payloads=partial
retained wrapper: inspection=malformed; encoded_size=97; decoded_size=101
wrapper limitation: GPU payload decompression failed or exceeded its declared length
no host entry retained

v2: file_bytes=136; ok=true; discovery=complete; payloads=not-requested
wrapper: inspection=complete; encoded_size=112; decoded_size=101
host entry: inspection=complete; encoded_size=20; decoded_size=20
bundle_id: host-x86_64-unknown-linux

Fixture SHA-256:

v1: cb91b35f40f983881ff62355d90a002669f29d86b2de6e80a9d9a275e4d92968
v2: 9c717fd1e783a26d8e6432dad8914dbb64bddfd43d0cac9414e773daace178f6

Typed outcome or recovery expected

A successful inventory with the decoded host entry and complete discovery for this standalone supported-format input.

Analysis or suspected cause

Source-derived: readLegacyCcobTotal searches encoded bytes for the first literal CCOB, without establishing a codec boundary. The HIP inventory loop uses that position as the compressed extent and then advances to it. The observed 97-byte encoded extent matches the marker offset, truncating the 112-byte zlib stream; the subsequent parse treats the remaining ordinary payload text as another CCOB header.

Additional context

Related closed report #6703 concerned a v1 bundle absorbing later aligned bundles. This case has only one container: its compressed content is incorrectly split. Closed #6737 concerns high-bit signature corruption, and #6719 concerns AMDGPU target triples. None describes this standalone payload collision. No source files were changed.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: The production provider rejects the bounded valid v1 fixture and loses its host inventory entry; the identical decoded content and compressed bytes succeed with a v2 wrapper.
Workaround status: unverified
Workaround basis: The equivalent v2 fixture succeeds, but a supported CLI/MCP workaround for existing affected v1 artifacts or a real artifact conversion workflow was not tested.

Originally created by @coygeek on GitHub (Oct 23, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6887 ## Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ## Area Native analysis or decompilation ## Observed behavior The production `GpuCodeObjectsProvider.inspect({ path, analysis: "inventory" })` rejects a standalone 132-byte CCOB v1 artifact whose valid zlib payload contains the ordinary text `ordinary CCOB marker`. It returns `BinaryTargetError: Cannot open artifact: CCOB header is truncated` and retains a malformed wrapper with an incorrectly shortened encoded size. A v2 wrapper around the identical compressed bytes succeeds and retains the host bundle entry. ## Reproduction From the source clone with its locked dependencies and Node 24.22.1, save the following as `../evidence/recordings/ccob-marker-probe.mjs`. The fixture contains only one host bundle entry and inert text; it executes no artifact content. ```js import { createHash } from 'node:crypto'; import { writeFile } from 'node:fs/promises'; import { resolve, join } from 'node:path'; import { pathToFileURL } from 'node:url'; import { deflateSync, inflateSync } from 'node:zlib'; const clone = resolve(process.argv[2] ?? process.cwd()); const output = resolve(process.argv[3] ?? '.'); const { GpuCodeObjectsProvider } = await import(pathToFileURL(join(clone, 'src/gpu/GpuCodeObjectsProvider.ts'))); const id = Buffer.from('host-x86_64-unknown-linux'); const payload = Buffer.from('ordinary CCOB marker'); const directory = Buffer.alloc(32 + 24 + id.length); directory.write('__CLANG_OFFLOAD_BUNDLE__'); directory.writeBigUInt64LE(1n, 24); directory.writeBigUInt64LE(BigInt(directory.length), 32); directory.writeBigUInt64LE(BigInt(payload.length), 40); directory.writeBigUInt64LE(BigInt(id.length), 48); id.copy(directory, 56); const decoded = Buffer.concat([directory, payload]); const compressed = deflateSync(decoded, { level: 0 }); console.log(JSON.stringify({ node: process.version, platform: process.platform, arch: process.arch, decoded_bytes: decoded.length, compressed_bytes: compressed.length, codec_round_trip: inflateSync(compressed).equals(decoded), marker_offset_in_compressed_data: compressed.indexOf('CCOB') })); for (const version of [1, 2]) { const width = version === 1 ? 20 : 24; const header = Buffer.alloc(width); header.write('CCOB'); header.writeUInt16LE(version, 4); header.writeUInt16LE(0, 6); if (version === 1) header.writeUInt32LE(decoded.length, 8); else { header.writeUInt32LE(width + compressed.length, 8); header.writeUInt32LE(decoded.length, 12); } createHash('md5').update(decoded).digest().copy(header, width - 8, 0, 8); const file = Buffer.concat([header, compressed]); const path = join(output, `marker-v${version}.ccob`); await writeFile(path, file); const provider = new GpuCodeObjectsProvider(); try { const result = await provider.inspect({ path, analysis: 'inventory' }); const report = result.ok ? result.value : result.error.partialObservation?.normalized_result; console.log(JSON.stringify({ version, file_bytes: file.length, sha256: createHash('sha256').update(file).digest('hex'), ok: result.ok, ...(result.ok ? {} : { error_tag: result.error._tag, message: result.error.message, cause: result.error.cause?.message }), coverage: report?.coverage, objects: report?.objects?.map(({kind, inspection, encoded_size, decoded_size, bundle_id, limitations}) => ({kind, inspection, encoded_size, decoded_size, bundle_id, limitations})) })); } finally { await provider.close(); } } ``` Run from the clone root after creating `../evidence/recordings`: ```sh mise exec node@24.22.1 -- node --import ./node_modules/tsx/dist/loader.mjs ../evidence/recordings/ccob-marker-probe.mjs . ../evidence/recordings ``` Each version was inspected once in the final file-backed probe. No CLI or MCP transport, GPU, or compiler-produced artifact was tested. ## Expected behavior The standalone v1 wrapper should decode and retain the host entry, as the v2 control does. REA [advertises CCOB v1/v2/v3 support](https://github.com/morluto/rea/blob/2443b949a48c3b2eba687e7381682e49e909d5b9/docs/gpu-code-objects.md#L3-L5). An occurrence of the magic inside compressed data is not another container. The [published LLVM 21.1.0 layout](https://github.com/llvm/llvm-project/blob/llvmorg-21.1.0/clang/include/clang/Driver/OffloadBundler.h#L87-L98) has no restriction on compressed payload bytes; its [standalone decompressor passes the payload to the codec](https://github.com/llvm/llvm-project/blob/llvmorg-21.1.0/clang/lib/Driver/OffloadBundler.cpp#L1142-L1181). These LLVM references are source evidence, not an observed LLVM runtime comparison. ## Impact and scope The tested valid v1 input cannot be inventoried, and its host entry is absent even from the retained partial observation. The same 112 compressed bytes round-trip to the exact 101-byte plain bundle and work inside v2. The fixture uses a legal zlib level-0 stream so the payload marker is deterministic; other compression levels, Zstd, real compiler-produced bundles, and frequency in real artifacts remain untested. ## REA version `rea-agents` 6.7.0 source revision `2443b949a48c3b2eba687e7381682e49e909d5b9`. ## Runtime and platform Node v24.22.1; Linux x86_64; locked source dependencies. No external analysis provider or GPU. ## Target and artifact context Synthetic standalone CCOB v1/v2, zlib method 0, correct declared decoded length and truncated MD5, wrapping a Clang offload bundle with one `host-x86_64-unknown-linux` entry containing `ordinary CCOB marker`. The two files differ only in wrapper header layout/version. ## Interface Other — production source API `GpuCodeObjectsProvider.inspect`. ## Earliest failure stage Native analysis or reconstruction ## Relevant output or evidence The probe observed: ```text Node v24.22.1; linux x64 Decoded bytes: 101; compressed bytes: 112; zlib round trip: true The payload's CCOB marker is at compressed-data offset 97 (file offset 117 in v1). v1: file_bytes=132; ok=false; error_tag=BinaryTargetError message: Cannot open artifact: CCOB header is truncated partial coverage: discovery=partial; payloads=partial retained wrapper: inspection=malformed; encoded_size=97; decoded_size=101 wrapper limitation: GPU payload decompression failed or exceeded its declared length no host entry retained v2: file_bytes=136; ok=true; discovery=complete; payloads=not-requested wrapper: inspection=complete; encoded_size=112; decoded_size=101 host entry: inspection=complete; encoded_size=20; decoded_size=20 bundle_id: host-x86_64-unknown-linux ``` Fixture SHA-256: - v1: `cb91b35f40f983881ff62355d90a002669f29d86b2de6e80a9d9a275e4d92968` - v2: `9c717fd1e783a26d8e6432dad8914dbb64bddfd43d0cac9414e773daace178f6` ## Typed outcome or recovery expected A successful inventory with the decoded host entry and complete discovery for this standalone supported-format input. ## Analysis or suspected cause Source-derived: [`readLegacyCcobTotal` searches encoded bytes for the first literal `CCOB`](https://github.com/morluto/rea/blob/2443b949a48c3b2eba687e7381682e49e909d5b9/src/gpu/ClangOffloadBundle.ts#L121-L146), without establishing a codec boundary. [The HIP inventory loop uses that position as the compressed extent and then advances to it](https://github.com/morluto/rea/blob/2443b949a48c3b2eba687e7381682e49e909d5b9/src/gpu/GpuInventory.ts#L555-L597). The observed 97-byte encoded extent matches the marker offset, truncating the 112-byte zlib stream; the subsequent parse treats the remaining ordinary payload text as another CCOB header. ## Additional context Related closed report [#6703](https://github.com/morluto/rea/issues/6703) concerned a v1 bundle absorbing later aligned bundles. This case has only one container: its compressed content is incorrectly split. Closed #6737 concerns high-bit signature corruption, and #6719 concerns AMDGPU target triples. None describes this standalone payload collision. No source files were changed. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: The production provider rejects the bounded valid v1 fixture and loses its host inventory entry; the identical decoded content and compressed bytes succeed with a v2 wrapper. - Workaround status: unverified - Workaround basis: The equivalent v2 fixture succeeds, but a supported CLI/MCP workaround for existing affected v1 artifacts or a real artifact conversion workflow was not tested.

bsakaguchi closed this issue

2026-10-31 00:47:51 +00:00

bsakaguchi referenced this issue

2026-10-31 01:12:07 +00:00

[PR #4223] [MERGED] fix(reference): resolve extensionless imports to .mts and .cts files #6135