[GH-ISSUE #6718] [Bug] Legacy CUDA compression flag is reported as uncompressed PTX #4085

New issue

Closed

opened 2026-10-31 00:46:43 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:46:43 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 23, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6718

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Native analysis or decompilation

Observed behavior

The production source service reports compression: "none" for a CUDA fatbinary entry carrying the documented legacy compression flag 0x1000. Inventory returns no limitation, uses the 96-byte padded encoded extent as both encoded_size and decoded_size, and discards the header's 91-byte decoded-length declaration. Kernel analysis then treats the compressed bytes as PTX text, marks the object malformed with PTX is not valid UTF-8 text, and returns no kernels. The otherwise comparable uncompressed control returns kernel kernel with complete payload coverage.

Reproduction

Save the following complete fixture/service probe as $WORK/gpu-cuda-legacy-repro.mts in a disposable directory. From a prepared source checkout with locked dependencies, run the command below. It writes only under $WORK, generates inert PTX bytes and a version-1 fatbinary with a version-0x101 entry, and invokes the production shared service; it does not execute a target or use ambient configuration.

import {pathToFileURL} from 'node:url';
import {join,resolve} from 'node:path';
import {writeFile} from 'node:fs/promises';
import {deflateSync,inflateSync} from 'node:zlib';
const repo=resolve(process.argv[2]),work=resolve(process.argv[3]);
const load=(p:string)=>import(pathToFileURL(join(repo,p)).href);
const {GpuCodeObjectsService}=await load('src/application/gpu/GpuCodeObjectsService.ts');
const {GpuCodeObjectsProvider}=await load('src/gpu/GpuCodeObjectsProvider.ts');
const ptx=Buffer.from('.version 7.0\n.target sm_80\n.address_size 64\n.visible .entry kernel(.param .u32 x) { ret; }\n');
const service=new GpuCodeObjectsService(new GpuCodeObjectsProvider());
try{for(const compressed of [false,true]){
 const data=compressed?deflateSync(ptx):ptx;
 const padded=Math.ceil(data.length/8)*8,record=Buffer.alloc(64+padded),head=Buffer.alloc(16);
 head.writeUInt32LE(0xba55ed50);head.writeUInt16LE(1,4);head.writeUInt16LE(16,6);head.writeBigUInt64LE(BigInt(record.length),8);
 record.writeUInt16LE(1);record.writeUInt16LE(0x101,2);record.writeUInt32LE(64,4);record.writeBigUInt64LE(BigInt(padded),8);
 record.writeUInt32LE(compressed?data.length:0,16);record.writeUInt32LE(80,28);record.writeBigUInt64LE(compressed?0x1015n:0x15n,40);record.writeBigUInt64LE(compressed?BigInt(ptx.length):0n,56);data.copy(record,64);
 if(compressed)console.log(JSON.stringify({control:'zlib-roundtrip',matches:inflateSync(data).equals(ptx),encoded:data.length,padded,decoded:ptx.length}));
 const path=join(work,compressed?'legacy-compressed.fatbin':'plain.fatbin');await writeFile(path,Buffer.concat([head,record]));
 for(const analysis of ['inventory','kernels'] as const){
  const result=await service.inspect({path,analysis,detail:'complete'});if(!result.ok)throw result.error;
  const r=result.value.normalized_result;
  console.log(JSON.stringify({compressed,analysis,coverage:r.coverage,objects:r.objects.map(({compression,encoded_size,decoded_size,inspection,limitations}:any)=>({compression,encoded_size,decoded_size,inspection,limitations})),kernels:r.kernels.map(({name}:any)=>name)}));
 }
}}finally{await service.close();}

Command actually run, with the checkout and disposable directory supplied as arguments:

node node_modules/tsx/dist/cli.mjs "$WORK/gpu-cuda-legacy-repro.mts" "$PWD" "$WORK"

The final standalone probe ran once per input/analysis combination and exited 0. A preliminary combined probe produced the same behavior before this standalone reproduction was written. The independent zlib round trip restored the exact 91-byte PTX source; encoded size was 95 bytes and aligned storage size was 96 bytes.

Expected behavior

Preserve the compression declaration and its encoded/decoded extents, and either decode a supported codec or explicitly classify it as unsupported. It must not report compressed bytes as uncompressed PTX. NVIDIA's authored fatbinary header, mirrored by Gklee, defines FATBIN_FLAG_COMPRESS = 0x1000 and states that a changed compression algorithm needs a new flag. REA's docs/gpu-code-objects.md promises explicit limitations for unknown codecs. This report requests correct classification, not an additional decoder.

Impact and scope

The demonstrated static inventory loses an explicit compression declaration and its declared decoded size; selected kernel analysis gives a misleading text-format failure. The uncompressed control succeeds. This is a bounded correctness defect in the advertised CUDA fatbinary inventory path. NOT_ENOUGH_INFO: real-producer frequency, historical driver acceptance of this self-made fixture, and a practical workaround were not established. The fixture uses ordinary zlib to make the compressed-versus-text distinction observable; the NVIDIA header alone does not specify the codec or the internal entry layout.

REA version

Source main 8c4f96e, package version 6.7.0. Latest published tag rea-agents-6.7.0 contains no src/gpu tree. This is a main-only source baseline, not a demonstrated released-package regression.

Runtime and platform

Node.js v26.12.1, Linux x64, repository-locked tsx. No GPU, CUDA/ROCm runtime, Hopper, Ghidra, built CLI, or MCP transport was used.

Target and artifact context

Self-made CUDA fatbinary with one PTX entry for sm_80, header flags 0x1015, 95 encoded bytes, 96 padded storage bytes, and 91 declared decoded bytes. The control contains the same PTX source without compression. This is a static format-boundary fixture, not a compiler-produced or runtime-loadability claim.

Interface

Other

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

Selected fields from the actual service output:

zlib roundtrip: matches=true; encoded=95; padded=96; decoded=91
plain inventory: compression=none; encoded_size=96; decoded_size=96; limitations=[]
plain kernels: inspection=complete; payloads=complete; kernels=[kernel]
legacy-compressed inventory: compression=none; encoded_size=96; decoded_size=96; limitations=[]
legacy-compressed kernels: inspection=malformed; payloads=partial; kernels=[]
limitation: PTX is not valid UTF-8 text

Evidence provenance: observed production-service results; flag meaning is separately source-derived from the NVIDIA-authored header.
Local verification: reproduced.
Reproduction completeness: complete for the synthetic static format/service boundary. Producer/driver acceptance and CLI/MCP transport were not run.

The public CLI and MCP use this same GpuCodeObjectsService; the complete request passes its input schema and the provider's ordinary-file admission, hashing, parsing, and output validation. The scripts and outputs contain only synthetic data.

Typed outcome or recovery expected

The flagged entry should retain a compressed/unsupported state, the 95-byte encoded payload extent, and the 91-byte decoded-size declaration. Unsupported decoding should produce an explicit codec limitation rather than feeding compressed bytes into UTF-8 PTX decoding. The plain control must still return its kernel. This is a proposed restoration criterion; no fix was made.

Analysis or suspected cause

Source-derived: CudaFatbinary.ts:43 tests only flags & 0xa000, which covers the separately recognized LZ4/Zstd bits and excludes 0x1000. The same boolean selects padded bytes and replaces decoded length at lines 46 and 67. GpuInventory.ts consequently enters its uncompressed PTX decoding path. The source permalink is given in REA version.

Additional context

The initial open-issue snapshot and related closed reports were reviewed. Exact all-state searches for FATBIN_FLAG_COMPRESS, plus related CUDA/fatbinary/compression searches, found no matching report. The submission checkbox stays unchecked for the filing agent's immediate duplicate check. Independent semantic review accepted the supported inventory defect before this draft was written. No runtime recheck or report review followed draft creation.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: The observed supported inventory path misrepresents a documented compression declaration and decoded size, then misclassifies selected inspection as malformed PTX; the defect is bounded and visible without data mutation or target execution.
Workaround status: unverified
Workaround basis: No practical recovery workflow for an existing affected artifact was demonstrated. The uncompressed control is evidence that parsing ordinary PTX succeeds, not a verified workaround for an existing compressed input.

Originally created by @coygeek on GitHub (Oct 23, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6718 ## Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ## Area Native analysis or decompilation ## Observed behavior The production source service reports `compression: "none"` for a CUDA fatbinary entry carrying the documented legacy compression flag `0x1000`. Inventory returns no limitation, uses the 96-byte padded encoded extent as both `encoded_size` and `decoded_size`, and discards the header's 91-byte decoded-length declaration. Kernel analysis then treats the compressed bytes as PTX text, marks the object malformed with `PTX is not valid UTF-8 text`, and returns no kernels. The otherwise comparable uncompressed control returns kernel `kernel` with complete payload coverage. ## Reproduction Save the following complete fixture/service probe as `$WORK/gpu-cuda-legacy-repro.mts` in a disposable directory. From a prepared source checkout with locked dependencies, run the command below. It writes only under `$WORK`, generates inert PTX bytes and a version-1 fatbinary with a version-`0x101` entry, and invokes the production shared service; it does not execute a target or use ambient configuration. ```ts import {pathToFileURL} from 'node:url'; import {join,resolve} from 'node:path'; import {writeFile} from 'node:fs/promises'; import {deflateSync,inflateSync} from 'node:zlib'; const repo=resolve(process.argv[2]),work=resolve(process.argv[3]); const load=(p:string)=>import(pathToFileURL(join(repo,p)).href); const {GpuCodeObjectsService}=await load('src/application/gpu/GpuCodeObjectsService.ts'); const {GpuCodeObjectsProvider}=await load('src/gpu/GpuCodeObjectsProvider.ts'); const ptx=Buffer.from('.version 7.0\n.target sm_80\n.address_size 64\n.visible .entry kernel(.param .u32 x) { ret; }\n'); const service=new GpuCodeObjectsService(new GpuCodeObjectsProvider()); try{for(const compressed of [false,true]){ const data=compressed?deflateSync(ptx):ptx; const padded=Math.ceil(data.length/8)*8,record=Buffer.alloc(64+padded),head=Buffer.alloc(16); head.writeUInt32LE(0xba55ed50);head.writeUInt16LE(1,4);head.writeUInt16LE(16,6);head.writeBigUInt64LE(BigInt(record.length),8); record.writeUInt16LE(1);record.writeUInt16LE(0x101,2);record.writeUInt32LE(64,4);record.writeBigUInt64LE(BigInt(padded),8); record.writeUInt32LE(compressed?data.length:0,16);record.writeUInt32LE(80,28);record.writeBigUInt64LE(compressed?0x1015n:0x15n,40);record.writeBigUInt64LE(compressed?BigInt(ptx.length):0n,56);data.copy(record,64); if(compressed)console.log(JSON.stringify({control:'zlib-roundtrip',matches:inflateSync(data).equals(ptx),encoded:data.length,padded,decoded:ptx.length})); const path=join(work,compressed?'legacy-compressed.fatbin':'plain.fatbin');await writeFile(path,Buffer.concat([head,record])); for(const analysis of ['inventory','kernels'] as const){ const result=await service.inspect({path,analysis,detail:'complete'});if(!result.ok)throw result.error; const r=result.value.normalized_result; console.log(JSON.stringify({compressed,analysis,coverage:r.coverage,objects:r.objects.map(({compression,encoded_size,decoded_size,inspection,limitations}:any)=>({compression,encoded_size,decoded_size,inspection,limitations})),kernels:r.kernels.map(({name}:any)=>name)})); } }}finally{await service.close();} ``` Command actually run, with the checkout and disposable directory supplied as arguments: ```sh node node_modules/tsx/dist/cli.mjs "$WORK/gpu-cuda-legacy-repro.mts" "$PWD" "$WORK" ``` The final standalone probe ran once per input/analysis combination and exited 0. A preliminary combined probe produced the same behavior before this standalone reproduction was written. The independent zlib round trip restored the exact 91-byte PTX source; encoded size was 95 bytes and aligned storage size was 96 bytes. ## Expected behavior Preserve the compression declaration and its encoded/decoded extents, and either decode a supported codec or explicitly classify it as unsupported. It must not report compressed bytes as uncompressed PTX. NVIDIA's authored [fatbinary header, mirrored by Gklee](https://github.com/Geof23/Gklee/blob/master/Gklee/include/cuda/fatbinary.h#L103-L104), defines `FATBIN_FLAG_COMPRESS = 0x1000` and states that a changed compression algorithm needs a new flag. REA's `docs/gpu-code-objects.md` promises explicit limitations for unknown codecs. This report requests correct classification, not an additional decoder. ## Impact and scope The demonstrated static inventory loses an explicit compression declaration and its declared decoded size; selected kernel analysis gives a misleading text-format failure. The uncompressed control succeeds. This is a bounded correctness defect in the advertised CUDA fatbinary inventory path. NOT_ENOUGH_INFO: real-producer frequency, historical driver acceptance of this self-made fixture, and a practical workaround were not established. The fixture uses ordinary zlib to make the compressed-versus-text distinction observable; the NVIDIA header alone does not specify the codec or the internal entry layout. ## REA version Source main [8c4f96e](https://github.com/morluto/rea/blob/8c4f96ef4759fd78da66c4b9726825ca71d7ee0c/src/gpu/CudaFatbinary.ts#L43-L67), package version 6.7.0. Latest published tag `rea-agents-6.7.0` contains no `src/gpu` tree. This is a main-only source baseline, not a demonstrated released-package regression. ## Runtime and platform Node.js v26.12.1, Linux x64, repository-locked tsx. No GPU, CUDA/ROCm runtime, Hopper, Ghidra, built CLI, or MCP transport was used. ## Target and artifact context Self-made CUDA fatbinary with one PTX entry for `sm_80`, header flags `0x1015`, 95 encoded bytes, 96 padded storage bytes, and 91 declared decoded bytes. The control contains the same PTX source without compression. This is a static format-boundary fixture, not a compiler-produced or runtime-loadability claim. ## Interface Other ## Earliest failure stage Native analysis or reconstruction ## Relevant output or evidence Selected fields from the actual service output: ```text zlib roundtrip: matches=true; encoded=95; padded=96; decoded=91 plain inventory: compression=none; encoded_size=96; decoded_size=96; limitations=[] plain kernels: inspection=complete; payloads=complete; kernels=[kernel] legacy-compressed inventory: compression=none; encoded_size=96; decoded_size=96; limitations=[] legacy-compressed kernels: inspection=malformed; payloads=partial; kernels=[] limitation: PTX is not valid UTF-8 text ``` - Evidence provenance: observed production-service results; flag meaning is separately source-derived from the NVIDIA-authored header. - Local verification: reproduced. - Reproduction completeness: complete for the synthetic static format/service boundary. Producer/driver acceptance and CLI/MCP transport were not run. The public CLI and MCP use this same `GpuCodeObjectsService`; the complete request passes its input schema and the provider's ordinary-file admission, hashing, parsing, and output validation. The scripts and outputs contain only synthetic data. ## Typed outcome or recovery expected The flagged entry should retain a compressed/unsupported state, the 95-byte encoded payload extent, and the 91-byte decoded-size declaration. Unsupported decoding should produce an explicit codec limitation rather than feeding compressed bytes into UTF-8 PTX decoding. The plain control must still return its kernel. This is a proposed restoration criterion; no fix was made. ## Analysis or suspected cause Source-derived: `CudaFatbinary.ts:43` tests only `flags & 0xa000`, which covers the separately recognized LZ4/Zstd bits and excludes `0x1000`. The same boolean selects padded bytes and replaces decoded length at lines 46 and 67. `GpuInventory.ts` consequently enters its uncompressed PTX decoding path. The source permalink is given in REA version. ## Additional context The initial open-issue snapshot and related closed reports were reviewed. Exact all-state searches for `FATBIN_FLAG_COMPRESS`, plus related CUDA/fatbinary/compression searches, found no matching report. The submission checkbox stays unchecked for the filing agent's immediate duplicate check. Independent semantic review accepted the supported inventory defect before this draft was written. No runtime recheck or report review followed draft creation. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: The observed supported inventory path misrepresents a documented compression declaration and decoded size, then misclassifies selected inspection as malformed PTX; the defect is bounded and visible without data mutation or target execution. - Workaround status: unverified - Workaround basis: No practical recovery workflow for an existing affected artifact was demonstrated. The uncompressed control is evidence that parsing ordinary PTX succeeds, not a verified workaround for an existing compressed input.

bsakaguchi closed this issue

2026-10-31 00:46:43 +00:00

bsakaguchi referenced this issue

2026-10-31 01:11:27 +00:00

[PR #4085] [MERGED] fix(domain): keep a decoded Swift alias category beside undecoded aliases #6048