morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #6618] [Bug] CUDA fatbinary inventory drops the family-specific f target suffix #4005

New issue

Closed

opened 2026-10-31 00:45:58 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:45:58 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 23, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6618

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Native analysis or decompilation

Observed behavior

A real NVIDIA nvFatbin-produced sm_100f container is inventoried as sm_100. Unfiltered kernel inspection then reports a false target conflict between inventory sm_100 and payload sm_100f. Selecting targets: ["sm_100f"] returns no kernels and marks the only object not-selected. The corresponding sm_100 producer control retains its target and returns its kernel with complete coverage.

The producer library and production REA shared service/provider ran with real files. No GPU driver, target execution, CUDA assembler, CLI/MCP transport or native decompiler was run.

Reproduction

Use the reported source checkout with locked dependencies, Python 3, and NVIDIA's nvidia-nvfatbin wheel version 13.4.92, extracted into a disposable directory. This was the latest version returned by PyPI during the probe. NVFATBIN_LIBRARY denotes its nvidia/cu13/lib/libnvfatbin.so.13; no system installation is required. The tested Linux x86_64 wheel's SHA-256 is 7fb72f39eaca2097e146b588411805074d2c6086e49412c55f553633e40e7f3c.

Save this generator as $WORK/make-fatbin.py, then run python "$WORK/make-fatbin.py" "$NVFATBIN_LIBRARY" "$WORK". All producer calls returned success, and nvFatbinVersion returned major 13, minor 4. The library packs supplied PTX into a fatbinary; it does not execute or compile the kernel.

import ctypes as C,pathlib,sys,struct,hashlib
lib=C.CDLL(sys.argv[1]);out=pathlib.Path(sys.argv[2]);
lib.nvFatbinAddPTX.argtypes=[C.c_void_p,C.c_char_p,C.c_size_t,C.c_char_p,C.c_char_p,C.c_char_p]
lib.nvFatbinCreate.argtypes=[C.POINTER(C.c_void_p),C.POINTER(C.c_char_p),C.c_size_t]
lib.nvFatbinSize.argtypes=[C.c_void_p,C.POINTER(C.c_size_t)]
lib.nvFatbinGet.argtypes=[C.c_void_p,C.c_void_p]
lib.nvFatbinDestroy.argtypes=[C.POINTER(C.c_void_p)]
def check(code):
 if code:raise RuntimeError(code)
major=C.c_uint();minor=C.c_uint();check(lib.nvFatbinVersion(C.byref(major),C.byref(minor)))
print('nvFatbinVersion',major.value,minor.value)
for arch in ['100','100f']:
 h=C.c_void_p();opts=(C.c_char_p*2)(b'-64',b'-compress=false')
 check(lib.nvFatbinCreate(C.byref(h),opts,2))
 try:
  ptx=f'.version 8.8\n.target sm_{arch}\n.address_size 64\n.visible .entry kernel() {{ret;}}\n'.encode()+b'\0'
  check(lib.nvFatbinAddPTX(h,ptx,len(ptx),arch.encode(),b'kernel',b''))
  size=C.c_size_t();check(lib.nvFatbinSize(h,C.byref(size)));buf=C.create_string_buffer(size.value);check(lib.nvFatbinGet(h,buf))
  data=buf.raw;(out/f'sm_{arch}.fatbin').write_bytes(data)
  print(arch,len(data),'flags',hex(struct.unpack_from('<Q',data,56)[0]),'sha256',hashlib.sha256(data).hexdigest())
 finally:check(lib.nvFatbinDestroy(C.byref(h)))

Save the following as $WORK/probe.mts. From the REA checkout run node node_modules/tsx/dist/cli.mjs "$WORK/probe.mts" "$PWD" "$WORK". It inspects each real producer artifact in inventory/kernel modes with and without its exact target filter. Each of the eight service combinations ran once, before drafting.

import {pathToFileURL} from 'node:url';
import {join,resolve} from 'node:path';
const repo=resolve(process.argv[2]),work=resolve(process.argv[3]);
const load=(p:string)=>import(pathToFileURL(join(repo,p)).href);
const {GpuCodeObjectsService}=await load('src/application/gpu/GpuCodeObjectsService.ts');
const {GpuCodeObjectsProvider}=await load('src/gpu/GpuCodeObjectsProvider.ts');
const service=new GpuCodeObjectsService(new GpuCodeObjectsProvider());
try{
 for(const arch of ['100','100f']){
  for(const analysis of ['inventory','kernels']){
   for(const filter of [false,true]){
    const result=await service.inspect({path:join(work,`sm_${arch}.fatbin`),analysis,detail:'complete',...(filter?{targets:[`sm_${arch}`]}:{})});
    if(!result.ok)throw result.error;
    const r=result.value.normalized_result;
    console.log(JSON.stringify({arch,analysis,filter,coverage:r.coverage,objects:r.objects.map(({target,payload_target,target_consistency,raw_flags,inspection,limitations}:any)=>({target,payload_target,target_consistency,raw_flags,inspection,limitations})),kernels:r.kernels.map(({name}:any)=>name)}));
   }
  }
 }
}finally{await service.close();}

Expected behavior

Inventory should retain sm_100f, agree with the matching PTX declaration, and allow the exact sm_100f filter to select kernel kernel. NVIDIA documents sm_100f as a supported distinct target, and its nvFatbin API validates the supplied PTX/architecture pair. The observed successful producer output establishes the concrete family flag representation without relying on a guessed header edit. REA documents target inventory, exact selection and inner/outer target agreement in docs/gpu-code-objects.md.

Impact and scope

The observed inventory loses the family qualification, a matching producer declaration is reported as conflicting, and exact family-target selection loses the only kernel. This affects the demonstrated real uncompressed PTX fatbinary; CUBIN and compressed variants were not tested. Broader family suffixes and frequency among user artifacts remain unknown. Omitting the target filter returns the kernel and retains payload_target: "sm_100f", making manual attribution possible; that workaround was observed. Source bytes remain unchanged.

REA version

Package 6.7.0; source revision 21b836a32710a575357715a798beadd0f3976b8a. This source package/revision is the REA baseline; an installed launcher was not used.

Runtime and platform

Node.js v26.12.1, Python 3, Linux x64, repository-locked tsx. Producer: NVIDIA nvFatbin 13.4.92 (runtime version API: 13.4), extracted from its public wheel into disposable storage. No GPU or driver was required.

Target and artifact context

Two real NVIDIA-produced 184-byte CUDA fatbinaries, each containing one uncompressed PTX 8.8 entry with kernel kernel and zero arguments. The control declares sm_100; the affected entry declares sm_100f. The latter PTX and fatbinary target declarations originate from the same successful API call.

Interface

Other

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

The generator produced:

nvFatbinVersion 13 4
100 184 flags 0x11 sha256 f26f448e9cbbfcb4153718721ba3b2a725cd4a1c18d11d75347e29024013838f
100f 184 flags 0x200011 sha256 f1bce13730470213e7635f7e2b2f813dfb03df68863010966b696d37585faa27

Selected service output for the feature case:

[
  {"analysis":"inventory","filter":false,"target":"sm_100","payload_target":null,"target_consistency":"unexamined","raw_flags":"0x200011","inspection":"inventory","discovery":"complete","payloads":"not-requested","kernels":[]},
  {"analysis":"kernels","filter":false,"target":"sm_100","payload_target":"sm_100f","target_consistency":"conflict","raw_flags":"0x200011","inspection":"partial","discovery":"complete","payloads":"partial","kernels":["kernel"]},
  {"analysis":"kernels","filter":true,"target":"sm_100","payload_target":null,"target_consistency":"unexamined","raw_flags":"0x200011","inspection":"not-selected","discovery":"complete","payloads":"partial","kernels":[]}
]

These are selected fields from the actual JSON results, flattened to retain the meaningful output. The unfiltered affected object limitation is exactly Conflicting target declarations: inventory sm_100; payload sm_100f. Target agreement is not established. Both kernel-mode sm_100 controls returned target: "sm_100", payload_target: "sm_100", target_consistency: "consistent", inspection: "complete", payloads: "complete" and ["kernel"].

Evidence provenance: observed producer output and production-service results; diagnosis below is source-derived.
Local verification: reproduced.
Reproduction completeness: complete for real producer generation and the static shared-service boundary. CLI/MCP transport and GPU execution were not run.

Typed outcome or recovery expected

Return target: "sm_100f" and consistent target agreement for this producer artifact, and retain the kernel under the exact family-target selector. A restoration check should preserve the target spelling and artifact bytes and keep the plain-target control passing.

Analysis or suspected cause

Source-derived: src/gpu/CudaFatbinary.ts:50 only appends a for flag 0x100000; it never appends the f represented by 0x200000 in the successful producer output. The exact target selection in src/gpu/GpuInventory.ts:119 then excludes sm_100f, while unfiltered payload decoding correctly reads that PTX target and compares it against the truncated inventory declaration. Public service admission accepts the artifact and both selector values, and no unsupported-header guard fires.

Additional context

The open-issue snapshot's GPU-related titles/bodies and a body search for sm_100f, family-specific targets and the relevant flags found no matching report. A hit in a generic CLI Evidence issue concerned a different operation. Open and closed issues were searched before submission. NOT_ENOUGH_INFO: the first affected release and prevalence in deployed binaries were not investigated.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: A supported target qualification is lost from a real producer container, creating incorrect inventory, a false declaration conflict and an empty exact-target kernel result; the plain-target producer control succeeds.
Workaround status: available
Workaround basis: Unfiltered kernel inspection returned the kernel and the correct payload target in the observed affected artifact, allowing manual filtering/attribution without changing input bytes.

Originally created by @coygeek on GitHub (Oct 23, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6618 ## Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ## Area Native analysis or decompilation ## Observed behavior A real NVIDIA nvFatbin-produced `sm_100f` container is inventoried as `sm_100`. Unfiltered kernel inspection then reports a false target conflict between inventory `sm_100` and payload `sm_100f`. Selecting `targets: ["sm_100f"]` returns no kernels and marks the only object `not-selected`. The corresponding `sm_100` producer control retains its target and returns its kernel with complete coverage. The producer library and production REA shared service/provider ran with real files. No GPU driver, target execution, CUDA assembler, CLI/MCP transport or native decompiler was run. ## Reproduction Use the reported source checkout with locked dependencies, Python 3, and NVIDIA's `nvidia-nvfatbin` wheel version 13.4.92, extracted into a disposable directory. This was the latest version returned by PyPI during the probe. `NVFATBIN_LIBRARY` denotes its `nvidia/cu13/lib/libnvfatbin.so.13`; no system installation is required. The tested Linux x86_64 wheel's SHA-256 is `7fb72f39eaca2097e146b588411805074d2c6086e49412c55f553633e40e7f3c`. Save this generator as `$WORK/make-fatbin.py`, then run `python "$WORK/make-fatbin.py" "$NVFATBIN_LIBRARY" "$WORK"`. All producer calls returned success, and `nvFatbinVersion` returned major 13, minor 4. The library packs supplied PTX into a fatbinary; it does not execute or compile the kernel. ```python import ctypes as C,pathlib,sys,struct,hashlib lib=C.CDLL(sys.argv[1]);out=pathlib.Path(sys.argv[2]); lib.nvFatbinAddPTX.argtypes=[C.c_void_p,C.c_char_p,C.c_size_t,C.c_char_p,C.c_char_p,C.c_char_p] lib.nvFatbinCreate.argtypes=[C.POINTER(C.c_void_p),C.POINTER(C.c_char_p),C.c_size_t] lib.nvFatbinSize.argtypes=[C.c_void_p,C.POINTER(C.c_size_t)] lib.nvFatbinGet.argtypes=[C.c_void_p,C.c_void_p] lib.nvFatbinDestroy.argtypes=[C.POINTER(C.c_void_p)] def check(code): if code:raise RuntimeError(code) major=C.c_uint();minor=C.c_uint();check(lib.nvFatbinVersion(C.byref(major),C.byref(minor))) print('nvFatbinVersion',major.value,minor.value) for arch in ['100','100f']: h=C.c_void_p();opts=(C.c_char_p*2)(b'-64',b'-compress=false') check(lib.nvFatbinCreate(C.byref(h),opts,2)) try: ptx=f'.version 8.8\n.target sm_{arch}\n.address_size 64\n.visible .entry kernel() {{ret;}}\n'.encode()+b'\0' check(lib.nvFatbinAddPTX(h,ptx,len(ptx),arch.encode(),b'kernel',b'')) size=C.c_size_t();check(lib.nvFatbinSize(h,C.byref(size)));buf=C.create_string_buffer(size.value);check(lib.nvFatbinGet(h,buf)) data=buf.raw;(out/f'sm_{arch}.fatbin').write_bytes(data) print(arch,len(data),'flags',hex(struct.unpack_from('<Q',data,56)[0]),'sha256',hashlib.sha256(data).hexdigest()) finally:check(lib.nvFatbinDestroy(C.byref(h))) ``` Save the following as `$WORK/probe.mts`. From the REA checkout run `node node_modules/tsx/dist/cli.mjs "$WORK/probe.mts" "$PWD" "$WORK"`. It inspects each real producer artifact in inventory/kernel modes with and without its exact target filter. Each of the eight service combinations ran once, before drafting. ```ts import {pathToFileURL} from 'node:url'; import {join,resolve} from 'node:path'; const repo=resolve(process.argv[2]),work=resolve(process.argv[3]); const load=(p:string)=>import(pathToFileURL(join(repo,p)).href); const {GpuCodeObjectsService}=await load('src/application/gpu/GpuCodeObjectsService.ts'); const {GpuCodeObjectsProvider}=await load('src/gpu/GpuCodeObjectsProvider.ts'); const service=new GpuCodeObjectsService(new GpuCodeObjectsProvider()); try{ for(const arch of ['100','100f']){ for(const analysis of ['inventory','kernels']){ for(const filter of [false,true]){ const result=await service.inspect({path:join(work,`sm_${arch}.fatbin`),analysis,detail:'complete',...(filter?{targets:[`sm_${arch}`]}:{})}); if(!result.ok)throw result.error; const r=result.value.normalized_result; console.log(JSON.stringify({arch,analysis,filter,coverage:r.coverage,objects:r.objects.map(({target,payload_target,target_consistency,raw_flags,inspection,limitations}:any)=>({target,payload_target,target_consistency,raw_flags,inspection,limitations})),kernels:r.kernels.map(({name}:any)=>name)})); } } } }finally{await service.close();} ``` ## Expected behavior Inventory should retain `sm_100f`, agree with the matching PTX declaration, and allow the exact `sm_100f` filter to select kernel `kernel`. NVIDIA documents [`sm_100f` as a supported distinct target](https://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/index.html#gpu-name-gpuname-arch), and its [nvFatbin API](https://docs.nvidia.com/cuda/nvfatbin/index.html#nvFatbinAddPTX) validates the supplied PTX/architecture pair. The observed successful producer output establishes the concrete family flag representation without relying on a guessed header edit. REA documents target inventory, exact selection and inner/outer target agreement in `docs/gpu-code-objects.md`. ## Impact and scope The observed inventory loses the family qualification, a matching producer declaration is reported as conflicting, and exact family-target selection loses the only kernel. This affects the demonstrated real uncompressed PTX fatbinary; CUBIN and compressed variants were not tested. Broader family suffixes and frequency among user artifacts remain unknown. Omitting the target filter returns the kernel and retains `payload_target: "sm_100f"`, making manual attribution possible; that workaround was observed. Source bytes remain unchanged. ## REA version Package 6.7.0; source revision `21b836a32710a575357715a798beadd0f3976b8a`. This source package/revision is the REA baseline; an installed launcher was not used. ## Runtime and platform Node.js v26.12.1, Python 3, Linux x64, repository-locked tsx. Producer: NVIDIA nvFatbin 13.4.92 (runtime version API: 13.4), extracted from its public wheel into disposable storage. No GPU or driver was required. ## Target and artifact context Two real NVIDIA-produced 184-byte CUDA fatbinaries, each containing one uncompressed PTX 8.8 entry with kernel `kernel` and zero arguments. The control declares `sm_100`; the affected entry declares `sm_100f`. The latter PTX and fatbinary target declarations originate from the same successful API call. ## Interface Other ## Earliest failure stage Native analysis or reconstruction ## Relevant output or evidence The generator produced: ```text nvFatbinVersion 13 4 100 184 flags 0x11 sha256 f26f448e9cbbfcb4153718721ba3b2a725cd4a1c18d11d75347e29024013838f 100f 184 flags 0x200011 sha256 f1bce13730470213e7635f7e2b2f813dfb03df68863010966b696d37585faa27 ``` Selected service output for the feature case: ```json [ {"analysis":"inventory","filter":false,"target":"sm_100","payload_target":null,"target_consistency":"unexamined","raw_flags":"0x200011","inspection":"inventory","discovery":"complete","payloads":"not-requested","kernels":[]}, {"analysis":"kernels","filter":false,"target":"sm_100","payload_target":"sm_100f","target_consistency":"conflict","raw_flags":"0x200011","inspection":"partial","discovery":"complete","payloads":"partial","kernels":["kernel"]}, {"analysis":"kernels","filter":true,"target":"sm_100","payload_target":null,"target_consistency":"unexamined","raw_flags":"0x200011","inspection":"not-selected","discovery":"complete","payloads":"partial","kernels":[]} ] ``` These are selected fields from the actual JSON results, flattened to retain the meaningful output. The unfiltered affected object limitation is exactly `Conflicting target declarations: inventory sm_100; payload sm_100f. Target agreement is not established.` Both kernel-mode `sm_100` controls returned `target: "sm_100"`, `payload_target: "sm_100"`, `target_consistency: "consistent"`, `inspection: "complete"`, `payloads: "complete"` and `["kernel"]`. - Evidence provenance: observed producer output and production-service results; diagnosis below is source-derived. - Local verification: reproduced. - Reproduction completeness: complete for real producer generation and the static shared-service boundary. CLI/MCP transport and GPU execution were not run. ## Typed outcome or recovery expected Return `target: "sm_100f"` and consistent target agreement for this producer artifact, and retain the kernel under the exact family-target selector. A restoration check should preserve the target spelling and artifact bytes and keep the plain-target control passing. ## Analysis or suspected cause Source-derived: `src/gpu/CudaFatbinary.ts:50` only appends `a` for flag `0x100000`; it never appends the `f` represented by `0x200000` in the successful producer output. The exact target selection in `src/gpu/GpuInventory.ts:119` then excludes `sm_100f`, while unfiltered payload decoding correctly reads that PTX target and compares it against the truncated inventory declaration. Public service admission accepts the artifact and both selector values, and no unsupported-header guard fires. ## Additional context The open-issue snapshot's GPU-related titles/bodies and a body search for `sm_100f`, family-specific targets and the relevant flags found no matching report. A hit in a generic CLI Evidence issue concerned a different operation. Open and closed issues were searched before submission. NOT_ENOUGH_INFO: the first affected release and prevalence in deployed binaries were not investigated. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: A supported target qualification is lost from a real producer container, creating incorrect inventory, a false declaration conflict and an empty exact-target kernel result; the plain-target producer control succeeds. - Workaround status: available - Workaround basis: Unfiltered kernel inspection returned the kernel and the correct payload target in the observed affected artifact, allowing manual filtering/attribution without changing input bytes.

bsakaguchi closed this issue

2026-10-31 00:45:58 +00:00

bsakaguchi referenced this issue

2026-10-31 00:52:08 +00:00

[GH-ISSUE #8032] [Bug] Process capture reports cleanup_outcome "not_required" after its cleanup killed a detached descendant #4661

bsakaguchi referenced this issue

2026-10-31 01:12:11 +00:00

[PR #4238] [MERGED] fix(lifecycle): preserve ownership, cancellation, and cleanup outcomes #6141