morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #6625] [Bug] Equivalent compute_80 PTX and sm_80 fatbinary targets are reported as conflicting #4011

New issue

Closed

opened 2026-10-31 00:46:01 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:46:01 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 23, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6625

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Native analysis or decompilation

Observed behavior

The production shared GPU service reports target_consistency: "conflict" and partial payload coverage for an NVIDIA nvFatbin-produced container whose PTX declares .target compute_80 and whose directory declares sm_80. Both spellings name the same architecture. Its kernel remains present. An otherwise identical .target sm_80 producer control reports consistent targets and complete coverage.

This ran the real NVIDIA packer and REA source service/provider with files, hashing, parsing, schema validation and Evidence construction. It did not run CLI/MCP transport, a GPU driver, target execution, or a decompiler.

Reproduction

Prerequisites: this source checkout with locked dependencies and a supported Node runtime, Python 3, HTTPS access to PyPI, and the Linux x86_64 NVIDIA nvFatbin wheel. The generator below downloads and extracts the wheel into its supplied scratch directory without installing it. The observed latest wheel was nvidia-nvfatbin 13.4.92; its SHA-256 was 7fb72f39eaca2097e146b588411805074d2c6086e49412c55f553633e40e7f3c, and the library reported 13.4. All producer calls succeeded.

From the checkout, create /tmp/rea-repro and its home, config, and cache directories. Save the following as /tmp/rea-repro/make-real-fatbin.py, then run:

HOME=/tmp/rea-repro/home XDG_CONFIG_HOME=/tmp/rea-repro/config python /tmp/rea-repro/make-real-fatbin.py /tmp/rea-repro

import ctypes as C,hashlib,json,pathlib,sys,urllib.request,zipfile
work=pathlib.Path(sys.argv[1]);meta=json.load(urllib.request.urlopen('https://pypi.org/pypi/nvidia-nvfatbin/13.4.92/json'))
url=next(u for u in meta['urls'] if 'manylinux' in u['filename'] and 'x86_64' in u['filename'])
wheel=work/url['filename'];data=urllib.request.urlopen(url['url']).read();assert hashlib.sha256(data).hexdigest()==url['digests']['sha256'];wheel.write_bytes(data)
with zipfile.ZipFile(wheel) as archive:
 library=next(n for n in archive.namelist() if n.endswith('libnvfatbin.so.13'))
 archive.extract(library,work/'nvfatbin')
lib=C.CDLL(str(work/'nvfatbin'/library))
lib.nvFatbinAddPTX.argtypes=[C.c_void_p,C.c_char_p,C.c_size_t,C.c_char_p,C.c_char_p,C.c_char_p]
lib.nvFatbinCreate.argtypes=[C.POINTER(C.c_void_p),C.POINTER(C.c_char_p),C.c_size_t]
lib.nvFatbinSize.argtypes=[C.c_void_p,C.POINTER(C.c_size_t)]
lib.nvFatbinGet.argtypes=[C.c_void_p,C.c_void_p]
lib.nvFatbinDestroy.argtypes=[C.POINTER(C.c_void_p)]
def check(code):
 if code: raise RuntimeError(code)
major=C.c_uint();minor=C.c_uint();check(lib.nvFatbinVersion(C.byref(major),C.byref(minor)))
print(json.dumps({'package':'nvidia-nvfatbin','version':meta['info']['version'],'wheel_sha256':url['digests']['sha256'],'library_version':[major.value,minor.value]}))
for target in ['sm_80','compute_80']:
 h=C.c_void_p();opts=(C.c_char_p*2)(b'-64',b'-compress=false');check(lib.nvFatbinCreate(C.byref(h),opts,2))
 try:
  ptx=f'.version 8.8\n.target {target}\n.address_size 64\n.visible .entry kernel() {{ ret; }}\n'.encode()+b'\0'
  check(lib.nvFatbinAddPTX(h,ptx,len(ptx),b'80',b'kernel',b''))
  size=C.c_size_t();check(lib.nvFatbinSize(h,C.byref(size)));buf=C.create_string_buffer(size.value);check(lib.nvFatbinGet(h,buf));(work/f'real-{target}.fatbin').write_bytes(buf.raw)
  print(json.dumps({'target':target,'producer_calls':'success','bytes':size.value}))
 finally:check(lib.nvFatbinDestroy(C.byref(h)))

Save this as /tmp/rea-repro/inspect-real-fatbins.mts:

import {pathToFileURL} from 'node:url';
import {join,resolve} from 'node:path';
import {platform,release} from 'node:os';
const repo=resolve(process.argv[2]),work=resolve(process.argv[3]);
const load=(p:string)=>import(pathToFileURL(join(repo,p)).href);
const {GpuCodeObjectsService}=await load('src/application/gpu/GpuCodeObjectsService.ts');
const {GpuCodeObjectsProvider}=await load('src/gpu/GpuCodeObjectsProvider.ts');
console.log(JSON.stringify({node:process.version,platform:platform(),release:release(),arch:process.arch}));
for (const target of ['sm_80','compute_80']) {
 const service=new GpuCodeObjectsService(new GpuCodeObjectsProvider());
 try {
  const result=await service.inspect({path:join(work,`real-${target}.fatbin`),analysis:'kernels',detail:'complete'});
  if (!result.ok) throw result.error;
  const report=result.value.normalized_result;
  console.log(JSON.stringify({input:target,payloads:report.coverage.payloads,objects:report.objects.map(({target,payload_target,target_consistency,inspection,limitations}:any)=>({target,payload_target,target_consistency,inspection,limitations})),kernel_names:report.kernels.map((k:any)=>k.name)}));
 } finally { await service.close(); }
}

Run from the same checkout:

HOME=/tmp/rea-repro/home XDG_CONFIG_HOME=/tmp/rea-repro/config XDG_CACHE_HOME=/tmp/rea-repro/cache node node_modules/tsx/dist/cli.mjs /tmp/rea-repro/inspect-real-fatbins.mts "$PWD" /tmp/rea-repro

The final real-producer pair ran once each with empty stderr. An earlier repository-fixture pair independently showed the same discrepancy before this producer check.

Expected behavior

Keep the source spellings in target and payload_target, but recognize their semantic agreement when comparing them. NVIDIA's PTX target directive accepts compute_xx as a synonym for sm_xx. REA's GPU guide explicitly documents this synonym and preservation of the raw spelling. Equivalent declarations should not introduce a conflict warning or downgrade otherwise complete kernel metadata coverage.

Impact and scope

For the observed valid input, retained evidence incorrectly asserts a target disagreement and marks the object/report partial. The kernel is still returned; no execution compatibility was measured. The paired sm_80 spelling is a demonstrated workaround when generating a fatbinary, but rewriting an existing artifact changes its digest. Other architectures and real-provider execution remain untested.

REA version

6.7.0 source revision 00a9e95de91a4bd4d2a41f8aee704a53fa575750.

Runtime and platform

Node.js v26.12.1; Linux 7.2.8-9-omarchy, x64 (Omarchy). NVIDIA nvFatbin 13.4 from wheel 13.4.92. No Hopper/Ghidra provider used.

Target and artifact context

Two inert uncompressed CUDA fatbinaries produced by NVIDIA nvFatbin, containing the same one-entry PTX body. Only .target sm_80 versus .target compute_80 differs in PTX; both are packed for architecture 80.

Interface

Other

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

Actual output from the real-producer service probe:

{"node":"v26.12.1","platform":"linux","release":"7.2.8-9-omarchy","arch":"x64"}
{"input":"sm_80","payloads":"complete","objects":[{"target":"sm_80","payload_target":"sm_80","target_consistency":"consistent","inspection":"complete","limitations":[]}],"kernel_names":["kernel"]}
{"input":"compute_80","payloads":"partial","objects":[{"target":"sm_80","payload_target":"compute_80","target_consistency":"conflict","inspection":"partial","limitations":["Conflicting target declarations: inventory sm_80; payload compute_80. Target agreement is not established."]}],"kernel_names":["kernel"]}

The producer returned success for both inputs, yielding 184-byte (sm_80) and 192-byte (compute_80) fatbinaries. No native GPU compiler or runtime was used; the packer accepts supplied PTX without compiling it.

Typed outcome or recovery expected

A successful GPU evidence result with both source declarations retained, agreement rather than conflict, and complete object/report payload coverage for this supported one-kernel fixture. Genuine architecture disagreement must remain partial.

Analysis or suspected cause

Source-derived cause: PTX payload inspection passes the raw spelling as an exact comparison key; target comparison compares those strings directly. sm_80 and compute_80 therefore fall through to conflict.

The CLI command and named MCP registration call the same shared service used by this probe.

Additional context

Proposed restoration check, not run: repeat the paired fixture after a fix and require agreement/complete coverage for both spellings while preserving the raw compute_80 payload target; retain an unequal-architecture control as a conflict. Last-known-good version: NOT_ENOUGH_INFO; no historical version was tested. Related closed #6494 concerned standalone-file admission; this report concerns a false comparison after a fatbinary and its PTX have both been admitted.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: A valid real-producer artifact deterministically receives false target-conflict evidence and partial coverage; its kernel remains available, bounding the observed failure.
Workaround status: unverified
Workaround basis: The generated .target sm_80 control succeeds, but repacking changes the source spelling and artifact digest. No workaround preserving the selected artifact was tested.

Originally created by @coygeek on GitHub (Oct 23, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6625 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Native analysis or decompilation ### Observed behavior The production shared GPU service reports `target_consistency: "conflict"` and partial payload coverage for an NVIDIA nvFatbin-produced container whose PTX declares `.target compute_80` and whose directory declares `sm_80`. Both spellings name the same architecture. Its kernel remains present. An otherwise identical `.target sm_80` producer control reports consistent targets and complete coverage. This ran the real NVIDIA packer and REA source service/provider with files, hashing, parsing, schema validation and Evidence construction. It did not run CLI/MCP transport, a GPU driver, target execution, or a decompiler. ### Reproduction Prerequisites: this source checkout with locked dependencies and a supported Node runtime, Python 3, HTTPS access to PyPI, and the Linux x86_64 NVIDIA nvFatbin wheel. The generator below downloads and extracts the wheel into its supplied scratch directory without installing it. The observed latest wheel was `nvidia-nvfatbin` 13.4.92; its SHA-256 was `7fb72f39eaca2097e146b588411805074d2c6086e49412c55f553633e40e7f3c`, and the library reported 13.4. All producer calls succeeded. From the checkout, create `/tmp/rea-repro` and its `home`, `config`, and `cache` directories. Save the following as `/tmp/rea-repro/make-real-fatbin.py`, then run: ```sh HOME=/tmp/rea-repro/home XDG_CONFIG_HOME=/tmp/rea-repro/config python /tmp/rea-repro/make-real-fatbin.py /tmp/rea-repro ``` ```python import ctypes as C,hashlib,json,pathlib,sys,urllib.request,zipfile work=pathlib.Path(sys.argv[1]);meta=json.load(urllib.request.urlopen('https://pypi.org/pypi/nvidia-nvfatbin/13.4.92/json')) url=next(u for u in meta['urls'] if 'manylinux' in u['filename'] and 'x86_64' in u['filename']) wheel=work/url['filename'];data=urllib.request.urlopen(url['url']).read();assert hashlib.sha256(data).hexdigest()==url['digests']['sha256'];wheel.write_bytes(data) with zipfile.ZipFile(wheel) as archive: library=next(n for n in archive.namelist() if n.endswith('libnvfatbin.so.13')) archive.extract(library,work/'nvfatbin') lib=C.CDLL(str(work/'nvfatbin'/library)) lib.nvFatbinAddPTX.argtypes=[C.c_void_p,C.c_char_p,C.c_size_t,C.c_char_p,C.c_char_p,C.c_char_p] lib.nvFatbinCreate.argtypes=[C.POINTER(C.c_void_p),C.POINTER(C.c_char_p),C.c_size_t] lib.nvFatbinSize.argtypes=[C.c_void_p,C.POINTER(C.c_size_t)] lib.nvFatbinGet.argtypes=[C.c_void_p,C.c_void_p] lib.nvFatbinDestroy.argtypes=[C.POINTER(C.c_void_p)] def check(code): if code: raise RuntimeError(code) major=C.c_uint();minor=C.c_uint();check(lib.nvFatbinVersion(C.byref(major),C.byref(minor))) print(json.dumps({'package':'nvidia-nvfatbin','version':meta['info']['version'],'wheel_sha256':url['digests']['sha256'],'library_version':[major.value,minor.value]})) for target in ['sm_80','compute_80']: h=C.c_void_p();opts=(C.c_char_p*2)(b'-64',b'-compress=false');check(lib.nvFatbinCreate(C.byref(h),opts,2)) try: ptx=f'.version 8.8\n.target {target}\n.address_size 64\n.visible .entry kernel() {{ ret; }}\n'.encode()+b'\0' check(lib.nvFatbinAddPTX(h,ptx,len(ptx),b'80',b'kernel',b'')) size=C.c_size_t();check(lib.nvFatbinSize(h,C.byref(size)));buf=C.create_string_buffer(size.value);check(lib.nvFatbinGet(h,buf));(work/f'real-{target}.fatbin').write_bytes(buf.raw) print(json.dumps({'target':target,'producer_calls':'success','bytes':size.value})) finally:check(lib.nvFatbinDestroy(C.byref(h))) ``` Save this as `/tmp/rea-repro/inspect-real-fatbins.mts`: ```ts import {pathToFileURL} from 'node:url'; import {join,resolve} from 'node:path'; import {platform,release} from 'node:os'; const repo=resolve(process.argv[2]),work=resolve(process.argv[3]); const load=(p:string)=>import(pathToFileURL(join(repo,p)).href); const {GpuCodeObjectsService}=await load('src/application/gpu/GpuCodeObjectsService.ts'); const {GpuCodeObjectsProvider}=await load('src/gpu/GpuCodeObjectsProvider.ts'); console.log(JSON.stringify({node:process.version,platform:platform(),release:release(),arch:process.arch})); for (const target of ['sm_80','compute_80']) { const service=new GpuCodeObjectsService(new GpuCodeObjectsProvider()); try { const result=await service.inspect({path:join(work,`real-${target}.fatbin`),analysis:'kernels',detail:'complete'}); if (!result.ok) throw result.error; const report=result.value.normalized_result; console.log(JSON.stringify({input:target,payloads:report.coverage.payloads,objects:report.objects.map(({target,payload_target,target_consistency,inspection,limitations}:any)=>({target,payload_target,target_consistency,inspection,limitations})),kernel_names:report.kernels.map((k:any)=>k.name)})); } finally { await service.close(); } } ``` Run from the same checkout: ```sh HOME=/tmp/rea-repro/home XDG_CONFIG_HOME=/tmp/rea-repro/config XDG_CACHE_HOME=/tmp/rea-repro/cache node node_modules/tsx/dist/cli.mjs /tmp/rea-repro/inspect-real-fatbins.mts "$PWD" /tmp/rea-repro ``` The final real-producer pair ran once each with empty stderr. An earlier repository-fixture pair independently showed the same discrepancy before this producer check. ### Expected behavior Keep the source spellings in `target` and `payload_target`, but recognize their semantic agreement when comparing them. NVIDIA's [PTX target directive](https://docs.nvidia.com/cuda/parallel-thread-execution/#ptx-module-directives-target) accepts `compute_xx` as a synonym for `sm_xx`. REA's [GPU guide](https://github.com/morluto/rea/blob/00a9e95de91a4bd4d2a41f8aee704a53fa575750/docs/gpu-code-objects.md#L79-L86) explicitly documents this synonym and preservation of the raw spelling. Equivalent declarations should not introduce a conflict warning or downgrade otherwise complete kernel metadata coverage. ### Impact and scope For the observed valid input, retained evidence incorrectly asserts a target disagreement and marks the object/report partial. The kernel is still returned; no execution compatibility was measured. The paired `sm_80` spelling is a demonstrated workaround when generating a fatbinary, but rewriting an existing artifact changes its digest. Other architectures and real-provider execution remain untested. ### REA version 6.7.0 source revision `00a9e95de91a4bd4d2a41f8aee704a53fa575750`. ### Runtime and platform Node.js v26.12.1; Linux 7.2.8-9-omarchy, x64 (Omarchy). NVIDIA nvFatbin 13.4 from wheel 13.4.92. No Hopper/Ghidra provider used. ### Target and artifact context Two inert uncompressed CUDA fatbinaries produced by NVIDIA nvFatbin, containing the same one-entry PTX body. Only `.target sm_80` versus `.target compute_80` differs in PTX; both are packed for architecture `80`. ### Interface Other ### Earliest failure stage Native analysis or reconstruction ### Relevant output or evidence Actual output from the real-producer service probe: ```jsonl {"node":"v26.12.1","platform":"linux","release":"7.2.8-9-omarchy","arch":"x64"} {"input":"sm_80","payloads":"complete","objects":[{"target":"sm_80","payload_target":"sm_80","target_consistency":"consistent","inspection":"complete","limitations":[]}],"kernel_names":["kernel"]} {"input":"compute_80","payloads":"partial","objects":[{"target":"sm_80","payload_target":"compute_80","target_consistency":"conflict","inspection":"partial","limitations":["Conflicting target declarations: inventory sm_80; payload compute_80. Target agreement is not established."]}],"kernel_names":["kernel"]} ``` The producer returned success for both inputs, yielding 184-byte (`sm_80`) and 192-byte (`compute_80`) fatbinaries. No native GPU compiler or runtime was used; the packer accepts supplied PTX without compiling it. ### Typed outcome or recovery expected A successful GPU evidence result with both source declarations retained, agreement rather than `conflict`, and complete object/report payload coverage for this supported one-kernel fixture. Genuine architecture disagreement must remain partial. ### Analysis or suspected cause Source-derived cause: [PTX payload inspection](https://github.com/morluto/rea/blob/00a9e95de91a4bd4d2a41f8aee704a53fa575750/src/gpu/GpuInventory.ts#L351-L357) passes the raw spelling as an exact comparison key; [target comparison](https://github.com/morluto/rea/blob/00a9e95de91a4bd4d2a41f8aee704a53fa575750/src/gpu/GpuInventory.ts#L391-L409) compares those strings directly. `sm_80` and `compute_80` therefore fall through to conflict. The [CLI command](https://github.com/morluto/rea/blob/00a9e95de91a4bd4d2a41f8aee704a53fa575750/src/cli/gpuCommands.ts#L99-L119) and [named MCP registration](https://github.com/morluto/rea/blob/00a9e95de91a4bd4d2a41f8aee704a53fa575750/src/server/registerGpuCodeObjectsTools.ts#L20-L35) call the same shared service used by this probe. ### Additional context Proposed restoration check, not run: repeat the paired fixture after a fix and require agreement/complete coverage for both spellings while preserving the raw `compute_80` payload target; retain an unequal-architecture control as a conflict. Last-known-good version: NOT_ENOUGH_INFO; no historical version was tested. Related closed [#6494](https://github.com/morluto/rea/issues/6494) concerned standalone-file admission; this report concerns a false comparison after a fatbinary and its PTX have both been admitted. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: A valid real-producer artifact deterministically receives false target-conflict evidence and partial coverage; its kernel remains available, bounding the observed failure. - Workaround status: unverified - Workaround basis: The generated `.target sm_80` control succeeds, but repacking changes the source spelling and artifact digest. No workaround preserving the selected artifact was tested.

bsakaguchi closed this issue

2026-10-31 00:46:02 +00:00

bsakaguchi referenced this issue

2026-10-31 01:12:58 +00:00

[PR #4435] [MERGED] fix(pwntools): preserve worker ownership across launcher exec #6241