morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #6739] [Bug] Rust inspection reports dependency metadata absent when only the ELF section-name table is compressed #4102

New issue

Closed

opened 2026-10-31 00:46:50 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:46:50 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 23, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6739

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

The shared production RustBinaryService.inspect({path}) workflow returns successful Evidence with auditable_metadata: null and “No .dep-v0 section was found” after elfutils compresses only an ELF's .shstrtab. The .dep-v0 section remains present, uncompressed at the ELF layer, and contains the same zlib-compressed package document. Uncompressed controls return both packages and their dependency edge.

This was executed against source modules, not through the CLI or MCP transport. Synthetic ELF32 and ELF64 executables both reproduce it. Elfutils validates all four inputs, and an independent reader that decompresses the name table recovers identical dependency payloads from them.

Reproduction

Prerequisites: this source checkout with its locked dependencies installed, Node v24.22.1, and elfutils (eu-elfcompress and eu-elflint). Save the following executed script as repro.mjs. Set CHECKOUT to the checkout's absolute path, create an empty WORK_DIR, then run:

node --import "$CHECKOUT/node_modules/tsx/dist/loader.mjs"   repro.mjs "$CHECKOUT" "$WORK_DIR"

import assert from 'node:assert/strict';
import { readFileSync, writeFileSync } from 'node:fs';
import { execFileSync } from 'node:child_process';
import { join } from 'node:path';
import { pathToFileURL } from 'node:url';
import { deflateSync } from 'node:zlib';
const [repo, out] = process.argv.slice(2);
const document = {format:1,packages:[{name:'example_dep',version:'1.2.3',source:'crates.io'},{name:'example_app',version:'0.1.0',source:'local',dependencies:[0],root:true}]};
const payload = deflateSync(Buffer.from(JSON.stringify(document)));
const names = Buffer.from('\0.dep-v0\0.shstrtab\0.text\0');
const b = Buffer.alloc(4097);
const put = (at, value, width=4) => width===8 ? b.writeBigUInt64LE(BigInt(value),at) : width===2 ? b.writeUInt16LE(value,at) : b.writeUInt32LE(value,at);
b.set([0x7f,0x45,0x4c,0x46,2,1,1]);
for (const [at,value,width] of [[16,2,2],[18,62,2],[20,1,4],[24,0x401000,8],[32,64,8],[40,128,8],[52,64,2],[54,56,2],[56,1,2],[58,64,2],[60,4,2],[62,2,2],[64,1,4],[68,5,4],[72,4096,8],[80,0x401000,8],[88,0x401000,8],[96,1,8],[104,1,8],[112,4096,8]]) put(at,value,width);
function section(index,name,type,flags,address,offset,size) {
 const h=128+index*64;
 put(h,name); put(h+4,type); put(h+8,flags,8); put(h+16,address,8);
 put(h+24,offset,8); put(h+32,size,8); put(h+48,1,8);
}
section(1,1,1,0,0,1024,payload.length);
section(2,9,3,0,0,512,names.length);
section(3,19,1,6,0x401000,4096,1);
names.copy(b,512); payload.copy(b,1024); b[4096]=0xc3;
const plain=join(out,'repro-control.elf'), compressed=join(out,'repro-compressed.elf'), recovered=join(out,'repro-recovered.elf');
writeFileSync(plain,b);
execFileSync('eu-elfcompress',['-f','-t','zlib','-n','.shstrtab','-o',compressed,plain]);
execFileSync('eu-elfcompress',['-f','-t','none','-n','.shstrtab','-o',recovered,compressed]);
const {RustBinaryProvider}=await import(pathToFileURL(join(repo,'src/rust/RustBinaryProvider.ts')));
const {RustBinaryService}=await import(pathToFileURL(join(repo,'src/application/rust/RustBinaryService.ts')));
const service=new RustBinaryService(new RustBinaryProvider());
const rows=[];
try {
 for (const [label,path] of [['control',plain],['compressed',compressed],['recovered',recovered]]) {
  const lint=execFileSync('eu-elflint',['--gnu-ld',path],{encoding:'utf8'}).trim();
  const before=readFileSync(path);
  const result=await service.inspect({path});
  assert(result.ok,JSON.stringify(result));
  assert.deepEqual(readFileSync(path),before);
  const r=result.value.normalized_result;
  const row={label,lint,packages:r.auditable_metadata?.packages.map(p=>p.name)??null,edges:r.auditable_metadata?.dependency_edges??null,metadata:r.auditable_metadata,limitations:r.limitations};
  rows.push(row); console.log(JSON.stringify(row));
 }
 assert.equal(rows[0].packages.length,2); assert.equal(rows[1].metadata,null);
 assert.deepEqual(rows[2].packages,rows[0].packages); assert.deepEqual(rows[2].edges,rows[0].edges);
} finally {await service.close();}
writeFileSync(join(out,'repro-evidence.json'),JSON.stringify(rows,null,2));

The script creates an inert ELF64 executable, compresses only its section-name table with normal elfutils options, and inspects the control, compressed copy, and a recovered copy. It never executes those files. The executable's one-byte text section merely makes the structural fixture a valid executable. eu-elflint --gnu-ld prints No errors for all three.

This script ran once before drafting. A separate two-class probe also ran once for each ELF32/ELF64 control/compressed case. Every inspected file remained byte-for-byte unchanged.

Expected behavior

Recover the same embedded package names and dependency edge when only the valid section-name table is compressed. If compressed names are unsupported, identify that limitation explicitly instead of concluding that the metadata section is absent.

REA's Rust guide promises .dep-v0 package and dependency inspection, and distinguishes missing sections from unsupported compressed dependency sections. The dependency section in these fixtures is unchanged; only .shstrtab has SHF_COMPRESSED.

The ELF compression rules permit compression of nonallocated sections and separate the compression header/physical bytes from the logical section contents. Section-name offsets must address the logical string table. Elfutils' successful validation and name decoding independently establish that these fixtures retain .dep-v0.

Impact and scope

Successful Rust Evidence silently loses the entire embedded two-package graph for a bounded class of valid ELF files. The misleading absence limitation can cause callers to stop looking for existing dependency facts. ELF32/i386 and ELF64/x86-64 little-endian synthetic executables were exercised; natural-producer prevalence, big-endian ELF, other compression algorithms, CLI/MCP execution, and the published package runtime were not tested.

A demonstrated workaround is to use eu-elfcompress -t none -n .shstrtab to create a separate decompressed copy. REA then returns the original package identities and edge from that copy. Its file digest and offsets correctly identify the transformed artifact, not the original. Independent name-table decompression and zlib/JSON extraction also recovered the original document without changing the original file.

REA version

Executed source revision 76b56957e94d804667f7890b244639ecc4725b32; package version 6.7.0. The published package runtime was not executed.

Runtime and platform

Node v24.22.1; Linux 7.2.8-9-omarchy x86_64; elfutils 0.197. No Rust compiler, disassembler, live process, or target execution was used.

Target and artifact context

Synthetic ELF32 and ELF64 little-endian executables with an uncompressed ELF SHT_PROGBITS .dep-v0 section containing cargo-auditable format-1 JSON. The document has example_dep 1.2.3 and root example_app 0.1.0, with dependency index 1 → 0. Compression affects only the nonallocated SHT_STRTAB section named .shstrtab.

Interface

Other — executed shared RustBinaryService and bundled RustBinaryProvider source modules.

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

Reduced output from the self-contained ELF64 reproduction:

control:    lint = No errors; packages = [example_dep, example_app]; edges = [{from_index:1,to_index:0}]
compressed: lint = No errors; auditable_metadata = null
recovered:  lint = No errors; packages = [example_dep, example_app]; edges = [{from_index:1,to_index:0}]

The compressed case includes this exact limitation:

No .dep-v0 section was found. Absence of .dep-v0 does not mean the binary has no dependencies.

The separate ELF32/ELF64 probe independently decoded each name table, extracted .dep-v0, inflated it, and parsed the same document. All four dependency payloads had SHA-256 6545c38991b1c6d820696afcb030016df29ef0c1e76b23c6b3313f7c2adae793. Source revision, runtime, synthetic containers, production-boundary results, independent extraction, and recovery were observed before this draft was written. No CLI/MCP reproduction or packaged-runtime equivalence is claimed.

Typed outcome or recovery expected

Return the dependency metadata, or a typed unsupported/unknown section-name decoding outcome with an accurate limitation. Do not substitute a successful absent-section result for unresolved compressed names.

Analysis or suspected cause

Source-derived: NativeSectionDirectory.ts:152-161 reads the physical .shstrtab buffer without checking SHF_COMPRESSED. 174-192 applies logical name offsets directly to that buffer. Here, .dep-v0's offset 1 reaches a zero byte in the compression header.

ElfImage.ts:70 drops the falsely empty-named section. RustBinaryImage.ts:76-84 then returns no dependency section, and RustBinaryProvider.ts:175-178 presents that loss as an absence limitation.

Additional context

Related #6692 concerns section-name loss in the separate standalone ElfNoteParser.ts. This report identifies the shared NativeSectionDirectory.ts boundary and loss of Rust package/dependency facts. Maintainers may prefer consolidation. #3397 and #6663 concern compiler-path inference, not discovery of a present dependency section.

No fix was made, and no last-known-good revision was established.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: Executed controls and validated compressed ELF32/ELF64 fixtures show successful Rust Evidence losing a complete two-package dependency graph while incorrectly describing its section as absent; the failure is bounded to compressed name tables.
Workaround status: available
Workaround basis: The executed ELF64 reproduction decompresses only .shstrtab into a separate copy and observes restoration of both package identities and the dependency edge. Independent extraction also recovers the original document without modifying the original artifact; REA Evidence for the unchanged original remains unavailable.

Originally created by @coygeek on GitHub (Oct 23, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6739 ## Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ## Area Evidence, artifacts, or reconstruction ## Observed behavior The shared production `RustBinaryService.inspect({path})` workflow returns successful Evidence with `auditable_metadata: null` and “No .dep-v0 section was found” after elfutils compresses only an ELF's `.shstrtab`. The `.dep-v0` section remains present, uncompressed at the ELF layer, and contains the same zlib-compressed package document. Uncompressed controls return both packages and their dependency edge. This was executed against source modules, not through the CLI or MCP transport. Synthetic ELF32 and ELF64 executables both reproduce it. Elfutils validates all four inputs, and an independent reader that decompresses the name table recovers identical dependency payloads from them. ## Reproduction Prerequisites: this source checkout with its locked dependencies installed, Node v24.22.1, and elfutils (`eu-elfcompress` and `eu-elflint`). Save the following executed script as `repro.mjs`. Set `CHECKOUT` to the checkout's absolute path, create an empty `WORK_DIR`, then run: ```sh node --import "$CHECKOUT/node_modules/tsx/dist/loader.mjs" repro.mjs "$CHECKOUT" "$WORK_DIR" ``` ```js import assert from 'node:assert/strict'; import { readFileSync, writeFileSync } from 'node:fs'; import { execFileSync } from 'node:child_process'; import { join } from 'node:path'; import { pathToFileURL } from 'node:url'; import { deflateSync } from 'node:zlib'; const [repo, out] = process.argv.slice(2); const document = {format:1,packages:[{name:'example_dep',version:'1.2.3',source:'crates.io'},{name:'example_app',version:'0.1.0',source:'local',dependencies:[0],root:true}]}; const payload = deflateSync(Buffer.from(JSON.stringify(document))); const names = Buffer.from('\0.dep-v0\0.shstrtab\0.text\0'); const b = Buffer.alloc(4097); const put = (at, value, width=4) => width===8 ? b.writeBigUInt64LE(BigInt(value),at) : width===2 ? b.writeUInt16LE(value,at) : b.writeUInt32LE(value,at); b.set([0x7f,0x45,0x4c,0x46,2,1,1]); for (const [at,value,width] of [[16,2,2],[18,62,2],[20,1,4],[24,0x401000,8],[32,64,8],[40,128,8],[52,64,2],[54,56,2],[56,1,2],[58,64,2],[60,4,2],[62,2,2],[64,1,4],[68,5,4],[72,4096,8],[80,0x401000,8],[88,0x401000,8],[96,1,8],[104,1,8],[112,4096,8]]) put(at,value,width); function section(index,name,type,flags,address,offset,size) { const h=128+index*64; put(h,name); put(h+4,type); put(h+8,flags,8); put(h+16,address,8); put(h+24,offset,8); put(h+32,size,8); put(h+48,1,8); } section(1,1,1,0,0,1024,payload.length); section(2,9,3,0,0,512,names.length); section(3,19,1,6,0x401000,4096,1); names.copy(b,512); payload.copy(b,1024); b[4096]=0xc3; const plain=join(out,'repro-control.elf'), compressed=join(out,'repro-compressed.elf'), recovered=join(out,'repro-recovered.elf'); writeFileSync(plain,b); execFileSync('eu-elfcompress',['-f','-t','zlib','-n','.shstrtab','-o',compressed,plain]); execFileSync('eu-elfcompress',['-f','-t','none','-n','.shstrtab','-o',recovered,compressed]); const {RustBinaryProvider}=await import(pathToFileURL(join(repo,'src/rust/RustBinaryProvider.ts'))); const {RustBinaryService}=await import(pathToFileURL(join(repo,'src/application/rust/RustBinaryService.ts'))); const service=new RustBinaryService(new RustBinaryProvider()); const rows=[]; try { for (const [label,path] of [['control',plain],['compressed',compressed],['recovered',recovered]]) { const lint=execFileSync('eu-elflint',['--gnu-ld',path],{encoding:'utf8'}).trim(); const before=readFileSync(path); const result=await service.inspect({path}); assert(result.ok,JSON.stringify(result)); assert.deepEqual(readFileSync(path),before); const r=result.value.normalized_result; const row={label,lint,packages:r.auditable_metadata?.packages.map(p=>p.name)??null,edges:r.auditable_metadata?.dependency_edges??null,metadata:r.auditable_metadata,limitations:r.limitations}; rows.push(row); console.log(JSON.stringify(row)); } assert.equal(rows[0].packages.length,2); assert.equal(rows[1].metadata,null); assert.deepEqual(rows[2].packages,rows[0].packages); assert.deepEqual(rows[2].edges,rows[0].edges); } finally {await service.close();} writeFileSync(join(out,'repro-evidence.json'),JSON.stringify(rows,null,2)); ``` The script creates an inert ELF64 executable, compresses only its section-name table with normal elfutils options, and inspects the control, compressed copy, and a recovered copy. It never executes those files. The executable's one-byte text section merely makes the structural fixture a valid executable. `eu-elflint --gnu-ld` prints `No errors` for all three. This script ran once before drafting. A separate two-class probe also ran once for each ELF32/ELF64 control/compressed case. Every inspected file remained byte-for-byte unchanged. ## Expected behavior Recover the same embedded package names and dependency edge when only the valid section-name table is compressed. If compressed names are unsupported, identify that limitation explicitly instead of concluding that the metadata section is absent. REA's [Rust guide](https://github.com/morluto/rea/blob/76b56957e94d804667f7890b244639ecc4725b32/docs/rust-binaries.md) promises `.dep-v0` package and dependency inspection, and distinguishes missing sections from unsupported compressed dependency sections. The dependency section in these fixtures is unchanged; only `.shstrtab` has `SHF_COMPRESSED`. The [ELF compression rules](https://gabi.xinuos.com/elf/03-sheader.html#compressed-sections) permit compression of nonallocated sections and separate the compression header/physical bytes from the logical section contents. Section-name offsets must address the logical string table. Elfutils' successful validation and name decoding independently establish that these fixtures retain `.dep-v0`. ## Impact and scope Successful Rust Evidence silently loses the entire embedded two-package graph for a bounded class of valid ELF files. The misleading absence limitation can cause callers to stop looking for existing dependency facts. ELF32/i386 and ELF64/x86-64 little-endian synthetic executables were exercised; natural-producer prevalence, big-endian ELF, other compression algorithms, CLI/MCP execution, and the published package runtime were not tested. A demonstrated workaround is to use `eu-elfcompress -t none -n .shstrtab` to create a separate decompressed copy. REA then returns the original package identities and edge from that copy. Its file digest and offsets correctly identify the transformed artifact, not the original. Independent name-table decompression and zlib/JSON extraction also recovered the original document without changing the original file. ## REA version Executed source revision `76b56957e94d804667f7890b244639ecc4725b32`; package version `6.7.0`. The published package runtime was not executed. ## Runtime and platform Node v24.22.1; Linux 7.2.8-9-omarchy x86_64; elfutils 0.197. No Rust compiler, disassembler, live process, or target execution was used. ## Target and artifact context Synthetic ELF32 and ELF64 little-endian executables with an uncompressed ELF `SHT_PROGBITS` `.dep-v0` section containing cargo-auditable format-1 JSON. The document has `example_dep` 1.2.3 and root `example_app` 0.1.0, with dependency index 1 → 0. Compression affects only the nonallocated `SHT_STRTAB` section named `.shstrtab`. ## Interface Other — executed shared `RustBinaryService` and bundled `RustBinaryProvider` source modules. ## Earliest failure stage Native analysis or reconstruction ## Relevant output or evidence Reduced output from the self-contained ELF64 reproduction: ```text control: lint = No errors; packages = [example_dep, example_app]; edges = [{from_index:1,to_index:0}] compressed: lint = No errors; auditable_metadata = null recovered: lint = No errors; packages = [example_dep, example_app]; edges = [{from_index:1,to_index:0}] ``` The compressed case includes this exact limitation: ```text No .dep-v0 section was found. Absence of .dep-v0 does not mean the binary has no dependencies. ``` The separate ELF32/ELF64 probe independently decoded each name table, extracted `.dep-v0`, inflated it, and parsed the same document. All four dependency payloads had SHA-256 `6545c38991b1c6d820696afcb030016df29ef0c1e76b23c6b3313f7c2adae793`. Source revision, runtime, synthetic containers, production-boundary results, independent extraction, and recovery were observed before this draft was written. No CLI/MCP reproduction or packaged-runtime equivalence is claimed. ## Typed outcome or recovery expected Return the dependency metadata, or a typed unsupported/unknown section-name decoding outcome with an accurate limitation. Do not substitute a successful absent-section result for unresolved compressed names. ## Analysis or suspected cause Source-derived: [`NativeSectionDirectory.ts:152-161`](https://github.com/morluto/rea/blob/76b56957e94d804667f7890b244639ecc4725b32/src/native/metadata/NativeSectionDirectory.ts#L152-L161) reads the physical `.shstrtab` buffer without checking `SHF_COMPRESSED`. [`174-192`](https://github.com/morluto/rea/blob/76b56957e94d804667f7890b244639ecc4725b32/src/native/metadata/NativeSectionDirectory.ts#L174-L192) applies logical name offsets directly to that buffer. Here, `.dep-v0`'s offset 1 reaches a zero byte in the compression header. [`ElfImage.ts:70`](https://github.com/morluto/rea/blob/76b56957e94d804667f7890b244639ecc4725b32/src/native/metadata/ElfImage.ts#L70) drops the falsely empty-named section. [`RustBinaryImage.ts:76-84`](https://github.com/morluto/rea/blob/76b56957e94d804667f7890b244639ecc4725b32/src/rust/RustBinaryImage.ts#L76-L84) then returns no dependency section, and [`RustBinaryProvider.ts:175-178`](https://github.com/morluto/rea/blob/76b56957e94d804667f7890b244639ecc4725b32/src/rust/RustBinaryProvider.ts#L175-L178) presents that loss as an absence limitation. ## Additional context Related [#6692](https://github.com/morluto/rea/issues/6692) concerns section-name loss in the separate standalone `ElfNoteParser.ts`. This report identifies the shared `NativeSectionDirectory.ts` boundary and loss of Rust package/dependency facts. Maintainers may prefer consolidation. [#3397](https://github.com/morluto/rea/issues/3397) and [#6663](https://github.com/morluto/rea/issues/6663) concern compiler-path inference, not discovery of a present dependency section. No fix was made, and no last-known-good revision was established. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: Executed controls and validated compressed ELF32/ELF64 fixtures show successful Rust Evidence losing a complete two-package dependency graph while incorrectly describing its section as absent; the failure is bounded to compressed name tables. - Workaround status: available - Workaround basis: The executed ELF64 reproduction decompresses only `.shstrtab` into a separate copy and observes restoration of both package identities and the dependency edge. Independent extraction also recovers the original document without modifying the original artifact; REA Evidence for the unchanged original remains unavailable.

bsakaguchi closed this issue

2026-10-31 00:46:50 +00:00

bsakaguchi referenced this issue

2026-11-01 21:46:09 +00:00

[PR #6219] [MERGED] feat(native): inspect a universal Mach-O slice in Go and Rust metadata readers #7206