morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #6692] [Bug] inspect-elf-notes silently loses note section names when .shstrtab is compressed #4067

New issue

Closed

opened 2026-10-31 00:46:32 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:46:32 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 23, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6692

Before submitting

I searched the open and closed issues for a related report (shstrtab, compressed section name, and inspect_elf_notes).
I removed credentials, tokens, private paths, and sensitive target data.

Area

Native analysis or decompilation

Observed behavior

The built public CLI exits 0 and returns tables[0].name: "" for valid ELF32 and ELF64 objects whose section-name string table is zlib-compressed. It reports complete coverage without a name-decoding limitation. The note section is actually named .note.gnu.build-id: eu-readelf resolves that name from the same files, and eu-elflint --gnu-ld reports No errors. Uncompressed controls return the correct name. GNU build-id decoding still succeeds in all four cases.

Reproduction

Run from the prepared source checkout at the revision below, with its locked dependencies and compiled runtime already available. Prerequisites: Node.js, Python 3, and elfutils (eu-elfcompress, eu-readelf, eu-elflint). The following is the focused probe actually executed: it builds inert minimal objects, uses elfutils to compress only .shstrtab, validates each file, and invokes the public CLI. It does not execute an object file.

work_dir="$(mktemp -d)"
cat > "$work_dir/repro.py" <<'PYTHON'
import hashlib
import json
import pathlib
import struct
import subprocess
import sys

repo = pathlib.Path(sys.argv[1]).resolve()
out = pathlib.Path(sys.argv[2]).resolve()
out.mkdir(parents=True, exist_ok=True)
commands = []

def run(label, args):
    p = subprocess.run(args, cwd=repo, text=True, capture_output=True)
    commands.append({'label': label, 'cwd': str(repo), 'command': args, 'returncode': p.returncode})
    (out / (label + '.stdout')).write_text(p.stdout)
    (out / (label + '.stderr')).write_text(p.stderr)
    if p.returncode:
        raise RuntimeError((label, p.returncode, p.stderr, p.stdout))
    return p.stdout

summary = []
for bits in (32, 64):
    wide = bits == 64
    ident = b'\x7fELF' + bytes((2 if wide else 1, 1, 1)) + bytes(9)
    names = b'\0.note.gnu.build-id\0.shstrtab\0'
    note = struct.pack('<III', 4, 20, 3) + b'GNU\0' + bytes(range(20))
    ehsize, shsize = (64, 64) if wide else (52, 40)
    noteoff = ehsize + 3 * shsize
    stroff = noteoff + len(note)
    if wide:
        header = struct.pack('<16sHHIQQQIHHHHHH', ident, 1, 62, 1, 0, 0, ehsize, 0, ehsize, 0, 0, shsize, 3, 2)
        def section(name, kind, off, size, align):
            return struct.pack('<IIQQQQIIQQ', name, kind, 0, 0, off, size, 0, 0, align, 0)
    else:
        header = struct.pack('<16sHHIIIIIHHHHHH', ident, 1, 3, 1, 0, 0, ehsize, 0, ehsize, 0, 0, shsize, 3, 2)
        def section(name, kind, off, size, align):
            return struct.pack('<IIIIIIIIII', name, kind, 0, 0, off, size, 0, 0, align, 0)
    plain = out / f'control{bits}.o'
    compressed = out / f'names-compressed{bits}.o'
    plain.write_bytes(header + bytes(shsize) + section(1, 7, noteoff, len(note), 4) + section(20, 3, stroff, len(names), 1) + note + names)
    run(f'compress{bits}', ['eu-elfcompress', '-f', '-t', 'zlib', '-n', '.shstrtab', '-o', str(compressed), str(plain)])
    for label, path in ((f'control{bits}', plain), (f'compressed{bits}', compressed)):
        before = hashlib.sha256(path.read_bytes()).hexdigest()
        lint = run(label + '-lint', ['eu-elflint', '--gnu-ld', str(path)])
        run(label + '-readelf', ['eu-readelf', '--section-headers', '--notes', str(path)])
        evidence = json.loads(run(label + '-cli', ['node', 'scripts/rea.mjs', 'inspect-elf-notes', str(path), '--json']))
        r = evidence['normalized_result']
        row = {'case': label, 'bytes': path.stat().st_size, 'sha256': before, 'unchanged': before == hashlib.sha256(path.read_bytes()).hexdigest(), 'lint': lint.strip(), 'tables': r['tables'], 'coverage': r['coverage'], 'build_id_hex': r['entries'][0]['gnu']['build_id_hex']}
        summary.append(row)
        print(json.dumps(row))
(out / 'commands.json').write_text(json.dumps(commands, indent=2))
(out / 'summary.json').write_text(json.dumps(summary, indent=2))
PYTHON
python3 "$work_dir/repro.py" "$PWD" "$work_dir/results"

Compare the printed tables fields for control32 versus compressed32, and control64 versus compressed64. The output directory retains every exact command, exit code, stdout, stderr, fixture, and summary. Both compressed outputs were produced by the normal elfutils compressor, without its permissive option.

Expected behavior

Retain .note.gnu.build-id as the note section's name when its valid section-name table is compressed, within admitted budgets. If compressed names are unsupported, preserve that uncertainty explicitly instead of presenting an empty name as an observed complete fact.

The ELF note contract includes section names when a string table is present. The ELF compression specification distinguishes compressed section bytes and their compression header from the uncompressed data; section-name offsets index the string table's logical contents. The paired elfutils results establish the expected names for these executed fixtures.

Impact and scope

A declared structural identity is silently replaced by an empty string in successful Evidence. This loses the name needed for name-based inventory comparison while claiming complete coverage. The two benign ELF classes reproduce the same omission; original files, note locations, owners, and build IDs remain intact. Inspection left all four files unchanged. Frequency in distributed binaries, other compression algorithms, big-endian files, and MCP execution were not measured. This report concerns the main-only standalone note inspector, not Flutter's separate build-id reader.

REA version

Built main revision 8c4f96ef4759fd78da66c4b9726825ca71d7ee0c; node scripts/rea.mjs --version prints 6.7.0.

Main-only finding: comparison with the latest published release tag rea-agents-6.7.0 (commit 27dc42d10249f0410904a75c0997842eb03b931c) found src/native/elf/ElfNoteParser.ts, src/application/binaryDiagnostics/ElfNotesService.ts, and docs/elf-notes.md absent from that release. This is not a claim that the published 6.7.0 package exposes the command or reproduces this defect.

Runtime and platform

Node.js v26.12.1; Python 3.14.8; elfutils 0.197; Linux 7.2.8-9-omarchy x86_64. No disassembler, live process, or native target execution was used.

Target and artifact context

ELF32 little-endian i386 and ELF64 little-endian x86-64 relocatable objects. Each has a null section, one 36-byte uncompressed SHT_NOTE section named .note.gnu.build-id, and one SHT_STRTAB section named .shstrtab. The compressed variants set SHF_COMPRESSED on the string table through eu-elfcompress; the note data remains uncompressed. The control/compressed sizes are 238/260 bytes for ELF32 and 322/360 bytes for ELF64.

Interface

CLI

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

All four CLI calls exit 0; all four elfutils validation calls print No errors. Reduced observations from the executed probe:

control32:    tables[0].name = ".note.gnu.build-id"
compressed32: tables[0].name = ""
control64:    tables[0].name = ".note.gnu.build-id"
compressed64: tables[0].name = ""
All four: coverage = {"status":"complete","examined_tables":1,"examined_entries":1}
All four: build_id_hex = "000102030405060708090a0b0c0d0e0f10111213"
All four: unchanged = true

For the compressed ELF64 object, eu-readelf --section-headers --notes reports:

[ 1] .note.gnu.build-id   NOTE         0000000000000000 00000040 00000024  0        0   0  4
[ 2] .shstrtab            STRTAB       0000000000000000 00000068 0000003e  0 C      0   0  8
Note section [ 1] '.note.gnu.build-id' of 36 bytes at offset 0x40:
  GNU                   20  GNU_BUILD_ID
    Build ID: 000102030405060708090a0b0c0d0e0f10111213

Compressed ELF64 SHA-256: 2f401f2efdaccc7d5f624554b05179238fd29f6061e05e003a714152528f919b. Compressed ELF32 SHA-256: ed332066973fdc9f101f193470eee943f2bb6410dbd99b49c178a2fa645266cf.

Failure provenance: observed through the built public CLI and independently compared with elfutils. Local verification: reproduced. Reproduction completeness: complete for the two synthetic classes; natural-producer prevalence remains unknown.

Typed outcome or recovery expected

Return the original section name, or an explicit unsupported/unknown name-decoding outcome and limitation. Retain the correctly observed note contents and stable file identity. Bounded decompression and any explicit unsupported outcome are proposed restoration choices, not tested fixes.

Analysis or suspected cause

Source-derived: ElfNoteParser.ts:325-334 checks that the name section is SHT_STRTAB, then reads its physical file bytes directly without examining SHF_COMPRESSED. ElfNoteParser.ts:373-381 applies the logical name offset to that physical buffer. Here, name offset 1 reaches a zero byte in the compression header, producing the observed empty string.

Additional context

No code changes or fix were made. No last-known-good version is established because the operation is absent from the latest release. Related #6563 concerns the separate Flutter reader's eight-byte note framing; this reproduction keeps note data uncompressed and four-byte-aligned.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: Executed CLI controls show successful Evidence silently losing an original note-section identity for two valid ELF classes while reporting complete coverage; the error is bounded metadata loss and does not prevent build-id decoding.
Workaround status: available
Workaround basis: eu-readelf --section-headers --notes recovered the correct names and build IDs directly from both unchanged compressed inputs in the executed probe.

Originally created by @coygeek on GitHub (Oct 23, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6692 ## Before submitting - [x] I searched the open and closed issues for a related report (`shstrtab`, `compressed section name`, and `inspect_elf_notes`). - [x] I removed credentials, tokens, private paths, and sensitive target data. ## Area Native analysis or decompilation ## Observed behavior The built public CLI exits 0 and returns `tables[0].name: ""` for valid ELF32 and ELF64 objects whose section-name string table is zlib-compressed. It reports complete coverage without a name-decoding limitation. The note section is actually named `.note.gnu.build-id`: `eu-readelf` resolves that name from the same files, and `eu-elflint --gnu-ld` reports `No errors`. Uncompressed controls return the correct name. GNU build-id decoding still succeeds in all four cases. ## Reproduction Run from the prepared source checkout at the revision below, with its locked dependencies and compiled runtime already available. Prerequisites: Node.js, Python 3, and elfutils (`eu-elfcompress`, `eu-readelf`, `eu-elflint`). The following is the focused probe actually executed: it builds inert minimal objects, uses elfutils to compress only `.shstrtab`, validates each file, and invokes the public CLI. It does not execute an object file. ```sh work_dir="$(mktemp -d)" cat > "$work_dir/repro.py" <<'PYTHON' import hashlib import json import pathlib import struct import subprocess import sys repo = pathlib.Path(sys.argv[1]).resolve() out = pathlib.Path(sys.argv[2]).resolve() out.mkdir(parents=True, exist_ok=True) commands = [] def run(label, args): p = subprocess.run(args, cwd=repo, text=True, capture_output=True) commands.append({'label': label, 'cwd': str(repo), 'command': args, 'returncode': p.returncode}) (out / (label + '.stdout')).write_text(p.stdout) (out / (label + '.stderr')).write_text(p.stderr) if p.returncode: raise RuntimeError((label, p.returncode, p.stderr, p.stdout)) return p.stdout summary = [] for bits in (32, 64): wide = bits == 64 ident = b'\x7fELF' + bytes((2 if wide else 1, 1, 1)) + bytes(9) names = b'\0.note.gnu.build-id\0.shstrtab\0' note = struct.pack('<III', 4, 20, 3) + b'GNU\0' + bytes(range(20)) ehsize, shsize = (64, 64) if wide else (52, 40) noteoff = ehsize + 3 * shsize stroff = noteoff + len(note) if wide: header = struct.pack('<16sHHIQQQIHHHHHH', ident, 1, 62, 1, 0, 0, ehsize, 0, ehsize, 0, 0, shsize, 3, 2) def section(name, kind, off, size, align): return struct.pack('<IIQQQQIIQQ', name, kind, 0, 0, off, size, 0, 0, align, 0) else: header = struct.pack('<16sHHIIIIIHHHHHH', ident, 1, 3, 1, 0, 0, ehsize, 0, ehsize, 0, 0, shsize, 3, 2) def section(name, kind, off, size, align): return struct.pack('<IIIIIIIIII', name, kind, 0, 0, off, size, 0, 0, align, 0) plain = out / f'control{bits}.o' compressed = out / f'names-compressed{bits}.o' plain.write_bytes(header + bytes(shsize) + section(1, 7, noteoff, len(note), 4) + section(20, 3, stroff, len(names), 1) + note + names) run(f'compress{bits}', ['eu-elfcompress', '-f', '-t', 'zlib', '-n', '.shstrtab', '-o', str(compressed), str(plain)]) for label, path in ((f'control{bits}', plain), (f'compressed{bits}', compressed)): before = hashlib.sha256(path.read_bytes()).hexdigest() lint = run(label + '-lint', ['eu-elflint', '--gnu-ld', str(path)]) run(label + '-readelf', ['eu-readelf', '--section-headers', '--notes', str(path)]) evidence = json.loads(run(label + '-cli', ['node', 'scripts/rea.mjs', 'inspect-elf-notes', str(path), '--json'])) r = evidence['normalized_result'] row = {'case': label, 'bytes': path.stat().st_size, 'sha256': before, 'unchanged': before == hashlib.sha256(path.read_bytes()).hexdigest(), 'lint': lint.strip(), 'tables': r['tables'], 'coverage': r['coverage'], 'build_id_hex': r['entries'][0]['gnu']['build_id_hex']} summary.append(row) print(json.dumps(row)) (out / 'commands.json').write_text(json.dumps(commands, indent=2)) (out / 'summary.json').write_text(json.dumps(summary, indent=2)) PYTHON python3 "$work_dir/repro.py" "$PWD" "$work_dir/results" ``` Compare the printed `tables` fields for `control32` versus `compressed32`, and `control64` versus `compressed64`. The output directory retains every exact command, exit code, stdout, stderr, fixture, and summary. Both compressed outputs were produced by the normal elfutils compressor, without its permissive option. ## Expected behavior Retain `.note.gnu.build-id` as the note section's name when its valid section-name table is compressed, within admitted budgets. If compressed names are unsupported, preserve that uncertainty explicitly instead of presenting an empty name as an observed complete fact. The [ELF note contract](https://github.com/morluto/rea/blob/8c4f96ef4759fd78da66c4b9726825ca71d7ee0c/docs/elf-notes.md#L25-L28) includes section names when a string table is present. The [ELF compression specification](https://gabi.xinuos.com/elf/03-sheader.html#compressed-sections) distinguishes compressed section bytes and their compression header from the uncompressed data; section-name offsets index the string table's logical contents. The paired elfutils results establish the expected names for these executed fixtures. ## Impact and scope A declared structural identity is silently replaced by an empty string in successful Evidence. This loses the name needed for name-based inventory comparison while claiming complete coverage. The two benign ELF classes reproduce the same omission; original files, note locations, owners, and build IDs remain intact. Inspection left all four files unchanged. Frequency in distributed binaries, other compression algorithms, big-endian files, and MCP execution were not measured. This report concerns the main-only standalone note inspector, not Flutter's separate build-id reader. ## REA version Built main revision `8c4f96ef4759fd78da66c4b9726825ca71d7ee0c`; `node scripts/rea.mjs --version` prints `6.7.0`. Main-only finding: comparison with the latest published release tag `rea-agents-6.7.0` (commit `27dc42d10249f0410904a75c0997842eb03b931c`) found `src/native/elf/ElfNoteParser.ts`, `src/application/binaryDiagnostics/ElfNotesService.ts`, and `docs/elf-notes.md` absent from that release. This is not a claim that the published 6.7.0 package exposes the command or reproduces this defect. ## Runtime and platform Node.js v26.12.1; Python 3.14.8; elfutils 0.197; Linux 7.2.8-9-omarchy x86_64. No disassembler, live process, or native target execution was used. ## Target and artifact context ELF32 little-endian i386 and ELF64 little-endian x86-64 relocatable objects. Each has a null section, one 36-byte uncompressed SHT_NOTE section named `.note.gnu.build-id`, and one SHT_STRTAB section named `.shstrtab`. The compressed variants set SHF_COMPRESSED on the string table through `eu-elfcompress`; the note data remains uncompressed. The control/compressed sizes are 238/260 bytes for ELF32 and 322/360 bytes for ELF64. ## Interface CLI ## Earliest failure stage Native analysis or reconstruction ## Relevant output or evidence All four CLI calls exit 0; all four elfutils validation calls print `No errors`. Reduced observations from the executed probe: ```text control32: tables[0].name = ".note.gnu.build-id" compressed32: tables[0].name = "" control64: tables[0].name = ".note.gnu.build-id" compressed64: tables[0].name = "" All four: coverage = {"status":"complete","examined_tables":1,"examined_entries":1} All four: build_id_hex = "000102030405060708090a0b0c0d0e0f10111213" All four: unchanged = true ``` For the compressed ELF64 object, `eu-readelf --section-headers --notes` reports: ```text [ 1] .note.gnu.build-id NOTE 0000000000000000 00000040 00000024 0 0 0 4 [ 2] .shstrtab STRTAB 0000000000000000 00000068 0000003e 0 C 0 0 8 Note section [ 1] '.note.gnu.build-id' of 36 bytes at offset 0x40: GNU 20 GNU_BUILD_ID Build ID: 000102030405060708090a0b0c0d0e0f10111213 ``` Compressed ELF64 SHA-256: `2f401f2efdaccc7d5f624554b05179238fd29f6061e05e003a714152528f919b`. Compressed ELF32 SHA-256: `ed332066973fdc9f101f193470eee943f2bb6410dbd99b49c178a2fa645266cf`. Failure provenance: observed through the built public CLI and independently compared with elfutils. Local verification: reproduced. Reproduction completeness: complete for the two synthetic classes; natural-producer prevalence remains unknown. ## Typed outcome or recovery expected Return the original section name, or an explicit unsupported/unknown name-decoding outcome and limitation. Retain the correctly observed note contents and stable file identity. Bounded decompression and any explicit unsupported outcome are proposed restoration choices, not tested fixes. ## Analysis or suspected cause Source-derived: [`ElfNoteParser.ts:325-334`](https://github.com/morluto/rea/blob/8c4f96ef4759fd78da66c4b9726825ca71d7ee0c/src/native/elf/ElfNoteParser.ts#L325-L334) checks that the name section is SHT_STRTAB, then reads its physical file bytes directly without examining SHF_COMPRESSED. [`ElfNoteParser.ts:373-381`](https://github.com/morluto/rea/blob/8c4f96ef4759fd78da66c4b9726825ca71d7ee0c/src/native/elf/ElfNoteParser.ts#L373-L381) applies the logical name offset to that physical buffer. Here, name offset 1 reaches a zero byte in the compression header, producing the observed empty string. ## Additional context No code changes or fix were made. No last-known-good version is established because the operation is absent from the latest release. Related #6563 concerns the separate Flutter reader's eight-byte note framing; this reproduction keeps note data uncompressed and four-byte-aligned. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: Executed CLI controls show successful Evidence silently losing an original note-section identity for two valid ELF classes while reporting complete coverage; the error is bounded metadata loss and does not prevent build-id decoding. - Workaround status: available - Workaround basis: `eu-readelf --section-headers --notes` recovered the correct names and build IDs directly from both unchanged compressed inputs in the executed probe.

bsakaguchi closed this issue

2026-10-31 00:46:32 +00:00

bsakaguchi referenced this issue

2026-10-31 00:50:17 +00:00

[GH-ISSUE #7556] [Bug] inspect-macho fails when the executable filename ends in parentheses #4495

bsakaguchi referenced this issue

2026-10-31 01:09:08 +00:00

[PR #3455] [CLOSED] fix(wasm): preserve local matches for dotted selected paths #5746

bsakaguchi referenced this issue

2026-10-31 01:09:42 +00:00

[PR #3561] [CLOSED] fix(evidence): preserve filesystem-selected JSON paths #5825

bsakaguchi referenced this issue