morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #6538] [Bug] Recorded-core validation rejects unchanged AArch64 program-header enum values #3930

New issue

Closed

opened 2026-10-31 00:45:19 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:45:19 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 23, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6538

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Native analysis or decompilation

Observed behavior

The recorded-core structure validator rejects the exact PT_AARCH64_UNWIND and PT_AARCH64_ARCHEXT names produced by the pinned pyelftools 0.33 parser for their original AArch64 program-header bytes. Both observations pass recordedCrashSchema first. Validation then says the reported structure differs from the original headers, although the type and all other reported header fields came from those bytes.

This was observed through real pyelftools parsing followed by production TypeScript schema and source validation. The report projection mirrors the bridge's program-header projection. The complete REA Python decoder, public CLI/MCP, pwntools, Unicorn, and debugger were not executed. The fixtures are synthetic metadata containers, not captured native crashes; no claim is made that a Linux core dumper naturally emits these auxiliary headers.

Reproduction

The initial probes ran once, before drafting. Install the repository's locked dependencies, use Node 24.18.0, and install pyelftools==0.33 into a disposable Python environment. Save the two scripts below together as segment-producer.py and segment-validator.mts in a disposable directory.

The producer makes five ELF64 little-endian ET_CORE fixtures. Each has one 56-byte program header at offset 64 and four file-backed bytes at offset 120; the architecture and numeric type are varied. It parses their actual bytes with pyelftools, constructs the same program-header fields as the REA bridge, and writes the observations:

import hashlib, io, json, struct
from pathlib import Path
from importlib.metadata import version
from elftools.elf.elffile import ELFFile

root=Path(__file__).parent
rows=[]
for name,machine,kind in [('arm-unwind',183,0x70000001),('arm-archext',183,0x70000000),('arm-opaque',183,0x70000003),('x64-same-number',62,0x70000001),('arm-gnu-stack',183,0x6474e551)]:
    ident=b'\x7fELF'+bytes([2,1,1,0,0])+bytes(7)
    header=struct.pack('<16sHHIQQQIHHHHHH',ident,4,machine,1,0,64,0,0,64,56,1,64,0,0)
    raw=header+struct.pack('<IIQQQQQQ',kind,4,120,0,0,4,4,4)+bytes(4)
    path=root/(name+'.core');path.write_bytes(raw)
    image=ELFFile(io.BytesIO(raw));segments=[]
    for index,segment in enumerate(image.iter_segments()):
        h=segment.header
        segments.append({'index':index,'type':h.p_type,'header_location':{'offset':hex(image.header.e_phoff+index*image.header.e_phentsize),'bytes':hex(image.header.e_phentsize)},'offset':hex(h.p_offset),'file_size':hex(h.p_filesz),'memory_size':hex(h.p_memsz),'virtual_address':hex(h.p_vaddr),'physical_address':hex(h.p_paddr),'alignment':hex(h.p_align),'flags':hex(h.p_flags),'file_backing':'file' if h.p_type!='PT_NULL' and h.p_filesz>0 else 'none'})
    report={'artifact':{'path':str(path),'sha256':hashlib.sha256(raw).hexdigest(),'bytes':len(raw)},'format':'elf-core','architecture':'aarch64-little-endian' if machine==183 else 'x86_64-little-endian','target_execution':'not-performed','live_process_identity':'unknown','segments':segments,'notes':[],'note_padding':[],'threads':[],'signals':[],'debugger':{'status':'not-requested'},'note_interpretation_completeness':'unknown','decoder_diagnostics':{'stdout':'','stderr':'','truncated':False},'diagnostics':{'stdout':'','stderr':'','truncated':False},'limitations':[]}
    (root/(name+'.json')).write_text(json.dumps(report,indent=2)+'\n')
    rows.append({'name':name,'machine':machine,'raw_type':hex(kind),'parser_type':segments[0]['type'],'sha256':report['artifact']['sha256']})
print(json.dumps({'pyelftools':version('pyelftools'),'python':__import__('sys').version.split()[0],'cases':rows},indent=2))

The TypeScript probe validates each complete observation, then executes REA's production source-binding function against the original bytes:

import { readFile } from 'node:fs/promises';
import { fileURLToPath } from 'node:url';
import { resolve,dirname } from 'node:path';
import { pathToFileURL } from 'node:url';
const repo=process.argv[2];
const {recordedCrashSchema}=await import(pathToFileURL(resolve(repo,'src/domain/native/recordedCrash.ts')).href);
const {validateRecordedCrashStructure}=await import(pathToFileURL(resolve(repo,'src/native/pwntools/RecordedCrashStructureBindings.ts')).href);
const root=dirname(fileURLToPath(import.meta.url));
for(const name of ['arm-unwind','arm-archext','arm-opaque','x64-same-number','arm-gnu-stack']){
 const report=recordedCrashSchema.parse(JSON.parse(await readFile(resolve(root,name+'.json'),'utf8')));
 const bytes=await readFile(resolve(root,name+'.core'));
 let accepted=true,error=null;
 try{validateRecordedCrashStructure(report,bytes,report.decoder_diagnostics)}catch(e){accepted=false;error=e.message}
 console.log(JSON.stringify({name,parserType:report.segments[0].type,schemaAccepted:true,structureAccepted:accepted,error}));
}

Run the producer with the disposable environment's Python. From the repository root, run:

node --import "$PWD/node_modules/tsx/dist/loader.mjs" /tmp/rea-crash-probe/segment-validator.mts "$PWD"

Replace the neutral probe directory with the directory containing the scripts. These are the operations executed locally, with only the temporary path changed for this report. Both AArch64 enum cases failed; the opaque AArch64 type, the same numeric value under x86-64, and the known GNU type passed.

Expected behavior

A source-consistency check should accept the parser's architecture-specific name when it denotes the exact original numeric p_type. It should preserve that header observation without pretending to interpret its payload. Unsupported or unknown auxiliary semantics can remain unknown; they do not establish a disagreement with the recorded bytes.

REA's recorded-crash contract includes AArch64, uses unchanged pyelftools structures, and retains original program headers. The pinned parser's enum table adds these two names for AArch64.

Impact and scope

The directly observed consequence is a false source-consistency refusal for these accepted observations. Source-derived consequence: the production provider invokes this validator after decoding, so a reply containing either name cannot complete its ordinary projection. This is a bounded metadata compatibility defect, not a claim about native crash prevalence or unwind decoding.

NOT_ENOUGH_INFO: no real captured crash with either header, full provider workflow, or input-preserving public workaround was tested. The three controls establish that the refusal depends on the architecture-specific symbolic spelling, rather than a general prohibition on opaque program headers.

REA version

Source revision e073f64be0a5a30c32d8b3cec23f9bd411080903, package metadata 6.7.0. The published npm runtime was not executed.

Runtime and platform

Node.js v24.18.0; Python 3.14.8; pyelftools 0.33; Linux x86_64, kernel 7.2.8-9-omarchy; repository-locked TypeScript dependencies.

Target and artifact context

Five self-authored 124-byte ELF64 little-endian ET_CORE metadata fixtures, with AArch64 or x86-64 machine identifiers. No target code, executable discovery, process attachment, native core capture, or debugger context.

Interface

Other

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

Initial output:

arm-unwind: parserType=PT_AARCH64_UNWIND, schemaAccepted=true, structureAccepted=false
arm-archext: parserType=PT_AARCH64_ARCHEXT, schemaAccepted=true, structureAccepted=false
arm-opaque: parserType=1879048195, schemaAccepted=true, structureAccepted=true
x64-same-number: parserType=1879048193, schemaAccepted=true, structureAccepted=true
arm-gnu-stack: parserType=PT_GNU_STACK, schemaAccepted=true, structureAccepted=true

Both failures report Invalid analysis output for inspect_recorded_crash: Reported core structure differs from its original ELF/note headers.

The arm-unwind fixture SHA-256 is 4fc100c2a3f9a51b0de7dac2e248906f095cca9f063c6e5faf8709a635fa951e. All source bytes and parser observations were retained by the initial probe. Evidence provenance: observed at the parser/schema/source-validation boundaries; full provider reachability is source-derived. Reproduction completeness is incomplete for the full configured REA decoder.

Typed outcome or recovery expected

Accept and retain the original header identity, or preserve an explicit semantic limitation separately. Do not classify an architecture-specific enum spelling as disagreement with the original bytes. Keep byte-disagreement rejection intact.

Analysis or suspected cause

Source-derived: the bridge retains h.p_type unchanged. The validator duplicates only the base/amd64 enum table. Its string branch requires a matching entry in that table; both AArch64 additions are absent. The same raw value remains numeric in the x86-64 control and is accepted by the numeric branch. The provider calls this check before completing projection.

Additional context

Open and closed searches for repo:morluto/rea is:issue "PT_AARCH64" returned no results. Searches also covered "recorded" "program headers", "core" "enum", "core" "architecture-specific" "segment", and RecordedCrashStructureBindings.

Closest report #5700 concerns owner-qualified note names and incorrect GNU note interpretation. This report has no notes and concerns omitted architecture-specific program-header enum names at a different source-binding boundary. The observed restoration predicate is accepting unchanged AArch64 header identities while preserving byte-consistency checks.

AI-assisted research. No reproduction or independent review pass was performed after this draft was written.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: Production source validation rejects two parser-generated AArch64 header identities despite matching original bytes and successful schema validation. Controls isolate the representation mismatch; real captured-crash occurrence is not established.
Workaround status: unverified
Workaround basis: No public option or workflow preserving the original artifact and completing this source-validation path was exercised.

Originally created by @coygeek on GitHub (Oct 23, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6538 ## Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ## Area Native analysis or decompilation ## Observed behavior The recorded-core structure validator rejects the exact `PT_AARCH64_UNWIND` and `PT_AARCH64_ARCHEXT` names produced by the pinned pyelftools 0.33 parser for their original AArch64 program-header bytes. Both observations pass `recordedCrashSchema` first. Validation then says the reported structure differs from the original headers, although the type and all other reported header fields came from those bytes. This was observed through real pyelftools parsing followed by production TypeScript schema and source validation. The report projection mirrors the bridge's program-header projection. The complete REA Python decoder, public CLI/MCP, pwntools, Unicorn, and debugger were not executed. The fixtures are synthetic metadata containers, not captured native crashes; no claim is made that a Linux core dumper naturally emits these auxiliary headers. ## Reproduction The initial probes ran once, before drafting. Install the repository's locked dependencies, use Node 24.18.0, and install `pyelftools==0.33` into a disposable Python environment. Save the two scripts below together as `segment-producer.py` and `segment-validator.mts` in a disposable directory. The producer makes five ELF64 little-endian ET_CORE fixtures. Each has one 56-byte program header at offset 64 and four file-backed bytes at offset 120; the architecture and numeric type are varied. It parses their actual bytes with pyelftools, constructs the same program-header fields as the REA bridge, and writes the observations: ```python import hashlib, io, json, struct from pathlib import Path from importlib.metadata import version from elftools.elf.elffile import ELFFile root=Path(__file__).parent rows=[] for name,machine,kind in [('arm-unwind',183,0x70000001),('arm-archext',183,0x70000000),('arm-opaque',183,0x70000003),('x64-same-number',62,0x70000001),('arm-gnu-stack',183,0x6474e551)]: ident=b'\x7fELF'+bytes([2,1,1,0,0])+bytes(7) header=struct.pack('<16sHHIQQQIHHHHHH',ident,4,machine,1,0,64,0,0,64,56,1,64,0,0) raw=header+struct.pack('<IIQQQQQQ',kind,4,120,0,0,4,4,4)+bytes(4) path=root/(name+'.core');path.write_bytes(raw) image=ELFFile(io.BytesIO(raw));segments=[] for index,segment in enumerate(image.iter_segments()): h=segment.header segments.append({'index':index,'type':h.p_type,'header_location':{'offset':hex(image.header.e_phoff+index*image.header.e_phentsize),'bytes':hex(image.header.e_phentsize)},'offset':hex(h.p_offset),'file_size':hex(h.p_filesz),'memory_size':hex(h.p_memsz),'virtual_address':hex(h.p_vaddr),'physical_address':hex(h.p_paddr),'alignment':hex(h.p_align),'flags':hex(h.p_flags),'file_backing':'file' if h.p_type!='PT_NULL' and h.p_filesz>0 else 'none'}) report={'artifact':{'path':str(path),'sha256':hashlib.sha256(raw).hexdigest(),'bytes':len(raw)},'format':'elf-core','architecture':'aarch64-little-endian' if machine==183 else 'x86_64-little-endian','target_execution':'not-performed','live_process_identity':'unknown','segments':segments,'notes':[],'note_padding':[],'threads':[],'signals':[],'debugger':{'status':'not-requested'},'note_interpretation_completeness':'unknown','decoder_diagnostics':{'stdout':'','stderr':'','truncated':False},'diagnostics':{'stdout':'','stderr':'','truncated':False},'limitations':[]} (root/(name+'.json')).write_text(json.dumps(report,indent=2)+'\n') rows.append({'name':name,'machine':machine,'raw_type':hex(kind),'parser_type':segments[0]['type'],'sha256':report['artifact']['sha256']}) print(json.dumps({'pyelftools':version('pyelftools'),'python':__import__('sys').version.split()[0],'cases':rows},indent=2)) ``` The TypeScript probe validates each complete observation, then executes REA's production source-binding function against the original bytes: ```ts import { readFile } from 'node:fs/promises'; import { fileURLToPath } from 'node:url'; import { resolve,dirname } from 'node:path'; import { pathToFileURL } from 'node:url'; const repo=process.argv[2]; const {recordedCrashSchema}=await import(pathToFileURL(resolve(repo,'src/domain/native/recordedCrash.ts')).href); const {validateRecordedCrashStructure}=await import(pathToFileURL(resolve(repo,'src/native/pwntools/RecordedCrashStructureBindings.ts')).href); const root=dirname(fileURLToPath(import.meta.url)); for(const name of ['arm-unwind','arm-archext','arm-opaque','x64-same-number','arm-gnu-stack']){ const report=recordedCrashSchema.parse(JSON.parse(await readFile(resolve(root,name+'.json'),'utf8'))); const bytes=await readFile(resolve(root,name+'.core')); let accepted=true,error=null; try{validateRecordedCrashStructure(report,bytes,report.decoder_diagnostics)}catch(e){accepted=false;error=e.message} console.log(JSON.stringify({name,parserType:report.segments[0].type,schemaAccepted:true,structureAccepted:accepted,error})); } ``` Run the producer with the disposable environment's Python. From the repository root, run: ```sh node --import "$PWD/node_modules/tsx/dist/loader.mjs" /tmp/rea-crash-probe/segment-validator.mts "$PWD" ``` Replace the neutral probe directory with the directory containing the scripts. These are the operations executed locally, with only the temporary path changed for this report. Both AArch64 enum cases failed; the opaque AArch64 type, the same numeric value under x86-64, and the known GNU type passed. ## Expected behavior A source-consistency check should accept the parser's architecture-specific name when it denotes the exact original numeric `p_type`. It should preserve that header observation without pretending to interpret its payload. Unsupported or unknown auxiliary semantics can remain unknown; they do not establish a disagreement with the recorded bytes. REA's [recorded-crash contract](https://github.com/morluto/rea/blob/e073f64be0a5a30c32d8b3cec23f9bd411080903/docs/recorded-crashes.md#L3-L27) includes AArch64, uses unchanged pyelftools structures, and retains original program headers. The [pinned parser's enum table](https://github.com/eliben/pyelftools/blob/v0.33/elftools/elf/enums.py#L431-L435) adds these two names for AArch64. ## Impact and scope The directly observed consequence is a false source-consistency refusal for these accepted observations. Source-derived consequence: the production provider invokes this validator after decoding, so a reply containing either name cannot complete its ordinary projection. This is a bounded metadata compatibility defect, not a claim about native crash prevalence or unwind decoding. NOT_ENOUGH_INFO: no real captured crash with either header, full provider workflow, or input-preserving public workaround was tested. The three controls establish that the refusal depends on the architecture-specific symbolic spelling, rather than a general prohibition on opaque program headers. ## REA version Source revision `e073f64be0a5a30c32d8b3cec23f9bd411080903`, package metadata `6.7.0`. The published npm runtime was not executed. ## Runtime and platform Node.js v24.18.0; Python 3.14.8; pyelftools 0.33; Linux x86_64, kernel 7.2.8-9-omarchy; repository-locked TypeScript dependencies. ## Target and artifact context Five self-authored 124-byte ELF64 little-endian ET_CORE metadata fixtures, with AArch64 or x86-64 machine identifiers. No target code, executable discovery, process attachment, native core capture, or debugger context. ## Interface Other ## Earliest failure stage Native analysis or reconstruction ## Relevant output or evidence Initial output: ```text arm-unwind: parserType=PT_AARCH64_UNWIND, schemaAccepted=true, structureAccepted=false arm-archext: parserType=PT_AARCH64_ARCHEXT, schemaAccepted=true, structureAccepted=false arm-opaque: parserType=1879048195, schemaAccepted=true, structureAccepted=true x64-same-number: parserType=1879048193, schemaAccepted=true, structureAccepted=true arm-gnu-stack: parserType=PT_GNU_STACK, schemaAccepted=true, structureAccepted=true ``` Both failures report `Invalid analysis output for inspect_recorded_crash: Reported core structure differs from its original ELF/note headers.` The `arm-unwind` fixture SHA-256 is `4fc100c2a3f9a51b0de7dac2e248906f095cca9f063c6e5faf8709a635fa951e`. All source bytes and parser observations were retained by the initial probe. Evidence provenance: observed at the parser/schema/source-validation boundaries; full provider reachability is source-derived. Reproduction completeness is incomplete for the full configured REA decoder. ## Typed outcome or recovery expected Accept and retain the original header identity, or preserve an explicit semantic limitation separately. Do not classify an architecture-specific enum spelling as disagreement with the original bytes. Keep byte-disagreement rejection intact. ## Analysis or suspected cause Source-derived: [the bridge retains `h.p_type` unchanged](https://github.com/morluto/rea/blob/e073f64be0a5a30c32d8b3cec23f9bd411080903/bridge/pwntools/recorded_crash_impl.py#L89-L101). [The validator](https://github.com/morluto/rea/blob/e073f64be0a5a30c32d8b3cec23f9bd411080903/src/native/pwntools/RecordedCrashStructureBindings.ts#L7-L50) duplicates only the base/amd64 enum table. Its string branch requires a matching entry in that table; both AArch64 additions are absent. The same raw value remains numeric in the x86-64 control and is accepted by the numeric branch. [The provider calls this check before completing projection](https://github.com/morluto/rea/blob/e073f64be0a5a30c32d8b3cec23f9bd411080903/src/native/pwntools/PwntoolsRecordedCrashProvider.ts#L86-L91). ## Additional context Open and closed searches for `repo:morluto/rea is:issue "PT_AARCH64"` returned no results. Searches also covered `"recorded" "program headers"`, `"core" "enum"`, `"core" "architecture-specific" "segment"`, and `RecordedCrashStructureBindings`. Closest report [#5700](https://github.com/morluto/rea/issues/5700) concerns owner-qualified note names and incorrect GNU note interpretation. This report has no notes and concerns omitted architecture-specific program-header enum names at a different source-binding boundary. The observed restoration predicate is accepting unchanged AArch64 header identities while preserving byte-consistency checks. AI-assisted research. No reproduction or independent review pass was performed after this draft was written. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: Production source validation rejects two parser-generated AArch64 header identities despite matching original bytes and successful schema validation. Controls isolate the representation mismatch; real captured-crash occurrence is not established. - Workaround status: unverified - Workaround basis: No public option or workflow preserving the original artifact and completing this source-validation path was exercised.

bsakaguchi closed this issue

2026-10-31 00:45:19 +00:00

bsakaguchi referenced this issue

2026-10-31 01:11:32 +00:00

[PR #4103] [MERGED] fix(providers): report provider-selection diagnostics when the selected provider refuses its profile #6059