[GH-ISSUE #7351] [Bug] Resuming an unreadable or conflicting export drops its retained function limitations from the index #4369

New issue

Closed

opened 2026-10-31 00:49:17 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:49:17 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 27, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/7351

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Native analysis or decompilation

Observed behavior

Through the registered MCP decompile_all tool with a fixed synthetic provider, an initial export stored 40 complete limitations in its function index record. Making that function file unreadable and resuming changed the record to failed / output_io, retained the earlier body and relative path, and removed limitations. A byte conflict produced the same omission with output_exists, including on a repeated resume. Reporting these file failures and preserving the caller's bytes are correct; discarding the retained observation's caveats violates the index contract. The original file header contains only a capped subset of these caveats.

Reproduction

Executed once through the production MCP SDK client, in-memory transport, registered handler and shared export workflow. Run the following from the baseline checkout after its locked dependencies and compiled runtime have been prepared, with pinned Node.js 24 on PATH. A non-root POSIX account that enforces mode-000 read denial is required. All fixture files, HOME and temporary output are isolated; no native engine or target program is opened or executed.

set -eu
REA_ROOT="$PWD"
WORK="$(mktemp -d)"
trap 'rm -rf "$WORK"' EXIT
mkdir -p "$WORK/home" "$WORK/tmp"
env -i PATH="$PATH" HOME="$WORK/home" TMPDIR="$WORK/tmp" REA_ROOT="$REA_ROOT" WORK="$WORK" node --input-type=module <<'JS'
import assert from 'node:assert/strict';
import {createRequire} from 'node:module';
import {pathToFileURL} from 'node:url';
import {join} from 'node:path';
import {chmod, readFile, writeFile, copyFile} from 'node:fs/promises';
import {release} from 'node:os';
const root=process.env.REA_ROOT, work=process.env.WORK;
const require=createRequire(join(root,'package.json'));
const {Client,InMemoryTransport}=require('@modelcontextprotocol/client');
const pino=require('pino');
const load=p=>import(pathToFileURL(join(root,'dist',p)).href);
const {EvidenceMcpServer}=await load('server/EvidenceMcpServer.js');
const {registerEnhancedTools}=await load('server/registerEnhancedTools.js');
const {createToolResultDelivery}=await load('server/toolResult.js');
const {withAdmittedAnalysis}=await load('server/analysisAdmission.js');
const {createAnalysisExecution}=await load('application/AnalysisProvider.js');
const {createAnalysisProfile}=await load('domain/analysisProfile.js');
const {ok}=await load('domain/result.js');
const provider={id:'fixture',name:'Fixture',version:'1'};
const profile=createAnalysisProfile(provider,{decompiler:'fixture'});
const subject={path:'fixture',sha256:'ab'.repeat(32),format:'elf',architecture:'x86_64'};
const caveats=Array.from({length:40},(_,i)=>`fixture caveat ${i}: ${'x'.repeat(150)}`);
const body={available:true,provenance:'fixture',ranges:[{start:'0x1000',end:'0x1000'}],total_bytes:1,span_bytes:1,non_contiguous:false,contains_entry:true};
const calls=[];
const analysis={execute:async op=>{
  calls.push(op);
  const value=op==='list_procedures'?[{address:'0x1000',value:'main'}]:op==='procedure_info'?{name:'main',entrypoint:'0x1000',basicblock_count:1,length:1,signature:null,locals:[],classification:null,body}:op==='procedure_pseudo_code'?'int main(void) { return 0; }':assert.fail(`unexpected ${op}`);
  return ok(createAnalysisExecution(value,provider,{subject,analysisProfile:profile,limitations:op==='list_procedures'?[]:caveats}));
}};
const server=new EvidenceMcpServer({name:'fixture-server',version:'1'},{},undefined,createToolResultDelivery(4*1024*1024));
registerEnhancedTools(server,{withAdmittedAnalysis:withAdmittedAnalysis({kind:'fixed',analysis}),logger:pino({enabled:false}),activeTarget:()=>undefined,analysisProfile:()=>profile,recordEvidence:undefined,recordUnknown:undefined,allowsSnapshotReplay:undefined,recordWorkflowSnapshot:undefined});
const client=new Client({name:'fixture-client',version:'1'});
const [ct,st]=InMemoryTransport.createLinkedPair();
const out=join(work,'out'), file=join(out,'functions','0x1000.c');
const record=async()=>JSON.parse(await readFile(join(out,'decompile-index.json'),'utf8')).functions[0];
const run=async phase=>{
  const start=calls.length;
  const reply=await client.callTool({name:'decompile_all',arguments:{output_directory:out,...(phase==='first'?{}:{resume:true})}});
  await writeFile(join(work,`${phase}-reply.json`),JSON.stringify(reply,null,2));
  assert.notEqual(reply.isError,true,JSON.stringify(reply));
  const r=await record();
  await copyFile(join(out,'decompile-index.json'),join(work,`${phase}-index.json`));
  console.log(JSON.stringify({phase,status:r.status,failure:r.failure?.code??null,limitations:r.limitations?.length??null,body:r.body,relative_path:r.relative_path,calls:calls.slice(start)}));
  return r;
};
try {
  await server.connect(st);await client.connect(ct);
  console.log(JSON.stringify({node:process.version,platform:process.platform,release:release(),architecture:process.arch,uid:process.getuid?.()}));
  const first=await run('first');assert.deepEqual(first.limitations,caveats);
  const original=await readFile(file);await writeFile(join(work,'original.c'),original);
  assert.ok(original.toString().includes('limitations_truncated: true'));
  assert.ok(!original.toString().includes(caveats[39]));
  console.log(JSON.stringify({header_truncated:true,last_caveat_absent:true}));
  await chmod(file,0);
  let denied=false;try{await readFile(file);}catch(e){assert.equal(e.code,'EACCES');denied=true;}assert.ok(denied);
  const io=await run('unreadable');assert.equal(io.status,'failed');assert.equal(io.failure.code,'output_io');assert.equal(io.limitations,undefined);assert.deepEqual(io.body,first.body);assert.equal(io.relative_path,first.relative_path);
  await chmod(file,0o600);assert.deepEqual(await readFile(file),original);
  const repaired=await run('readable');assert.equal(repaired.status,'written');assert.deepEqual(repaired.limitations,caveats);
  const conflict=Buffer.from('caller replacement\n');await writeFile(file,conflict);
  const bad=await run('conflict');assert.equal(bad.failure.code,'output_exists');assert.equal(bad.limitations,undefined);assert.deepEqual(bad.body,first.body);assert.deepEqual(await readFile(file),conflict);
  // Demonstrate repeat-failure persistence without asking provider to regenerate it.
  const repeated=await run('conflict-again');assert.equal(repeated.limitations,undefined);assert.deepEqual(repeated.body,first.body);
  await writeFile(file,original);
  const restored=await run('restored');assert.deepEqual(restored.limitations,caveats);assert.equal(restored.status,'written');
  const journal=(await readFile(join(out,'decompile-journal.jsonl'),'utf8')).trim().split('\n').map(JSON.parse);
  assert.equal(journal.filter(x=>x.kind==='function'&&x.status==='written'&&x.limitations?.length===40).length,3);
  console.log(JSON.stringify({journal_prior_complete_records_preserved:true,restoration_skipped_without_provider_reads:true,assertions:'passed'}));
}finally{await chmod(file,0o600).catch(()=>{});await Promise.allSettled([client.close(),server.close()]);}
JS

The assertions passed. The six calls returned successful MCP replies containing manifests; the file-error manifests correctly counted one failed function. Only the initial call read procedure_info and procedure_pseudo_code; each later call read only list_procedures.

Expected behavior

Every function's current index record must retain the complete limitations of the observations it uses, including when a resume reports a file error and retains an earlier body fact. Keep the intentional failed status and leave unreadable or conflicting function files unchanged. A capped header does not replace the complete index record.

Impact and scope

P2: consumers of the documented complete index lose the caveats associated with a retained function observation while its file remains unreadable or conflicting. One six-call synthetic-provider sequence reproduced the omission in the unreadable phase and both conflict phases. This is bounded, recoverable index loss: earlier complete records remain in the append-only journal. Restoring read permission and resuming restored all 40 limitations; restoring the original bytes after a conflict did the same without per-function provider reads. The test preserved the caller's conflicting bytes until its explicit restoration step. NOT_ENOUGH_INFO on real-provider caveat lengths, occurrence frequency in real exports, other operating systems or real native-engine behavior; none was exercised.

REA version

Source revision ace4dbf36e8c8f36a0e192aff9a11484ed428358, using the prepared compiled runtime and locked dependencies.

Runtime and platform

Node.js v24.22.1; macOS arm64, Darwin 27.0.0; non-root account with observed EACCES on the fixture's mode-000 file. MCP uses the locked SDK's in-memory transport.

Target and artifact context

One synthetic function at 0x1000, a valid matching provider/profile/target digest and 40 distinct fixture caveats accepted by the existing observation and journal schemas. The output is a disposable directory export. The fixed port does not select or launch a real provider and establishes no real-provider behavior.

Interface

MCP

Earliest failure stage

Artifact or evidence registration

Relevant output or evidence

Expected source: docs/native-investigation.md:210-218 promises complete limitations for every function record regardless of status and describes resume verification; src/contracts/enhancedToolContracts.ts:55 advertises complete per-function index limitations. src/application/decompileDirectory/functionFile.ts:67-77 explicitly directs readers of a truncated header to the complete index.
Failure source: The executed fixture above produced written/40 → failed/output_io/absent → written/40 → failed/output_exists/absent → failed/output_exists/absent → written/40. Each failed record retained the original body and relative path. Source corroboration: src/application/decompileDirectory/decompileAll.ts:409-440 creates both file-error records with the retained body but no limitations; src/application/decompileDirectory/exportTree.ts:159-227 commits each latest record and projects it into the index.
Evidence provenance: observed for the registered MCP calls, file denial, byte preservation, index contents and restoration; static for the implementation explanation and contract attribution.
Local verification: reproduced through production MCP registration, input validation, fixed-port admission, composed workflow, filesystem export and result delivery. No CLI invocation, stdio transport, native-provider selection or real-engine analysis was run.
Reproduction completeness: complete for the provider-independent export-record contract using the runnable synthetic provider fixture above. Real-provider occurrence and caveat-volume claims are unverified and are not required to generate the accepted fixture observations.

Additional context

Restoration check: during the same unreadable and conflicting resumes, the failed index record should keep all 40 limitations alongside the retained body/path, while retaining its appropriate failure code and preserving the caller's file bytes. The executed readable/original-byte controls already restore the previous complete written record via checksum verification, without another per-function provider observation. The historical complete records remain available in the journal; the test does not establish permanent loss of caveats.

Searched the supplied open/closed issue index and full relevant bodies. The empty-target issue and PR concern CLI target validation; the earlier directory-export feature and source-Evidence follow-up concern creation and source-link retention. None describes loss of per-function limitations from failed resume records.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: Observed file-error resumes remove complete limitations from a documented current index while retaining the associated body and path; the header contains only a subset in the accepted long-caveat fixture. File contents and historical complete journal records survive.
Workaround status: available
Workaround basis: Restoring file readability and resuming recovered all caveats in the permission case. Restoring known original bytes and resuming recovered them in the conflict case; the journal also preserves the earlier complete record without changing the conflicting file.

Originally created by @coygeek on GitHub (Oct 27, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/7351 ## Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ## Area Native analysis or decompilation ## Observed behavior Through the registered MCP `decompile_all` tool with a fixed synthetic provider, an initial export stored 40 complete limitations in its function index record. Making that function file unreadable and resuming changed the record to `failed` / `output_io`, retained the earlier body and relative path, and removed `limitations`. A byte conflict produced the same omission with `output_exists`, including on a repeated resume. Reporting these file failures and preserving the caller's bytes are correct; discarding the retained observation's caveats violates the index contract. The original file header contains only a capped subset of these caveats. ## Reproduction Executed once through the production MCP SDK client, in-memory transport, registered handler and shared export workflow. Run the following from the baseline checkout after its locked dependencies and compiled runtime have been prepared, with pinned Node.js 24 on PATH. A non-root POSIX account that enforces mode-000 read denial is required. All fixture files, HOME and temporary output are isolated; no native engine or target program is opened or executed. ```sh set -eu REA_ROOT="$PWD" WORK="$(mktemp -d)" trap 'rm -rf "$WORK"' EXIT mkdir -p "$WORK/home" "$WORK/tmp" env -i PATH="$PATH" HOME="$WORK/home" TMPDIR="$WORK/tmp" REA_ROOT="$REA_ROOT" WORK="$WORK" node --input-type=module <<'JS' import assert from 'node:assert/strict'; import {createRequire} from 'node:module'; import {pathToFileURL} from 'node:url'; import {join} from 'node:path'; import {chmod, readFile, writeFile, copyFile} from 'node:fs/promises'; import {release} from 'node:os'; const root=process.env.REA_ROOT, work=process.env.WORK; const require=createRequire(join(root,'package.json')); const {Client,InMemoryTransport}=require('@modelcontextprotocol/client'); const pino=require('pino'); const load=p=>import(pathToFileURL(join(root,'dist',p)).href); const {EvidenceMcpServer}=await load('server/EvidenceMcpServer.js'); const {registerEnhancedTools}=await load('server/registerEnhancedTools.js'); const {createToolResultDelivery}=await load('server/toolResult.js'); const {withAdmittedAnalysis}=await load('server/analysisAdmission.js'); const {createAnalysisExecution}=await load('application/AnalysisProvider.js'); const {createAnalysisProfile}=await load('domain/analysisProfile.js'); const {ok}=await load('domain/result.js'); const provider={id:'fixture',name:'Fixture',version:'1'}; const profile=createAnalysisProfile(provider,{decompiler:'fixture'}); const subject={path:'fixture',sha256:'ab'.repeat(32),format:'elf',architecture:'x86_64'}; const caveats=Array.from({length:40},(_,i)=>`fixture caveat ${i}: ${'x'.repeat(150)}`); const body={available:true,provenance:'fixture',ranges:[{start:'0x1000',end:'0x1000'}],total_bytes:1,span_bytes:1,non_contiguous:false,contains_entry:true}; const calls=[]; const analysis={execute:async op=>{ calls.push(op); const value=op==='list_procedures'?[{address:'0x1000',value:'main'}]:op==='procedure_info'?{name:'main',entrypoint:'0x1000',basicblock_count:1,length:1,signature:null,locals:[],classification:null,body}:op==='procedure_pseudo_code'?'int main(void) { return 0; }':assert.fail(`unexpected ${op}`); return ok(createAnalysisExecution(value,provider,{subject,analysisProfile:profile,limitations:op==='list_procedures'?[]:caveats})); }}; const server=new EvidenceMcpServer({name:'fixture-server',version:'1'},{},undefined,createToolResultDelivery(4*1024*1024)); registerEnhancedTools(server,{withAdmittedAnalysis:withAdmittedAnalysis({kind:'fixed',analysis}),logger:pino({enabled:false}),activeTarget:()=>undefined,analysisProfile:()=>profile,recordEvidence:undefined,recordUnknown:undefined,allowsSnapshotReplay:undefined,recordWorkflowSnapshot:undefined}); const client=new Client({name:'fixture-client',version:'1'}); const [ct,st]=InMemoryTransport.createLinkedPair(); const out=join(work,'out'), file=join(out,'functions','0x1000.c'); const record=async()=>JSON.parse(await readFile(join(out,'decompile-index.json'),'utf8')).functions[0]; const run=async phase=>{ const start=calls.length; const reply=await client.callTool({name:'decompile_all',arguments:{output_directory:out,...(phase==='first'?{}:{resume:true})}}); await writeFile(join(work,`${phase}-reply.json`),JSON.stringify(reply,null,2)); assert.notEqual(reply.isError,true,JSON.stringify(reply)); const r=await record(); await copyFile(join(out,'decompile-index.json'),join(work,`${phase}-index.json`)); console.log(JSON.stringify({phase,status:r.status,failure:r.failure?.code??null,limitations:r.limitations?.length??null,body:r.body,relative_path:r.relative_path,calls:calls.slice(start)})); return r; }; try { await server.connect(st);await client.connect(ct); console.log(JSON.stringify({node:process.version,platform:process.platform,release:release(),architecture:process.arch,uid:process.getuid?.()})); const first=await run('first');assert.deepEqual(first.limitations,caveats); const original=await readFile(file);await writeFile(join(work,'original.c'),original); assert.ok(original.toString().includes('limitations_truncated: true')); assert.ok(!original.toString().includes(caveats[39])); console.log(JSON.stringify({header_truncated:true,last_caveat_absent:true})); await chmod(file,0); let denied=false;try{await readFile(file);}catch(e){assert.equal(e.code,'EACCES');denied=true;}assert.ok(denied); const io=await run('unreadable');assert.equal(io.status,'failed');assert.equal(io.failure.code,'output_io');assert.equal(io.limitations,undefined);assert.deepEqual(io.body,first.body);assert.equal(io.relative_path,first.relative_path); await chmod(file,0o600);assert.deepEqual(await readFile(file),original); const repaired=await run('readable');assert.equal(repaired.status,'written');assert.deepEqual(repaired.limitations,caveats); const conflict=Buffer.from('caller replacement\n');await writeFile(file,conflict); const bad=await run('conflict');assert.equal(bad.failure.code,'output_exists');assert.equal(bad.limitations,undefined);assert.deepEqual(bad.body,first.body);assert.deepEqual(await readFile(file),conflict); // Demonstrate repeat-failure persistence without asking provider to regenerate it. const repeated=await run('conflict-again');assert.equal(repeated.limitations,undefined);assert.deepEqual(repeated.body,first.body); await writeFile(file,original); const restored=await run('restored');assert.deepEqual(restored.limitations,caveats);assert.equal(restored.status,'written'); const journal=(await readFile(join(out,'decompile-journal.jsonl'),'utf8')).trim().split('\n').map(JSON.parse); assert.equal(journal.filter(x=>x.kind==='function'&&x.status==='written'&&x.limitations?.length===40).length,3); console.log(JSON.stringify({journal_prior_complete_records_preserved:true,restoration_skipped_without_provider_reads:true,assertions:'passed'})); }finally{await chmod(file,0o600).catch(()=>{});await Promise.allSettled([client.close(),server.close()]);} JS ``` The assertions passed. The six calls returned successful MCP replies containing manifests; the file-error manifests correctly counted one failed function. Only the initial call read `procedure_info` and `procedure_pseudo_code`; each later call read only `list_procedures`. ## Expected behavior Every function's current index record must retain the complete limitations of the observations it uses, including when a resume reports a file error and retains an earlier body fact. Keep the intentional failed status and leave unreadable or conflicting function files unchanged. A capped header does not replace the complete index record. ## Impact and scope P2: consumers of the documented complete index lose the caveats associated with a retained function observation while its file remains unreadable or conflicting. One six-call synthetic-provider sequence reproduced the omission in the unreadable phase and both conflict phases. This is bounded, recoverable index loss: earlier complete records remain in the append-only journal. Restoring read permission and resuming restored all 40 limitations; restoring the original bytes after a conflict did the same without per-function provider reads. The test preserved the caller's conflicting bytes until its explicit restoration step. NOT_ENOUGH_INFO on real-provider caveat lengths, occurrence frequency in real exports, other operating systems or real native-engine behavior; none was exercised. ## REA version Source revision `ace4dbf36e8c8f36a0e192aff9a11484ed428358`, using the prepared compiled runtime and locked dependencies. ## Runtime and platform Node.js v24.22.1; macOS arm64, Darwin 27.0.0; non-root account with observed EACCES on the fixture's mode-000 file. MCP uses the locked SDK's in-memory transport. ## Target and artifact context One synthetic function at `0x1000`, a valid matching provider/profile/target digest and 40 distinct fixture caveats accepted by the existing observation and journal schemas. The output is a disposable directory export. The fixed port does not select or launch a real provider and establishes no real-provider behavior. ## Interface MCP ## Earliest failure stage Artifact or evidence registration ## Relevant output or evidence - Expected source: `docs/native-investigation.md:210-218` promises complete limitations for every function record regardless of status and describes resume verification; `src/contracts/enhancedToolContracts.ts:55` advertises complete per-function index limitations. `src/application/decompileDirectory/functionFile.ts:67-77` explicitly directs readers of a truncated header to the complete index. - Failure source: The executed fixture above produced `written/40 → failed/output_io/absent → written/40 → failed/output_exists/absent → failed/output_exists/absent → written/40`. Each failed record retained the original body and relative path. Source corroboration: `src/application/decompileDirectory/decompileAll.ts:409-440` creates both file-error records with the retained body but no limitations; `src/application/decompileDirectory/exportTree.ts:159-227` commits each latest record and projects it into the index. - Evidence provenance: observed for the registered MCP calls, file denial, byte preservation, index contents and restoration; static for the implementation explanation and contract attribution. - Local verification: reproduced through production MCP registration, input validation, fixed-port admission, composed workflow, filesystem export and result delivery. No CLI invocation, stdio transport, native-provider selection or real-engine analysis was run. - Reproduction completeness: complete for the provider-independent export-record contract using the runnable synthetic provider fixture above. Real-provider occurrence and caveat-volume claims are unverified and are not required to generate the accepted fixture observations. ## Additional context Restoration check: during the same unreadable and conflicting resumes, the failed index record should keep all 40 limitations alongside the retained body/path, while retaining its appropriate failure code and preserving the caller's file bytes. The executed readable/original-byte controls already restore the previous complete `written` record via checksum verification, without another per-function provider observation. The historical complete records remain available in the journal; the test does not establish permanent loss of caveats. Searched the supplied open/closed issue index and full relevant bodies. The empty-target issue and PR concern CLI target validation; the earlier directory-export feature and source-Evidence follow-up concern creation and source-link retention. None describes loss of per-function limitations from failed resume records. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: Observed file-error resumes remove complete limitations from a documented current index while retaining the associated body and path; the header contains only a subset in the accepted long-caveat fixture. File contents and historical complete journal records survive. - Workaround status: available - Workaround basis: Restoring file readability and resuming recovered all caveats in the permission case. Restoring known original bytes and resuming recovered them in the conflict case; the journal also preserves the earlier complete record without changing the conflicting file.

bsakaguchi closed this issue

2026-10-31 00:49:17 +00:00

bsakaguchi referenced this issue

2026-10-31 01:12:56 +00:00

[PR #4429] fix(native): isolate empty Swift conformance names #6237

bsakaguchi referenced this issue

2026-10-31 01:12:57 +00:00

[PR #4434] [CLOSED] fix(native): retain valid Swift vtables beside empty class names #6240

bsakaguchi referenced this issue

2026-10-31 01:13:05 +00:00

[PR #4469] [MERGED] fix(native): reject empty Swift class vtable owner before slot insertion #6260