[GH-ISSUE #6067] [Bug] PyInstaller extraction reports changed source admission as unreadable I/O #3709

New issue

Closed

opened 2026-10-31 00:43:16 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:43:16 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 21, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6067

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

Through extract_artifact, a source-name replacement observed during PyInstaller reader admission returns artifact_operation_failed / execution_failure, reason io, with PyInstaller archive could not be opened and generic file-access advice. The real descriptor was opened successfully; openRegularFile subsequently raised RegularFileChangedError because the selected name no longer matched it.

This was executed through production MCP composition and transport on macOS with the no-O_NOFOLLOW platform branch explicitly simulated. It is not a real Windows or naturally occurring race observation. Initial and restored-source extraction controls succeeded, and the rejected admission descriptor closed.

Reproduction

Save the following executed focused test as tests/process-global/pyinstallerAdmissionChange.test.ts in a source checkout with locked dependencies. Set REPRO_ROOT to an existing disposable writable directory outside the checkout and run:

node node_modules/vitest/vitest.mjs run tests/process-global/pyinstallerAdmissionChange.test.ts --maxWorkers=1

The constants mock sets O_NOFOLLOW to zero, selecting the fallback used where that flag is unavailable. The open wrapper still opens a real descriptor, then renames the source and recreates its name with identical bytes. The production comparison generates RegularFileChangedError; the test does not inject that exception. Arming at prepareContainer confines replacement to the extraction reader's later admission, after inventory. The test characterizes current behavior, so it passes while the bug is present.

import { Client, InMemoryTransport } from '@modelcontextprotocol/client';
import { mkdir, mkdtemp, writeFile, readFile, rename, rm, realpath } from 'node:fs/promises';
import { join } from 'node:path';
import { expect, it, vi } from 'vitest';

const state = vi.hoisted(() => ({ path: '', arm: false, replacements: 0, handles: [] as any[] }));
vi.mock('node:fs', async (original) => {
  const actual = await original<typeof import('node:fs')>();
  return { ...actual, constants: { ...actual.constants, O_NOFOLLOW: 0 } };
});
vi.mock('node:fs/promises', async (original) => {
  const actual = await original<typeof import('node:fs/promises')>();
  return { ...actual, open: async (...args: Parameters<typeof actual.open>) => {
    const handle = await actual.open(...args);
    if (state.arm && String(args[0]) === state.path) {
      state.arm = false;
      state.handles.push(handle);
      const bytes = await actual.readFile(state.path);
      await actual.rename(state.path, state.path + '.original');
      await actual.writeFile(state.path, bytes);
      state.replacements++;
    }
    return handle;
  }};
});
import { createArtifactBinarySession } from '../../src/composition/binary.js';
import { createServer } from '../../src/server/createServer.js';
import { PyInstallerArtifactReader } from '../../src/artifacts/PyInstallerArtifactReader.js';
import { RegularFileChangedError } from '../../src/filesystem/RegularFile.js';
import * as admission from '../../src/artifacts/pyinstaller/PyInstallerFileAdmission.js';
import { cArchiveExecutable } from '../../src/artifacts/pyinstaller/CArchive.fixture.js';
import { parseEvidence } from '../../src/domain/evidence.js';

const root = async () => realpath(await mkdtemp(join(process.env.REPRO_ROOT!, 'case-')));
const connect = async () => {
  const session = await createArtifactBinarySession(process.env);
  const server = createServer({kind:'session',session});
  const client = new Client({name:'artifact-reproduction',version:'1'});
  const [a,b] = InMemoryTransport.createLinkedPair();
  await server.connect(b); await client.connect(a);
  return {client, close: async () => { await client.close();await server.close();await session.close(); }};
};
const error = (r:any) => JSON.parse(r.content.find((x:any)=>x.type==='text').text).error;

it('PyInstaller extraction maps an observed admission identity change to io through MCP',async()=>{
 const dir=await root();const path=join(dir,'app');state.path=path;state.replacements=0;state.handles=[];
 await writeFile(path,cArchiveExecutable({layout:'2.1',pythonVersion:312,pythonLibrary:'libpython3.12.so',members:[{name:'data.txt',typeCode:'x',payload:Buffer.from('data'),compress:false}]}));
 const mcp=await connect();const original=PyInstallerArtifactReader.prototype.prepareContainer;
 const admissionSpy=vi.spyOn(admission,'pyInstallerFileAdmissionFailure');
 let arm=false;const outputs:string[]=[];
 const spy=vi.spyOn(PyInstallerArtifactReader.prototype,'prepareContainer').mockImplementation(async function(...args){if(arm)state.arm=true;return original.apply(this,args);});
 const invoke=()=>mcp.client.callTool({name:'extract_artifact',arguments:{}});
 const success=async()=>{const r=await invoke();expect(r.isError,JSON.stringify(r)).not.toBe(true);const result:any=parseEvidence(r.structuredContent).normalized_result;outputs.push(result.output_root);expect(await readFile(join(result.output_root,'data.txt'),'utf8')).toBe('data');};
 try {
  expect((await mcp.client.callTool({name:'open_binary',arguments:{path}})).isError).not.toBe(true);
  await success();arm=true;const response=await invoke();arm=false;
  expect(state.replacements).toBe(1);
  expect(admissionSpy).toHaveBeenCalled();expect(admissionSpy.mock.calls.at(-1)![1]).toBeInstanceOf(RegularFileChangedError);
  expect(response.isError).toBe(true);const projected=error(response);
  expect(projected).toMatchObject({code:'artifact_operation_failed',category:'execution_failure',details:{reason:'io',detail:'PyInstaller archive could not be opened'}});
  expect(state.handles.every(h=>h.fd===-1)).toBe(true);
  console.log('PYINSTALLER_ADMISSION',JSON.stringify({actualCause:(admissionSpy.mock.calls.at(-1)![1] as Error).name,projected,replacements:state.replacements,handlesClosed:state.handles.every(h=>h.fd===-1)}).replaceAll(dir,'<fixture-root>'));
  await rm(path);await rename(path+'.original',path);await success();
 } finally {state.arm=false;spy.mockRestore();admissionSpy.mockRestore();await mcp.close();for(const output of outputs)await rm(output,{recursive:true,force:true});await rm(dir,{recursive:true,force:true});}
},30000);

Expected behavior

Preserve the observed changed-file condition as an integrity outcome rather than an unreadable archive. The reader's existing post-read source check already uses reason integrity. RegularFileChangedError and its admission check explicitly identify a mismatch after a successful open.

This report does not require a new error code or automatic retryability. The existing integrity category and guidance toward a stable artifact would preserve the reason.

Impact and scope

P2: a bounded admission race receives the wrong diagnosis and recovery advice. The operation still refuses unstable input, and no successful extraction from the mismatched source or descriptor leak was observed. Restoring the stable original source made the same MCP extraction succeed and returned the exact data.txt bytes.

Observed scope: extract_artifact with the no-O_NOFOLLOW fallback simulated on macOS. Real Windows filesystem semantics, natural race frequency and private PYZ-image admission were not tested. Inventory normally supplies an already-admitted root descriptor to PyInstallerArtifactReader, so this reproduction does not establish that inspect_artifact reaches the same direct-reader admission mapper.

REA version

rea-agents 6.7.0; source revision 52415975942a4b9c53a817915b313ccc404f280e. Built CLI and source MCP tests at this revision.

Runtime and platform

macOS 27.0.1 arm64; Node.js v26.12.0, non-root account. Repository Node pin: 24.18.0. No Hopper, Ghidra, IDA, ibtool or target application execution. Repository-pinned Vitest 5.0.3; SDK InMemoryTransport. Imported O_NOFOLLOW is set to zero solely to exercise the fallback branch. Host remains macOS.

Target and artifact context

Synthetic PyInstaller 2.1 executable with one uncompressed type-x member data.txt containing data. The source name is replaced with a byte-identical regular file after the real descriptor opens. The executable is inspected and extracted, never launched.

Interface

MCP

Earliest failure stage

Artifact or evidence registration

Relevant output or evidence

The final standalone run passed one test. The observed source replacement count is one. A spy at the adapter observed the actual cause name RegularFileChangedError, and all faulted admission handles had fd -1 after the request.

The public error contained:

{
  "code": "artifact_operation_failed",
  "category": "execution_failure",
  "retryable": false,
  "message": "Artifact could not be read or written. Check file access and try again.",
  "details": {
    "operation": "extract_artifact",
    "reason": "io",
    "detail": "PyInstaller archive could not be opened"
  }
}

Readable and restored-source controls returned successful Evidence and extracted data.txt with exactly data.

Typed outcome or recovery expected

An integrity/changed-artifact outcome retaining the selected source identity failure and recovery toward a stable copy. Preserve ordinary unknown I/O, non-regular input, cancellation and cleanup classifications.

Analysis or suspected cause

Source-derived: extraction constructs a new PyInstallerArtifactReader without a retained source and calls prepareContainer, reaching reader admission. pyInstallerFileAdmissionFailure handles cancellation, retained cleanup, existing artifact errors and non-regular selections, but has no RegularFileChangedError branch. Its generic io fallback explains the observed public response. Handle the existing changed-file type at this adapter instead of changing all unknown admission failures.

Additional context

Fetched origin/main at d4c637f4cfcb7ae3b48d0c3047f0e9171d43b26d. The relevant producer and classification lines are unchanged. Changes elsewhere in the two shared projection/presentation files concern captured-output inclusion and fixed resource allowance remediation. Later main was source-inspected, not executed. No repaired software or known-good revision is established.

#5836 concerns PyInstaller private-storage capacity and lost resource-boundary recovery, not source-identity admission. Other PyInstaller issues/PRs found concern archive format support, filenames, inventory coverage, release metadata or source-module bytecode headers. #5953, #5970, #5991, #5997 and #6004 concern independent filesystem classifiers. The private PYZ store calls the same admission helper, but applicability there is source-derived only. No known-good revision or repaired-behavior run is established.

Originally created by @coygeek on GitHub (Oct 21, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6067 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior Through `extract_artifact`, a source-name replacement observed during PyInstaller reader admission returns `artifact_operation_failed` / `execution_failure`, reason `io`, with `PyInstaller archive could not be opened` and generic file-access advice. The real descriptor was opened successfully; `openRegularFile` subsequently raised `RegularFileChangedError` because the selected name no longer matched it. This was executed through production MCP composition and transport on macOS with the no-O_NOFOLLOW platform branch explicitly simulated. It is not a real Windows or naturally occurring race observation. Initial and restored-source extraction controls succeeded, and the rejected admission descriptor closed. ### Reproduction Save the following executed focused test as `tests/process-global/pyinstallerAdmissionChange.test.ts` in a source checkout with locked dependencies. Set `REPRO_ROOT` to an existing disposable writable directory outside the checkout and run: ```sh node node_modules/vitest/vitest.mjs run tests/process-global/pyinstallerAdmissionChange.test.ts --maxWorkers=1 ``` The constants mock sets O_NOFOLLOW to zero, selecting the fallback used where that flag is unavailable. The open wrapper still opens a real descriptor, then renames the source and recreates its name with identical bytes. The production comparison generates RegularFileChangedError; the test does not inject that exception. Arming at prepareContainer confines replacement to the extraction reader's later admission, after inventory. The test characterizes current behavior, so it passes while the bug is present. ```ts import { Client, InMemoryTransport } from '@modelcontextprotocol/client'; import { mkdir, mkdtemp, writeFile, readFile, rename, rm, realpath } from 'node:fs/promises'; import { join } from 'node:path'; import { expect, it, vi } from 'vitest'; const state = vi.hoisted(() => ({ path: '', arm: false, replacements: 0, handles: [] as any[] })); vi.mock('node:fs', async (original) => { const actual = await original<typeof import('node:fs')>(); return { ...actual, constants: { ...actual.constants, O_NOFOLLOW: 0 } }; }); vi.mock('node:fs/promises', async (original) => { const actual = await original<typeof import('node:fs/promises')>(); return { ...actual, open: async (...args: Parameters<typeof actual.open>) => { const handle = await actual.open(...args); if (state.arm && String(args[0]) === state.path) { state.arm = false; state.handles.push(handle); const bytes = await actual.readFile(state.path); await actual.rename(state.path, state.path + '.original'); await actual.writeFile(state.path, bytes); state.replacements++; } return handle; }}; }); import { createArtifactBinarySession } from '../../src/composition/binary.js'; import { createServer } from '../../src/server/createServer.js'; import { PyInstallerArtifactReader } from '../../src/artifacts/PyInstallerArtifactReader.js'; import { RegularFileChangedError } from '../../src/filesystem/RegularFile.js'; import * as admission from '../../src/artifacts/pyinstaller/PyInstallerFileAdmission.js'; import { cArchiveExecutable } from '../../src/artifacts/pyinstaller/CArchive.fixture.js'; import { parseEvidence } from '../../src/domain/evidence.js'; const root = async () => realpath(await mkdtemp(join(process.env.REPRO_ROOT!, 'case-'))); const connect = async () => { const session = await createArtifactBinarySession(process.env); const server = createServer({kind:'session',session}); const client = new Client({name:'artifact-reproduction',version:'1'}); const [a,b] = InMemoryTransport.createLinkedPair(); await server.connect(b); await client.connect(a); return {client, close: async () => { await client.close();await server.close();await session.close(); }}; }; const error = (r:any) => JSON.parse(r.content.find((x:any)=>x.type==='text').text).error; it('PyInstaller extraction maps an observed admission identity change to io through MCP',async()=>{ const dir=await root();const path=join(dir,'app');state.path=path;state.replacements=0;state.handles=[]; await writeFile(path,cArchiveExecutable({layout:'2.1',pythonVersion:312,pythonLibrary:'libpython3.12.so',members:[{name:'data.txt',typeCode:'x',payload:Buffer.from('data'),compress:false}]})); const mcp=await connect();const original=PyInstallerArtifactReader.prototype.prepareContainer; const admissionSpy=vi.spyOn(admission,'pyInstallerFileAdmissionFailure'); let arm=false;const outputs:string[]=[]; const spy=vi.spyOn(PyInstallerArtifactReader.prototype,'prepareContainer').mockImplementation(async function(...args){if(arm)state.arm=true;return original.apply(this,args);}); const invoke=()=>mcp.client.callTool({name:'extract_artifact',arguments:{}}); const success=async()=>{const r=await invoke();expect(r.isError,JSON.stringify(r)).not.toBe(true);const result:any=parseEvidence(r.structuredContent).normalized_result;outputs.push(result.output_root);expect(await readFile(join(result.output_root,'data.txt'),'utf8')).toBe('data');}; try { expect((await mcp.client.callTool({name:'open_binary',arguments:{path}})).isError).not.toBe(true); await success();arm=true;const response=await invoke();arm=false; expect(state.replacements).toBe(1); expect(admissionSpy).toHaveBeenCalled();expect(admissionSpy.mock.calls.at(-1)![1]).toBeInstanceOf(RegularFileChangedError); expect(response.isError).toBe(true);const projected=error(response); expect(projected).toMatchObject({code:'artifact_operation_failed',category:'execution_failure',details:{reason:'io',detail:'PyInstaller archive could not be opened'}}); expect(state.handles.every(h=>h.fd===-1)).toBe(true); console.log('PYINSTALLER_ADMISSION',JSON.stringify({actualCause:(admissionSpy.mock.calls.at(-1)![1] as Error).name,projected,replacements:state.replacements,handlesClosed:state.handles.every(h=>h.fd===-1)}).replaceAll(dir,'<fixture-root>')); await rm(path);await rename(path+'.original',path);await success(); } finally {state.arm=false;spy.mockRestore();admissionSpy.mockRestore();await mcp.close();for(const output of outputs)await rm(output,{recursive:true,force:true});await rm(dir,{recursive:true,force:true});} },30000); ``` ### Expected behavior Preserve the observed changed-file condition as an integrity outcome rather than an unreadable archive. The reader's existing [post-read source check already uses reason integrity](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/artifacts/PyInstallerArtifactReader.ts#L541-L552). [RegularFileChangedError and its admission check](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/filesystem/RegularFile.ts#L103-L123) explicitly identify a mismatch after a successful open. This report does not require a new error code or automatic retryability. The existing integrity category and guidance toward a stable artifact would preserve the reason. ### Impact and scope P2: a bounded admission race receives the wrong diagnosis and recovery advice. The operation still refuses unstable input, and no successful extraction from the mismatched source or descriptor leak was observed. Restoring the stable original source made the same MCP extraction succeed and returned the exact data.txt bytes. Observed scope: `extract_artifact` with the no-O_NOFOLLOW fallback simulated on macOS. Real Windows filesystem semantics, natural race frequency and private PYZ-image admission were not tested. Inventory normally supplies an already-admitted root descriptor to PyInstallerArtifactReader, so this reproduction does not establish that `inspect_artifact` reaches the same direct-reader admission mapper. ### REA version `rea-agents` 6.7.0; source revision `52415975942a4b9c53a817915b313ccc404f280e`. Built CLI and source MCP tests at this revision. ### Runtime and platform macOS 27.0.1 arm64; Node.js v26.12.0, non-root account. Repository Node pin: 24.18.0. No Hopper, Ghidra, IDA, ibtool or target application execution. Repository-pinned Vitest 5.0.3; SDK InMemoryTransport. Imported O_NOFOLLOW is set to zero solely to exercise the fallback branch. Host remains macOS. ### Target and artifact context Synthetic PyInstaller 2.1 executable with one uncompressed type-x member `data.txt` containing `data`. The source name is replaced with a byte-identical regular file after the real descriptor opens. The executable is inspected and extracted, never launched. ### Interface MCP ### Earliest failure stage Artifact or evidence registration ### Relevant output or evidence The final standalone run passed one test. The observed source replacement count is one. A spy at the adapter observed the actual cause name `RegularFileChangedError`, and all faulted admission handles had fd -1 after the request. The public error contained: ```json { "code": "artifact_operation_failed", "category": "execution_failure", "retryable": false, "message": "Artifact could not be read or written. Check file access and try again.", "details": { "operation": "extract_artifact", "reason": "io", "detail": "PyInstaller archive could not be opened" } } ``` Readable and restored-source controls returned successful Evidence and extracted `data.txt` with exactly `data`. ### Typed outcome or recovery expected An integrity/changed-artifact outcome retaining the selected source identity failure and recovery toward a stable copy. Preserve ordinary unknown I/O, non-regular input, cancellation and cleanup classifications. ### Analysis or suspected cause Source-derived: extraction [constructs a new PyInstallerArtifactReader without a retained source](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/artifacts/extraction/ArtifactExtraction.ts#L471-L485) and [calls prepareContainer](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/artifacts/extraction/ArtifactExtraction.ts#L242-L267), reaching [reader admission](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/artifacts/PyInstallerArtifactReader.ts#L378-L391). [pyInstallerFileAdmissionFailure](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/artifacts/pyinstaller/PyInstallerFileAdmission.ts#L1-L35) handles cancellation, retained cleanup, existing artifact errors and non-regular selections, but has no RegularFileChangedError branch. Its generic io fallback explains the observed public response. Handle the existing changed-file type at this adapter instead of changing all unknown admission failures. ### Additional context Fetched `origin/main` at `d4c637f4cfcb7ae3b48d0c3047f0e9171d43b26d`. The relevant producer and classification lines are unchanged. Changes elsewhere in the two shared projection/presentation files concern captured-output inclusion and fixed resource allowance remediation. Later main was source-inspected, not executed. No repaired software or known-good revision is established. #5836 concerns PyInstaller private-storage capacity and lost resource-boundary recovery, not source-identity admission. Other PyInstaller issues/PRs found concern archive format support, filenames, inventory coverage, release metadata or source-module bytecode headers. #5953, #5970, #5991, #5997 and #6004 concern independent filesystem classifiers. The private PYZ store calls the same admission helper, but applicability there is source-derived only. No known-good revision or repaired-behavior run is established.

bsakaguchi closed this issue

2026-10-31 00:43:16 +00:00

bsakaguchi referenced this issue

2026-11-01 21:10:57 +00:00

[GH-ISSUE #3796] [Tracker] Open feature requests by area (October 2026) #1471