morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #6004] [Bug] Model inspection misclassifies host I/O and descriptor failures as unavailable targets #3671

New issue

Closed

opened 2026-10-31 00:42:55 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-31 00:42:55 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 21, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6004

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

The public inspect_model_artifact MCP tool reports host descriptor and read failures as target_unavailable / unavailable and tells callers to check whether the selected path exists, is readable, and has a supported format.

This reproduced for both valid GGUF and safetensors fixtures with an injected EMFILE at selected-file open and an injected EIO at FileHandle.read. An uncoded read error through the same path correctly reached the general execution_failure fallback. The same files inspected successfully before and immediately after each fault; all opened handles closed.

These were controlled filesystem faults, not naturally occurring descriptor exhaustion or hardware failure. The actual provider, service, production MCP registration, public error projection and SDK in-memory transport ran normally.

Reproduction

In a prepared source checkout with dependencies and existing build artifacts, save the following as tests/process-global/modelHostErrors.repro.test.ts. Set REA_REPRO_ROOT to an existing disposable writable directory and run:

node node_modules/vitest/vitest.mjs run tests/process-global/modelHostErrors.repro.test.ts --maxWorkers=1

The executed test passed both parameterized cases. Its assertions intentionally characterize the faulty result. It creates and removes the two selected files; no external analysis provider is used.

import { writeFile, rm, type FileHandle } from "node:fs/promises";
import { join } from "node:path";
import { expect, vi } from "vitest";
const fault = vi.hoisted(() => ({ path: "", stage: "", cause: undefined as Error | undefined, handles: [] as FileHandle[] }));
vi.mock("node:fs/promises", async (importOriginal) => {
  const original = await importOriginal<typeof import("node:fs/promises")>();
  return { ...original, open: async (...args: Parameters<typeof original.open>) => {
    if (String(args[0]) === fault.path && fault.stage === "open") throw fault.cause;
    const handle = await original.open(...args);
    if (String(args[0]) === fault.path) {
      fault.handles.push(handle);
      if (fault.stage === "read") handle.read = async () => { throw fault.cause; };
    }
    return handle;
  }};
});
import { ModelArtifactProvider } from "../../src/model/ModelArtifactProvider.js";
import { ModelArtifactService } from "../../src/application/model/ModelArtifactService.js";
import { EvidenceMcpServer } from "../../src/server/EvidenceMcpServer.js";
import { registerModelArtifactTools } from "../../src/server/registerModelArtifactTools.js";
import { createToolResultDelivery } from "../../src/server/toolResult.js";
import { silentLogger } from "../../src/logger.js";
import { ggufFixture, safetensorsFixture } from "../fixtures/model/model.js";
import { parseMcpToolError } from "../fixtures/mcpToolError.js";
import { mcpTest } from "../support/mcp/mcpFixture.js";

mcpTest.for(["gguf", "safetensors"])("host error classification for %s", async (format, { mcp }) => {
  const root = process.env.REA_REPRO_ROOT!;
  const path = join(root, `selected.${format}`);
  const bytes = format === "gguf" ? ggufFixture().bytes : safetensorsFixture().bytes;
  await writeFile(path, bytes);
  const provider = new ModelArtifactProvider();
  const server = new EvidenceMcpServer({ name: "model-errors-test", version: "1" }, {}, undefined, createToolResultDelivery(undefined));
  registerModelArtifactTools(server, new ModelArtifactService(provider), silentLogger);
  const client = await mcp.connect(server);
  const call = () => client.callTool({ name: "inspect_model_artifact", arguments: { path, detail: "complete" } });
  try {
    expect((await call()).isError).not.toBe(true);
    fault.path = path;
    for (const [stage, code] of [["open", "EMFILE"], ["read", "EIO"], ["read", ""]]) {
      fault.stage = stage!;
      fault.cause = new Error(`controlled ${stage} failure${code ? ` (${code})` : ""}`);
      if (code) Object.assign(fault.cause, { code });
      const response = await call();
      const { error } = parseMcpToolError(response);
      console.log(JSON.stringify({ format, stage, injected_code: code || null, error }));
      expect(error).toMatchObject({ code: code ? "target_unavailable" : "execution_failure", retryable: false });
      if (code) {
        expect(error.category).toBe("unavailable");
        expect(error.message).toContain("Check that the path exists, is readable, and points to a supported file");
        expect(error.details).not.toHaveProperty("system_code");
      }
      expect(fault.handles.every(handle => handle.fd === -1)).toBe(true);
      fault.stage = "";
      expect((await call()).isError).not.toBe(true);
    }
    console.log(JSON.stringify({format, controls: "initial and after each fault succeeded", all_handles_closed: fault.handles.every(handle => handle.fd === -1)}));
  } finally {
    fault.stage = ""; fault.path = "";
    await provider.close();
    await rm(path, { force: true });
  }
});

Expected behavior

Keep host I/O/resource failure distinct from selected-path or supported-format refusal. Preserve the failure reason and use an execution/resource outcome appropriate to the actual host condition; do not infer that the selected model path is unavailable merely because an Error carries a code property.

The repository boundary rule requires meaningful reasons and typed expected failures. The existing Go admission mapping distinguishes permission and path-shaped errno values from remaining provider failures. The model operation already has an execution-failure fallback, demonstrated by the uncoded control.

This does not require making every host failure automatically retryable. Genuine missing-path, access-denied and format refusals should retain their own semantics.

Impact and scope

P2: callers receive the wrong machine-readable diagnosis and path-correction advice for an otherwise valid selected model. The underlying refusal remains effective, the raw reason text survives, and no incorrect successful Evidence, corruption, descriptor leak or security-boundary crossing was observed.

A practical manual workaround is to use the retained host diagnostic rather than reselecting the model. Removing the injected condition restored successful inspection of the unchanged file after every fault. Actual operating-system fault incidence and downstream automated-recovery effects were not measured.

Executed scope is EMFILE on open and EIO on the first header read, for both GGUF and safetensors. Other errno values, mid-stream faults, actual descriptor exhaustion, real device failures, other operating systems and CLI execution were not tested.

REA version

rea-agents 6.7.0; source revision 52415975942a4b9c53a817915b313ccc404f280e.

Fetched origin/main at 6d356b84344659085f2b912857b5e27fe0160c9b: the model reader, filesystem admission and relevant classification/presentation logic remain unchanged. That newer revision was source-inspected, not executed.

Runtime and platform

macOS 27.0.1 arm64; Node.js v26.12.0; Vitest 5.0.3. Repository Node pin: 24.18.0. Controlled filesystem fault injection with real provider and MCP transport; no external provider or model execution.

Target and artifact context

Repository-generated valid GGUF and safetensors declaration fixtures with tensor payload bytes. The selected bytes do not change between successful controls and faulted requests.

Interface

MCP

Earliest failure stage

Setup or provider admission

Relevant output or evidence

Both formats returned the following selected fields for each injected host fault:

{
  "code": "target_unavailable",
  "category": "unavailable",
  "retryable": false
}

For the open fault, details.reason was controlled open failure (EMFILE), and the message and remediation action were:

Cannot open artifact: controlled open failure (EMFILE). Check that the path exists, is readable, and points to a supported file.

For the read fault, the equivalent reason was controlled read failure (EIO), with the same path-checking advice. Neither error had a structured system_code. The uncoded read control returned execution_failure / execution_failure, with details.provider_id: "model-artifact" and the original reason under details.diagnostics. All eight successful controls completed, and all observed handles closed.

Typed outcome or recovery expected

Preserve an execution/resource failure for non-path host errors, including the selected path and actual diagnostic. Reserve selected-target refusal for errors that establish a selected-path or format problem. Preserve the existing independent permission, integrity, cancellation and cleanup handling.

Analysis or suspected cause

Source-derived explanation of the observed behavior: regular-file admission forwards ordinary open errors, and the header reader forwards physical read errors. After handling its known typed and permission cases, ModelArtifactProvider's final coded-error branch converts every remaining Error with a code to BinaryTargetError, bypassing its own ProviderAdapterError fallback.

That error's public code and presentation explain the measured path-specific outcome. Classification should distinguish known selected-path errors from other host failures at the model acquisition/read owner.

Additional context

Related but distinct: #5953 concerns firmware source admission, #5970 concerns denied JavaScript recovery snapshots, #5991 concerns lost JavaScript application filesystem diagnostics, and #5997 concerns bound Mach-O dispatch reads. They involve separate producing boundaries and repairs.

Existing model reports #5895 / PR #5918 concern safetensors-versus-pickle detection; #5749 / PR #5789 concern unsupported GGUF version diagnostics; #5625 concerns partial observations on cancellation. None changes this model host-error classification. No known-good revision or repaired-behavior run is established.

Originally created by @coygeek on GitHub (Oct 21, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6004 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior The public `inspect_model_artifact` MCP tool reports host descriptor and read failures as `target_unavailable` / `unavailable` and tells callers to check whether the selected path exists, is readable, and has a supported format. This reproduced for both valid GGUF and safetensors fixtures with an injected `EMFILE` at selected-file open and an injected `EIO` at `FileHandle.read`. An uncoded read error through the same path correctly reached the general `execution_failure` fallback. The same files inspected successfully before and immediately after each fault; all opened handles closed. These were controlled filesystem faults, not naturally occurring descriptor exhaustion or hardware failure. The actual provider, service, production MCP registration, public error projection and SDK in-memory transport ran normally. ### Reproduction In a prepared source checkout with dependencies and existing build artifacts, save the following as `tests/process-global/modelHostErrors.repro.test.ts`. Set `REA_REPRO_ROOT` to an existing disposable writable directory and run: ```sh node node_modules/vitest/vitest.mjs run tests/process-global/modelHostErrors.repro.test.ts --maxWorkers=1 ``` The executed test passed both parameterized cases. Its assertions intentionally characterize the faulty result. It creates and removes the two selected files; no external analysis provider is used. ```ts import { writeFile, rm, type FileHandle } from "node:fs/promises"; import { join } from "node:path"; import { expect, vi } from "vitest"; const fault = vi.hoisted(() => ({ path: "", stage: "", cause: undefined as Error | undefined, handles: [] as FileHandle[] })); vi.mock("node:fs/promises", async (importOriginal) => { const original = await importOriginal<typeof import("node:fs/promises")>(); return { ...original, open: async (...args: Parameters<typeof original.open>) => { if (String(args[0]) === fault.path && fault.stage === "open") throw fault.cause; const handle = await original.open(...args); if (String(args[0]) === fault.path) { fault.handles.push(handle); if (fault.stage === "read") handle.read = async () => { throw fault.cause; }; } return handle; }}; }); import { ModelArtifactProvider } from "../../src/model/ModelArtifactProvider.js"; import { ModelArtifactService } from "../../src/application/model/ModelArtifactService.js"; import { EvidenceMcpServer } from "../../src/server/EvidenceMcpServer.js"; import { registerModelArtifactTools } from "../../src/server/registerModelArtifactTools.js"; import { createToolResultDelivery } from "../../src/server/toolResult.js"; import { silentLogger } from "../../src/logger.js"; import { ggufFixture, safetensorsFixture } from "../fixtures/model/model.js"; import { parseMcpToolError } from "../fixtures/mcpToolError.js"; import { mcpTest } from "../support/mcp/mcpFixture.js"; mcpTest.for(["gguf", "safetensors"])("host error classification for %s", async (format, { mcp }) => { const root = process.env.REA_REPRO_ROOT!; const path = join(root, `selected.${format}`); const bytes = format === "gguf" ? ggufFixture().bytes : safetensorsFixture().bytes; await writeFile(path, bytes); const provider = new ModelArtifactProvider(); const server = new EvidenceMcpServer({ name: "model-errors-test", version: "1" }, {}, undefined, createToolResultDelivery(undefined)); registerModelArtifactTools(server, new ModelArtifactService(provider), silentLogger); const client = await mcp.connect(server); const call = () => client.callTool({ name: "inspect_model_artifact", arguments: { path, detail: "complete" } }); try { expect((await call()).isError).not.toBe(true); fault.path = path; for (const [stage, code] of [["open", "EMFILE"], ["read", "EIO"], ["read", ""]]) { fault.stage = stage!; fault.cause = new Error(`controlled ${stage} failure${code ? ` (${code})` : ""}`); if (code) Object.assign(fault.cause, { code }); const response = await call(); const { error } = parseMcpToolError(response); console.log(JSON.stringify({ format, stage, injected_code: code || null, error })); expect(error).toMatchObject({ code: code ? "target_unavailable" : "execution_failure", retryable: false }); if (code) { expect(error.category).toBe("unavailable"); expect(error.message).toContain("Check that the path exists, is readable, and points to a supported file"); expect(error.details).not.toHaveProperty("system_code"); } expect(fault.handles.every(handle => handle.fd === -1)).toBe(true); fault.stage = ""; expect((await call()).isError).not.toBe(true); } console.log(JSON.stringify({format, controls: "initial and after each fault succeeded", all_handles_closed: fault.handles.every(handle => handle.fd === -1)})); } finally { fault.stage = ""; fault.path = ""; await provider.close(); await rm(path, { force: true }); } }); ``` ### Expected behavior Keep host I/O/resource failure distinct from selected-path or supported-format refusal. Preserve the failure reason and use an execution/resource outcome appropriate to the actual host condition; do not infer that the selected model path is unavailable merely because an Error carries a `code` property. The [repository boundary rule](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/AGENTS.md#architecture-and-cleanup) requires meaningful reasons and typed expected failures. The existing [Go admission mapping](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/go/GoBinaryProvider.ts#L108-L132) distinguishes permission and path-shaped errno values from remaining provider failures. The model operation already has an execution-failure fallback, demonstrated by the uncoded control. This does not require making every host failure automatically retryable. Genuine missing-path, access-denied and format refusals should retain their own semantics. ### Impact and scope P2: callers receive the wrong machine-readable diagnosis and path-correction advice for an otherwise valid selected model. The underlying refusal remains effective, the raw reason text survives, and no incorrect successful Evidence, corruption, descriptor leak or security-boundary crossing was observed. A practical manual workaround is to use the retained host diagnostic rather than reselecting the model. Removing the injected condition restored successful inspection of the unchanged file after every fault. Actual operating-system fault incidence and downstream automated-recovery effects were not measured. Executed scope is `EMFILE` on open and `EIO` on the first header read, for both GGUF and safetensors. Other errno values, mid-stream faults, actual descriptor exhaustion, real device failures, other operating systems and CLI execution were not tested. ### REA version rea-agents 6.7.0; source revision `52415975942a4b9c53a817915b313ccc404f280e`. Fetched `origin/main` at `6d356b84344659085f2b912857b5e27fe0160c9b`: the model reader, filesystem admission and relevant classification/presentation logic remain unchanged. That newer revision was source-inspected, not executed. ### Runtime and platform macOS 27.0.1 arm64; Node.js v26.12.0; Vitest 5.0.3. Repository Node pin: 24.18.0. Controlled filesystem fault injection with real provider and MCP transport; no external provider or model execution. ### Target and artifact context Repository-generated valid GGUF and safetensors declaration fixtures with tensor payload bytes. The selected bytes do not change between successful controls and faulted requests. ### Interface MCP ### Earliest failure stage Setup or provider admission ### Relevant output or evidence Both formats returned the following selected fields for each injected host fault: ```json { "code": "target_unavailable", "category": "unavailable", "retryable": false } ``` For the open fault, `details.reason` was `controlled open failure (EMFILE)`, and the message and remediation action were: ```text Cannot open artifact: controlled open failure (EMFILE). Check that the path exists, is readable, and points to a supported file. ``` For the read fault, the equivalent reason was `controlled read failure (EIO)`, with the same path-checking advice. Neither error had a structured `system_code`. The uncoded read control returned `execution_failure` / `execution_failure`, with `details.provider_id: "model-artifact"` and the original reason under `details.diagnostics`. All eight successful controls completed, and all observed handles closed. ### Typed outcome or recovery expected Preserve an execution/resource failure for non-path host errors, including the selected path and actual diagnostic. Reserve selected-target refusal for errors that establish a selected-path or format problem. Preserve the existing independent permission, integrity, cancellation and cleanup handling. ### Analysis or suspected cause Source-derived explanation of the observed behavior: [regular-file admission](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/filesystem/RegularFile.ts#L81-L95) forwards ordinary open errors, and [the header reader](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/model/ModelHeaderReader.ts#L66-L79) forwards physical read errors. After handling its known typed and permission cases, [ModelArtifactProvider's final coded-error branch](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/model/ModelArtifactProvider.ts#L151-L193) converts every remaining Error with a `code` to `BinaryTargetError`, bypassing its own `ProviderAdapterError` fallback. That [error's public code](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/domain/analysisErrorProjection.ts#L161-L166) and [presentation](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/domain/analysisErrorPresentation.ts#L296-L302) explain the measured path-specific outcome. Classification should distinguish known selected-path errors from other host failures at the model acquisition/read owner. ### Additional context Related but distinct: #5953 concerns firmware source admission, #5970 concerns denied JavaScript recovery snapshots, #5991 concerns lost JavaScript application filesystem diagnostics, and #5997 concerns bound Mach-O dispatch reads. They involve separate producing boundaries and repairs. Existing model reports #5895 / PR #5918 concern safetensors-versus-pickle detection; #5749 / PR #5789 concern unsupported GGUF version diagnostics; #5625 concerns partial observations on cancellation. None changes this model host-error classification. No known-good revision or repaired-behavior run is established.

bsakaguchi closed this issue

2026-10-31 00:42:55 +00:00

bsakaguchi commented

2026-10-31 00:42:56 +00:00

Author

Owner

@amyanger commented on GitHub (Oct 21, 2026):

I'm working on this. Reproduced on main 01788bac8 through the public MCP tool: an injected EMFILE at open returns target_unavailable for both GGUF and safetensors. The plan is to keep target refusal only for errors about the selected path itself (ENOENT, ENOTDIR, ELOOP, ENAMETOOLONG, non-regular files, changed files), following the Go provider's mapping, and let other host errors reach the existing execution_failure fallback with the path and reason preserved. A draft PR will follow.

@amyanger commented on GitHub (Oct 21, 2026): I'm working on this. Reproduced on main 01788bac8 through the public MCP tool: an injected EMFILE at open returns target_unavailable for both GGUF and safetensors. The plan is to keep target refusal only for errors about the selected path itself (ENOENT, ENOTDIR, ELOOP, ENAMETOOLONG, non-regular files, changed files), following the Go provider's mapping, and let other host errors reach the existing execution_failure fallback with the path and reason preserved. A draft PR will follow.

bsakaguchi referenced this issue

2026-10-31 00:43:21 +00:00

[GH-ISSUE #6083] [Bug] JavaScript temporary-file cleanup failure drops the application expansion guard #3718

bsakaguchi referenced this issue

2026-10-31 00:43:22 +00:00

[GH-ISSUE #6084] [Bug] Source-map cleanup remains incomplete after the command successfully retries removal #3722

bsakaguchi referenced this issue

2026-10-31 00:43:22 +00:00

[GH-ISSUE #6085] [Bug] Source-map cleanup lists a codec run after verified process release #3723

bsakaguchi referenced this issue

2026-10-31 00:51:34 +00:00

[GH-ISSUE #7735] [Bug] analyze_javascript_application: application-projection estimate uses up the parent result budget after ~10 files; later files get a 0-byte semantic budget #4627