[GH-ISSUE #5977] [Bug] inspect_asset_catalog reports unreadable assetutil output as an invalid artifact #3644

New issue

Closed

opened 2026-10-31 00:42:40 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:42:40 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 21, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/5977

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

A successful assetutil invocation with malformed JSON or an unexpected record structure is reported as an invalid or changed artifact. Supplied successful stdout not json, {}, and [null] each produced artifact_operation_failed, reason format, and “Get a fresh copy and try again.” The record-shape parser’s more precise AnalysisOutputError reason is lost from the canonical result. Supplied [] succeeds on the same fixture.

Reproduction

Run the following focused test from a macOS source checkout with locked dependencies installed. Save it under src/artifacts/apple/assetCatalogFailureClassification.test.ts, then run:

node node_modules/vitest/vitest.mjs run src/artifacts/apple/assetCatalogFailureClassification.test.ts --maxWorkers=1

This test was executed with the same content except for the temporary fixture directory location. It exercises the production artifact provider and canonical error projection. The provider boundary is simulated: no native catalog validity, real assetutil output failure, or CLI/MCP transport result is asserted. The deadline case uses a real Node child and real execFileOutput timeout handling, shortening only the deadline to 100 ms after checking the production options specify 60,000 ms and no stopSignal. The other cases supply successful subprocess stdout. All five cases passed, including the valid [] control.

import { mkdtemp, mkdir, writeFile, rm } from "node:fs/promises";
import { join } from "node:path";
import { tmpdir } from "node:os";
import { describe, it, expect, vi } from "vitest";
import { ArtifactProvider } from "../ArtifactProvider.js";
import { projectAnalysisError } from "../../domain/analysisErrorProjection.js";
import { execFileOutput } from "../../process/ExecFileOutput.js";

vi.mock("../../process/ExecFileOutput.js", async (importOriginal) => {
  const actual = await importOriginal<typeof import("../../process/ExecFileOutput.js")>();
  return { ...actual, execFileOutput: vi.fn(actual.execFileOutput) };
});
const actual = await vi.importActual<typeof import("../../process/ExecFileOutput.js")>("../../process/ExecFileOutput.js");
const scratch = tmpdir();
async function run() {
  const bundle = await mkdtemp(join(scratch, "fixture-"));
  await mkdir(join(bundle, "Contents", "Resources"), { recursive: true });
  await writeFile(join(bundle, "Contents", "Resources", "Assets.car"), "inert fixture");
  const client = new ArtifactProvider({}, "darwin").createBundleResourceClient({
    callerPath: bundle, canonicalPath: bundle,
    infoPlistPath: join(bundle, "Contents", "Info.plist"),
    layout: "contents", executableSha256: null,
  });
  try { return await client.execute("inspect_asset_catalog", {}); }
  finally { await client.close(); await rm(bundle, { recursive: true, force: true }); }
}
describe("assetutil failure classification", () => {
  it("observes the real non-stoppable subprocess timeout shape", async () => {
    let childCode: unknown = "not observed";
    vi.mocked(execFileOutput).mockImplementationOnce(async (command, args, options) => {
      expect(command).toBe("/usr/bin/assetutil");
      expect(args[0]).toBe("--info");
      expect(options?.timeout).toBe(60_000);
      expect(options?.stopSignal).toBeUndefined();
      try {
        return await actual.execFileOutput(process.execPath, ["-e", "setInterval(() => {}, 1000)"], { ...options, timeout: 100 });
      } catch (cause) {
        childCode = Reflect.get(cause as object, "code");
        throw cause;
      }
    });
    const result = await run();
    expect(result.ok).toBe(false);
    expect(childCode).toBeNull();
    if (result.ok) throw new Error("expected failure");
    const projected = projectAnalysisError(result.error);
    console.log("actual-timeout", JSON.stringify(projected));
    expect(projected).toMatchObject({code: "artifact_operation_failed", category: "execution_failure", retryable: false, details: { reason: "io", detail: "assetutil failed (null)" }});
  });
  it.each(["not json", "{}", "[null]"])("classifies supplied output %s", async (stdout) => {
    vi.mocked(execFileOutput).mockResolvedValueOnce({ stdout, stderr: "", stdoutBytes: Buffer.byteLength(stdout), stderrBytes: 0 });
    const result = await run();
    expect(result.ok).toBe(false);
    if (result.ok) throw new Error("expected failure");
    const projected = projectAnalysisError(result.error);
    console.log("supplied-output", JSON.stringify({stdout, projected}));
    expect(projected).toMatchObject({code: "artifact_operation_failed", category: "execution_failure", retryable: false, details: {reason: "format"}});
  });
  it("accepts supplied valid empty output", async () => {
    vi.mocked(execFileOutput).mockResolvedValueOnce({stdout: "[]", stderr: "", stdoutBytes: 2, stderrBytes: 0});
    const result = await run();
    expect(result.ok).toBe(true);
  });
});

Expected behavior

Keep provider stdout failures distinct from invalid artifact bytes. The utility-output parser deliberately throws AnalysisOutputError for non-array output and invalid records; its existing test asserts that classification. The canonical error mapping calls that unreadable_output. Preserve the structural or JSON parsing reason and offer recovery concerning the utility output rather than asserting that the artifact changed.

Impact and scope

P2: provider-output incompatibility produces incorrect artifact-replacement advice and loses useful structural diagnostics. This affects callers that interpret canonical error codes or follow recovery instructions. Each of three supplied malformed stdout values reproduced the issue once; the [] control succeeded. The inspected files were unchanged. No real assetutil version producing these successful malformed responses was identified, so native occurrence frequency is unknown. Both actual and expected classifications are non-retryable; retryability alone is not the defect.

REA version

rea-agents 6.7.0; source revision 52415975942a4b9c53a817915b313ccc404f280e. Latest npm version checked: 6.7.0. Fetched origin/main at 25e1a24a3ed3671a0148757555f18d16c29d936a leaves the producing paths unchanged; that newer revision was compared as source, not executed.

Runtime and platform

macOS 27.0.1 (26A434), arm64; Node.js v26.12.0; locked Vitest 5.0.3. No deep-analysis engine was involved.

Target and artifact context

Synthetic macOS-layout resource directory with one regular Contents/Resources/Assets.car holding inert fixture bytes. The app resources are inspected as data; no target code runs. Successful utility output is supplied at the subprocess boundary; this does not establish native catalog validity.

Interface

Other

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

Observed projection for supplied {} and [null]:

{"code":"artifact_operation_failed","category":"execution_failure","message":"Artifact is invalid or has changed. Get a fresh copy and try again.","retryable":false,"remediation":{"action":"Artifact is invalid or has changed. Get a fresh copy and try again."},"details":{"operation":"inspect_asset_catalog","reason":"format","detail":"assetutil returned an unsupported record for Contents/Resources/Assets.car"}}

not json produces the same classification with details.detail: "assetutil returned invalid JSON for Contents/Resources/Assets.car". All outputs are from supplied successful utility stdout, not native assetutil invocations. The source identity and digest checks ran against unchanged fixture bytes; that establishes stability, not native catalog validity.

Typed outcome or recovery expected

Use/preserve AnalysisOutputError through the provider boundary, with code: unreadable_output and the JSON or record-index reason in canonical details. For example, {} should retain “assetutil output must be a JSON array” and [null] should retain “assetutil record 0 is not a JSON object.” Keep malformed target-byte errors separate.

Analysis or suspected cause

Source-derived: inspectAssetCatalogEntry verifies the file around the command and then converts both JSON syntax failure and the parser’s AnalysisOutputError to ArtifactReaderFailure("format"). The artifact translation boundary turns this into ArtifactOperationError; it also would discard AnalysisOutputError if only the local catch were removed. Both syntax and shape branches need provider-output provenance preserved.

Additional context

The runtime result comes from the production artifact-provider boundary, not a public CLI/MCP invocation. The CLI registration dispatches this operation through the same provider; that reachability is source-derived. No real-app prevalence, known-good revision, or software fix was established. Related #5698 covers a nonzero assetutil exit for damaged catalog bytes; #5111 concerns empty-inventory pagination; #5887 concerns skipped-catalog join coverage; #5635 and #2176 concern resource-key recognition; #3914/merged PR #4094 concern filename selection; PR #3285 concerns bundle-walk concurrency. Those fixes have different restoration signals.

Originally created by @coygeek on GitHub (Oct 21, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/5977 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior A successful assetutil invocation with malformed JSON or an unexpected record structure is reported as an invalid or changed artifact. Supplied successful stdout `not json`, `{}`, and `[null]` each produced `artifact_operation_failed`, reason `format`, and “Get a fresh copy and try again.” The record-shape parser’s more precise AnalysisOutputError reason is lost from the canonical result. Supplied `[]` succeeds on the same fixture. ### Reproduction Run the following focused test from a macOS source checkout with locked dependencies installed. Save it under `src/artifacts/apple/assetCatalogFailureClassification.test.ts`, then run: ```sh node node_modules/vitest/vitest.mjs run src/artifacts/apple/assetCatalogFailureClassification.test.ts --maxWorkers=1 ``` This test was executed with the same content except for the temporary fixture directory location. It exercises the production artifact provider and canonical error projection. The provider boundary is simulated: no native catalog validity, real assetutil output failure, or CLI/MCP transport result is asserted. The deadline case uses a real Node child and real execFileOutput timeout handling, shortening only the deadline to 100 ms after checking the production options specify 60,000 ms and no stopSignal. The other cases supply successful subprocess stdout. All five cases passed, including the valid `[]` control. ```ts import { mkdtemp, mkdir, writeFile, rm } from "node:fs/promises"; import { join } from "node:path"; import { tmpdir } from "node:os"; import { describe, it, expect, vi } from "vitest"; import { ArtifactProvider } from "../ArtifactProvider.js"; import { projectAnalysisError } from "../../domain/analysisErrorProjection.js"; import { execFileOutput } from "../../process/ExecFileOutput.js"; vi.mock("../../process/ExecFileOutput.js", async (importOriginal) => { const actual = await importOriginal<typeof import("../../process/ExecFileOutput.js")>(); return { ...actual, execFileOutput: vi.fn(actual.execFileOutput) }; }); const actual = await vi.importActual<typeof import("../../process/ExecFileOutput.js")>("../../process/ExecFileOutput.js"); const scratch = tmpdir(); async function run() { const bundle = await mkdtemp(join(scratch, "fixture-")); await mkdir(join(bundle, "Contents", "Resources"), { recursive: true }); await writeFile(join(bundle, "Contents", "Resources", "Assets.car"), "inert fixture"); const client = new ArtifactProvider({}, "darwin").createBundleResourceClient({ callerPath: bundle, canonicalPath: bundle, infoPlistPath: join(bundle, "Contents", "Info.plist"), layout: "contents", executableSha256: null, }); try { return await client.execute("inspect_asset_catalog", {}); } finally { await client.close(); await rm(bundle, { recursive: true, force: true }); } } describe("assetutil failure classification", () => { it("observes the real non-stoppable subprocess timeout shape", async () => { let childCode: unknown = "not observed"; vi.mocked(execFileOutput).mockImplementationOnce(async (command, args, options) => { expect(command).toBe("/usr/bin/assetutil"); expect(args[0]).toBe("--info"); expect(options?.timeout).toBe(60_000); expect(options?.stopSignal).toBeUndefined(); try { return await actual.execFileOutput(process.execPath, ["-e", "setInterval(() => {}, 1000)"], { ...options, timeout: 100 }); } catch (cause) { childCode = Reflect.get(cause as object, "code"); throw cause; } }); const result = await run(); expect(result.ok).toBe(false); expect(childCode).toBeNull(); if (result.ok) throw new Error("expected failure"); const projected = projectAnalysisError(result.error); console.log("actual-timeout", JSON.stringify(projected)); expect(projected).toMatchObject({code: "artifact_operation_failed", category: "execution_failure", retryable: false, details: { reason: "io", detail: "assetutil failed (null)" }}); }); it.each(["not json", "{}", "[null]"])("classifies supplied output %s", async (stdout) => { vi.mocked(execFileOutput).mockResolvedValueOnce({ stdout, stderr: "", stdoutBytes: Buffer.byteLength(stdout), stderrBytes: 0 }); const result = await run(); expect(result.ok).toBe(false); if (result.ok) throw new Error("expected failure"); const projected = projectAnalysisError(result.error); console.log("supplied-output", JSON.stringify({stdout, projected})); expect(projected).toMatchObject({code: "artifact_operation_failed", category: "execution_failure", retryable: false, details: {reason: "format"}}); }); it("accepts supplied valid empty output", async () => { vi.mocked(execFileOutput).mockResolvedValueOnce({stdout: "[]", stderr: "", stdoutBytes: 2, stderrBytes: 0}); const result = await run(); expect(result.ok).toBe(true); }); }); ``` ### Expected behavior Keep provider stdout failures distinct from invalid artifact bytes. [The utility-output parser](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/domain/apple/appleAssetCatalog.ts#L87-L108) deliberately throws AnalysisOutputError for non-array output and invalid records; [its existing test](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/domain/apple/appleAssetCatalog.test.ts#L115-L133) asserts that classification. [The canonical error mapping](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/domain/analysisErrorProjection.ts#L242-L254) calls that `unreadable_output`. Preserve the structural or JSON parsing reason and offer recovery concerning the utility output rather than asserting that the artifact changed. ### Impact and scope P2: provider-output incompatibility produces incorrect artifact-replacement advice and loses useful structural diagnostics. This affects callers that interpret canonical error codes or follow recovery instructions. Each of three supplied malformed stdout values reproduced the issue once; the `[]` control succeeded. The inspected files were unchanged. No real assetutil version producing these successful malformed responses was identified, so native occurrence frequency is unknown. Both actual and expected classifications are non-retryable; retryability alone is not the defect. ### REA version `rea-agents` 6.7.0; source revision `52415975942a4b9c53a817915b313ccc404f280e`. Latest npm version checked: 6.7.0. Fetched `origin/main` at `25e1a24a3ed3671a0148757555f18d16c29d936a` leaves the producing paths unchanged; that newer revision was compared as source, not executed. ### Runtime and platform macOS 27.0.1 (26A434), arm64; Node.js v26.12.0; locked Vitest 5.0.3. No deep-analysis engine was involved. ### Target and artifact context Synthetic macOS-layout resource directory with one regular `Contents/Resources/Assets.car` holding inert fixture bytes. The app resources are inspected as data; no target code runs. Successful utility output is supplied at the subprocess boundary; this does not establish native catalog validity. ### Interface Other ### Earliest failure stage Native analysis or reconstruction ### Relevant output or evidence Observed projection for supplied `{}` and `[null]`: ```json {"code":"artifact_operation_failed","category":"execution_failure","message":"Artifact is invalid or has changed. Get a fresh copy and try again.","retryable":false,"remediation":{"action":"Artifact is invalid or has changed. Get a fresh copy and try again."},"details":{"operation":"inspect_asset_catalog","reason":"format","detail":"assetutil returned an unsupported record for Contents/Resources/Assets.car"}} ``` `not json` produces the same classification with `details.detail: "assetutil returned invalid JSON for Contents/Resources/Assets.car"`. All outputs are from supplied successful utility stdout, not native assetutil invocations. The source identity and digest checks ran against unchanged fixture bytes; that establishes stability, not native catalog validity. ### Typed outcome or recovery expected Use/preserve `AnalysisOutputError` through the provider boundary, with `code: unreadable_output` and the JSON or record-index reason in canonical details. For example, `{}` should retain “assetutil output must be a JSON array” and `[null]` should retain “assetutil record 0 is not a JSON object.” Keep malformed target-byte errors separate. ### Analysis or suspected cause Source-derived: [inspectAssetCatalogEntry](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/artifacts/apple/AppleAssetCatalogAnalysis.ts#L178-L224) verifies the file around the command and then converts both JSON syntax failure and the parser’s AnalysisOutputError to ArtifactReaderFailure("format"). [The artifact translation boundary](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/artifacts/ArtifactOperationFailure.ts#L16-L49) turns this into ArtifactOperationError; it also would discard AnalysisOutputError if only the local catch were removed. Both syntax and shape branches need provider-output provenance preserved. ### Additional context The runtime result comes from the production artifact-provider boundary, not a public CLI/MCP invocation. The CLI registration dispatches this operation through the same provider; that reachability is source-derived. No real-app prevalence, known-good revision, or software fix was established. Related #5698 covers a nonzero assetutil exit for damaged catalog bytes; #5111 concerns empty-inventory pagination; #5887 concerns skipped-catalog join coverage; #5635 and #2176 concern resource-key recognition; #3914/merged PR #4094 concern filename selection; PR #3285 concerns bundle-walk concurrency. Those fixes have different restoration signals.

bsakaguchi closed this issue

2026-10-31 00:42:40 +00:00

bsakaguchi referenced this issue

2026-11-01 21:10:58 +00:00

[GH-ISSUE #3796] [Tracker] Open feature requests by area (October 2026) #1471