[GH-ISSUE #6029] [Bug] Web script export misclassifies an existing destination as an execution failure #3677

New issue

Closed

opened 2026-10-31 00:42:58 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:42:58 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 21, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6029

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

export_web_scripts correctly refuses an existing destination, but reports the refusal as artifact_operation_failed / execution_failure rather than invalid caller input. It provides no output_directory field issue or input-contract recovery. remediation.action simply repeats the failure sentence.

Executed both the built export-web-scripts CLI and the registered MCP tool using a synthetic saved browser-scenario capture. An ordinary existing directory and a symlink each produced the incorrect classification through both adapters. The original marker bytes and directory contents were preserved. The same capture exported exact script bytes successfully to an absent destination through the CLI.

Reproduction

From the stated source checkout with locked dependencies and the runtime already built, save this test as tests/boundary/filesystem/scriptExportDestination.test.ts, then run:

REA_EXPORT_REPRO_ROOT="$(mktemp -d)" node node_modules/vitest/vitest.mjs run tests/boundary/filesystem/scriptExportDestination.test.ts --maxWorkers 1

The test below was executed. Its assertions describe the current incorrect result. It uses the repository's saved-capture fixture, the real built CLI subprocess, real filesystem operations, and the production MCP handler through the repository-standard in-memory client/server transport. It does not launch a browser or simulate a provider failure.

import { spawnSync } from 'node:child_process';
import { mkdtemp, mkdir, writeFile, readFile, readdir, symlink, rm } from 'node:fs/promises';
import { join, resolve } from 'node:path';
import { Client, InMemoryTransport } from '@modelcontextprotocol/client';
import { STDIO_DEFAULT_MAX_BUFFER_SIZE } from '@modelcontextprotocol/server';
import { expect, it } from 'vitest';
import { ArtifactResourceScope } from '../../../src/artifacts/ArtifactResourceScope.js';
import { silentLogger } from '../../../src/logger.js';
import { EvidenceMcpServer } from '../../../src/server/EvidenceMcpServer.js';
import { registerWebScriptTool } from '../../../src/server/registerWebScriptTool.js';
import { ToolResultDelivery } from '../../../src/server/toolResult.js';
import { scriptScenarioFixture } from '../../fixtures/webScriptCapture.js';
import { parseMcpToolError } from '../../fixtures/mcpToolError.js';

it('projects existing script-export destinations through CLI and MCP', async () => {
  const root = await mkdtemp(join(process.env.REA_EXPORT_REPRO_ROOT!, 'case-'));
  const resources = new ArtifactResourceScope();
  const server = new EvidenceMcpServer(
    { name: 'script-export-repro', version: '1' }, {}, undefined,
    new ToolResultDelivery(STDIO_DEFAULT_MAX_BUFFER_SIZE),
  );
  registerWebScriptTool(server, { logger: silentLogger, recordEvidence: undefined, artifactResources: resources });
  const client = new Client({ name: 'script-export-repro-client', version: '1' });
  const [clientTransport, serverTransport] = InMemoryTransport.createLinkedPair();
  try {
    await server.connect(serverTransport);
    await client.connect(clientTransport);
    const capturePath = join(root, 'capture.json');
    await writeFile(capturePath, JSON.stringify(scriptScenarioFixture()));
    for (const kind of ['directory', 'symlink'] as const) {
      const retained = join(root, `retained-${kind}`);
      await mkdir(retained);
      await writeFile(join(retained, 'marker.txt'), 'keep');
      const output = kind === 'directory' ? retained : join(root, 'output-link');
      if (kind === 'symlink') await symlink(retained, output, 'dir');
      const cli = spawnSync(process.execPath, [resolve('scripts/rea.mjs'), 'export-web-scripts', capturePath, output, '--json'], {
        env: { ...process.env, REA_LOG_LEVEL: 'silent' }, encoding: 'utf8', timeout: 20000,
      });
      expect(cli.error).toBeUndefined();
      expect(cli.status).toBe(1);
      const cliError = JSON.parse(cli.stdout);
      const mcpError = parseMcpToolError(await client.callTool({
        name: 'export_web_scripts', arguments: { capture_path: capturePath, output_directory: output },
      })).error;
      for (const [adapter, error] of [['CLI', cliError], ['MCP', mcpError]] as const) {
        expect(error).toMatchObject({ code: 'artifact_operation_failed', category: 'execution_failure', retryable: false });
        expect(error.message).toContain('Extraction output root already exists');
        expect(error.remediation.action).toBe(error.message);
        expect(error.recovery).toBeUndefined();
        expect(error.details.issues).toBeUndefined();
        console.log(JSON.stringify({ adapter, kind, code: error.code, category: error.category, retryable: error.retryable, remediationRepeatsMessage: error.remediation.action === error.message, recoveryAbsent: error.recovery === undefined }));
      }
      expect(await readFile(join(retained, 'marker.txt'), 'utf8')).toBe('keep');
      expect(await readdir(retained)).toEqual(['marker.txt']);
    }
    const output = join(root, 'absent-output');
    const cli = spawnSync(process.execPath, [resolve('scripts/rea.mjs'), 'export-web-scripts', capturePath, output, '--json'], {
      env: { ...process.env, REA_LOG_LEVEL: 'silent' }, encoding: 'utf8', timeout: 20000,
    });
    expect(cli.error).toBeUndefined();
    expect(cli.status).toBe(0);
    const result = JSON.parse(cli.stdout).normalized_result;
    const script = result.scripts.find((item: any) => item.content.state === 'exported');
    expect(await readFile(join(result.analysis_input.input_path, script.content.relative_path), 'utf8')).toBe("export const marker = 'fixture-source';\n");
    console.log(JSON.stringify({ adapter: 'CLI', kind: 'absent-directory-control', exit: cli.status, scriptBytesMatch: true }));
  } finally {
    await client.close();
    await server.close();
    await resources.close();
    await rm(root, { recursive: true, force: true });
  }
}, 60000);

Expected behavior

Keep the safe refusal and existing contents, but identify output_directory as invalid caller input: invalid_request / invalid_input, a field issue explaining that the selected destination already exists, and guidance to choose a new absent directory. The standard input-contract recovery should name export_web_scripts.

The export guide requires an absent destination, and the input contract advertises that precondition. Existing Evidence output classification treats an already-existing caller-selected path as invalid input. The environment bundle writer similarly converts destination-admission refusals to an input issue while retaining separate semantics for later write failures.

Impact and scope

P2: callers that branch on typed failures cannot distinguish a rejected output argument from a failed artifact operation, and receive no machine-readable field correction or recovery step. This reproduced for both tested destination kinds through both adapters. Choosing an absent destination is a verified workaround. Existing output was preserved; no data loss or security boundary crossing was observed. Natural incidence and behavior on other operating systems were not measured.

REA version

rea-agents 6.7.0; source revision 52415975942a4b9c53a817915b313ccc404f280e.

Runtime and platform

macOS 27.0.1 arm64; Node.js v26.12.0; locked Vitest 5.0.3. The repository's exact Node.js 24.18.0 pin was not exercised.

Target and artifact context

Synthetic saved capture_browser_scenario normalized JSON from tests/fixtures/webScriptCapture.ts, containing one retained JavaScript response. Local existing directory and symlink destinations containing a marker file; an absent-directory success control. No captured code was executed or fetched.

Interface

CLI

Earliest failure stage

Request parsing or validation

Relevant output or evidence

The single focused test passed, confirming the current incorrect diagnostic for four combinations. Selected actual output:

{"adapter":"CLI","kind":"directory","code":"artifact_operation_failed","category":"execution_failure","retryable":false,"remediationRepeatsMessage":true,"recoveryAbsent":true}
{"adapter":"MCP","kind":"directory","code":"artifact_operation_failed","category":"execution_failure","retryable":false,"remediationRepeatsMessage":true,"recoveryAbsent":true}
{"adapter":"CLI","kind":"symlink","code":"artifact_operation_failed","category":"execution_failure","retryable":false,"remediationRepeatsMessage":true,"recoveryAbsent":true}
{"adapter":"MCP","kind":"symlink","code":"artifact_operation_failed","category":"execution_failure","retryable":false,"remediationRepeatsMessage":true,"recoveryAbsent":true}
{"adapter":"CLI","kind":"absent-directory-control","exit":0,"scriptBytesMatch":true}

Each refusal exited 1 in the CLI. Each message contained Extraction output root already exists; remediation.action equaled that complete message, recovery was absent, and no field issues were returned. After both adapters ran, each refused destination still contained only the unchanged marker file. MCP was exercised through an in-memory transport, not a stdio server process.

Typed outcome or recovery expected

code: invalid_request, category: invalid_input, an issue on ["output_directory"], actionable correction guidance, and recovery: {"kind":"inspect_input_contract","operation":"export_web_scripts"}. Preserve the destination and the diagnostic reason. Genuine failures after destination admission must retain their artifact/publication failure semantics.

Analysis or suspected cause

Source-derived: SafeOutputTree.create throws ArtifactReaderFailure("path") when the requested root already exists, before an output tree is acquired. The web export mapper wraps that refusal in WebScriptExportError. The artifact code mapping and category mapping then report an operation failure. The remediation fallback repeats its message, and input recovery applies only to the input error type.

A repair should preserve admission origin at the producing boundary. A generic conversion of every path failure would also catch genuine later filesystem failures.

Additional context

Fetched origin/main at 6d356b84344659085f2b912857b5e27fe0160c9b. Source comparison found the destination-admission, web mapper, category and recovery logic unchanged. An unrelated captured-output projection refactor does not change this outcome; newer main was not executed.

Related #5956 concerns cancellation plus failed rollback, after output acquisition. #5746, #5899 and #5900 concern firmware destination classification. PR #5902 adds a shared admission-failure subtype but converts it to input errors only in the firmware mapper; the unchanged web mapper still accepts that ArtifactReaderFailure subclass and wraps it as an artifact failure. This conclusion comes from its diff, not an executed PR checkout. PR #1240 fixed capture_path selection and explicitly left output publication classification unchanged. PR #5672 addresses source-stream cleanup and leaves destination admission unchanged. PR #5581 separates environment-bundle admission from later write failures. None supplies the web destination conversion described here. No regression range or post-fix result is established.

Originally created by @coygeek on GitHub (Oct 21, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6029 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior `export_web_scripts` correctly refuses an existing destination, but reports the refusal as `artifact_operation_failed` / `execution_failure` rather than invalid caller input. It provides no `output_directory` field issue or input-contract recovery. `remediation.action` simply repeats the failure sentence. Executed both the built `export-web-scripts` CLI and the registered MCP tool using a synthetic saved browser-scenario capture. An ordinary existing directory and a symlink each produced the incorrect classification through both adapters. The original marker bytes and directory contents were preserved. The same capture exported exact script bytes successfully to an absent destination through the CLI. ### Reproduction From the stated source checkout with locked dependencies and the runtime already built, save this test as `tests/boundary/filesystem/scriptExportDestination.test.ts`, then run: ```sh REA_EXPORT_REPRO_ROOT="$(mktemp -d)" node node_modules/vitest/vitest.mjs run tests/boundary/filesystem/scriptExportDestination.test.ts --maxWorkers 1 ``` The test below was executed. Its assertions describe the current incorrect result. It uses the repository's saved-capture fixture, the real built CLI subprocess, real filesystem operations, and the production MCP handler through the repository-standard in-memory client/server transport. It does not launch a browser or simulate a provider failure. ```ts import { spawnSync } from 'node:child_process'; import { mkdtemp, mkdir, writeFile, readFile, readdir, symlink, rm } from 'node:fs/promises'; import { join, resolve } from 'node:path'; import { Client, InMemoryTransport } from '@modelcontextprotocol/client'; import { STDIO_DEFAULT_MAX_BUFFER_SIZE } from '@modelcontextprotocol/server'; import { expect, it } from 'vitest'; import { ArtifactResourceScope } from '../../../src/artifacts/ArtifactResourceScope.js'; import { silentLogger } from '../../../src/logger.js'; import { EvidenceMcpServer } from '../../../src/server/EvidenceMcpServer.js'; import { registerWebScriptTool } from '../../../src/server/registerWebScriptTool.js'; import { ToolResultDelivery } from '../../../src/server/toolResult.js'; import { scriptScenarioFixture } from '../../fixtures/webScriptCapture.js'; import { parseMcpToolError } from '../../fixtures/mcpToolError.js'; it('projects existing script-export destinations through CLI and MCP', async () => { const root = await mkdtemp(join(process.env.REA_EXPORT_REPRO_ROOT!, 'case-')); const resources = new ArtifactResourceScope(); const server = new EvidenceMcpServer( { name: 'script-export-repro', version: '1' }, {}, undefined, new ToolResultDelivery(STDIO_DEFAULT_MAX_BUFFER_SIZE), ); registerWebScriptTool(server, { logger: silentLogger, recordEvidence: undefined, artifactResources: resources }); const client = new Client({ name: 'script-export-repro-client', version: '1' }); const [clientTransport, serverTransport] = InMemoryTransport.createLinkedPair(); try { await server.connect(serverTransport); await client.connect(clientTransport); const capturePath = join(root, 'capture.json'); await writeFile(capturePath, JSON.stringify(scriptScenarioFixture())); for (const kind of ['directory', 'symlink'] as const) { const retained = join(root, `retained-${kind}`); await mkdir(retained); await writeFile(join(retained, 'marker.txt'), 'keep'); const output = kind === 'directory' ? retained : join(root, 'output-link'); if (kind === 'symlink') await symlink(retained, output, 'dir'); const cli = spawnSync(process.execPath, [resolve('scripts/rea.mjs'), 'export-web-scripts', capturePath, output, '--json'], { env: { ...process.env, REA_LOG_LEVEL: 'silent' }, encoding: 'utf8', timeout: 20000, }); expect(cli.error).toBeUndefined(); expect(cli.status).toBe(1); const cliError = JSON.parse(cli.stdout); const mcpError = parseMcpToolError(await client.callTool({ name: 'export_web_scripts', arguments: { capture_path: capturePath, output_directory: output }, })).error; for (const [adapter, error] of [['CLI', cliError], ['MCP', mcpError]] as const) { expect(error).toMatchObject({ code: 'artifact_operation_failed', category: 'execution_failure', retryable: false }); expect(error.message).toContain('Extraction output root already exists'); expect(error.remediation.action).toBe(error.message); expect(error.recovery).toBeUndefined(); expect(error.details.issues).toBeUndefined(); console.log(JSON.stringify({ adapter, kind, code: error.code, category: error.category, retryable: error.retryable, remediationRepeatsMessage: error.remediation.action === error.message, recoveryAbsent: error.recovery === undefined })); } expect(await readFile(join(retained, 'marker.txt'), 'utf8')).toBe('keep'); expect(await readdir(retained)).toEqual(['marker.txt']); } const output = join(root, 'absent-output'); const cli = spawnSync(process.execPath, [resolve('scripts/rea.mjs'), 'export-web-scripts', capturePath, output, '--json'], { env: { ...process.env, REA_LOG_LEVEL: 'silent' }, encoding: 'utf8', timeout: 20000, }); expect(cli.error).toBeUndefined(); expect(cli.status).toBe(0); const result = JSON.parse(cli.stdout).normalized_result; const script = result.scripts.find((item: any) => item.content.state === 'exported'); expect(await readFile(join(result.analysis_input.input_path, script.content.relative_path), 'utf8')).toBe("export const marker = 'fixture-source';\n"); console.log(JSON.stringify({ adapter: 'CLI', kind: 'absent-directory-control', exit: cli.status, scriptBytesMatch: true })); } finally { await client.close(); await server.close(); await resources.close(); await rm(root, { recursive: true, force: true }); } }, 60000); ``` ### Expected behavior Keep the safe refusal and existing contents, but identify `output_directory` as invalid caller input: `invalid_request` / `invalid_input`, a field issue explaining that the selected destination already exists, and guidance to choose a new absent directory. The standard input-contract recovery should name `export_web_scripts`. The [export guide](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/docs/website-script-export.md#L30-L34) requires an absent destination, and the [input contract](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/domain/webScriptExport.ts#L8-L15) advertises that precondition. Existing [Evidence output classification](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/domain/analysisErrorProjection.ts#L206-L212) treats an already-existing caller-selected path as invalid input. The [environment bundle writer](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/application/environment/environmentBundleWrite.ts#L128-L147) similarly converts destination-admission refusals to an input issue while retaining separate semantics for later write failures. ### Impact and scope P2: callers that branch on typed failures cannot distinguish a rejected output argument from a failed artifact operation, and receive no machine-readable field correction or recovery step. This reproduced for both tested destination kinds through both adapters. Choosing an absent destination is a verified workaround. Existing output was preserved; no data loss or security boundary crossing was observed. Natural incidence and behavior on other operating systems were not measured. ### REA version rea-agents 6.7.0; source revision `52415975942a4b9c53a817915b313ccc404f280e`. ### Runtime and platform macOS 27.0.1 arm64; Node.js v26.12.0; locked Vitest 5.0.3. The repository's exact Node.js 24.18.0 pin was not exercised. ### Target and artifact context Synthetic saved `capture_browser_scenario` normalized JSON from `tests/fixtures/webScriptCapture.ts`, containing one retained JavaScript response. Local existing directory and symlink destinations containing a marker file; an absent-directory success control. No captured code was executed or fetched. ### Interface CLI ### Earliest failure stage Request parsing or validation ### Relevant output or evidence The single focused test passed, confirming the current incorrect diagnostic for four combinations. Selected actual output: ```json {"adapter":"CLI","kind":"directory","code":"artifact_operation_failed","category":"execution_failure","retryable":false,"remediationRepeatsMessage":true,"recoveryAbsent":true} {"adapter":"MCP","kind":"directory","code":"artifact_operation_failed","category":"execution_failure","retryable":false,"remediationRepeatsMessage":true,"recoveryAbsent":true} {"adapter":"CLI","kind":"symlink","code":"artifact_operation_failed","category":"execution_failure","retryable":false,"remediationRepeatsMessage":true,"recoveryAbsent":true} {"adapter":"MCP","kind":"symlink","code":"artifact_operation_failed","category":"execution_failure","retryable":false,"remediationRepeatsMessage":true,"recoveryAbsent":true} {"adapter":"CLI","kind":"absent-directory-control","exit":0,"scriptBytesMatch":true} ``` Each refusal exited 1 in the CLI. Each message contained `Extraction output root already exists`; `remediation.action` equaled that complete message, `recovery` was absent, and no field issues were returned. After both adapters ran, each refused destination still contained only the unchanged marker file. MCP was exercised through an in-memory transport, not a stdio server process. ### Typed outcome or recovery expected `code: invalid_request`, `category: invalid_input`, an issue on `["output_directory"]`, actionable correction guidance, and `recovery: {"kind":"inspect_input_contract","operation":"export_web_scripts"}`. Preserve the destination and the diagnostic reason. Genuine failures after destination admission must retain their artifact/publication failure semantics. ### Analysis or suspected cause Source-derived: [SafeOutputTree.create](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/artifacts/SafeOutputTree.ts#L118-L128) throws `ArtifactReaderFailure("path")` when the requested root already exists, before an output tree is acquired. [The web export mapper](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/application/WebScriptExportService.ts#L191-L197) wraps that refusal in `WebScriptExportError`. The [artifact code mapping](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/domain/analysisErrorProjection.ts#L196-L204) and [category mapping](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/domain/analysisErrorPresentation.ts#L215-L223) then report an operation failure. The [remediation fallback](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/domain/analysisErrorPresentation.ts#L133-L142) repeats its message, and [input recovery](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/domain/analysisErrorProjection.ts#L126-L140) applies only to the input error type. A repair should preserve admission origin at the producing boundary. A generic conversion of every `path` failure would also catch genuine later filesystem failures. ### Additional context Fetched `origin/main` at `6d356b84344659085f2b912857b5e27fe0160c9b`. Source comparison found the destination-admission, web mapper, category and recovery logic unchanged. An unrelated captured-output projection refactor does not change this outcome; newer main was not executed. Related #5956 concerns cancellation plus failed rollback, after output acquisition. #5746, #5899 and #5900 concern firmware destination classification. PR #5902 adds a shared admission-failure subtype but converts it to input errors only in the firmware mapper; the unchanged web mapper still accepts that ArtifactReaderFailure subclass and wraps it as an artifact failure. This conclusion comes from its diff, not an executed PR checkout. PR #1240 fixed `capture_path` selection and explicitly left output publication classification unchanged. PR #5672 addresses source-stream cleanup and leaves destination admission unchanged. PR #5581 separates environment-bundle admission from later write failures. None supplies the web destination conversion described here. No regression range or post-fix result is established.

bsakaguchi closed this issue

2026-10-31 00:42:59 +00:00

bsakaguchi referenced this issue

2026-11-01 21:10:58 +00:00

[GH-ISSUE #3796] [Tracker] Open feature requests by area (October 2026) #1471