morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #5948] [Bug] Native UI helper timeouts return execution_failure at the hidden 30-second limit #3616

New issue

Closed

opened 2026-10-31 00:42:25 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-31 00:42:25 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 21, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/5948

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Process capture or runtime observation

Observed behavior

A production native UI provider call stops its helper subprocess after 30000 ms and returns execution_failure, retryable: false, with generic rea doctor recovery and Xcode/permissions advice. It does this while the operation deadline signal is still active. The scenario schema documents a 180-second operation deadline; the shorter helper limit is not advertised there.

This was executed through NativeMacOSProvider.createClient(...).execute("observe_native_ui", ...) with a substituted compiler/helper launch, using the real runtime, actual owned subprocess, real 30-second timer, and error projection. No slow real application or live CLI/MCP transport was exercised.

Reproduction

From the prepared source checkout with locked dependencies, save the following test as src/native/nativeUiHelperTimeoutReproduction.test.ts and run node node_modules/vitest/vitest.mjs run src/native/nativeUiHelperTimeoutReproduction.test.ts --maxWorkers 1. The test asserts the faulty outcome and therefore passes when this bug is present.

The compilation seam returns immediately; the helper launch is replaced with an owned Node process scheduled to finish after 31 seconds. All timeout/signal options from NativeUiHelperRuntime pass unchanged to the real process runner.

import { expect, it, vi } from 'vitest';
import { NativeMacOSProvider } from './NativeMacOSProvider.js';
import { NativeFixtureRunner, nativeMachoTarget } from '../../tests/fixtures/nativeCommands.js';
import { projectAnalysisError } from '../domain/analysisErrorProjection.js';

vi.mock('../process/ExecFileOutput.js', async (importOriginal) => {
  const actual = await importOriginal<typeof import('../process/ExecFileOutput.js')>();
  return {
    ...actual,
    execFileOutput: async (command: string, args: readonly string[], options: any) => {
      if (command === '/usr/bin/xcrun') return { stdout: '', stderr: '' };
      if (command.endsWith('/observer')) {
        expect(options.timeout).toBe(30_000);
        expect(options.signal.aborted).toBe(false);
        try {
          return await actual.execFileOutput(process.execPath,
            ['-e', 'setTimeout(() => console.log(JSON.stringify({ok:true})), 31000)'],
            options);
        } catch (cause) {
          expect(options.signal.aborted).toBe(false);
          throw cause;
        }
      }
      return actual.execFileOutput(command, args, options);
    },
  };
});

it('runs the real native UI runtime timeout through the public provider client', async () => {
  const client = new NativeMacOSProvider({}, new NativeFixtureRunner(), 'darwin')
    .createClient(nativeMachoTarget('/fixture/App'));
  const started = performance.now();
  try {
    const result = await client.execute('observe_native_ui', {
      pid: 123, window_id: 456, screenshot: false, accessibility: true,
    });
    const elapsed = Math.round(performance.now() - started);
    expect(result.ok).toBe(false);
    if (result.ok) throw new Error('Expected timeout');
    expect(result.error._tag).toBe('ProviderAdapterError');
    expect(Reflect.get(result.error.cause!, 'code')).toBe('ETIMEDOUT');
    const projected = projectAnalysisError(result.error);
    expect(projected.code).toBe('execution_failure');
    expect(elapsed).toBeGreaterThanOrEqual(29_000);
    expect(elapsed).toBeLessThan(90_000);
    console.log(JSON.stringify({elapsed_ms: elapsed, tag: result.error._tag,
      cause_code: Reflect.get(result.error.cause!, 'code'), projection: projected}));
  } finally {
    expect((await client.close()).ok).toBe(true);
  }
}, 90_000);

Two executions reproduced the same result. The final execution additionally asserted that the operation signal had not aborted when the subprocess timeout arrived. Delete the temporary test after use.

Expected behavior

A reached server/provider deadline should produce provider_timeout with its actual budget and timeout-specific recovery. Preserve caller cancellation and physical helper failures as distinct outcomes. Either let the documented operation deadline own helper execution, or explicitly describe and type any shorter supported invocation deadline.

The scenario wait schema names the 180-second operation deadline; the native UI guide describes it, and the recovery contract identifies provider_timeout as REA's server/provider deadline. This report does not assume every helper call is guaranteed the whole operation budget; the definite defect is the reached helper timeout being classified as an execution failure.

Impact and scope

P2: a bounded native UI subprocess timeout aborts capture with the wrong structured code and recovery. The real 30-second stop and its misclassification reproduced twice at the controlled provider boundary. Source inspection shows the same helper runtime serves capture_native_ui_scenario, but that operation was not executed here. Real-world frequency and whether any specific accessibility tree or screenshot exceeds the limit are unknown. No target data mutation or security-boundary crossing was exercised. No complete-capture workaround was demonstrated.

REA version

rea-agents 6.7.0; source 52415975942a4b9c53a817915b313ccc404f280e.

Runtime and platform

macOS 27.0.1 arm64; Node.js v26.12.0; repository-pinned Vitest 5.0.3. Simulated native compiler/helper launch with a real Node subprocess and real process timeout. No Swift compiler or GUI provider qualification.

Target and artifact context

Synthetic Mach-O target metadata, selected PID/window values, and an owned delayed subprocess. The metadata does not identify or launch a real application.

Interface

Other

Earliest failure stage

Capture or observation

Relevant output or evidence

Final focused execution, reduced to relevant fields:

1 test passed (asserts current faulty behavior)
elapsed_ms: 30008
error tag: ProviderAdapterError
underlying code: ETIMEDOUT
operation signal at rejection: not aborted
projected code: execution_failure
retryable: false
message/remediation: Analysis could not complete. Run `rea doctor` and review the reported details; repeating the unchanged request is not expected to succeed.
diagnostics.remediation: Native helper failed, timed out, or returned malformed capture data; install compatible Xcode command-line tools and inspect local OS permissions
provider client close: successful

Source reachability: the production provider creates this runtime for observe_native_ui and capture_native_ui_scenario; these are public tool contracts. A live MCP call was not run.

Typed outcome or recovery expected

provider_timeout, with the reached invocation budget if the shorter limit is intentional, or the owned 180000 ms operation budget if it owns execution. Include timeout-specific recovery and preserve any already collected scenario observations. Do not turn a known timeout into non-retryable toolchain/permission failure.

Analysis or suspected cause

Source-derived: NativeUiHelperRuntime passes timeout: 30_000 in addition to its 180-second signal. The process runner constructs a plain ETIMEDOUT error. NativeUiObservation.take special-cases cancellation but wraps the timeout as ProviderAdapterError; the shared projection assigns execution_failure.

Additional context

Fetched main 705058f3e2df82503f1c601fc464aea7cbb626b8 has no changes from the tested baseline in the runtime, interruption owner, observation catch, process runner, or cited native UI guide. Current-main applicability is source-derived; runtime evidence is at the baseline.

Related reports #5056 and #4748 cover the owned deadline's cancellation/finalization provenance, not this earlier per-command timeout. PR #3482 only extracts the operation timeout constant, and PR #854 retains helper diagnostics without classifying timeouts. No fix or restoration was implemented or tested. Last known good revision: NOT_ENOUGH_INFO.

Originally created by @coygeek on GitHub (Oct 21, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/5948 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Process capture or runtime observation ### Observed behavior A production native UI provider call stops its helper subprocess after 30000 ms and returns `execution_failure`, `retryable: false`, with generic `rea doctor` recovery and Xcode/permissions advice. It does this while the operation deadline signal is still active. The scenario schema documents a 180-second operation deadline; the shorter helper limit is not advertised there. This was executed through `NativeMacOSProvider.createClient(...).execute("observe_native_ui", ...)` with a substituted compiler/helper launch, using the real runtime, actual owned subprocess, real 30-second timer, and error projection. No slow real application or live CLI/MCP transport was exercised. ### Reproduction From the prepared source checkout with locked dependencies, save the following test as `src/native/nativeUiHelperTimeoutReproduction.test.ts` and run `node node_modules/vitest/vitest.mjs run src/native/nativeUiHelperTimeoutReproduction.test.ts --maxWorkers 1`. The test asserts the faulty outcome and therefore passes when this bug is present. The compilation seam returns immediately; the helper launch is replaced with an owned Node process scheduled to finish after 31 seconds. All timeout/signal options from `NativeUiHelperRuntime` pass unchanged to the real process runner. ```ts import { expect, it, vi } from 'vitest'; import { NativeMacOSProvider } from './NativeMacOSProvider.js'; import { NativeFixtureRunner, nativeMachoTarget } from '../../tests/fixtures/nativeCommands.js'; import { projectAnalysisError } from '../domain/analysisErrorProjection.js'; vi.mock('../process/ExecFileOutput.js', async (importOriginal) => { const actual = await importOriginal<typeof import('../process/ExecFileOutput.js')>(); return { ...actual, execFileOutput: async (command: string, args: readonly string[], options: any) => { if (command === '/usr/bin/xcrun') return { stdout: '', stderr: '' }; if (command.endsWith('/observer')) { expect(options.timeout).toBe(30_000); expect(options.signal.aborted).toBe(false); try { return await actual.execFileOutput(process.execPath, ['-e', 'setTimeout(() => console.log(JSON.stringify({ok:true})), 31000)'], options); } catch (cause) { expect(options.signal.aborted).toBe(false); throw cause; } } return actual.execFileOutput(command, args, options); }, }; }); it('runs the real native UI runtime timeout through the public provider client', async () => { const client = new NativeMacOSProvider({}, new NativeFixtureRunner(), 'darwin') .createClient(nativeMachoTarget('/fixture/App')); const started = performance.now(); try { const result = await client.execute('observe_native_ui', { pid: 123, window_id: 456, screenshot: false, accessibility: true, }); const elapsed = Math.round(performance.now() - started); expect(result.ok).toBe(false); if (result.ok) throw new Error('Expected timeout'); expect(result.error._tag).toBe('ProviderAdapterError'); expect(Reflect.get(result.error.cause!, 'code')).toBe('ETIMEDOUT'); const projected = projectAnalysisError(result.error); expect(projected.code).toBe('execution_failure'); expect(elapsed).toBeGreaterThanOrEqual(29_000); expect(elapsed).toBeLessThan(90_000); console.log(JSON.stringify({elapsed_ms: elapsed, tag: result.error._tag, cause_code: Reflect.get(result.error.cause!, 'code'), projection: projected})); } finally { expect((await client.close()).ok).toBe(true); } }, 90_000); ``` Two executions reproduced the same result. The final execution additionally asserted that the operation signal had not aborted when the subprocess timeout arrived. Delete the temporary test after use. ### Expected behavior A reached server/provider deadline should produce `provider_timeout` with its actual budget and timeout-specific recovery. Preserve caller cancellation and physical helper failures as distinct outcomes. Either let the documented operation deadline own helper execution, or explicitly describe and type any shorter supported invocation deadline. The [scenario wait schema](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/domain/native/nativeUiObservation.ts#L47-L54) names the 180-second operation deadline; the [native UI guide](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/docs/native-investigation.md#L534-L539) describes it, and the [recovery contract](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/docs/mcp-contracts.md#L243-L250) identifies `provider_timeout` as REA's server/provider deadline. This report does not assume every helper call is guaranteed the whole operation budget; the definite defect is the reached helper timeout being classified as an execution failure. ### Impact and scope P2: a bounded native UI subprocess timeout aborts capture with the wrong structured code and recovery. The real 30-second stop and its misclassification reproduced twice at the controlled provider boundary. Source inspection shows the same helper runtime serves `capture_native_ui_scenario`, but that operation was not executed here. Real-world frequency and whether any specific accessibility tree or screenshot exceeds the limit are unknown. No target data mutation or security-boundary crossing was exercised. No complete-capture workaround was demonstrated. ### REA version `rea-agents` 6.7.0; source `52415975942a4b9c53a817915b313ccc404f280e`. ### Runtime and platform macOS 27.0.1 arm64; Node.js v26.12.0; repository-pinned Vitest 5.0.3. Simulated native compiler/helper launch with a real Node subprocess and real process timeout. No Swift compiler or GUI provider qualification. ### Target and artifact context Synthetic Mach-O target metadata, selected PID/window values, and an owned delayed subprocess. The metadata does not identify or launch a real application. ### Interface Other ### Earliest failure stage Capture or observation ### Relevant output or evidence Final focused execution, reduced to relevant fields: ```text 1 test passed (asserts current faulty behavior) elapsed_ms: 30008 error tag: ProviderAdapterError underlying code: ETIMEDOUT operation signal at rejection: not aborted projected code: execution_failure retryable: false message/remediation: Analysis could not complete. Run `rea doctor` and review the reported details; repeating the unchanged request is not expected to succeed. diagnostics.remediation: Native helper failed, timed out, or returned malformed capture data; install compatible Xcode command-line tools and inspect local OS permissions provider client close: successful ``` Source reachability: the [production provider](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/native/NativeMacOSProvider.ts#L403-L442) creates this runtime for `observe_native_ui` and `capture_native_ui_scenario`; these are [public tool contracts](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/contracts/native/nativeToolContracts.ts#L78-L88). A live MCP call was not run. ### Typed outcome or recovery expected `provider_timeout`, with the reached invocation budget if the shorter limit is intentional, or the owned 180000 ms operation budget if it owns execution. Include timeout-specific recovery and preserve any already collected scenario observations. Do not turn a known timeout into non-retryable toolchain/permission failure. ### Analysis or suspected cause Source-derived: [NativeUiHelperRuntime](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/native/NativeUiHelperRuntime.ts#L79-L98) passes `timeout: 30_000` in addition to its [180-second signal](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/native/NativeUiHelperRuntime.ts#L24-L28). The [process runner](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/process/ExecFileOutput.ts#L37-L58) constructs a plain `ETIMEDOUT` error. [NativeUiObservation.take](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/native/NativeUiObservation.ts#L209-L226) special-cases cancellation but wraps the timeout as `ProviderAdapterError`; the [shared projection](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/domain/analysisErrorProjection.ts#L242-L261) assigns `execution_failure`. ### Additional context Fetched main `705058f3e2df82503f1c601fc464aea7cbb626b8` has no changes from the tested baseline in the runtime, interruption owner, observation catch, process runner, or cited native UI guide. Current-main applicability is source-derived; runtime evidence is at the baseline. Related reports #5056 and #4748 cover the owned deadline's cancellation/finalization provenance, not this earlier per-command timeout. PR #3482 only extracts the operation timeout constant, and PR #854 retains helper diagnostics without classifying timeouts. No fix or restoration was implemented or tested. Last known good revision: NOT_ENOUGH_INFO.

bsakaguchi closed this issue

2026-10-31 00:42:25 +00:00

bsakaguchi commented

2026-10-31 00:42:26 +00:00

Author

Owner

@coygeek commented on GitHub (Oct 24, 2026):

Checked the reported fix on main at d87e73f2db.

The hidden 30-second helper timeout is removed: compile and invoke both use the selected native UI deadline. ETIMEDOUT is explicitly mapped via interruption.deadlineFailure instead of ProviderAdapterError. Regression injects ETIMEDOUT with an active deadline signal and asserts provider_timeout, the selected 2500 ms budget and timeout recovery; caller-cancellation precedence has separate tests.

Implementation: NativeUiHelperRuntime.ts, NativeUiObservation.ts.

The relevant regression checks in NativeUiHelperRuntime.test.ts, NativeUiObservation.test.ts passed in the focused test run on this revision. This closure is based on the current source and those checks; provider fixtures do not establish a real-engine run. Closing as completed on main.

@coygeek commented on GitHub (Oct 24, 2026): Checked the reported fix on `main` at d87e73f2db50be72d33a4b3c7199af35518b441c. The hidden 30-second helper timeout is removed: compile and invoke both use the selected native UI deadline. ETIMEDOUT is explicitly mapped via interruption.deadlineFailure instead of ProviderAdapterError. Regression injects ETIMEDOUT with an active deadline signal and asserts provider_timeout, the selected 2500 ms budget and timeout recovery; caller-cancellation precedence has separate tests. Implementation: [NativeUiHelperRuntime.ts](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/src/native/NativeUiHelperRuntime.ts#L78), [NativeUiObservation.ts](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/src/native/NativeUiObservation.ts#L249). The relevant regression checks in [NativeUiHelperRuntime.test.ts](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/src/native/NativeUiHelperRuntime.test.ts), [NativeUiObservation.test.ts](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/src/native/NativeUiObservation.test.ts) passed in the focused test run on this revision. This closure is based on the current source and those checks; provider fixtures do not establish a real-engine run. Closing as completed on main.

bsakaguchi referenced this issue

2026-11-01 21:10:58 +00:00

[GH-ISSUE #3796] [Tracker] Open feature requests by area (October 2026) #1471

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#3616

No description provided.

Rows
Columns