morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #2566] fix(native): return seed resolution errors from UI action traces #884

New issue

Closed

opened 2026-10-15 18:29:17 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-15 18:29:17 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 13, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/2566

Summary

For a trace_native_ui_action request whose selector is absent from the decoded UI graph, a provider failure while resolving that selector as a native function produces a successful not-found trace. This can make an analysis failure look like evidence that no action or native function matches the input.

Runtime or environment

Failure was verified at a2d8b94dd5 and is still present at 8b181716ac by source recheck; Node v26.10.0; lockfile dependencies installed; focused service test lane.

Steps to reproduce

At the stated baseline, add the following temporary test at src/application/native/nativeSeedError.test.ts. The fixture supplies a valid empty UI graph and an empty native name inventory for one target, then makes native seed resolution return an output error.
Run npm run test:focused -- src/application/native/nativeSeedError.test.ts and inspect the first test's returned result. The second test is a cancellation control.

import { expect, it } from "vitest";
import { createAnalysisExecution, type AnalysisOperationPort } from "../AnalysisProvider.js";
import { EnhancedTools } from "../EnhancedTools.js";
import { AnalysisCancelledError, AnalysisCapabilityUnavailableError, AnalysisOutputError } from "../../domain/analysisErrorCore.js";
import { err, ok } from "../../domain/result.js";

const sha256 = "a".repeat(64);
const provider = { id: "fixture", name: "Fixture", version: "1" };
const subject = { path: "fixture", sha256, format: "mach-o" as const, architecture: "arm64" as const };
const makePort = (cancelled: boolean): AnalysisOperationPort => ({
  execute: async (operation) => {
    if (operation === "inspect_native_dispatch_metadata")
      return err(new AnalysisCapabilityUnavailableError("fixture", operation, "no binary metadata"));
    if (operation === "list_names") return ok(createAnalysisExecution([], provider, { subject }));
    if (operation === "decode_interface_builder")
      return ok(createAnalysisExecution({
        target_sha256: sha256,
        documents: [],
        graph: { target_sha256: sha256, provider: { id: "fixture", version: "1", tool_version: "fixture" }, nodes: [], edges: [], coverage: [], truncated: false },
        limitations: [],
      }, provider, { subject }));
    if (operation === "procedure_address")
      return err(cancelled ? new AnalysisCancelledError(operation) : new AnalysisOutputError(operation, "provider failed"));
    throw new Error(`unexpected operation ${operation}`);
  },
});

it("reports a failed native seed lookup", async () => {
  const result = await new EnhancedTools(makePort(false)).execute("trace_native_ui_action", { action: "missing-selector" });
  expect(result).toMatchObject({ ok: true, value: { reason: "ui_action_or_object_not_found" } });
});

it("retains cancellation as an error", async () => {
  const result = await new EnhancedTools(makePort(true)).execute("trace_native_ui_action", { action: "missing-selector" });
  expect(result).toMatchObject({ ok: false, error: { _tag: "AnalysisCancelledError" } });
});

Expected behavior

The trace accepts a native symbol or address as its seed. If the UI graph has no match and native resolution fails because the provider produced an invalid result, the operation should report that failure rather than claiming the action or function was not found. A missing-capability result can continue to use the existing not-found behavior.

Actual behavior

In the focused fixture, the output-error case returned ok: true and reason: "ui_action_or_object_not_found"; the cancellation control returned ok: false with AnalysisCancelledError. The provider output error was absent from the trace result.

Evidence

Expected source: docs/native-investigation.md:210-215 documents accepted native seeds; AGENTS.md requires unknown identity and meaningful errors to be preserved; src/application/ProcedureAddressResolution.ts:19-20 shows error propagation for native address resolution.
Failure source: src/application/native/NativeUiActionTrace.ts:119-177 returns cancellation, checks a successful address, then emits not-found for other failed native resolution results.
Evidence provenance: observed
Local verification: reproduced
Reproduction completeness: complete
The focused test passed both observations: output error -> {"ok":true,"reason":"ui_action_or_object_not_found"}; cancellation -> {"ok":false,"tag":"AnalysisCancelledError"}. The existing MCP boundary test at tests/boundary/mcp/enhancedTools.01-lists-the-complete-target-open.test.ts:391-394 covers capability-unavailable not-found separately.

Restoration check

With the same empty graph, target identity, and output-error result, the trace should return the provider error instead of a successful not-found result. Cancellation should still return its error, and the capability-unavailable case should retain its established behavior. These cases distinguish error preservation from treating every unresolved native seed alike.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: A failed native seed lookup becomes a positive not-found answer for affected trace requests. The reproduction establishes this bounded result when native resolution returns an output error; its frequency with real providers is unknown.
Workaround status: unverified
Workaround basis: Calling native address resolution separately may reveal the provider failure, but this public workflow was not exercised end to end and does not recover the failed trace.

Originally created by @coygeek on GitHub (Oct 13, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/2566 ## Summary For a `trace_native_ui_action` request whose selector is absent from the decoded UI graph, a provider failure while resolving that selector as a native function produces a successful not-found trace. This can make an analysis failure look like evidence that no action or native function matches the input. ## Runtime or environment Failure was verified at a2d8b94dd50dab8d966239dd63ab84718aa8b4b4 and is still present at 8b181716aca2cb467be499cba226abfb5ae19141 by source recheck; Node v26.10.0; lockfile dependencies installed; focused service test lane. ## Steps to reproduce 1. At the stated baseline, add the following temporary test at `src/application/native/nativeSeedError.test.ts`. The fixture supplies a valid empty UI graph and an empty native name inventory for one target, then makes native seed resolution return an output error. 2. Run `npm run test:focused -- src/application/native/nativeSeedError.test.ts` and inspect the first test's returned result. The second test is a cancellation control. ```ts import { expect, it } from "vitest"; import { createAnalysisExecution, type AnalysisOperationPort } from "../AnalysisProvider.js"; import { EnhancedTools } from "../EnhancedTools.js"; import { AnalysisCancelledError, AnalysisCapabilityUnavailableError, AnalysisOutputError } from "../../domain/analysisErrorCore.js"; import { err, ok } from "../../domain/result.js"; const sha256 = "a".repeat(64); const provider = { id: "fixture", name: "Fixture", version: "1" }; const subject = { path: "fixture", sha256, format: "mach-o" as const, architecture: "arm64" as const }; const makePort = (cancelled: boolean): AnalysisOperationPort => ({ execute: async (operation) => { if (operation === "inspect_native_dispatch_metadata") return err(new AnalysisCapabilityUnavailableError("fixture", operation, "no binary metadata")); if (operation === "list_names") return ok(createAnalysisExecution([], provider, { subject })); if (operation === "decode_interface_builder") return ok(createAnalysisExecution({ target_sha256: sha256, documents: [], graph: { target_sha256: sha256, provider: { id: "fixture", version: "1", tool_version: "fixture" }, nodes: [], edges: [], coverage: [], truncated: false }, limitations: [], }, provider, { subject })); if (operation === "procedure_address") return err(cancelled ? new AnalysisCancelledError(operation) : new AnalysisOutputError(operation, "provider failed")); throw new Error(`unexpected operation ${operation}`); }, }); it("reports a failed native seed lookup", async () => { const result = await new EnhancedTools(makePort(false)).execute("trace_native_ui_action", { action: "missing-selector" }); expect(result).toMatchObject({ ok: true, value: { reason: "ui_action_or_object_not_found" } }); }); it("retains cancellation as an error", async () => { const result = await new EnhancedTools(makePort(true)).execute("trace_native_ui_action", { action: "missing-selector" }); expect(result).toMatchObject({ ok: false, error: { _tag: "AnalysisCancelledError" } }); }); ``` ## Expected behavior The trace accepts a native symbol or address as its seed. If the UI graph has no match and native resolution fails because the provider produced an invalid result, the operation should report that failure rather than claiming the action or function was not found. A missing-capability result can continue to use the existing not-found behavior. ## Actual behavior In the focused fixture, the output-error case returned `ok: true` and `reason: "ui_action_or_object_not_found"`; the cancellation control returned `ok: false` with `AnalysisCancelledError`. The provider output error was absent from the trace result. ## Evidence - Expected source: `docs/native-investigation.md:210-215` documents accepted native seeds; `AGENTS.md` requires unknown identity and meaningful errors to be preserved; `src/application/ProcedureAddressResolution.ts:19-20` shows error propagation for native address resolution. - Failure source: `src/application/native/NativeUiActionTrace.ts:119-177` returns cancellation, checks a successful address, then emits not-found for other failed native resolution results. - Evidence provenance: observed - Local verification: reproduced - Reproduction completeness: complete - The focused test passed both observations: output error -> `{"ok":true,"reason":"ui_action_or_object_not_found"}`; cancellation -> `{"ok":false,"tag":"AnalysisCancelledError"}`. The existing MCP boundary test at `tests/boundary/mcp/enhancedTools.01-lists-the-complete-target-open.test.ts:391-394` covers capability-unavailable not-found separately. ## Restoration check With the same empty graph, target identity, and output-error result, the trace should return the provider error instead of a successful not-found result. Cancellation should still return its error, and the capability-unavailable case should retain its established behavior. These cases distinguish error preservation from treating every unresolved native seed alike. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: A failed native seed lookup becomes a positive not-found answer for affected trace requests. The reproduction establishes this bounded result when native resolution returns an output error; its frequency with real providers is unknown. - Workaround status: unverified - Workaround basis: Calling native address resolution separately may reveal the provider failure, but this public workflow was not exercised end to end and does not recover the failed trace.

bsakaguchi

2026-10-15 18:29:17 +00:00

closed this issue
added the
area: native

bug
labels

bsakaguchi commented

2026-10-15 18:29:18 +00:00

Author

Owner

@morluto commented on GitHub (Oct 14, 2026):

Verified the reported defect is addressed on main by 55dd21a5e2.

Root cause and resolution: A native seed resolution error was collapsed into empty UI-selector matching and successful not-found output.

Validation: src/application/native/EnhancedNativeInvestigation.test.ts — returns native seed resolution failures instead of reporting not found asserts a procedure_address AnalysisOutputError is returned; cancellation is separately tested.

Evidence: CI run 500e: owner unchanged and assertion passed.

Verification scope: Shared workflow error propagation; no live native provider run.

Current main CI has separate failures under repair; this closure does not claim the complete main workflow is green.

@morluto commented on GitHub (Oct 14, 2026): Verified the reported defect is addressed on main by 55dd21a5e2172187dfe7167df684f85ecb536f5e. Root cause and resolution: A native seed resolution error was collapsed into empty UI-selector matching and successful not-found output. Validation: `src/application/native/EnhancedNativeInvestigation.test.ts` — `returns native seed resolution failures instead of reporting not found` asserts a `procedure_address` AnalysisOutputError is returned; cancellation is separately tested. Evidence: CI run 500e: owner unchanged and assertion passed. Verification scope: Shared workflow error propagation; no live native provider run. Current main CI has separate failures under repair; this closure does not claim the complete main workflow is green.

bsakaguchi referenced this issue

2026-10-17 21:12:11 +00:00

[PR #1533] [MERGED] feat(wasm): inspect local artifacts with WABT #2951

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#884

No description provided.

Rows
Columns