morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #5840] [Bug] get_objc_classes discards the provider explanation for unavailable import bindings #3536

New issue

Closed

opened 2026-10-31 00:41:37 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-31 00:41:37 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 21, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/5840

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Native analysis or decompilation

Observed behavior

An executed in-memory MCP call to get_objc_classes with {} returned the class inventory and identity_coverage.status: "unknown", but replaced the provider's capability explanation with this fixed text:

Native class-import binding evidence is unavailable (AnalysisCapabilityUnavailableError).

Two fake-provider cases supplied different explanations for the unavailable inspect_native_dispatch_metadata operation: a profile limitation and unsupported 32-bit metadata. Both produced that same text. Neither original explanation appeared anywhere in the full MCP response. This reproduction used the production server through the repository's existing MCP harness; no external analysis engine was run.

Reproduction

With the baseline's locked dependencies installed, select its pinned Node.js version. Save the following as src/application/repro.test.ts, then run npm run test:local -- src/application/repro.test.ts:

import { afterEach, expect, it } from "vitest";
import {
  closeEnhancedToolResources,
  connect,
  jsonResult,
} from "../../tests/boundary/mcp/enhancedToolsHarness.js";
import { observed } from "../../tests/fixtures/analysisExecution.js";
import { AnalysisCapabilityUnavailableError } from "../domain/analysisErrorCore.js";
import { err } from "../domain/result.js";

afterEach(closeEnhancedToolResources);

it.each([
  "This profile does not expose direct class-import bindings.",
  "The native metadata reader does not support 32-bit Mach-O images.",
])("retains the capability explanation: %s", async (reason) => {
  const calls: string[] = [];
  const client = await connect({
    execute: async (operation) => {
      calls.push(operation);
      if (operation === "list_names")
        return observed([{ address: "0x1000", value: "_OBJC_CLASS_$_Sample" }]);
      expect(operation).toBe("inspect_native_dispatch_metadata");
      return err(new AnalysisCapabilityUnavailableError("fixture", operation, reason));
    },
  });
  const response = await client.callTool({ name: "get_objc_classes", arguments: {} });
  const result = jsonResult(response);
  console.log(JSON.stringify({ input_reason: reason, result,
    reason_present_anywhere_in_response: JSON.stringify(response).includes(reason) }));
  expect(calls).toEqual(["list_names", "inspect_native_dispatch_metadata"]);
  expect(response.isError).not.toBe(true);
  expect(result).toMatchObject({ count: 1,
    classes: [{ address: "0x1000", name: "_OBJC_CLASS_$_Sample" }],
    identity_coverage: { status: "unknown", reason: expect.stringContaining(reason) },
  });
});

Both cases ran once. Both failed the assertion requiring the supplied reason; the operation-sequence and successful-response checks passed. Remove the temporary test afterward. The file is beside the application module because test:local rejects the MCP boundary directory as a built test project; the test still invokes the production MCP server using its existing harness.

Expected behavior

Keep the useful class-label fallback and unknown identity status, while preserving the meaningful provider explanation in the returned limitation. The exact sentence format is not prescribed.

AGENTS.md explicitly requires preserving partial facts and meaningful failure reasons. The tool design contract requires coverage and relevant limitations inline. The class-discovery documentation documents retaining class labels with unknown import identity when recognized formats are unsupported.

Impact and scope

P2: the caller loses the specific diagnostic context needed to distinguish capability limitations. The two executed inputs became indistinguishable on that point, even though the error retained different explanations. Class count, class label, and unknown identity status remained intact. No target modification or security boundary crossing occurred in these fake-provider cases.

Source-derived scope is every ordinary capability-unavailable error reaching this fallback with a nonempty selected class inventory. Real-provider frequency and recovery behavior were not measured. No recovery workaround was executed; this report does not claim that changing providers restores coverage. CLI behavior was not executed.

REA version

rea-agents 6.7.0; source revision a5f982de8d

Runtime and platform

Node.js 24.18.0; macOS 27.0.1 arm64; Vitest 5.0.3; in-memory MCP with fake analysis providers

Target and artifact context

Synthetic inventory containing one Objective-C class label at 0x1000; no target binary or external engine required

Interface

MCP

Earliest failure stage

Transport or result projection

Relevant output or evidence

Trimmed JSON printed by the executed test:

{"input_reason":"This profile does not expose direct class-import bindings.","result":{"count":1,"classes":[{"address":"0x1000","name":"_OBJC_CLASS_$_Sample"}],"identity_coverage":{"status":"unknown","reason":"Native class-import binding evidence is unavailable (AnalysisCapabilityUnavailableError)."}},"reason_present_anywhere_in_response":false}
{"input_reason":"The native metadata reader does not support 32-bit Mach-O images.","result":{"count":1,"classes":[{"address":"0x1000","name":"_OBJC_CLASS_$_Sample"}],"identity_coverage":{"status":"unknown","reason":"Native class-import binding evidence is unavailable (AnalysisCapabilityUnavailableError)."}},"reason_present_anywhere_in_response":false}

The focused run ended with Tests 2 failed (2). Each failure compared the original reason with the constant sentence above. Those failures demonstrate the missing explanation; they are not a passing regression check or a tested fix.

The public MCP registration accepts the call, and the shared workflow dispatch invokes EnhancedTools. Its class-inventory branch reaches the fallback after successful name selection and a capability error. The result projection returns the chosen coverage reason unchanged.

Typed outcome or recovery expected

Return a successful inventory with unknown identity coverage and the actual explanation for the unavailable binding scan. Provider and operation attribution may help, but the essential missing fact is the retained error reason.

Analysis or suspected cause

Source-derived: the fallback interpolates error._tag, which the capability-error class defines as a constant. The explanation is available separately as error.reason. Preserving that explanation at this conversion boundary would address the demonstrated loss; no fix was attempted.

Additional context

Read-only searches covered open and closed issues and PRs using class-discovery, import-binding, and coverage-reason terms. No report of this same failure was found. The closest inspected reports concern duplicate import aliases and omitted stripped class labels, which are different behaviors.

The two reason strings above are synthetic test inputs, not captured output from a real provider. As a source-derived example of an applicable production input, the Mach-O reader raises a format-specific unsupported explanation for recognized 32-bit metadata, and the native adapter retains it in a capability error. That real-provider path was not run here.

Originally created by @coygeek on GitHub (Oct 21, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/5840 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Native analysis or decompilation ### Observed behavior An executed in-memory MCP call to `get_objc_classes` with `{}` returned the class inventory and `identity_coverage.status: "unknown"`, but replaced the provider's capability explanation with this fixed text: ```text Native class-import binding evidence is unavailable (AnalysisCapabilityUnavailableError). ``` Two fake-provider cases supplied different explanations for the unavailable `inspect_native_dispatch_metadata` operation: a profile limitation and unsupported 32-bit metadata. Both produced that same text. Neither original explanation appeared anywhere in the full MCP response. This reproduction used the production server through the repository's existing MCP harness; no external analysis engine was run. ### Reproduction With the baseline's locked dependencies installed, select its pinned Node.js version. Save the following as `src/application/repro.test.ts`, then run `npm run test:local -- src/application/repro.test.ts`: ```ts import { afterEach, expect, it } from "vitest"; import { closeEnhancedToolResources, connect, jsonResult, } from "../../tests/boundary/mcp/enhancedToolsHarness.js"; import { observed } from "../../tests/fixtures/analysisExecution.js"; import { AnalysisCapabilityUnavailableError } from "../domain/analysisErrorCore.js"; import { err } from "../domain/result.js"; afterEach(closeEnhancedToolResources); it.each([ "This profile does not expose direct class-import bindings.", "The native metadata reader does not support 32-bit Mach-O images.", ])("retains the capability explanation: %s", async (reason) => { const calls: string[] = []; const client = await connect({ execute: async (operation) => { calls.push(operation); if (operation === "list_names") return observed([{ address: "0x1000", value: "_OBJC_CLASS_$_Sample" }]); expect(operation).toBe("inspect_native_dispatch_metadata"); return err(new AnalysisCapabilityUnavailableError("fixture", operation, reason)); }, }); const response = await client.callTool({ name: "get_objc_classes", arguments: {} }); const result = jsonResult(response); console.log(JSON.stringify({ input_reason: reason, result, reason_present_anywhere_in_response: JSON.stringify(response).includes(reason) })); expect(calls).toEqual(["list_names", "inspect_native_dispatch_metadata"]); expect(response.isError).not.toBe(true); expect(result).toMatchObject({ count: 1, classes: [{ address: "0x1000", name: "_OBJC_CLASS_$_Sample" }], identity_coverage: { status: "unknown", reason: expect.stringContaining(reason) }, }); }); ``` Both cases ran once. Both failed the assertion requiring the supplied reason; the operation-sequence and successful-response checks passed. Remove the temporary test afterward. The file is beside the application module because `test:local` rejects the MCP boundary directory as a built test project; the test still invokes the production MCP server using its existing harness. ### Expected behavior Keep the useful class-label fallback and unknown identity status, while preserving the meaningful provider explanation in the returned limitation. The exact sentence format is not prescribed. [AGENTS.md](https://github.com/morluto/rea/blob/a5f982de8dbe4b092cc20ab89d492b7e7028b758/AGENTS.md#L31-L31) explicitly requires preserving partial facts and meaningful failure reasons. The [tool design contract](https://github.com/morluto/rea/blob/a5f982de8dbe4b092cc20ab89d492b7e7028b758/docs/tool-design.md#L51-L58) requires coverage and relevant limitations inline. The [class-discovery documentation](https://github.com/morluto/rea/blob/a5f982de8dbe4b092cc20ab89d492b7e7028b758/docs/native-investigation.md#L498-L506) documents retaining class labels with unknown import identity when recognized formats are unsupported. ### Impact and scope P2: the caller loses the specific diagnostic context needed to distinguish capability limitations. The two executed inputs became indistinguishable on that point, even though the error retained different explanations. Class count, class label, and unknown identity status remained intact. No target modification or security boundary crossing occurred in these fake-provider cases. Source-derived scope is every ordinary capability-unavailable error reaching this fallback with a nonempty selected class inventory. Real-provider frequency and recovery behavior were not measured. No recovery workaround was executed; this report does not claim that changing providers restores coverage. CLI behavior was not executed. ### REA version rea-agents 6.7.0; source revision a5f982de8dbe4b092cc20ab89d492b7e7028b758 ### Runtime and platform Node.js 24.18.0; macOS 27.0.1 arm64; Vitest 5.0.3; in-memory MCP with fake analysis providers ### Target and artifact context Synthetic inventory containing one Objective-C class label at 0x1000; no target binary or external engine required ### Interface MCP ### Earliest failure stage Transport or result projection ### Relevant output or evidence Trimmed JSON printed by the executed test: ```json {"input_reason":"This profile does not expose direct class-import bindings.","result":{"count":1,"classes":[{"address":"0x1000","name":"_OBJC_CLASS_$_Sample"}],"identity_coverage":{"status":"unknown","reason":"Native class-import binding evidence is unavailable (AnalysisCapabilityUnavailableError)."}},"reason_present_anywhere_in_response":false} {"input_reason":"The native metadata reader does not support 32-bit Mach-O images.","result":{"count":1,"classes":[{"address":"0x1000","name":"_OBJC_CLASS_$_Sample"}],"identity_coverage":{"status":"unknown","reason":"Native class-import binding evidence is unavailable (AnalysisCapabilityUnavailableError)."}},"reason_present_anywhere_in_response":false} ``` The focused run ended with `Tests 2 failed (2)`. Each failure compared the original reason with the constant sentence above. Those failures demonstrate the missing explanation; they are not a passing regression check or a tested fix. The public [MCP registration](https://github.com/morluto/rea/blob/a5f982de8dbe4b092cc20ab89d492b7e7028b758/src/server/registerEnhancedTools.ts#L97-L104) accepts the call, and the [shared workflow dispatch](https://github.com/morluto/rea/blob/a5f982de8dbe4b092cc20ab89d492b7e7028b758/src/server/registerEnhancedTools.ts#L244-L250) invokes `EnhancedTools`. Its [class-inventory branch](https://github.com/morluto/rea/blob/a5f982de8dbe4b092cc20ab89d492b7e7028b758/src/application/EnhancedTools.ts#L300-L342) reaches the fallback after successful name selection and a capability error. The [result projection](https://github.com/morluto/rea/blob/a5f982de8dbe4b092cc20ab89d492b7e7028b758/src/domain/symbolAnalysis.ts#L70-L78) returns the chosen coverage reason unchanged. ### Typed outcome or recovery expected Return a successful inventory with unknown identity coverage and the actual explanation for the unavailable binding scan. Provider and operation attribution may help, but the essential missing fact is the retained error reason. ### Analysis or suspected cause Source-derived: the fallback interpolates `error._tag`, which the [capability-error class](https://github.com/morluto/rea/blob/a5f982de8dbe4b092cc20ab89d492b7e7028b758/src/domain/analysisErrorCore.ts#L124-L147) defines as a constant. The explanation is available separately as `error.reason`. Preserving that explanation at this conversion boundary would address the demonstrated loss; no fix was attempted. ### Additional context Read-only searches covered open and closed issues and PRs using class-discovery, import-binding, and coverage-reason terms. No report of this same failure was found. The closest inspected reports concern [duplicate import aliases](https://github.com/morluto/rea/issues/5607) and [omitted stripped class labels](https://github.com/morluto/rea/issues/5617), which are different behaviors. The two reason strings above are synthetic test inputs, not captured output from a real provider. As a source-derived example of an applicable production input, the [Mach-O reader](https://github.com/morluto/rea/blob/a5f982de8dbe4b092cc20ab89d492b7e7028b758/src/native/AppleMachoSelection.ts#L201-L208) raises a format-specific unsupported explanation for recognized 32-bit metadata, and the [native adapter](https://github.com/morluto/rea/blob/a5f982de8dbe4b092cc20ab89d492b7e7028b758/src/native/AppleDispatchMetadata.ts#L359-L365) retains it in a capability error. That real-provider path was not run here.

bsakaguchi closed this issue

2026-10-31 00:41:37 +00:00

bsakaguchi commented

2026-10-31 00:41:38 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 21, 2026):

Confirmed — the fallback interpolating error._tag (a constant) instead of error.reason makes two genuinely different capability failures (profile limitation vs unsupported 32-bit metadata) byte-identical in the response, which directly contradicts AGENTS.md's failure-reason preservation requirement. Since the class inventory and unknown status are already retained correctly, swapping the projection to carry error.reason through the limitation is a small, well-scoped fix that restores actionable diagnostics without changing the success shape.

@XinwangTao commented on GitHub (Oct 21, 2026): Confirmed — the fallback interpolating error._tag (a constant) instead of error.reason makes two genuinely different capability failures (profile limitation vs unsupported 32-bit metadata) byte-identical in the response, which directly contradicts AGENTS.md's failure-reason preservation requirement. Since the class inventory and unknown status are already retained correctly, swapping the projection to carry error.reason through the limitation is a small, well-scoped fix that restores actionable diagnostics without changing the success shape.

bsakaguchi commented

2026-10-31 00:41:38 +00:00

Author

Owner

@coygeek commented on GitHub (Oct 24, 2026):

Checked the reported fix on main at d87e73f2db.

The shared get_objc_classes fallback now passes bindingsExecution.error.reason directly into unknown coverage, retaining the class inventory. The inspected regression supplies the nonconstant reason No static Mach-O reader and explicitly expects that exact reason, unknown status and retained class labels/count; the following test asserts that non-capability failures still fail rather than silently falling back. This shared reason projection covers either original synthetic explanation by source inspection, but the existing assertion uses one different explanation and the workflow test is not a new in-memory MCP replay.

Implementation: EnhancedTools.ts, EnhancedTools.objcClasses.test.ts.

The relevant regression checks in EnhancedTools.objcClasses.test.ts passed in the focused test run on this revision. This closure is based on the current source and those checks; provider fixtures do not establish a real-engine run. Closing as completed on main.

@coygeek commented on GitHub (Oct 24, 2026): Checked the reported fix on `main` at d87e73f2db50be72d33a4b3c7199af35518b441c. The shared get_objc_classes fallback now passes bindingsExecution.error.reason directly into unknown coverage, retaining the class inventory. The inspected regression supplies the nonconstant reason No static Mach-O reader and explicitly expects that exact reason, unknown status and retained class labels/count; the following test asserts that non-capability failures still fail rather than silently falling back. This shared reason projection covers either original synthetic explanation by source inspection, but the existing assertion uses one different explanation and the workflow test is not a new in-memory MCP replay. Implementation: [EnhancedTools.ts](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/src/application/EnhancedTools.ts#L335), [EnhancedTools.objcClasses.test.ts](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/src/application/EnhancedTools.objcClasses.test.ts#L312). The relevant regression checks in [EnhancedTools.objcClasses.test.ts](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/src/application/EnhancedTools.objcClasses.test.ts) passed in the focused test run on this revision. This closure is based on the current source and those checks; provider fixtures do not establish a real-engine run. Closing as completed on main.

bsakaguchi referenced this issue

2026-10-31 00:48:46 +00:00

[GH-ISSUE #7227] [Bug] Module-level webPreferences constants are reported as dynamic when the window is created inside a function #4307

bsakaguchi referenced this issue

2026-10-31 01:12:44 +00:00

[PR #4400] [MERGED] feat(electron): resolve stable local constant window options #6213

bsakaguchi referenced this issue

2026-11-01 21:11:31 +00:00

[GH-ISSUE #8201] [Bug] Electron graph reports 0 IPC channels and 0 BrowserWindows on a real Rolldown bundle (Signal Desktop 8.33.0) #6482

bsakaguchi referenced this issue

2026-11-03 13:39:57 +00:00

[PR #7229] fix(electron): resolve module constant window options read inside functions #7731