morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #6857] [Bug] P/Invoke verification drops decorated candidates for default and Auto charsets #4197

New issue

Closed

opened 2026-10-31 00:47:37 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-31 00:47:37 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 23, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6857

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

verify-managed-native-boundaries returns unresolved / no-native-candidate with an empty candidate list for a default-charset P/Invoke Example and supplied PE function Evidence named ExampleA. Changing only the declaration to explicit ANSI retains the same function as inferred / decorated-name-candidate. The corresponding Auto / ExampleW case also loses its candidate, while explicit Unicode retains it.

These are executed CLI results over production inspection output from synthetic PE/CLI metadata and authenticated synthetic native function Evidence. Managed inspection has complete coverage. No .NET or native target code was executed; this does not claim a live Windows binding test.

Reproduction

From the prepared source checkout, with locked dependencies and compiled runtime available, save the script below as /tmp/rea-charset-repro.mts, then run:

export REA_ROOT="$PWD"
export REPRO_DIR="$(mktemp -d)"
mkdir -p "$REPRO_DIR"/{home,config,cache,data,state}
node --import "$PWD/node_modules/tsx/dist/loader.mjs" /tmp/rea-charset-repro.mts

The exact script was executed once for each of its six cases. It creates only disposable requests/results and invokes the supported CLI.

import { writeFileSync } from 'node:fs';
import { spawnSync } from 'node:child_process';
import { pathToFileURL } from 'node:url';
const repo = process.env.REA_ROOT!;
const work = process.env.REPRO_DIR!;
const load = (name: string) => import(pathToFileURL(`${repo}/src/${name}.ts`).href);
const { buildManagedPeFixture, managedPeFixtureTarget } = await load('dotnet/ManagedPe.fixture');
const { inspectManagedNativeBoundariesBytes } = await load('dotnet/ManagedNativeBoundaryInspector');
const { createEvidence } = await load('domain/evidence');
const { functionDossierSchema } = await load('domain/hopperValues');
const provider = { id: 'synthetic-provider', name: 'Synthetic provider', version: '1' };
const cases = [
  ['default-ansi-suffix', 0x0200, 'ExampleA'],
  ['explicit-ansi-control', 0x0202, 'ExampleA'],
  ['auto-unicode-suffix', 0x0206, 'ExampleW'],
  ['explicit-unicode-control', 0x0204, 'ExampleW'],
  ['auto-exact-control', 0x0206, 'Example'],
  ['exactspelling-control', 0x0205, 'ExampleW'],
] as const;
const results = [];
for (const [label, mappingFlags, name] of cases) {
  const bytes = buildManagedPeFixture({ targetFramework: null, pinvoke: { mappingFlags, importName: 'Example', moduleName: 'example.dll' } });
  const target = managedPeFixtureTarget(bytes);
  const inspection = inspectManagedNativeBoundariesBytes(bytes, target);
  const managed = createEvidence(target, { id: 'rea-dotnet-static', name: 'REA managed static analysis provider', version: '1' }, {
    operation: 'inspect_managed_native_boundaries', parameters: {}, result: inspection, rawResult: null, limitations: inspection.limitations,
  });
  const native = createEvidence({ path: '/examples/example.dll', sha256: '8'.repeat(64), format: 'pe', architecture: 'x86_64' }, provider, {
    operation: 'analyze_function', parameters: {}, rawResult: null,
    result: functionDossierSchema.parse({
      procedure: { address: '0x1000', name, classification: { external: false, thunk: false, thunk_target: null, provenance: 'synthetic-provider' }, body: { available: false, reason: 'Synthetic fixture' }, signature: null, locals: [] },
      pseudocode: '', assembly: [], comments: [], callers: [], callees: [], incoming_references: [], outgoing_references: [], referenced_strings: [], referenced_names: [], basic_blocks: [], native_api: null, native_value_flow: null, limitations: [],
    }),
  });
  const request = { managed_boundaries: managed, native_observations: [native] };
  writeFileSync(`${work}/${label}.json`, JSON.stringify(request));
  const run = spawnSync(process.execPath, [`${repo}/scripts/rea.mjs`, 'verify-managed-native-boundaries', `${work}/${label}.json`, '--json'], {
    cwd: repo, encoding: 'utf8', maxBuffer: 4 * 1024 * 1024,
    env: { PATH: `${process.execPath.slice(0, process.execPath.lastIndexOf('/'))}:/usr/bin:/bin`, HOME: `${work}/home`, XDG_CONFIG_HOME: `${work}/config`, XDG_CACHE_HOME: `${work}/cache`, XDG_DATA_HOME: `${work}/data`, XDG_STATE_HOME: `${work}/state`, REA_LOG_LEVEL: 'silent' },
  });
  writeFileSync(`${work}/${label}.stdout.json`, run.stdout);
  writeFileSync(`${work}/${label}.stderr.txt`, run.stderr);
  const parsed = JSON.parse(run.stdout);
  const normalized = parsed.normalized_result ?? parsed.result?.normalized_result ?? parsed.data?.normalized_result ?? parsed;
  const item = normalized.pinvoke_imports?.[0];
  results.push({ label, exit: run.status, stderr: run.stderr, declaration: inspection.pinvoke_imports[0], sourceCoverage: inspection.coverage, result: item ?? parsed });
}
writeFileSync(`${work}/charset-results.json`, JSON.stringify(results, null, 2));
console.log(JSON.stringify(results.map(({ label, exit, declaration, sourceCoverage, result }) => ({ label, exit, charset: declaration.char_set, no_mangle: declaration.no_mangle, coverage: sourceCoverage.state, status: result.status, basis: result.basis, candidates: result.candidates?.map((x: any) => x.name) })), null, 2));

Expected behavior

Retain ExampleA as an inferred candidate for the default charset, consistently with the explicit ANSI control. For supplied Windows PE context, retain ExampleW for Auto, consistently with the explicit Unicode control. Preserve the no_mangle / ExactSpelling restriction and keep this a static candidate inference, not proof of runtime binding.

Microsoft documents that the default CharSet.None behaves like ANSI in Charsets and marshalling, and describes the A/W name-matching alternatives when ExactSpelling is false in Specifying a character set. The first page also identifies Unicode as the Windows Auto behavior. REA's managed/native contract limits these checks to supplied declaration/name Evidence.

Impact and scope

Default C# P/Invoke declarations and Auto declarations can lose supported Windows decorated-name correspondences. Observed in two synthetic cases, with the equivalent explicit-charset controls succeeding; occurrence in real assemblies was not measured. Inputs remain readable and no artifact is changed. Affected automated verification reports no candidate even though the supplied function is the conventional candidate for that charset.

REA version

Executed source/build: 4526862728af01f1135714079c0d8a8c754eb364. The relevant candidateNames implementation is unchanged in latest published tag rea-agents-6.7.0 (27dc42d10249f0410904a75c0997842eb03b931c); that tag's runtime was not executed separately.

Runtime and platform

Node.js v24.22.1; Linux x86_64. Windows PE module identities are supplied as synthetic native function Evidence. No external provider is required.

Target and artifact context

Byte-built managed PE/CLI P/Invoke metadata: entry point Example, module example.dll, cdecl, charset flag values 0x0000, 0x0002, 0x0004, or 0x0006. Native Evidence describes one synthetic function in /examples/example.dll.

Interface

CLI

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

All six CLI calls exited 0 with empty stderr. Reduced output:

[
  {
    "case": "default-ansi-suffix",
    "charset": "not-specified",
    "no_mangle": false,
    "exit": 0,
    "status": "unresolved",
    "basis": "no-native-candidate",
    "candidates": []
  },
  {
    "case": "explicit-ansi-control",
    "charset": "ansi",
    "no_mangle": false,
    "exit": 0,
    "status": "inferred",
    "basis": "decorated-name-candidate",
    "candidates": [
      "ExampleA"
    ]
  },
  {
    "case": "auto-unicode-suffix",
    "charset": "auto",
    "no_mangle": false,
    "exit": 0,
    "status": "unresolved",
    "basis": "no-native-candidate",
    "candidates": []
  },
  {
    "case": "explicit-unicode-control",
    "charset": "unicode",
    "no_mangle": false,
    "exit": 0,
    "status": "inferred",
    "basis": "decorated-name-candidate",
    "candidates": [
      "ExampleW"
    ]
  },
  {
    "case": "auto-exact-control",
    "charset": "auto",
    "no_mangle": false,
    "exit": 0,
    "status": "verified",
    "basis": "exact-function-name",
    "candidates": [
      "Example"
    ]
  },
  {
    "case": "exactspelling-control",
    "charset": "unicode",
    "no_mangle": true,
    "exit": 0,
    "status": "unresolved",
    "basis": "no-native-candidate",
    "candidates": []
  }
]

The producer maps 0x0000 to not-specified and 0x0006 to auto in ManagedNativeBoundaryHelpers.ts. Candidate generation tests only unicode and ansi in managedNativeVerificationMatch.ts. The supported CLI routes the request into this workflow in managedCommands.ts.

Typed outcome or recovery expected

inferred / decorated-name-candidate, retaining the corresponding ExampleA or ExampleW function Evidence. The ExactSpelling control must continue returning no suffixed candidate, and exact unsuffixed names must remain available.

Analysis or suspected cause

Source-derived: the parsed charset states are preserved correctly, but candidate enumeration does not account for the default ANSI equivalence or the Auto charset's Windows meaning. The explicit-charset controls isolate enumeration from module identity and Evidence validation.

Additional context

Fresh searches across open and closed issues for P/Invoke charset, not-specified candidate, and Auto decorated returned no related report. The open-issue snapshot was also checked. This concerns missing static decorated candidates, not execution, marshaling correctness, or loader authenticity.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: Supported metadata states deterministically discard useful supplied native candidates in the executed CLI, while equivalent explicit charset states retain them; this is bounded static correspondence loss with no mutation.
Workaround status: unverified
Workaround basis: Manual comparison of the preserved declaration and native function names is plausible, but no separate supported automatic recovery preserving the original Evidence was verified.

Originally created by @coygeek on GitHub (Oct 23, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6857 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior `verify-managed-native-boundaries` returns `unresolved / no-native-candidate` with an empty candidate list for a default-charset P/Invoke `Example` and supplied PE function Evidence named `ExampleA`. Changing only the declaration to explicit ANSI retains the same function as `inferred / decorated-name-candidate`. The corresponding `Auto` / `ExampleW` case also loses its candidate, while explicit Unicode retains it. These are executed CLI results over production inspection output from synthetic PE/CLI metadata and authenticated synthetic native function Evidence. Managed inspection has complete coverage. No .NET or native target code was executed; this does not claim a live Windows binding test. ### Reproduction From the prepared source checkout, with locked dependencies and compiled runtime available, save the script below as `/tmp/rea-charset-repro.mts`, then run: ```sh export REA_ROOT="$PWD" export REPRO_DIR="$(mktemp -d)" mkdir -p "$REPRO_DIR"/{home,config,cache,data,state} node --import "$PWD/node_modules/tsx/dist/loader.mjs" /tmp/rea-charset-repro.mts ``` The exact script was executed once for each of its six cases. It creates only disposable requests/results and invokes the supported CLI. ```ts import { writeFileSync } from 'node:fs'; import { spawnSync } from 'node:child_process'; import { pathToFileURL } from 'node:url'; const repo = process.env.REA_ROOT!; const work = process.env.REPRO_DIR!; const load = (name: string) => import(pathToFileURL(`${repo}/src/${name}.ts`).href); const { buildManagedPeFixture, managedPeFixtureTarget } = await load('dotnet/ManagedPe.fixture'); const { inspectManagedNativeBoundariesBytes } = await load('dotnet/ManagedNativeBoundaryInspector'); const { createEvidence } = await load('domain/evidence'); const { functionDossierSchema } = await load('domain/hopperValues'); const provider = { id: 'synthetic-provider', name: 'Synthetic provider', version: '1' }; const cases = [ ['default-ansi-suffix', 0x0200, 'ExampleA'], ['explicit-ansi-control', 0x0202, 'ExampleA'], ['auto-unicode-suffix', 0x0206, 'ExampleW'], ['explicit-unicode-control', 0x0204, 'ExampleW'], ['auto-exact-control', 0x0206, 'Example'], ['exactspelling-control', 0x0205, 'ExampleW'], ] as const; const results = []; for (const [label, mappingFlags, name] of cases) { const bytes = buildManagedPeFixture({ targetFramework: null, pinvoke: { mappingFlags, importName: 'Example', moduleName: 'example.dll' } }); const target = managedPeFixtureTarget(bytes); const inspection = inspectManagedNativeBoundariesBytes(bytes, target); const managed = createEvidence(target, { id: 'rea-dotnet-static', name: 'REA managed static analysis provider', version: '1' }, { operation: 'inspect_managed_native_boundaries', parameters: {}, result: inspection, rawResult: null, limitations: inspection.limitations, }); const native = createEvidence({ path: '/examples/example.dll', sha256: '8'.repeat(64), format: 'pe', architecture: 'x86_64' }, provider, { operation: 'analyze_function', parameters: {}, rawResult: null, result: functionDossierSchema.parse({ procedure: { address: '0x1000', name, classification: { external: false, thunk: false, thunk_target: null, provenance: 'synthetic-provider' }, body: { available: false, reason: 'Synthetic fixture' }, signature: null, locals: [] }, pseudocode: '', assembly: [], comments: [], callers: [], callees: [], incoming_references: [], outgoing_references: [], referenced_strings: [], referenced_names: [], basic_blocks: [], native_api: null, native_value_flow: null, limitations: [], }), }); const request = { managed_boundaries: managed, native_observations: [native] }; writeFileSync(`${work}/${label}.json`, JSON.stringify(request)); const run = spawnSync(process.execPath, [`${repo}/scripts/rea.mjs`, 'verify-managed-native-boundaries', `${work}/${label}.json`, '--json'], { cwd: repo, encoding: 'utf8', maxBuffer: 4 * 1024 * 1024, env: { PATH: `${process.execPath.slice(0, process.execPath.lastIndexOf('/'))}:/usr/bin:/bin`, HOME: `${work}/home`, XDG_CONFIG_HOME: `${work}/config`, XDG_CACHE_HOME: `${work}/cache`, XDG_DATA_HOME: `${work}/data`, XDG_STATE_HOME: `${work}/state`, REA_LOG_LEVEL: 'silent' }, }); writeFileSync(`${work}/${label}.stdout.json`, run.stdout); writeFileSync(`${work}/${label}.stderr.txt`, run.stderr); const parsed = JSON.parse(run.stdout); const normalized = parsed.normalized_result ?? parsed.result?.normalized_result ?? parsed.data?.normalized_result ?? parsed; const item = normalized.pinvoke_imports?.[0]; results.push({ label, exit: run.status, stderr: run.stderr, declaration: inspection.pinvoke_imports[0], sourceCoverage: inspection.coverage, result: item ?? parsed }); } writeFileSync(`${work}/charset-results.json`, JSON.stringify(results, null, 2)); console.log(JSON.stringify(results.map(({ label, exit, declaration, sourceCoverage, result }) => ({ label, exit, charset: declaration.char_set, no_mangle: declaration.no_mangle, coverage: sourceCoverage.state, status: result.status, basis: result.basis, candidates: result.candidates?.map((x: any) => x.name) })), null, 2)); ``` ### Expected behavior Retain `ExampleA` as an inferred candidate for the default charset, consistently with the explicit ANSI control. For supplied Windows PE context, retain `ExampleW` for `Auto`, consistently with the explicit Unicode control. Preserve the `no_mangle` / ExactSpelling restriction and keep this a static candidate inference, not proof of runtime binding. Microsoft documents that the default `CharSet.None` behaves like ANSI in [Charsets and marshalling](https://learn.microsoft.com/en-us/dotnet/standard/native-interop/charset), and describes the `A`/`W` name-matching alternatives when ExactSpelling is false in [Specifying a character set](https://learn.microsoft.com/en-us/dotnet/standard/native-interop/specifying-a-character-set). The first page also identifies Unicode as the Windows Auto behavior. REA's [managed/native contract](https://github.com/morluto/rea/blob/4526862728af01f1135714079c0d8a8c754eb364/docs/managed-code-analysis.md#managed-native-boundary-rules) limits these checks to supplied declaration/name Evidence. ### Impact and scope Default C# P/Invoke declarations and Auto declarations can lose supported Windows decorated-name correspondences. Observed in two synthetic cases, with the equivalent explicit-charset controls succeeding; occurrence in real assemblies was not measured. Inputs remain readable and no artifact is changed. Affected automated verification reports no candidate even though the supplied function is the conventional candidate for that charset. ### REA version Executed source/build: `4526862728af01f1135714079c0d8a8c754eb364`. The relevant `candidateNames` implementation is unchanged in latest published tag `rea-agents-6.7.0` (`27dc42d10249f0410904a75c0997842eb03b931c`); that tag's runtime was not executed separately. ### Runtime and platform Node.js `v24.22.1`; Linux x86_64. Windows PE module identities are supplied as synthetic native function Evidence. No external provider is required. ### Target and artifact context Byte-built managed PE/CLI P/Invoke metadata: entry point `Example`, module `example.dll`, cdecl, charset flag values `0x0000`, `0x0002`, `0x0004`, or `0x0006`. Native Evidence describes one synthetic function in `/examples/example.dll`. ### Interface CLI ### Earliest failure stage Native analysis or reconstruction ### Relevant output or evidence All six CLI calls exited 0 with empty stderr. Reduced output: ```json [ { "case": "default-ansi-suffix", "charset": "not-specified", "no_mangle": false, "exit": 0, "status": "unresolved", "basis": "no-native-candidate", "candidates": [] }, { "case": "explicit-ansi-control", "charset": "ansi", "no_mangle": false, "exit": 0, "status": "inferred", "basis": "decorated-name-candidate", "candidates": [ "ExampleA" ] }, { "case": "auto-unicode-suffix", "charset": "auto", "no_mangle": false, "exit": 0, "status": "unresolved", "basis": "no-native-candidate", "candidates": [] }, { "case": "explicit-unicode-control", "charset": "unicode", "no_mangle": false, "exit": 0, "status": "inferred", "basis": "decorated-name-candidate", "candidates": [ "ExampleW" ] }, { "case": "auto-exact-control", "charset": "auto", "no_mangle": false, "exit": 0, "status": "verified", "basis": "exact-function-name", "candidates": [ "Example" ] }, { "case": "exactspelling-control", "charset": "unicode", "no_mangle": true, "exit": 0, "status": "unresolved", "basis": "no-native-candidate", "candidates": [] } ] ``` The producer maps `0x0000` to `not-specified` and `0x0006` to `auto` in [ManagedNativeBoundaryHelpers.ts](https://github.com/morluto/rea/blob/4526862728af01f1135714079c0d8a8c754eb364/src/dotnet/ManagedNativeBoundaryHelpers.ts#L244-L256). Candidate generation tests only `unicode` and `ansi` in [managedNativeVerificationMatch.ts](https://github.com/morluto/rea/blob/4526862728af01f1135714079c0d8a8c754eb364/src/domain/managed/managedNativeVerificationMatch.ts#L250-L265). The supported CLI routes the request into this workflow in [managedCommands.ts](https://github.com/morluto/rea/blob/4526862728af01f1135714079c0d8a8c754eb364/src/cli/managedCommands.ts#L212-L234). ### Typed outcome or recovery expected `inferred / decorated-name-candidate`, retaining the corresponding `ExampleA` or `ExampleW` function Evidence. The ExactSpelling control must continue returning no suffixed candidate, and exact unsuffixed names must remain available. ### Analysis or suspected cause Source-derived: the parsed charset states are preserved correctly, but candidate enumeration does not account for the default ANSI equivalence or the Auto charset's Windows meaning. The explicit-charset controls isolate enumeration from module identity and Evidence validation. ### Additional context Fresh searches across open and closed issues for `P/Invoke charset`, `not-specified candidate`, and `Auto decorated` returned no related report. The open-issue snapshot was also checked. This concerns missing static decorated candidates, not execution, marshaling correctness, or loader authenticity. ### Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: Supported metadata states deterministically discard useful supplied native candidates in the executed CLI, while equivalent explicit charset states retain them; this is bounded static correspondence loss with no mutation. - Workaround status: unverified - Workaround basis: Manual comparison of the preserved declaration and native function names is plausible, but no separate supported automatic recovery preserving the original Evidence was verified.

bsakaguchi closed this issue

2026-10-31 00:47:37 +00:00

bsakaguchi commented

2026-10-31 00:47:37 +00:00

Author

Owner

@gensokyo-dev commented on GitHub (Oct 23, 2026):

Resolved on main by db923eba4. Verified default→ExampleA and Auto→ExampleW in PE context through the compiled CLI and stdio MCP using real .NET compiler metadata and export-backed test-supplied native Evidence. ExactSpelling suppression and exact-name controls were preserved. All 31 existing conformance/adapter tests passed.

These remain static candidate inferences; no live Windows binding or native provider body/ABI analysis was run. A separate real Linux .NET control confirms that default ELF declarations must not gain Windows suffix probing. The duplicate #6877 is closed; a distinct pre-existing explicit-charset ELF problem discovered during review is tracked at https://github.com/morluto/rea/issues/6926. Full verification and cost limits are recorded on #6877.

@gensokyo-dev commented on GitHub (Oct 23, 2026): Resolved on main by [db923eba4](https://github.com/morluto/rea/commit/db923eba44f8703708c380966f3a7498db6fdd69). Verified default→`ExampleA` and Auto→`ExampleW` in PE context through the compiled CLI and stdio MCP using real .NET compiler metadata and export-backed test-supplied native Evidence. ExactSpelling suppression and exact-name controls were preserved. All 31 existing conformance/adapter tests passed. These remain static candidate inferences; no live Windows binding or native provider body/ABI analysis was run. A separate real Linux .NET control confirms that default ELF declarations must not gain Windows suffix probing. The duplicate #6877 is closed; a distinct pre-existing explicit-charset ELF problem discovered during review is tracked at https://github.com/morluto/rea/issues/6926. Full verification and cost limits are recorded on #6877.

bsakaguchi referenced this issue

2026-10-31 01:12:07 +00:00

[PR #4220] [CLOSED] fix(native): map Go and Rust ppc reports to the powerpc subject #6133

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#4197

No description provided.

Rows
Columns