morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #6647] [Bug] ELF export inventory renders UTF-8 names as Latin-1 without encoding metadata #4025

New issue

Closed

opened 2026-10-31 00:46:09 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:46:09 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 23, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6647

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Native analysis or decompilation

Observed behavior

The production NativeExportsService.inspect boundary reports a GCC-built ELF export named café as cafÃ©. The ASCII control plain is unchanged. The result says coverage is complete, has no issues, and gives no encoding or raw-byte field explaining the name mapping.

This is an executed source-service reproduction. CLI and MCP transport were not exercised.

Reproduction

From this source checkout with locked dependencies installed, build this harmless shared library; it is inspected without being loaded or executed:

cat > /tmp/rea-unicode-export.c <<'C'
int café(void) { return 7; }
int plain(void) { return 8; }
C
gcc -shared -fPIC -finput-charset=UTF-8 -o /tmp/rea-unicode-export.so /tmp/rea-unicode-export.c
nm -D --defined-only /tmp/rea-unicode-export.so
node --import tsx --input-type=module <<'JS'
import { NativeExportsProvider } from './src/native/exports/NativeExportsProvider.ts';
import { NativeExportsService } from './src/application/binaryDiagnostics/NativeExportsService.ts';
const provider = new NativeExportsProvider();
try {
  const result = await new NativeExportsService(provider).inspect({path:'/tmp/rea-unicode-export.so'});
  console.log(JSON.stringify(result.ok ? result.value.normalized_result : result.error));
} finally {
  await provider.close();
}
JS

The investigation ran the same C contents and service call once, from external scratch files. Paths and import placement above are normalized for a checkout-local reproduction.

Expected behavior

For a valid UTF-8 symbol, expose its correct decoded spelling, or explicitly identify the name as a byte-preserving Latin-1 representation and provide unambiguous raw bytes/encoding. A plain JSON name should not silently present the mapped bytes as the original Unicode symbol.

The export contract says rows reproduce defined dynamic symbols. The fixture's nm output and independently compared UTF-8 bytes establish the discrepancy. ELF string tables do not universally require UTF-8; this report does not propose replacing arbitrary invalid bytes with U+FFFD.

Impact and scope

An analyst or consumer using the returned name as ordinary UTF-8 gets a different symbol identity: source café encodes as 636166c3a9, while reported cafÃ© encodes as 636166c383c2a9. This affects valid non-ASCII UTF-8 export names; only this two-export Linux fixture was tested. Addresses and ASCII names remain correct, and the source file is unchanged.

For this known-UTF-8 input, Buffer.from(name, 'latin1').toString('utf8') recovers café; that recovery was checked before drafting. It requires outside knowledge of the source encoding.

REA version

Source revision 23def04096869b85c4d3c14a73ed09168483c9f6 on main; package version 6.7.0. The separately published 6.7.0 package was not reproduced.

Runtime and platform

Node.js v24.22.1; Linux x86_64; GCC 16.2.1; GNU nm 2.47. No native-analysis engine was used.

Target and artifact context

A locally compiled ELF64 x86-64 shared library with two defined functions. The compiler received UTF-8 source explicitly. No functions were called.

Interface

Other

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

Observed nm -D --defined-only output:

00000000000010e9 T café
00000000000010f4 T plain

Selected source-service result:

{
  "exports": [
    {"name":"plain","value":"0x10f4","size":"11","type":"function","binding":"global","version":null},
    {"name":"cafÃ©","value":"0x10e9","size":"11","type":"function","binding":"global","version":null}
  ],
  "coverage":{"status":"complete","reason":null,"examined_entries":6,"exports":2},
  "issues":[]
}

The input was 15,176 bytes. The output limitations discuss export reachability and version lookup, not text encoding. The byte comparison and recovery above were executed separately before this draft. No reproduction was rerun after drafting.

Typed outcome or recovery expected

Correct UTF-8 display text for this input, with explicit byte identity/encoding for names whose text interpretation is not established.

Analysis or suspected cause

Source-derived: readElfString always decodes string-table bytes with latin1. The export parser publishes that string directly as name. The mapping preserves bytes internally but does not disclose how consumers should recover them.

Additional context

Open and closed searches covered mojibake, exports/Unicode, exports/UTF, ELF/Latin-1, and Latin/symbol terms. No matching issue was found. No last-known-good version is established.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: The executed production service publishes a different ordinary UTF-8 symbol spelling for a valid GCC-produced non-ASCII export while claiming complete coverage; ASCII names and addresses remain correct.
Workaround status: available
Workaround basis: For the demonstrated known-UTF-8 input, decoding the reported Latin-1 byte mapping as UTF-8 recovers the original name; this was directly checked before drafting.

Originally created by @coygeek on GitHub (Oct 23, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6647 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Native analysis or decompilation ### Observed behavior The production `NativeExportsService.inspect` boundary reports a GCC-built ELF export named `café` as `cafÃ©`. The ASCII control `plain` is unchanged. The result says coverage is complete, has no issues, and gives no encoding or raw-byte field explaining the name mapping. This is an executed source-service reproduction. CLI and MCP transport were not exercised. ### Reproduction From this source checkout with locked dependencies installed, build this harmless shared library; it is inspected without being loaded or executed: ```sh cat > /tmp/rea-unicode-export.c <<'C' int café(void) { return 7; } int plain(void) { return 8; } C gcc -shared -fPIC -finput-charset=UTF-8 -o /tmp/rea-unicode-export.so /tmp/rea-unicode-export.c nm -D --defined-only /tmp/rea-unicode-export.so node --import tsx --input-type=module <<'JS' import { NativeExportsProvider } from './src/native/exports/NativeExportsProvider.ts'; import { NativeExportsService } from './src/application/binaryDiagnostics/NativeExportsService.ts'; const provider = new NativeExportsProvider(); try { const result = await new NativeExportsService(provider).inspect({path:'/tmp/rea-unicode-export.so'}); console.log(JSON.stringify(result.ok ? result.value.normalized_result : result.error)); } finally { await provider.close(); } JS ``` The investigation ran the same C contents and service call once, from external scratch files. Paths and import placement above are normalized for a checkout-local reproduction. ### Expected behavior For a valid UTF-8 symbol, expose its correct decoded spelling, or explicitly identify the name as a byte-preserving Latin-1 representation and provide unambiguous raw bytes/encoding. A plain JSON `name` should not silently present the mapped bytes as the original Unicode symbol. The [export contract](https://github.com/morluto/rea/blob/23def04096869b85c4d3c14a73ed09168483c9f6/docs/binary-diagnostics.md#L332-L340) says rows reproduce defined dynamic symbols. The fixture's `nm` output and independently compared UTF-8 bytes establish the discrepancy. ELF string tables do not universally require UTF-8; this report does not propose replacing arbitrary invalid bytes with U+FFFD. ### Impact and scope An analyst or consumer using the returned name as ordinary UTF-8 gets a different symbol identity: source `café` encodes as `636166c3a9`, while reported `cafÃ©` encodes as `636166c383c2a9`. This affects valid non-ASCII UTF-8 export names; only this two-export Linux fixture was tested. Addresses and ASCII names remain correct, and the source file is unchanged. For this known-UTF-8 input, `Buffer.from(name, 'latin1').toString('utf8')` recovers `café`; that recovery was checked before drafting. It requires outside knowledge of the source encoding. ### REA version Source revision `23def04096869b85c4d3c14a73ed09168483c9f6` on main; package version `6.7.0`. The separately published 6.7.0 package was not reproduced. ### Runtime and platform Node.js v24.22.1; Linux x86_64; GCC 16.2.1; GNU nm 2.47. No native-analysis engine was used. ### Target and artifact context A locally compiled ELF64 x86-64 shared library with two defined functions. The compiler received UTF-8 source explicitly. No functions were called. ### Interface Other ### Earliest failure stage Native analysis or reconstruction ### Relevant output or evidence Observed `nm -D --defined-only` output: ```text 00000000000010e9 T café 00000000000010f4 T plain ``` Selected source-service result: ```json { "exports": [ {"name":"plain","value":"0x10f4","size":"11","type":"function","binding":"global","version":null}, {"name":"cafÃ©","value":"0x10e9","size":"11","type":"function","binding":"global","version":null} ], "coverage":{"status":"complete","reason":null,"examined_entries":6,"exports":2}, "issues":[] } ``` The input was 15,176 bytes. The output limitations discuss export reachability and version lookup, not text encoding. The byte comparison and recovery above were executed separately before this draft. No reproduction was rerun after drafting. ### Typed outcome or recovery expected Correct UTF-8 display text for this input, with explicit byte identity/encoding for names whose text interpretation is not established. ### Analysis or suspected cause Source-derived: [`readElfString`](https://github.com/morluto/rea/blob/23def04096869b85c4d3c14a73ed09168483c9f6/src/native/metadata/ElfSymbol.ts#L21-L35) always decodes string-table bytes with `latin1`. [The export parser](https://github.com/morluto/rea/blob/23def04096869b85c4d3c14a73ed09168483c9f6/src/native/exports/NativeExportParser.ts#L563-L592) publishes that string directly as `name`. The mapping preserves bytes internally but does not disclose how consumers should recover them. ### Additional context Open and closed searches covered mojibake, exports/Unicode, exports/UTF, ELF/Latin-1, and Latin/symbol terms. No matching issue was found. No last-known-good version is established. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: The executed production service publishes a different ordinary UTF-8 symbol spelling for a valid GCC-produced non-ASCII export while claiming complete coverage; ASCII names and addresses remain correct. - Workaround status: available - Workaround basis: For the demonstrated known-UTF-8 input, decoding the reported Latin-1 byte mapping as UTF-8 recovers the original name; this was directly checked before drafting.

bsakaguchi closed this issue

2026-10-31 00:46:09 +00:00

bsakaguchi referenced this issue

2026-10-31 01:11:18 +00:00

[PR #4064] [CLOSED] fix(mcp): advertise boolean literals as typed booleans #6029

bsakaguchi referenced this issue

2026-10-31 01:11:27 +00:00

[PR #4087] [CLOSED] fix(mcp): normalize advertised boolean literals for Gemini compatibility (#4025) #6047

bsakaguchi referenced this issue

2026-11-01 21:43:12 +00:00

[PR #5203] [CLOSED] fix(mcp): advertise empty-string literals as typed strings #6811