morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 07:47:29 +00:00

[GH-ISSUE #6716] [Bug] Go inline metadata is rejected when its strings extend beyond the named discovery section #4084

New issue

Closed

opened 2026-10-31 00:46:43 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:46:43 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 23, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6716

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Native analysis or decompilation

Observed behavior

The production readGoBinaryImage reader and GoBinaryProvider.inspect reject an ELF64 fixture with Go build-info varint is truncated when .go.buildinfo covers the complete 32-byte header and the inline strings continue within its file-backed load segment. The version and module bytes are present and mapped. Go 1.27.3's standard debug/buildinfo.ReadFile recovers go1.27.3 and main module example.com/app from the same file.

Changing only the section's sh_size from 32 to 124 makes REA recover both fields. The actual metadata and load mapping are identical. This was reproduced at production-module/provider boundaries; the public CLI and MCP were not invoked.

Reproduction

Use the stated source revision and its locked dependencies. Save this script as /tmp/rea-go-section-repro.mjs and run it from the repository root:

node --import tsx /tmp/rea-go-section-repro.mjs /tmp/rea-go-section

import { mkdirSync, writeFileSync } from 'node:fs';
import { pathToFileURL } from 'node:url';
const { readGoBinaryImage } = await import(pathToFileURL(`${process.cwd()}/src/go/GoBinaryImage.ts`));
const { GoBinaryProvider } = await import(pathToFileURL(`${process.cwd()}/src/go/GoBinaryProvider.ts`));
const root = process.argv[2];
mkdirSync(root, { recursive: true });
const v = Buffer.from('go1.27.3');
const m = Buffer.concat([
 Buffer.from('3077af0c9274080241e1c107e6d618e6', 'hex'),
 Buffer.from('path\texample.com/app\nmod\texample.com/app\t(devel)\t\n'),
 Buffer.from('f932433186182072008242104116d8f2', 'hex'),
]);
const data = Buffer.concat([
 Buffer.from('ff20476f206275696c64696e663a080200000000000000000000000000000000','hex'),
 Buffer.from([v.length]), v, Buffer.from([m.length]), m,
]);
for (const span of [data.length, 32]) {
 const b = Buffer.alloc(1024);
 b.set([0x7f,0x45,0x4c,0x46,2,1,1]);
 const u16 = (a,v) => b.writeUInt16LE(v,a);
 const u32 = (a,v) => b.writeUInt32LE(v,a);
 const u64 = (a,v) => b.writeBigUInt64LE(BigInt(v),a);
 u16(16,2); u16(18,62); u32(20,1); u64(32,64); u64(40,512);
 u16(52,64); u16(54,56); u16(56,1); u16(58,64); u16(60,3); u16(62,1);
 u32(64,1); u32(68,6); u64(72,0); u64(80,0x400000);
 u64(96,b.length); u64(104,b.length); u64(112,1);
 const names = Buffer.from('\0.shstrtab\0.go.buildinfo\0'); names.copy(b,800);
 u32(576,1); u32(580,3); u64(600,800); u64(608,names.length); u64(624,1);
 u32(640,11); u32(644,1); u64(648,3); u64(656,0x400100);
 u64(664,256); u64(672,span); u64(688,16);
 data.copy(b,256);
 const path = `${root}/section-${span}.elf`; writeFileSync(path,b);
 try { console.log(JSON.stringify({span,layer:'parser',go_version:readGoBinaryImage(b).build_info?.go_version})); }
 catch(e) { console.log(JSON.stringify({span,layer:'parser',kind:e.kind,message:e.message})); }
 const p = new GoBinaryProvider();
 try {
  const r = await p.inspect({path});
  console.log(JSON.stringify({span,layer:'provider',ok:r.ok,...(r.ok ? {
   go_version:r.value.build_info?.go_version,
   main:r.value.build_info?.module.main,
  } : {tag:r.error._tag,issues:r.error.issues})}));
 } finally { await p.close(); }
}

For the independent reader comparison, save this as /tmp/rea-go-oracle.go and run:

go run /tmp/rea-go-oracle.go /tmp/rea-go-section/section-124.elf /tmp/rea-go-section/section-32.elf

package main
import("debug/buildinfo"; "encoding/json"; "os")
func main(){ for _,path:=range os.Args[1:] { v,e:=buildinfo.ReadFile(path); r:=map[string]any{"file":path}; if e!=nil {r["error"]=e.Error()} else {r["go_version"]=v.GoVersion;r["path"]=v.Path;r["main"]=v.Main;r["deps"]=v.Deps;r["settings"]=v.Settings};json.NewEncoder(os.Stdout).Encode(r) } }

The complete version-and-module case above was run once through each REA boundary and once through the Go oracle. A preceding version-only reduction independently showed the same section-boundary failure. No reproduction was rerun after this draft was saved.

Expected behavior

Recover the same compiler and module facts from both files, while retaining each selected artifact's identity and enforcing the existing mapped-range and byte limits. A section used to discover the header should not truncate strings that remain inside the valid load mapping.

The documented supported boundary includes inline build-info encoding in ELF, and named-section priority describes discovery. Go's inline decoder and ELF reader read string contents by mapped address, separately from the header discovery span. The executed standard-reader comparison grounds this compatibility expectation.

Impact and scope

P2, supported: metadata inspection rejects this bounded ELF representation despite its complete mapped Go record. REA supplies neither compiler nor module results for the affected file. The input is never modified. The fixture is synthetic; prevalence in compiler-produced or rewritten executables is unknown. No Mach-O, PE, pointer encoding, public CLI, MCP or real-provider behavior is established by this probe.

The executed Go standard-reader helper is an available workaround for the demonstrated version and module facts. It reads the original file without rewriting it. Enlarging the section in an artifact copy is only a control, because that changes the artifact's bytes and digest.

REA version

Source revision 5d6b8a0ab693c03ab85cf0807d40bedf021e76d8, package version 6.7.0. Production TypeScript source was loaded with the checkout's locked tsx; no published npm-package runtime was executed.

Runtime and platform

Node.js v24.22.1, Linux x86_64. Oracle: Go 1.27.3 linux/amd64. No Erlang/Rust runtime, disassembler, native analysis provider or target execution was used.

Target and artifact context

Two synthetic 1024-byte little-endian x86-64 ELF64 files. One RW, non-executable PT_LOAD maps the complete file at 0x400000. The aligned Go header starts at file offset 256 / address 0x400100. Its inline version is go1.27.3; the correctly framed module body names example.com/app at (devel). Both files have identical metadata, program headers and section tables except for one byte of the .go.buildinfo section's 64-bit size field at file offset 672: 124 versus 32. These are format fixtures, not compiler-built executable programs.

Interface

Other

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

Executed production-boundary output, with the path omitted from the error text:

{"span":124,"layer":"parser","go_version":"go1.27.3"}
{"span":124,"layer":"provider","ok":true,"go_version":"go1.27.3","main":{"path":"example.com/app","version":"(devel)","sum":"","replacement":null}}
{"span":32,"layer":"parser","kind":"malformed","message":"Go build-info varint is truncated"}
{"span":32,"layer":"provider","ok":false,"tag":"AnalysisInputError"}

The provider's issue has path: ["path"] and reason: "invalid_format". The Go oracle returned the following facts for both files:

{"go_version":"go1.27.3","path":"example.com/app","main":{"Path":"example.com/app","Version":"(devel)"}}

A byte comparison reported exactly one differing byte, the size field identified above. Evidence provenance: observed production-module and provider execution plus an independent standard reader. Local verification: reproduced. Reproduction completeness: complete for these synthetic ELF cases; no public CLI/MCP or compiler-produced corpus coverage.

Typed outcome or recovery expected

Return successful metadata with the observed compiler string, module path and main module for the 32-byte discovery-section fixture. Continue to reject strings that leave valid file mappings or exceed the existing resource limits; those restoration checks were not run against repaired software.

Analysis or suspected cause

Source-derived: section discovery returns the named section extent. Inline decoding reuses that discovery extent as the available string-byte boundary. It therefore supplies zero remaining bytes at header + 32 even though the load mapping contains both strings. The production provider calls this reader before module parsing. The observations above match that source path; no fix was implemented or tested.

Additional context

The initial open-issue inventory was filtered for Go build-info, inline encoding, varints, section ranges and truncation. Related #5333 concerns PE discovery with VirtualSize == 0; #5076 concerns malformed Mach-O command alignment; neither describes this inline ELF extent failure. A fresh open-and-closed duplicate search was completed immediately before submission; no matching report was found. No last-known-good version is established.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: Both the production reader and provider reject the complete mapped inline record when only its named discovery section ends after the header; the standard Go reader recovers the same version and module as the larger-section control. Real-world frequency is unknown.
Workaround status: available
Workaround basis: The executed Go 1.27.3 standard-reader helper recovered the demonstrated compiler and module facts directly from the unchanged affected file; it requires a Go installation.

Originally created by @coygeek on GitHub (Oct 23, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6716 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Native analysis or decompilation ### Observed behavior The production `readGoBinaryImage` reader and `GoBinaryProvider.inspect` reject an ELF64 fixture with `Go build-info varint is truncated` when `.go.buildinfo` covers the complete 32-byte header and the inline strings continue within its file-backed load segment. The version and module bytes are present and mapped. Go 1.27.3's standard `debug/buildinfo.ReadFile` recovers `go1.27.3` and main module `example.com/app` from the same file. Changing only the section's `sh_size` from 32 to 124 makes REA recover both fields. The actual metadata and load mapping are identical. This was reproduced at production-module/provider boundaries; the public CLI and MCP were not invoked. ### Reproduction Use the stated source revision and its locked dependencies. Save this script as `/tmp/rea-go-section-repro.mjs` and run it from the repository root: ```sh node --import tsx /tmp/rea-go-section-repro.mjs /tmp/rea-go-section ``` ```js import { mkdirSync, writeFileSync } from 'node:fs'; import { pathToFileURL } from 'node:url'; const { readGoBinaryImage } = await import(pathToFileURL(`${process.cwd()}/src/go/GoBinaryImage.ts`)); const { GoBinaryProvider } = await import(pathToFileURL(`${process.cwd()}/src/go/GoBinaryProvider.ts`)); const root = process.argv[2]; mkdirSync(root, { recursive: true }); const v = Buffer.from('go1.27.3'); const m = Buffer.concat([ Buffer.from('3077af0c9274080241e1c107e6d618e6', 'hex'), Buffer.from('path\texample.com/app\nmod\texample.com/app\t(devel)\t\n'), Buffer.from('f932433186182072008242104116d8f2', 'hex'), ]); const data = Buffer.concat([ Buffer.from('ff20476f206275696c64696e663a080200000000000000000000000000000000','hex'), Buffer.from([v.length]), v, Buffer.from([m.length]), m, ]); for (const span of [data.length, 32]) { const b = Buffer.alloc(1024); b.set([0x7f,0x45,0x4c,0x46,2,1,1]); const u16 = (a,v) => b.writeUInt16LE(v,a); const u32 = (a,v) => b.writeUInt32LE(v,a); const u64 = (a,v) => b.writeBigUInt64LE(BigInt(v),a); u16(16,2); u16(18,62); u32(20,1); u64(32,64); u64(40,512); u16(52,64); u16(54,56); u16(56,1); u16(58,64); u16(60,3); u16(62,1); u32(64,1); u32(68,6); u64(72,0); u64(80,0x400000); u64(96,b.length); u64(104,b.length); u64(112,1); const names = Buffer.from('\0.shstrtab\0.go.buildinfo\0'); names.copy(b,800); u32(576,1); u32(580,3); u64(600,800); u64(608,names.length); u64(624,1); u32(640,11); u32(644,1); u64(648,3); u64(656,0x400100); u64(664,256); u64(672,span); u64(688,16); data.copy(b,256); const path = `${root}/section-${span}.elf`; writeFileSync(path,b); try { console.log(JSON.stringify({span,layer:'parser',go_version:readGoBinaryImage(b).build_info?.go_version})); } catch(e) { console.log(JSON.stringify({span,layer:'parser',kind:e.kind,message:e.message})); } const p = new GoBinaryProvider(); try { const r = await p.inspect({path}); console.log(JSON.stringify({span,layer:'provider',ok:r.ok,...(r.ok ? { go_version:r.value.build_info?.go_version, main:r.value.build_info?.module.main, } : {tag:r.error._tag,issues:r.error.issues})})); } finally { await p.close(); } } ``` For the independent reader comparison, save this as `/tmp/rea-go-oracle.go` and run: ```sh go run /tmp/rea-go-oracle.go /tmp/rea-go-section/section-124.elf /tmp/rea-go-section/section-32.elf ``` ```go package main import("debug/buildinfo"; "encoding/json"; "os") func main(){ for _,path:=range os.Args[1:] { v,e:=buildinfo.ReadFile(path); r:=map[string]any{"file":path}; if e!=nil {r["error"]=e.Error()} else {r["go_version"]=v.GoVersion;r["path"]=v.Path;r["main"]=v.Main;r["deps"]=v.Deps;r["settings"]=v.Settings};json.NewEncoder(os.Stdout).Encode(r) } } ``` The complete version-and-module case above was run once through each REA boundary and once through the Go oracle. A preceding version-only reduction independently showed the same section-boundary failure. No reproduction was rerun after this draft was saved. ### Expected behavior Recover the same compiler and module facts from both files, while retaining each selected artifact's identity and enforcing the existing mapped-range and byte limits. A section used to discover the header should not truncate strings that remain inside the valid load mapping. The [documented supported boundary](https://github.com/morluto/rea/blob/5d6b8a0ab693c03ab85cf0807d40bedf021e76d8/docs/go-binaries.md#L112-L117) includes inline build-info encoding in ELF, and [named-section priority](https://github.com/morluto/rea/blob/5d6b8a0ab693c03ab85cf0807d40bedf021e76d8/docs/go-binaries.md#L129-L133) describes discovery. Go's [inline decoder](https://github.com/golang/go/blob/go1.27.3/src/debug/buildinfo/buildinfo.go#L229-L237) and [ELF reader](https://github.com/golang/go/blob/go1.27.3/src/debug/buildinfo/buildinfo.go#L445-L459) read string contents by mapped address, separately from the header discovery span. The executed standard-reader comparison grounds this compatibility expectation. ### Impact and scope P2, supported: metadata inspection rejects this bounded ELF representation despite its complete mapped Go record. REA supplies neither compiler nor module results for the affected file. The input is never modified. The fixture is synthetic; prevalence in compiler-produced or rewritten executables is unknown. No Mach-O, PE, pointer encoding, public CLI, MCP or real-provider behavior is established by this probe. The executed Go standard-reader helper is an available workaround for the demonstrated version and module facts. It reads the original file without rewriting it. Enlarging the section in an artifact copy is only a control, because that changes the artifact's bytes and digest. ### REA version Source revision [`5d6b8a0ab693c03ab85cf0807d40bedf021e76d8`](https://github.com/morluto/rea/commit/5d6b8a0ab693c03ab85cf0807d40bedf021e76d8), package version 6.7.0. Production TypeScript source was loaded with the checkout's locked `tsx`; no published npm-package runtime was executed. ### Runtime and platform Node.js v24.22.1, Linux x86_64. Oracle: Go 1.27.3 linux/amd64. No Erlang/Rust runtime, disassembler, native analysis provider or target execution was used. ### Target and artifact context Two synthetic 1024-byte little-endian x86-64 ELF64 files. One RW, non-executable `PT_LOAD` maps the complete file at `0x400000`. The aligned Go header starts at file offset 256 / address `0x400100`. Its inline version is `go1.27.3`; the correctly framed module body names `example.com/app` at `(devel)`. Both files have identical metadata, program headers and section tables except for one byte of the `.go.buildinfo` section's 64-bit size field at file offset 672: 124 versus 32. These are format fixtures, not compiler-built executable programs. ### Interface Other ### Earliest failure stage Native analysis or reconstruction ### Relevant output or evidence Executed production-boundary output, with the path omitted from the error text: ```json {"span":124,"layer":"parser","go_version":"go1.27.3"} {"span":124,"layer":"provider","ok":true,"go_version":"go1.27.3","main":{"path":"example.com/app","version":"(devel)","sum":"","replacement":null}} {"span":32,"layer":"parser","kind":"malformed","message":"Go build-info varint is truncated"} {"span":32,"layer":"provider","ok":false,"tag":"AnalysisInputError"} ``` The provider's issue has `path: ["path"]` and `reason: "invalid_format"`. The Go oracle returned the following facts for **both** files: ```json {"go_version":"go1.27.3","path":"example.com/app","main":{"Path":"example.com/app","Version":"(devel)"}} ``` A byte comparison reported exactly one differing byte, the size field identified above. Evidence provenance: observed production-module and provider execution plus an independent standard reader. Local verification: reproduced. Reproduction completeness: complete for these synthetic ELF cases; no public CLI/MCP or compiler-produced corpus coverage. ### Typed outcome or recovery expected Return successful metadata with the observed compiler string, module path and main module for the 32-byte discovery-section fixture. Continue to reject strings that leave valid file mappings or exceed the existing resource limits; those restoration checks were not run against repaired software. ### Analysis or suspected cause Source-derived: [section discovery](https://github.com/morluto/rea/blob/5d6b8a0ab693c03ab85cf0807d40bedf021e76d8/src/go/GoBuildInfoSearch.ts#L69-L89) returns the named section extent. [Inline decoding](https://github.com/morluto/rea/blob/5d6b8a0ab693c03ab85cf0807d40bedf021e76d8/src/go/GoBinaryImage.ts#L104-L114) reuses that discovery extent as the available string-byte boundary. It therefore supplies zero remaining bytes at `header + 32` even though the load mapping contains both strings. The [production provider](https://github.com/morluto/rea/blob/5d6b8a0ab693c03ab85cf0807d40bedf021e76d8/src/go/GoBinaryProvider.ts#L240-L274) calls this reader before module parsing. The observations above match that source path; no fix was implemented or tested. ### Additional context The initial open-issue inventory was filtered for Go build-info, inline encoding, varints, section ranges and truncation. Related #5333 concerns PE discovery with `VirtualSize == 0`; #5076 concerns malformed Mach-O command alignment; neither describes this inline ELF extent failure. A fresh open-and-closed duplicate search was completed immediately before submission; no matching report was found. No last-known-good version is established. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: Both the production reader and provider reject the complete mapped inline record when only its named discovery section ends after the header; the standard Go reader recovers the same version and module as the larger-section control. Real-world frequency is unknown. - Workaround status: available - Workaround basis: The executed Go 1.27.3 standard-reader helper recovered the demonstrated compiler and module facts directly from the unchanged affected file; it requires a Go installation.

bsakaguchi closed this issue

2026-10-31 00:46:43 +00:00

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#4084

No description provided.

Rows
Columns