morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #6779] [Bug] Mach-O Go inline metadata is rejected when strings extend beyond the header discovery section #4133

New issue

Closed

opened 2026-10-31 00:47:03 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:47:03 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 23, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6779

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Native analysis or decompilation

Observed behavior

inspect-go-binary rejects an authored Mach-O containing a complete mapped inline Go build-info record when __go_buildinfo declares only the 32-byte header. The CLI exits 1 with invalid_request, reason invalid_format, and Go build-info varint is truncated. Go 1.27.3's go version -m reads the compiler, package, main module and build setting from the same file without executing it.

Changing only the section-size byte at file offset 144 from 32 to 150 makes REA recover the metadata. Both 4096-byte fixtures otherwise have identical bytes and segment mappings. Each fixture was inspected once by each public command before drafting.

Reproduction

Use the source revision below with its locked dependencies and compiled runtime (npm ci, then npm run build:cached) and Node 24.22.1 plus Go 1.27.3 on PATH. Run this self-contained version of the executed probe from the checkout root. It creates inert fixtures and isolated configuration under /tmp; neither command executes the target.

python3 - <<'PY_REPRO'
import json,os,pathlib,subprocess,struct,tempfile
root=pathlib.Path(tempfile.mkdtemp(prefix='rea-go-macho-',dir='/tmp'))
for name in ('home','config','cache'):(root/name).mkdir()
repo=pathlib.Path.cwd()
node='node'
body=b'path\texample.invalid/app\nmod\texample.invalid/app\t(devel)\t\nbuild\tGOOS=darwin\n'
module=bytes.fromhex('3077af0c9274080241e1c107e6d618e6')+body+bytes.fromhex('f932433186182072008242104116d8f2')
version=b'go1.27.3'
def varint(n):
 b=bytearray()
 while n>127:b.append((n&127)|128);n>>=7
 b.append(n);return b
payload=b'\xff Go buildinf:'+bytes([8,2])+bytes(16)+varint(len(version))+version+varint(len(module))+module
for name,size in [('complete',len(payload)),('header-only',32)]:
 b=bytearray(4096)
 struct.pack_into('<8I',b,0,0xfeedfacf,0x1000007,3,2,1,152,0,0)
 struct.pack_into('<II16sQQQQIIII',b,32,0x19,152,b'__DATA',0x10000,4096,0,4096,3,3,1,0)
 struct.pack_into('<16s16sQQIIIIIIII',b,104,b'__go_buildinfo',b'__DATA',0x10400,size,1024,4,0,0,0,0,0,0)
 b[1024:1024+len(payload)]=payload
 path=root/(name+'.macho');path.write_bytes(b)
 env=os.environ.copy();env.update(HOME=str(root/'home'),XDG_CONFIG_HOME=str(root/'config'),XDG_CACHE_HOME=str(root/'cache'))
 for reader,args in [('rea',[node,str(repo/'scripts/rea.mjs'),'inspect-go-binary',str(path),'--json']),('go',['go','version','-m',str(path)])]:
  p=subprocess.run(args,cwd=repo,env=env,capture_output=True,text=True,timeout=60)
  if reader=='rea':
   d=json.loads(p.stdout);r=d.get('normalized_result');out={'go_version':r['build_info']['go_version'],'module':r['build_info']['module']['path']} if r else d
  else:out=p.stdout
  print(json.dumps({'case':name,'reader':reader,'exit':p.returncode,'result':out}))
PY_REPRO

Expected behavior

Recover the same compiler and module facts from both files while preserving the selected artifact identity and existing file-mapping and string-size checks. The named discovery section should identify the header; the inline strings remain available within the mapped segment.

The REA supported boundary includes inline Go metadata in thin Mach-O. Go's inline decoder reads strings separately from header discovery, and its Mach-O reader makes the enclosing segment available. The executed standard-reader comparison grounds this compatibility expectation.

Impact and scope

The public CLI rejects all metadata for this bounded Mach-O representation, although the standard Go reader recovers it. The inputs are synthetic format fixtures, not compiler-built executable programs. Prevalence in compiler-produced or rewritten binaries, other platforms and MCP transport were not measured. No target was executed or modified.

go version -m is an observed workaround for the demonstrated compiler/module/settings facts, requiring a Go installation. Enlarging the section size is only a control because it changes the artifact and digest.

REA version

Source build at 98ca6c82705bf8c44f814620636852f938d827ca; manifest and public --version both report 6.7.0. This is a built source checkout, not an execution of the published npm package.

Runtime and platform

Node.js v24.22.1; Linux x86_64 (Linux 7.2.8-9-omarchy). Independent reader: Go 1.27.3 linux/amd64. No disassembler or native analysis provider was involved.

Target and artifact context

Two authored 4096-byte little-endian x86-64 thin Mach-O files. One RW LC_SEGMENT_64 maps the complete file at 0x10000. The aligned Go header starts at offset 1024 / address 0x10400. Its version is go1.27.3, and its correctly framed module body identifies example.invalid/app at (devel) with GOOS=darwin. Only the __DATA,__go_buildinfo section size differs: 150 versus 32 bytes. All 150 bytes remain file-backed in the same segment.

Interface

CLI

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

Executed public-command results, reduced to the relevant fields:

complete (section size 150): REA exit 0, go_version=go1.27.3, module.path=example.invalid/app
header-only (section size 32): REA exit 1, invalid_request / invalid_format / Go build-info varint is truncated
both files: go version -m exit 0, go1.27.3; path example.invalid/app; mod example.invalid/app (devel); build GOOS=darwin
byte comparison: exactly one differing file offset, 144

Evidence provenance: observed public CLI execution and independent standard-reader comparison. Local verification: reproduced. Reproduction completeness: complete for these synthetic Mach-O cases. No compiler-produced trigger, MCP call or repaired implementation was tested. No reproducer was rerun after this draft was saved.

Typed outcome or recovery expected

Successful observed Go metadata for the original header-only fixture, with its own artifact digest and exact source offsets. Continue rejecting strings that leave the valid mapping or exceed supported limits. Those are proposed restoration checks, not executed fix verification.

Analysis or suspected cause

Source-derived: Mach-O discovery returns the named section extent. Inline decoding uses inlineDataEnd, whose non-ELF branch stops at that discovery extent. It supplies zero available string bytes at header + 32 even though the same mapped segment contains the complete record.

Additional context

Related closed #6716 demonstrated the analogous ELF failure and explicitly stated that Mach-O was untested. Current code extends inline reads into ELF load mappings only; this report demonstrates the remaining Mach-O case through the public CLI. The report can be treated as a scoped follow-up to that issue rather than a new diagnosis of its ELF case. Open #6763 concerns the separate near-1-MiB ELF string-budget boundary; this fixture's complete record is only 150 bytes. Open #5333 concerns zero-VirtualSize PE discovery.

The saved open-issue inventory and fresh GitHub open-and-closed searches for "Go" "Mach-O" "truncated", "build-info" "section", and "__go_buildinfo" found #6716 but no dedicated Mach-O report. No last-known-good version is established.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: The public metadata CLI rejects a complete mapped 150-byte Go record when the named discovery section declares only its full header; the otherwise identical control and independent Go reader recover the expected metadata. Real-world frequency is unknown.
Workaround status: available
Workaround basis: Executed go version -m recovered the demonstrated compiler, module and build-setting facts directly from the unchanged affected file, requiring a Go installation.

Originally created by @coygeek on GitHub (Oct 23, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6779 ## Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ## Area Native analysis or decompilation ## Observed behavior `inspect-go-binary` rejects an authored Mach-O containing a complete mapped inline Go build-info record when `__go_buildinfo` declares only the 32-byte header. The CLI exits 1 with `invalid_request`, reason `invalid_format`, and `Go build-info varint is truncated`. Go 1.27.3's `go version -m` reads the compiler, package, main module and build setting from the same file without executing it. Changing only the section-size byte at file offset 144 from 32 to 150 makes REA recover the metadata. Both 4096-byte fixtures otherwise have identical bytes and segment mappings. Each fixture was inspected once by each public command before drafting. ## Reproduction Use the source revision below with its locked dependencies and compiled runtime (`npm ci`, then `npm run build:cached`) and Node 24.22.1 plus Go 1.27.3 on `PATH`. Run this self-contained version of the executed probe from the checkout root. It creates inert fixtures and isolated configuration under `/tmp`; neither command executes the target. ```sh python3 - <<'PY_REPRO' import json,os,pathlib,subprocess,struct,tempfile root=pathlib.Path(tempfile.mkdtemp(prefix='rea-go-macho-',dir='/tmp')) for name in ('home','config','cache'):(root/name).mkdir() repo=pathlib.Path.cwd() node='node' body=b'path\texample.invalid/app\nmod\texample.invalid/app\t(devel)\t\nbuild\tGOOS=darwin\n' module=bytes.fromhex('3077af0c9274080241e1c107e6d618e6')+body+bytes.fromhex('f932433186182072008242104116d8f2') version=b'go1.27.3' def varint(n): b=bytearray() while n>127:b.append((n&127)|128);n>>=7 b.append(n);return b payload=b'\xff Go buildinf:'+bytes([8,2])+bytes(16)+varint(len(version))+version+varint(len(module))+module for name,size in [('complete',len(payload)),('header-only',32)]: b=bytearray(4096) struct.pack_into('<8I',b,0,0xfeedfacf,0x1000007,3,2,1,152,0,0) struct.pack_into('<II16sQQQQIIII',b,32,0x19,152,b'__DATA',0x10000,4096,0,4096,3,3,1,0) struct.pack_into('<16s16sQQIIIIIIII',b,104,b'__go_buildinfo',b'__DATA',0x10400,size,1024,4,0,0,0,0,0,0) b[1024:1024+len(payload)]=payload path=root/(name+'.macho');path.write_bytes(b) env=os.environ.copy();env.update(HOME=str(root/'home'),XDG_CONFIG_HOME=str(root/'config'),XDG_CACHE_HOME=str(root/'cache')) for reader,args in [('rea',[node,str(repo/'scripts/rea.mjs'),'inspect-go-binary',str(path),'--json']),('go',['go','version','-m',str(path)])]: p=subprocess.run(args,cwd=repo,env=env,capture_output=True,text=True,timeout=60) if reader=='rea': d=json.loads(p.stdout);r=d.get('normalized_result');out={'go_version':r['build_info']['go_version'],'module':r['build_info']['module']['path']} if r else d else:out=p.stdout print(json.dumps({'case':name,'reader':reader,'exit':p.returncode,'result':out})) PY_REPRO ``` ## Expected behavior Recover the same compiler and module facts from both files while preserving the selected artifact identity and existing file-mapping and string-size checks. The named discovery section should identify the header; the inline strings remain available within the mapped segment. The [REA supported boundary](https://github.com/morluto/rea/blob/98ca6c82705bf8c44f814620636852f938d827ca/docs/go-binaries.md#L112-L117) includes inline Go metadata in thin Mach-O. Go's [inline decoder](https://github.com/golang/go/blob/go1.27.3/src/debug/buildinfo/buildinfo.go#L229-L237) reads strings separately from header discovery, and its [Mach-O reader](https://github.com/golang/go/blob/go1.27.3/src/debug/buildinfo/buildinfo.go#L515-L541) makes the enclosing segment available. The executed standard-reader comparison grounds this compatibility expectation. ## Impact and scope The public CLI rejects all metadata for this bounded Mach-O representation, although the standard Go reader recovers it. The inputs are synthetic format fixtures, not compiler-built executable programs. Prevalence in compiler-produced or rewritten binaries, other platforms and MCP transport were not measured. No target was executed or modified. `go version -m` is an observed workaround for the demonstrated compiler/module/settings facts, requiring a Go installation. Enlarging the section size is only a control because it changes the artifact and digest. ## REA version Source build at [`98ca6c82705bf8c44f814620636852f938d827ca`](https://github.com/morluto/rea/commit/98ca6c82705bf8c44f814620636852f938d827ca); manifest and public `--version` both report 6.7.0. This is a built source checkout, not an execution of the published npm package. ## Runtime and platform Node.js v24.22.1; Linux x86_64 (`Linux 7.2.8-9-omarchy`). Independent reader: Go 1.27.3 linux/amd64. No disassembler or native analysis provider was involved. ## Target and artifact context Two authored 4096-byte little-endian x86-64 thin Mach-O files. One RW `LC_SEGMENT_64` maps the complete file at `0x10000`. The aligned Go header starts at offset 1024 / address `0x10400`. Its version is `go1.27.3`, and its correctly framed module body identifies `example.invalid/app` at `(devel)` with `GOOS=darwin`. Only the `__DATA,__go_buildinfo` section size differs: 150 versus 32 bytes. All 150 bytes remain file-backed in the same segment. ## Interface CLI ## Earliest failure stage Native analysis or reconstruction ## Relevant output or evidence Executed public-command results, reduced to the relevant fields: ```text complete (section size 150): REA exit 0, go_version=go1.27.3, module.path=example.invalid/app header-only (section size 32): REA exit 1, invalid_request / invalid_format / Go build-info varint is truncated both files: go version -m exit 0, go1.27.3; path example.invalid/app; mod example.invalid/app (devel); build GOOS=darwin byte comparison: exactly one differing file offset, 144 ``` Evidence provenance: observed public CLI execution and independent standard-reader comparison. Local verification: reproduced. Reproduction completeness: complete for these synthetic Mach-O cases. No compiler-produced trigger, MCP call or repaired implementation was tested. No reproducer was rerun after this draft was saved. ## Typed outcome or recovery expected Successful observed Go metadata for the original header-only fixture, with its own artifact digest and exact source offsets. Continue rejecting strings that leave the valid mapping or exceed supported limits. Those are proposed restoration checks, not executed fix verification. ## Analysis or suspected cause Source-derived: [Mach-O discovery](https://github.com/morluto/rea/blob/98ca6c82705bf8c44f814620636852f938d827ca/src/go/GoBuildInfoSearch.ts#L90-L128) returns the named section extent. [Inline decoding](https://github.com/morluto/rea/blob/98ca6c82705bf8c44f814620636852f938d827ca/src/go/GoBinaryImage.ts#L121-L126) uses `inlineDataEnd`, whose [non-ELF branch](https://github.com/morluto/rea/blob/98ca6c82705bf8c44f814620636852f938d827ca/src/go/GoBinaryImage.ts#L196-L216) stops at that discovery extent. It supplies zero available string bytes at `header + 32` even though the same mapped segment contains the complete record. ## Additional context Related closed [#6716](https://github.com/morluto/rea/issues/6716) demonstrated the analogous ELF failure and explicitly stated that Mach-O was untested. Current code extends inline reads into ELF load mappings only; this report demonstrates the remaining Mach-O case through the public CLI. The report can be treated as a scoped follow-up to that issue rather than a new diagnosis of its ELF case. Open [#6763](https://github.com/morluto/rea/issues/6763) concerns the separate near-1-MiB ELF string-budget boundary; this fixture's complete record is only 150 bytes. Open #5333 concerns zero-`VirtualSize` PE discovery. The saved open-issue inventory and fresh GitHub open-and-closed searches for `"Go" "Mach-O" "truncated"`, `"build-info" "section"`, and `"__go_buildinfo"` found #6716 but no dedicated Mach-O report. No last-known-good version is established. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: The public metadata CLI rejects a complete mapped 150-byte Go record when the named discovery section declares only its full header; the otherwise identical control and independent Go reader recover the expected metadata. Real-world frequency is unknown. - Workaround status: available - Workaround basis: Executed `go version -m` recovered the demonstrated compiler, module and build-setting facts directly from the unchanged affected file, requiring a Go installation.

bsakaguchi closed this issue

2026-10-31 00:47:03 +00:00

bsakaguchi referenced this issue

2026-10-31 01:11:41 +00:00

[PR #4133] feat(evidence): cluster inspected artifacts into families by root digest #6077

bsakaguchi referenced this issue

2026-10-31 01:13:35 +00:00

[PR #4589] feat(evidence): cluster artifacts from retained Evidence references and summaries #6325

bsakaguchi referenced this issue

2026-11-01 21:10:57 +00:00

[GH-ISSUE #3796] [Tracker] Open feature requests by area (October 2026) #1471

bsakaguchi referenced this issue