[GH-ISSUE #2167] fix(go): inspect-go-binary rejects valid Mach-O binaries with unmapped debug segments #712

New issue

Closed

opened 2026-10-15 18:27:38 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-15 18:27:38 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 12, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/2167

Summary

The built-in Go metadata inspector rejects compiler-produced thin Mach-O executables that contain an unmapped __DWARF segment. Both inspect-go-binary and MCP inspect_go_binary return an input-format error instead of compiler, module and build-setting Evidence for the selected file. Default Darwin arm64 and amd64 builds reproduced this in source. The latest tested package does not yet expose this capability, so this report concerns the source implementation.

Runtime or environment

Source revision c22111eca2, using its prepared compiled CLI/MCP runtime. Separately tested published package rea-agents 6.4.0. Node v24.21.0 satisfies the package engines range, but differs from the checkout's .nvmrc contributor pin of 24.18.0. Host macOS 27.0.1 arm64. Fixture compiler Go go1.27.1 darwin/arm64, with CGO disabled and no external dependencies. No Hopper, Ghidra, IDA or active binary target is required.

Area: native analysis or decompilation. Interfaces: CLI and MCP. Earliest failure stage: capture or observation, when admitting the selected native container. Target: compiler-produced thin Mach-O executable with Go inline build information and a non-mapped debug segment. Impact is compatibility and missing metadata; the inspection leaves the artifact unchanged.

Steps to reproduce

Use the source-built rea from the stated baseline and an existing Go compiler. Create this local module. The cache and module paths below isolate the fixture build, and module fetching is disabled.

work_dir=$(mktemp -d)
cd "$work_dir"
cat > go.mod <<'EOF'
module example.invalid/app

go 1.27
EOF
cat > main.go <<'EOF'
package main

func main() {}
EOF
export GOCACHE="$work_dir/cache"
export GOMODCACHE="$work_dir/modules"
export GOPATH="$work_dir/gopath"
export GOPROXY=off GOSUMDB=off GOTOOLCHAIN=local GOENV=off
export CGO_ENABLED=0 GOMAXPROCS=2
GOOS=darwin GOARCH=arm64 go build -p=2 -o app .
go version -m app
rea inspect-go-binary "$work_dir/app" --json

These fixture and inspection operations were executed with the stated compiler and source runtime. The reference command reads the file successfully; REA exits 1. Repeat the build with GOARCH=amd64 to observe the same source failure. A real stdio MCP session also reproduces it by calling inspect_go_binary with the absolute path to app, without opening a binary session first.

For a discriminating control, build a separate file with GOOS=darwin GOARCH=arm64 go build -p=2 -ldflags=-w -o app-nodwarf . and inspect it. This succeeded locally. Linux arm64 ELF and Windows amd64 PE builds of the same module also succeeded through the source CLI. The original debug-bearing file is never modified. Removing debug information changes its bytes and digest, so that control does not inspect the same artifact.

Expected behavior

The documented built-in reader supports thin Mach-O images and returns the selected artifact's SHA-256, byte length, format and original build-information observations without requiring Go. A valid non-mapped debug segment must not make this supported executable malformed. The same unmodified file should produce successful Evidence with format: "macho", build_info.go_version: "go1.27.1", the embedded module and build settings, and source ranges referring to the original bytes. The CLI and MCP must agree through their shared service.

Apple's loader permits file bytes exceeding virtual memory size for a non-mapped segment when its virtual memory size and initial protection are both zero, while still checking the file range. The compiler-generated debug segments meet those conditions.

Actual behavior

The source CLI exits 1 with the following meaningful error fields. The selected file path is omitted from the message here.

{
  "code": "invalid_request",
  "category": "invalid_input",
  "details": {
    "operation": "inspect_go_binary",
    "issues": [
      {
        "path": ["path"],
        "reason": "invalid_format",
        "message": "Mach-O segment addresses or sizes are invalid"
      }
    ]
  }
}

This is an excerpt of the observed response, not its complete JSON projection. MCP returns isError: true with the same typed failure. Neither returns the promised metadata Evidence. The fresh native arm64 executable runs and exits 0, and go version -m reads its compiler and module metadata. The debug segment has vmaddr 0x100148000, vmsize 0, fileoff 1163264, filesize 547390, maxprot 0 and initprot 0; its file range lies within the 1831778-byte artifact. It shares its virtual address with __LINKEDIT but declares no virtual memory allocation. The amd64 fixture has the same zero-memory and zero-protection conditions and also fails.

The source controls return populated metadata and the correct artifact digests. Both before and after inspection, the failing file's SHA-256 was identical. The separately tested release returns COMMAND_NOT_FOUND for this CLI command on all five fixtures, and its real MCP tool list does not advertise inspect_go_binary. This is not evidence of a fixed released parser.

Evidence

Expected source: docs/go-binaries.md, Supported boundary and Observations, plus the source CLI help's "Local ELF, PE or thin Mach-O image" argument.
Failure source: Fresh source CLI calls on compiler-produced arm64 and amd64 artifacts and a stdio MCP call on the arm64 artifact; src/go/GoMachoImage.ts:66 rejects fileSize > memorySize unconditionally and src/go/GoBinaryProvider.ts:54-65 projects the malformed-container failure to the observed path error.
Evidence provenance: observed
Local verification: reproduced
Reproduction completeness: complete

The independent format reference is Apple's dyld segment validation. It explicitly names non-allocatable Go __DWARF, checks that file contents stay within the file, and allows the zero-memory, zero-initial-protection case. Go's installed distribution independently emits this segment in src/cmd/link/internal/ld/macho.go:750-761 and explains the loader conditions in src/cmd/link/internal/ld/macho_combine_dwarf.go:374-387.

Both public adapters reach GoBinaryService and GoBinaryProvider through src/cli/goCommands.ts, src/server/registerGoTools.ts and src/composition/go.ts. The path-only input schema, stable regular-file read and thin Mach-O dispatch all admit the fixture before the segment-size guard fails. The source tool was advertised and called successfully past MCP admission; no deep provider was started.

Existing synthetic Mach-O fixtures write an ordinary __DATA segment with equal file and memory sizes in tests/fixtures/go/image.ts:328-344. The compiler verification lane in scripts/verify-go-binary.mjs:105-118 passes -ldflags=-s -w to every platform fixture, removing the Darwin debug segment. Existing ambiguous-mapping and virtual-overflow checks remain relevant.

Restoration check

Inspect the original debug-bearing arm64 and amd64 files through both public adapters. They must return populated metadata Evidence for the same bytes and digest, with module/build observations agreeing with the reference reader. Retain successful ELF, PE and no-DWARF controls. Treat the debug segment according to its actual non-mapped semantics; it must not introduce a virtual file mapping that conflicts with the real __LINKEDIT mapping at the same address.

Preserve malformed-input rejection. Three separately constructed controls were rejected by the source CLI: a declared segment file range beyond EOF, a virtual span overflowing the 64-bit address width, and a segment with file bytes and zero virtual memory size but nonzero initial protection. Those must remain invalid after restoration. A segment with arbitrary nonzero memory size and no permissions is not established as valid by this reproduction. Leave the implementation and regression-test design to the maintainer.

Additional context

The issue and PR searches found another author's open PR proposing this exact repair. Its body and comments were inspected; it is unmerged and provides reference context, not proof of restoration in either tested runtime. The proposed branch was not executed for this report.

The same original file is accepted by inspect-macho, but that command supplies container Evidence rather than Go module/compiler/build-string observations. go version -m is a useful external diagnostic on the original file when Go is installed. It does not supply the built-in reader's metadata schema, original string ranges or exact preserved build-string bytes, and the documented inspection contract does not require that toolchain. No supported alternative provider or segment override exists in the Go inspector's path-only input.

Triage assessment

Impact level: P1
Assessment status: supported
Impact basis: The primary documented Go metadata path returns no result for the demonstrated default Darwin arm64 and amd64 compiler outputs through CLI, with the arm64 failure also reproduced through MCP, although their valid debug segments do not affect the embedded metadata. This blocks that source capability for the affected input class. It does not establish failure of other REA native workflows, historical Go versions or every Mach-O kind.
Workaround status: none
Workaround basis: The shared inspector has one reader and no provider or segment override. Stripping or rebuilding changes the selected artifact. Container inspection does not return Go metadata, and the external Go reference requires an additional toolchain and does not preserve the promised metadata Evidence and source ranges. No supported same-artifact alternative restoring this contract was identified.

Originally created by @coygeek on GitHub (Oct 12, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/2167 ## Summary The built-in Go metadata inspector rejects compiler-produced thin Mach-O executables that contain an unmapped `__DWARF` segment. Both `inspect-go-binary` and MCP `inspect_go_binary` return an input-format error instead of compiler, module and build-setting Evidence for the selected file. Default Darwin arm64 and amd64 builds reproduced this in source. The latest tested package does not yet expose this capability, so this report concerns the source implementation. ## Runtime or environment Source revision c22111eca2c4bc84a084f12711f405b3e7422a40, using its prepared compiled CLI/MCP runtime. Separately tested published package rea-agents 6.4.0. Node v24.21.0 satisfies the package engines range, but differs from the checkout's .nvmrc contributor pin of 24.18.0. Host macOS 27.0.1 arm64. Fixture compiler Go go1.27.1 darwin/arm64, with CGO disabled and no external dependencies. No Hopper, Ghidra, IDA or active binary target is required. Area: native analysis or decompilation. Interfaces: CLI and MCP. Earliest failure stage: capture or observation, when admitting the selected native container. Target: compiler-produced thin Mach-O executable with Go inline build information and a non-mapped debug segment. Impact is compatibility and missing metadata; the inspection leaves the artifact unchanged. ## Steps to reproduce Use the source-built `rea` from the stated baseline and an existing Go compiler. Create this local module. The cache and module paths below isolate the fixture build, and module fetching is disabled. ```sh work_dir=$(mktemp -d) cd "$work_dir" cat > go.mod <<'EOF' module example.invalid/app go 1.27 EOF cat > main.go <<'EOF' package main func main() {} EOF export GOCACHE="$work_dir/cache" export GOMODCACHE="$work_dir/modules" export GOPATH="$work_dir/gopath" export GOPROXY=off GOSUMDB=off GOTOOLCHAIN=local GOENV=off export CGO_ENABLED=0 GOMAXPROCS=2 GOOS=darwin GOARCH=arm64 go build -p=2 -o app . go version -m app rea inspect-go-binary "$work_dir/app" --json ``` These fixture and inspection operations were executed with the stated compiler and source runtime. The reference command reads the file successfully; REA exits 1. Repeat the build with `GOARCH=amd64` to observe the same source failure. A real stdio MCP session also reproduces it by calling `inspect_go_binary` with the absolute path to `app`, without opening a binary session first. For a discriminating control, build a separate file with `GOOS=darwin GOARCH=arm64 go build -p=2 -ldflags=-w -o app-nodwarf .` and inspect it. This succeeded locally. Linux arm64 ELF and Windows amd64 PE builds of the same module also succeeded through the source CLI. The original debug-bearing file is never modified. Removing debug information changes its bytes and digest, so that control does not inspect the same artifact. ## Expected behavior The documented built-in reader supports thin Mach-O images and returns the selected artifact's SHA-256, byte length, format and original build-information observations without requiring Go. A valid non-mapped debug segment must not make this supported executable malformed. The same unmodified file should produce successful Evidence with `format: "macho"`, `build_info.go_version: "go1.27.1"`, the embedded module and build settings, and source ranges referring to the original bytes. The CLI and MCP must agree through their shared service. Apple's loader permits file bytes exceeding virtual memory size for a non-mapped segment when its virtual memory size and initial protection are both zero, while still checking the file range. The compiler-generated debug segments meet those conditions. ## Actual behavior The source CLI exits 1 with the following meaningful error fields. The selected file path is omitted from the message here. ```json { "code": "invalid_request", "category": "invalid_input", "details": { "operation": "inspect_go_binary", "issues": [ { "path": ["path"], "reason": "invalid_format", "message": "Mach-O segment addresses or sizes are invalid" } ] } } ``` This is an excerpt of the observed response, not its complete JSON projection. MCP returns `isError: true` with the same typed failure. Neither returns the promised metadata Evidence. The fresh native arm64 executable runs and exits 0, and `go version -m` reads its compiler and module metadata. The debug segment has `vmaddr 0x100148000`, `vmsize 0`, `fileoff 1163264`, `filesize 547390`, `maxprot 0` and `initprot 0`; its file range lies within the 1831778-byte artifact. It shares its virtual address with `__LINKEDIT` but declares no virtual memory allocation. The amd64 fixture has the same zero-memory and zero-protection conditions and also fails. The source controls return populated metadata and the correct artifact digests. Both before and after inspection, the failing file's SHA-256 was identical. The separately tested release returns `COMMAND_NOT_FOUND` for this CLI command on all five fixtures, and its real MCP tool list does not advertise `inspect_go_binary`. This is not evidence of a fixed released parser. ## Evidence - Expected source: `docs/go-binaries.md`, Supported boundary and Observations, plus the source CLI help's "Local ELF, PE or thin Mach-O image" argument. - Failure source: Fresh source CLI calls on compiler-produced arm64 and amd64 artifacts and a stdio MCP call on the arm64 artifact; `src/go/GoMachoImage.ts:66` rejects `fileSize > memorySize` unconditionally and `src/go/GoBinaryProvider.ts:54-65` projects the malformed-container failure to the observed path error. - Evidence provenance: observed - Local verification: reproduced - Reproduction completeness: complete The independent format reference is Apple's [dyld segment validation](https://github.com/apple-oss-distributions/dyld/blob/e9da5ae571b4191dcdbcfd827363f19847c84561/mach_o/UnsafeHeader.cpp#L791-L805). It explicitly names non-allocatable Go `__DWARF`, checks that file contents stay within the file, and allows the zero-memory, zero-initial-protection case. Go's installed distribution independently emits this segment in `src/cmd/link/internal/ld/macho.go:750-761` and explains the loader conditions in `src/cmd/link/internal/ld/macho_combine_dwarf.go:374-387`. Both public adapters reach `GoBinaryService` and `GoBinaryProvider` through `src/cli/goCommands.ts`, `src/server/registerGoTools.ts` and `src/composition/go.ts`. The path-only input schema, stable regular-file read and thin Mach-O dispatch all admit the fixture before the segment-size guard fails. The source tool was advertised and called successfully past MCP admission; no deep provider was started. Existing synthetic Mach-O fixtures write an ordinary `__DATA` segment with equal file and memory sizes in `tests/fixtures/go/image.ts:328-344`. The compiler verification lane in `scripts/verify-go-binary.mjs:105-118` passes `-ldflags=-s -w` to every platform fixture, removing the Darwin debug segment. Existing ambiguous-mapping and virtual-overflow checks remain relevant. ## Restoration check Inspect the original debug-bearing arm64 and amd64 files through both public adapters. They must return populated metadata Evidence for the same bytes and digest, with module/build observations agreeing with the reference reader. Retain successful ELF, PE and no-DWARF controls. Treat the debug segment according to its actual non-mapped semantics; it must not introduce a virtual file mapping that conflicts with the real `__LINKEDIT` mapping at the same address. Preserve malformed-input rejection. Three separately constructed controls were rejected by the source CLI: a declared segment file range beyond EOF, a virtual span overflowing the 64-bit address width, and a segment with file bytes and zero virtual memory size but nonzero initial protection. Those must remain invalid after restoration. A segment with arbitrary nonzero memory size and no permissions is not established as valid by this reproduction. Leave the implementation and regression-test design to the maintainer. ## Additional context The issue and PR searches found another author's open [PR proposing this exact repair](https://github.com/morluto/rea/pull/2089). Its body and comments were inspected; it is unmerged and provides reference context, not proof of restoration in either tested runtime. The proposed branch was not executed for this report. The same original file is accepted by `inspect-macho`, but that command supplies container Evidence rather than Go module/compiler/build-string observations. `go version -m` is a useful external diagnostic on the original file when Go is installed. It does not supply the built-in reader's metadata schema, original string ranges or exact preserved build-string bytes, and the documented inspection contract does not require that toolchain. No supported alternative provider or segment override exists in the Go inspector's path-only input. ## Triage assessment - Impact level: P1 - Assessment status: supported - Impact basis: The primary documented Go metadata path returns no result for the demonstrated default Darwin arm64 and amd64 compiler outputs through CLI, with the arm64 failure also reproduced through MCP, although their valid debug segments do not affect the embedded metadata. This blocks that source capability for the affected input class. It does not establish failure of other REA native workflows, historical Go versions or every Mach-O kind. - Workaround status: none - Workaround basis: The shared inspector has one reader and no provider or segment override. Stripping or rebuilding changes the selected artifact. Container inspection does not return Go metadata, and the external Go reference requires an additional toolchain and does not preserve the promised metadata Evidence and source ranges. No supported same-artifact alternative restoring this contract was identified.

bsakaguchi

2026-10-15 18:27:38 +00:00