morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #2297] [Feature] Recover function names, source mappings and types from stripped Go binaries #762

New issue

Open

opened 2026-10-15 18:28:04 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-15 18:28:04 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 12, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/2297

Before submitting

I searched the open and closed issues and pull requests for a related proposal.
I understand the contribution scope must be reviewed before integration; open questions are welcome here.
This request does not include credentials, private samples, or sensitive target data.

Area

Native analysis or decompilation

Current limitation

On main (aeee5074d1), inspect_go_binary returns the artifact identity, container format and architecture, and the embedded build_info record (compiler string, main module, dependencies, replacements and build settings with exact source ranges). docs/go-binaries.md, the tool's limitations array and its MCP description all state that function names, source mappings, types and decompiled code are outside its coverage, and send those questions to a disassembler.

For current Go releases the disassembler route does not fill the gap either. Ghidra 12.1.4, run directly with default auto-analysis (not through REA's open_binary), imported a stripped (-ldflags="-s -w") linux/amd64 test binary with the golang compiler spec. Its Golang Symbols and Strings analyzers then stopped with Untested Go version [1.26.5], leaving 1342 of 1344 functions as FUN_<address>, including main.main. A Go 1.27.1 build behaved the same way (1350 of 1352). Hopper and IDA were not tested.

Use cases

Triage a stripped Go service or CLI: list its main and third-party package functions with address ranges, without a configured disassembler.
Map an address seen in a crash, trace or disassembly back to a Go function, source file and line.
See which runtime types a binary declares, and their struct layouts, before deeper native analysis.

Desired outcome and success criteria

A static, read-only capability, extending inspect_go_binary or added as a sibling tool on both the CLI and MCP, that reads Go's own runtime metadata (pclntab, moduledata) from one selected ELF, PE or thin Mach-O image. It returns inline observed Evidence with: function names and start and end addresses, plus file offsets where the range is file-backed; recorded source files and per-function file and line or PC-to-line mapping where available; reachable type names and, where recovered, struct fields; and the interpreted table addresses and layout version. Unsupported layouts or Go versions and malformed tables return a typed failure or an explicit limitation, never a complete-looking empty list. Recovered names are labeled as file content, not authenticated identity.

Success criteria:

For stripped fixtures covering ELF, PE and Mach-O on amd64 and arm64, main function names and start addresses equal Go's own symbol tooling (go tool nm or debug/gosym) on an unstripped build of the same source and toolchain, and a selected function's file and line agree with debug/gosym.
A struct type made reachable at runtime appears among the recovered types, and any reported fields match its declaration.
The newest Go release supported by the chosen engine succeeds, and a binary with an unsupported layout gets a typed failure or a limitation naming what was observed.
Fixtures are byte-identical after inspection, with no target execution and no network access.
Inventories larger than one MCP response are complete across pages or retained Evidence export.
docs/go-binaries.md, the contract text and the limitations describe the new boundary, and a real Go verification lane (extending npm run verify:go:binary or alongside it) checks all of the above through the public CLI and MCP.

Expected interaction with targets

Read existing files or captured data

Target and scope details (optional)

The target is one explicitly selected local executable or library, read without execution. If the chosen engine is an external helper such as GoReSym, REA starts that helper as a local subprocess on the selected file, and the tool's effects and capability inventory should declare it. A built-in reader would start no subprocess. Out of scope: decompiling Go to source, reversing deliberate symbol renaming or obfuscation, fat Mach-O selection (outside today's reader too), and writing names into a disassembler session automatically. Because a new provider may be involved, this needs a maintainer scope decision before integration.

Verification available so far

I have a prototype or observed results

Verification details (optional)

Fixture: one small main package with a //go:noinline function main.reaProbeCompute and a struct type, built with CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -ldflags="-s -w" under Go 1.26.5 and Go 1.27.1.

GoReSym v3.4.1, built from source at 722897e0a80da0a06c00c5e40591a253a551f04e and run with -t -p on the Go 1.26.5 binary, reported Version 1.26.5, pclntab layout 1.20, main.reaProbeCompute and main.main with start and end addresses, and 244 source paths. Its function records carry no per-function file or line.
On the Go 1.27.1 binary, stripped and unstripped, GoReSym returned {"error": "Failed to parse file: no valid pclntab found"}, consistent with its open Go 1.27 support issue.
Ghidra 12.1.4 analyzeHeadless gave the FUN_ results described above for both builds.
Not tested: the struct type, which the fixture never made reachable through an interface; PE and Mach-O; arm64; Hopper; and IDA.

Alternatives considered

GoReSym as an optional local helper: MIT, JSON output (functions, files, types, interfaces, build info, pclntab and moduledata addresses). Its README claims Mach-O, ELF and PE on ARM64 and x86/x64 in both byte orders, pclntab parsing from Go 1.2 and type parsing from Go 1.5. It currently supports up to Go 1.26, and per-function line mapping would need another source.
REA's own TypeScript parser of pclntab and moduledata, extending the existing built-in reader that already handles ELF, PE and Mach-O mappings without Go or a disassembler. It needs no third-party runtime, but REA would track Go's per-release table layout changes itself.
redress: AGPL-3.0, as is its gore library, while REA is MIT. Its README documents text-table output and a radare2 r2pipe mode, with no JSON output.
Relying on deep providers' Go analyzers: Ghidra 12.1.4 refused Go 1.26.5 and 1.27.1 as untested versions. Coverage would follow each provider's release cadence and would need a configured disassembler.

Possible direction

Either engine above could produce the same Evidence contract, and the real verification lane can compare it with Go's own debug/gosym on unstripped builds, as the existing lane does with debug/buildinfo. As a later, separate step, REA already offers set_address_name / set_addresses_names for Hopper sessions and for Ghidra sessions on Linux and macOS, so recovered names could be applied to an open session on explicit request. REA's IDA client currently rejects that mutation, and the cross-provider address mapping has not been verified.

References

#2258 and the follow-up mapping (gap 3): https://github.com/morluto/rea/issues/2258#issuecomment-6130226337
docs/go-binaries.md: https://github.com/morluto/rea/blob/main/docs/go-binaries.md
GoReSym: https://github.com/mandiant/GoReSym
GoReSym Go 1.27 support issue: https://github.com/mandiant/GoReSym/issues/90
redress: https://github.com/goretk/redress
#1363 (the merged inspect_go_binary PR); #2167, #2089 and #2246 cover Mach-O build-info reading and are distinct from this request.

Originally created by @coygeek on GitHub (Oct 12, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/2297 ### Before submitting - [x] I searched the open and closed issues and pull requests for a related proposal. - [x] I understand the [contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope) must be reviewed before integration; open questions are welcome here. - [x] This request does not include credentials, private samples, or sensitive target data. ### Area Native analysis or decompilation ### Current limitation On `main` (aeee5074d124414239b6dd440dc79c26bcbd58a4), `inspect_go_binary` returns the artifact identity, container format and architecture, and the embedded `build_info` record (compiler string, main module, dependencies, replacements and build settings with exact source ranges). `docs/go-binaries.md`, the tool's `limitations` array and its MCP description all state that function names, source mappings, types and decompiled code are outside its coverage, and send those questions to a disassembler. For current Go releases the disassembler route does not fill the gap either. Ghidra 12.1.4, run directly with default auto-analysis (not through REA's `open_binary`), imported a stripped (`-ldflags="-s -w"`) linux/amd64 test binary with the `golang` compiler spec. Its Golang Symbols and Strings analyzers then stopped with `Untested Go version [1.26.5]`, leaving 1342 of 1344 functions as `FUN_<address>`, including `main.main`. A Go 1.27.1 build behaved the same way (1350 of 1352). Hopper and IDA were not tested. ### Use cases - Triage a stripped Go service or CLI: list its `main` and third-party package functions with address ranges, without a configured disassembler. - Map an address seen in a crash, trace or disassembly back to a Go function, source file and line. - See which runtime types a binary declares, and their struct layouts, before deeper native analysis. ### Desired outcome and success criteria A static, read-only capability, extending `inspect_go_binary` or added as a sibling tool on both the CLI and MCP, that reads Go's own runtime metadata (`pclntab`, `moduledata`) from one selected ELF, PE or thin Mach-O image. It returns inline observed Evidence with: function names and start and end addresses, plus file offsets where the range is file-backed; recorded source files and per-function file and line or PC-to-line mapping where available; reachable type names and, where recovered, struct fields; and the interpreted table addresses and layout version. Unsupported layouts or Go versions and malformed tables return a typed failure or an explicit limitation, never a complete-looking empty list. Recovered names are labeled as file content, not authenticated identity. Success criteria: - For stripped fixtures covering ELF, PE and Mach-O on amd64 and arm64, `main` function names and start addresses equal Go's own symbol tooling (`go tool nm` or `debug/gosym`) on an unstripped build of the same source and toolchain, and a selected function's file and line agree with `debug/gosym`. - A struct type made reachable at runtime appears among the recovered types, and any reported fields match its declaration. - The newest Go release supported by the chosen engine succeeds, and a binary with an unsupported layout gets a typed failure or a limitation naming what was observed. - Fixtures are byte-identical after inspection, with no target execution and no network access. - Inventories larger than one MCP response are complete across pages or retained Evidence export. - `docs/go-binaries.md`, the contract text and the limitations describe the new boundary, and a real Go verification lane (extending `npm run verify:go:binary` or alongside it) checks all of the above through the public CLI and MCP. ### Expected interaction with targets Read existing files or captured data ### Target and scope details (optional) The target is one explicitly selected local executable or library, read without execution. If the chosen engine is an external helper such as GoReSym, REA starts that helper as a local subprocess on the selected file, and the tool's effects and capability inventory should declare it. A built-in reader would start no subprocess. Out of scope: decompiling Go to source, reversing deliberate symbol renaming or obfuscation, fat Mach-O selection (outside today's reader too), and writing names into a disassembler session automatically. Because a new provider may be involved, this needs a maintainer scope decision before integration. ### Verification available so far I have a prototype or observed results ### Verification details (optional) Fixture: one small `main` package with a `//go:noinline` function `main.reaProbeCompute` and a struct type, built with `CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -ldflags="-s -w"` under Go 1.26.5 and Go 1.27.1. - GoReSym v3.4.1, built from source at 722897e0a80da0a06c00c5e40591a253a551f04e and run with `-t -p` on the Go 1.26.5 binary, reported `Version 1.26.5`, pclntab layout `1.20`, `main.reaProbeCompute` and `main.main` with start and end addresses, and 244 source paths. Its function records carry no per-function file or line. - On the Go 1.27.1 binary, stripped and unstripped, GoReSym returned `{"error": "Failed to parse file: no valid pclntab found"}`, consistent with its open Go 1.27 support issue. - Ghidra 12.1.4 `analyzeHeadless` gave the `FUN_` results described above for both builds. - Not tested: the struct type, which the fixture never made reachable through an interface; PE and Mach-O; arm64; Hopper; and IDA. ### Alternatives considered - GoReSym as an optional local helper: MIT, JSON output (functions, files, types, interfaces, build info, pclntab and moduledata addresses). Its README claims Mach-O, ELF and PE on ARM64 and x86/x64 in both byte orders, pclntab parsing from Go 1.2 and type parsing from Go 1.5. It currently supports up to Go 1.26, and per-function line mapping would need another source. - REA's own TypeScript parser of `pclntab` and `moduledata`, extending the existing built-in reader that already handles ELF, PE and Mach-O mappings without Go or a disassembler. It needs no third-party runtime, but REA would track Go's per-release table layout changes itself. - redress: AGPL-3.0, as is its `gore` library, while REA is MIT. Its README documents text-table output and a radare2 `r2pipe` mode, with no JSON output. - Relying on deep providers' Go analyzers: Ghidra 12.1.4 refused Go 1.26.5 and 1.27.1 as untested versions. Coverage would follow each provider's release cadence and would need a configured disassembler. ### Possible direction Either engine above could produce the same Evidence contract, and the real verification lane can compare it with Go's own `debug/gosym` on unstripped builds, as the existing lane does with `debug/buildinfo`. As a later, separate step, REA already offers `set_address_name` / `set_addresses_names` for Hopper sessions and for Ghidra sessions on Linux and macOS, so recovered names could be applied to an open session on explicit request. REA's IDA client currently rejects that mutation, and the cross-provider address mapping has not been verified. ### References - #2258 and the follow-up mapping (gap 3): https://github.com/morluto/rea/issues/2258#issuecomment-6130226337 - `docs/go-binaries.md`: https://github.com/morluto/rea/blob/main/docs/go-binaries.md - GoReSym: https://github.com/mandiant/GoReSym - GoReSym Go 1.27 support issue: https://github.com/mandiant/GoReSym/issues/90 - redress: https://github.com/goretk/redress - #1363 (the merged `inspect_go_binary` PR); #2167, #2089 and #2246 cover Mach-O build-info reading and are distinct from this request.

bsakaguchi added the

area: native

enhancement

P2: medium

labels

2026-10-15 18:28:04 +00:00

bsakaguchi commented

2026-10-15 18:28:05 +00:00

Author

Owner

@matbrgz commented on GitHub (Oct 12, 2026):

Design & Go Runtime Symbol Recovery Notes

Recovering function names, types, and source mappings from stripped Go binaries is one of the highest-leverage static analysis additions:

Format Structures: Go binaries preserve pclntab (PC Line Table / runtime pclntab) across Go 1.2 through 1.23+ (0xfffffff0 / 0xfffffff1 magic bytes). Even when ELF/PE symbol tables are stripped with -ldflags="-s -w", pclntab remains present in .gopclntab (or embedded in .rodata/.text) because the Go runtime needs it for stack traces.
Implementation Strategy:
- Implement a pure TypeScript parser for Go pclntab versions (Go 1.16, 1.18, 1.20+ format revisions).
- Feed recovered function addresses and names into Ghidra/IDA symbol re-labeling or surface them directly as static Go procedure Evidence.

@matbrgz commented on GitHub (Oct 12, 2026): ### Design & Go Runtime Symbol Recovery Notes Recovering function names, types, and source mappings from stripped Go binaries is one of the highest-leverage static analysis additions: 1. **Format Structures:** Go binaries preserve `pclntab` (PC Line Table / runtime pclntab) across Go 1.2 through 1.23+ (`0xfffffff0` / `0xfffffff1` magic bytes). Even when ELF/PE symbol tables are stripped with `-ldflags="-s -w"`, `pclntab` remains present in `.gopclntab` (or embedded in `.rodata`/`.text`) because the Go runtime needs it for stack traces. 2. **Implementation Strategy:** - Implement a pure TypeScript parser for Go `pclntab` versions (Go 1.16, 1.18, 1.20+ format revisions). - Feed recovered function addresses and names into Ghidra/IDA symbol re-labeling or surface them directly as static Go procedure Evidence.

bsakaguchi commented

2026-10-15 18:28:05 +00:00

Author

Owner

@jmalfonsi commented on GitHub (Oct 13, 2026):

Data point: garble-obfuscated binaries change the `pclntab` magic and scramble `_func.entryOff`

The design note above locates pclntab by its magic (0xfffffff0 / 0xfffffff1). That will miss binaries built with garble, which is common in Go malware. I hit this on a garble-obfuscated Windows x64 ransomware sample (PE32+, Go ≥ 1.24 judging by runtime packages, stripped, build info reads unknown). Static analysis only; the sample was never run.

Observed:

The header magic is 0xd78908d1 (bytes d1 08 89 d7) instead of 0xfffffff1, and the file contains no 0xfffffff1 header at all. The rest of the header is standard Go 1.20+: two zero pad bytes, minLC = 1, ptrSize = 8, then nfunc, nfiles, textStart and the five table offsets.
The table sits in .rdata (the PE has no .gopclntab section), with 2675 functions and 7030 files.
The functab entryoff values are plain: monotonic, and adding them to textStart gives correct function starts.
_func.entryOff differs from the matching functab entry for 2674 of 2675 functions. This matches garble's linker patch that encrypts that field. A parser that takes entry addresses from _func will produce garbage.
Function names in funcnametab are readable. They are garble's renamed identifiers (main.d8KcLKg, renamed stdlib packages), not the originals, but they still give Go function boundaries and package grouping.
REA 6.4.0 with Ghidra 12.1.4 reported 2490 functions with no Go names. inspect_go_binary has nothing to report because build info is stripped.

Workaround that worked (about 40 lines of Python, outside REA): find the header, read functab (textStart + entryoff, nameoff → funcnametab), then push the 2675 names into the live Ghidra session with set_addresses_names before batch_decompile. That made the decompiled code readable.

Suggestions for the parser:

Find the header by structure rather than by magic. Look for 4-byte-aligned offsets where bytes 4–5 are 00 00, minLC ∈ {1, 2, 4}, ptrSize ∈ {4, 8} and textStart equals the start of the executable section. Then check that nfunc is plausible, the table offsets fall inside the section, and functab is monotonic. On this sample, the first three checks plus the textStart match already give exactly one hit. A non-standard magic would be reported as an observation (it is also a useful garble indicator), not as a failure.
Take entry addresses from functab, not _func.entryOff. At minimum, cross-check the two and report the mismatch rather than trusting _func.
Label names as observed funcnametab strings. Do not present them as original source identifiers, since garble rewrites them before linking.

I can't share the sample, but this is reproducible without malware: build any small program with garble build (recent garble, Go 1.24+) and compare with a plain go build -ldflags="-s -w". That pair would also make a good fixture. I'm happy to help with the fixture or the parser once the scope is settled.

@jmalfonsi commented on GitHub (Oct 13, 2026): ### Data point: garble-obfuscated binaries change the `pclntab` magic and scramble `_func.entryOff` The design note above locates `pclntab` by its magic (`0xfffffff0` / `0xfffffff1`). That will miss binaries built with [garble](https://github.com/burrowers/garble), which is common in Go malware. I hit this on a garble-obfuscated Windows x64 ransomware sample (PE32+, Go ≥ 1.24 judging by runtime packages, stripped, build info reads `unknown`). Static analysis only; the sample was never run. **Observed:** - The header magic is `0xd78908d1` (bytes `d1 08 89 d7`) instead of `0xfffffff1`, and the file contains no `0xfffffff1` header at all. The rest of the header is standard Go 1.20+: two zero pad bytes, `minLC = 1`, `ptrSize = 8`, then `nfunc`, `nfiles`, `textStart` and the five table offsets. - The table sits in `.rdata` (the PE has no `.gopclntab` section), with 2675 functions and 7030 files. - The `functab` `entryoff` values are plain: monotonic, and adding them to `textStart` gives correct function starts. - `_func.entryOff` differs from the matching `functab` entry for 2674 of 2675 functions. This matches garble's linker patch that encrypts that field. A parser that takes entry addresses from `_func` will produce garbage. - Function names in `funcnametab` are readable. They are garble's renamed identifiers (`main.d8KcLKg`, renamed stdlib packages), not the originals, but they still give Go function boundaries and package grouping. - REA 6.4.0 with Ghidra 12.1.4 reported 2490 functions with no Go names. `inspect_go_binary` has nothing to report because build info is stripped. **Workaround that worked (about 40 lines of Python, outside REA):** find the header, read `functab` (`textStart + entryoff`, `nameoff` → `funcnametab`), then push the 2675 names into the live Ghidra session with `set_addresses_names` before `batch_decompile`. That made the decompiled code readable. **Suggestions for the parser:** 1. **Find the header by structure rather than by magic.** Look for 4-byte-aligned offsets where bytes 4–5 are `00 00`, `minLC ∈ {1, 2, 4}`, `ptrSize ∈ {4, 8}` and `textStart` equals the start of the executable section. Then check that `nfunc` is plausible, the table offsets fall inside the section, and `functab` is monotonic. On this sample, the first three checks plus the `textStart` match already give exactly one hit. A non-standard magic would be reported as an observation (it is also a useful garble indicator), not as a failure. 2. **Take entry addresses from `functab`, not `_func.entryOff`.** At minimum, cross-check the two and report the mismatch rather than trusting `_func`. 3. **Label names as observed `funcnametab` strings.** Do not present them as original source identifiers, since garble rewrites them before linking. I can't share the sample, but this is reproducible without malware: build any small program with `garble build` (recent garble, Go 1.24+) and compare with a plain `go build -ldflags="-s -w"`. That pair would also make a good fixture. I'm happy to help with the fixture or the parser once the scope is settled.

bsakaguchi referenced this issue

2026-10-15 18:28:32 +00:00

[GH-ISSUE #2383] [Tracker] Audit analysis and debugging coverage for the TIOBE top 100 languages (October 2026) #794

bsakaguchi referenced this issue

2026-10-16 21:15:54 +00:00

[PR #762] [CLOSED] fix(native): preserve thin architectures across echoed filename lines #2223

bsakaguchi referenced this issue

2026-10-16 21:16:27 +00:00

[PR #815] [MERGED] fix: integrate 24 reviewed browser/native/semantics corrections #2262

bsakaguchi added

enhancement

and removed