morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[PR #1054] [MERGED] feat(native): inspect recorded crashes through pwntools and pwndbg #2609

New issue

Closed

opened 2026-10-17 21:08:10 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-17 21:08:10 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/1054
Author: @N0zoM1z0
Created: 10/8/2026
Status: ✅ Merged
Merged: 10/8/2026
Merged by: @N0zoM1z0

Base: main ← Head: feat/recorded-crash

📝 Commits (6)

cfd3627 feat(native): inspect recorded crashes through pwntools and pwndbg
818d70e fix(native): bind core note semantics to raw owner bytes
b2cd148 fix(native): validate recorded core ABI and evidence scalars
9147e89 fix(native): require complete recorded note interpretations
9ba594c fix(native): bind physical core notes to original headers
be6c54e chore: sync main for recorded crash validation

📊 Changes

58 files changed (+4623 additions, -490 deletions)

View changed files

➕ .github/workflows/real-recorded-crash.yml (+99 -0)
📝 README.md (+1 -0)
📝 README_ar.md (+1 -1)
📝 README_ja.md (+1 -1)
📝 README_ko.md (+1 -1)
📝 README_zh.md (+14 -14)
➕ bridge/pwndbg/core_context.py (+101 -0)
➕ bridge/pwndbg/launch.py (+39 -0)
➕ bridge/pwndbg/launch_impl.py (+34 -0)
➕ bridge/pwntools/decoder_runtime.py (+59 -0)
📝 bridge/pwntools/layout_impl.py (+11 -58)
➕ bridge/pwntools/recorded_crash.py (+39 -0)
➕ bridge/pwntools/recorded_crash_impl.py (+143 -0)
📝 docs/architecture.mermaid (+1 -1)
📝 docs/product-catalog.json (+12 -10)
➕ docs/recorded-crashes.md (+88 -0)
📝 docs/testing.md (+22 -0)
📝 docs/verification/managed-conformance-manifest.json (+2 -2)
📝 package.json (+11 -2)
➕ scripts/fixtures/recorded-crash.c (+24 -0)

...and 38 more files

📄 Description

Summary

Refs #969. Add a general recorded-crash capability: inspect an explicit Linux core independently of an active disassembler or live process. This is useful for application crash investigation and binary analysis; CTF is only one possible application.

Problem and expected behavior

REA's offline ELF tool deliberately excludes cores. Agents need every recorded thread/register, original note bytes and source locations, with optional debugger mapping context. Historical PIDs must remain recorded metadata and must never select a live process.

Change and scope

Reuse unchanged pwntools/pyelftools structures and Linux amd64 ctypes. Extract the existing snapshot/process/reply boundary into a shared typed decoder; keep ELF-specific semantics and core semantics in separate adapters.
Add a portable report, application port/shared Evidence workflow and named CLI/MCP adapters. Source-bound uint64 registers remain lossless; each supported raw note requires exactly one interpretation and each recorded thread requires all27 supported register identities bound to their ABI slots. Note owners/descriptors and unfamiliar kinds are preserved.
Optional unchanged BYO pwndbg/GDB enrichment queries mapping candidates from the same snapshot, in an owned core-only session with no executable pairing. Preserve exact observed versions, diagnostics and unknown current file identities. Zero flags leave permissions unknown.
Own both stage lifetimes; preserve diagnostics across invalid output, cancellation and cleanup failure. Retain tighter inherited soft/hard limits. Keep resource-branch markers distinct from unverified launcher exits.
Retain upstream MIT license/provenance; add modular source-owned real verification and a conditional installed-package CI lane using the exact upstream commit/frozen lockfile.

Contract and boundary impact

Semantic owner and earliest changed stage: pure domain/native/recordedCrash; source parsing and ABI interpretation stay in provider bridges.
CLI and MCP/tool-catalog contract: inspect-recorded-crash / inspect_recorded_crash, explicit path and optional mapping-context facet; catalog139/skill33. No approval boolean. Coverage-bearing ELF/note headers, canonical kind labels and owner displays are independently bound to the selected snapshot; exact physical note/padding coverage prevents hidden or duplicated observations.
Provider/format/platform: Linux x64, ELF64 amd64 little-endian ET_CORE; pwntools4.15.0/pyelftools0.33/Unicorn2.1.2, optional pwndbg2026.09.15. Incompatible ELF class/endian/machine/type, declared OSABI or ABI version return target-specific failures before Linux note interpretation. System V/unspecified ABI alone does not prove OS origin. Shared ELF target rejections now use the existing unsupported_target category rather than provider absence.
Evidence/provenance: full raw/normalized report inline, SHA-256/size, original source coordinates, all recorded thread registers, signed signals, unknown note completeness and signal-thread association. Mapping candidates and an envelope containing them are explicitly derived; basic recorded evidence remains observed. PID/current-signal and meaningful SIGINFO scalars are independently bound to original descriptor bytes alongside registers.
Process/authority/cleanup: basic owned Python; optional owned GDB, core-only connection verified before map queries, no paired executable or caller GDB command language. Input is never launched by inspection. BYO engine code is trusted and process limits are not a sandbox.
Package/generated metadata: original pwndbg license and fixed bridges included; generated catalog/skill/package metadata regenerated. No installer or host-default configuration changes.

Evidence and regression coverage

Added domain/service/provider/SDK/schema tests, including source bytes, uint64 values, invalid context, requested unavailable context, cancellation and two-stage cleanup diagnostics.
Executed source-owned two-thread core fixture: high RDI/R12, signed signal code, fault-address meaning, raw/opaque/missing/malformed notes, negative/historical PIDs, missing/wrong reported plugin profile and unsupported ET_EXEC carrier.
A live owned sentinel collides with the recorded PID. CLI/MCP inspection traces preserve attempted executions and reject ptrace/process-memory access, provider lookups of that PID's /proc files, and attempted Internet sockets. Exact ownership ps enumeration and observed upstream iconv -l/selected Git version lookup are separately accounted for. Sentinel survives; no descendants remain after cleanup. Fixture generation explicitly executes its own test program.
User-visible invocation:

rea inspect-recorded-crash /artifacts/crash.core --debugger-context --json

{"name":"inspect_recorded_crash","arguments":{"path":"/artifacts/crash.core","include_debugger_context":true}}

The normalized report includes target_execution: "not-performed", live_process_identity: "unknown", source-bound threads[].registers[], raw notes[], and debugger with core connection, no executable, derived maps and unknown current file identities. This describes inspection, separately from fixture generation.

Remaining coverage: full unwinding, executable/library matching, live debugging, other architectures/core formats and other hosts are outside this increment. Shared decoder resource markers, late cancellation and cleanup have deterministic boundary regressions; real GDB memory/CPU exhaustion is not claimed.

Observed, derived, and inferred claims remain distinguishable.
Artifact identity, source provenance, and failed attempts remain preserved.
Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

Focused domain/service/provider/SDK/schema/catalog tests: 93 passed, one host skip across10 files; after fixture refactoring, latest20 provider/service tests passed. Raw-owner follow-up:31 domain/service/provider checks passed; both thread/signal display-label counterexamples reproduced against the previous installed validator. ABI/scalar/confidence review follow-up:39 domain/service/provider/SDK checks passed; six scalar mismatches and the mixed-confidence envelope first reproduced as7 failing regressions, then passed after correction. Interpretation completeness follow-up:51 domain/service/provider/SDK checks passed;12 missing/duplicate-row, incomplete/unknown-register and wrong-register-slot counterexamples failed before the fix and passed after it. Final physical-source follow-up:64 focused checks passed;11 physical-identity/coverage/header counterexamples plus an unknown-owner display forgery were reproduced before correction. Exact physical coverage and independent source-header/enum/count binding close those paths.
npm run check:fast, npm run format:check, npm run docs:check — passed.
npm run verify:test-discovery — all576 test files exactly once.
Fresh npm pack --ignore-scripts / isolated installation — required bridges/runtime/original licenses included; private plans, reference clones, game data and binary fixtures excluded. Original upstream checkout remains unmodified.
Installed node scripts/verify-recorded-crash.mjs <installed>/scripts/rea.mjs — 42 actual CLI/MCP/trace cases passed on the latest9ba594c0 installed package (including GNU/Linux, Solaris, FreeBSD, unsupported ABI-version, extended program counts, zero/nonzero note tails and envelope confidence checks), complete139-tool schema/inventory/session parity; original inputs unchanged, sentinel alive and verifier descendants empty.
Local real debugger: GDB13.1 with unchanged pwndbg2026.09.15 build55596aa; The hosted installed-package workflow passed28 cases on818d70ef after provisioning GDB on the disposable runner, with no VPS changes. The previous9147e89b hosted36-case proof and its local6-case owner smoke also passed. Synced current main b9fb4388 (shared Evidence and CLI JSON changes) without conflicts, producing be6c54e3. All102 relevant core/Evidence/CLI JSON checks and a freshly installed6-case CLI/MCP smoke passed with139 schemas. Latest-head hosted42-case verification and all remaining CI must pass before merge.

Compatibility, safety, and release

Breaking changes/migration: none; additive target-free capability and optional evidence facet. Existing ELF workflow retains its behavior apart from the intentional target-error classification correction.
Real Hopper/Ghidra/browser: not run; independent capability. Linux actual GDB/pwndbg verification performed as above; no other OS claim.
Package/release: new bridges and original license, generated catalog/skill updated. No release requested or triggered.
Local-only evidence, supplied paths and diagnostics are retained. No ambient environment persistence, live PID authority, implicit host-file pairing or host-default changes.

Review checklist

One focused outcome; Conventional Commit title.
Related issue linked.
Changed behavior and meaningful failure paths tested.
Owning docs/contracts/generated metadata updated.
Representative CLI/MCP invocation and result semantics included.
Final diff checked for private data, unrelated changes and unsupported claims.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/1054 **Author:** [@N0zoM1z0](https://github.com/N0zoM1z0) **Created:** 10/8/2026 **Status:** ✅ Merged **Merged:** 10/8/2026 **Merged by:** [@N0zoM1z0](https://github.com/N0zoM1z0) **Base:** `main` ← **Head:** `feat/recorded-crash` --- ### 📝 Commits (6) - [`cfd3627`](https://github.com/morluto/rea/commit/cfd3627b521a26e2865918d06a5844339fed2d07) feat(native): inspect recorded crashes through pwntools and pwndbg - [`818d70e`](https://github.com/morluto/rea/commit/818d70efac83f7acbcbe5560d60384a5234e2cfb) fix(native): bind core note semantics to raw owner bytes - [`b2cd148`](https://github.com/morluto/rea/commit/b2cd148bb301e05df2763796b717fe7be9ebfdda) fix(native): validate recorded core ABI and evidence scalars - [`9147e89`](https://github.com/morluto/rea/commit/9147e89b90b160b39b5c1db7a8bbc17e0d6f96e7) fix(native): require complete recorded note interpretations - [`9ba594c`](https://github.com/morluto/rea/commit/9ba594c0ede20e334b5a571d25a3236a3a478ac6) fix(native): bind physical core notes to original headers - [`be6c54e`](https://github.com/morluto/rea/commit/be6c54e3ad75bfe5a1c2fe274545ef3a255e58ea) chore: sync main for recorded crash validation ### 📊 Changes **58 files changed** (+4623 additions, -490 deletions) <details> <summary>View changed files</summary> ➕ `.github/workflows/real-recorded-crash.yml` (+99 -0) 📝 `README.md` (+1 -0) 📝 `README_ar.md` (+1 -1) 📝 `README_ja.md` (+1 -1) 📝 `README_ko.md` (+1 -1) 📝 `README_zh.md` (+14 -14) ➕ `bridge/pwndbg/core_context.py` (+101 -0) ➕ `bridge/pwndbg/launch.py` (+39 -0) ➕ `bridge/pwndbg/launch_impl.py` (+34 -0) ➕ `bridge/pwntools/decoder_runtime.py` (+59 -0) 📝 `bridge/pwntools/layout_impl.py` (+11 -58) ➕ `bridge/pwntools/recorded_crash.py` (+39 -0) ➕ `bridge/pwntools/recorded_crash_impl.py` (+143 -0) 📝 `docs/architecture.mermaid` (+1 -1) 📝 `docs/product-catalog.json` (+12 -10) ➕ `docs/recorded-crashes.md` (+88 -0) 📝 `docs/testing.md` (+22 -0) 📝 `docs/verification/managed-conformance-manifest.json` (+2 -2) 📝 `package.json` (+11 -2) ➕ `scripts/fixtures/recorded-crash.c` (+24 -0) _...and 38 more files_ </details> ### 📄 Description ## Summary Refs #969. Add a general recorded-crash capability: inspect an explicit Linux core independently of an active disassembler or live process. This is useful for application crash investigation and binary analysis; CTF is only one possible application. ## Problem and expected behavior REA's offline ELF tool deliberately excludes cores. Agents need every recorded thread/register, original note bytes and source locations, with optional debugger mapping context. Historical PIDs must remain recorded metadata and must never select a live process. ## Change and scope - Reuse unchanged pwntools/pyelftools structures and Linux amd64 ctypes. Extract the existing snapshot/process/reply boundary into a shared typed decoder; keep ELF-specific semantics and core semantics in separate adapters. - Add a portable report, application port/shared Evidence workflow and named CLI/MCP adapters. Source-bound uint64 registers remain lossless; each supported raw note requires exactly one interpretation and each recorded thread requires all27 supported register identities bound to their ABI slots. Note owners/descriptors and unfamiliar kinds are preserved. - Optional unchanged BYO pwndbg/GDB enrichment queries mapping candidates from the same snapshot, in an owned core-only session with no executable pairing. Preserve exact observed versions, diagnostics and unknown current file identities. Zero flags leave permissions unknown. - Own both stage lifetimes; preserve diagnostics across invalid output, cancellation and cleanup failure. Retain tighter inherited soft/hard limits. Keep resource-branch markers distinct from unverified launcher exits. - Retain upstream MIT license/provenance; add modular source-owned real verification and a conditional installed-package CI lane using the exact upstream commit/frozen lockfile. ## Contract and boundary impact - Semantic owner and earliest changed stage: pure `domain/native/recordedCrash`; source parsing and ABI interpretation stay in provider bridges. - CLI and MCP/tool-catalog contract: `inspect-recorded-crash` / `inspect_recorded_crash`, explicit path and optional mapping-context facet; catalog139/skill33. No approval boolean. Coverage-bearing ELF/note headers, canonical kind labels and owner displays are independently bound to the selected snapshot; exact physical note/padding coverage prevents hidden or duplicated observations. - Provider/format/platform: Linux x64, ELF64 amd64 little-endian ET_CORE; pwntools4.15.0/pyelftools0.33/Unicorn2.1.2, optional pwndbg2026.09.15. Incompatible ELF class/endian/machine/type, declared OSABI or ABI version return target-specific failures before Linux note interpretation. System V/unspecified ABI alone does not prove OS origin. Shared ELF target rejections now use the existing `unsupported_target` category rather than provider absence. - Evidence/provenance: full raw/normalized report inline, SHA-256/size, original source coordinates, all recorded thread registers, signed signals, unknown note completeness and signal-thread association. Mapping candidates and an envelope containing them are explicitly derived; basic recorded evidence remains observed. PID/current-signal and meaningful SIGINFO scalars are independently bound to original descriptor bytes alongside registers. - Process/authority/cleanup: basic owned Python; optional owned GDB, core-only connection verified before map queries, no paired executable or caller GDB command language. Input is never launched by inspection. BYO engine code is trusted and process limits are not a sandbox. - Package/generated metadata: original pwndbg license and fixed bridges included; generated catalog/skill/package metadata regenerated. No installer or host-default configuration changes. ## Evidence and regression coverage - Added domain/service/provider/SDK/schema tests, including source bytes, uint64 values, invalid context, requested unavailable context, cancellation and two-stage cleanup diagnostics. - Executed source-owned two-thread core fixture: high RDI/R12, signed signal code, fault-address meaning, raw/opaque/missing/malformed notes, negative/historical PIDs, missing/wrong reported plugin profile and unsupported ET_EXEC carrier. - A live owned sentinel collides with the recorded PID. CLI/MCP inspection traces preserve attempted executions and reject ptrace/process-memory access, provider lookups of that PID's `/proc` files, and attempted Internet sockets. Exact ownership `ps` enumeration and observed upstream `iconv -l`/selected Git version lookup are separately accounted for. Sentinel survives; no descendants remain after cleanup. Fixture generation explicitly executes its own test program. - User-visible invocation: ```sh rea inspect-recorded-crash /artifacts/crash.core --debugger-context --json ``` ```json {"name":"inspect_recorded_crash","arguments":{"path":"/artifacts/crash.core","include_debugger_context":true}} ``` The normalized report includes `target_execution: "not-performed"`, `live_process_identity: "unknown"`, source-bound `threads[].registers[]`, raw `notes[]`, and `debugger` with core connection, no executable, derived maps and unknown current file identities. This describes inspection, separately from fixture generation. Remaining coverage: full unwinding, executable/library matching, live debugging, other architectures/core formats and other hosts are outside this increment. Shared decoder resource markers, late cancellation and cleanup have deterministic boundary regressions; real GDB memory/CPU exhaustion is not claimed. - [x] Observed, derived, and inferred claims remain distinguishable. - [x] Artifact identity, source provenance, and failed attempts remain preserved. - [x] Unsupported, incomplete, unavailable, or uncertain outcomes remain visible. ## Validation performed - Focused domain/service/provider/SDK/schema/catalog tests: **93 passed, one host skip across10 files**; after fixture refactoring, latest20 provider/service tests passed. Raw-owner follow-up:31 domain/service/provider checks passed; both thread/signal display-label counterexamples reproduced against the previous installed validator. ABI/scalar/confidence review follow-up:39 domain/service/provider/SDK checks passed; six scalar mismatches and the mixed-confidence envelope first reproduced as7 failing regressions, then passed after correction. Interpretation completeness follow-up:51 domain/service/provider/SDK checks passed;12 missing/duplicate-row, incomplete/unknown-register and wrong-register-slot counterexamples failed before the fix and passed after it. Final physical-source follow-up:64 focused checks passed;11 physical-identity/coverage/header counterexamples plus an unknown-owner display forgery were reproduced before correction. Exact physical coverage and independent source-header/enum/count binding close those paths. - `npm run check:fast`, `npm run format:check`, `npm run docs:check` — passed. - `npm run verify:test-discovery` — all576 test files exactly once. - Fresh `npm pack --ignore-scripts` / isolated installation — required bridges/runtime/original licenses included; private plans, reference clones, game data and binary fixtures excluded. Original upstream checkout remains unmodified. - Installed `node scripts/verify-recorded-crash.mjs <installed>/scripts/rea.mjs` — **42 actual CLI/MCP/trace cases passed on the latest9ba594c0 installed package** (including GNU/Linux, Solaris, FreeBSD, unsupported ABI-version, extended program counts, zero/nonzero note tails and envelope confidence checks), complete139-tool schema/inventory/session parity; original inputs unchanged, sentinel alive and verifier descendants empty. - Local real debugger: GDB13.1 with unchanged pwndbg2026.09.15 build55596aa; The hosted installed-package workflow passed28 cases on818d70ef after provisioning GDB on the disposable runner, with no VPS changes. The previous9147e89b hosted36-case proof and its local6-case owner smoke also passed. Synced current main b9fb4388 (shared Evidence and CLI JSON changes) without conflicts, producing be6c54e3. All102 relevant core/Evidence/CLI JSON checks and a freshly installed6-case CLI/MCP smoke passed with139 schemas. Latest-head hosted42-case verification and all remaining CI must pass before merge. ## Compatibility, safety, and release - Breaking changes/migration: none; additive target-free capability and optional evidence facet. Existing ELF workflow retains its behavior apart from the intentional target-error classification correction. - Real Hopper/Ghidra/browser: not run; independent capability. Linux actual GDB/pwndbg verification performed as above; no other OS claim. - Package/release: new bridges and original license, generated catalog/skill updated. No release requested or triggered. - Local-only evidence, supplied paths and diagnostics are retained. No ambient environment persistence, live PID authority, implicit host-file pairing or host-default changes. ## Review checklist - [x] One focused outcome; Conventional Commit title. - [x] Related issue linked. - [x] Changed behavior and meaningful failure paths tested. - [x] Owning docs/contracts/generated metadata updated. - [x] Representative CLI/MCP invocation and result semantics included. - [x] Final diff checked for private data, unrelated changes and unsupported claims. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-17 21:08:10 +00:00

closed this issue
added the
pull-request
label

bsakaguchi referenced this issue

2026-10-31 01:07:54 +00:00

[PR #3082] [MERGED] fix(browser): preserve passive network inventory uncertainty #5602

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#2609

No description provided.

Rows
Columns