[GH-ISSUE #4397] [Docs] Bundled skill has no route for native runtime observation (Frida, GDB, Rizin debug, process capture) #1752

New issue

Closed

opened 2026-10-16 21:09:49 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-16 21:09:49 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 16, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/4397

Before submitting

I searched the existing issues for a related documentation report.
I removed credentials, tokens, private paths, and sensitive target data.

Documentation location

.agents/skills/reverse-engineer-anything/SKILL.md (target routing and guide index) and .agents/skills/reverse-engineer-anything/references/runtime-observation.md

Problem

The skill's guide index sends runtime questions only to "Browser/Electron observation (passive and scenario) and static/runtime reconciliation" (.agents/skills/reverse-engineer-anything/SKILL.md), and references/runtime-observation.md is titled "Browser and Electron runtime observation" and contains no native-process content. A search of the skill directory finds no mention of frida, rizin, cutter, start_gdb_session, gdb_console, capture_process_scenario, compare_process_captures, observe_native_ui, capture_native_ui_scenario, or trace_native_values.

Native runtime coverage in the skill is limited to observe_native_calls for Mach-O under LLDB and GDB/pwndbg as debugger context for recorded core files (references/native-and-artifacts.md). The routing table sends native executables to open_binary and static analysis; a question about a native program's runtime behavior has no route.

The repository documents these tools elsewhere (docs/frida-instrumentation.md, docs/reverse-engineering-tools.md, docs/process-capture.md), and they carry significant effects that agents need to know before calling them: Frida scripts "can read or change target memory and behavior", GDB and Rizin debug consoles are unrestricted, spawned processes start paused, GDB and Rizin debug are Linux-only, and REA never kills an attached target. An agent working only from the skill does not learn any of this.

Affected: agents that rely on the installed skill to choose tools. Examples:

An agent has a static hypothesis about which function handles a message in a Linux binary and wants to confirm it by observing calls or values at runtime.
An agent on macOS wants to see arguments passed to a Mach-O function without writing a debugger script.
An agent needs to compare a program's file and process effects between two inputs.

Expected content

The skill should give native runtime questions a route, with the following guidance:

The skill's routing table and guide index include a native runtime route, and a reference file covers native runtime observation.
For each tool family, the guidance states: the question it answers, supported host platforms, target effects (launch, attach, foreign code, memory or register changes), ownership and close behavior, and when to prefer a passive alternative.
The guidance includes a "confirm static hypotheses at runtime, then reconcile with correlate_static_and_runtime" step, with the expected before/after observation and what counts as failure stated before running.
Effects language matches the tool contracts and TOOL_EFFECTS declarations; no tool is described as read-only when its contract says otherwise.
The existing generated-skill checks continue to pass.

Documentation only: this describes existing tools and their existing gates and adds no capability. It should not include guidance for anti-debugging evasion or protection bypass, which CONTRIBUTING.md excludes.

Supporting evidence

Checked against main at commit 8ab5024 (rea-agents 6.5.0).

grep -ri over .agents/skills/reverse-engineer-anything/ for frida, rizin, cutter, start_gdb_session, gdb_console, capture_process_scenario, compare_process_captures, observe_native_ui, capture_native_ui_scenario, and trace_native_values returns no matches.
The guide index entry for runtime is "Browser/Electron observation (passive and scenario) and static/runtime reconciliation" (SKILL.md), and references/runtime-observation.md begins "# Browser and Electron runtime observation".
The tools are documented outside the skill: docs/frida-instrumentation.md, docs/reverse-engineering-tools.md, docs/process-capture.md. The installed skill does not ship docs/.
load_frida_script says "Scripts can read or change target memory and behavior" (src/contracts/fridaToolContracts.ts).

Additional context

No REA document or test claims the skill covers native runtime tools, so this is missing guidance rather than incorrect guidance. A possible check: give a fresh agent only the installed skill and the question "confirm at runtime which function in this Linux ELF handles input X", and check that it picks a native runtime tool and states the correct platform and effects, rather than stopping at static analysis or trying a browser tool.

Related open work touching the same skill: #1587, PR #3411, PR #4204.

Originally created by @coygeek on GitHub (Oct 16, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/4397 ### Before submitting - [x] I searched the existing issues for a related documentation report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Documentation location `.agents/skills/reverse-engineer-anything/SKILL.md` (target routing and guide index) and `.agents/skills/reverse-engineer-anything/references/runtime-observation.md` ### Problem The skill's guide index sends runtime questions only to "Browser/Electron observation (passive and scenario) and static/runtime reconciliation" (`.agents/skills/reverse-engineer-anything/SKILL.md`), and `references/runtime-observation.md` is titled "Browser and Electron runtime observation" and contains no native-process content. A search of the skill directory finds no mention of `frida`, `rizin`, `cutter`, `start_gdb_session`, `gdb_console`, `capture_process_scenario`, `compare_process_captures`, `observe_native_ui`, `capture_native_ui_scenario`, or `trace_native_values`. Native runtime coverage in the skill is limited to `observe_native_calls` for Mach-O under LLDB and GDB/pwndbg as debugger context for recorded core files (`references/native-and-artifacts.md`). The routing table sends native executables to `open_binary` and static analysis; a question about a native program's runtime behavior has no route. The repository documents these tools elsewhere (`docs/frida-instrumentation.md`, `docs/reverse-engineering-tools.md`, `docs/process-capture.md`), and they carry significant effects that agents need to know before calling them: Frida scripts "can read or change target memory and behavior", GDB and Rizin debug consoles are unrestricted, spawned processes start paused, GDB and Rizin debug are Linux-only, and REA never kills an attached target. An agent working only from the skill does not learn any of this. Affected: agents that rely on the installed skill to choose tools. Examples: - An agent has a static hypothesis about which function handles a message in a Linux binary and wants to confirm it by observing calls or values at runtime. - An agent on macOS wants to see arguments passed to a Mach-O function without writing a debugger script. - An agent needs to compare a program's file and process effects between two inputs. ### Expected content The skill should give native runtime questions a route, with the following guidance: - The skill's routing table and guide index include a native runtime route, and a reference file covers native runtime observation. - For each tool family, the guidance states: the question it answers, supported host platforms, target effects (launch, attach, foreign code, memory or register changes), ownership and close behavior, and when to prefer a passive alternative. - The guidance includes a "confirm static hypotheses at runtime, then reconcile with `correlate_static_and_runtime`" step, with the expected before/after observation and what counts as failure stated before running. - Effects language matches the tool contracts and `TOOL_EFFECTS` declarations; no tool is described as read-only when its contract says otherwise. - The existing generated-skill checks continue to pass. Documentation only: this describes existing tools and their existing gates and adds no capability. It should not include guidance for anti-debugging evasion or protection bypass, which `CONTRIBUTING.md` excludes. ### Supporting evidence Checked against `main` at commit 8ab5024 (`rea-agents` 6.5.0). - `grep -ri` over `.agents/skills/reverse-engineer-anything/` for `frida`, `rizin`, `cutter`, `start_gdb_session`, `gdb_console`, `capture_process_scenario`, `compare_process_captures`, `observe_native_ui`, `capture_native_ui_scenario`, and `trace_native_values` returns no matches. - The guide index entry for runtime is "Browser/Electron observation (passive and scenario) and static/runtime reconciliation" (`SKILL.md`), and `references/runtime-observation.md` begins "# Browser and Electron runtime observation". - The tools are documented outside the skill: `docs/frida-instrumentation.md`, `docs/reverse-engineering-tools.md`, `docs/process-capture.md`. The installed skill does not ship `docs/`. - `load_frida_script` says "Scripts can read or change target memory and behavior" (`src/contracts/fridaToolContracts.ts`). ### Additional context No REA document or test claims the skill covers native runtime tools, so this is missing guidance rather than incorrect guidance. A possible check: give a fresh agent only the installed skill and the question "confirm at runtime which function in this Linux ELF handles input X", and check that it picks a native runtime tool and states the correct platform and effects, rather than stopping at static analysis or trying a browser tool. Related open work touching the same skill: #1587, PR #3411, PR #4204.

bsakaguchi commented

2026-10-17 21:03:22 +00:00

Author

Owner

@1052326311 commented on GitHub (Oct 16, 2026):

Prepared a documentation candidate: d45c7f71. It adds native routing/index entries and a focused native section in the existing shipped runtime reference, avoiding a new installer inventory. The guide covers tool questions, hosts, execution/mutation effects, ownership/close behavior, passive alternatives and declared hypothesis/negative-control checks.

Source tracing also distinguishes static trace_native_values from runtime observation, and explains that correlate_static_and_runtime requires supported comparison Evidence rather than arbitrary Frida/console/UI records. All16 setupSkill tests, check and docs:check pass; links, native anchor and referenced tool names resolve. This is source/bundle validation, not real-provider execution or an empirical agent-routing evaluation.

Publication is currently blocked by HTTPS connectivity and explicit CreateRef permission denial. The exact commit object is uploaded and verified, but no topic branch or PR exists yet. Human initial-readiness declarations remain for the human owner.

@1052326311 commented on GitHub (Oct 16, 2026): Prepared a documentation candidate: [d45c7f71](https://github.com/1052326311/rea/commit/d45c7f7128638385814ca065f6f50a6482c13674). It adds native routing/index entries and a focused native section in the existing shipped runtime reference, avoiding a new installer inventory. The guide covers tool questions, hosts, execution/mutation effects, ownership/close behavior, passive alternatives and declared hypothesis/negative-control checks. Source tracing also distinguishes static trace_native_values from runtime observation, and explains that correlate_static_and_runtime requires supported comparison Evidence rather than arbitrary Frida/console/UI records. All16 setupSkill tests, check and docs:check pass; links, native anchor and referenced tool names resolve. This is source/bundle validation, not real-provider execution or an empirical agent-routing evaluation. Publication is currently blocked by HTTPS connectivity and explicit CreateRef permission denial. The exact commit object is uploaded and verified, but no topic branch or PR exists yet. Human initial-readiness declarations remain for the human owner.

bsakaguchi closed this issue

2026-10-31 00:32:32 +00:00

bsakaguchi commented

2026-10-31 00:32:33 +00:00

Author

Owner

@coygeek commented on GitHub (Oct 24, 2026):

Resolved in the current skill guidance on main at d87e73f2db.

The target-routing section now has a native runtime route, and the guide index includes native instrumentation, debugging, process capture, and static/runtime comparison. The new native runtime section covers LLDB call observation, Frida, GDB/Rizin, process capture, and native UI, with host requirements, execution/mutation effects, ownership, cleanup, and passive alternatives. It also requires expected/failure/stopping conditions and explains the supported correlate_static_and_runtime inputs and limitations.

I inspected that guidance against the current effect declarations and confirmed the built runtime-observation reference matches its authored source byte-for-byte. This is verification of the documentation fix, without executing native instrumentation or debugger workflows. Closing as completed on main.

@coygeek commented on GitHub (Oct 24, 2026): Resolved in the current skill guidance on `main` at d87e73f2db50be72d33a4b3c7199af35518b441c. The [target-routing section now has a native runtime route](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/.agents/skills/reverse-engineer-anything/SKILL.md#L125-L130), and the guide index includes native instrumentation, debugging, process capture, and static/runtime comparison. The [new native runtime section](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/.agents/skills/reverse-engineer-anything/references/runtime-observation.md#L3-L47) covers LLDB call observation, Frida, GDB/Rizin, process capture, and native UI, with host requirements, execution/mutation effects, ownership, cleanup, and passive alternatives. It also requires expected/failure/stopping conditions and explains the supported `correlate_static_and_runtime` inputs and limitations. I inspected that guidance against the current effect declarations and confirmed the built runtime-observation reference matches its authored source byte-for-byte. This is verification of the documentation fix, without executing native instrumentation or debugger workflows. Closing as completed on main.

bsakaguchi referenced this issue

2026-10-31 00:55:07 +00:00

[PR #1752] [MERGED] docs: celebrate 80,000 GitHub stars 🎉 #4793

bsakaguchi referenced this issue

2026-10-31 00:55:09 +00:00

[PR #1758] [MERGED] docs(i18n): celebrate 80,000 GitHub stars in translated READMEs #4798