morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[PR #2341] [MERGED] feat(native): search analyzed memory for masked byte patterns #5175

New issue

Closed

opened 2026-10-31 01:04:33 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 01:04:33 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/2341
Author: @mvanhorn
Created: 10/12/2026
Status: ✅ Merged
Merged: 10/13/2026
Merged by: @morluto

Base: main ← Head: fix/masked-byte-pattern-search

📝 Commits (5)

527278a feat(native): search analyzed memory for masked byte patterns
742a02b fix(native): limit unbounded byte search to loaded memory
3b87a5a fix(native): reconcile byte search coverage and target provenance
c661f8f fix(native): preserve implicit address spaces in range consumers
82cc51f fix(native): reject present malformed byte search state

📊 Changes

24 files changed (+980 additions, -36 deletions)

View changed files

📝 bridge/ghidra/ReaGhidraBridge.java (+162 -12)
📝 docs/ghidra-provider.md (+29 -0)
📝 docs/native-investigation.md (+14 -0)
📝 docs/user-visible-errors.csv (+3 -0)
📝 scripts/verify-real-ghidra-raw.mjs (+88 -0)
📝 src/application/DirectAnalysis.ts (+1 -0)
📝 src/cli/coreNativeCommands.ts (+40 -0)
📝 src/cliCommandNames.ts (+1 -0)
📝 src/contracts/hopperMemoryContracts.ts (+19 -0)
📝 src/contracts/toolContractExamples.ts (+1 -0)
📝 src/contracts/toolEffects.ts (+1 -0)
📝 src/contracts/toolOutputSchemaGroups.ts (+2 -0)
📝 src/domain/hopperValues.ts (+5 -14)
➕ src/domain/native/addressCoordinate.ts (+11 -0)
➕ src/domain/native/bytePattern.test.ts (+33 -0)
➕ src/domain/native/bytePattern.ts (+125 -0)
📝 src/ghidra/GhidraInventoryValues.ts (+34 -0)
📝 src/ghidra/GhidraProviderCapabilities.ts (+5 -0)
📝 src/hopper/HopperProviderCapabilities.ts (+7 -3)
📝 tests/boundary/mcp/ghidraInventoryMcp.test.ts (+270 -2)

...and 4 more files

📄 Description

Summary

Agents can now find where a byte signature occurs in an opened native target: search_bytes (CLI search-bytes) takes hexadecimal byte pairs with ?? wildcards and an optional bit mask, searches Ghidra's initialized analyzed memory, and returns ordered matches with segment, source file offset and containing procedure, plus exact page coverage and a bound continuation cursor. Hopper advertises the operation as unavailable with a typed reason.

No issue is linked. This is a new native tool, proposed here with the implementation.

Demo

Full-quality MP4

The video shows real output from this branch's MCP server and Ghidra provider code, driven against the repository's fake Ghidra fixture (tests/fixtures/fakeGhidra.mjs), since Ghidra is not installed on this host. The match addresses come from the fixture. It shows 48 8b ?? ?? 89 matching at 0x1000 and 0x1005 with .bss listed as skipped, the same search paged with limit: 1 and next_cursor, and ?? alone rejected as invalid.

Problem and expected behavior

read_bytes reads bytes at an address the caller already knows, and search_strings and procedure search cover names and strings. On upstream/main (2e45b4d) nothing answers "where does 48 8B ?? ?? 89 occur", so locating a constant table, a prologue, a magic header or an opcode sequence with wildcarded operands means reading memory page by page and matching client side.

With this change, one call such as {"pattern": "48 8B ?? ?? 89", "limit": 100} on an open Ghidra session returns observed Evidence with matches (address, segment, file_offset or null, procedure {address, name} or null), searched (blocks and bytes examined on this page, skipped_uninitialized), complete and next_cursor. Repeating the same query with cursor: next_cursor continues until complete is true.

Change and scope

Pattern grammar is owned by src/domain/native/bytePattern.ts and shared by the MCP contract, the Ghidra inventory schema and the CLI. A pattern is hexadecimal byte pairs (either case) or ??, with optional whitespace between pairs. Empty patterns, odd nibbles (4, 4?), non-hex characters and patterns where every bit is unconstrained (??, or ff with mask 00) are rejected as typed input errors before dispatch. The optional mask is hexadecimal pairs of the same length; each mask byte selects which bits of the pattern byte must match (pattern: "a0", mask: "f0" matches a0 through af). ?? stays fully unconstrained even when a mask is given, and an omitted mask means ff for every non-wildcard byte.

Optional start and end are inclusive and must lie in one address space with start <= end. Without bounds every memory block is eligible, in Ghidra address order. limit accepts 1 to 1000 and defaults to 100.

The Ghidra bridge (searchMemoryBytes in bridge/ghidra/ReaGhidraBridge.java) walks blocks sorted by start address and calls Memory.findBytes per initialized block with the end bound pulled back by length - 1, so a match never crosses a block boundary. After each hit it resumes at the next address, so overlapping matches are kept (00 00 over 00 00 00 gives two hits). Uninitialized blocks inside the bounds are never searched and are listed in searched.skipped_uninitialized on every page. searched.blocks reports the inclusive ranges examined for this page, including the lookahead that found the next unreturned match, so coverage can overlap between pages and should not be summed as unique memory.

The cursor is <binding>:<address>. The binding is SHA-256 over a JSON array of the target SHA-256, profile digest, normalized pattern bytes, masks, canonical start and end, and the document. The address is the first match not yet returned, so resuming neither skips nor repeats a match. The bridge rejects a cursor whose binding differs from the current target and query, or whose address is outside initialized memory or the query bounds. Cancellation is checked between blocks and around each findBytes call; it returns a typed cancelled error, discards the active Ghidra database and never produces a successful partial page.

The bridge re-validates its request at the socket boundary: required keys document, pattern, limit, allowed extras mask, start, end, cursor, anything else is invalid_request. It also re-applies the pattern, mask, constraint and bounds rules instead of trusting the TypeScript layer.

file_offset comes from a new shared uniqueFileOffset helper that returns an offset only when exactly one file-backed source mapping covers the address and the offset fits a JavaScript safe integer. address_to_file_offset now uses the same helper. That is a small visible change: its failure text changes from Address has N authoritative source mappings: <addr> to Address lacks a unique safe file offset: <addr>, and an offset above Number.MAX_SAFE_INTEGER is now rejected instead of returned.

Hopper lists search_bytes in its capabilities with available: false and the reason "Masked byte search has not been verified for Hopper.", so the call fails as capability_unavailable without launching Hopper.

Out of scope: a Hopper implementation, searching uninitialized or overlay memory, matches spanning block boundaries, string or regex search over bytes, and any write, patch or execution.

Contract and boundary impact

Semantic owner and earliest changed stage: domain grammar, parser and result schema in src/domain/native/bytePattern.ts (bytePatternFields, parseBytePattern, validateBytePattern, byteSearchResultSchema); Ghidra provider input/result schemas in src/ghidra/GhidraInventoryValues.ts; search in bridge/ghidra/ReaGhidraBridge.java.
Contribution scope fit and linked maintainer decision: a static query over memory Ghidra has already loaded for the caller's target; no target execution, injection, mutation or protection recovery, so no maintainer scope decision is required. Not applicable.
CLI and MCP/tool-catalog contract: new MCP tool search_bytes (src/contracts/hopperMemoryContracts.ts, strict input with pattern, mask?, document, start?, end?, limit 1 to 1000 default 100, cursor?; output schema in src/contracts/toolOutputSchemaGroups.ts; example {pattern: "48 8B ?? ?? 89"} in src/contracts/toolContractExamples.ts; effect evidence in src/contracts/toolEffects.ts). New CLI command search-bytes <path> <pattern> --mask --start --end --cursor --limit plus the existing native open options (src/cli/coreNativeCommands.ts, src/cliCommandNames.ts, src/application/DirectAnalysis.ts). The result schema rejects pages where complete disagrees with next_cursor or searched.bytes disagrees with the block sum, and the Ghidra schema also requires canonical addresses.
Provider, bridge, target-format, or platform compatibility: Ghidra gains search_bytes in its inventory, bridge dispatch and capability limitations (src/ghidra/GhidraProviderCapabilities.ts); same effects as other Ghidra reads, snapshot cache policy. Hopper reports the operation unavailable (src/hopper/HopperProviderCapabilities.ts). address_to_file_offset error text changes as described above.
Evidence, artifact, provenance, or reconstruction contract: observed Evidence bound to the session's target and profile. Each page carries its own coverage and skipped uninitialized blocks; the capability limitations state that coverage is per page, that uninitialized memory and cross-block spans are not searched, and that cancellation yields no partial page.
Target selection, authority, upstream behavior, and cleanup details (if applicable): uses the target the caller opened (or the CLI path) through the existing Ghidra provider path; reads loader-adjusted memory only. No new processes, writes or temp files beyond what every Ghidra operation already uses. Cancellation discards the active Ghidra database through the existing provider path.
Generated metadata (docs/public/product-catalog.json), package, or installation impact: docs in docs/ghidra-provider.md (Masked byte search) and docs/native-investigation.md (Locate a byte signature); three rows in docs/user-visible-errors.csv (unconstrained pattern, unequal mask length, cursor reused with another query). The new tool and command appear in the generated catalog; no authored skill text changed. metadata:check and docs:check pass; no build outputs committed.

Expected effects (check known effects; use documentation-only on its own).
Unchecked items do not establish absence; leave uncertainties visible.

Read existing files or captured data.
Launch or attach to a process.
Inject instrumentation or run scripts/code.
Change process memory, registers, files, or configuration.
Contact network services.
Recover protected code or work around protections.
Documentation or usability only; no target interaction.
Not sure yet; investigation or maintainer guidance needed.

The only process involved is the headless Ghidra analyzer that every Ghidra operation already starts; its capability effects are unchanged and the target itself is never launched or attached.

Evidence and regression coverage

Tests added or updated: src/domain/native/bytePattern.test.ts (whitespace, tab and no-break-space separators; ?? and bit masks combined; empty, odd-nibble, 4?, non-hex, all-wildcard, all-zero-mask, unequal-length and non-hex masks rejected with typed issues on the right field); tests/boundary/mcp/ghidraInventoryMcp.test.ts (a page and its continuation retained as Evidence; a page whose complete contradicts next_cursor rejected as unreadable_output; invalid patterns rejected before any provider call; cancellation projected as typed cancelled with no structured content; an end-to-end run through GhidraProvider and the socket bridge using tests/fixtures/fakeGhidra.mjs, covering pagination with limit: 1, cursor continuation, skipped .bss coverage, file offsets and procedure); tests/boundary/mcp/officialArguments.test.ts (CLI and MCP dispatch the same arguments with limit: 100 defaulted; ??, unequal mask, 4?, limit: 1001 and cursor: null rejected before dispatch); tests/boundary/mcp/officialTools.test.ts (Hopper returns capability_unavailable for search_bytes without launching; strict schema rejects unknown keys and defaults limit to 100). scripts/verify-real-ghidra-raw.mjs gains real-Ghidra assertions (see gaps).
Base reproduction or other evidence: executed. The three updated boundary test files plus the updated fake Ghidra fixture, copied onto a clone at 2e45b4d, give 7 failed and 9 passed: every new byte-search test fails (Tool search_bytes not found, or the contract lookup returns undefined), and the pre-existing tests in those files still pass. bytePattern.test.ts was not run on the base because its module does not exist there.
User-visible CLI/MCP output (if applicable): from the fake Ghidra fixture, the first page of {"pattern": "48 8b ?? ?? 89", "limit": 1} is matches: [{address: "0x1000", segment: ".text", file_offset: 0, procedure: {address: "0x1000", name: "fixture_main"}}], searched: {blocks: [{segment: ".text", start: "0x1000", end: "0x1009", bytes: 10}], bytes: 10, skipped_uninitialized: [{segment: ".bss", start: "0x2000", end: "0x200f", bytes: 16}]}, complete: false; the continuation returns 0x1005 with file_offset: 5 and complete: true. {"pattern": "??"} fails with Pattern must constrain at least one bit. On Hopper the call fails with capability_unavailable, provider_id: "hopper", operation: "search_bytes". These values are fixture output, not a real Ghidra run.
Remaining proof gaps: real Ghidra is not installed on this host, so npm run verify:ghidra:raw (needs GHIDRA_INSTALL_DIR) was not run, and the new assertions in scripts/verify-real-ghidra-raw.mjs were not executed. Those assertions cover a single-match signature with file_offset: 0 and procedure, a 1005-match file paged as 1000 then 5 with the continuation starting at 0x2770 (no skip or repeat), a bf/f0 nibble mask, overlapping 00 00 matches at 0x1002 and 0x1003, a cursor rejected for a different pattern, the target file unchanged after close, and first-search timing. Until that lane runs, the Java search loop, cursor binding and resume, block clamping and the address_to_file_offset refactor are source-derived; the TypeScript side is covered by the fixture boundary tests and unit tests above.

For evidence-bearing changes:

Observed, derived, and inferred claims remain distinguishable.
Artifact identity, source provenance, and failed attempts remain preserved.
Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

All on macOS arm64 with Node 26.5.0 and npm 12.2.0.

npm run test:focused -- src/domain/native/bytePattern.test.ts \
  tests/boundary/mcp/ghidraInventoryMcp.test.ts \
  tests/boundary/mcp/officialArguments.test.ts \
  tests/boundary/mcp/officialTools.test.ts     # pass: 4 files, 25 tests
npm run check:fast                             # pass (typecheck, lint)
npm run check                                  # pass (typecheck, lint:built 0 errors, format:check, knip, metadata:check, architecture guards across 1213 files)
npm run docs:check                             # pass
npm run verify:test-discovery                  # pass: 951 test files discovered once
npm run metadata:check                         # pass

On a clone at 2e45b4d with only the updated boundary tests and fixture applied, the same focused run over the three boundary files fails 7 of 16 tests, all of them the new byte-search cases.

npm run verify:ghidra:raw was not run (no Ghidra install on this host). npm run check:pr was not run for this change.

Compatibility, safety, and release

Breaking changes or migration steps: none for callers of existing tools beyond the address_to_file_offset failure text, which now reads Address lacks a unique safe file offset: <addr>, and its rejection of offsets above Number.MAX_SAFE_INTEGER, which previously could not be represented exactly as a JavaScript number anyway. The new tool and command are additive.
Real Hopper/Ghidra, browser, or OS coverage: no real Ghidra run; fixture and unit coverage only, on macOS. Hopper reports the operation unavailable by design and is not exercised beyond that.
Package or release metadata impact: new tool and command appear in the generated catalog; no dependency changes.
Security, privacy, process, or containment review: read-only query over memory Ghidra has already loaded; no target execution, writes to the target, or network. Results are bounded at 1000 matches per page, cursors are bound to target, profile and query, and cancellation is checked inside the search loop.

Before merge

This PR addresses a concrete problem or an agreed enhancement.
Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
The PR has one focused outcome and the title follows type(scope): outcome.
Related issue is linked, or the reason for not linking one is stated above.
Tests cover changed observable behavior and meaningful failure paths.
Owning docs, contracts, and generated metadata are updated where needed.
User-visible CLI/MCP changes include representative output.
I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/2341 **Author:** [@mvanhorn](https://github.com/mvanhorn) **Created:** 10/12/2026 **Status:** ✅ Merged **Merged:** 10/13/2026 **Merged by:** [@morluto](https://github.com/morluto) **Base:** `main` ← **Head:** `fix/masked-byte-pattern-search` --- ### 📝 Commits (5) - [`527278a`](https://github.com/morluto/rea/commit/527278ac3696e0a20e036685d13df1e5aef68cab) feat(native): search analyzed memory for masked byte patterns - [`742a02b`](https://github.com/morluto/rea/commit/742a02b86b7050aa6ea6f4884570e05b4397a4b8) fix(native): limit unbounded byte search to loaded memory - [`3b87a5a`](https://github.com/morluto/rea/commit/3b87a5a565b07a1e7950d448c20402b8aa25547c) fix(native): reconcile byte search coverage and target provenance - [`c661f8f`](https://github.com/morluto/rea/commit/c661f8f428d3e3b711ba17d540a41dca1983f39c) fix(native): preserve implicit address spaces in range consumers - [`82cc51f`](https://github.com/morluto/rea/commit/82cc51fa0ee309acfff52980847a582bb82783c1) fix(native): reject present malformed byte search state ### 📊 Changes **24 files changed** (+980 additions, -36 deletions) <details> <summary>View changed files</summary> 📝 `bridge/ghidra/ReaGhidraBridge.java` (+162 -12) 📝 `docs/ghidra-provider.md` (+29 -0) 📝 `docs/native-investigation.md` (+14 -0) 📝 `docs/user-visible-errors.csv` (+3 -0) 📝 `scripts/verify-real-ghidra-raw.mjs` (+88 -0) 📝 `src/application/DirectAnalysis.ts` (+1 -0) 📝 `src/cli/coreNativeCommands.ts` (+40 -0) 📝 `src/cliCommandNames.ts` (+1 -0) 📝 `src/contracts/hopperMemoryContracts.ts` (+19 -0) 📝 `src/contracts/toolContractExamples.ts` (+1 -0) 📝 `src/contracts/toolEffects.ts` (+1 -0) 📝 `src/contracts/toolOutputSchemaGroups.ts` (+2 -0) 📝 `src/domain/hopperValues.ts` (+5 -14) ➕ `src/domain/native/addressCoordinate.ts` (+11 -0) ➕ `src/domain/native/bytePattern.test.ts` (+33 -0) ➕ `src/domain/native/bytePattern.ts` (+125 -0) 📝 `src/ghidra/GhidraInventoryValues.ts` (+34 -0) 📝 `src/ghidra/GhidraProviderCapabilities.ts` (+5 -0) 📝 `src/hopper/HopperProviderCapabilities.ts` (+7 -3) 📝 `tests/boundary/mcp/ghidraInventoryMcp.test.ts` (+270 -2) _...and 4 more files_ </details> ### 📄 Description ## Summary Agents can now find where a byte signature occurs in an opened native target: `search_bytes` (CLI `search-bytes`) takes hexadecimal byte pairs with `??` wildcards and an optional bit mask, searches Ghidra's initialized analyzed memory, and returns ordered matches with segment, source file offset and containing procedure, plus exact page coverage and a bound continuation cursor. Hopper advertises the operation as unavailable with a typed reason. No issue is linked. This is a new native tool, proposed here with the implementation. ## Demo ![Masked byte search demo](https://raw.githubusercontent.com/mvanhorn/pr-demo-assets/main/rea/2026-10-12-021-feat-masked-byte-pattern-search/preview.gif) [Full-quality MP4](https://raw.githubusercontent.com/mvanhorn/pr-demo-assets/main/rea/2026-10-12-021-feat-masked-byte-pattern-search/out.mp4) The video shows real output from this branch's MCP server and Ghidra provider code, driven against the repository's fake Ghidra fixture (`tests/fixtures/fakeGhidra.mjs`), since Ghidra is not installed on this host. The match addresses come from the fixture. It shows `48 8b ?? ?? 89` matching at 0x1000 and 0x1005 with `.bss` listed as skipped, the same search paged with `limit: 1` and `next_cursor`, and `??` alone rejected as invalid. ## Problem and expected behavior `read_bytes` reads bytes at an address the caller already knows, and `search_strings` and procedure search cover names and strings. On `upstream/main` (`2e45b4d`) nothing answers "where does `48 8B ?? ?? 89` occur", so locating a constant table, a prologue, a magic header or an opcode sequence with wildcarded operands means reading memory page by page and matching client side. With this change, one call such as `{"pattern": "48 8B ?? ?? 89", "limit": 100}` on an open Ghidra session returns observed Evidence with `matches` (`address`, `segment`, `file_offset` or `null`, `procedure` `{address, name}` or `null`), `searched` (`blocks` and `bytes` examined on this page, `skipped_uninitialized`), `complete` and `next_cursor`. Repeating the same query with `cursor: next_cursor` continues until `complete` is true. ## Change and scope Pattern grammar is owned by `src/domain/native/bytePattern.ts` and shared by the MCP contract, the Ghidra inventory schema and the CLI. A pattern is hexadecimal byte pairs (either case) or `??`, with optional whitespace between pairs. Empty patterns, odd nibbles (`4`, `4?`), non-hex characters and patterns where every bit is unconstrained (`??`, or `ff` with mask `00`) are rejected as typed input errors before dispatch. The optional `mask` is hexadecimal pairs of the same length; each mask byte selects which bits of the pattern byte must match (`pattern: "a0", mask: "f0"` matches `a0` through `af`). `??` stays fully unconstrained even when a mask is given, and an omitted mask means `ff` for every non-wildcard byte. Optional `start` and `end` are inclusive and must lie in one address space with `start <= end`. Without bounds every memory block is eligible, in Ghidra address order. `limit` accepts 1 to 1000 and defaults to 100. The Ghidra bridge (`searchMemoryBytes` in `bridge/ghidra/ReaGhidraBridge.java`) walks blocks sorted by start address and calls `Memory.findBytes` per initialized block with the end bound pulled back by `length - 1`, so a match never crosses a block boundary. After each hit it resumes at the next address, so overlapping matches are kept (`00 00` over `00 00 00` gives two hits). Uninitialized blocks inside the bounds are never searched and are listed in `searched.skipped_uninitialized` on every page. `searched.blocks` reports the inclusive ranges examined for this page, including the lookahead that found the next unreturned match, so coverage can overlap between pages and should not be summed as unique memory. The cursor is `<binding>:<address>`. The binding is SHA-256 over a JSON array of the target SHA-256, profile digest, normalized pattern bytes, masks, canonical start and end, and the document. The address is the first match not yet returned, so resuming neither skips nor repeats a match. The bridge rejects a cursor whose binding differs from the current target and query, or whose address is outside initialized memory or the query bounds. Cancellation is checked between blocks and around each `findBytes` call; it returns a typed `cancelled` error, discards the active Ghidra database and never produces a successful partial page. The bridge re-validates its request at the socket boundary: required keys `document`, `pattern`, `limit`, allowed extras `mask`, `start`, `end`, `cursor`, anything else is `invalid_request`. It also re-applies the pattern, mask, constraint and bounds rules instead of trusting the TypeScript layer. `file_offset` comes from a new shared `uniqueFileOffset` helper that returns an offset only when exactly one file-backed source mapping covers the address and the offset fits a JavaScript safe integer. `address_to_file_offset` now uses the same helper. That is a small visible change: its failure text changes from `Address has N authoritative source mappings: <addr>` to `Address lacks a unique safe file offset: <addr>`, and an offset above `Number.MAX_SAFE_INTEGER` is now rejected instead of returned. Hopper lists `search_bytes` in its capabilities with `available: false` and the reason "Masked byte search has not been verified for Hopper.", so the call fails as `capability_unavailable` without launching Hopper. Out of scope: a Hopper implementation, searching uninitialized or overlay memory, matches spanning block boundaries, string or regex search over bytes, and any write, patch or execution. ## Contract and boundary impact - Semantic owner and earliest changed stage: domain grammar, parser and result schema in `src/domain/native/bytePattern.ts` (`bytePatternFields`, `parseBytePattern`, `validateBytePattern`, `byteSearchResultSchema`); Ghidra provider input/result schemas in `src/ghidra/GhidraInventoryValues.ts`; search in `bridge/ghidra/ReaGhidraBridge.java`. - Contribution scope fit and linked maintainer decision: a static query over memory Ghidra has already loaded for the caller's target; no target execution, injection, mutation or protection recovery, so no maintainer scope decision is required. Not applicable. - CLI and MCP/tool-catalog contract: new MCP tool `search_bytes` (`src/contracts/hopperMemoryContracts.ts`, strict input with `pattern`, `mask?`, `document`, `start?`, `end?`, `limit` 1 to 1000 default 100, `cursor?`; output schema in `src/contracts/toolOutputSchemaGroups.ts`; example `{pattern: "48 8B ?? ?? 89"}` in `src/contracts/toolContractExamples.ts`; effect `evidence` in `src/contracts/toolEffects.ts`). New CLI command `search-bytes <path> <pattern> --mask --start --end --cursor --limit` plus the existing native open options (`src/cli/coreNativeCommands.ts`, `src/cliCommandNames.ts`, `src/application/DirectAnalysis.ts`). The result schema rejects pages where `complete` disagrees with `next_cursor` or `searched.bytes` disagrees with the block sum, and the Ghidra schema also requires canonical addresses. - Provider, bridge, target-format, or platform compatibility: Ghidra gains `search_bytes` in its inventory, bridge dispatch and capability limitations (`src/ghidra/GhidraProviderCapabilities.ts`); same effects as other Ghidra reads, snapshot cache policy. Hopper reports the operation unavailable (`src/hopper/HopperProviderCapabilities.ts`). `address_to_file_offset` error text changes as described above. - Evidence, artifact, provenance, or reconstruction contract: observed Evidence bound to the session's target and profile. Each page carries its own coverage and skipped uninitialized blocks; the capability limitations state that coverage is per page, that uninitialized memory and cross-block spans are not searched, and that cancellation yields no partial page. - Target selection, authority, upstream behavior, and cleanup details (if applicable): uses the target the caller opened (or the CLI path) through the existing Ghidra provider path; reads loader-adjusted memory only. No new processes, writes or temp files beyond what every Ghidra operation already uses. Cancellation discards the active Ghidra database through the existing provider path. - Generated metadata (`docs/public/product-catalog.json`), package, or installation impact: docs in `docs/ghidra-provider.md` (Masked byte search) and `docs/native-investigation.md` (Locate a byte signature); three rows in `docs/user-visible-errors.csv` (unconstrained pattern, unequal mask length, cursor reused with another query). The new tool and command appear in the generated catalog; no authored skill text changed. `metadata:check` and `docs:check` pass; no build outputs committed. Expected effects (check known effects; use documentation-only on its own). Unchecked items do not establish absence; leave uncertainties visible. - [x] Read existing files or captured data. - [ ] Launch or attach to a process. - [ ] Inject instrumentation or run scripts/code. - [ ] Change process memory, registers, files, or configuration. - [ ] Contact network services. - [ ] Recover protected code or work around protections. - [ ] Documentation or usability only; no target interaction. - [ ] Not sure yet; investigation or maintainer guidance needed. The only process involved is the headless Ghidra analyzer that every Ghidra operation already starts; its capability effects are unchanged and the target itself is never launched or attached. ## Evidence and regression coverage - Tests added or updated: `src/domain/native/bytePattern.test.ts` (whitespace, tab and no-break-space separators; `??` and bit masks combined; empty, odd-nibble, `4?`, non-hex, all-wildcard, all-zero-mask, unequal-length and non-hex masks rejected with typed issues on the right field); `tests/boundary/mcp/ghidraInventoryMcp.test.ts` (a page and its continuation retained as Evidence; a page whose `complete` contradicts `next_cursor` rejected as `unreadable_output`; invalid patterns rejected before any provider call; cancellation projected as typed `cancelled` with no structured content; an end-to-end run through `GhidraProvider` and the socket bridge using `tests/fixtures/fakeGhidra.mjs`, covering pagination with `limit: 1`, cursor continuation, skipped `.bss` coverage, file offsets and procedure); `tests/boundary/mcp/officialArguments.test.ts` (CLI and MCP dispatch the same arguments with `limit: 100` defaulted; `??`, unequal mask, `4?`, `limit: 1001` and `cursor: null` rejected before dispatch); `tests/boundary/mcp/officialTools.test.ts` (Hopper returns `capability_unavailable` for `search_bytes` without launching; strict schema rejects unknown keys and defaults `limit` to 100). `scripts/verify-real-ghidra-raw.mjs` gains real-Ghidra assertions (see gaps). - Base reproduction or other evidence: executed. The three updated boundary test files plus the updated fake Ghidra fixture, copied onto a clone at `2e45b4d`, give 7 failed and 9 passed: every new byte-search test fails (`Tool search_bytes not found`, or the contract lookup returns `undefined`), and the pre-existing tests in those files still pass. `bytePattern.test.ts` was not run on the base because its module does not exist there. - User-visible CLI/MCP output (if applicable): from the fake Ghidra fixture, the first page of `{"pattern": "48 8b ?? ?? 89", "limit": 1}` is `matches: [{address: "0x1000", segment: ".text", file_offset: 0, procedure: {address: "0x1000", name: "fixture_main"}}]`, `searched: {blocks: [{segment: ".text", start: "0x1000", end: "0x1009", bytes: 10}], bytes: 10, skipped_uninitialized: [{segment: ".bss", start: "0x2000", end: "0x200f", bytes: 16}]}`, `complete: false`; the continuation returns `0x1005` with `file_offset: 5` and `complete: true`. `{"pattern": "??"}` fails with `Pattern must constrain at least one bit`. On Hopper the call fails with `capability_unavailable`, `provider_id: "hopper"`, `operation: "search_bytes"`. These values are fixture output, not a real Ghidra run. - Remaining proof gaps: real Ghidra is not installed on this host, so `npm run verify:ghidra:raw` (needs `GHIDRA_INSTALL_DIR`) was not run, and the new assertions in `scripts/verify-real-ghidra-raw.mjs` were not executed. Those assertions cover a single-match signature with `file_offset: 0` and procedure, a 1005-match file paged as 1000 then 5 with the continuation starting at `0x2770` (no skip or repeat), a `bf`/`f0` nibble mask, overlapping `00 00` matches at `0x1002` and `0x1003`, a cursor rejected for a different pattern, the target file unchanged after close, and first-search timing. Until that lane runs, the Java search loop, cursor binding and resume, block clamping and the `address_to_file_offset` refactor are source-derived; the TypeScript side is covered by the fixture boundary tests and unit tests above. For evidence-bearing changes: - [x] Observed, derived, and inferred claims remain distinguishable. - [x] Artifact identity, source provenance, and failed attempts remain preserved. - [x] Unsupported, incomplete, unavailable, or uncertain outcomes remain visible. ## Validation performed All on macOS arm64 with Node 26.5.0 and npm 12.2.0. ``` npm run test:focused -- src/domain/native/bytePattern.test.ts \ tests/boundary/mcp/ghidraInventoryMcp.test.ts \ tests/boundary/mcp/officialArguments.test.ts \ tests/boundary/mcp/officialTools.test.ts # pass: 4 files, 25 tests npm run check:fast # pass (typecheck, lint) npm run check # pass (typecheck, lint:built 0 errors, format:check, knip, metadata:check, architecture guards across 1213 files) npm run docs:check # pass npm run verify:test-discovery # pass: 951 test files discovered once npm run metadata:check # pass ``` On a clone at `2e45b4d` with only the updated boundary tests and fixture applied, the same focused run over the three boundary files fails 7 of 16 tests, all of them the new byte-search cases. `npm run verify:ghidra:raw` was not run (no Ghidra install on this host). `npm run check:pr` was not run for this change. ## Compatibility, safety, and release - Breaking changes or migration steps: none for callers of existing tools beyond the `address_to_file_offset` failure text, which now reads `Address lacks a unique safe file offset: <addr>`, and its rejection of offsets above `Number.MAX_SAFE_INTEGER`, which previously could not be represented exactly as a JavaScript number anyway. The new tool and command are additive. - Real Hopper/Ghidra, browser, or OS coverage: no real Ghidra run; fixture and unit coverage only, on macOS. Hopper reports the operation unavailable by design and is not exercised beyond that. - Package or release metadata impact: new tool and command appear in the generated catalog; no dependency changes. - Security, privacy, process, or containment review: read-only query over memory Ghidra has already loaded; no target execution, writes to the target, or network. Results are bounded at 1000 matches per page, cursors are bound to target, profile and query, and cancellation is checked inside the search loop. ## Before merge - [x] This PR addresses a concrete problem or an agreed enhancement. - [x] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff. - [x] The PR has one focused outcome and the title follows `type(scope): outcome`. - [x] Related issue is linked, or the reason for not linking one is stated above. - [x] Tests cover changed observable behavior and meaningful failure paths. - [x] Owning docs, contracts, and generated metadata are updated where needed. - [x] User-visible CLI/MCP changes include representative output. - [x] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-31 01:04:33 +00:00

closed this issue
added the
pull-request
label

bsakaguchi referenced this issue

2026-11-01 21:43:08 +00:00

[PR #5175] [CLOSED] fix(frida): count lone high surrogates in JSON byte budgets #6801

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#5175

No description provided.

Rows
Columns