morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[PR #6233] [MERGED] feat(native): let callers raise the Apple dispatch metadata input budget #7214

New issue

Closed

opened 2026-11-01 21:46:11 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-11-01 21:46:11 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/6233
Author: @kaoru0822-kitauji
Created: 10/22/2026
Status: ✅ Merged
Merged: 10/22/2026
Merged by: @kaoru0822-kitauji

Base: main ← Head: feat/dispatch-metadata-input-budget

📝 Commits (3)

cc34b58 feat(native): let callers raise the Apple dispatch metadata input budget
167fc0c fix(cli): describe --max-input-bytes in inspect-native-dispatch-metadata help
60d9d60 test(native): cover max_input_bytes forwarding through CLI and MCP

📊 Changes

13 files changed (+386 additions, -55 deletions)

View changed files

📝 docs/native-investigation.md (+8 -0)
📝 src/application/EnhancedTools.ts (+6 -20)
📝 src/application/native/NativeDispatchMetadataInspection.ts (+14 -3)
📝 src/application/native/NativeUiActionTrace.ts (+1 -1)
📝 src/cli/coreNativeCommands.ts (+10 -1)
📝 src/contracts/enhancedInputs.ts (+10 -0)
📝 src/contracts/enhancedToolContracts.ts (+1 -1)
📝 src/domain/native/objcSwiftMetadata.ts (+26 -0)
📝 src/native/AppleDispatchMetadata.ts (+71 -26)
📝 src/native/NativeMacOSProvider.ts (+7 -3)
➕ tests/boundary/cli/nativeDispatchInputBudget.test.ts (+62 -0)
➕ tests/boundary/filesystem/appleDispatchMetadataInputBudget.test.ts (+139 -0)
📝 tests/boundary/mcp/enhancedTools.04-uses-complete-inventories-inline.test.ts (+31 -0)

📄 Description

What broke: inspect_native_dispatch_metadata refused every Mach-O over a fixed 64 MiB before decoding. Its recovery said "no runtime override" and suggested more memory, which cannot help. Most large macOS frameworks are over that size.
Fix: caller-selected max_input_bytes (CLI --max-input-bytes), default 64 MiB, at most 1 GiB, recorded in results as input_admission; the refusal names the observed size, the budget, its source and what to select.
Risk: Low. The default and decoding are unchanged; under-budget output is identical to main plus the additive input_admission field.

Summary

Fixes #3917 (proposal in the issue thread).

Before: a 202 MB framework failed with Apple static metadata input exceeds its 67108864-byte (64 MiB) snapshot budget. This release has no runtime override. After: the same call returns max_input_bytes guidance, and --max-input-bytes 268435456 decodes it.

Problem and expected behavior

The byte reader holds the whole selected file in memory and refused anything over a hard-coded 64 MiB, so --max-records could not help. Expected per the issue: keep the 64 MiB default and the typed refusal, add an explicit, finite opt-in, report the selected limit in Evidence and errors, and give recovery that names an actual option.

Change and scope

max_input_bytes joins the shared inspect_native_dispatch_metadata input contract, so the MCP tool, the enhanced workflow and the CLI (--max-input-bytes, built from the same schema field) validate identically. It is a positive integer of at most 1 GiB.
inspectAppleDispatchMetadata (the owner) applies the default when omitted. It refuses before reading when the file is larger (and still if the file grows during the read), and records input_admission: {maximum_bytes, source: "default" | "caller", retained_bytes} in the result. Its optional arguments became one options object. inspectNativeDispatch likewise takes an options object, and its UI-action-trace caller keeps the default.
The refusal reports observed_input_bytes, input_bytes, input_bytes_source and maximum_selectable_input_bytes. Its remediation names the exact max_input_bytes to select. Above 1 GiB it points to a deep analysis provider instead.
Why 1 GiB: this reader keeps the whole file in REA's own heap. Larger images belong to a deep provider that loads them in its own process. Measured cost is about 2.9× the file size in peak RSS (below).
Not included: reading only the session-selected slice of a universal file, which my issue comment proposed as point 1. The decoder reports whole-file offsets from about 46 sites that index the full buffer, so that is a separate refactor; I'll follow up on the issue. With this PR the budget counts the whole file, as the docs state.

Contract and boundary impact

Semantic owner and earliest changed stage: inspectAppleDispatchMetadata admission, before any read.
Contribution scope fit and linked maintainer decision: static read of the already-bound target; no new authority. Proposal posted on #3917.
CLI and MCP/tool-catalog contract: additive optional max_input_bytes / --max-input-bytes; additive optional input_admission in results; refusal message and details changed (old text claimed no override).
Provider, bridge, target-format, or platform compatibility: macOS native Mach-O reader only; symbol-only fallback unchanged (no input_admission).
Evidence, artifact, provenance, or reconstruction contract: digest verification and all decoding unchanged.
Target selection, authority, upstream behavior, and cleanup details: unchanged.
Generated metadata, package, or installation impact: tool catalog description/schema regenerate in CI.

Expected effects:

Read existing files or captured data.
Launch or attach to a process.
Inject instrumentation or run scripts/code.
Change process memory, registers, files, or configuration.
Contact network services.
Recover protected code or work around protections.
Documentation or usability only; no target interaction.
Not sure yet; investigation or maintainer guidance needed.

Evidence and regression coverage

Tests added (tests/boundary/filesystem/appleDispatchMetadataInputBudget.test.ts):
- the default and the caller budget are recorded, and an exact-size budget decodes identically to the default;
- a refusal one byte under the file size checks the message, the remediation, reported_limits, and the absence of "no runtime override";
- sparse files just over the default (no opt-in) and over the 1 GiB maximum (deep-provider guidance);
- schema rejection of 0, -1, 1.5 and max+1;
- public paths: tests/boundary/cli/nativeDispatchInputBudget.test.ts runs the real CLI on macOS. It checks input_admission as default without the flag and caller with --max-input-bytes <size>, and a caller-sourced refusal one byte under. enhancedTools.04 asserts the MCP tool forwards max_input_bytes to the provider operation and omits it when unset.

Real-binary evidence, executed on macOS arm64 (Node 26.5.0, rea inspect-native-dispatch-metadata). Baseline = merge-base origin/main build:

Target	Build	Result	wall	CPU user+sys	peak RSS
Notion `Electron Framework` (202 MB, arm64)	baseline	`resource_constraint`, "no runtime override"	1.59 s	1.56 s	339 MiB
same	this PR, default	`resource_constraint` naming 201996992 bytes and `--max-input-bytes`	1.74 s	1.48 s	331 MiB
same	this PR, `--max-input-bytes 268435456`	decoded: 167 ObjC classes; `input_admission` caller/retained 201996992	1.75 s	2.35 s	866–869 MiB
Firefox `XUL` (353 MB, universal)	this PR, default	refused with guidance	1.40 s	1.57 s	336 MiB
same	this PR, `--max-input-bytes 402653184`	decoded: 234 classes, ObjC coverage complete	2.24 s	3.02 s	1022 MiB
Pages (47 MB, universal)	baseline vs this PR	output identical except `input_admission` (default, 46707264)	1.47 / 1.48 s	~1.8 s	501–527 / 504–508 MiB
BetterDisplay (35 MB, universal)	baseline vs this PR	output identical except `input_admission`	1.54 / 1.60 s	1.84 / 1.84 s	457 / 458 MiB

Under-budget cost is unchanged. Responses grow by 90 bytes for the new field; the existing max_records still bounds them (1.5–2.7 MB at the default 5000).

User-visible refusal (default, Notion):

code: resource_constraint
message: Apple static metadata input (201996992 bytes) exceeds the default input budget of 67108864 bytes (64 MiB).
remediation.action: Retry with max_input_bytes (CLI --max-input-bytes) of at least 201996992, at most 1073741824 bytes (1 GiB); REA holds that many bytes in memory for the read.
details.reported_limits: { input_bytes: 67108864, input_bytes_source: "default", maximum_selectable_input_bytes: 1073741824, observed_input_bytes: 201996992 }

Remaining proof gaps: no MCP-session run against a real >64 MiB binary (the CLI uses the same enhanced workflow and contract); Linux/Windows hosts return the existing capability refusal for this macOS reader.

For evidence-bearing changes:

Observed, derived, and inferred claims remain distinguishable.
Artifact identity, source provenance, and failed attempts remain preserved.
Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

npm run test:focused -- tests/boundary/filesystem/appleDispatchMetadataInputBudget.test.ts <19 existing files covering inspect_native_dispatch_metadata / inspectNativeDispatch / AppleDispatchMetadata>: 20 files; 358 passed, 2 skipped.
npm run test:focused -- tests/boundary/cli/nativeDispatchInputBudget.test.ts tests/boundary/mcp/enhancedTools.04-uses-complete-inventories-inline.test.ts tests/boundary/cli/cliOptionAliases.test.ts: all passed.
npm run check:fast — passed.
npm run format:check — passed.
npm run docs:check — passed.
Real-binary runs above (npm run build:cached on both trees).

Compatibility, safety, and release

Breaking changes or migration steps: none. Only the refusal wording changes; the error code stays resource_constraint.
Real Hopper/Ghidra, browser, or OS coverage: macOS arm64 with real Notion, Firefox, Pages and BetterDisplay binaries; no deep provider involved.
Package or release metadata impact: none.
Security, privacy, process, or containment review: the opt-in only raises a caller-chosen in-memory budget up to a finite 1 GiB; the size check, digest verification and cancellation are unchanged.

Author readiness

Human owner: @kaoru0822-kitauji

I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
I declare this PR ready for initial maintainer review.

Before merge

This PR addresses a concrete problem or an agreed enhancement.
Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
The PR has one focused outcome and the title follows type(scope): outcome.
Related issue is linked, or the reason for not linking one is stated above.
Tests cover changed observable behavior and meaningful failure paths.
Owning docs, contracts, and generated metadata are updated where needed.
User-visible CLI/MCP changes include representative output.
I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

🤖 Generated with Claude Code

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/6233 **Author:** [@kaoru0822-kitauji](https://github.com/kaoru0822-kitauji) **Created:** 10/22/2026 **Status:** ✅ Merged **Merged:** 10/22/2026 **Merged by:** [@kaoru0822-kitauji](https://github.com/kaoru0822-kitauji) **Base:** `main` ← **Head:** `feat/dispatch-metadata-input-budget` --- ### 📝 Commits (3) - [`cc34b58`](https://github.com/morluto/rea/commit/cc34b584d7b813716d181ae5ca4c3a31ea00ff17) feat(native): let callers raise the Apple dispatch metadata input budget - [`167fc0c`](https://github.com/morluto/rea/commit/167fc0c724a292b426f27c51cfb1a387529cc25f) fix(cli): describe --max-input-bytes in inspect-native-dispatch-metadata help - [`60d9d60`](https://github.com/morluto/rea/commit/60d9d6076e24c80a114c5ec3e6d97eb4430be9dd) test(native): cover max_input_bytes forwarding through CLI and MCP ### 📊 Changes **13 files changed** (+386 additions, -55 deletions) <details> <summary>View changed files</summary> 📝 `docs/native-investigation.md` (+8 -0) 📝 `src/application/EnhancedTools.ts` (+6 -20) 📝 `src/application/native/NativeDispatchMetadataInspection.ts` (+14 -3) 📝 `src/application/native/NativeUiActionTrace.ts` (+1 -1) 📝 `src/cli/coreNativeCommands.ts` (+10 -1) 📝 `src/contracts/enhancedInputs.ts` (+10 -0) 📝 `src/contracts/enhancedToolContracts.ts` (+1 -1) 📝 `src/domain/native/objcSwiftMetadata.ts` (+26 -0) 📝 `src/native/AppleDispatchMetadata.ts` (+71 -26) 📝 `src/native/NativeMacOSProvider.ts` (+7 -3) ➕ `tests/boundary/cli/nativeDispatchInputBudget.test.ts` (+62 -0) ➕ `tests/boundary/filesystem/appleDispatchMetadataInputBudget.test.ts` (+139 -0) 📝 `tests/boundary/mcp/enhancedTools.04-uses-complete-inventories-inline.test.ts` (+31 -0) </details> ### 📄 Description > **What broke:** `inspect_native_dispatch_metadata` refused every Mach-O over a fixed 64 MiB before decoding. Its recovery said "no runtime override" and suggested more memory, which cannot help. Most large macOS frameworks are over that size. > **Fix:** caller-selected `max_input_bytes` (CLI `--max-input-bytes`), default 64 MiB, at most 1 GiB, recorded in results as `input_admission`; the refusal names the observed size, the budget, its source and what to select. > **Risk:** Low. The default and decoding are unchanged; under-budget output is identical to `main` plus the additive `input_admission` field. ## Summary Fixes #3917 (proposal in the issue thread). Before: a 202 MB framework failed with `Apple static metadata input exceeds its 67108864-byte (64 MiB) snapshot budget. This release has no runtime override.` After: the same call returns `max_input_bytes` guidance, and `--max-input-bytes 268435456` decodes it. ## Problem and expected behavior The byte reader holds the whole selected file in memory and refused anything over a hard-coded 64 MiB, so `--max-records` could not help. Expected per the issue: keep the 64 MiB default and the typed refusal, add an explicit, finite opt-in, report the selected limit in Evidence and errors, and give recovery that names an actual option. ## Change and scope - `max_input_bytes` joins the shared `inspect_native_dispatch_metadata` input contract, so the MCP tool, the enhanced workflow and the CLI (`--max-input-bytes`, built from the same schema field) validate identically. It is a positive integer of at most 1 GiB. - `inspectAppleDispatchMetadata` (the owner) applies the default when omitted. It refuses before reading when the file is larger (and still if the file grows during the read), and records `input_admission: {maximum_bytes, source: "default" | "caller", retained_bytes}` in the result. Its optional arguments became one options object. `inspectNativeDispatch` likewise takes an options object, and its UI-action-trace caller keeps the default. - The refusal reports `observed_input_bytes`, `input_bytes`, `input_bytes_source` and `maximum_selectable_input_bytes`. Its remediation names the exact `max_input_bytes` to select. Above 1 GiB it points to a deep analysis provider instead. - Why 1 GiB: this reader keeps the whole file in REA's own heap. Larger images belong to a deep provider that loads them in its own process. Measured cost is about 2.9× the file size in peak RSS (below). - Not included: reading only the session-selected slice of a universal file, which my issue comment proposed as point 1. The decoder reports whole-file offsets from about 46 sites that index the full buffer, so that is a separate refactor; I'll follow up on the issue. With this PR the budget counts the whole file, as the docs state. ## Contract and boundary impact - Semantic owner and earliest changed stage: `inspectAppleDispatchMetadata` admission, before any read. - Contribution scope fit and linked maintainer decision: static read of the already-bound target; no new authority. Proposal posted on #3917. - CLI and MCP/tool-catalog contract: additive optional `max_input_bytes` / `--max-input-bytes`; additive optional `input_admission` in results; refusal message and details changed (old text claimed no override). - Provider, bridge, target-format, or platform compatibility: macOS native Mach-O reader only; symbol-only fallback unchanged (no `input_admission`). - Evidence, artifact, provenance, or reconstruction contract: digest verification and all decoding unchanged. - Target selection, authority, upstream behavior, and cleanup details: unchanged. - Generated metadata, package, or installation impact: tool catalog description/schema regenerate in CI. Expected effects: - [x] Read existing files or captured data. - [ ] Launch or attach to a process. - [ ] Inject instrumentation or run scripts/code. - [ ] Change process memory, registers, files, or configuration. - [ ] Contact network services. - [ ] Recover protected code or work around protections. - [ ] Documentation or usability only; no target interaction. - [ ] Not sure yet; investigation or maintainer guidance needed. ## Evidence and regression coverage - Tests added (`tests/boundary/filesystem/appleDispatchMetadataInputBudget.test.ts`): - the default and the caller budget are recorded, and an exact-size budget decodes identically to the default; - a refusal one byte under the file size checks the message, the remediation, `reported_limits`, and the absence of "no runtime override"; - sparse files just over the default (no opt-in) and over the 1 GiB maximum (deep-provider guidance); - schema rejection of `0`, `-1`, `1.5` and max+1; - public paths: `tests/boundary/cli/nativeDispatchInputBudget.test.ts` runs the real CLI on macOS. It checks `input_admission` as `default` without the flag and `caller` with `--max-input-bytes <size>`, and a caller-sourced refusal one byte under. `enhancedTools.04` asserts the MCP tool forwards `max_input_bytes` to the provider operation and omits it when unset. - Real-binary evidence, executed on macOS arm64 (Node 26.5.0, `rea inspect-native-dispatch-metadata`). Baseline = merge-base `origin/main` build: | Target | Build | Result | wall | CPU user+sys | peak RSS | | --- | --- | --- | --- | --- | --- | | Notion `Electron Framework` (202 MB, arm64) | baseline | `resource_constraint`, "no runtime override" | 1.59 s | 1.56 s | 339 MiB | | same | this PR, default | `resource_constraint` naming 201996992 bytes and `--max-input-bytes` | 1.74 s | 1.48 s | 331 MiB | | same | this PR, `--max-input-bytes 268435456` | decoded: 167 ObjC classes; `input_admission` caller/retained 201996992 | 1.75 s | 2.35 s | 866–869 MiB | | Firefox `XUL` (353 MB, universal) | this PR, default | refused with guidance | 1.40 s | 1.57 s | 336 MiB | | same | this PR, `--max-input-bytes 402653184` | decoded: 234 classes, ObjC coverage complete | 2.24 s | 3.02 s | 1022 MiB | | Pages (47 MB, universal) | baseline vs this PR | output identical except `input_admission` (default, 46707264) | 1.47 / 1.48 s | ~1.8 s | 501–527 / 504–508 MiB | | BetterDisplay (35 MB, universal) | baseline vs this PR | output identical except `input_admission` | 1.54 / 1.60 s | 1.84 / 1.84 s | 457 / 458 MiB | Under-budget cost is unchanged. Responses grow by 90 bytes for the new field; the existing `max_records` still bounds them (1.5–2.7 MB at the default 5000). - User-visible refusal (default, Notion): ``` code: resource_constraint message: Apple static metadata input (201996992 bytes) exceeds the default input budget of 67108864 bytes (64 MiB). remediation.action: Retry with max_input_bytes (CLI --max-input-bytes) of at least 201996992, at most 1073741824 bytes (1 GiB); REA holds that many bytes in memory for the read. details.reported_limits: { input_bytes: 67108864, input_bytes_source: "default", maximum_selectable_input_bytes: 1073741824, observed_input_bytes: 201996992 } ``` - Remaining proof gaps: no MCP-session run against a real >64 MiB binary (the CLI uses the same enhanced workflow and contract); Linux/Windows hosts return the existing capability refusal for this macOS reader. For evidence-bearing changes: - [x] Observed, derived, and inferred claims remain distinguishable. - [x] Artifact identity, source provenance, and failed attempts remain preserved. - [x] Unsupported, incomplete, unavailable, or uncertain outcomes remain visible. ## Validation performed - `npm run test:focused -- tests/boundary/filesystem/appleDispatchMetadataInputBudget.test.ts <19 existing files covering inspect_native_dispatch_metadata / inspectNativeDispatch / AppleDispatchMetadata>`: 20 files; 358 passed, 2 skipped. - `npm run test:focused -- tests/boundary/cli/nativeDispatchInputBudget.test.ts tests/boundary/mcp/enhancedTools.04-uses-complete-inventories-inline.test.ts tests/boundary/cli/cliOptionAliases.test.ts`: all passed. - `npm run check:fast` — passed. - `npm run format:check` — passed. - `npm run docs:check` — passed. - Real-binary runs above (`npm run build:cached` on both trees). ## Compatibility, safety, and release - Breaking changes or migration steps: none. Only the refusal wording changes; the error code stays `resource_constraint`. - Real Hopper/Ghidra, browser, or OS coverage: macOS arm64 with real Notion, Firefox, Pages and BetterDisplay binaries; no deep provider involved. - Package or release metadata impact: none. - Security, privacy, process, or containment review: the opt-in only raises a caller-chosen in-memory budget up to a finite 1 GiB; the size check, digest verification and cancellation are unchanged. ## Author readiness Human owner: @kaoru0822-kitauji - [x] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [x] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [x] I declare this PR ready for initial maintainer review. ## Before merge - [x] This PR addresses a concrete problem or an agreed enhancement. - [x] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff. - [x] The PR has one focused outcome and the title follows `type(scope): outcome`. - [x] Related issue is linked, or the reason for not linking one is stated above. - [x] Tests cover changed observable behavior and meaningful failure paths. - [x] Owning docs, contracts, and generated metadata are updated where needed. - [x] User-visible CLI/MCP changes include representative output. - [x] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-11-01 21:46:11 +00:00

closed this issue
added the
pull-request
label

bsakaguchi referenced this issue

2026-11-03 13:39:54 +00:00

[PR #7220] [MERGED] fix(provider-selection): name whether a rejection came from availability or target support #7723

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#7214

No description provided.

Rows
Columns