mirror of
https://github.com/morluto/rea.git
synced 2026-11-05 06:45:22 +00:00
[PR #4318] [MERGED] fix(kuna): stream string scans instead of capping them at 16 MiB #6168
Closed
opened 2026-10-31 01:12:23 +00:00 by bsakaguchi
·
0 comments
No Branch/Tag specified
main
codex/procedure-reference-stacked
release-please--branches--main--components--rea-agents
release-please--branches--main--components--rea-agents--release-notes
codex/kuna-independent-retry
codex/byte-search-overlay-coverage
codex/open-failure-target-receipt
codex/kuna-xref-evidence
codex/ghidra-ambiguous-procedure-recovery
codex/function-source-bundle
codex/trace-function-integration-prototype
codex/snapshot-source-closure
codex/recorded-evidence-array-pages
codex/evidence-recovery-redesign
codex/trace-feature-address-scope
codex/call-discovery-scope
codex/ghidra-literal-candidate-evidence
codex/decompile-directory-independent-retry
codex/retry-provenance-base
codex/value-flow-capability-experiment
codex/ghidra-containing-instruction
cursor/readme-discord-button-ef9f
dependabot/github_actions/actions/setup-node-7.1.0
dependabot/github_actions/actions/upload-artifact-7.0.2
dependabot/npm_and_yarn/npm-minor-and-patch-d8d97ce6df
codex/kuna-1989
fix/8559-native-ui-fixture-ownership
fix/8555-go-verifier-mcp-ownership
codex/provider-capability-recovery
fix/8642-rizin-target-stdin
codex/ghidra-memory-effect-scope
codex/ghidra-annotation-entry-selector
codex/ghidra-readonly-recovery
codex/ghidra-regex-verifier-recovery
codex/evidence-discovery-view-guidance
fix/8545-bulk-append-overflow
fix/8428-navigation-refusal-alternatives
codex/address-context-recovery
codex/matching-member-limitations
codex/unknown-scope-contract
codex/evidence-view-guidance
feat/8373-native-trace-views
codex/ghidra-snapshot-verifier-binding
codex/elf-export-visibility
codex/evidence-view-recovery
codex/native-reference-cli-help
codex/provider-candidate-observation-scope
codex/kuna-session-verification
codex/ghidra-switch-lifetime-guidance
codex/pseudocode-paging-discovery
codex/clarify-evidence-view-support
codex/kuna-xrefs-selector-recovery
codex/context-capacity-recovery
codex/evidence-closure-large-links
codex/same-function-evidence-identity
codex/ghidra-verifier-inspected-jdk
codex/audit-guided-workflows
codex/crash-prompt-retained-dossiers
codex/analysis-view-error-recovery
codex/kuna-verifier-target-selection
codex/ghidra-selected-project-call-coverage
codex/docs-native-analysis-scope
codex/ghidra-java-socket-boundary
codex/ghidra-retained-snapshot-verifier
codex/ghidra-search-inventory-facts
fix/8284-zero-argument-parameter-bindings
fix/8198-ui-action-handler-offset
fix/8257-elf-entry-label-identity
fix/8304-deferred-output-schema-projection
fix/8277-call-path-computed-target-uncertainty
fix/pr-8089-conflicts
fix/ci-static-duplicate-export
dependabot/npm_and_yarn/lint-staged-17.6.0
dependabot/github_actions/actions/download-artifact-8.0.2
dependabot/npm_and_yarn/write-file-atomic-8.0.0
dependabot/npm_and_yarn/canonicalize-5.1.0
dependabot/github_actions/actions/cache/restore-6.1.0
dependabot/npm_and_yarn/toon-format/toon-4.4.0
dependabot/github_actions/actions/deploy-pages-5.0.1
codex/review-fixes-pr-2403
codex/review-fixes-pr-2400
codex/process-terminal-width
codex/test-brittleness-cleanup
verify/7167-browser-connect-macos
fix/7167-browser-connect
feat/native-matching-allowance-setting
feat/browser-scenario-dry-run
test/managed-caller-parameters-20261024
codex/parser-fortythird-boundaries
fix/6926-pinvoke-platform-decorations-20261023
fix/6889-har-body-encoding-20261023
fix/6888-sqlite-schema-budget-20261023
fix/dependency-security-gates-6708
cursor/evidence-overwrite-rebase-b4ac
feat/4057-verify-native-exports
fix/windows-acl-observer-overhead-20261022
cursor/erlang-beam-restack-4476
cursor/pe-certificate-review-bf32
fix/6116-javascript-application-fs-errors
fix/6061-unmapped-runtime-script-unknown
feat/browser-scenario-summary
fix/pe-certificates
codex/verify-3407
cursor/rebase-5699
cursor/rebase-5352
cursor/rebase-5600
cursor/fix-cli-schema-omission-10a8
codex/audit-hopper-shutdown-reasons-20261021
fix/demangle-swift-omitted-record
fix/nib-boolean-value-types
cursor/cancellation-partial-observation-a5ac
fix/ib-object-limit-reasons
fix/electron-fuse-limit-advice
fix/kuna-doctor-dangling-cache
fix/fixed-budget-refusal-advice
fix/js-ipc-listener-dedup
fix/js-return-member-provenance
fix/dmg-integrity-retry-advice
fix/ida-doctor-readiness
fix/kuna-version-probe-output-drain
fix/legacy-swift-runtime-name-matching
fix/dylib-lazy-load-commands
fix/browser-scenario-startup-cancel
fix/swift-nested-punycode-classification
fix/compare-unselected-scopes
fix/objc-method-budget-coverage
fix/dmg-mounted-root-observations
audit/ci-tests-20261019-round2
release/6.7.0
fix/wabt-cleanup-retry-resolution
fix/electron-fuse-elf-pe-architecture
fix/native-plist-descriptor-ownership
fix/kuna-caller-cleanup-ownership
experiment/atomic-codemode-20261016
review/first-contributor-3776
fix/call-resolution-full-read-cycles
fix/member-mutation-selection-repeat
experiment/atomic-re-tools-20261015
perf/member-mutation-lazy-origins
perf/static-network-mutation-index
fix/hopper-startup-lifecycle
audit/3050-windows-proof-20261015
feat/windows-window-capture-2183
audit/3049-windows-proof-20261015
audit/3735-macos-fixtures-20261015
fix/batch-decompile-null-pseudocode
fix/kuna-inventory-honors-jobs
experiment/browser-codemode-tool-design-20261015
fix/hopper-clean-runtime-argv
codex/review-2049-27f1b367
review-fixes/pr-2787-routing
fix/3360-scoped-this-emitters
fix/3367-replaced-global-effects
fix/3838-compact-any-value-items
test/hopper-download-probe-20261015
codex/windows-runtime-creation-red-green-20261015
codex/windows-readonly-root-red-green-20261015
codex/windows-root-share-red-green-20261015
codex/exp-1685-win-mcp-startup
codex/bug-sweep-20261015
investigate/2190-windows-package-phases
investigate/3543-macos-cli-deadline
perf/3281-reverse-tools-feedback-quality
fix/3406-hopper-doctor-fixture
fix/3341-wrapper-require-test-context
fix/3301-owned-uninstall-fixtures
fix/3164-compiler-input-ownership
fix/pcap-cleanup-stdout-newline-20261014
perf/interface-builder-summary
codex/review-fixes-pr-1977
fix/ghidra-effective-jvm-platform
fix/docs-turkish-rust-link-20261013
fix/main-maintenance-ci-20261010
fix/recover-binary-session-process-effects
codex/review-fixes-pr-2457
codex/ci-required-cancellation-gate
fix/main-format-baseline-20261013
fix/setup-preserve-entry-fields
cursor/restore-main-checks-1ffc
fix/ghidra-bridge-hashmap-import
codex/shared-ci-jadx-json-default-20261013
codex/shared-ci-ghidra-test-groups-20261013
codex/shared-ci-java-fixture-format-20261013
codex/fix-intrinsic-invocation-boundaries
test/release-preflight-reusable-jobs
test/js-parameter-provenance-expectation
fix/linux-hopper-verification-current
test/2419-ghidra-elf-probe-size
followup/pr-1544-native-ida
codex/fold-pr-2230-contribution
docs/readme-responsible-use
fix/issue-2064
fix/issue-2050
fix/issue-2053
fix/issue-1968
fix/issue-1945
experiment/codemode-bridge-20261011
fix/tool-effects-negative-wording-1904-20261012
fix/1842-electron-interop-provenance
feat/recover-javascript-multi-input
refactor/website-i18n-prep
fix/nativeaot-pe-virtual-extent
fix/browser-verifier-ci-hang
fix/pe-resources-input-selection
docs/beginner-friendly-guides
docs/beginner-setup-guide
fix/javascript-analysis-worker-1563
fix/1649-ghidra-duplicate-names
fix/javascript-export-keys-integration-1369
test/maintenance-boundary-contracts-20261011
review/javascript-export-keys-1369
review/process-live-progress-1483
test/javascript-1495-growth-ratio
fix/ci-escape-timing
docs/normalized-terminal-geometry-1450
release/6.3.0
archive/release-6.3.0-pr1381
release/6.2.0
fix/electron-preload-dynamic-override
fix/javascript-self-referential-bundler-edges
release/6.1.0
cursor/grok-client-cherry-pick-0b56
feat/signature-posture
feat/dyld-shared-cache
feat/artifact-integrity-policy
feat/pkg-inventory
release/6.0.0
chore/sync-release-5.0.0
release/5.0.0
cursor/cloud-agent-node-787e
fix/javascript-xhr-open-endpoints
review/pr-711-realpath-fallback-wording
cleanup/replay-confinement
codex/test-suite-performance
codex/issue-foundations
codex/address-open-issues-2026-07-20
agent/javascript-vite-rollup-esbuild-adapters
agent/process-cleanup
agent/triage-integration
agent/unknown-commands-registry
feat/mcp-identity-metadata
morlvps/issue_cleanup
codex/installer-redesign
morl/roadmap-investigation-core
morl/roadmap-investigation-docs
release-please--branches--main--components--rea
rea-agents-6.8.0
rea-agents-6.7.0
rea-agents-6.6.0
experiment/atomic-codemode-wait-runtime-20261016
rea-agents-6.5.0
rea-agents-6.4.0
rea-agents-6.3.0
rea-agents-6.2.0
rea-agents-6.1.0
rea-agents-6.0.0
rea-agents-5.0.0
rea-agents-4.1.0
rea-agents-4.0.1
rea-agents-4.0.0
rea-agents-3.2.1
rea-agents-3.2.0
rea-agents-3.1.0
rea-agents-3.0.0
rea-agents-2.7.0
rea-agents-2.6.0
rea-agents-2.5.1
rea-agents-2.5.0
rea-agents-2.4.0
rea-agents-2.3.0
rea-agents-2.2.0
rea-agents-2.1.0
rea-agents-2.0.0
rea-agents-1.7.0
rea-agents-1.6.0
rea-agents-1.5.0
rea-agents-1.4.0
rea-agents-1.3.0
rea-agents-1.2.0
rea-agents-1.1.0
rea-agents-1.0.0
rea-agents-0.5.0
rea-agents-0.4.0
rea-agents-0.3.0
rea-0.2.1
rea-0.2.0
Labels
Clear labels
area: bridge
area: bridge
area: build
area: build
area: build
area: cli
area: domain
area: evidence
area: mcp
area: native
area: providers
area: runtime
area: server
area: setup
area: tests
bug
design
documentation
enhancement
enhancement
enhancement
experiment
help wanted
i18n
invalid
P0: critical
P1: high
P2: medium
P2: medium
P2: medium
P3: low
performance
provider-request
pull-request
Mirrored from GitHub Pull Request
question
roadmap
triage
No labels
area: bridge
area: bridge
area: build
area: build
area: build
area: cli
area: domain
area: evidence
area: mcp
area: native
area: providers
area: runtime
area: server
area: setup
area: tests
bug
design
documentation
enhancement
enhancement
enhancement
experiment
help wanted
i18n
invalid
P0: critical
P1: high
P2: medium
P2: medium
P2: medium
P3: low
performance
provider-request
pull-request
question
roadmap
triage
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".
No due date set.
Dependencies
No dependencies set.
Reference
morluto/rea#6168
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
📋 Pull Request Information
Original PR: https://github.com/morluto/rea/pull/4318
Author: @kaoru0822-kitauji
Created: 10/16/2026
Status: ✅ Merged
Merged: 10/16/2026
Merged by: @kaoru0822-kitauji
Base:
main← Head:fix/kuna-stream-string-scan📝 Commits (5)
48abd8ffix(kuna): stream string scans instead of capping them at 16 MiB46f6feefix(kuna): admit string scan text as it streams and drop sinks with their command0ce0dbdfix(kuna): type the stream-json tokenizer where it is imported8770e38fix(kuna): bound key and number assembly in streamed Kuna JSON72f2de0fix(kuna): stream the call graph instead of capping it at 32 MiB (#4329)📊 Changes
20 files changed (+1408 additions, -271 deletions)
View changed files
📝
docs/kuna-provider.md(+2 -2)📝
src/kuna/KunaCommand.ts(+5 -1)📝
src/kuna/KunaJson.ts(+19 -11)➕
src/kuna/KunaJsonStream.ts(+554 -0)📝
src/kuna/KunaLimits.ts(+3 -2)📝
src/kuna/KunaProjectExport.ts(+3 -4)📝
src/kuna/KunaQueryWork.ts(+43 -1)📝
src/kuna/KunaReferenceFiles.ts(+151 -74)📝
src/kuna/KunaReferenceQueries.ts(+20 -29)📝
src/kuna/KunaStringInventory.ts(+85 -54)📝
src/kuna/KunaStringQueries.ts(+27 -47)📝
src/kuna/KunaTargetCommands.ts(+3 -0)📝
src/process/OwnedCommand.ts(+58 -11)📝
tests/boundary/process/ownedCommand.test.ts(+70 -1)📝
tests/boundary/providers/kuna/fakeKuna.ts(+19 -5)📝
tests/boundary/providers/kuna/kunaCommandFailures.test.ts(+32 -3)📝
tests/boundary/providers/kuna/kunaReferenceFiles.test.ts(+105 -11)📝
tests/boundary/providers/kuna/kunaReferences.test.ts(+4 -1)📝
tests/boundary/providers/kuna/kunaStringInventory.test.ts(+152 -13)📝
tests/boundary/providers/kuna/kunaStrings.test.ts(+53 -1)📄 Description
Summary
Kuna
list_stringsandsearch_stringsfailed on every real app above roughly 10 MB. REA buffered Kuna's whole string JSON and refused anything above 16 MiB, but Kuna writes about 290 bytes per row. REA now parses the scan as Kuna writes it, so these apps get answers. Refs #1957.Problem and expected behavior
Trigger:
rea search <app> ghostty --provider kunaon a thin arm64 app binary. Onorigin/mainwith real Kuna 1.833 (macOS arm64):kuna strings --jsonoutputXULThe 16 MiB cap came from no format or resource constraint, and the original validation only covered SQLite and zlib. Expected: a string question about a large target gets an answer, with memory bounded by what REA keeps rather than by the size of the transport.
Change and scope
runOwnedCommand(src/process/OwnedCommand.ts) accepts an optional stdout sink.KunaJsonDocumentReader(src/kuna/KunaJsonStream.ts) owns the token mechanics.jsonParser(already a dependency; its shape is declared at the import site because stream-json's typings omit it).kunaAddressFromJsonNumber, whichparseKunaJsonnow uses too;JsonInputFile, instead of a fixed scan size. Kept text is copied out of the decoded chunk it was sliced from.KunaStringInventoryReader(src/kuna/KunaStringInventory.ts) keeps only the string-scan semantics: the same schema and fact checks per row as before. It replacesparseKunaStringInventory.KUNA_DIAGNOSTIC_BYTESis now shared with the project export, andKUNA_STRING_OUTPUT_BYTESis removed.Contribution scope: a fix inside the existing Kuna provider. No new authority, target execution, injection, mutation or protection recovery. A maintainer scope decision is not applicable. There are no new dependencies and no new MCP tool names or input fields.
Contract and boundary impact
KunaStringInventoryReaderowns parsing and validation of the string scan;runOwnedCommandgains an optional stdout sink.resource_constraint.raw_result.stringskeeps its digest, size, scope, policy and counts. The SHA-256 now covers the streamed bytes, which are the same bytes as before.runOwnedCommand.Expected effects:
Evidence and regression coverage
kunaStrings.test.ts: a 24 MiB scan answers. It fails on base with the 16 MiBresource_constraint.kunaStrings.test.ts: a contradictory first row followed by a hang returnsAnalysisOutputErrorwithin about 1 s. On base it waits for the 60 s project deadline.kunaStrings.test.ts: the oversize-retention case now floods stderr pastKUNA_DIAGNOSTIC_BYTES, keeping coverage that deterministic buffer failures are retained until close.kunaStringInventory.test.tsnow drives the reader through 1–3 byte chunks (multi-byte UTF-8 across chunk boundaries, digest equal to the whole text). It also covers:__proto__members;getHeapStatistics), both for a retained row and for a kept string while it is still unterminated;The existing contradiction cases are kept.
ownedCommand.test.ts:final;finalwhen the deadline elapses is destroyed.rea search t/ghostty ghostty --provider kuna:ok,inventory_count: 107587,selected_count: 383,raw_result.strings.bytes: 30952152.For evidence-bearing changes:
Validation performed
npm run test:focused -- tests/boundary/providers/kuna/*.test.ts tests/boundary/process/ownedCommand.test.ts: 22 files, 231 tests passed.npm run check:fast(typecheck and lint): passed.npm run knip,npm run docs:check,npm run jscpd: passed.npx tsc --noEmit -p tsconfig.reverse-tools-mutation.json: passes. This is the partial project the Reverse tools quality lane checks; it failed on the earlier.d.tsaugmentation and is fixed in0ce0dbd41.npm run format:check: clean for the changed files. It reports one existing issue insrc/adb/AdbProvider.tson main, which this PR does not touch.REA_KUNA_COMMAND=<kuna 1.833 macos-arm64> npm run verify:kunaon macOS 26.7.1 arm64: passed, MCP workflow including strings,cleanup: "complete". Strings:inventory_count4, cold scan 501 ms, warm search 2 ms.Performance
Same CLI question,
rea search <target> ghostty --provider kuna, run as a cold CLI process each time. Real Kuna 1.833 on macOS arm64, three samples each. Peak RSS is for the whole process family (REA plus Kuna), sampled every 50 ms. CPU is family user+sys time.Final code, measured with a load average of about 4 (base and head runs interleaved):
origin/main)resource_constraint, 1.11–1.17 s, 501–509 MiBresource_constraint, 1.30–1.78 s, 552–596 MiBresource_constraint, 2.56–2.69 s, 1447–1507 MiBJSON.parseof the same file with the cap lifted takes 2.5 s at a 1.65 GiB peak.--token-count), same question: base returns a 194-token error for all three apps. This PR returns 691 (XUL), 755 (Raycast) and 53,644 (Ghostty, 383 matching rows). The search result contract is unchanged and has no paging.Compatibility, safety, and release
runOwnedCommandownership and cleanup.__proto__and other unknown members are skipped, not assigned.Author readiness
Human owner: @Kaoru0822
Before merge
type(scope): outcome.🤖 Generated with Claude Code
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.