morluto/rea

Fork

You've already forked rea

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

Code Issues 1.3k Projects Packages Wiki Activity

[PR #7125] fix(reference): bound historical source imports #7660

New issue

Open

opened 2026-11-03 13:39:27 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-11-03 13:39:27 +00:00

Owner

Copy link

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/7125
Author: @abdulwasea89
Created: 10/25/2026
Status: 🔄 Open

Base: main ← Head: fix/reference-source-import-budget

📝 Commits (3)

2573a9f fix(reference): bound historical source imports
455026e fix(reference): keep partial import errors JSON-safe
3cb7942 fix(reference): surface oversized Git metadata

📊 Changes

11 files changed (+332 additions, -13 deletions)

View changed files

📝 docs/cli.md (+5 -0)
📝 src/application/ReferenceSourceImportPolicy.ts (+27 -4)
📝 src/application/ReferenceSourceImportTypes.test.ts (+11 -2)
📝 src/application/ReferenceSourceImportTypes.ts (+31 -2)
📝 src/application/ReferenceSourceVcsAdapter.ts (+17 -2)
➕ src/reference/ReferenceSourceLimits.ts (+12 -0)
📝 src/reference/ReferenceSourceReaderEntries.ts (+8 -0)
📝 src/reference/ReferenceSourceReaderTypes.ts (+2 -1)
📝 tests/boundary/cli/referenceSourceImportStatus.test.ts (+133 -1)
📝 tests/boundary/filesystem/referenceSourceImport.test.ts (+58 -0)
📝 tests/boundary/filesystem/referenceSourceMetadataCleanup.test.ts (+28 -1)

📄 Description

Summary

Bound historical source imports so an oversized local tree is rejected before unbounded source or metadata reads.

Problem and expected behavior

Source-derived finding: the importer retained each file's bytes through tree traversal without a total byte cap. A large sparse regular file could drive excessive memory use. The regression uses sparse files to verify early rejection; an out-of-memory run was not performed.

Change and scope

Limit retained source bytes to 64 MiB across one import.
Limit each local .gitignore and Git metadata read to 1 MiB.
Return resource_constraint with the observed partial inventory, and document the limits.
This is a security fix within existing analysis boundaries; no new provider or target execution capability is added.

Contract and boundary impact

Semantic owner and earliest changed stage: reference source reader and local metadata admission.
Contribution scope fit and linked maintainer decision: ordinary security fix; advance scope decision not applicable.
CLI and MCP/tool-catalog contract: CLI import reports the existing canonical resource_constraint category; no MCP tool catalog change.
Provider, bridge, target-format, or platform compatibility: no provider or bridge change; oversized inputs now fail explicitly.
Evidence, artifact, provenance, or reconstruction contract: partial observed inventory is retained on source byte limit.
Target selection, authority, upstream behavior, and cleanup: caller-selected local directory; no new authority or side effects.
Generated metadata, package, or installation impact: none.

Expected effects:

Read existing files or captured data.
Launch or attach to a process.
Inject instrumentation or run scripts/code.
Change process memory, registers, files, or configuration.
Contact network services.
Recover protected code or work around protections.

Evidence and regression coverage

Added filesystem regressions for a sparse oversized file, aggregate bytes, and oversized .gitignore.
Added compiled CLI regressions for resource_constraint, exit code 1, and aggregate-limit failure after a file was read; error diagnostics retain path and size but omit file bytes so JSON output stays bounded.
The original unlimited read is established from source inspection; no OOM reproduction or before/after memory benchmark was run.
Windows-specific CLI cases were skipped on this Linux host. Large counts of tiny files are outside this byte-limit fix.

Validation performed

npm run test:focused -- tests/boundary/filesystem/referenceSourceMetadataCleanup.test.ts tests/boundary/cli/referenceSourceImportStatus.test.ts — 15 passed, 2 Windows skips after the Git metadata fix.
npm run check:fast — passed after the Git metadata fix.
npm run docs:check — passed with pinned Node 24.22.1 outside the sandbox.
npm run check:pr — stopped after 19 minutes. Its full parallel test run reported 3 failures in modelArtifactCli.test.ts and javascriptWorkerUrlCli.test.ts; rerunning those two files alone passed all 25 tests. The full gate did not complete.

Review response

Greptile identified that aggregate-limit errors included previously read file bytes in JSON diagnostics, creating an oversized response. The error projection now includes only each read file's path, kind, status, and size; the compiled CLI regression verifies that an aggregate-limit error after a prior read produces small, valid JSON.
Greptile identified that oversized Git metadata became a successful unknown VCS result. The adapter now returns the typed limit failure; filesystem and compiled CLI regressions verify resource_constraint is reported.

Compatibility, safety, and release

Previously accepted source trees over the new limits now fail with recovery guidance; users can exclude large files and retry.
No real provider run is applicable. The full check:pr gate and memory benchmark were not run.
No issue is linked; this draft follows the source-derived finding and fix in the current investigation.

Author readiness

Human owner: @abdulwasea89

I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
I declare this PR ready for initial maintainer review.

Before merge

Human owner has completed readiness declarations.
Required checks and maintainer review have completed.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/7125 **Author:** [@abdulwasea89](https://github.com/abdulwasea89) **Created:** 10/25/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `fix/reference-source-import-budget` --- ### 📝 Commits (3) - [`2573a9f`](https://github.com/morluto/rea/commit/2573a9fc9994f39d27b9a1fc68ab6e18d0a65729) fix(reference): bound historical source imports - [`455026e`](https://github.com/morluto/rea/commit/455026e01fe428042a9c9d21393c7cba0a2baf42) fix(reference): keep partial import errors JSON-safe - [`3cb7942`](https://github.com/morluto/rea/commit/3cb7942acec9c1ae4a46e71487ceda80a1df3def) fix(reference): surface oversized Git metadata ### 📊 Changes **11 files changed** (+332 additions, -13 deletions) <details> <summary>View changed files</summary> 📝 `docs/cli.md` (+5 -0) 📝 `src/application/ReferenceSourceImportPolicy.ts` (+27 -4) 📝 `src/application/ReferenceSourceImportTypes.test.ts` (+11 -2) 📝 `src/application/ReferenceSourceImportTypes.ts` (+31 -2) 📝 `src/application/ReferenceSourceVcsAdapter.ts` (+17 -2) ➕ `src/reference/ReferenceSourceLimits.ts` (+12 -0) 📝 `src/reference/ReferenceSourceReaderEntries.ts` (+8 -0) 📝 `src/reference/ReferenceSourceReaderTypes.ts` (+2 -1) 📝 `tests/boundary/cli/referenceSourceImportStatus.test.ts` (+133 -1) 📝 `tests/boundary/filesystem/referenceSourceImport.test.ts` (+58 -0) 📝 `tests/boundary/filesystem/referenceSourceMetadataCleanup.test.ts` (+28 -1) </details> ### 📄 Description ## Summary Bound historical source imports so an oversized local tree is rejected before unbounded source or metadata reads. ## Problem and expected behavior Source-derived finding: the importer retained each file's bytes through tree traversal without a total byte cap. A large sparse regular file could drive excessive memory use. The regression uses sparse files to verify early rejection; an out-of-memory run was not performed. ## Change and scope - Limit retained source bytes to 64 MiB across one import. - Limit each local `.gitignore` and Git metadata read to 1 MiB. - Return `resource_constraint` with the observed partial inventory, and document the limits. - This is a security fix within existing analysis boundaries; no new provider or target execution capability is added. ## Contract and boundary impact - Semantic owner and earliest changed stage: reference source reader and local metadata admission. - Contribution scope fit and linked maintainer decision: ordinary security fix; advance scope decision not applicable. - CLI and MCP/tool-catalog contract: CLI import reports the existing canonical `resource_constraint` category; no MCP tool catalog change. - Provider, bridge, target-format, or platform compatibility: no provider or bridge change; oversized inputs now fail explicitly. - Evidence, artifact, provenance, or reconstruction contract: partial observed inventory is retained on source byte limit. - Target selection, authority, upstream behavior, and cleanup: caller-selected local directory; no new authority or side effects. - Generated metadata, package, or installation impact: none. Expected effects: - [x] Read existing files or captured data. - [x] Launch or attach to a process. - [x] Inject instrumentation or run scripts/code. - [x] Change process memory, registers, files, or configuration. - [x] Contact network services. - [x] Recover protected code or work around protections. ## Evidence and regression coverage - Added filesystem regressions for a sparse oversized file, aggregate bytes, and oversized `.gitignore`. - Added compiled CLI regressions for `resource_constraint`, exit code 1, and aggregate-limit failure after a file was read; error diagnostics retain path and size but omit file bytes so JSON output stays bounded. - The original unlimited read is established from source inspection; no OOM reproduction or before/after memory benchmark was run. - Windows-specific CLI cases were skipped on this Linux host. Large counts of tiny files are outside this byte-limit fix. ## Validation performed - `npm run test:focused -- tests/boundary/filesystem/referenceSourceMetadataCleanup.test.ts tests/boundary/cli/referenceSourceImportStatus.test.ts` — 15 passed, 2 Windows skips after the Git metadata fix. - `npm run check:fast` — passed after the Git metadata fix. - `npm run docs:check` — passed with pinned Node 24.22.1 outside the sandbox. - `npm run check:pr` — stopped after 19 minutes. Its full parallel test run reported 3 failures in `modelArtifactCli.test.ts` and `javascriptWorkerUrlCli.test.ts`; rerunning those two files alone passed all 25 tests. The full gate did not complete. ## Review response - Greptile identified that aggregate-limit errors included previously read file bytes in JSON diagnostics, creating an oversized response. The error projection now includes only each read file's path, kind, status, and size; the compiled CLI regression verifies that an aggregate-limit error after a prior read produces small, valid JSON. - Greptile identified that oversized Git metadata became a successful `unknown` VCS result. The adapter now returns the typed `limit` failure; filesystem and compiled CLI regressions verify `resource_constraint` is reported. ## Compatibility, safety, and release - Previously accepted source trees over the new limits now fail with recovery guidance; users can exclude large files and retry. - No real provider run is applicable. The full `check:pr` gate and memory benchmark were not run. - No issue is linked; this draft follows the source-derived finding and fix in the current investigation. ## Author readiness Human owner: @abdulwasea89 - [x] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [x] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [x] I declare this PR ready for initial maintainer review. ## Before merge - [x] Human owner has completed readiness declarations. - [x] Required checks and maintainer review have completed. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi added the

pull-request

label

2026-11-03 13:39:27 +00:00

No Branch/Tag specified

main

codex/procedure-reference-stacked

release-please--branches--main--components--rea-agents

release-please--branches--main--components--rea-agents--release-notes

codex/kuna-independent-retry

codex/byte-search-overlay-coverage

codex/open-failure-target-receipt

codex/kuna-xref-evidence

codex/ghidra-ambiguous-procedure-recovery

codex/function-source-bundle

codex/trace-function-integration-prototype

codex/snapshot-source-closure

codex/recorded-evidence-array-pages

codex/evidence-recovery-redesign

codex/trace-feature-address-scope

codex/call-discovery-scope

codex/ghidra-literal-candidate-evidence

codex/decompile-directory-independent-retry

codex/retry-provenance-base

codex/value-flow-capability-experiment

codex/ghidra-containing-instruction

cursor/readme-discord-button-ef9f

dependabot/github_actions/actions/setup-node-7.1.0

dependabot/github_actions/actions/upload-artifact-7.0.2

dependabot/npm_and_yarn/npm-minor-and-patch-d8d97ce6df

codex/kuna-1989

fix/8559-native-ui-fixture-ownership

fix/8555-go-verifier-mcp-ownership

codex/provider-capability-recovery

fix/8642-rizin-target-stdin

codex/ghidra-memory-effect-scope

codex/ghidra-annotation-entry-selector

codex/ghidra-readonly-recovery

codex/ghidra-regex-verifier-recovery

codex/evidence-discovery-view-guidance

fix/8545-bulk-append-overflow

fix/8428-navigation-refusal-alternatives

codex/address-context-recovery

codex/matching-member-limitations

codex/unknown-scope-contract

codex/evidence-view-guidance

feat/8373-native-trace-views

codex/ghidra-snapshot-verifier-binding

codex/elf-export-visibility

codex/evidence-view-recovery

codex/native-reference-cli-help

codex/provider-candidate-observation-scope

codex/kuna-session-verification

codex/ghidra-switch-lifetime-guidance

codex/pseudocode-paging-discovery

codex/clarify-evidence-view-support

codex/kuna-xrefs-selector-recovery

codex/context-capacity-recovery

codex/evidence-closure-large-links

codex/same-function-evidence-identity

codex/ghidra-verifier-inspected-jdk

codex/audit-guided-workflows

codex/crash-prompt-retained-dossiers

codex/analysis-view-error-recovery

codex/kuna-verifier-target-selection

codex/ghidra-selected-project-call-coverage

codex/docs-native-analysis-scope

codex/ghidra-java-socket-boundary

codex/ghidra-retained-snapshot-verifier

codex/ghidra-search-inventory-facts

fix/8284-zero-argument-parameter-bindings

fix/8198-ui-action-handler-offset

fix/8257-elf-entry-label-identity

fix/8304-deferred-output-schema-projection

fix/8277-call-path-computed-target-uncertainty

fix/pr-8089-conflicts

fix/ci-static-duplicate-export

dependabot/npm_and_yarn/lint-staged-17.6.0

dependabot/github_actions/actions/download-artifact-8.0.2

dependabot/npm_and_yarn/write-file-atomic-8.0.0

dependabot/npm_and_yarn/canonicalize-5.1.0

dependabot/github_actions/actions/cache/restore-6.1.0

dependabot/npm_and_yarn/toon-format/toon-4.4.0

dependabot/github_actions/actions/deploy-pages-5.0.1

codex/review-fixes-pr-2403

codex/review-fixes-pr-2400

codex/process-terminal-width

codex/test-brittleness-cleanup

verify/7167-browser-connect-macos

fix/7167-browser-connect

feat/native-matching-allowance-setting

feat/browser-scenario-dry-run

test/managed-caller-parameters-20261024

codex/parser-fortythird-boundaries

fix/6926-pinvoke-platform-decorations-20261023

fix/6889-har-body-encoding-20261023

fix/6888-sqlite-schema-budget-20261023

fix/dependency-security-gates-6708

cursor/evidence-overwrite-rebase-b4ac

feat/4057-verify-native-exports

fix/windows-acl-observer-overhead-20261022

cursor/erlang-beam-restack-4476

cursor/pe-certificate-review-bf32

fix/6116-javascript-application-fs-errors

fix/6061-unmapped-runtime-script-unknown

feat/browser-scenario-summary

fix/pe-certificates

codex/verify-3407

cursor/rebase-5699

cursor/rebase-5352

cursor/rebase-5600

cursor/fix-cli-schema-omission-10a8

codex/audit-hopper-shutdown-reasons-20261021

fix/demangle-swift-omitted-record

fix/nib-boolean-value-types

cursor/cancellation-partial-observation-a5ac

fix/ib-object-limit-reasons

fix/electron-fuse-limit-advice

fix/kuna-doctor-dangling-cache

fix/fixed-budget-refusal-advice

fix/js-ipc-listener-dedup

fix/js-return-member-provenance

fix/dmg-integrity-retry-advice

fix/ida-doctor-readiness

fix/kuna-version-probe-output-drain

fix/legacy-swift-runtime-name-matching

fix/dylib-lazy-load-commands

fix/browser-scenario-startup-cancel

fix/swift-nested-punycode-classification

fix/compare-unselected-scopes

fix/objc-method-budget-coverage

fix/dmg-mounted-root-observations

audit/ci-tests-20261019-round2

release/6.7.0

fix/wabt-cleanup-retry-resolution

fix/electron-fuse-elf-pe-architecture

fix/native-plist-descriptor-ownership

fix/kuna-caller-cleanup-ownership

experiment/atomic-codemode-20261016

review/first-contributor-3776

fix/call-resolution-full-read-cycles

fix/member-mutation-selection-repeat

experiment/atomic-re-tools-20261015

perf/member-mutation-lazy-origins

perf/static-network-mutation-index

fix/hopper-startup-lifecycle

audit/3050-windows-proof-20261015

feat/windows-window-capture-2183

audit/3049-windows-proof-20261015

audit/3735-macos-fixtures-20261015

fix/batch-decompile-null-pseudocode

fix/kuna-inventory-honors-jobs

experiment/browser-codemode-tool-design-20261015

fix/hopper-clean-runtime-argv

codex/review-2049-27f1b367

review-fixes/pr-2787-routing

fix/3360-scoped-this-emitters

fix/3367-replaced-global-effects

fix/3838-compact-any-value-items

test/hopper-download-probe-20261015

codex/windows-runtime-creation-red-green-20261015

codex/windows-readonly-root-red-green-20261015

codex/windows-root-share-red-green-20261015

codex/exp-1685-win-mcp-startup

codex/bug-sweep-20261015

investigate/2190-windows-package-phases

investigate/3543-macos-cli-deadline

perf/3281-reverse-tools-feedback-quality

fix/3406-hopper-doctor-fixture

fix/3341-wrapper-require-test-context

fix/3301-owned-uninstall-fixtures

fix/3164-compiler-input-ownership

fix/pcap-cleanup-stdout-newline-20261014

perf/interface-builder-summary

codex/review-fixes-pr-1977

fix/ghidra-effective-jvm-platform

fix/docs-turkish-rust-link-20261013

fix/main-maintenance-ci-20261010

fix/recover-binary-session-process-effects

codex/review-fixes-pr-2457

codex/ci-required-cancellation-gate

fix/main-format-baseline-20261013

fix/setup-preserve-entry-fields

cursor/restore-main-checks-1ffc

fix/ghidra-bridge-hashmap-import

codex/shared-ci-jadx-json-default-20261013

codex/shared-ci-ghidra-test-groups-20261013

codex/shared-ci-java-fixture-format-20261013

codex/fix-intrinsic-invocation-boundaries

test/release-preflight-reusable-jobs

test/js-parameter-provenance-expectation

fix/linux-hopper-verification-current

test/2419-ghidra-elf-probe-size

followup/pr-1544-native-ida

codex/fold-pr-2230-contribution

docs/readme-responsible-use

fix/issue-2064

fix/issue-2050

fix/issue-2053

fix/issue-1968

fix/issue-1945

experiment/codemode-bridge-20261011

fix/tool-effects-negative-wording-1904-20261012

fix/1842-electron-interop-provenance

feat/recover-javascript-multi-input

refactor/website-i18n-prep

fix/nativeaot-pe-virtual-extent

fix/browser-verifier-ci-hang

fix/pe-resources-input-selection

docs/beginner-friendly-guides

docs/beginner-setup-guide

fix/javascript-analysis-worker-1563

fix/1649-ghidra-duplicate-names

fix/javascript-export-keys-integration-1369

test/maintenance-boundary-contracts-20261011

review/javascript-export-keys-1369

review/process-live-progress-1483

test/javascript-1495-growth-ratio

fix/ci-escape-timing

docs/normalized-terminal-geometry-1450

release/6.3.0

archive/release-6.3.0-pr1381

release/6.2.0

fix/electron-preload-dynamic-override

fix/javascript-self-referential-bundler-edges

release/6.1.0

cursor/grok-client-cherry-pick-0b56

feat/signature-posture

feat/dyld-shared-cache

feat/artifact-integrity-policy

feat/pkg-inventory

release/6.0.0

chore/sync-release-5.0.0

release/5.0.0

cursor/cloud-agent-node-787e

fix/javascript-xhr-open-endpoints

review/pr-711-realpath-fallback-wording

cleanup/replay-confinement

codex/test-suite-performance

codex/issue-foundations

codex/address-open-issues-2026-07-20

agent/javascript-vite-rollup-esbuild-adapters

agent/process-cleanup

agent/triage-integration

agent/unknown-commands-registry

feat/mcp-identity-metadata

morlvps/issue_cleanup

codex/installer-redesign

morl/roadmap-investigation-core

morl/roadmap-investigation-docs

release-please--branches--main--components--rea

rea-agents-6.8.0

rea-agents-6.7.0

rea-agents-6.6.0

experiment/atomic-codemode-wait-runtime-20261016

rea-agents-6.5.0

rea-agents-6.4.0

rea-agents-6.3.0

rea-agents-6.2.0

rea-agents-6.1.0

rea-agents-6.0.0

rea-agents-5.0.0

rea-agents-4.1.0

rea-agents-4.0.1

rea-agents-4.0.0

rea-agents-3.2.1

rea-agents-3.2.0

rea-agents-3.1.0

rea-agents-3.0.0

rea-agents-2.7.0

rea-agents-2.6.0

rea-agents-2.5.1

rea-agents-2.5.0

rea-agents-2.4.0

rea-agents-2.3.0

rea-agents-2.2.0

rea-agents-2.1.0

rea-agents-2.0.0

rea-agents-1.7.0

rea-agents-1.6.0

rea-agents-1.5.0

rea-agents-1.4.0

rea-agents-1.3.0

rea-agents-1.2.0

rea-agents-1.1.0

rea-agents-1.0.0

rea-agents-0.5.0

rea-agents-0.4.0

rea-agents-0.3.0

rea-0.2.1

rea-0.2.0

Labels

Clear labels

Mirrored from GitHub Pull Request

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#7660

Reference in a new issue

Repository

morluto/rea

Title

Body

No description provided.

Delete branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?

Rows
Columns