[PR #6301] [MERGED] feat(process): make the regex worker heap request configurable #7266

Closed
opened 2026-11-01 21:46:36 +00:00 by bsakaguchi · 0 comments
Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/6301
Author: @morluto
Created: 10/22/2026
Status: ✅ Merged
Merged: 10/22/2026
Merged by: @morluto

Base: main ← Head: cursor/regex-worker-heap-request-d806


📝 Commits (3)

  • cffa07d feat(process): make the regex worker heap request configurable
  • 6ee3677 fix(process): set the regex worker heap floor to the measured startup bound
  • 8b9ad21 fix(process): raise the regex worker heap floor to 7 MiB

📊 Changes

24 files changed (+564 additions, -45 deletions)

View changed files

📝 docs/kuna-provider.md (+1 -1)
📝 docs/native-investigation.md (+11 -5)
📝 src/application/Doctor.test.ts (+1 -0)
📝 src/application/SetupRegistrationEnvironment.ts (+1 -0)
📝 src/composition/binary.ts (+1 -0)
📝 src/config/environment.ts (+2 -0)
📝 src/config/parseConfig.ts (+8 -0)
➕ src/config/regexWorkerHeapRequest.test.ts (+159 -0)
➕ src/config/regexWorkerHeapRequest.ts (+106 -0)
📝 src/config/types.ts (+3 -0)
📝 src/contracts/officialToolContracts.ts (+1 -1)
📝 src/hopper/HopperProvider.ts (+5 -1)
📝 src/hopper/HopperProviderCapabilities.ts (+6 -1)
📝 src/hopper/HopperRegexSearch.ts (+6 -0)
📝 src/kuna/KunaProvider.ts (+15 -0)
📝 src/kuna/KunaProviderCapabilities.ts (+9 -3)
📝 src/kuna/KunaQueryWork.ts (+6 -0)
📝 src/kuna/KunaSessionClient.ts (+11 -2)
📝 src/process/RegexMatcher.test.ts (+62 -1)
📝 src/process/RegexMatcher.ts (+69 -29)

...and 4 more files

📄 Description

Summary

Part of #6176. The regex worker old-generation request is now REA_REGEX_WORKER_HEAP_REQUEST_MIB. Unset still requests 128 MiB. This does not cover the issue's decoder heap or MCP startup budgets.

Problem and expected behavior

The regex worker requested a fixed 128 MiB old generation, and capability and recovery text said that request had no runtime override. Callers need a named setting whose default is that constant. Invalid values must fail configuration and name the setting. A positive Node --max-old-space-size, including NODE_OPTIONS, still overrides the request and is not clamped by the setting.

Unset behavior stays a 128 MiB request. A value from 7 to 16384 MiB is the request. Any other value fails startup, doctor, and client-registration configuration and names REA_REGEX_WORKER_HEAP_REQUEST_MIB. It does not fall back to 128. Setup copies a caller-selected value through SELECTED_REGISTRATION_SETTINGS.

Recovery, capability, and limitation text name the setting, the effective request, the source (default or setting), the default, the floor, the maximum, and how to change it. They also name the process effective heap when launch flags establish it.

Change and scope

src/config/regexWorkerHeapRequest.ts owns the setting. Startup (parseConfig) and doctor (parseEnvironment) resolve it with the same schema. The effective request is stored on AppConfig and passed into Hopper and Kuna, which pass it to RegexMatcher. A direct KunaProvider resolves the selected environment instead of ambient process.env. An omitted matcher request means the 128 MiB default, not ambient process.env.

Setup keeps REA_REGEX_WORKER_HEAP_REQUEST_MIB in SELECTED_REGISTRATION_SETTINGS. A present value is copied into the generated registration. An unset value is omitted.

The floor is 7 MiB. On Linux, the production worker (^a against ["a"], no NODE_OPTIONS) returns ERR_WORKER_OUT_OF_MEMORY at 6 MiB on Node 24.21.0 and 24.22.1, and on most runs of Node 24.11.0, 24.18.0, and 26.0.0. 7 MiB returned that match on Node 22.19.0, 22.23.4, 24.11.0, 24.18.0, 24.21.0, 24.22.1, 26.0.0, and 26.12.1, so a smaller request is not a matching bound on every supported Node line. The maximum is 16384 MiB, the same ceiling REA already uses for one isolated JavaScript analysis worker. Node starts with a larger flag; this maximum bounds only the REA request. A positive --max-old-space-size is reported as the effective heap and is not limited by this floor or maximum.

Follow the contribution scope. This is configuration of an existing regex worker request. It does not add a provider or a capability that executes, injects, mutates, or recovers a target. Maintainer scope decision: not applicable. The product decision to make this ceiling a setting is recorded in #6176.

Contract and boundary impact

  • Semantic owner and earliest changed stage: src/config/regexWorkerHeapRequest.ts parses the setting; parseConfig resolves it before providers match. explicitSetupRegistrationEnvironment preserves a caller-selected value when setup writes a registration.
  • Contribution scope fit and linked maintainer decision: existing regex matching only; not applicable.
  • CLI and MCP/tool-catalog contract: search_procedures and search_strings descriptions name REA_REGEX_WORKER_HEAP_REQUEST_MIB, the 128 MiB default, the 7 MiB floor, and the 16384 MiB maximum. Capability and heap failures name the effective request and source.
  • Provider, bridge, target-format, or platform compatibility: Hopper and Kuna share the request. A positive Node --max-old-space-size, including NODE_OPTIONS, still overrides it. Percentage, --max-heap-size, env-file, and experimental config-file launches still leave the effective heap unestablished.
  • Evidence, artifact, provenance, or reconstruction contract: heap failures add requested_worker_heap_source beside requested_worker_heap_mib.
  • Target selection, authority, upstream behavior, and cleanup details (if applicable): none.
  • Generated metadata (docs/public/product-catalog.json), package, or installation impact (CI generates and validates ignored catalogs and skills/; edit authored instructions in .agents/skills/reverse-engineer-anything/ and never commit build outputs): tool descriptions changed in source contracts. No authored skill text claimed the heap request had no override.

Expected effects (check known effects; use documentation-only on its own).
Unchecked items do not establish absence; leave uncertainties visible.

  • Read existing files or captured data.
  • Launch or attach to a process.
  • Inject instrumentation or run scripts/code.
  • Change process memory, registers, files, or configuration.
  • Contact network services.
  • Recover protected code or work around protections.
  • Documentation or usability only; no target interaction.
  • Not sure yet; investigation or maintainer guidance needed.

Evidence and regression coverage

  • Tests added or updated: parser accepts 7 and rejects 6; doctor and client-registration rejection of 6; Hopper/Kuna capability text for a 256 MiB request while ambient process.env is 8; a selected request of 256 stays in effect, and --max-old-space-size=64 still overrides it to 64 MiB without clamping. A clean Node process, with NODE_OPTIONS removed and without --import tsx on execArgv, calls production matchRegexValues at the floor and matches ^a against ["a"].
  • Base reproduction or other evidence: on main the request constant is 128 and recovery says there is no runtime override for that request. The 6 MiB floor was only a Node 22.23.4 startup bound. Reproducing the review on Linux Node 24.21.0 with the production matcher returned ERR_WORKER_OUT_OF_MEMORY at 6 MiB (5/5) and {ok:true, indices:[0]} at 7 MiB (5/5). The same 7 MiB production call matched on Node 22.19.0, 22.23.4, 24.11.0, 24.18.0, 26.0.0, and 26.12.1 (3/3 each). Raw worker repeats at 6 MiB were 0/10 matches on 24.21.0 and 24.22.1, 1/10 on 24.11.0 and 24.18.0, and 2/10 on 26.0.0; 7 MiB was 10/10 on those builds.
  • User-visible CLI/MCP output (if applicable): the default recovery says The effective REA_REGEX_WORKER_HEAP_REQUEST_MIB value is 128 MiB (source default; default 128 MiB, floor 7 MiB, maximum 16384 MiB). Set REA_REGEX_WORKER_HEAP_REQUEST_MIB in the environment that starts REA to a decimal integer from 7 to 16384, then restart REA. A 256 MiB setting uses source setting and 256 MiB.
  • Remaining proof gaps: no real Hopper or Kuna binary was used. Windows was not executed. Node 22 and Node 26.12.1 can complete the trivial match at 6 MiB, but the setting rejects 6 so one accepted value works on every supported Node line. A heap above 128 was checked through resolveRegexWorkerHeap and capability text, not by exhausting a 256 MiB worker.

For evidence-bearing changes:

  • Observed, derived, and inferred claims remain distinguishable.
  • Artifact identity, source provenance, and failed attempts remain preserved.
  • Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

  • npm run check:fast — typecheck and lint, 2 tasks successful (6.695s).
  • npm run format:check — all matched files use the correct format (4944 files).
  • npm run test:local -- src/config/regexWorkerHeapRequest.test.ts src/process/RegexMatcher.test.ts src/application/Doctor.test.ts — 124 passed.
  • npm run test:focused -- tests/boundary/process/regexMatcher.test.ts tests/boundary/filesystem/clientConfigurationEnvironment.test.ts — 43 passed, 10 skipped. The real-worker floor test is in the process-boundary file.

Compatibility, safety, and release

  • Breaking changes or migration steps: none for an unset environment. The request remains 128 MiB. REA_REGEX_WORKER_HEAP_REQUEST_MIB=6 is now a configuration error. It previously passed configuration and then failed regex search on supported Node 24 and Node 26.0.0. A positive --max-old-space-size still overrides the request.
  • Real Hopper/Ghidra, browser, or OS coverage: not run. Heap-request behavior was measured with the production regex worker on the Linux Node versions listed above.
  • Package or release metadata impact: none.
  • Security, privacy, process, or containment review: the worker still uses resourceLimits.maxOldGenerationSizeMb. The setting changes only that REA request. It does not raise a Node --max-old-space-size already applied to the process, and it does not change target memory.

Author readiness

Human owner: @morluto

  • I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
  • I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
  • I declare this PR ready for initial maintainer review.

Before merge

  • This PR addresses a concrete problem or an agreed enhancement.
  • Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
  • The PR has one focused outcome and the title follows type(scope): outcome.
  • Related issue is linked, or the reason for not linking one is stated above.
  • Tests cover changed observable behavior and meaningful failure paths.
  • Owning docs, contracts, and generated metadata are updated where needed.
  • User-visible CLI/MCP changes include representative output.
  • I checked the final diff for secrets, unrelated cleanup, and unsupported claims.
Open in Web Open in Cursor 

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/6301 **Author:** [@morluto](https://github.com/morluto) **Created:** 10/22/2026 **Status:** ✅ Merged **Merged:** 10/22/2026 **Merged by:** [@morluto](https://github.com/morluto) **Base:** `main` ← **Head:** `cursor/regex-worker-heap-request-d806` --- ### 📝 Commits (3) - [`cffa07d`](https://github.com/morluto/rea/commit/cffa07df86680aefb4d6313aeb08667f6b6a0ca6) feat(process): make the regex worker heap request configurable - [`6ee3677`](https://github.com/morluto/rea/commit/6ee36777bd6417e813c69b5af92ba2b8af35f5ed) fix(process): set the regex worker heap floor to the measured startup bound - [`8b9ad21`](https://github.com/morluto/rea/commit/8b9ad2161488beed0e3670a1017882b309de80a8) fix(process): raise the regex worker heap floor to 7 MiB ### 📊 Changes **24 files changed** (+564 additions, -45 deletions) <details> <summary>View changed files</summary> 📝 `docs/kuna-provider.md` (+1 -1) 📝 `docs/native-investigation.md` (+11 -5) 📝 `src/application/Doctor.test.ts` (+1 -0) 📝 `src/application/SetupRegistrationEnvironment.ts` (+1 -0) 📝 `src/composition/binary.ts` (+1 -0) 📝 `src/config/environment.ts` (+2 -0) 📝 `src/config/parseConfig.ts` (+8 -0) ➕ `src/config/regexWorkerHeapRequest.test.ts` (+159 -0) ➕ `src/config/regexWorkerHeapRequest.ts` (+106 -0) 📝 `src/config/types.ts` (+3 -0) 📝 `src/contracts/officialToolContracts.ts` (+1 -1) 📝 `src/hopper/HopperProvider.ts` (+5 -1) 📝 `src/hopper/HopperProviderCapabilities.ts` (+6 -1) 📝 `src/hopper/HopperRegexSearch.ts` (+6 -0) 📝 `src/kuna/KunaProvider.ts` (+15 -0) 📝 `src/kuna/KunaProviderCapabilities.ts` (+9 -3) 📝 `src/kuna/KunaQueryWork.ts` (+6 -0) 📝 `src/kuna/KunaSessionClient.ts` (+11 -2) 📝 `src/process/RegexMatcher.test.ts` (+62 -1) 📝 `src/process/RegexMatcher.ts` (+69 -29) _...and 4 more files_ </details> ### 📄 Description <!-- CURSOR_AGENT_PR_BODY_BEGIN --> ## Summary Part of #6176. The regex worker old-generation request is now `REA_REGEX_WORKER_HEAP_REQUEST_MIB`. Unset still requests 128 MiB. This does not cover the issue's decoder heap or MCP startup budgets. ## Problem and expected behavior The regex worker requested a fixed 128 MiB old generation, and capability and recovery text said that request had no runtime override. Callers need a named setting whose default is that constant. Invalid values must fail configuration and name the setting. A positive Node `--max-old-space-size`, including `NODE_OPTIONS`, still overrides the request and is not clamped by the setting. Unset behavior stays a 128 MiB request. A value from 7 to 16384 MiB is the request. Any other value fails startup, doctor, and client-registration configuration and names `REA_REGEX_WORKER_HEAP_REQUEST_MIB`. It does not fall back to 128. Setup copies a caller-selected value through `SELECTED_REGISTRATION_SETTINGS`. Recovery, capability, and limitation text name the setting, the effective request, the source (`default` or `setting`), the default, the floor, the maximum, and how to change it. They also name the process effective heap when launch flags establish it. ## Change and scope `src/config/regexWorkerHeapRequest.ts` owns the setting. Startup (`parseConfig`) and doctor (`parseEnvironment`) resolve it with the same schema. The effective request is stored on `AppConfig` and passed into Hopper and Kuna, which pass it to `RegexMatcher`. A direct `KunaProvider` resolves the selected environment instead of ambient `process.env`. An omitted matcher request means the 128 MiB default, not ambient `process.env`. Setup keeps `REA_REGEX_WORKER_HEAP_REQUEST_MIB` in `SELECTED_REGISTRATION_SETTINGS`. A present value is copied into the generated registration. An unset value is omitted. The floor is 7 MiB. On Linux, the production worker (`^a` against `["a"]`, no `NODE_OPTIONS`) returns `ERR_WORKER_OUT_OF_MEMORY` at 6 MiB on Node 24.21.0 and 24.22.1, and on most runs of Node 24.11.0, 24.18.0, and 26.0.0. 7 MiB returned that match on Node 22.19.0, 22.23.4, 24.11.0, 24.18.0, 24.21.0, 24.22.1, 26.0.0, and 26.12.1, so a smaller request is not a matching bound on every supported Node line. The maximum is 16384 MiB, the same ceiling REA already uses for one isolated JavaScript analysis worker. Node starts with a larger flag; this maximum bounds only the REA request. A positive `--max-old-space-size` is reported as the effective heap and is not limited by this floor or maximum. Follow the [contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope). This is configuration of an existing regex worker request. It does not add a provider or a capability that executes, injects, mutates, or recovers a target. Maintainer scope decision: not applicable. The product decision to make this ceiling a setting is recorded in #6176. ## Contract and boundary impact - Semantic owner and earliest changed stage: `src/config/regexWorkerHeapRequest.ts` parses the setting; `parseConfig` resolves it before providers match. `explicitSetupRegistrationEnvironment` preserves a caller-selected value when setup writes a registration. - Contribution scope fit and linked maintainer decision: existing regex matching only; not applicable. - CLI and MCP/tool-catalog contract: `search_procedures` and `search_strings` descriptions name `REA_REGEX_WORKER_HEAP_REQUEST_MIB`, the 128 MiB default, the 7 MiB floor, and the 16384 MiB maximum. Capability and heap failures name the effective request and source. - Provider, bridge, target-format, or platform compatibility: Hopper and Kuna share the request. A positive Node `--max-old-space-size`, including `NODE_OPTIONS`, still overrides it. Percentage, `--max-heap-size`, env-file, and experimental config-file launches still leave the effective heap unestablished. - Evidence, artifact, provenance, or reconstruction contract: heap failures add `requested_worker_heap_source` beside `requested_worker_heap_mib`. - Target selection, authority, upstream behavior, and cleanup details (if applicable): none. - Generated metadata (`docs/public/product-catalog.json`), package, or installation impact (CI generates and validates ignored catalogs and `skills/`; edit authored instructions in `.agents/skills/reverse-engineer-anything/` and never commit build outputs): tool descriptions changed in source contracts. No authored skill text claimed the heap request had no override. Expected effects (check known effects; use documentation-only on its own). Unchecked items do not establish absence; leave uncertainties visible. - [x] Read existing files or captured data. - [ ] Launch or attach to a process. - [ ] Inject instrumentation or run scripts/code. - [ ] Change process memory, registers, files, or configuration. - [ ] Contact network services. - [ ] Recover protected code or work around protections. - [ ] Documentation or usability only; no target interaction. - [ ] Not sure yet; investigation or maintainer guidance needed. ## Evidence and regression coverage - Tests added or updated: parser accepts 7 and rejects 6; doctor and client-registration rejection of `6`; Hopper/Kuna capability text for a 256 MiB request while ambient `process.env` is `8`; a selected request of 256 stays in effect, and `--max-old-space-size=64` still overrides it to 64 MiB without clamping. A clean Node process, with `NODE_OPTIONS` removed and without `--import tsx` on `execArgv`, calls production `matchRegexValues` at the floor and matches `^a` against `["a"]`. - Base reproduction or other evidence: on main the request constant is 128 and recovery says there is no runtime override for that request. The 6 MiB floor was only a Node 22.23.4 startup bound. Reproducing the review on Linux Node 24.21.0 with the production matcher returned `ERR_WORKER_OUT_OF_MEMORY` at 6 MiB (5/5) and `{ok:true, indices:[0]}` at 7 MiB (5/5). The same 7 MiB production call matched on Node 22.19.0, 22.23.4, 24.11.0, 24.18.0, 26.0.0, and 26.12.1 (3/3 each). Raw worker repeats at 6 MiB were 0/10 matches on 24.21.0 and 24.22.1, 1/10 on 24.11.0 and 24.18.0, and 2/10 on 26.0.0; 7 MiB was 10/10 on those builds. - User-visible CLI/MCP output (if applicable): the default recovery says `The effective REA_REGEX_WORKER_HEAP_REQUEST_MIB value is 128 MiB (source default; default 128 MiB, floor 7 MiB, maximum 16384 MiB). Set REA_REGEX_WORKER_HEAP_REQUEST_MIB in the environment that starts REA to a decimal integer from 7 to 16384, then restart REA.` A 256 MiB setting uses `source setting` and `256 MiB`. - Remaining proof gaps: no real Hopper or Kuna binary was used. Windows was not executed. Node 22 and Node 26.12.1 can complete the trivial match at 6 MiB, but the setting rejects 6 so one accepted value works on every supported Node line. A heap above 128 was checked through `resolveRegexWorkerHeap` and capability text, not by exhausting a 256 MiB worker. For evidence-bearing changes: - [x] Observed, derived, and inferred claims remain distinguishable. - [x] Artifact identity, source provenance, and failed attempts remain preserved. - [x] Unsupported, incomplete, unavailable, or uncertain outcomes remain visible. ## Validation performed - `npm run check:fast` — typecheck and lint, 2 tasks successful (6.695s). - `npm run format:check` — all matched files use the correct format (4944 files). - `npm run test:local -- src/config/regexWorkerHeapRequest.test.ts src/process/RegexMatcher.test.ts src/application/Doctor.test.ts` — 124 passed. - `npm run test:focused -- tests/boundary/process/regexMatcher.test.ts tests/boundary/filesystem/clientConfigurationEnvironment.test.ts` — 43 passed, 10 skipped. The real-worker floor test is in the process-boundary file. ## Compatibility, safety, and release - Breaking changes or migration steps: none for an unset environment. The request remains 128 MiB. `REA_REGEX_WORKER_HEAP_REQUEST_MIB=6` is now a configuration error. It previously passed configuration and then failed regex search on supported Node 24 and Node 26.0.0. A positive `--max-old-space-size` still overrides the request. - Real Hopper/Ghidra, browser, or OS coverage: not run. Heap-request behavior was measured with the production regex worker on the Linux Node versions listed above. - Package or release metadata impact: none. - Security, privacy, process, or containment review: the worker still uses `resourceLimits.maxOldGenerationSizeMb`. The setting changes only that REA request. It does not raise a Node `--max-old-space-size` already applied to the process, and it does not change target memory. ## Author readiness Human owner: @morluto - [x] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [x] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [x] I declare this PR ready for initial maintainer review. ## Before merge - [x] This PR addresses a concrete problem or an agreed enhancement. - [x] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff. - [x] The PR has one focused outcome and the title follows `type(scope): outcome`. - [x] Related issue is linked, or the reason for not linking one is stated above. - [x] Tests cover changed observable behavior and meaningful failure paths. - [x] Owning docs, contracts, and generated metadata are updated where needed. - [x] User-visible CLI/MCP changes include representative output. - [x] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. <!-- CURSOR_AGENT_PR_BODY_END --> <div><a href="https://cursor.com/agents/bc-d912cecc-68c2-58d9-95b2-4a200e86d806?cursor_ref=pr_footer&cursor_cta=open_in_web"><picture><source media="(prefers-color-scheme: dark)" srcset="https://cursor.com/assets/images/open-in-web-dark.png"><source media="(prefers-color-scheme: light)" srcset="https://cursor.com/assets/images/open-in-web-light.png"><img alt="Open in Web" width="114" height="28" src="https://cursor.com/assets/images/open-in-web-dark.png"></picture></a>&nbsp;<a href="https://cursor.com/background-agent?bcId=bc-d912cecc-68c2-58d9-95b2-4a200e86d806&cursor_ref=pr_footer&cursor_cta=open_in_cursor"><picture><source media="(prefers-color-scheme: dark)" srcset="https://cursor.com/assets/images/open-in-cursor-dark.png"><source media="(prefers-color-scheme: light)" srcset="https://cursor.com/assets/images/open-in-cursor-light.png"><img alt="Open in Cursor" width="131" height="28" src="https://cursor.com/assets/images/open-in-cursor-dark.png"></picture></a>&nbsp;</div> --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
bsakaguchi 2026-11-01 21:46:36 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
morluto/rea#7266
No description provided.