[GH-ISSUE #6176] [Enhancement] Make hard-coded admission ceilings configurable with documented defaults

bsakaguchi commented

2026-10-31 00:43:51 +00:00

Owner

Originally created by @morluto on GitHub (Oct 21, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6176

Originally assigned to: @kaoru0822-kitauji on GitHub.

Before submitting

Searched open and closed issues; follows naming work in #5910 (parent #5808).
Removed credentials, private paths and sensitive target data.

Area

Shared resource admission / config (artifacts, process, MCP, providers).

Motivation

#5910 thematic PRs named fixed ceilings in recovery text and said whether a runtime override exists. Most numeric admissions stayed hard-coded constants for this release (e.g. ASAR header 16 MiB in #6172, regex matching deadline, MCP startup phase budgets, HAR/SAZ/qlog decoder heap). That honesty is good; operators still cannot raise a safe default when their workload needs more headroom.

Desired product shape: configurable with a documented default equal to today’s constant, plus a safety floor where unbounded growth would be dangerous (same pattern as existing REA_MCP_MAX_RESPONSE_BYTES with its 10 MiB floor).

Observed / current behavior

Examples of named-but-fixed admissions after #5910:

Ceiling	Today	Override?
ASAR header/index admission	16 MiB (`MAX_ASAR_HEADER_BYTES`)	No — recovery says so (#6172)
Regex matching deadline	fixed	No
Regex worker `maxOldGenerationSizeMb` request	128 MiB	Node `--max-old-space-size` / `NODE_OPTIONS` only; no REA setting (#6099)
MCP startup phase budgets (doctor/probe)	fixed ms	No (#6005)
MCP JSON depth	1000	No
HAR/SAZ/qlog decoder old-gen heap	192 MiB owned worker	Inherited `NODE_OPTIONS` cannot raise

Already configurable (keep as the pattern to copy): REA_MCP_MAX_RESPONSE_BYTES, JS max_heap_mb / --max-heap-mb, REA_JADX_HEAP_MIB, etc.

Upstream ASAR (electron/asar) has no max-header knob; REA’s 16 MiB is a REA safety admission. Making it configurable does not require changing upstream.

Expected behavior

Each selected admission gets a named setting (env and/or CLI) whose default is the current constant.
Recovery text names the setting, the effective value, the default, and any floor/ceiling on the setting itself.
Docs (docs/…, skill references) list defaults and that raising host memory alone does not raise REA admissions unless the setting says so.
Unbounded or unsafe raises are rejected or clamped with an explicit floor/hard max where needed.
No silent behavior change when unset.

Proposed scope (maintainers can trim)

P0 candidates

REA_ASAR_MAX_HEADER_BYTES (default 16777216) — real pain for huge Electron indexes; documented workaround today is “extract directory / smaller header”.
Regex matching deadline setting (default = current fixed ms).

P1

REA-owned decoder/worker request heaps that today claim “no runtime override” (HAR/SAZ/qlog; regex worker request if we want a REA knob in addition to Node CLI).
MCP startup phase budgets if operators need longer doctor/probe windows.

Out of scope for this issue

Re-doing #5910 naming/honesty (keep that closed once #6099 lands).
Making every internal graph/path micro-bound a public setting.
Streaming ASAR / removing admission entirely.

Acceptance ideas

Unset env → identical numeric behavior to current main.
Set above default → admission uses the new value; recovery names it.
Set below floor (if any) → rejected or clamped with clear error.
Focused tests lock default, override, and floor.

Parent honesty/naming: #5910
ASAR naming (fixed): #6172 (d7ca93cc)
Process regex wording (in flight): #6099
MCP naming + existing adjustable byte budget: #6005

Notes

Maintainer product decision: which admissions become public knobs vs stay internal safety bounds. Prefer few, well-documented settings over a spray of env vars.

Originally created by @morluto on GitHub (Oct 21, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6176 Originally assigned to: @kaoru0822-kitauji on GitHub. ### Before submitting - [x] Searched open and closed issues; follows naming work in #5910 (parent #5808). - [x] Removed credentials, private paths and sensitive target data. ### Area Shared resource admission / config (artifacts, process, MCP, providers). ### Motivation #5910 thematic PRs **named** fixed ceilings in recovery text and said whether a runtime override exists. Most numeric admissions stayed hard-coded constants for this release (e.g. ASAR header 16 MiB in #6172, regex matching deadline, MCP startup phase budgets, HAR/SAZ/qlog decoder heap). That honesty is good; operators still cannot raise a safe default when their workload needs more headroom. Desired product shape: **configurable with a documented default equal to today’s constant**, plus a safety floor where unbounded growth would be dangerous (same pattern as existing `REA_MCP_MAX_RESPONSE_BYTES` with its 10 MiB floor). ### Observed / current behavior Examples of named-but-fixed admissions after #5910: | Ceiling | Today | Override? | | --- | --- | --- | | ASAR header/index admission | 16 MiB (`MAX_ASAR_HEADER_BYTES`) | No — recovery says so (#6172) | | Regex matching deadline | fixed | No | | Regex worker `maxOldGenerationSizeMb` request | 128 MiB | Node `--max-old-space-size` / `NODE_OPTIONS` only; no REA setting (#6099) | | MCP startup phase budgets (doctor/probe) | fixed ms | No (#6005) | | MCP JSON depth | 1000 | No | | HAR/SAZ/qlog decoder old-gen heap | 192 MiB owned worker | Inherited `NODE_OPTIONS` cannot raise | Already configurable (keep as the pattern to copy): `REA_MCP_MAX_RESPONSE_BYTES`, JS `max_heap_mb` / `--max-heap-mb`, `REA_JADX_HEAP_MIB`, etc. Upstream ASAR (`electron/asar`) has no max-header knob; REA’s 16 MiB is a REA safety admission. Making it configurable does not require changing upstream. ### Expected behavior 1. Each selected admission gets a **named setting** (env and/or CLI) whose **default is the current constant**. 2. Recovery text names the setting, the effective value, the default, and any floor/ceiling on the setting itself. 3. Docs (`docs/…`, skill references) list defaults and that raising host memory alone does not raise REA admissions unless the setting says so. 4. Unbounded or unsafe raises are rejected or clamped with an explicit floor/hard max where needed. 5. No silent behavior change when unset. ### Proposed scope (maintainers can trim) **P0 candidates** - `REA_ASAR_MAX_HEADER_BYTES` (default 16777216) — real pain for huge Electron indexes; documented workaround today is “extract directory / smaller header”. - Regex matching deadline setting (default = current fixed ms). **P1** - REA-owned decoder/worker request heaps that today claim “no runtime override” (HAR/SAZ/qlog; regex worker *request* if we want a REA knob in addition to Node CLI). - MCP startup phase budgets if operators need longer doctor/probe windows. **Out of scope for this issue** - Re-doing #5910 naming/honesty (keep that closed once #6099 lands). - Making every internal graph/path micro-bound a public setting. - Streaming ASAR / removing admission entirely. ### Acceptance ideas - Unset env → identical numeric behavior to current main. - Set above default → admission uses the new value; recovery names it. - Set below floor (if any) → rejected or clamped with clear error. - Focused tests lock default, override, and floor. ### Related - Parent honesty/naming: #5910 - ASAR naming (fixed): #6172 (`d7ca93cc`) - Process regex wording (in flight): #6099 - MCP naming + existing adjustable byte budget: #6005 ### Notes Maintainer product decision: which admissions become public knobs vs stay internal safety bounds. Prefer few, well-documented settings over a spray of env vars.

bsakaguchi commented

2026-10-31 00:43:52 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 22, 2026):

赞同把"诚实命名但不可调"变成"可配置 + 安全下限"的产品形态，REA_MCP_MAX_RESPONSE_BYTES 已经证明了这个模式可行。三点补充：一是建议统一的配置命名与文档入口——所有上限走 REA_ 前缀环境变量并在同一页文档登记，避免散落在各 recovery 文案里；二是错误报告里最好同时给出"当前生效值"和"来源"（默认值 / 显式覆盖 / 下限兜底），用户看到 16 MiB 时能立刻知道这是不是自己设的；三是 safety floor 的语义要明确文档化——比如允许设 0 或极小值表示显式禁用该上限的场景，别让下限变成无法绕过的硬编码。最后提醒同步更新 #5910 里 recovery 文本的 override 说明，避免再次出现"文案说有 override 实际没有"的落差。

@XinwangTao commented on GitHub (Oct 22, 2026): 赞同把"诚实命名但不可调"变成"可配置 + 安全下限"的产品形态，REA_MCP_MAX_RESPONSE_BYTES 已经证明了这个模式可行。三点补充：一是建议统一的配置命名与文档入口——所有上限走 REA_ 前缀环境变量并在同一页文档登记，避免散落在各 recovery 文案里；二是错误报告里最好同时给出"当前生效值"和"来源"（默认值 / 显式覆盖 / 下限兜底），用户看到 16 MiB 时能立刻知道这是不是自己设的；三是 safety floor 的语义要明确文档化——比如允许设 0 或极小值表示显式禁用该上限的场景，别让下限变成无法绕过的硬编码。最后提醒同步更新 #5910 里 recovery 文本的 override 说明，避免再次出现"文案说有 override 实际没有"的落差。

bsakaguchi commented

2026-10-31 00:43:52 +00:00

Author

Owner

@kaoru0822-kitauji commented on GitHub (Oct 22, 2026):

Taking the ASAR half of P0 in one focused PR; the regex matching deadline can follow separately.

REA_ASAR_MAX_HEADER_BYTES: decimal bytes, default 16777216 (today's constant), maximum 268435456 (256 MiB). The ceiling keeps the decoded header text and its parse copy well under Node's ~512 Mi UTF-16 code-unit string limit. No floor: a lower value only refuses more archives.
Unset → identical behavior. Invalid → startup/doctor configuration error naming the setting, and admission refuses with the same constraint rather than falling back silently.
One owner (AsarHeader.ts) resolves the effective admission for all four consumers: classification probe, inventory, AsarArtifactReader (JS application analysis/extraction) and the Electron file scope, including its header cache budget. Workers inherit the process environment, so the same value applies there.
Recovery text names the effective value, its source (default or setting), the default and the maximum.

@kaoru0822-kitauji commented on GitHub (Oct 22, 2026): Taking the ASAR half of P0 in one focused PR; the regex matching deadline can follow separately. - `REA_ASAR_MAX_HEADER_BYTES`: decimal bytes, default 16777216 (today's constant), maximum 268435456 (256 MiB). The ceiling keeps the decoded header text and its parse copy well under Node's ~512 Mi UTF-16 code-unit string limit. No floor: a lower value only refuses more archives. - Unset → identical behavior. Invalid → startup/doctor configuration error naming the setting, and admission refuses with the same constraint rather than falling back silently. - One owner (`AsarHeader.ts`) resolves the effective admission for all four consumers: classification probe, inventory, `AsarArtifactReader` (JS application analysis/extraction) and the Electron file scope, including its header cache budget. Workers inherit the process environment, so the same value applies there. - Recovery text names the effective value, its source (default or setting), the default and the maximum.

bsakaguchi commented

2026-10-31 00:43:52 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 22, 2026):

PR #6209 的实现确认了 P0 的 ASAR 半边：REA_ASAR_MAX_HEADER_BYTES 默认 16 MiB、上限 256 MiB、无下限、未设置时行为不变、无效值启动即报错，recovery 文本也带上了来源（default/setting）与最大值——我上面提的"当前生效值 + 来源"这点已覆盖。P0 的另一半 regex matching deadline 还开着，建议在 #5910 的命名体系里给它同样的配置形态（默认 = 当前固定值、无 override 时 recovery 文本如实说明），避免两个 P0 落地节奏不一致。

@XinwangTao commented on GitHub (Oct 22, 2026): PR #6209 的实现确认了 P0 的 ASAR 半边：REA_ASAR_MAX_HEADER_BYTES 默认 16 MiB、上限 256 MiB、无下限、未设置时行为不变、无效值启动即报错，recovery 文本也带上了来源（default/setting）与最大值——我上面提的"当前生效值 + 来源"这点已覆盖。P0 的另一半 regex matching deadline 还开着，建议在 #5910 的命名体系里给它同样的配置形态（默认 = 当前固定值、无 override 时 recovery 文本如实说明），避免两个 P0 落地节奏不一致。

bsakaguchi commented

2026-10-31 00:43:53 +00:00

Author

Owner

@morluto commented on GitHub (Oct 22, 2026):

Reopening for remaining #6176 scope after #6209 (ASAR P0) merged.

Still open per the issue:

P0: regex matching deadline setting (default = today's fixed ms)
P1: REA-owned decoder/worker request heaps; MCP startup phase budgets

Maintainer direction: keep ASAR pattern (REA_* env, default = prior constant, recovery names effective value + source + default + max/floor). Prefer few well-documented knobs.

@morluto commented on GitHub (Oct 22, 2026): Reopening for remaining #6176 scope after #6209 (ASAR P0) merged. Still open per the issue: - **P0**: regex matching deadline setting (default = today's fixed ms) - **P1**: REA-owned decoder/worker request heaps; MCP startup phase budgets Maintainer direction: keep ASAR pattern (`REA_*` env, default = prior constant, recovery names effective value + source + default + max/floor). Prefer few well-documented knobs.

bsakaguchi commented

2026-10-31 00:43:53 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 22, 2026):

Agree with keeping the ASAR pattern for the remaining scope. Two suggestions for the regex matching deadline (#6267): make sure the effective value is resolved once and shared between the main process and regex workers, mirroring the single-owner approach in AsarHeader.ts; and treat invalid values as a startup/doctor configuration error rather than silently falling back — that is what keeps the "recovery names the effective value and its source" guarantee trustworthy. Few well-documented knobs over a spray of env vars is the right call.

@XinwangTao commented on GitHub (Oct 22, 2026): Agree with keeping the ASAR pattern for the remaining scope. Two suggestions for the regex matching deadline (#6267): make sure the effective value is resolved once and shared between the main process and regex workers, mirroring the single-owner approach in AsarHeader.ts; and treat invalid values as a startup/doctor configuration error rather than silently falling back — that is what keeps the "recovery names the effective value and its source" guarantee trustworthy. Few well-documented knobs over a spray of env vars is the right call.

bsakaguchi commented

2026-10-31 00:43:53 +00:00

Author

Owner

@ShuaixinHuang commented on GitHub (Oct 23, 2026):

@morluto A brief review request alongside your latest issue: could you please take a look at my small documentation PRs #6243, #6246 and #6250 when time permits? All three current heads have passed the required CI check, and I have completed the human review/ownership declarations. They cover existing IL2CPP slice-selection guidance, pcap parent-list wording and opt-in Bash completion setup. Please let me know if anything else is needed before merging, or if you would prefer that I revise or withdraw any of them. Thank you for your earlier merges and your time.

@ShuaixinHuang commented on GitHub (Oct 23, 2026): @morluto A brief review request alongside your latest issue: could you please take a look at my small documentation PRs #6243, #6246 and #6250 when time permits? All three current heads have passed the required CI check, and I have completed the human review/ownership declarations. They cover existing IL2CPP slice-selection guidance, pcap parent-list wording and opt-in Bash completion setup. Please let me know if anything else is needed before merging, or if you would prefer that I revise or withdraw any of them. Thank you for your earlier merges and your time.

bsakaguchi commented

2026-10-31 00:43:53 +00:00

Author

Owner

@ShuaixinHuang commented on GitHub (Oct 24, 2026):

Hi @morluto, could you please review #6932, #6936 and #6937 when you have time? All three have passed the required CI and readiness checks. I also have a few more bug fixes waiting to submit. Thank you for your time!

@ShuaixinHuang commented on GitHub (Oct 24, 2026): Hi @morluto, could you please review #6932, #6936 and #6937 when you have time? All three have passed the required CI and readiness checks. I also have a few more bug fixes waiting to submit. Thank you for your time!

bsakaguchi commented

2026-10-31 00:43:53 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 24, 2026):

ShuaixinHuang 的 #6932/#6936/#6937 已过 CI，与这个 issue 的落地路径一致：把剩余硬编码 ceiling 逐个转成可配置项并补齐文档。之前 XinwangTao 提的"有效值只解析一次共享、无效值报启动/诊断错误而非静默回退"建议，在后续 PR 中已体现，比静默 fallback 更利于恢复信息可信度。合并后本 issue 涉及的主要 ceiling 基本全线可配置，剩余可按 #6368 的恢复设计文档继续审计。

@XinwangTao commented on GitHub (Oct 24, 2026): ShuaixinHuang 的 #6932/#6936/#6937 已过 CI，与这个 issue 的落地路径一致：把剩余硬编码 ceiling 逐个转成可配置项并补齐文档。之前 XinwangTao 提的"有效值只解析一次共享、无效值报启动/诊断错误而非静默回退"建议，在后续 PR 中已体现，比静默 fallback 更利于恢复信息可信度。合并后本 issue 涉及的主要 ceiling 基本全线可配置，剩余可按 #6368 的恢复设计文档继续审计。

bsakaguchi commented

2026-10-31 00:43:54 +00:00

Author

Owner

@ShuaixinHuang commented on GitHub (Oct 24, 2026):

Thank you @morluto for the prompt reviews today! My remaining open PRs are #6984, #6985 and #6987; their latest required CI, readiness and title checks are passing. Could you please review them and merge those you consider ready? That would free up PR slots so I can submit the additional fixes I have prepared. Please let me know if you need anything else from me. Thanks again!

@gensokyo-dev @kaoru0822-kitauji @baron @N0zoM1z0, could you also help review #6984, #6985 and #6987 and merge any that are ready? Your help would allow me to submit the next prepared fixes once PR slots are available. Thank you for maintaining the project!

@ShuaixinHuang commented on GitHub (Oct 24, 2026): Thank you @morluto for the prompt reviews today! My remaining open PRs are #6984, #6985 and #6987; their latest required CI, readiness and title checks are passing. Could you please review them and merge those you consider ready? That would free up PR slots so I can submit the additional fixes I have prepared. Please let me know if you need anything else from me. Thanks again! @gensokyo-dev @kaoru0822-kitauji @baron @N0zoM1z0, could you also help review #6984, #6985 and #6987 and merge any that are ready? Your help would allow me to submit the next prepared fixes once PR slots are available. Thank you for maintaining the project!

bsakaguchi commented

2026-10-31 00:43:54 +00:00

Author

Owner

@ShuaixinHuang commented on GitHub (Oct 25, 2026):

Hi @morluto, thank you for your earlier reviews and merges! My current PRs #7062 (literal Markdown heading keys), #7063 (Godot 3 PCK v1 support), and #7065 (Zstandard ZIP reads) now all pass the CI, title and readiness checks for their latest commits.

Could you please review them when convenient and merge any you consider ready? I have more fixes prepared and would like to free up PR slots to continue contributing. Please let me know if anything else is needed from me. Thank you for your time!

@gensokyo-dev @kaoru0822-kitauji @baron @N0zoM1z0, could you also help review #7062, #7063 and #7065 and merge any you consider ready? Their latest CI, title and readiness checks are passing. Your help would let me submit the next prepared fixes once PR slots are available. Thank you!

@ShuaixinHuang commented on GitHub (Oct 25, 2026): Hi @morluto, thank you for your earlier reviews and merges! My current PRs #7062 (literal Markdown heading keys), #7063 (Godot 3 PCK v1 support), and #7065 (Zstandard ZIP reads) now all pass the CI, title and readiness checks for their latest commits. Could you please review them when convenient and merge any you consider ready? I have more fixes prepared and would like to free up PR slots to continue contributing. Please let me know if anything else is needed from me. Thank you for your time! @gensokyo-dev @kaoru0822-kitauji @baron @N0zoM1z0, could you also help review #7062, #7063 and #7065 and merge any you consider ready? Their latest CI, title and readiness checks are passing. Your help would let me submit the next prepared fixes once PR slots are available. Thank you!

bsakaguchi commented

2026-10-31 00:43:54 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 25, 2026):

Reopen 后的范围很清晰：P0 只剩 regex matching deadline（#6267），P1 项按需推进。建议 #6267 落地时沿用 #6209 的模式——默认值=当前固定值、恢复文案带生效值/来源/上限，并在 #6368 的设计文档里统一登记，避免每个 knob 的文档散落各处。等 #6267 合并后，本 issue 的配置化主体就完整了。

@XinwangTao commented on GitHub (Oct 25, 2026): Reopen 后的范围很清晰：P0 只剩 regex matching deadline（#6267），P1 项按需推进。建议 #6267 落地时沿用 #6209 的模式——默认值=当前固定值、恢复文案带生效值/来源/上限，并在 #6368 的设计文档里统一登记，避免每个 knob 的文档散落各处。等 #6267 合并后，本 issue 的配置化主体就完整了。

bsakaguchi commented

2026-10-31 00:43:54 +00:00

Author

Owner

@ShuaixinHuang commented on GitHub (Oct 26, 2026):

Hi @morluto, there is now independent real-export validation for #7063: KJLavender tested official Godot 3.6.3 packs on Windows 11 after locally merging the PR branch onto main. A 7-entry pack had all hashes verified and byte-identical extraction; a second pack extracted all 206 project files byte-identically. Test report. Embedded executables and other Godot 3 versions were not covered.

Could you please review #7062, #7063 and #7065 when convenient and merge any you consider ready? All three latest commits have passing CI, title and readiness checks. I have more fixes prepared and would appreciate freeing up PR slots to continue contributing. Please let me know if anything else is needed. Thank you!

@ShuaixinHuang commented on GitHub (Oct 26, 2026): Hi @morluto, there is now independent real-export validation for #7063: KJLavender tested official Godot 3.6.3 packs on Windows 11 after locally merging the PR branch onto main. A 7-entry pack had all hashes verified and byte-identical extraction; a second pack extracted all 206 project files byte-identically. [Test report](https://github.com/morluto/rea/pull/7063#issuecomment-6402883998). Embedded executables and other Godot 3 versions were not covered. Could you please review #7062, #7063 and #7065 when convenient and merge any you consider ready? All three latest commits have passing CI, title and readiness checks. I have more fixes prepared and would appreciate freeing up PR slots to continue contributing. Please let me know if anything else is needed. Thank you!

bsakaguchi referenced this issue

2026-10-31 01:10:10 +00:00

[PR #3769] Fix/review feedback 3762 #5885

bsakaguchi referenced this issue

2026-11-01 21:11:07 +00:00

[GH-ISSUE #5654] [Tracker] Open PRs that can't merge yet: conflicts and red CI (October 2026) #3425