morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #5797] [Feature] Make WABT command deadlines adjustable with actionable timeout recovery #3512

New issue

Closed

opened 2026-10-31 00:41:20 +00:00 by bsakaguchi · 3 comments

bsakaguchi commented

2026-10-31 00:41:20 +00:00

Owner

Originally created by @gensokyo-dev on GitHub (Oct 21, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/5797

Before submitting

Searched related issues/PRs; #5598 covers general recovery UX, while #5506 concerns a distinct fixed comparison allowance.
Reviewed contribution scope; this stays within existing read-only WASM inspection.
No credentials or sensitive samples included.

Area

MCP or agent integration

Current limitation

Source-derived on d672cc23998b6cbb21c485f118595ebfb82fed33: WABT inspection applies a hardcoded 30,000 ms deadline to every version/analysis command (WabtRelease.ts, WabtArtifactProvider.ts). Neither the named MCP input nor rea inspect-wasm-artifact offers a timeout control. WabtFailures.ts projects this through generic provider-timeout recovery, without identifying whether the budget is adjustable. Extending the MCP client's deadline cannot change this internal budget.

Use cases

An agent inspecting a valid module should be able to correct an insufficient command budget from the returned error, rather than guessing a nonexistent parameter, restarting unnecessarily, or repeatedly invoking the same failing request. Frequency on real slow modules: NOT_ENOUGH_INFO; no natural large-module timeout has been measured in this report.

Desired outcome and success criteria

Add one optional per-call command_timeout_ms control shared by CLI/MCP, preserving the 30,000 ms default and existing owned-command supervision.
Validate positive integer milliseconds against the runtime timer range. Report the selected value in Evidence and timeout errors.
On timeout, name the actual MCP field and CLI option, explain per-command versus whole-call scope, distinguish client delivery deadlines, and require unresolved cleanup to finish before retrying.
Keep input/output/representation safety allowances fixed. Increasing a deadline, switching delivery to summary, or adding host memory must not be advertised as raising them.

Expected interaction with targets

Read existing files; launch the already-supported caller-supplied WABT tools in private workspaces. No WASM execution, network acquisition, installation, GUI control, or new target authority.

Verification available so far

Source investigation; runtime and agent checks are proposed, not yet claimed passing.

Verification details

Planned: fail-before/pass-after contracts; actual CLI/MCP with upstream WABT 1.0.42 and source-owned modules; bounded delay injection for repeatable deadlines; actual Codex CLI adjusting a small budget and completing the same observation; fixed input/output allowance negative controls; cancellation/cleanup regressions. Record whole agent usage, elapsed/CPU and sampled process-family RSS, with cold/warm and fixture limits explicit.

Alternatives considered

Only changing error prose leaves no supported adjustment. Increasing MCP client timeout alone does not repair the server budget. A launch-environment-only setting unnecessarily requires server restart. Globally removing deadlines or raising safety limits is outside scope.

Possible direction

Own the request budget in the canonical WASM input; pass it to the existing owned-command supervisor. Keep specific recovery at the WABT producing boundary rather than guessing provider controls in generic error projection.

References

#5598 (RepoPrompt comparison and agent trials), #5767/#5768 (WABT configuration diagnostics), #5506 (fixed comparison allowance; separately addressed in current main).

Originally created by @gensokyo-dev on GitHub (Oct 21, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/5797 ### Before submitting - [x] Searched related issues/PRs; #5598 covers general recovery UX, while #5506 concerns a distinct fixed comparison allowance. - [x] Reviewed contribution scope; this stays within existing read-only WASM inspection. - [x] No credentials or sensitive samples included. ### Area MCP or agent integration ### Current limitation Source-derived on `d672cc23998b6cbb21c485f118595ebfb82fed33`: WABT inspection applies a hardcoded 30,000 ms deadline to every version/analysis command (`WabtRelease.ts`, `WabtArtifactProvider.ts`). Neither the named MCP input nor `rea inspect-wasm-artifact` offers a timeout control. `WabtFailures.ts` projects this through generic provider-timeout recovery, without identifying whether the budget is adjustable. Extending the MCP client's deadline cannot change this internal budget. ### Use cases An agent inspecting a valid module should be able to correct an insufficient command budget from the returned error, rather than guessing a nonexistent parameter, restarting unnecessarily, or repeatedly invoking the same failing request. Frequency on real slow modules: NOT_ENOUGH_INFO; no natural large-module timeout has been measured in this report. ### Desired outcome and success criteria - Add one optional per-call `command_timeout_ms` control shared by CLI/MCP, preserving the 30,000 ms default and existing owned-command supervision. - Validate positive integer milliseconds against the runtime timer range. Report the selected value in Evidence and timeout errors. - On timeout, name the actual MCP field and CLI option, explain per-command versus whole-call scope, distinguish client delivery deadlines, and require unresolved cleanup to finish before retrying. - Keep input/output/representation safety allowances fixed. Increasing a deadline, switching delivery to summary, or adding host memory must not be advertised as raising them. ### Expected interaction with targets Read existing files; launch the already-supported caller-supplied WABT tools in private workspaces. No WASM execution, network acquisition, installation, GUI control, or new target authority. ### Verification available so far Source investigation; runtime and agent checks are proposed, not yet claimed passing. ### Verification details Planned: fail-before/pass-after contracts; actual CLI/MCP with upstream WABT 1.0.42 and source-owned modules; bounded delay injection for repeatable deadlines; actual Codex CLI adjusting a small budget and completing the same observation; fixed input/output allowance negative controls; cancellation/cleanup regressions. Record whole agent usage, elapsed/CPU and sampled process-family RSS, with cold/warm and fixture limits explicit. ### Alternatives considered Only changing error prose leaves no supported adjustment. Increasing MCP client timeout alone does not repair the server budget. A launch-environment-only setting unnecessarily requires server restart. Globally removing deadlines or raising safety limits is outside scope. ### Possible direction Own the request budget in the canonical WASM input; pass it to the existing owned-command supervisor. Keep specific recovery at the WABT producing boundary rather than guessing provider controls in generic error projection. ### References #5598 (RepoPrompt comparison and agent trials), #5767/#5768 (WABT configuration diagnostics), #5506 (fixed comparison allowance; separately addressed in current main).

bsakaguchi closed this issue

2026-10-31 00:41:20 +00:00

bsakaguchi commented

2026-10-31 00:41:21 +00:00

Author

Owner

@gensokyo-dev commented on GitHub (Oct 21, 2026):

WABT timeout recovery: executed evidence

Baseline d672cc23998b6cbb21c485f118595ebfb82fed33; implementation 06f075479a56505a1fef509759fab5733fd8878b. Linux x64, Node 24.18.0/npm 11.16.0, Codex CLI 0.161.0, gpt-6.1-sol. Issue #5797 / PR #5798. Selected host surface: only inspect_wasm_artifact, explicitly preapproved; not a full-catalog agent qualification.

Contract ledger and decisive cases

Promise	Variation and independent oracle	Observed outcome
Selected per-command budget is enforced and honestly reported	Empty WASM bytes `00 61 73 6d 01 00 00 00`; native delay launcher before real WABT, then actual validator/decoder; selected budget versus observed error and original digest	Three source regressions fail before; public CLI/MCP and agent retries complete after
Default deadline can actually be raised, not just described	Exact same native 31-second delay launcher/file before and after; default 30,000 ms then agent-selected higher budget	Baseline blocked; after returned timeout, agent selected 120,000 ms and completed real WABT summary (two delayed commands, over 60 seconds combined)
Safety allowance is not an adjustable deadline	Sparse 8 MiB + 1-byte admission fixture, no truncation allowed; file size independently known	`resource_constraint`, `retryable: false`; agent stops after one call without inventing a timeout/memory workaround
Timer maximum does not invite overflow	Boundary 2,147,483,647 ms error seam, SDK-converted schema accepts maximum and rejects maximum + 1, zero, fractions, wrong types	Error at maximum explicitly says no higher supported timer; no recommendation to increase
Cleanup failure takes precedence over retry	Existing cancellation/retained-resource regressions; initial invalid shell-launcher experiment	Cleanup remains explicit; agent does not retry incomplete cleanup

Actual Codex sessions

Nine serial sessions / 14 recorded REA calls, with all raw prompts, JSONL, final answers, stderr, /usr/bin/time and reports retained locally.

Initial shell-script delay experiment: harness limitation, not a pure-timeout success. Command identity was /bin/sh <script> rather than the configured executable. Returned cleanup_incomplete; agent correctly stopped. Raw receipt preserved. The process later exited; no remaining process identity was sampled. One small provider workspace was retained by the fail-closed owner; it was not manually removed or called clean.
Native-launcher baseline: default 30,000 ms timeout, agent explicitly identified that the old tool contract had no supported budget control and stopped.
3–4. Version-stage 5 ms timeout, twice: agent selected 30,000 ms and completed same-file summary.
Analysis-stage 200 ms timeout: agent selected 30,000 ms and completed same-file summary.
Invalid zero budget: invalid_request; agent corrected to 30,000 ms and completed same-file summary.
Fixed input allowance: correctly blocked, no changed target, truncation or unsupported knob.
Omitted-budget normal control: successful summary with 30,000 ms profile.
Original default timeout, after fix: agent selected 120,000 ms and completed the same observation, proving an actual budget above the former hardcoded default.

The independent verifier checks exact first arguments, all recorded call counts, selected budgets in actual returned profiles, original SHA-256 93a44bbb96c751218e4c00d479e4c14358122a389acca16205b1e4d0dc5f9476, validation, summary coverage, retained parent reference, final Evidence ID, and no target/network execution. Zero unchanged failed REA retries; no observed OOM; no sampled remaining owned process identities. Several model messages reported host-side initial dispatch mistakes before REA received the request; these are preserved and included in whole-session usage, not counted as successful first actions or hidden by the REA-call counter.

Executed correctness checks

66 source tests + 11 MCP boundary tests passed. Three new regressions failed before implementation. Final combined focused rerun is recorded separately when complete.
npm run check, npm run docs:check (5 README checks), npm run build:cached: passed.
Real WABT 1.0.42 verifier: 51 CLI/MCP cases passed, including actual selected 60,000 ms versus omitted default budgets, tool/source digests, malformed input, partial Evidence, bundle readback and SDK schemas. Run 4821e1e1-b71f-49ab-b0a8-b1e1d0d6324d, verified descendants [].
Additional actual CLI/MCP parity: selected 20 ms timeout; corrected 2,000 ms successful complete Evidence; fixed oversized-input refusal with 60,000 ms budget. Passed with matching recovery actions. An initial parity harness assertion used a double-slash MCP literal versus CLI-resolved path; canonicalizing the test input, not product identity, resolved that harness mismatch.
Independent read-only Codex source review: no actionable findings. CI/automated review tracked on the PR, separately from these executed local checks.

Whole-agent costs and limitations

All nine trials, including failed/blocked/control experiments: 1,239,298 input tokens (1,059,840 cached), 5,566 output tokens; summed session wall 470.563 s, process-family CPU 136.86 s, maximum sampled process-family RSS 1,200.109 MiB (~1.17 GiB). Child identities sampled every 500 ms using PID + start time, including retained/reparented descendants; CPU from /usr/bin/time for the client/server/tool family. This RSS sum is not a kernel/cgroup peak: it may double-count shared pages and miss between-sample spikes.

Fresh client/server processes; OS file and model prompt caches warm/uncontrolled. No fully cold run. Source builds, fixture preparation, source review, deterministic tests, real-tool verifier and sampler CPU are outside these agent-session totals. Baseline is blocked whereas corrected flow completes: no equivalent-completed-task speedup or token reduction is claimed. All delay behavior is injected by source-owned native launchers before the actual trusted WABT executable, not a naturally slow module or qualification of all WABT/platform behavior. Empty-module trials do not establish large-application analysis performance. No global tool/configuration installation or target execution occurred.

Bounded source audit / remaining work

This is not a claim that every REA timeout/limit has been audited.

Owner	Existing control / remaining gap (source-derived unless measured above)
WABT	Fixed 30 s command budget now per-call adjustable; fixed 8 MiB input / 32 MiB output unchanged. Completed here.
JavaScript analysis	Existing `analysis_timeout_ms` per source and semantic selectors; prior #5608 recovery guidance. Summary is delivery only. No new runtime qualification this round.
Ghidra startup	Existing `REA_GHIDRA_STARTUP_TIMEOUT_MS`, 330,000 ms default; producer guidance explains launch environment, export/close/restart/reopen and client headroom. Not a per-call knob; no new target analysis here.
Hopper startup	Existing `REA_HOPPER_STARTUP_TIMEOUT_MS`, 120,000 ms default; configured startup guidance. No macOS qualification this round.
SQLite	Worker 30,000 ms budget still fixed; timeout projection is generic. Exact table/columns/sample selection exists, but no timeout control is advertised. Follow-up required; do not invent `analysis_timeout_ms` on the current tool.
Firmware	Analysis budget remains fixed at 120,000 ms; publication separately exposes `publication_timeout_ms`. Publication guidance already names that knob. Analysis timeout recovery merits a separate follow-up; publication control must not be suggested for the analysis phase.
Shared fixed representation allowances	Current main contains `fixedAllowanceOptions` and several non-retryable corrections, including WASM comparison. These must not be converted to temporary admission retries or configurable heap claims. Source-only check, not repeated large-fixture qualification here.

Each follow-up should independently verify its actual advertised controls and preserve partial Evidence/cleanup, rather than adding a generic guessed control map.

@gensokyo-dev commented on GitHub (Oct 21, 2026): # WABT timeout recovery: executed evidence Baseline `d672cc23998b6cbb21c485f118595ebfb82fed33`; implementation `06f075479a56505a1fef509759fab5733fd8878b`. Linux x64, Node 24.18.0/npm 11.16.0, Codex CLI 0.161.0, `gpt-6.1-sol`. Issue #5797 / PR #5798. Selected host surface: only `inspect_wasm_artifact`, explicitly preapproved; not a full-catalog agent qualification. ## Contract ledger and decisive cases | Promise | Variation and independent oracle | Observed outcome | | --- | --- | --- | | Selected per-command budget is enforced and honestly reported | Empty WASM bytes `00 61 73 6d 01 00 00 00`; native delay launcher before real WABT, then actual validator/decoder; selected budget versus observed error and original digest | Three source regressions fail before; public CLI/MCP and agent retries complete after | | Default deadline can actually be raised, not just described | Exact same native 31-second delay launcher/file before and after; default 30,000 ms then agent-selected higher budget | Baseline blocked; after returned timeout, agent selected 120,000 ms and completed real WABT summary (two delayed commands, over 60 seconds combined) | | Safety allowance is not an adjustable deadline | Sparse 8 MiB + 1-byte admission fixture, no truncation allowed; file size independently known | `resource_constraint`, `retryable: false`; agent stops after one call without inventing a timeout/memory workaround | | Timer maximum does not invite overflow | Boundary 2,147,483,647 ms error seam, SDK-converted schema accepts maximum and rejects maximum + 1, zero, fractions, wrong types | Error at maximum explicitly says no higher supported timer; no recommendation to increase | | Cleanup failure takes precedence over retry | Existing cancellation/retained-resource regressions; initial invalid shell-launcher experiment | Cleanup remains explicit; agent does not retry incomplete cleanup | ## Actual Codex sessions Nine serial sessions / 14 recorded REA calls, with all raw prompts, JSONL, final answers, stderr, `/usr/bin/time` and reports retained locally. 1. Initial shell-script delay experiment: **harness limitation**, not a pure-timeout success. Command identity was `/bin/sh <script>` rather than the configured executable. Returned `cleanup_incomplete`; agent correctly stopped. Raw receipt preserved. The process later exited; no remaining process identity was sampled. One small provider workspace was retained by the fail-closed owner; it was not manually removed or called clean. 2. Native-launcher baseline: default 30,000 ms timeout, agent explicitly identified that the old tool contract had no supported budget control and stopped. 3–4. Version-stage 5 ms timeout, twice: agent selected 30,000 ms and completed same-file summary. 5. Analysis-stage 200 ms timeout: agent selected 30,000 ms and completed same-file summary. 6. Invalid zero budget: `invalid_request`; agent corrected to 30,000 ms and completed same-file summary. 7. Fixed input allowance: correctly blocked, no changed target, truncation or unsupported knob. 8. Omitted-budget normal control: successful summary with 30,000 ms profile. 9. Original default timeout, after fix: agent selected 120,000 ms and completed the same observation, proving an actual budget above the former hardcoded default. The independent verifier checks exact first arguments, all recorded call counts, selected budgets in actual returned profiles, original SHA-256 `93a44bbb96c751218e4c00d479e4c14358122a389acca16205b1e4d0dc5f9476`, validation, summary coverage, retained parent reference, final Evidence ID, and no target/network execution. Zero unchanged failed REA retries; no observed OOM; no sampled remaining owned process identities. Several model messages reported host-side initial dispatch mistakes before REA received the request; these are preserved and included in whole-session usage, not counted as successful first actions or hidden by the REA-call counter. ## Executed correctness checks - 66 source tests + 11 MCP boundary tests passed. Three new regressions failed before implementation. Final combined focused rerun is recorded separately when complete. - `npm run check`, `npm run docs:check` (5 README checks), `npm run build:cached`: passed. - Real WABT 1.0.42 verifier: **51 CLI/MCP cases passed**, including actual selected 60,000 ms versus omitted default budgets, tool/source digests, malformed input, partial Evidence, bundle readback and SDK schemas. Run `4821e1e1-b71f-49ab-b0a8-b1e1d0d6324d`, verified descendants `[]`. - Additional actual CLI/MCP parity: selected 20 ms timeout; corrected 2,000 ms successful complete Evidence; fixed oversized-input refusal with 60,000 ms budget. Passed with matching recovery actions. An initial parity harness assertion used a double-slash MCP literal versus CLI-resolved path; canonicalizing the **test input**, not product identity, resolved that harness mismatch. - Independent read-only Codex source review: no actionable findings. CI/automated review tracked on the PR, separately from these executed local checks. ## Whole-agent costs and limitations All nine trials, including failed/blocked/control experiments: **1,239,298 input tokens** (1,059,840 cached), **5,566 output tokens**; summed session wall **470.563 s**, process-family CPU **136.86 s**, maximum sampled process-family RSS **1,200.109 MiB** (~1.17 GiB). Child identities sampled every 500 ms using PID + start time, including retained/reparented descendants; CPU from `/usr/bin/time` for the client/server/tool family. This RSS sum is not a kernel/cgroup peak: it may double-count shared pages and miss between-sample spikes. Fresh client/server processes; OS file and model prompt caches warm/uncontrolled. No fully cold run. Source builds, fixture preparation, source review, deterministic tests, real-tool verifier and sampler CPU are outside these agent-session totals. Baseline is blocked whereas corrected flow completes: **no equivalent-completed-task speedup or token reduction is claimed**. All delay behavior is injected by source-owned native launchers before the actual trusted WABT executable, not a naturally slow module or qualification of all WABT/platform behavior. Empty-module trials do not establish large-application analysis performance. No global tool/configuration installation or target execution occurred. ## Bounded source audit / remaining work This is not a claim that every REA timeout/limit has been audited. | Owner | Existing control / remaining gap (source-derived unless measured above) | | --- | --- | | WABT | Fixed 30 s command budget now per-call adjustable; fixed 8 MiB input / 32 MiB output unchanged. Completed here. | | JavaScript analysis | Existing `analysis_timeout_ms` per source and semantic selectors; prior #5608 recovery guidance. Summary is delivery only. No new runtime qualification this round. | | Ghidra startup | Existing `REA_GHIDRA_STARTUP_TIMEOUT_MS`, 330,000 ms default; producer guidance explains launch environment, export/close/restart/reopen and client headroom. Not a per-call knob; no new target analysis here. | | Hopper startup | Existing `REA_HOPPER_STARTUP_TIMEOUT_MS`, 120,000 ms default; configured startup guidance. No macOS qualification this round. | | SQLite | Worker 30,000 ms budget still fixed; timeout projection is generic. Exact table/columns/sample selection exists, but no timeout control is advertised. Follow-up required; do not invent `analysis_timeout_ms` on the current tool. | | Firmware | Analysis budget remains fixed at 120,000 ms; publication separately exposes `publication_timeout_ms`. Publication guidance already names that knob. Analysis timeout recovery merits a separate follow-up; publication control must not be suggested for the analysis phase. | | Shared fixed representation allowances | Current main contains `fixedAllowanceOptions` and several non-retryable corrections, including WASM comparison. These must not be converted to temporary admission retries or configurable heap claims. Source-only check, not repeated large-fixture qualification here. | Each follow-up should independently verify its actual advertised controls and preserve partial Evidence/cleanup, rather than adding a generic guessed control map.

bsakaguchi commented

2026-10-31 00:41:21 +00:00

Author

Owner

@gensokyo-dev commented on GitHub (Oct 21, 2026):

Final timeout/limit recovery evidence — PR #5798

Final candidate: e9cdf873c58bd32e8f43b69f8a9a6d1ceb50d645 (base d672cc23998b6cbb21c485f118595ebfb82fed33). This supplements the initial evidence and bounded source audit, not a claim that all providers are audited.

Observable result

WABT command budgets are adjustable through actual MCP command_timeout_ms / CLI --command-timeout-ms, default 30,000 ms. Error messages identify milliseconds, per-command scope, client headroom, fixed safety limits and cleanup-before-retry.
Same 31-second native delay fixture: old default-timeout flow blocked; after the returned error, actual Codex selected 120,000 ms, completed version checking and validation, and returned the same selected module's correct digest and summary. No target/configuration change or restart.
Automated Codex/Greptile found the combined timeout/cleanup gap. Fixed at the producer by classifying timeout once before cleanup precedence; structured previous_error retains selected budget/recovery without a second command-output copy. Cleanup remains non-retryable. Two fail-before/pass-after guards plus actual 100 ms combined-failure agent trials verify it.
A second automated finding exposed duplicate tool-profile construction that omitted budgets in rejected-module Evidence. Success and validation failure now reuse one admitted profile. Three validation/cancellation/cleanup assertions failed before the fix and pass after. Real CLI/MCP partial Evidence carries default and selected 42,000 ms budgets plus the fixed output allowance.
A malformed-module Codex trial distinguished invalid bytes from a timeout. It selected get_evidence_bundle with exact Evidence-ID filtering and detail: summary, recovered the correct original digest, and reported the validation blocker without repeating analysis or increasing the budget. It did not fetch the whole report.

Executed checks

80 focused tests passed, including source/service, SDK-advertised MCP schemas, cancellation, resource retention and cleanup recovery. Final test-orchestration refactor additionally reran all 33 provider cases successfully.
51 real WABT 1.0.42 CLI/MCP cases passed on the final production code, including the new rejected-profile assertions. Run 97c404d6-57f3-40ae-8e35-a67b2f060c83, verified descendants []. Earlier 51-case runs are retained but are not 153 distinct scenarios.
Additional actual CLI/MCP timeout / corrected retry / fixed oversized-input parity passed.
Final npm run check, npm run docs:check (5 README cases) and runtime build passed. During iteration, static checks caught an overlong test helper; the validation-cleanup assertions were moved to their owning helper, rather than suppressing the guard. A pair of concurrent artifact-check attempts correctly refused the checkout lock, then ran serially successfully.
Independent initial and cleanup-follow-up Codex source reviews completed; the first follow-up attempt hit model capacity, is retained as unavailable, and was successfully rerun. Automated findings above were fixed rather than ignored.

Actual-agent coverage and measured costs

13 serial Codex sessions / 20 recorded REA calls: seven completed summaries; five deliberate/legitimate blockers (old fixed timeout, immutable input allowance, two cleanup-uncertainty cases, malformed bytes with retained-Evidence read); one initial shell-launcher experiment that exposed a harness identity mismatch and is not a pure-timeout recovery pass. All raw prompts/transcripts/final answers, stderr, time files and independent oracle receipts remain locally under .cache/budget-recovery-20261021/.

Whole-session actual usage: 1,812,606 input tokens, including 1,522,688 cached; 7,699 output tokens. Summed elapsed 644.665 s, child process-family CPU 182.62 s, maximum 500 ms sampled family RSS 1,200.109 MiB (~1.17 GiB). Zero unchanged failed REA retries, no observed OOM, zero sampled remaining owned process identities. Host-side pre-dispatch tool mistakes appear in several transcripts; their usage is included, not called perfect first-action routing.

Costs include failed/blocked/control agent sessions, but exclude fixture preparation, source reviews, builds, deterministic checks, native verifier and sampler CPU. Fresh clients/servers with warm/uncontrolled OS/model caches; no truly cold benchmark. RSS is a sampled process sum (shared pages can be double-counted and spikes missed), not a kernel/cgroup peak. Delays are source-owned native launchers before real WABT, not naturally slow modules or large-app performance. A blocked baseline and a completed corrected task are not equivalent completed workflows: no speedup or token-reduction claim.

No global installation/configuration edits, target execution or network acquisition. Deliberate shell-identity/cleanup controls leave three small provider workspaces retained; they are not claimed cleaned or manually removed. Linux x64 only; no new macOS/Windows, IDA/Ghidra engine or full-catalog qualification. Remaining SQLite/fixed firmware analysis timeout gaps are source-derived follow-ups in the initial audit, not fixes included here.

@gensokyo-dev commented on GitHub (Oct 21, 2026): ## Final timeout/limit recovery evidence — PR #5798 Final candidate: `e9cdf873c58bd32e8f43b69f8a9a6d1ceb50d645` (base `d672cc23998b6cbb21c485f118595ebfb82fed33`). This supplements the [initial evidence and bounded source audit](https://github.com/morluto/rea/issues/5797#issuecomment-6309752108), not a claim that all providers are audited. ### Observable result - WABT command budgets are adjustable through actual MCP `command_timeout_ms` / CLI `--command-timeout-ms`, default 30,000 ms. Error messages identify milliseconds, per-command scope, client headroom, fixed safety limits and cleanup-before-retry. - Same 31-second native delay fixture: old default-timeout flow blocked; after the returned error, actual Codex selected **120,000 ms**, completed version checking and validation, and returned the same selected module's correct digest and summary. No target/configuration change or restart. - Automated Codex/Greptile found the combined timeout/cleanup gap. Fixed at the producer by classifying timeout once before cleanup precedence; structured `previous_error` retains selected budget/recovery without a second command-output copy. Cleanup remains non-retryable. Two fail-before/pass-after guards plus actual 100 ms combined-failure agent trials verify it. - A second automated finding exposed duplicate tool-profile construction that omitted budgets in rejected-module Evidence. Success and validation failure now reuse one admitted profile. Three validation/cancellation/cleanup assertions failed before the fix and pass after. Real CLI/MCP partial Evidence carries default and selected 42,000 ms budgets plus the fixed output allowance. - A malformed-module Codex trial distinguished invalid bytes from a timeout. It selected `get_evidence_bundle` **with exact Evidence-ID filtering and `detail: summary`**, recovered the correct original digest, and reported the validation blocker without repeating analysis or increasing the budget. It did not fetch the whole report. ### Executed checks - **80 focused tests passed**, including source/service, SDK-advertised MCP schemas, cancellation, resource retention and cleanup recovery. Final test-orchestration refactor additionally reran all 33 provider cases successfully. - **51 real WABT 1.0.42 CLI/MCP cases passed** on the final production code, including the new rejected-profile assertions. Run `97c404d6-57f3-40ae-8e35-a67b2f060c83`, verified descendants `[]`. Earlier 51-case runs are retained but are not 153 distinct scenarios. - Additional actual CLI/MCP timeout / corrected retry / fixed oversized-input parity passed. - Final `npm run check`, `npm run docs:check` (5 README cases) and runtime build passed. During iteration, static checks caught an overlong test helper; the validation-cleanup assertions were moved to their owning helper, rather than suppressing the guard. A pair of concurrent artifact-check attempts correctly refused the checkout lock, then ran serially successfully. - Independent initial and cleanup-follow-up Codex source reviews completed; the first follow-up attempt hit model capacity, is retained as unavailable, and was successfully rerun. Automated findings above were fixed rather than ignored. ### Actual-agent coverage and measured costs **13 serial Codex sessions / 20 recorded REA calls**: seven completed summaries; five deliberate/legitimate blockers (old fixed timeout, immutable input allowance, two cleanup-uncertainty cases, malformed bytes with retained-Evidence read); one initial shell-launcher experiment that exposed a harness identity mismatch and is **not** a pure-timeout recovery pass. All raw prompts/transcripts/final answers, stderr, time files and independent oracle receipts remain locally under `.cache/budget-recovery-20261021/`. Whole-session actual usage: **1,812,606 input tokens**, including **1,522,688 cached**; **7,699 output tokens**. Summed elapsed **644.665 s**, child process-family CPU **182.62 s**, maximum 500 ms sampled family RSS **1,200.109 MiB** (~1.17 GiB). **Zero unchanged failed REA retries**, no observed OOM, zero sampled remaining owned process identities. Host-side pre-dispatch tool mistakes appear in several transcripts; their usage is included, not called perfect first-action routing. Costs include failed/blocked/control agent sessions, but exclude fixture preparation, source reviews, builds, deterministic checks, native verifier and sampler CPU. Fresh clients/servers with warm/uncontrolled OS/model caches; no truly cold benchmark. RSS is a sampled process sum (shared pages can be double-counted and spikes missed), not a kernel/cgroup peak. Delays are source-owned native launchers before real WABT, not naturally slow modules or large-app performance. A blocked baseline and a completed corrected task are not equivalent completed workflows: **no speedup or token-reduction claim**. No global installation/configuration edits, target execution or network acquisition. Deliberate shell-identity/cleanup controls leave three small provider workspaces retained; they are not claimed cleaned or manually removed. Linux x64 only; no new macOS/Windows, IDA/Ghidra engine or full-catalog qualification. Remaining SQLite/fixed firmware analysis timeout gaps are source-derived follow-ups in the initial audit, not fixes included here.

bsakaguchi commented

2026-10-31 00:41:21 +00:00

Author

Owner

@gensokyo-dev commented on GitHub (Oct 21, 2026):

Merged #5798 as a5f982de8dbe4b092cc20ab89d492b7e7028b758 — feat(wasm): make command deadlines adjustable with recovery guidance (#5798). #5797 is closed. All 13 touched files match tested final head e9cdf873c58bd32e8f43b69f8a9a6d1ceb50d645; unrelated concurrent main changes are not claimed independently qualified. The actual CLI/MCP timeout/corrected-retry/fixed-input parity smoke passed again after merge.

Three independent source review passes completed across the implementation/review fixes; latest final-profile review found no actionable bugs. All actionable automated threads were fixed and resolved, Greptile and static checks succeeded. Admin override used per explicit user authorization after ordinary policy refusal. Human-review attestations remain unchecked; author-readiness failure and still-running code CI are disclosed, not reported green. Full executed evidence, warm-cache/token/resource accounting and remaining qualification gaps: https://github.com/morluto/rea/issues/5797#issuecomment-6309988721.

The squash commit contains Assisted-by: Kogasa (gensokyo-skills:kogasa-surprise-testing) for material fail-before/pass-after negative and combined-failure regressions. No Co-authored-by trailer was added.

Remaining provider timeout/limit recovery is recorded separately in #5808 (source-derived SQLite/firmware gaps and audit criteria). Those providers are not claimed fixed by this PR.

@gensokyo-dev commented on GitHub (Oct 21, 2026): Merged #5798 as **`a5f982de8dbe4b092cc20ab89d492b7e7028b758` — `feat(wasm): make command deadlines adjustable with recovery guidance (#5798)`**. #5797 is closed. All 13 touched files match tested final head `e9cdf873c58bd32e8f43b69f8a9a6d1ceb50d645`; unrelated concurrent main changes are not claimed independently qualified. The actual CLI/MCP timeout/corrected-retry/fixed-input parity smoke passed again after merge. Three independent source review passes completed across the implementation/review fixes; latest final-profile review found no actionable bugs. All actionable automated threads were fixed and resolved, Greptile and static checks succeeded. Admin override used per explicit user authorization after ordinary policy refusal. Human-review attestations remain unchecked; author-readiness failure and still-running code CI are disclosed, not reported green. Full executed evidence, warm-cache/token/resource accounting and remaining qualification gaps: https://github.com/morluto/rea/issues/5797#issuecomment-6309988721. The squash commit contains `Assisted-by: Kogasa (gensokyo-skills:kogasa-surprise-testing)` for material fail-before/pass-after negative and combined-failure regressions. **No `Co-authored-by` trailer** was added. Remaining provider timeout/limit recovery is recorded separately in **#5808** (source-derived SQLite/firmware gaps and audit criteria). Those providers are not claimed fixed by this PR.

bsakaguchi referenced this issue

2026-10-31 01:09:29 +00:00

[PR #3512] [CLOSED] Feat/web UI command 15374833743637920269 #5793

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#3512

No description provided.

Rows
Columns