morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #5812] [Feature] Expose SQLite worker deadlines with precise timeout and limit recovery #3518

New issue

Closed

opened 2026-10-31 00:41:24 +00:00 by bsakaguchi · 4 comments

bsakaguchi commented

2026-10-31 00:41:24 +00:00

Owner

Originally created by @gensokyo-dev on GitHub (Oct 21, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/5812

Before submitting

Searched related reports; #5808 tracks remaining provider budgets, #5798 fixes the separate WABT boundary.
Reviewed contribution scope. Existing read-only SQLite inspection only; no new target execution, SQL authority, installation or source mutation.
No credentials or sensitive samples.

Area

MCP or agent integration

Current limitation

Source-derived on a5f982de8dbe4b092cc20ab89d492b7e7028b758: SQLite's owned worker always receives SQLITE_DATABASE_LIMITS.timeoutMs = 30000. Its input contract/CLI have no supported deadline control; SqliteDatabaseFailures.ts emits generic timeout recovery with that constant. A caller cannot adjust this worker budget or identify the actual supported repair from the error. Raising the MCP client timeout alone does not change it.

The worker deadline is distinct from acquisition of the frozen DB/WAL and from final reply reads/cleanup. Existing input/SQLite value/SQL/reply/heap allowances are separate safety controls, not timeout settings.

Desired outcome and success criteria

Optional per-call worker_timeout_ms / CLI --worker-timeout-ms, default 30000, positive integer milliseconds within the runtime timer range.
One producer chooses the effective budget and applies it to the existing supervisor. Success and completed partial Evidence preserve the actual value; caller-selected parameters remain explicit and omission remains omission.
Timeout guidance names exact controls, scope, client headroom, cleanup-before-retry and the timer maximum. Timeout plus cleanup failure retains the primary facts without making cleanup retryable or duplicating output.
Fixed resource refusal must not suggest raising a timeout. Where applicable, name supported exact table/column/schema/sample selection rather than inventing a larger safety allowance; changing the analytical question is not an automatic recovery.

Use cases / alternatives

An agent inspecting a legitimate application database should recover from an insufficient worker deadline using the actual named contract. Exact row columns and compact schema inventory can reduce unnecessary analysis/output, but are not substitutes when the caller needs the omitted facts. Client-only timeout changes, global heap changes and repeated unchanged calls cannot repair a fixed worker budget.

Expected interaction with targets

Read existing database/WAL snapshots; launch the existing owned Node SQLite worker on private copies. No arbitrary SQL, view execution, target launch, native provider, network acquisition, repair or installation.

Verification available so far

Source investigation. Natural slow-database timeout frequency/performance: NOT_ENOUGH_INFO; no such runtime claim yet. Proposed checks are not completed results.

Verification details

Fail-before/pass-after contract and production-boundary tests; real CLI/MCP on public Chinook plus small source-owned DB/WAL, malformed and generated-large-value fixtures; actual Codex CLI selecting a corrected small deadline and completing the same question. Test supported field selection versus fixed value/reply limits and immutable input refusal. Retain cancellation/cleanup and Evidence identity coverage. Record whole-agent tokens, elapsed, CPU and sampled family RSS with cold/warm and failure limitations explicit.

Originally created by @gensokyo-dev on GitHub (Oct 21, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/5812 ### Before submitting - [x] Searched related reports; #5808 tracks remaining provider budgets, #5798 fixes the separate WABT boundary. - [x] Reviewed contribution scope. Existing read-only SQLite inspection only; no new target execution, SQL authority, installation or source mutation. - [x] No credentials or sensitive samples. ### Area MCP or agent integration ### Current limitation Source-derived on `a5f982de8dbe4b092cc20ab89d492b7e7028b758`: SQLite's owned worker always receives `SQLITE_DATABASE_LIMITS.timeoutMs = 30000`. Its input contract/CLI have no supported deadline control; `SqliteDatabaseFailures.ts` emits generic timeout recovery with that constant. A caller cannot adjust this worker budget or identify the actual supported repair from the error. Raising the MCP client timeout alone does not change it. The worker deadline is distinct from acquisition of the frozen DB/WAL and from final reply reads/cleanup. Existing input/SQLite value/SQL/reply/heap allowances are separate safety controls, not timeout settings. ### Desired outcome and success criteria - Optional per-call `worker_timeout_ms` / CLI `--worker-timeout-ms`, default 30000, positive integer milliseconds within the runtime timer range. - One producer chooses the effective budget and applies it to the existing supervisor. Success and completed partial Evidence preserve the actual value; caller-selected parameters remain explicit and omission remains omission. - Timeout guidance names exact controls, scope, client headroom, cleanup-before-retry and the timer maximum. Timeout plus cleanup failure retains the primary facts without making cleanup retryable or duplicating output. - Fixed resource refusal must not suggest raising a timeout. Where applicable, name supported exact table/column/schema/sample selection rather than inventing a larger safety allowance; changing the analytical question is not an automatic recovery. ### Use cases / alternatives An agent inspecting a legitimate application database should recover from an insufficient worker deadline using the actual named contract. Exact row columns and compact schema inventory can reduce unnecessary analysis/output, but are not substitutes when the caller needs the omitted facts. Client-only timeout changes, global heap changes and repeated unchanged calls cannot repair a fixed worker budget. ### Expected interaction with targets Read existing database/WAL snapshots; launch the existing owned Node SQLite worker on private copies. No arbitrary SQL, view execution, target launch, native provider, network acquisition, repair or installation. ### Verification available so far Source investigation. Natural slow-database timeout frequency/performance: NOT_ENOUGH_INFO; no such runtime claim yet. Proposed checks are not completed results. ### Verification details Fail-before/pass-after contract and production-boundary tests; real CLI/MCP on public Chinook plus small source-owned DB/WAL, malformed and generated-large-value fixtures; actual Codex CLI selecting a corrected small deadline and completing the same question. Test supported field selection versus fixed value/reply limits and immutable input refusal. Retain cancellation/cleanup and Evidence identity coverage. Record whole-agent tokens, elapsed, CPU and sampled family RSS with cold/warm and failure limitations explicit.

bsakaguchi closed this issue

2026-10-31 00:41:24 +00:00

bsakaguchi commented

2026-10-31 00:41:26 +00:00

Author

Owner

@gensokyo-dev commented on GitHub (Oct 21, 2026):

Executed SQLite recovery verification (2026-10-21)

Implemented against a5f982de8dbe4b092cc20ab89d492b7e7028b758: optional per-call worker_timeout_ms / --worker-timeout-ms, default 30000, integer range 1..2147483647. The parent owns the actual worker budget; report metadata, explicit Evidence parameters and timeout diagnostics agree. Historical missing metadata remains unknown. Worker startup/queries/reply writing are covered, not snapshot acquisition or final read/cleanup/delivery. Fixed byte/heap bounds remain fixed.

Correctness and boundaries

npm run test:sqlite: 150 passed, 3 skipped, 12 files, final run 71.60 s on Linux / Node 24.18.0. Includes real compiled CLI/stdio MCP, custom budget retained through cancellation/cleanup, frozen DB/WAL identity, exact columns, fixed combined input admission and owned worker cleanup. The controlled startup-delay boundary test times out at 50 ms and completes the same database at 2000 ms.
Six focused guards failed before implementation: selected-budget service acceptance/mismatch checks and timeout projection with/without cleanup and at the timer maximum. They pass now; this is not a claim that every new test was run fail-before.
npm run check: passed typecheck, formatting, lint/module boundaries, unused-code checks, metadata and architecture guards. Existing lint warnings remain. An earlier formatting failure was corrected with the repository formatter.
npm run docs:check and npm run verify:test-discovery: passed again after the final test additions (five README checks; all 1900 test files discovered exactly once).
An independent public verifier passed six CLI/MCP pairs: default, selected 60000, 1 ms timeout, fixed value refusal despite 60000, authorized exact-column success, and oversized-input refusal despite 60000. It validates SDK-converted input/output JSON Schemas with AJV 2020, valid timer endpoints and six invalid values, exact retained Evidence readback, and original input identity. Its first run used unsupported complete-bundle filters; the verifier was corrected to the declared complete-bundle contract, not the product.
A separate read-only Codex source review reported no actionable introduced findings. That review did not execute tests and is not human review.

Actual Codex CLI behavior

Model gpt-6.1-sol, Codex CLI 0.161.0, fresh ephemeral sessions, only inspect_sqlite_database enabled, at most six sequential calls. Prompts seed the exact first request and preserve the analytical question; no recovery procedure is supplied. No shell, installation, edits, alternate engines or arbitrary SQL are authorized.

Public Chinook SQLite, SHA-256 7651ba378ac2fcd0dfc3c66fb101f7a7eed3ba39a612ec642b96e20702061f15, 3503 tracks. The task selects two TrackId/Name rows with compact inventory. An independent Python SQLite read-only/immutable query verifies returned IDs/names; a separate verifier checks every receipt, final Evidence ID, original digest, selections, absence of mutations/unchanged failed retries and remaining owned processes.

Trial	Observed behavior	REA calls	Input / cached / output tokens	Wall / CPU s	Peak family RSS MiB*
Before, normal first attempt	Host tool discovery stopped before REA; not a recovery pass	0	46941 / 24576 / 253	18.23 / 7.31	1034.2
Before, unsupported knob	`invalid_request`; removes unsupported field, finishes same task	2	107101 / 87808 / 581	29.57 / 9.02	1083.2
After, 1 ms	`provider_timeout`; independently selects 30000 and finishes same task	2	159261 / 136192 / 679	57.08 / 14.86	1086.2
After, repeated 1 ms	Same corrected success	2	105812 / 89216 / 583	35.18 / 13.50	1121.2
After, invalid 0	`invalid_request`; selects legal 30000, finishes same task	2	135448 / 114176 / 681	42.90 / 13.37	1044.3
After, generated-large value	Fixed 16 MiB refusal; selects authorized `id/title`, completes	2	104161 / 86912 / 512	33.46 / 14.56	1112.0
After, oversized first attempt	Host discovery stopped before REA; not a recovery pass	0	46703 / 30592 / 273	24.93 / 11.56	1056.4
After, normal	One compact call completes	1	94943 / 76672 / 539	31.98 / 10.26	1091.8
After, oversized repeat	Receives fixed 256 MiB refusal; honestly stops, no invented larger budget or changed input	1	78439 / 63360 / 386	30.39 / 8.03	1032.0
Before, normal repeat	One compact call completes on pre-change `ea24207b5`	1	97707 / 71168 / 463	27.08 / 8.48	1098.8

Ten recorded sessions: eight reached REA, two host-discovery blockers. Seven completed answers and one legitimate fixed-input blocker. No unchanged failed retries, shell commands, unexpected tools, or surviving sampled process identities. Whole trial totals include failed host attempts: 976516 input tokens (780672 cached), 4950 output, 330.81 wall seconds, 110.95 CPU seconds. Token counts include host/tool discovery and retries, not just response bytes. No claim that this patch makes model runs faster or cheaper: the one-call before/after controls are noisy and not statistical performance evidence.

*RSS samples every 500 ms, process identity tracked by PID plus start time including observed descendants; not a kernel peak or native-memory cap. CPU from /usr/bin/time includes the agent/server/worker descendants, not preparation/tests/review or the sampler. Runs use fresh processes; OS/model caches are uncontrolled, so not a cold-cache versus warm-cache benchmark. Raw prompts, JSONL, stderr, timing, reports and verification scripts are retained locally in .cache/sqlite-recovery-20261021/, not committed or publicly attached. No OOM observed; serial model sessions and bounded Node heaps were used.

Limits of this evidence / follow-up

The tiny 1 ms deadline is deliberate fault injection, not a naturally slow 30-second database. The generated-value and sparse 256 MiB+1 files are source-owned resource controls, not healthy large-application qualification. macOS/Windows, other Node versions, natural slow databases, total native memory bounds and other providers are unverified here. The existing native SQLITE_NOMEM versus JavaScript-heap classification is separate follow-up; this change does not claim to fix it. Remaining provider boundaries stay tracked in #5808.

@gensokyo-dev commented on GitHub (Oct 21, 2026): ## Executed SQLite recovery verification (2026-10-21) Implemented against `a5f982de8dbe4b092cc20ab89d492b7e7028b758`: optional per-call `worker_timeout_ms` / `--worker-timeout-ms`, default 30000, integer range 1..2147483647. The parent owns the actual worker budget; report metadata, explicit Evidence parameters and timeout diagnostics agree. Historical missing metadata remains unknown. Worker startup/queries/reply writing are covered, not snapshot acquisition or final read/cleanup/delivery. Fixed byte/heap bounds remain fixed. ### Correctness and boundaries - `npm run test:sqlite`: **150 passed, 3 skipped, 12 files**, final run 71.60 s on Linux / Node 24.18.0. Includes real compiled CLI/stdio MCP, custom budget retained through cancellation/cleanup, frozen DB/WAL identity, exact columns, fixed combined input admission and owned worker cleanup. The controlled startup-delay boundary test times out at 50 ms and completes the same database at 2000 ms. - Six focused guards failed before implementation: selected-budget service acceptance/mismatch checks and timeout projection with/without cleanup and at the timer maximum. They pass now; this is not a claim that every new test was run fail-before. - `npm run check`: passed typecheck, formatting, lint/module boundaries, unused-code checks, metadata and architecture guards. Existing lint warnings remain. An earlier formatting failure was corrected with the repository formatter. - `npm run docs:check` and `npm run verify:test-discovery`: passed again after the final test additions (five README checks; all 1900 test files discovered exactly once). - An independent public verifier passed **six CLI/MCP pairs**: default, selected 60000, 1 ms timeout, fixed value refusal despite 60000, authorized exact-column success, and oversized-input refusal despite 60000. It validates SDK-converted input/output JSON Schemas with AJV 2020, valid timer endpoints and six invalid values, exact retained Evidence readback, and original input identity. Its first run used unsupported complete-bundle filters; the verifier was corrected to the declared complete-bundle contract, not the product. - A separate read-only Codex source review reported no actionable introduced findings. That review did not execute tests and is not human review. ### Actual Codex CLI behavior Model `gpt-6.1-sol`, Codex CLI 0.161.0, fresh ephemeral sessions, only `inspect_sqlite_database` enabled, at most six sequential calls. Prompts seed the exact first request and preserve the analytical question; **no recovery procedure is supplied**. No shell, installation, edits, alternate engines or arbitrary SQL are authorized. Public [Chinook SQLite](https://github.com/lerocha/chinook-database/blob/master/ChinookDatabase/DataSources/Chinook_Sqlite.sqlite), SHA-256 `7651ba378ac2fcd0dfc3c66fb101f7a7eed3ba39a612ec642b96e20702061f15`, 3503 tracks. The task selects two `TrackId/Name` rows with compact inventory. An independent Python SQLite read-only/immutable query verifies returned IDs/names; a separate verifier checks every receipt, final Evidence ID, original digest, selections, absence of mutations/unchanged failed retries and remaining owned processes. | Trial | Observed behavior | REA calls | Input / cached / output tokens | Wall / CPU s | Peak family RSS MiB* | | --- | --- | ---: | ---: | ---: | ---: | | Before, normal first attempt | Host tool discovery stopped before REA; **not a recovery pass** | 0 | 46941 / 24576 / 253 | 18.23 / 7.31 | 1034.2 | | Before, unsupported knob | `invalid_request`; removes unsupported field, finishes same task | 2 | 107101 / 87808 / 581 | 29.57 / 9.02 | 1083.2 | | After, 1 ms | `provider_timeout`; independently selects 30000 and finishes same task | 2 | 159261 / 136192 / 679 | 57.08 / 14.86 | 1086.2 | | After, repeated 1 ms | Same corrected success | 2 | 105812 / 89216 / 583 | 35.18 / 13.50 | 1121.2 | | After, invalid 0 | `invalid_request`; selects legal 30000, finishes same task | 2 | 135448 / 114176 / 681 | 42.90 / 13.37 | 1044.3 | | After, generated-large value | Fixed 16 MiB refusal; selects authorized `id/title`, completes | 2 | 104161 / 86912 / 512 | 33.46 / 14.56 | 1112.0 | | After, oversized first attempt | Host discovery stopped before REA; **not a recovery pass** | 0 | 46703 / 30592 / 273 | 24.93 / 11.56 | 1056.4 | | After, normal | One compact call completes | 1 | 94943 / 76672 / 539 | 31.98 / 10.26 | 1091.8 | | After, oversized repeat | Receives fixed 256 MiB refusal; honestly stops, no invented larger budget or changed input | 1 | 78439 / 63360 / 386 | 30.39 / 8.03 | 1032.0 | | Before, normal repeat | One compact call completes on pre-change `ea24207b5` | 1 | 97707 / 71168 / 463 | 27.08 / 8.48 | 1098.8 | Ten recorded sessions: eight reached REA, two host-discovery blockers. Seven completed answers and one legitimate fixed-input blocker. **No unchanged failed retries, shell commands, unexpected tools, or surviving sampled process identities**. Whole trial totals include failed host attempts: 976516 input tokens (780672 cached), 4950 output, 330.81 wall seconds, 110.95 CPU seconds. Token counts include host/tool discovery and retries, not just response bytes. No claim that this patch makes model runs faster or cheaper: the one-call before/after controls are noisy and not statistical performance evidence. *RSS samples every 500 ms, process identity tracked by PID plus start time including observed descendants; not a kernel peak or native-memory cap. CPU from `/usr/bin/time` includes the agent/server/worker descendants, not preparation/tests/review or the sampler. Runs use fresh processes; OS/model caches are uncontrolled, so **not** a cold-cache versus warm-cache benchmark. Raw prompts, JSONL, stderr, timing, reports and verification scripts are retained locally in `.cache/sqlite-recovery-20261021/`, not committed or publicly attached. No OOM observed; serial model sessions and bounded Node heaps were used. ### Limits of this evidence / follow-up The tiny 1 ms deadline is deliberate fault injection, not a naturally slow 30-second database. The generated-value and sparse 256 MiB+1 files are source-owned resource controls, not healthy large-application qualification. macOS/Windows, other Node versions, natural slow databases, total native memory bounds and other providers are unverified here. The existing native `SQLITE_NOMEM` versus JavaScript-heap classification is separate follow-up; this change does not claim to fix it. Remaining provider boundaries stay tracked in #5808.

bsakaguchi commented

2026-10-31 00:41:26 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 21, 2026):

The success criteria are well scoped — in particular the rule that "fixed resource refusal must not suggest raising a timeout" and "changing the analytical question is not an automatic recovery" keep the guidance honest, so a caller can't paper over a genuine 256 MiB refusal by bumping the deadline. Two things I'd want covered before this lands: (1) the timeout+cleanup-failure path — per the criteria, primary facts must be retained without making cleanup retryable; worth a dedicated test where the worker times out and the subsequent cleanup also fails, asserting the timeout error stays primary; (2) the CLI/MCP parity — the per-call worker_timeout_ms and the CLI flag should resolve to the same effective budget, ideally with a test that both entry points produce the identical timeout error shape for the same deadline value. The before/after evidence with the 1 ms fault injection is a nice honest way to prove the pass-after contract without pretending it measures real-world slow databases.

@XinwangTao commented on GitHub (Oct 21, 2026): The success criteria are well scoped — in particular the rule that "fixed resource refusal must not suggest raising a timeout" and "changing the analytical question is not an automatic recovery" keep the guidance honest, so a caller can't paper over a genuine 256 MiB refusal by bumping the deadline. Two things I'd want covered before this lands: (1) the timeout+cleanup-failure path — per the criteria, primary facts must be retained without making cleanup retryable; worth a dedicated test where the worker times out and the subsequent cleanup also fails, asserting the timeout error stays primary; (2) the CLI/MCP parity — the per-call worker_timeout_ms and the CLI flag should resolve to the same effective budget, ideally with a test that both entry points produce the identical timeout error shape for the same deadline value. The before/after evidence with the 1 ms fault injection is a nice honest way to prove the pass-after contract without pretending it measures real-world slow databases.

bsakaguchi commented

2026-10-31 00:41:26 +00:00

Author

Owner

@gensokyo-dev commented on GitHub (Oct 21, 2026):

Final validation and review fixes: `b58ca531f1f978a5813d3fe5b8e054171fe0466b`

Supersedes the initial counts in the ten-session record; raw earlier failures and sessions remain retained, not erased.

All three actionable automated findings were fixed at their owners:

Fresh versus historical results: only persisted report parsing permits an unknown/missing budget. Every fresh shared-service execution must report its selected-or-default value. A new guard reproduced the incorrect accepted result at f1fbd1d62; default omission, explicit selection, missing/mismatched metadata and historical parsing now have distinct coverage.
Actual retained-cleanup path: a real owned Node worker times out at selected 3000 ms, cleanup confirmation is deliberately denied, the owned snapshot remains retained, and a second inspection is blocked without launching another worker. After allowing exact owned cleanup, the original error becomes provider_timeout with the same budget. The new provider test reproduced two serialized output copies before the fix. This is a controlled production-boundary process/cleanup fault, not a naturally slow SQLite query or a real OS permission failure.
Typed primary and single output owner: isolated OwnedCommandFailure cleanup previously kept only serialized timeout facts. A fail-before guard proved the missing cleanupPrimary. A shared SQLite cleanup composer now preserves that typed primary and canonical captured output while keeping history metadata-only. Both paths retain exactly one output payload before and after cleanup resolution; cleanup remains non-retryable while unresolved. Related descriptor-cleanup assertions now reflect canonical outer output ownership rather than requiring another nested payload. Windows-specific assertions were source-updated but not locally executed.

Final executed checks on this code:

npm run check: passed (existing warnings unchanged).
npm run test:sqlite: 155 passed, 3 skipped, 12 files, 69.39 s.
npm run docs:check: passed, including five README consistency checks.
npm run verify:test-discovery: all 1900 test files discovered exactly once.
Public CLI/MCP verifier rerun after the cleanup refactor: all six pairs passed, including SDK-converted JSON Schema validation, timer endpoints/invalid values, exact retained Evidence readback, independent Python SQLite oracle and unchanged input identity.

Three additional actual Codex CLI sessions

Same model, exact-first-call prompt, restrictions and measurement method as the original record; no recovery procedure was supplied.

Trial	Outcome	REA calls	Input / cached / output tokens	Wall / CPU s	Sampled peak family RSS MiB
`normal-final`	Host discovery stopped before REA; retained as a failed control	0	46752 / 30592 / 282	19.61 / 7.53	1079.1
`timeout-final`	1 ms timeout → model selects 30000 → same Chinook question succeeds	2	110095 / 91392 / 627	32.95 / 8.46	1035.0
`repeat-final` (final cleanup code)	Same independently corrected success	2	159397 / 120064 / 700	37.06 / 8.10	1113.8

13 total recorded sessions: ten reached REA; nine completed answers and one legitimate fixed-input blocker. Three pre-REA host-discovery failures are not product recovery passes. Four distinct 1 ms timeout trials all selected 30000 and completed the unchanged requested Chinook rows. The independent receipt verifier checked exact inputs, selected columns/sample/scope, final Evidence IDs/digests/rows, and all recorded failure classifications. No unchanged failed retries, shell commands, unexpected tools or surviving sampled process identities.

Whole 13-session totals (including failed controls): 1292760 input tokens, 1022720 cached input tokens, 6559 output tokens; 420.43 wall s; 135.04 CPU s; 1121.2 MiB maximum sampled process-family RSS. Sampling is every 500 ms, not a kernel peak. Preparation/tests/reviews/sampler excluded from these task costs. Fresh processes, uncontrolled OS/model caches; no cold-versus-warm claim or statistical speedup claim. No OOM observed. Natural 30-second database workloads and total native memory bounds remain unmeasured.

The three pre-dispatch sessions contain the model's claim that the tool was unavailable, with no REA tool calls and no transport error captured in stderr. Whether this is model discovery, host registration/routing or the constrained prompt is not established. They remain unresolved agent/host-discovery failures, not product recovery passes or proven REA transport defects. A SQLite timeout message cannot repair a request that never reaches the tool. Raw receipts remain in the ignored local verification directory; no binaries, private data or credentials were committed.

CI distinction

Current-head CI/review results are recorded on #5818. On previous head 200f493d9, test shard 4 failed only in cliInputSchemaRequired.test.ts for WABT commandTimeoutMs, not SQLite. The same failure was reproduced in the earlier WABT worktree e9cdf873c58bd32e8f43b69f8a9a6d1ceb50d645, before this SQLite change. Its test oracle checks a standalone field with undefined, which does not represent object-key omission for .exactOptional(). This is tracked in #5845 rather than changing the WABT contract or relaxing the SQLite tests in this PR. Human-review declarations are not fabricated; readiness-policy status is not represented as all green.

@gensokyo-dev commented on GitHub (Oct 21, 2026): ## Final validation and review fixes: `b58ca531f1f978a5813d3fe5b8e054171fe0466b` Supersedes the initial counts in [the ten-session record](https://github.com/morluto/rea/issues/5812#issuecomment-6311080292); raw earlier failures and sessions remain retained, not erased. All three actionable automated findings were fixed at their owners: 1. **Fresh versus historical results**: only persisted report parsing permits an unknown/missing budget. Every fresh shared-service execution must report its selected-or-default value. A new guard reproduced the incorrect accepted result at `f1fbd1d62`; default omission, explicit selection, missing/mismatched metadata and historical parsing now have distinct coverage. 2. **Actual retained-cleanup path**: a real owned Node worker times out at selected 3000 ms, cleanup confirmation is deliberately denied, the owned snapshot remains retained, and a second inspection is blocked without launching another worker. After allowing exact owned cleanup, the original error becomes `provider_timeout` with the same budget. The new provider test reproduced two serialized output copies before the fix. This is a controlled production-boundary process/cleanup fault, **not** a naturally slow SQLite query or a real OS permission failure. 3. **Typed primary and single output owner**: isolated `OwnedCommandFailure` cleanup previously kept only serialized timeout facts. A fail-before guard proved the missing `cleanupPrimary`. A shared SQLite cleanup composer now preserves that typed primary and canonical captured output while keeping history metadata-only. Both paths retain **exactly one** output payload before and after cleanup resolution; cleanup remains non-retryable while unresolved. Related descriptor-cleanup assertions now reflect canonical outer output ownership rather than requiring another nested payload. Windows-specific assertions were source-updated but not locally executed. Final executed checks on this code: - `npm run check`: passed (existing warnings unchanged). - `npm run test:sqlite`: **155 passed, 3 skipped, 12 files, 69.39 s**. - `npm run docs:check`: passed, including five README consistency checks. - `npm run verify:test-discovery`: all **1900** test files discovered exactly once. - Public CLI/MCP verifier rerun after the cleanup refactor: **all six pairs passed**, including SDK-converted JSON Schema validation, timer endpoints/invalid values, exact retained Evidence readback, independent Python SQLite oracle and unchanged input identity. ### Three additional actual Codex CLI sessions Same model, exact-first-call prompt, restrictions and measurement method as the original record; no recovery procedure was supplied. | Trial | Outcome | REA calls | Input / cached / output tokens | Wall / CPU s | Sampled peak family RSS MiB | | --- | --- | ---: | ---: | ---: | ---: | | `normal-final` | Host discovery stopped before REA; retained as a failed control | 0 | 46752 / 30592 / 282 | 19.61 / 7.53 | 1079.1 | | `timeout-final` | 1 ms timeout → model selects 30000 → same Chinook question succeeds | 2 | 110095 / 91392 / 627 | 32.95 / 8.46 | 1035.0 | | `repeat-final` (final cleanup code) | Same independently corrected success | 2 | 159397 / 120064 / 700 | 37.06 / 8.10 | 1113.8 | **13 total recorded sessions**: ten reached REA; nine completed answers and one legitimate fixed-input blocker. Three pre-REA host-discovery failures are **not** product recovery passes. Four distinct 1 ms timeout trials all selected 30000 and completed the unchanged requested Chinook rows. The independent receipt verifier checked exact inputs, selected columns/sample/scope, final Evidence IDs/digests/rows, and all recorded failure classifications. No unchanged failed retries, shell commands, unexpected tools or surviving sampled process identities. Whole 13-session totals (including failed controls): **1292760 input tokens, 1022720 cached input tokens, 6559 output tokens; 420.43 wall s; 135.04 CPU s; 1121.2 MiB maximum sampled process-family RSS**. Sampling is every 500 ms, not a kernel peak. Preparation/tests/reviews/sampler excluded from these task costs. Fresh processes, uncontrolled OS/model caches; no cold-versus-warm claim or statistical speedup claim. No OOM observed. Natural 30-second database workloads and total native memory bounds remain unmeasured. The three pre-dispatch sessions contain the model's claim that the tool was unavailable, with no REA tool calls and no transport error captured in stderr. Whether this is model discovery, host registration/routing or the constrained prompt is **not established**. They remain unresolved agent/host-discovery failures, not product recovery passes or proven REA transport defects. A SQLite timeout message cannot repair a request that never reaches the tool. Raw receipts remain in the ignored local verification directory; no binaries, private data or credentials were committed. ### CI distinction Current-head CI/review results are recorded on #5818. On previous head `200f493d9`, test shard 4 failed only in `cliInputSchemaRequired.test.ts` for **WABT `commandTimeoutMs`**, not SQLite. The same failure was reproduced in the earlier WABT worktree `e9cdf873c58bd32e8f43b69f8a9a6d1ceb50d645`, before this SQLite change. Its test oracle checks a standalone field with `undefined`, which does not represent object-key omission for `.exactOptional()`. This is tracked in #5845 rather than changing the WABT contract or relaxing the SQLite tests in this PR. Human-review declarations are not fabricated; readiness-policy status is not represented as all green.

bsakaguchi commented

2026-10-31 00:41:26 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 22, 2026):

Both points I raised are now directly covered: the retained-cleanup provider test exercises a real owned worker timing out at 3000 ms with cleanup deliberately denied, snapshot retained and a second inspection blocked — proving the timeout stays primary without making cleanup retryable; and the six CLI/MCP verifier pairs (default, selected 60000, 1 ms, invalid 0, exact-column, oversized-refusal) confirm both entry points resolve to the same effective budget with identical error shapes. The three actionable findings (missing-budget guard, cleanup-primary, single output owner) all being fixed at their owners closes the loop cleanly. Nice work on the honest before/after record.

@XinwangTao commented on GitHub (Oct 22, 2026): Both points I raised are now directly covered: the retained-cleanup provider test exercises a real owned worker timing out at 3000 ms with cleanup deliberately denied, snapshot retained and a second inspection blocked — proving the timeout stays primary without making cleanup retryable; and the six CLI/MCP verifier pairs (default, selected 60000, 1 ms, invalid 0, exact-column, oversized-refusal) confirm both entry points resolve to the same effective budget with identical error shapes. The three actionable findings (missing-budget guard, cleanup-primary, single output owner) all being fixed at their owners closes the loop cleanly. Nice work on the honest before/after record.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#3518

No description provided.

Rows
Columns

[GH-ISSUE #5812] [Feature] Expose SQLite worker deadlines with precise timeout and limit recovery #3518

Before submitting

Area

Current limitation

Desired outcome and success criteria

Use cases / alternatives

Expected interaction with targets

Verification available so far

Verification details

Executed SQLite recovery verification (2026-10-21)

Correctness and boundaries

Actual Codex CLI behavior

Limits of this evidence / follow-up

Final validation and review fixes: b58ca531f1f978a5813d3fe5b8e054171fe0466b

Three additional actual Codex CLI sessions

CI distinction

Final validation and review fixes: `b58ca531f1f978a5813d3fe5b8e054171fe0466b`