morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[PR #4602] [MERGED] fix(firmware): separate analysis and publication deadlines #6330

New issue

Closed

opened 2026-10-31 01:13:37 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 01:13:37 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/4602
Author: @Nitin13-git
Created: 10/16/2026
Status: ✅ Merged
Merged: 10/20/2026
Merged by: @gensokyo-dev

Base: main ← Head: fix/issue-4176

📝 Commits (6)

82835ef fix(firmware): budget extraction publication separately from the extraction deadline
c3ca905 fix(firmware): reject an expired extraction before starting the publication deadline
28abfe9 Merge commit '5120eb44' into fix-4176-review
1b06627 Merge commit '28abfe950f' into fix/4176-firmware-publication-budget
e058908 fix(firmware): preserve publication budgets and timeout provenance
3296539 fix(firmware): enforce monotonic phase expiry at completion boundaries

📊 Changes

11 files changed (+726 additions, -102 deletions)

View changed files

📝 docs/firmware-analysis.md (+11 -1)
📝 src/application/firmware/FirmwareAnalysisService.ts (+3 -1)
📝 src/cli/firmwareCommands.ts (+2 -0)
📝 src/domain/firmware/firmwareAnalysis.ts (+10 -0)
➕ src/firmware/FirmwareDeadline.test.ts (+111 -0)
➕ src/firmware/FirmwareDeadline.ts (+22 -0)
📝 src/firmware/FirmwareFailures.ts (+71 -21)
📝 src/firmware/FirmwareProvider.ts (+59 -28)
📝 src/firmware/FirmwarePublication.ts (+34 -23)
📝 tests/process-global/firmwareFileLifetime.test.ts (+58 -28)
➕ tests/process-global/firmwarePublicationDeadline.test.ts (+345 -0)

📄 Description

Summary

Fixes #4176. extract_firmware used one 120-second deadline for admission, extraction, inventory and durable publication. Slow output filesystems could consume that deadline after extraction completed. Publication now starts a separate 120-second budget, selectable with MCP publication_timeout_ms / CLI --publication-timeout-ms (1–2,147,483,647 ms, the timer representation limit).

Timeouts retain the selected phase/budget and verified file progress, with advice to inspect any publication receipt before retrying or choosing a faster destination. Caller cancellation remains active through both phases and final cleanup. Absolute monotonic expiry is checked at phase handoff, before commit and after final cleanup, including when the native timer callback is still pending. Expired analysis cannot start publication; committed output and its partial observation survive late cancellation/timeout and cleanup retry. Already-aborted admission also preserves the original caller reason without launching work.

The shared firmware input contract owns the default and limits; CLI and MCP use the same provider workflow. Phase/signal/budget and monotonic expiry have one owner, and failure projection uses named context rather than positional metadata. Regular-file inventory indexing avoids intermediate filter/map arrays. This retains existing extraction/publication authority and resource bounds; extracted target code is never executed.

Executed validation

Linux x64, Node 24.18.0, npm 11.16.0:

Exact main c374c2fe7: a controlled analysis deadline firing during real file publication reproduces provider_timeout. Candidate passes. Admission regression independently reproduced the lost caller reason before its fix.
101 distinct targeted tests passed across the shared contract, real-subprocess firmware boundaries, MCP, descriptor ownership, publication rollback, queued controls, late finalization/cleanup retry and SDK-converted schema validity. Full npm run check passed after the absolute-expiry review fix; docs:check passed, including 5 consistency tests.
Public CLI on the exact reported OpenWrt 24.10.8 image: 7,012,680 bytes, SHA-256 feeab4e3142feb865450ab044a42e198af9510d263e4b0f6d1cdcc701d296003. Unblob 26.6.4 with upstream Sasquatch 4.5.1-6 publishes 1,101 regular files / 34,083,451 bytes. Every returned file's size/digest is verified, input stays immutable, temporary roots are empty. Coverage remains partial; 210 directories/links are explicitly unpublished and other unknown/depth facts remain visible. The selected 240,000-ms budget is retained in Evidence parameters.
Real public MCP: use the provider-reported LZMA root range offset 64 / length 2,627,728, depth 1; verify its one 8,756,685-byte file, exact selection, provider identity, selected budget, full Evidence export and cleanup. The real SDK also verifies that the new input advertises its numeric bounds/default and remains optional. Protocol 2026-07-28, full schema profile, 210 registered tools.
Integration with newer main e50a271c7 (local merge tree adeb229daf75f953dcfd963cd3814f57ffaf9b5e): 77 tests passed, including its revised error projection, the phase lifecycle and the actual MCP boundary. Real OpenWrt selected-range MCP also passes on that combined runtime, with byte verification, Evidence export and cleanup. This control does not change the published PR head or restart its CI.

The original Windows/Docker Desktop bind mount was unavailable. Expiry interleavings use synchronized production boundaries; real analysis/publication runs use a Linux local filesystem. Initial runs missing Sasquatch published only eight files and are excluded from the bulk-workflow comparison. A discarded MCP harness attempt incorrectly required a Ghidra-style analysis profile from firmware; the corrected check uses firmware's actual provider/engine provenance. Neither attempt is claimed as a successful representative task.

Performance and context

One CPU; 8-GiB hard / 7-GiB early process-family memory bounds; no swap. Completed public CLI tasks include verification of every output file:

Fresh process	Elapsed	Family CPU	Peak family memory
Main, first	30.237 s	27.730 s	563.8 MiB
Candidate, first	56.290 s	50.039 s	577.7 MiB
Main, repeat	30.230 s	27.402 s	566.5 MiB
Candidate, repeat	28.517 s	26.149 s	556.0 MiB
Candidate, reverse-order repeat	59.499 s	50.006 s	564.9 MiB
Main, reverse-order repeat	54.790 s	44.720 s	554.0 MiB
Candidate, final follow-up	29.925 s	26.378 s	593.2 MiB
Main, final paired control	29.522 s	26.321 s	559.6 MiB

Each run starts a new process/private home; OS caches are uncontrolled. The slower first candidate sample is retained and its phase attribution is unmeasured. Instrumented repeats show CLI-process CPU 16.811 / 16.149 s and extraction-child elapsed 13.421 / 12.378 s (main/candidate). Final paired instrumentation gives CLI-process CPU 16.138 / 15.839 s and extractor elapsed 12.737 / 13.441 s (main/candidate). These small samples establish no general speedup. No analysis-result cache is introduced.

Selected-range MCP operation time is 11.247 / 11.340 s; complete SDK task including catalog, export, startup/close is 19.761 / 19.348 s, CPU 19.283 / 18.993 s, peak 1,134.8 / 1,087.3 MiB. Final-head MCP also passes: selected operation 12.425 s, complete task 21.688 s, CPU 20.424 s, peak 1136.8 MiB. All completed controls have positive cleanup evidence and no OOM. All eight full-image runs have identical relative-path/size/SHA-256 maps.

Captured full-schema MCP JSON traffic, including tools/list, extraction and Evidence export, estimates 863,547 / 863,620 cl100k_base tokens; the named extraction input schema is 335 / 381 tokens. This counts serialized protocol schemas/output as well as messages, and is not actual model-context usage. Bulk CLI stdout is about 885k–891k tokens and is retained as a local Evidence artifact; the probe forwards only a compact summary to the agent. Model prompts/actual usage and negotiation outside the capture hooks are unmeasured. Range/depth selection keeps the real MCP extraction response to one file (about 2,070–2,118 cl100k_base text tokens) rather than the complete tree. The existing compact input-schema profile also passes the actual SDK task and retains the new numeric bounds/optionality; compact intentionally omits default annotations. A name/description/inputSchema-only catalog proxy measures 193,337 / 193,383 tokens in full mode and 44,470 / 44,489 in compact mode (main/candidate). The new field adds 46 full / 19 compact schema tokens. These catalog proxies are not measured client/model prompt usage; serialized output schemas account for much of the larger protocol total.

Author readiness

Human owner: @gensokyo-dev

The task owner explicitly delegated validation, readiness declarations and merge to this agent. These declarations are supplied under that authorization; no new separate manual human review by the original author is asserted.

I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
I declare this PR ready for initial maintainer review.

CI attribution and merge decision

Latest head 329653961: Codex completes with no findings, Greptile 5/5, all review threads resolved. Static checks fail only the existing duplication gate (3.53%, exact main 3.52789%). Source tests report 3 failed / 12,275 passed / 9 skipped; the same three URL-message assertions fail on exact main (3 failed / 10 passed in its targeted control). Controlled attribution and exact-base EVM control are recorded separately.

At merge assessment, build, docs, readiness and title checks pass; five general built-test shards are still running. The task owner explicitly authorized bypassing unrelated failing gates after qualification. The decision uses the completed local/static, 101 targeted tests, 77 latest-main integration tests, real CLI/MCP checks and completed latest-head reviews; it does not claim the remaining CI shards passed. No unrelated code or gate thresholds were changed. The previous head's built-shard failures are the same two EVM cases independently reproduced on main (2 failed / 28 passed / 2 skipped).

Post-merge follow-up: built shards 1/5 and 3/5 pass. Shard 5/5 reports exactly the two controlled EVM cases (2 failed / 2,568 passed / 25 skipped); shards 2/5 and 4/5 are cancelled by the matrix. No new firmware failure was reported. A smoke check of the actual merged tree cf5194f68 also passes 5 tests, including the public MCP firmware boundary and phase contract.

Before merge

Scope/authority and applicable lifecycle verification resolved; caller choices, source identity, partial facts and exact cleanup ownership retained.
Local implementation/validation complete; no targets, binaries, credentials, dependencies or local reports in the diff.
Completed latest-head CI results and actionable review findings assessed; pending shards and bypass authorization recorded above.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/4602 **Author:** [@Nitin13-git](https://github.com/Nitin13-git) **Created:** 10/16/2026 **Status:** ✅ Merged **Merged:** 10/20/2026 **Merged by:** [@gensokyo-dev](https://github.com/gensokyo-dev) **Base:** `main` ← **Head:** `fix/issue-4176` --- ### 📝 Commits (6) - [`82835ef`](https://github.com/morluto/rea/commit/82835ef01fecaaeafbbbce54a2f4df8ba6379d65) fix(firmware): budget extraction publication separately from the extraction deadline - [`c3ca905`](https://github.com/morluto/rea/commit/c3ca9055347030fb592590e1dccab794f12e0a56) fix(firmware): reject an expired extraction before starting the publication deadline - [`28abfe9`](https://github.com/morluto/rea/commit/28abfe950f37fdf725ce6a0cf2edcb2939f46d06) Merge commit '5120eb44' into fix-4176-review - [`1b06627`](https://github.com/morluto/rea/commit/1b06627ffd371d67c53e8fbf2dc1f7ec9e342b00) Merge commit '28abfe950f37fdf725ce6a0cf2edcb2939f46d06' into fix/4176-firmware-publication-budget - [`e058908`](https://github.com/morluto/rea/commit/e058908095b1a1bf4b9f7cb21764e7c67ad56a88) fix(firmware): preserve publication budgets and timeout provenance - [`3296539`](https://github.com/morluto/rea/commit/329653961ad0ecc016a4291509ec08e3e4d36ba7) fix(firmware): enforce monotonic phase expiry at completion boundaries ### 📊 Changes **11 files changed** (+726 additions, -102 deletions) <details> <summary>View changed files</summary> 📝 `docs/firmware-analysis.md` (+11 -1) 📝 `src/application/firmware/FirmwareAnalysisService.ts` (+3 -1) 📝 `src/cli/firmwareCommands.ts` (+2 -0) 📝 `src/domain/firmware/firmwareAnalysis.ts` (+10 -0) ➕ `src/firmware/FirmwareDeadline.test.ts` (+111 -0) ➕ `src/firmware/FirmwareDeadline.ts` (+22 -0) 📝 `src/firmware/FirmwareFailures.ts` (+71 -21) 📝 `src/firmware/FirmwareProvider.ts` (+59 -28) 📝 `src/firmware/FirmwarePublication.ts` (+34 -23) 📝 `tests/process-global/firmwareFileLifetime.test.ts` (+58 -28) ➕ `tests/process-global/firmwarePublicationDeadline.test.ts` (+345 -0) </details> ### 📄 Description ## Summary Fixes #4176. `extract_firmware` used one 120-second deadline for admission, extraction, inventory and durable publication. Slow output filesystems could consume that deadline after extraction completed. Publication now starts a separate 120-second budget, selectable with MCP `publication_timeout_ms` / CLI `--publication-timeout-ms` (1–2,147,483,647 ms, the timer representation limit). Timeouts retain the selected phase/budget and verified file progress, with advice to inspect any publication receipt before retrying or choosing a faster destination. Caller cancellation remains active through both phases and final cleanup. Absolute monotonic expiry is checked at phase handoff, before commit and after final cleanup, including when the native timer callback is still pending. Expired analysis cannot start publication; committed output and its partial observation survive late cancellation/timeout and cleanup retry. Already-aborted admission also preserves the original caller reason without launching work. The shared firmware input contract owns the default and limits; CLI and MCP use the same provider workflow. Phase/signal/budget and monotonic expiry have one owner, and failure projection uses named context rather than positional metadata. Regular-file inventory indexing avoids intermediate filter/map arrays. This retains existing extraction/publication authority and resource bounds; extracted target code is never executed. ## Executed validation Linux x64, Node 24.18.0, npm 11.16.0: - Exact main `c374c2fe7`: a controlled analysis deadline firing during real file publication reproduces `provider_timeout`. Candidate passes. Admission regression independently reproduced the lost caller reason before its fix. - **101 distinct targeted tests passed** across the shared contract, real-subprocess firmware boundaries, MCP, descriptor ownership, publication rollback, queued controls, late finalization/cleanup retry and SDK-converted schema validity. Full `npm run check` passed after the absolute-expiry review fix; `docs:check` passed, including **5 consistency tests**. - Public CLI on the exact reported [OpenWrt 24.10.8 image](https://downloads.openwrt.org/releases/24.10.8/targets/ath79/generic/sha256sums): **7,012,680 bytes**, SHA-256 `feeab4e3142feb865450ab044a42e198af9510d263e4b0f6d1cdcc701d296003`. Unblob **26.6.4** with upstream Sasquatch **4.5.1-6** publishes **1,101 regular files / 34,083,451 bytes**. Every returned file's size/digest is verified, input stays immutable, temporary roots are empty. Coverage remains **partial**; 210 directories/links are explicitly unpublished and other unknown/depth facts remain visible. The selected 240,000-ms budget is retained in Evidence parameters. - Real public MCP: use the provider-reported LZMA root range **offset 64 / length 2,627,728**, depth 1; verify its **one 8,756,685-byte file**, exact selection, provider identity, selected budget, full Evidence export and cleanup. The real SDK also verifies that the new input advertises its numeric bounds/default and remains optional. Protocol 2026-07-28, full schema profile, 210 registered tools. - Integration with newer main `e50a271c7` (local merge tree `adeb229daf75f953dcfd963cd3814f57ffaf9b5e`): **77 tests passed**, including its revised error projection, the phase lifecycle and the actual MCP boundary. Real OpenWrt selected-range MCP also passes on that combined runtime, with byte verification, Evidence export and cleanup. This control does not change the published PR head or restart its CI. The original Windows/Docker Desktop bind mount was unavailable. Expiry interleavings use synchronized production boundaries; real analysis/publication runs use a Linux local filesystem. Initial runs missing Sasquatch published only eight files and are excluded from the bulk-workflow comparison. A discarded MCP harness attempt incorrectly required a Ghidra-style analysis profile from firmware; the corrected check uses firmware's actual provider/engine provenance. Neither attempt is claimed as a successful representative task. ## Performance and context One CPU; 8-GiB hard / 7-GiB early process-family memory bounds; no swap. Completed public CLI tasks include verification of every output file: | Fresh process | Elapsed | Family CPU | Peak family memory | | --- | ---: | ---: | ---: | | Main, first | 30.237 s | 27.730 s | 563.8 MiB | | Candidate, first | 56.290 s | 50.039 s | 577.7 MiB | | Main, repeat | 30.230 s | 27.402 s | 566.5 MiB | | Candidate, repeat | 28.517 s | 26.149 s | 556.0 MiB | | Candidate, reverse-order repeat | 59.499 s | 50.006 s | 564.9 MiB | | Main, reverse-order repeat | 54.790 s | 44.720 s | 554.0 MiB | | Candidate, final follow-up | 29.925 s | 26.378 s | 593.2 MiB | | Main, final paired control | 29.522 s | 26.321 s | 559.6 MiB | Each run starts a new process/private home; OS caches are uncontrolled. The slower first candidate sample is retained and its phase attribution is unmeasured. Instrumented repeats show CLI-process CPU **16.811 / 16.149 s** and extraction-child elapsed **13.421 / 12.378 s** (main/candidate). Final paired instrumentation gives CLI-process CPU **16.138 / 15.839 s** and extractor elapsed **12.737 / 13.441 s** (main/candidate). These small samples establish no general speedup. No analysis-result cache is introduced. Selected-range MCP operation time is **11.247 / 11.340 s**; complete SDK task including catalog, export, startup/close is **19.761 / 19.348 s**, CPU **19.283 / 18.993 s**, peak **1,134.8 / 1,087.3 MiB**. Final-head MCP also passes: selected operation **12.425 s**, complete task **21.688 s**, CPU **20.424 s**, peak **1136.8 MiB**. All completed controls have positive cleanup evidence and no OOM. All eight full-image runs have identical relative-path/size/SHA-256 maps. Captured full-schema MCP JSON traffic, including `tools/list`, extraction and Evidence export, estimates **863,547 / 863,620 cl100k_base tokens**; the named extraction input schema is **335 / 381 tokens**. This counts serialized protocol schemas/output as well as messages, and is not actual model-context usage. Bulk CLI stdout is about **885k–891k tokens** and is retained as a local Evidence artifact; the probe forwards only a compact summary to the agent. Model prompts/actual usage and negotiation outside the capture hooks are unmeasured. Range/depth selection keeps the real MCP extraction response to one file (about 2,070–2,118 cl100k_base text tokens) rather than the complete tree. The existing compact input-schema profile also passes the actual SDK task and retains the new numeric bounds/optionality; compact intentionally omits default annotations. A name/description/inputSchema-only catalog proxy measures **193,337 / 193,383 tokens** in full mode and **44,470 / 44,489** in compact mode (main/candidate). The new field adds **46 full / 19 compact** schema tokens. These catalog proxies are not measured client/model prompt usage; serialized output schemas account for much of the larger protocol total. ## Author readiness Human owner: @gensokyo-dev The task owner explicitly delegated validation, readiness declarations and merge to this agent. These declarations are supplied under that authorization; no new separate manual human review by the original author is asserted. - [x] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [x] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [x] I declare this PR ready for initial maintainer review. ## CI attribution and merge decision Latest head `329653961`: Codex completes with **no findings**, Greptile **5/5**, all review threads resolved. Static checks fail only the existing duplication gate (**3.53%**, exact main **3.52789%**). Source tests report **3 failed / 12,275 passed / 9 skipped**; the same three URL-message assertions fail on exact main (**3 failed / 10 passed** in its targeted control). [Controlled attribution](https://github.com/morluto/rea/pull/4602#issuecomment-6291813433) and [exact-base EVM control](https://github.com/morluto/rea/pull/4602#issuecomment-6292315433) are recorded separately. At merge assessment, build, docs, readiness and title checks pass; five general built-test shards are still running. The task owner explicitly authorized bypassing unrelated failing gates after qualification. The decision uses the completed local/static, 101 targeted tests, 77 latest-main integration tests, real CLI/MCP checks and completed latest-head reviews; it does not claim the remaining CI shards passed. No unrelated code or gate thresholds were changed. The previous head's built-shard failures are the same two EVM cases independently reproduced on main (**2 failed / 28 passed / 2 skipped**). Post-merge follow-up: built shards **1/5 and 3/5 pass**. Shard **5/5** reports exactly the two controlled EVM cases (**2 failed / 2,568 passed / 25 skipped**); shards **2/5 and 4/5** are cancelled by the matrix. No new firmware failure was reported. A smoke check of the actual merged tree `cf5194f68` also passes **5 tests**, including the public MCP firmware boundary and phase contract. ## Before merge - [x] Scope/authority and applicable lifecycle verification resolved; caller choices, source identity, partial facts and exact cleanup ownership retained. - [x] Local implementation/validation complete; no targets, binaries, credentials, dependencies or local reports in the diff. - [x] Completed latest-head CI results and actionable review findings assessed; pending shards and bypass authorization recorded above. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-31 01:13:37 +00:00

closed this issue
added the
pull-request
label

bsakaguchi referenced this issue

2026-11-01 21:46:46 +00:00

[PR #6330] [CLOSED] Arena/7cd23a21 rea #7288

bsakaguchi changed title from ~~[PR #4602] [MERGED] fix(firmware): separate analysis and publication deadlines~~ to [PR #4602] fix(firmware): separate analysis and publication deadlines

2026-11-03 13:32:48 +00:00

bsakaguchi changed title from ~~[PR #4602] fix(firmware): separate analysis and publication deadlines~~ to [PR #4602] [MERGED] fix(firmware): separate analysis and publication deadlines

2026-11-03 14:34:33 +00:00

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#6330

No description provided.

Rows
Columns