mirror of
https://github.com/morluto/rea.git
synced 2026-11-05 06:45:22 +00:00
[GH-ISSUE #3888] [Bug] Firmware extraction byte budget also caps the mandatory producer report #1507
Closed
opened 2026-10-16 21:06:20 +00:00 by bsakaguchi
·
3 comments
No Branch/Tag specified
main
codex/procedure-reference-stacked
release-please--branches--main--components--rea-agents
release-please--branches--main--components--rea-agents--release-notes
codex/kuna-independent-retry
codex/byte-search-overlay-coverage
codex/open-failure-target-receipt
codex/kuna-xref-evidence
codex/ghidra-ambiguous-procedure-recovery
codex/function-source-bundle
codex/trace-function-integration-prototype
codex/snapshot-source-closure
codex/recorded-evidence-array-pages
codex/evidence-recovery-redesign
codex/trace-feature-address-scope
codex/call-discovery-scope
codex/ghidra-literal-candidate-evidence
codex/decompile-directory-independent-retry
codex/retry-provenance-base
codex/value-flow-capability-experiment
codex/ghidra-containing-instruction
cursor/readme-discord-button-ef9f
dependabot/github_actions/actions/setup-node-7.1.0
dependabot/github_actions/actions/upload-artifact-7.0.2
dependabot/npm_and_yarn/npm-minor-and-patch-d8d97ce6df
codex/kuna-1989
fix/8559-native-ui-fixture-ownership
fix/8555-go-verifier-mcp-ownership
codex/provider-capability-recovery
fix/8642-rizin-target-stdin
codex/ghidra-memory-effect-scope
codex/ghidra-annotation-entry-selector
codex/ghidra-readonly-recovery
codex/ghidra-regex-verifier-recovery
codex/evidence-discovery-view-guidance
fix/8545-bulk-append-overflow
fix/8428-navigation-refusal-alternatives
codex/address-context-recovery
codex/matching-member-limitations
codex/unknown-scope-contract
codex/evidence-view-guidance
feat/8373-native-trace-views
codex/ghidra-snapshot-verifier-binding
codex/elf-export-visibility
codex/evidence-view-recovery
codex/native-reference-cli-help
codex/provider-candidate-observation-scope
codex/kuna-session-verification
codex/ghidra-switch-lifetime-guidance
codex/pseudocode-paging-discovery
codex/clarify-evidence-view-support
codex/kuna-xrefs-selector-recovery
codex/context-capacity-recovery
codex/evidence-closure-large-links
codex/same-function-evidence-identity
codex/ghidra-verifier-inspected-jdk
codex/audit-guided-workflows
codex/crash-prompt-retained-dossiers
codex/analysis-view-error-recovery
codex/kuna-verifier-target-selection
codex/ghidra-selected-project-call-coverage
codex/docs-native-analysis-scope
codex/ghidra-java-socket-boundary
codex/ghidra-retained-snapshot-verifier
codex/ghidra-search-inventory-facts
fix/8284-zero-argument-parameter-bindings
fix/8198-ui-action-handler-offset
fix/8257-elf-entry-label-identity
fix/8304-deferred-output-schema-projection
fix/8277-call-path-computed-target-uncertainty
fix/pr-8089-conflicts
fix/ci-static-duplicate-export
dependabot/npm_and_yarn/lint-staged-17.6.0
dependabot/github_actions/actions/download-artifact-8.0.2
dependabot/npm_and_yarn/write-file-atomic-8.0.0
dependabot/npm_and_yarn/canonicalize-5.1.0
dependabot/github_actions/actions/cache/restore-6.1.0
dependabot/npm_and_yarn/toon-format/toon-4.4.0
dependabot/github_actions/actions/deploy-pages-5.0.1
codex/review-fixes-pr-2403
codex/review-fixes-pr-2400
codex/process-terminal-width
codex/test-brittleness-cleanup
verify/7167-browser-connect-macos
fix/7167-browser-connect
feat/native-matching-allowance-setting
feat/browser-scenario-dry-run
test/managed-caller-parameters-20261024
codex/parser-fortythird-boundaries
fix/6926-pinvoke-platform-decorations-20261023
fix/6889-har-body-encoding-20261023
fix/6888-sqlite-schema-budget-20261023
fix/dependency-security-gates-6708
cursor/evidence-overwrite-rebase-b4ac
feat/4057-verify-native-exports
fix/windows-acl-observer-overhead-20261022
cursor/erlang-beam-restack-4476
cursor/pe-certificate-review-bf32
fix/6116-javascript-application-fs-errors
fix/6061-unmapped-runtime-script-unknown
feat/browser-scenario-summary
fix/pe-certificates
codex/verify-3407
cursor/rebase-5699
cursor/rebase-5352
cursor/rebase-5600
cursor/fix-cli-schema-omission-10a8
codex/audit-hopper-shutdown-reasons-20261021
fix/demangle-swift-omitted-record
fix/nib-boolean-value-types
cursor/cancellation-partial-observation-a5ac
fix/ib-object-limit-reasons
fix/electron-fuse-limit-advice
fix/kuna-doctor-dangling-cache
fix/fixed-budget-refusal-advice
fix/js-ipc-listener-dedup
fix/js-return-member-provenance
fix/dmg-integrity-retry-advice
fix/ida-doctor-readiness
fix/kuna-version-probe-output-drain
fix/legacy-swift-runtime-name-matching
fix/dylib-lazy-load-commands
fix/browser-scenario-startup-cancel
fix/swift-nested-punycode-classification
fix/compare-unselected-scopes
fix/objc-method-budget-coverage
fix/dmg-mounted-root-observations
audit/ci-tests-20261019-round2
release/6.7.0
fix/wabt-cleanup-retry-resolution
fix/electron-fuse-elf-pe-architecture
fix/native-plist-descriptor-ownership
fix/kuna-caller-cleanup-ownership
experiment/atomic-codemode-20261016
review/first-contributor-3776
fix/call-resolution-full-read-cycles
fix/member-mutation-selection-repeat
experiment/atomic-re-tools-20261015
perf/member-mutation-lazy-origins
perf/static-network-mutation-index
fix/hopper-startup-lifecycle
audit/3050-windows-proof-20261015
feat/windows-window-capture-2183
audit/3049-windows-proof-20261015
audit/3735-macos-fixtures-20261015
fix/batch-decompile-null-pseudocode
fix/kuna-inventory-honors-jobs
experiment/browser-codemode-tool-design-20261015
fix/hopper-clean-runtime-argv
codex/review-2049-27f1b367
review-fixes/pr-2787-routing
fix/3360-scoped-this-emitters
fix/3367-replaced-global-effects
fix/3838-compact-any-value-items
test/hopper-download-probe-20261015
codex/windows-runtime-creation-red-green-20261015
codex/windows-readonly-root-red-green-20261015
codex/windows-root-share-red-green-20261015
codex/exp-1685-win-mcp-startup
codex/bug-sweep-20261015
investigate/2190-windows-package-phases
investigate/3543-macos-cli-deadline
perf/3281-reverse-tools-feedback-quality
fix/3406-hopper-doctor-fixture
fix/3341-wrapper-require-test-context
fix/3301-owned-uninstall-fixtures
fix/3164-compiler-input-ownership
fix/pcap-cleanup-stdout-newline-20261014
perf/interface-builder-summary
codex/review-fixes-pr-1977
fix/ghidra-effective-jvm-platform
fix/docs-turkish-rust-link-20261013
fix/main-maintenance-ci-20261010
fix/recover-binary-session-process-effects
codex/review-fixes-pr-2457
codex/ci-required-cancellation-gate
fix/main-format-baseline-20261013
fix/setup-preserve-entry-fields
cursor/restore-main-checks-1ffc
fix/ghidra-bridge-hashmap-import
codex/shared-ci-jadx-json-default-20261013
codex/shared-ci-ghidra-test-groups-20261013
codex/shared-ci-java-fixture-format-20261013
codex/fix-intrinsic-invocation-boundaries
test/release-preflight-reusable-jobs
test/js-parameter-provenance-expectation
fix/linux-hopper-verification-current
test/2419-ghidra-elf-probe-size
followup/pr-1544-native-ida
codex/fold-pr-2230-contribution
docs/readme-responsible-use
fix/issue-2064
fix/issue-2050
fix/issue-2053
fix/issue-1968
fix/issue-1945
experiment/codemode-bridge-20261011
fix/tool-effects-negative-wording-1904-20261012
fix/1842-electron-interop-provenance
feat/recover-javascript-multi-input
refactor/website-i18n-prep
fix/nativeaot-pe-virtual-extent
fix/browser-verifier-ci-hang
fix/pe-resources-input-selection
docs/beginner-friendly-guides
docs/beginner-setup-guide
fix/javascript-analysis-worker-1563
fix/1649-ghidra-duplicate-names
fix/javascript-export-keys-integration-1369
test/maintenance-boundary-contracts-20261011
review/javascript-export-keys-1369
review/process-live-progress-1483
test/javascript-1495-growth-ratio
fix/ci-escape-timing
docs/normalized-terminal-geometry-1450
release/6.3.0
archive/release-6.3.0-pr1381
release/6.2.0
fix/electron-preload-dynamic-override
fix/javascript-self-referential-bundler-edges
release/6.1.0
cursor/grok-client-cherry-pick-0b56
feat/signature-posture
feat/dyld-shared-cache
feat/artifact-integrity-policy
feat/pkg-inventory
release/6.0.0
chore/sync-release-5.0.0
release/5.0.0
cursor/cloud-agent-node-787e
fix/javascript-xhr-open-endpoints
review/pr-711-realpath-fallback-wording
cleanup/replay-confinement
codex/test-suite-performance
codex/issue-foundations
codex/address-open-issues-2026-07-20
agent/javascript-vite-rollup-esbuild-adapters
agent/process-cleanup
agent/triage-integration
agent/unknown-commands-registry
feat/mcp-identity-metadata
morlvps/issue_cleanup
codex/installer-redesign
morl/roadmap-investigation-core
morl/roadmap-investigation-docs
release-please--branches--main--components--rea
rea-agents-6.8.0
rea-agents-6.7.0
rea-agents-6.6.0
experiment/atomic-codemode-wait-runtime-20261016
rea-agents-6.5.0
rea-agents-6.4.0
rea-agents-6.3.0
rea-agents-6.2.0
rea-agents-6.1.0
rea-agents-6.0.0
rea-agents-5.0.0
rea-agents-4.1.0
rea-agents-4.0.1
rea-agents-4.0.0
rea-agents-3.2.1
rea-agents-3.2.0
rea-agents-3.1.0
rea-agents-3.0.0
rea-agents-2.7.0
rea-agents-2.6.0
rea-agents-2.5.1
rea-agents-2.5.0
rea-agents-2.4.0
rea-agents-2.3.0
rea-agents-2.2.0
rea-agents-2.1.0
rea-agents-2.0.0
rea-agents-1.7.0
rea-agents-1.6.0
rea-agents-1.5.0
rea-agents-1.4.0
rea-agents-1.3.0
rea-agents-1.2.0
rea-agents-1.1.0
rea-agents-1.0.0
rea-agents-0.5.0
rea-agents-0.4.0
rea-agents-0.3.0
rea-0.2.1
rea-0.2.0
Labels
Clear labels
area: bridge
area: bridge
area: build
area: build
area: build
area: cli
area: domain
area: evidence
area: mcp
area: native
area: providers
area: runtime
area: server
area: setup
area: tests
bug
design
documentation
enhancement
enhancement
enhancement
experiment
help wanted
i18n
invalid
P0: critical
P1: high
P2: medium
P2: medium
P2: medium
P3: low
performance
provider-request
pull-request
Mirrored from GitHub Pull Request
question
roadmap
triage
No labels
area: bridge
area: bridge
area: build
area: build
area: build
area: cli
area: domain
area: evidence
area: mcp
area: native
area: providers
area: runtime
area: server
area: setup
area: tests
bug
design
documentation
enhancement
enhancement
enhancement
experiment
help wanted
i18n
invalid
P0: critical
P1: high
P2: medium
P2: medium
P2: medium
P3: low
performance
provider-request
pull-request
question
roadmap
triage
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".
No due date set.
Dependencies
No dependencies set.
Reference
morluto/rea#1507
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @coygeek on GitHub (Oct 15, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3888
Before submitting
Area
Evidence, artifacts, or reconstruction
Observed behavior
extract-firmware --max-output-bytesis forwarded as the process-wideprlimit --fsizeceiling, so it limits not only staged payload files but also the mandatoryreport.jsonthe producer must write; with the accepted value 1, no valid report can fit even when no extracted file exceeds the budget.The extraction passes its staging byte budget into
runFirmwareCommand, which emits it as the process-wide--fsizeceiling at src/firmware/FirmwareCommand.ts:125, and util-linux applies RLIMIT_FSIZE to every regular file the producer writes, including the mandatory report and log outside the staging tree.With the accepted value 1 no valid required report can fit: an empty JSON array needs two bytes, and an accepted report requires a root task, a StatReport, and a SHA-256 HashReport per src/firmware/FirmwareReports.ts:186-200 and lines 317-322.
After the command src/firmware/FirmwareProvider.ts:356 unconditionally reads that report and src/firmware/FirmwareFiles.ts:150-172 requires complete valid JSON, so failure precedes result publication regardless of whether extracted payload files fit the selected budget, and the 8 MiB report-reader allowance never helps.
Reproduction
A local probe imported
dist/domain/firmware/firmwareAnalysis.jsanddist/firmware/FirmwareCommand.jsfrom a built source checkout, parsed extraction input withmax_output_bytes: 1, then calledrunFirmwareCommandwith that parsed budget and an injected launcher that printed the file-size argument and threw a sentinel before spawning any process:The public CLI was also run locally with
--max-output-bytes 1; it passed option validation and returned the expected macOScapability_unavailablerefusal, which does not establish Linux extraction behavior.Expected behavior
src/domain/firmware/firmwareAnalysis.ts:40 admits
max_output_bytesfrom 1 and describes it as the staging file-byte budget, and docs/firmware-analysis.md:161 gives producer JSON reports a separate 8 MiB budget while lines 164-166 describe the configurable staging budget.Selecting a small staging budget should constrain extracted files rather than make the mandatory report impossible to write, even when no extracted file exceeds that budget.
src/firmware/FirmwareRelease.ts:31 and src/firmware/FirmwareFiles.ts:154 separately allow reports up to 8 MiB, establishing the intended separation of the two budgets.
Impact and scope
low extraction budgets can prevent analysis solely because mandatory metadata or logs exceed the payload allowance, and the one-byte accepted boundary cannot produce any valid report even for an analysis yielding no published payload; impact is bounded to selected budgets.
Workaround: unverified. raising the budget to the default is a plausible workaround but was not verified with a real producer and gives up the requested tighter staging limit.
Reporter's impact estimate: P2 (supported).
REA version
6.5.0, built from source at main commit
034342ba28(latest npm release is 6.5.0)Runtime and platform
Node.js v26.10.0; macOS 27.0.1 arm64; This is a source-only finding for the public Linux defect; schema acceptance and the production launcher-argument generation were executed locally, but no real Unblob/binwalk producer was spawned. End-to-end runtime reproduction with a real producer was not run.
Target and artifact context
Firmware image file (Linux Unblob extraction)
Interface
CLI
Earliest failure stage
Capture or observation
Relevant output or evidence
--fsize=1emitted from the accepted budget (output below); static chain at src/firmware/FirmwareProvider.ts:350-353 (passesmax_output_bytesasoutputBudget), src/firmware/FirmwareCommand.ts:125 (emits--fsize), and src/firmware/FirmwareReports.ts:186-200 (minimum valid report cannot fit one byte)--fsizelauncher argument were observed. End-to-end runtime reproduction with a real producer was not run.Typed outcome or recovery expected
A small --max-output-bytes staging budget should constrain extracted payload files only; the mandatory report.json and log should be written under their own report-size allowance so a non-over-budget extraction still publishes a valid result.
Restoration check:
Failing:
extract-firmware --max-output-bytes 1admits the request and emits--fsize=1, which caps the mandatoryreport.jsonso no valid report can be written even for an analysis yielding no published payload.Passing: the same small staging budget constrains only extracted payload files, while the mandatory report and log are written under their own report-size allowance, so an extraction that produces no over-budget payload still completes and publishes a valid report.
Analysis or suspected cause
FirmwareProvider.ts:350-353 forwards request.input.max_output_bytes as outputBudget; FirmwareCommand.ts:125 emits it as the process-wide --fsize ceiling, which util-linux applies to every regular file the producer writes, including report.json and unblob.log outside the staging tree. A minimum valid report (root task + StatReport + SHA-256 HashReport, FirmwareReports.ts:186-200/317-322) cannot fit one byte, and FirmwareProvider.ts:356 unconditionally reads it, so the 8 MiB report allowance (FirmwareRelease.ts:31, FirmwareFiles.ts:154) never applies.
Additional context
The preliminary version probe uses the independent report-size default and has no extraction budget, so selecting 1 is not intentionally rejected during admission; larger small budgets also depend on report and log size rather than only staging bytes.
Related issues: open #3756 concerns false
coverage: "complete"when nothing was extracted, not failure to write metadata under a staging limit; open #3757 and #3758 concern process prerequisites and doctor coverage; closed #1501 concerns teardown after completed extraction. No matching report found.This report came from AI-assisted source review with local reproduction where stated above; the evidence labels distinguish observed runs from source-only reasoning.
@XinwangTao commented on GitHub (Oct 15, 2026):
Good catch — using the staging budget as a process-wide RLIMIT_FSIZE is a layering violation: the byte cap is meant to bound extracted payload files, but --fsize also clamps the mandatory report.json/log that the producer must write to complete the run. So a tiny budget doesn't just trim outputs, it makes the whole extraction fail before results can be published, and the 8 MiB report-reader allowance becomes dead code. The budget should apply only to files inside the staging tree (or be applied per-file after the run), and the mandatory report should be exempt from the ceiling. A regression test with max_output_bytes: 1 asserting that a valid report still gets produced would lock this down.
@XinwangTao commented on GitHub (Oct 15, 2026):
Good catch — using the staging budget as a process-wide RLIMIT_FSIZE is a layering violation: the byte cap is meant to bound extracted payload files, but --fsize also clamps the mandatory report.json/log that the producer must write to complete the run. So a tiny budget doesn't just trim outputs, it makes the whole extraction fail before results can be published, and the 8 MiB report-reader allowance becomes dead code. The budget should apply only to files inside the staging tree (or be applied per-file after the run), and the mandatory report should be exempt from the ceiling. A regression test with max_output_bytes: 1 asserting that a valid report still gets produced would lock this down.
@coygeek commented on GitHub (Oct 24, 2026):
Checked the reported fix on
mainatd87e73f2db.Process-wide file size allowance is now max(selected payload budget, report budget, diagnostic budget), capped by the inherited file ceiling; separate output inventory still enforces aggregate selected payload bytes/entries. The command regression explicitly tests selected=1 produces 8 MiB fsize headroom, large payload and inherited-ceiling controls. This fixes the mandatory report impossible-at-one-byte launch policy.
Implementation: FirmwareCommand.ts, FirmwareCommand.ts.
The relevant regression checks in FirmwareCommand.test.ts passed in the focused test run on this revision. This closure is based on the current source and those checks; provider fixtures do not establish a real-engine run. Closing as completed on main.