morluto/rea

Fork

You've already forked rea

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

Code Issues 1.3k Projects Packages Wiki Activity

[GH-ISSUE #113] Return reliable nonzero CLI exit status for structured operation failures #71

New issue

Closed

opened 2026-10-15 18:20:48 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-15 18:20:48 +00:00

Owner

Copy link

Originally created by @morluto on GitHub (Jul 13, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/113

Summary

Ensure every REA CLI command returns a nonzero process exit status when the requested operation fails, even when the failure is represented as valid structured JSON/TOON/YAML output.

Observed behavior in REA 1.3.0

Commands such as inventory-artifact and investigate-versions can return shell exit status 0 while output contains an operational failure, for example:

ArtifactOperationError / integrity_mismatch;
InvestigationWorkspaceError / disabled.

The envelope may itself serialize successfully, but serialization success is not operation success.

User problem

Shell scripts, CI, coding agents, and command runners conventionally use the process status as the primary success signal. Exit 0 with an embedded error can:

be mistaken for valid Evidence;
allow pipelines to continue;
overwrite prior reports;
produce false-positive verification;
require every caller to parse REA-specific envelopes before checking success.

Required semantics

Separate three concepts:

command parsed and output encoded;
requested operation succeeded;
process successfully wrote the selected output format.

Suggested exit classes:

0: requested operation succeeded;
nonzero stable code: invalid input or policy/permission failure;
nonzero stable code: analysis/provider/integrity failure;
nonzero stable code: cancellation or timeout;
nonzero stable code: output/serialization failure;
conventional signal-derived status where appropriate.

Exact numeric codes may remain simple, but they must be documented and stable enough for automation.

Structured output

On expected failure:

still emit one valid object in the selected format;
include the typed error contract from #100;
write machine-readable output to stdout when requested;
route incidental diagnostics/progress to stderr;
exit nonzero after output flush succeeds.

Broken pipes and serialization failures must not be reported as successful analysis.

Composition and pipeline requirements

Piped commands must preserve failure through pipefail-compatible use.
--full-output and filtered output must not change exit status.
A resumed investigate-versions run that fails before a checkpoint must return nonzero.
Partial/truncated success must have explicit documented semantics; truncation must not be conflated with execution failure.
Setup outcomes such as needs_human should have a deliberate documented status.

Acceptance criteria

Every CLI command has an explicit success/failure exit-status contract.
Expected operation failures exit nonzero in every output format.
Structured failure output remains valid and machine-readable.
Exit status is independent of --format, --full-output, filtering, and token limits.
Cancellation, timeout, permission failure, integrity mismatch, provider failure, and serialization failure are distinguishable.
Success with bounded/truncated evidence has documented status semantics.
Tests execute packaged CLI commands and assert both output and process status.
Shell pipeline examples use pipefail where relevant.
Documentation includes an exit-status table.
MCP behavior remains unaffected; this issue concerns terminal process semantics.

Non-goals

Printing raw stack traces.
Removing structured errors.
Assigning a unique numeric exit code to every internal exception.
Treating warnings or explicit unknown evidence as process failure when the requested operation completed truthfully.

Relationships

#100 defines structured error bodies.
#111 governs diagnostic detail preservation.
Verification tooling should treat this issue as required to prevent false green automation.

Originally created by @morluto on GitHub (Jul 13, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/113 ## Summary Ensure every REA CLI command returns a nonzero process exit status when the requested operation fails, even when the failure is represented as valid structured JSON/TOON/YAML output. ## Observed behavior in REA 1.3.0 Commands such as inventory-artifact and investigate-versions can return shell exit status 0 while output contains an operational failure, for example: - ArtifactOperationError / integrity_mismatch; - InvestigationWorkspaceError / disabled. The envelope may itself serialize successfully, but serialization success is not operation success. ## User problem Shell scripts, CI, coding agents, and command runners conventionally use the process status as the primary success signal. Exit 0 with an embedded error can: - be mistaken for valid Evidence; - allow pipelines to continue; - overwrite prior reports; - produce false-positive verification; - require every caller to parse REA-specific envelopes before checking success. ## Required semantics Separate three concepts: 1. command parsed and output encoded; 2. requested operation succeeded; 3. process successfully wrote the selected output format. Suggested exit classes: - 0: requested operation succeeded; - nonzero stable code: invalid input or policy/permission failure; - nonzero stable code: analysis/provider/integrity failure; - nonzero stable code: cancellation or timeout; - nonzero stable code: output/serialization failure; - conventional signal-derived status where appropriate. Exact numeric codes may remain simple, but they must be documented and stable enough for automation. ## Structured output On expected failure: - still emit one valid object in the selected format; - include the typed error contract from #100; - write machine-readable output to stdout when requested; - route incidental diagnostics/progress to stderr; - exit nonzero after output flush succeeds. Broken pipes and serialization failures must not be reported as successful analysis. ## Composition and pipeline requirements - Piped commands must preserve failure through pipefail-compatible use. - --full-output and filtered output must not change exit status. - A resumed investigate-versions run that fails before a checkpoint must return nonzero. - Partial/truncated success must have explicit documented semantics; truncation must not be conflated with execution failure. - Setup outcomes such as needs_human should have a deliberate documented status. ## Acceptance criteria - [ ] Every CLI command has an explicit success/failure exit-status contract. - [ ] Expected operation failures exit nonzero in every output format. - [ ] Structured failure output remains valid and machine-readable. - [ ] Exit status is independent of --format, --full-output, filtering, and token limits. - [ ] Cancellation, timeout, permission failure, integrity mismatch, provider failure, and serialization failure are distinguishable. - [ ] Success with bounded/truncated evidence has documented status semantics. - [ ] Tests execute packaged CLI commands and assert both output and process status. - [ ] Shell pipeline examples use pipefail where relevant. - [ ] Documentation includes an exit-status table. - [ ] MCP behavior remains unaffected; this issue concerns terminal process semantics. ## Non-goals - Printing raw stack traces. - Removing structured errors. - Assigning a unique numeric exit code to every internal exception. - Treating warnings or explicit unknown evidence as process failure when the requested operation completed truthfully. ## Relationships - #100 defines structured error bodies. - #111 governs diagnostic detail preservation. - Verification tooling should treat this issue as required to prevent false green automation.

bsakaguchi

2026-10-15 18:20:48 +00:00

closed this issue
added the
area: cli

enhancement
labels

bsakaguchi referenced this issue

2026-10-15 18:22:45 +00:00

[GH-ISSUE #385] Design and implement bounded Objective-C runtime observation #233

bsakaguchi referenced this issue

2026-10-15 18:24:20 +00:00

[GH-ISSUE #968] [Feature] Modular native runtime instrumentation through a Frida adapter #367

bsakaguchi referenced this issue

2026-10-15 18:24:22 +00:00

[GH-ISSUE #969] [Feature] Modular binary diagnostics and debugger adapters using pwntools and pwndbg #370

bsakaguchi referenced this issue

2026-10-15 18:28:29 +00:00

[GH-ISSUE #2383] [Tracker] Audit analysis and debugging coverage for the TIOBE top 100 languages (October 2026) #794

~~bsakaguchi referenced this issue 2026-10-16 21:05:48 +00:00~~

[GH-ISSUE #3796] [Tracker] Open feature requests by area (October 2026) #1471

bsakaguchi referenced this issue

2026-10-16 21:05:50 +00:00

[GH-ISSUE #3796] [Tracker] Open feature requests by area (October 2026) #1471

bsakaguchi referenced this issue

2026-10-16 21:05:51 +00:00

[GH-ISSUE #3796] [Tracker] Open feature requests by area (October 2026) #1471

bsakaguchi added

enhancement

and removed

enhancement

labels

2026-10-31 00:28:42 +00:00

No Branch/Tag specified

main

codex/procedure-reference-stacked

release-please--branches--main--components--rea-agents

release-please--branches--main--components--rea-agents--release-notes

codex/kuna-independent-retry

codex/byte-search-overlay-coverage

codex/open-failure-target-receipt

codex/kuna-xref-evidence

codex/ghidra-ambiguous-procedure-recovery

codex/function-source-bundle

codex/trace-function-integration-prototype

codex/snapshot-source-closure

codex/recorded-evidence-array-pages

codex/evidence-recovery-redesign

codex/trace-feature-address-scope

codex/call-discovery-scope

codex/ghidra-literal-candidate-evidence

codex/decompile-directory-independent-retry

codex/retry-provenance-base

codex/value-flow-capability-experiment

codex/ghidra-containing-instruction

cursor/readme-discord-button-ef9f

dependabot/github_actions/actions/setup-node-7.1.0

dependabot/github_actions/actions/upload-artifact-7.0.2

dependabot/npm_and_yarn/npm-minor-and-patch-d8d97ce6df

codex/kuna-1989

fix/8559-native-ui-fixture-ownership

fix/8555-go-verifier-mcp-ownership

codex/provider-capability-recovery

fix/8642-rizin-target-stdin

codex/ghidra-memory-effect-scope

codex/ghidra-annotation-entry-selector

codex/ghidra-readonly-recovery

codex/ghidra-regex-verifier-recovery

codex/evidence-discovery-view-guidance

fix/8545-bulk-append-overflow

fix/8428-navigation-refusal-alternatives

codex/address-context-recovery

codex/matching-member-limitations

codex/unknown-scope-contract

codex/evidence-view-guidance

feat/8373-native-trace-views

codex/ghidra-snapshot-verifier-binding

codex/elf-export-visibility

codex/evidence-view-recovery

codex/native-reference-cli-help

codex/provider-candidate-observation-scope

codex/kuna-session-verification

codex/ghidra-switch-lifetime-guidance

codex/pseudocode-paging-discovery

codex/clarify-evidence-view-support

codex/kuna-xrefs-selector-recovery

codex/context-capacity-recovery

codex/evidence-closure-large-links

codex/same-function-evidence-identity

codex/ghidra-verifier-inspected-jdk

codex/audit-guided-workflows

codex/crash-prompt-retained-dossiers

codex/analysis-view-error-recovery

codex/kuna-verifier-target-selection

codex/ghidra-selected-project-call-coverage

codex/docs-native-analysis-scope

codex/ghidra-java-socket-boundary

codex/ghidra-retained-snapshot-verifier

codex/ghidra-search-inventory-facts

fix/8284-zero-argument-parameter-bindings

fix/8198-ui-action-handler-offset

fix/8257-elf-entry-label-identity

fix/8304-deferred-output-schema-projection

fix/8277-call-path-computed-target-uncertainty

fix/pr-8089-conflicts

fix/ci-static-duplicate-export

dependabot/npm_and_yarn/lint-staged-17.6.0

dependabot/github_actions/actions/download-artifact-8.0.2

dependabot/npm_and_yarn/write-file-atomic-8.0.0

dependabot/npm_and_yarn/canonicalize-5.1.0

dependabot/github_actions/actions/cache/restore-6.1.0

dependabot/npm_and_yarn/toon-format/toon-4.4.0

dependabot/github_actions/actions/deploy-pages-5.0.1

codex/review-fixes-pr-2403

codex/review-fixes-pr-2400

codex/process-terminal-width

codex/test-brittleness-cleanup

verify/7167-browser-connect-macos

fix/7167-browser-connect

feat/native-matching-allowance-setting

feat/browser-scenario-dry-run

test/managed-caller-parameters-20261024

codex/parser-fortythird-boundaries

fix/6926-pinvoke-platform-decorations-20261023

fix/6889-har-body-encoding-20261023

fix/6888-sqlite-schema-budget-20261023

fix/dependency-security-gates-6708

cursor/evidence-overwrite-rebase-b4ac

feat/4057-verify-native-exports

fix/windows-acl-observer-overhead-20261022

cursor/erlang-beam-restack-4476

cursor/pe-certificate-review-bf32

fix/6116-javascript-application-fs-errors

fix/6061-unmapped-runtime-script-unknown

feat/browser-scenario-summary

fix/pe-certificates

codex/verify-3407

cursor/rebase-5699

cursor/rebase-5352

cursor/rebase-5600

cursor/fix-cli-schema-omission-10a8

codex/audit-hopper-shutdown-reasons-20261021

fix/demangle-swift-omitted-record

fix/nib-boolean-value-types

cursor/cancellation-partial-observation-a5ac

fix/ib-object-limit-reasons

fix/electron-fuse-limit-advice

fix/kuna-doctor-dangling-cache

fix/fixed-budget-refusal-advice

fix/js-ipc-listener-dedup

fix/js-return-member-provenance

fix/dmg-integrity-retry-advice

fix/ida-doctor-readiness

fix/kuna-version-probe-output-drain

fix/legacy-swift-runtime-name-matching

fix/dylib-lazy-load-commands

fix/browser-scenario-startup-cancel

fix/swift-nested-punycode-classification

fix/compare-unselected-scopes

fix/objc-method-budget-coverage

fix/dmg-mounted-root-observations

audit/ci-tests-20261019-round2

release/6.7.0

fix/wabt-cleanup-retry-resolution

fix/electron-fuse-elf-pe-architecture

fix/native-plist-descriptor-ownership

fix/kuna-caller-cleanup-ownership

experiment/atomic-codemode-20261016

review/first-contributor-3776

fix/call-resolution-full-read-cycles

fix/member-mutation-selection-repeat

experiment/atomic-re-tools-20261015

perf/member-mutation-lazy-origins

perf/static-network-mutation-index

fix/hopper-startup-lifecycle

audit/3050-windows-proof-20261015

feat/windows-window-capture-2183

audit/3049-windows-proof-20261015

audit/3735-macos-fixtures-20261015

fix/batch-decompile-null-pseudocode

fix/kuna-inventory-honors-jobs

experiment/browser-codemode-tool-design-20261015

fix/hopper-clean-runtime-argv

codex/review-2049-27f1b367

review-fixes/pr-2787-routing

fix/3360-scoped-this-emitters

fix/3367-replaced-global-effects

fix/3838-compact-any-value-items

test/hopper-download-probe-20261015

codex/windows-runtime-creation-red-green-20261015

codex/windows-readonly-root-red-green-20261015

codex/windows-root-share-red-green-20261015

codex/exp-1685-win-mcp-startup

codex/bug-sweep-20261015

investigate/2190-windows-package-phases

investigate/3543-macos-cli-deadline

perf/3281-reverse-tools-feedback-quality

fix/3406-hopper-doctor-fixture

fix/3341-wrapper-require-test-context

fix/3301-owned-uninstall-fixtures

fix/3164-compiler-input-ownership

fix/pcap-cleanup-stdout-newline-20261014

perf/interface-builder-summary

codex/review-fixes-pr-1977

fix/ghidra-effective-jvm-platform

fix/docs-turkish-rust-link-20261013

fix/main-maintenance-ci-20261010

fix/recover-binary-session-process-effects

codex/review-fixes-pr-2457

codex/ci-required-cancellation-gate

fix/main-format-baseline-20261013

fix/setup-preserve-entry-fields

cursor/restore-main-checks-1ffc

fix/ghidra-bridge-hashmap-import

codex/shared-ci-jadx-json-default-20261013

codex/shared-ci-ghidra-test-groups-20261013

codex/shared-ci-java-fixture-format-20261013

codex/fix-intrinsic-invocation-boundaries

test/release-preflight-reusable-jobs

test/js-parameter-provenance-expectation

fix/linux-hopper-verification-current

test/2419-ghidra-elf-probe-size

followup/pr-1544-native-ida

codex/fold-pr-2230-contribution

docs/readme-responsible-use

fix/issue-2064

fix/issue-2050

fix/issue-2053

fix/issue-1968

fix/issue-1945

experiment/codemode-bridge-20261011

fix/tool-effects-negative-wording-1904-20261012

fix/1842-electron-interop-provenance

feat/recover-javascript-multi-input

refactor/website-i18n-prep

fix/nativeaot-pe-virtual-extent

fix/browser-verifier-ci-hang

fix/pe-resources-input-selection

docs/beginner-friendly-guides

docs/beginner-setup-guide

fix/javascript-analysis-worker-1563

fix/1649-ghidra-duplicate-names

fix/javascript-export-keys-integration-1369

test/maintenance-boundary-contracts-20261011

review/javascript-export-keys-1369

review/process-live-progress-1483

test/javascript-1495-growth-ratio

fix/ci-escape-timing

docs/normalized-terminal-geometry-1450

release/6.3.0

archive/release-6.3.0-pr1381

release/6.2.0

fix/electron-preload-dynamic-override

fix/javascript-self-referential-bundler-edges

release/6.1.0

cursor/grok-client-cherry-pick-0b56

feat/signature-posture

feat/dyld-shared-cache

feat/artifact-integrity-policy

feat/pkg-inventory

release/6.0.0

chore/sync-release-5.0.0

release/5.0.0

cursor/cloud-agent-node-787e

fix/javascript-xhr-open-endpoints

review/pr-711-realpath-fallback-wording

cleanup/replay-confinement

codex/test-suite-performance

codex/issue-foundations

codex/address-open-issues-2026-07-20

agent/javascript-vite-rollup-esbuild-adapters

agent/process-cleanup

agent/triage-integration

agent/unknown-commands-registry

feat/mcp-identity-metadata

morlvps/issue_cleanup

codex/installer-redesign

morl/roadmap-investigation-core

morl/roadmap-investigation-docs

release-please--branches--main--components--rea

rea-agents-6.8.0

rea-agents-6.7.0

rea-agents-6.6.0

experiment/atomic-codemode-wait-runtime-20261016

rea-agents-6.5.0

rea-agents-6.4.0

rea-agents-6.3.0

rea-agents-6.2.0

rea-agents-6.1.0

rea-agents-6.0.0

rea-agents-5.0.0

rea-agents-4.1.0

rea-agents-4.0.1

rea-agents-4.0.0

rea-agents-3.2.1

rea-agents-3.2.0

rea-agents-3.1.0

rea-agents-3.0.0

rea-agents-2.7.0

rea-agents-2.6.0

rea-agents-2.5.1

rea-agents-2.5.0

rea-agents-2.4.0

rea-agents-2.3.0

rea-agents-2.2.0

rea-agents-2.1.0

rea-agents-2.0.0

rea-agents-1.7.0

rea-agents-1.6.0

rea-agents-1.5.0

rea-agents-1.4.0

rea-agents-1.3.0

rea-agents-1.2.0

rea-agents-1.1.0

rea-agents-1.0.0

rea-agents-0.5.0

rea-agents-0.4.0

rea-agents-0.3.0

rea-0.2.1

rea-0.2.0

Labels

Clear labels

Mirrored from GitHub Pull Request

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#71

Reference in a new issue

Repository

morluto/rea

Title

Body

No description provided.

Delete branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?

Rows
Columns