morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #5448] [Bug] CLI --filter-output removes failure diagnostics and prints an empty object #3308

New issue

Closed

opened 2026-10-31 00:39:00 +00:00 by bsakaguchi · 3 comments

bsakaguchi commented

2026-10-31 00:39:00 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 20, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/5448

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

CLI

Observed behavior

trace-application-feature ./missing-evidence.json --json --filter-output foo exits 1 but prints only {} and nothing on stderr. Without the filter, the same missing-file invocation returns an invalid_request diagnostic with a message, remediation, and ENOENT details. Selecting a successful-result field therefore removes the available explanation when the operation fails.

Reproduction

From the built source checkout below, leave missing-evidence.json absent and run this independently replayed reproduction. No analysis provider or external service is required.

probe=$(mktemp -d)
rea_script="$PWD/scripts/rea.mjs"
mkdir -p "$probe/home" "$probe/config" "$probe/data" "$probe/cache" "$probe/state"
run_probe() {
  (
    cd "$probe" || exit 2
    env -i PATH="$PATH" HOME="$probe/home" \
      XDG_CONFIG_HOME="$probe/config" XDG_DATA_HOME="$probe/data" \
      XDG_CACHE_HOME="$probe/cache" XDG_STATE_HOME="$probe/state" \
      TMPDIR="$probe" REA_LOG_LEVEL=silent \
      node "$rea_script" trace-application-feature ./missing-evidence.json --json "$@"
  )
  probe_status=$?
  printf 'exit=%s\n' "$probe_status"
}
run_probe
run_probe --filter-output foo
run_probe --filter-output code
run_probe --filter-output foo --full-output

Each variant was independently observed once at the reported baseline. The missing file is read and fails with ENOENT before application analysis starts.

Expected behavior

Retain an identifying, actionable failure diagnostic when a data selector does not match the error. Keep exit status 1 and the existing full-output failure representation. Successful data filtering, including legitimate scalar selections, should remain available.

docs/cli.md, "Output and exit status", says status 1 means the operation could not complete, with structured output identifying the failure when available. The unfiltered control establishes that this diagnostic is available. Source-derived supporting evidence: Incur 0.6.1, src/Cli.ts, filters successful output.data but bypasses that filter for output.error.

Impact and scope

A script selecting successful-result fields loses the failure identity and recovery instructions. The process status remains correctly nonzero. Omitting --filter-output or adding --full-output returned identifying diagnostics in the replay and provides a practical workaround. No target mutation, data loss, or security impact was demonstrated. This report establishes ordinary failed JSON output for this command; other commands and formats were not reproduced.

REA version

6.7.0, source commit 074768901f3cdc8205b61773fb1faf6f6c7b3ca8.

Runtime and platform

Node.js v26.12.0; macOS 27.0.1 arm64; Incur 0.6.1. No Hopper or Ghidra provider was used.

Target and artifact context

Absent local JSON input file, ./missing-evidence.json. No binary target or Evidence fixture is required.

Interface

CLI

Earliest failure stage

Transport or result projection

Relevant output or evidence

All four commands exit 1 with empty stderr:

Unfiltered:
  code: "invalid_request"
  category: "invalid_input"
  message: "Analysis input is invalid. Check the arguments and try again."
  retryable: true
  remediation.action: "Correct the listed arguments and retry."
  input_reason: "read-failed"
  details include ENOENT for ./missing-evidence.json

--filter-output foo:
{}

--filter-output code:
"invalid_request"

--filter-output foo --full-output:
  ok: false
  error contains the classified diagnostic and input details
  meta contains command metadata

The first and last entries summarize captured JSON; the middle two are exact stdout documents. A separate missing-argument control, trace-application-feature --json --filter-output foo, still returned VALIDATION_ERROR and its message with exit 1.

Typed outcome or recovery expected

An available failure code and explanatory diagnostic should survive successful-data selection. Preserve the current missing-input refusal and the full-output error envelope.

Analysis or suspected cause

Source-derived: src/cli/cliResultOutput.ts:222-233 obtains the canonical failure payload only when fullOutput is enabled, then filters the original value. Lines 299-321 preserve full-output failures but replace an undefined ordinary selection with {}. src/cli/applicationCommands.ts:207-215 returns the file-input failure, and src/cliLogging.ts sets exit status 1 before formatting. This explains the observed diagnostic loss without changing operation status.

Additional context

Prepared with AI assistance and independently reproduced through the public CLI before submission.

PR #2333 preserves selected JSON null values. Its merge commit is already an ancestor of this baseline; it does not bypass filtering for ordinary failure documents. A fresh fetch found no subsequent changes to the cited contract, failure-path, or boundary-test files through 052956af3416fa01d93837e54fdec3ad2739f6d1. Related issue #912 concerns successful commands producing no JSON, and #2273 concerns contradictory full-output success envelopes; this report concerns ordinary failure diagnostics disappearing under filtering.

Originally created by @coygeek on GitHub (Oct 20, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/5448 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area CLI ### Observed behavior `trace-application-feature ./missing-evidence.json --json --filter-output foo` exits 1 but prints only `{}` and nothing on stderr. Without the filter, the same missing-file invocation returns an `invalid_request` diagnostic with a message, remediation, and ENOENT details. Selecting a successful-result field therefore removes the available explanation when the operation fails. ### Reproduction From the built source checkout below, leave `missing-evidence.json` absent and run this independently replayed reproduction. No analysis provider or external service is required. ```sh probe=$(mktemp -d) rea_script="$PWD/scripts/rea.mjs" mkdir -p "$probe/home" "$probe/config" "$probe/data" "$probe/cache" "$probe/state" run_probe() { ( cd "$probe" || exit 2 env -i PATH="$PATH" HOME="$probe/home" \ XDG_CONFIG_HOME="$probe/config" XDG_DATA_HOME="$probe/data" \ XDG_CACHE_HOME="$probe/cache" XDG_STATE_HOME="$probe/state" \ TMPDIR="$probe" REA_LOG_LEVEL=silent \ node "$rea_script" trace-application-feature ./missing-evidence.json --json "$@" ) probe_status=$? printf 'exit=%s\n' "$probe_status" } run_probe run_probe --filter-output foo run_probe --filter-output code run_probe --filter-output foo --full-output ``` Each variant was independently observed once at the reported baseline. The missing file is read and fails with ENOENT before application analysis starts. ### Expected behavior Retain an identifying, actionable failure diagnostic when a data selector does not match the error. Keep exit status 1 and the existing full-output failure representation. Successful data filtering, including legitimate scalar selections, should remain available. `docs/cli.md`, "Output and exit status", says status 1 means the operation could not complete, with structured output identifying the failure when available. The unfiltered control establishes that this diagnostic is available. Source-derived supporting evidence: Incur 0.6.1, `src/Cli.ts`, filters successful `output.data` but bypasses that filter for `output.error`. ### Impact and scope A script selecting successful-result fields loses the failure identity and recovery instructions. The process status remains correctly nonzero. Omitting `--filter-output` or adding `--full-output` returned identifying diagnostics in the replay and provides a practical workaround. No target mutation, data loss, or security impact was demonstrated. This report establishes ordinary failed JSON output for this command; other commands and formats were not reproduced. ### REA version 6.7.0, source commit `074768901f3cdc8205b61773fb1faf6f6c7b3ca8`. ### Runtime and platform Node.js v26.12.0; macOS 27.0.1 arm64; Incur 0.6.1. No Hopper or Ghidra provider was used. ### Target and artifact context Absent local JSON input file, `./missing-evidence.json`. No binary target or Evidence fixture is required. ### Interface CLI ### Earliest failure stage Transport or result projection ### Relevant output or evidence All four commands exit 1 with empty stderr: ```text Unfiltered: code: "invalid_request" category: "invalid_input" message: "Analysis input is invalid. Check the arguments and try again." retryable: true remediation.action: "Correct the listed arguments and retry." input_reason: "read-failed" details include ENOENT for ./missing-evidence.json --filter-output foo: {} --filter-output code: "invalid_request" --filter-output foo --full-output: ok: false error contains the classified diagnostic and input details meta contains command metadata ``` The first and last entries summarize captured JSON; the middle two are exact stdout documents. A separate missing-argument control, `trace-application-feature --json --filter-output foo`, still returned `VALIDATION_ERROR` and its message with exit 1. ### Typed outcome or recovery expected An available failure code and explanatory diagnostic should survive successful-data selection. Preserve the current missing-input refusal and the full-output error envelope. ### Analysis or suspected cause Source-derived: `src/cli/cliResultOutput.ts:222-233` obtains the canonical failure payload only when `fullOutput` is enabled, then filters the original value. Lines 299-321 preserve full-output failures but replace an undefined ordinary selection with `{}`. `src/cli/applicationCommands.ts:207-215` returns the file-input failure, and `src/cliLogging.ts` sets exit status 1 before formatting. This explains the observed diagnostic loss without changing operation status. ### Additional context Prepared with AI assistance and independently reproduced through the public CLI before submission. PR #2333 preserves selected JSON null values. Its merge commit is already an ancestor of this baseline; it does not bypass filtering for ordinary failure documents. A fresh fetch found no subsequent changes to the cited contract, failure-path, or boundary-test files through `052956af3416fa01d93837e54fdec3ad2739f6d1`. Related issue #912 concerns successful commands producing no JSON, and #2273 concerns contradictory full-output success envelopes; this report concerns ordinary failure diagnostics disappearing under filtering.

bsakaguchi closed this issue

2026-10-31 00:39:00 +00:00

bsakaguchi commented

2026-10-31 00:39:01 +00:00

Author

Owner

@coygeek commented on GitHub (Oct 20, 2026):

Additional reproduced cases (same producing guard)

Replayed through the public CLI at 074768901f3cdc8205b61773fb1faf6f6c7b3ca8 (REA 6.7.0), with isolated configuration and the input files absent. Commands below are arguments after node scripts/rea.mjs. Every failing case exited 1 with 0 stderr bytes.

Default TOON can erase stdout completely as well:

Command	stdout bytes
`evidence-import ./nothere.json --filter-output missing`	0
`inspect-artifact ./nothere.bin --filter-output missing`	0
`trace-application-feature ./nothere.json --filter-output foo`	0

Additional command/format coverage:

Command	Observed stdout
`evidence-import ./nothere.json --json --filter-output missing`	Exactly `{}\n` (3 bytes)
`evidence-import ./nothere.json --json --filter-output code`	Exactly `"invalid_request"\n` (18 bytes), a bare JSON scalar
`inspect-artifact ./nothere.bin --json --filter-output message`	Only a JSON message string; no code, category, retryable flag, or remediation object
`evidence-import ./nothere.json --format yaml --filter-output code`	Exactly `invalid_request\n` (16 bytes)
`evidence-import ./nothere.json --format md --filter-output code`	Exactly `invalid_request\n` (16 bytes)

Unfiltered evidence-import ./nothere.json --json retained code: invalid_request, category: invalid_input, the missing-file message, retryable: true, and remediation. Adding --full-output --filter-output code retained the complete diagnostic under error with ok: false. Unfiltered inspect-artifact ./nothere.bin --json retained target_unavailable, unavailable, retryable: false, its message, remediation, and details. Both workarounds therefore remain available. The successful control capabilities --json --filter-output open exited 0 and printed exactly false\n.

These cases share the CLI result-filter boundary in src/cli/cliResultOutput.ts: the classified failure is retained only for full output, while ordinary failure results remain subject to data filtering; bare non-JSON rendering delegates that result to Incur. They extend this issue's command and format coverage.

@coygeek commented on GitHub (Oct 20, 2026): Additional reproduced cases (same producing guard) Replayed through the public CLI at `074768901f3cdc8205b61773fb1faf6f6c7b3ca8` (REA 6.7.0), with isolated configuration and the input files absent. Commands below are arguments after `node scripts/rea.mjs`. Every failing case exited 1 with **0 stderr bytes**. Default TOON can erase stdout completely as well: | Command | stdout bytes | | --- | ---: | | `evidence-import ./nothere.json --filter-output missing` | 0 | | `inspect-artifact ./nothere.bin --filter-output missing` | 0 | | `trace-application-feature ./nothere.json --filter-output foo` | 0 | Additional command/format coverage: | Command | Observed stdout | | --- | --- | | `evidence-import ./nothere.json --json --filter-output missing` | Exactly `{}\n` (3 bytes) | | `evidence-import ./nothere.json --json --filter-output code` | Exactly `"invalid_request"\n` (18 bytes), a bare JSON scalar | | `inspect-artifact ./nothere.bin --json --filter-output message` | Only a JSON message string; no code, category, retryable flag, or remediation object | | `evidence-import ./nothere.json --format yaml --filter-output code` | Exactly `invalid_request\n` (16 bytes) | | `evidence-import ./nothere.json --format md --filter-output code` | Exactly `invalid_request\n` (16 bytes) | Unfiltered `evidence-import ./nothere.json --json` retained `code: invalid_request`, `category: invalid_input`, the missing-file message, `retryable: true`, and remediation. Adding `--full-output --filter-output code` retained the complete diagnostic under `error` with `ok: false`. Unfiltered `inspect-artifact ./nothere.bin --json` retained `target_unavailable`, `unavailable`, `retryable: false`, its message, remediation, and details. Both workarounds therefore remain available. The successful control `capabilities --json --filter-output open` exited 0 and printed exactly `false\n`. These cases share the CLI result-filter boundary in `src/cli/cliResultOutput.ts`: the classified failure is retained only for full output, while ordinary failure results remain subject to data filtering; bare non-JSON rendering delegates that result to Incur. They extend this issue's command and format coverage.

bsakaguchi commented

2026-10-31 00:39:01 +00:00

Author

Owner

@matbrgz commented on GitHub (Oct 20, 2026):

Cross-referencing with sibling CLI diagnostic output formatting defect #5110 ([Bug] Exhausted CLI failure token windows display a reversed range). Both isolate how failure diagnostics and token ranges are formatted and preserved under CLI filters (--filter-output).

Consolidated under Cluster AQ (CLI Output Filtering & Diagnostics Suppression) in tracker #4330.

@matbrgz commented on GitHub (Oct 20, 2026): Cross-referencing with sibling CLI diagnostic output formatting defect #5110 (`[Bug] Exhausted CLI failure token windows display a reversed range`). Both isolate how failure diagnostics and token ranges are formatted and preserved under CLI filters (`--filter-output`). Consolidated under **Cluster AQ (CLI Output Filtering & Diagnostics Suppression)** in tracker #4330.

bsakaguchi commented

2026-10-31 00:39:01 +00:00

Author

Owner

@CassieFox23 commented on GitHub (Oct 20, 2026):

TL;DR: I could not reproduce this on main on Linux. The diagnostic still shows with --filter-output.

Checked on main b3aadaf47 (Node 24.18.0). Run trace-application-feature on a missing file with --json --filter-output foo. It exits 1. It prints the full invalid_request diagnostic, not {}. --filter-output code gives the same. Default TOON output also keeps it. The report was made at 074768901. Commit 4714c7eaa3 (preserve typed failures through output filters) landed after that. It looks like the fix. But I did not run 074768901. So that link is unconfirmed.

Not run

Running 074768901 to confirm 4714c7eaa3 is the cause.
The inspect-artifact variant and a relative ./missing-evidence.json path.

Thanks, coygeek, for the clear report.

Commands

node scripts/rea.mjs trace-application-feature /tmp/probe/missing-evidence.json --json --filter-output foo

Exit 1, stderr 0 bytes, stdout 753 bytes: {"code":"invalid_request", ... "remediation":{"action":"Read the declared input schema ..."}, "details":{... "message":"The JSON input file could not be read (ENOENT): ...", "input_reason":"read-failed"}}.

--filter-output code: exit 1, same 753 bytes.
--filter-output foo --full-output: exit 1, 839 bytes, {"ok":false,"error":{...},"meta":...}.
Without --json, --filter-output foo: exit 1, 741 bytes of TOON diagnostic.
evidence-import <missing> --filter-output missing: exit 1, 342 bytes of diagnostic (with and without --json).

Test setup

Box: Nobara 44 (Fedora 44 base), Linux 7.2.11
CPU: AMD Ryzen 9 5900X, 12 cores / 24 threads
RAM: 32 GB DDR4-3600
Node: 24.18.0
Head / base: b3aadaf47 / 074768901
Other: another job shared the box during the run

🤖 Generated with Claude Code (Sonnet 5.5, verified by Opus 5.5) 🦊

@CassieFox23 commented on GitHub (Oct 20, 2026): **TL;DR:** I could not reproduce this on main on Linux. The diagnostic still shows with `--filter-output`. Checked on main b3aadaf47 (Node 24.18.0). Run `trace-application-feature` on a missing file with `--json --filter-output foo`. It exits 1. It prints the full `invalid_request` diagnostic, not `{}`. `--filter-output code` gives the same. Default TOON output also keeps it. The report was made at 074768901. Commit 4714c7eaa3 (`preserve typed failures through output filters`) landed after that. It looks like the fix. But I did not run 074768901. So that link is unconfirmed. ### Not run - Running 074768901 to confirm 4714c7eaa3 is the cause. - The `inspect-artifact` variant and a relative `./missing-evidence.json` path. Thanks, coygeek, for the clear report. <details> <summary>Commands</summary> ```sh node scripts/rea.mjs trace-application-feature /tmp/probe/missing-evidence.json --json --filter-output foo ``` Exit 1, stderr 0 bytes, stdout 753 bytes: `{"code":"invalid_request", ... "remediation":{"action":"Read the declared input schema ..."}, "details":{... "message":"The JSON input file could not be read (ENOENT): ...", "input_reason":"read-failed"}}`. - `--filter-output code`: exit 1, same 753 bytes. - `--filter-output foo --full-output`: exit 1, 839 bytes, `{"ok":false,"error":{...},"meta":...}`. - Without `--json`, `--filter-output foo`: exit 1, 741 bytes of TOON diagnostic. - `evidence-import <missing> --filter-output missing`: exit 1, 342 bytes of diagnostic (with and without `--json`). </details> <details> <summary>Test setup</summary> - Box: Nobara 44 (Fedora 44 base), Linux 7.2.11 - CPU: AMD Ryzen 9 5900X, 12 cores / 24 threads - RAM: 32 GB DDR4-3600 - Node: 24.18.0 - Head / base: b3aadaf47 / 074768901 - Other: another job shared the box during the run </details> 🤖 Generated with Claude Code (Sonnet 5.5, verified by Opus 5.5) 🦊

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#3308

No description provided.

Rows
Columns