[PR #4807] fix(cli): stream default TOON results past the string limit #6419

Open
opened 2026-10-31 01:14:19 +00:00 by bsakaguchi · 0 comments
Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/4807
Author: @AniketR10
Created: 10/17/2026
Status: 🔄 Open

Base: main ← Head: fix/cli-stream-toon-output


📝 Commits (5)

  • ab00d03 fix(cli): stream default TOON results past the string limit
  • bf0f07b fix(cli): format non-object TOON values with Incur
  • 44e66ac fix(cli): keep human terminal TOON on Incur's renderer
  • 61f0b27 Merge remote-tracking branch 'upstream/main' into fix/cli-stream-toon-output
  • 9a3fb67 fix(cli): stream oversized TOON results from capacity recovery

📊 Changes

5 files changed (+364 additions, -14 deletions)

View changed files

📝 package-lock.json (+1 -0)
📝 package.json (+1 -0)
📝 scripts/rea.mjs (+3 -1)
📝 src/cli/cliResultOutput.ts (+100 -12)
📝 tests/boundary/cli/jsonOutput.test.ts (+259 -1)

📄 Description

Summary

Refs #4806

Main answers a TOON result that cannot fit in one runtime string with a typed resource_constraint (fd6da7092). This PR adds the remaining streaming behavior for successful results: when Incur's one-string TOON formatting hits the limit, capacity recovery writes the result line by line instead of the failure. All other output, including every result that fits, is unchanged.

Problem and expected behavior

Incur formats TOON with encode(), which joins every line into one string. Above buffer.constants.MAX_STRING_LENGTH that throws, and main's recoverFormattingFailure replaces the result with a small typed failure. JSON already streams past the limit; a successful default-TOON result could not be delivered at all.

Expected: such a result is written completely as the bytes Incur's formatter would produce, while everything else keeps main's path and encoding cost.

Change and scope

  • src/cli/cliResultOutput.ts:
    • writeResult keeps main's flow. It only records the TOON document that recovery may stream. The delegation check moves after the document is built so it can record it; there is no other branch change.
    • A result is recorded only when all of these hold:
      • the format is TOON and the result is successful (no operation failure or logged failure);
      • there is no --token-count, --token-limit or --token-offset;
      • Incur is not rendering for a human (--format/--json explicit, or stdout not a terminal).
    • recoverFormattingFailure: for a recorded document, lines are encoded with encodeLines from @toon-format/toon 2.3.1, the version and function Incur's formatter uses. Up to the limit they are encoded before anything is written. If a line cannot be built by then, recovery writes main's typed failure. Otherwise it streams the lines through the existing 64 KiB bufferedJsonParts and sets recoveredByStreaming.
  • scripts/rea.mjs: after a streamed recovery the launcher keeps the command's exit status instead of forcing 1. Failures already set their status through logCliCommand.
  • Unchanged:
    • Results that fit are still encoded once, by Incur, with no added work.
    • Failures and full-output failure envelopes, terminal rendering, token windows and counts, YAML, Markdown, JSON and JSONL keep main's behavior, including the compact capacity failure.
  • @toon-format/toon becomes a direct dependency at 2.3.1, the version already resolved through incur. The lockfile changes by one line.
  • Known gap, by design: if more than one runtime string of TOON has already streamed and a later single line cannot be built (over 1 GB of TOON), output is partial. Handling that would need buffering beyond the limit.

Contribution scope: ordinary output fix; maintainer scope decision not applicable.

Contract and boundary impact

  • Semantic owner and earliest changed stage: the shared CLI result output boundary (cliResultOutput.ts) and its launcher recovery hook.
  • Contribution scope fit and linked maintainer decision: fits existing scope; not applicable.
  • CLI and MCP/tool-catalog contract: a successful TOON result above the string limit, whose lines fit, outside terminal and token modes, is written completely with exit 0 instead of the typed capacity failure. Every other CLI output is unchanged. MCP is unaffected.
  • Provider, bridge, target-format, or platform compatibility: none.
  • Evidence, artifact, provenance, or reconstruction contract: none.
  • Target selection, authority, upstream behavior, and cleanup details (if applicable): not applicable.
  • Generated metadata (docs/public/product-catalog.json), package, or installation impact: one new direct dependency, already installed transitively at the same version.

Expected effects (check known effects; use documentation-only on its own).
Unchecked items do not establish absence; leave uncertainties visible.

  • Read existing files or captured data.
  • Launch or attach to a process.
  • Inject instrumentation or run scripts/code.
  • Change process memory, registers, files, or configuration.
  • Contact network services.
  • Recover protected code or work around protections.
  • Documentation or usability only; no target interaction.
  • Not sure yet; investigation or maintainer guidance needed.

Evidence and regression coverage

  • Tests in tests/boundary/cli/jsonOutput.test.ts:
    • Results that fit:
      • for default format, --format toon and --full-output, output equals Incur's own output for a nested object, an empty object, null, "", false, 0 and a multiline string;
      • owned --filter-output projections keep main's text and trailing newline, including an empty selection.
    • Above the limit:
      • a successful tabular result whose Incur formatting throws RangeError is streamed by recovery, exactly as the launcher drives it. Byte count and SHA-256 match the expected TOON, with no typed error.
    • Main's typed capacity failure kept:
      • an oversized full-output failure envelope;
      • oversized TOON on a default-format terminal;
      • oversized TOON with --token-count;
      • a document with one TOON line longer than the limit.
    • tests/boundary/cli/formattingRecovery.test.ts (main's recovery tests) passes unchanged.
  • Base reproduction (executed): with cliResultOutput.ts from main, only the streaming test fails (1 failed, 55 passed in the file).
  • Not executed: the launcher exit-status change is source-derived; no real CLI run produced over 512 MB of TOON. No real large application was run.

Validation performed

  • npm run test:focused -- tests/boundary/cli/jsonOutput.test.ts tests/boundary/cli/formattingRecovery.test.ts tests/boundary/cli/dispatcher.test.ts tests/boundary/cli/cliSetup.test.ts: 4 files, 93 tests passed.
  • Same jsonOutput.test.ts with main's cliResultOutput.ts: 1 failed (the streaming test), 55 passed.
  • npm run check: typecheck, lint, format and metadata pass. knip reports one unused export and three unused types in untouched files (javascriptStaticAnalysisHelpers.ts, androidJniLibraryFacts.ts, androidManifestComponents.ts, objcSwiftMetadata.ts), also present on main.
  • npm run docs:check: passed on the previous revision; this revision changes no documentation or contracts.
  • Earlier full npx vitest run --project cli-boundary on this branch: 6 failures. The five in analyzeFunctionDetail, electronCli and ghidraJavaDoctor fail identically with main's cliResultOutput.ts on this host; the npx setup case depends on the npm registry.

Compatibility, safety, and release

  • Breaking changes or migration steps: none. Only a successful TOON result that currently becomes a capacity failure changes.
  • Real Hopper/Ghidra, browser, or OS coverage: not applicable; Linux only.
  • Package or release metadata impact: new direct dependency @toon-format/toon@2.3.1, already shipped through incur.
  • Security, privacy, process, or containment review: output formatting only; no new I/O or side effects.

Author readiness

Human owner: @AniketR10

  • I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
  • I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
  • I declare this PR ready for initial maintainer review.

Before merge

  • This PR addresses a concrete problem or an agreed enhancement.
  • Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
  • The PR has one focused outcome and the title follows type(scope): outcome.
  • Related issue is linked, or the reason for not linking one is stated above.
  • Tests cover changed observable behavior and meaningful failure paths.
  • Owning docs, contracts, and generated metadata are updated where needed.
  • User-visible CLI/MCP changes include representative output.
  • I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/4807 **Author:** [@AniketR10](https://github.com/AniketR10) **Created:** 10/17/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `fix/cli-stream-toon-output` --- ### 📝 Commits (5) - [`ab00d03`](https://github.com/morluto/rea/commit/ab00d03b2c4e41da326ef2e7f860d1e7fe1723ec) fix(cli): stream default TOON results past the string limit - [`bf0f07b`](https://github.com/morluto/rea/commit/bf0f07b619f5377aa6961b34e4e9fe3ff3a0e8c3) fix(cli): format non-object TOON values with Incur - [`44e66ac`](https://github.com/morluto/rea/commit/44e66acd98d9cadb1c0e29a58d95aa276d38744c) fix(cli): keep human terminal TOON on Incur's renderer - [`61f0b27`](https://github.com/morluto/rea/commit/61f0b273fa97e072d2e633999d6b7e140dde64ad) Merge remote-tracking branch 'upstream/main' into fix/cli-stream-toon-output - [`9a3fb67`](https://github.com/morluto/rea/commit/9a3fb672a8f7901387beedf5bb1ea453595d98d1) fix(cli): stream oversized TOON results from capacity recovery ### 📊 Changes **5 files changed** (+364 additions, -14 deletions) <details> <summary>View changed files</summary> 📝 `package-lock.json` (+1 -0) 📝 `package.json` (+1 -0) 📝 `scripts/rea.mjs` (+3 -1) 📝 `src/cli/cliResultOutput.ts` (+100 -12) 📝 `tests/boundary/cli/jsonOutput.test.ts` (+259 -1) </details> ### 📄 Description ## Summary Refs #4806 Main answers a TOON result that cannot fit in one runtime string with a typed `resource_constraint` (fd6da7092). This PR adds the remaining streaming behavior for successful results: when Incur's one-string TOON formatting hits the limit, capacity recovery writes the result line by line instead of the failure. All other output, including every result that fits, is unchanged. ## Problem and expected behavior Incur formats TOON with `encode()`, which joins every line into one string. Above `buffer.constants.MAX_STRING_LENGTH` that throws, and main's `recoverFormattingFailure` replaces the result with a small typed failure. JSON already streams past the limit; a successful default-TOON result could not be delivered at all. Expected: such a result is written completely as the bytes Incur's formatter would produce, while everything else keeps main's path and encoding cost. ## Change and scope - `src/cli/cliResultOutput.ts`: - `writeResult` keeps main's flow. It only records the TOON document that recovery may stream. The delegation check moves after the document is built so it can record it; there is no other branch change. - A result is recorded only when all of these hold: - the format is TOON and the result is successful (no operation failure or logged failure); - there is no `--token-count`, `--token-limit` or `--token-offset`; - Incur is not rendering for a human (`--format`/`--json` explicit, or stdout not a terminal). - `recoverFormattingFailure`: for a recorded document, lines are encoded with `encodeLines` from `@toon-format/toon` 2.3.1, the version and function Incur's formatter uses. Up to the limit they are encoded before anything is written. If a line cannot be built by then, recovery writes main's typed failure. Otherwise it streams the lines through the existing 64 KiB `bufferedJsonParts` and sets `recoveredByStreaming`. - `scripts/rea.mjs`: after a streamed recovery the launcher keeps the command's exit status instead of forcing 1. Failures already set their status through `logCliCommand`. - Unchanged: - Results that fit are still encoded once, by Incur, with no added work. - Failures and full-output failure envelopes, terminal rendering, token windows and counts, YAML, Markdown, JSON and JSONL keep main's behavior, including the compact capacity failure. - `@toon-format/toon` becomes a direct dependency at 2.3.1, the version already resolved through `incur`. The lockfile changes by one line. - Known gap, by design: if more than one runtime string of TOON has already streamed and a later single line cannot be built (over 1 GB of TOON), output is partial. Handling that would need buffering beyond the limit. Contribution scope: ordinary output fix; maintainer scope decision not applicable. ## Contract and boundary impact - Semantic owner and earliest changed stage: the shared CLI result output boundary (`cliResultOutput.ts`) and its launcher recovery hook. - Contribution scope fit and linked maintainer decision: fits existing scope; not applicable. - CLI and MCP/tool-catalog contract: a successful TOON result above the string limit, whose lines fit, outside terminal and token modes, is written completely with exit 0 instead of the typed capacity failure. Every other CLI output is unchanged. MCP is unaffected. - Provider, bridge, target-format, or platform compatibility: none. - Evidence, artifact, provenance, or reconstruction contract: none. - Target selection, authority, upstream behavior, and cleanup details (if applicable): not applicable. - Generated metadata (`docs/public/product-catalog.json`), package, or installation impact: one new direct dependency, already installed transitively at the same version. Expected effects (check known effects; use documentation-only on its own). Unchecked items do not establish absence; leave uncertainties visible. - [ ] Read existing files or captured data. - [ ] Launch or attach to a process. - [ ] Inject instrumentation or run scripts/code. - [ ] Change process memory, registers, files, or configuration. - [ ] Contact network services. - [ ] Recover protected code or work around protections. - [ ] Documentation or usability only; no target interaction. - [ ] Not sure yet; investigation or maintainer guidance needed. ## Evidence and regression coverage - Tests in `tests/boundary/cli/jsonOutput.test.ts`: - Results that fit: - for default format, `--format toon` and `--full-output`, output equals Incur's own output for a nested object, an empty object, `null`, `""`, `false`, `0` and a multiline string; - owned `--filter-output` projections keep main's text and trailing newline, including an empty selection. - Above the limit: - a successful tabular result whose Incur formatting throws `RangeError` is streamed by recovery, exactly as the launcher drives it. Byte count and SHA-256 match the expected TOON, with no typed error. - Main's typed capacity failure kept: - an oversized full-output failure envelope; - oversized TOON on a default-format terminal; - oversized TOON with `--token-count`; - a document with one TOON line longer than the limit. - `tests/boundary/cli/formattingRecovery.test.ts` (main's recovery tests) passes unchanged. - Base reproduction (executed): with `cliResultOutput.ts` from main, only the streaming test fails (1 failed, 55 passed in the file). - Not executed: the launcher exit-status change is source-derived; no real CLI run produced over 512 MB of TOON. No real large application was run. ## Validation performed - `npm run test:focused -- tests/boundary/cli/jsonOutput.test.ts tests/boundary/cli/formattingRecovery.test.ts tests/boundary/cli/dispatcher.test.ts tests/boundary/cli/cliSetup.test.ts`: 4 files, 93 tests passed. - Same `jsonOutput.test.ts` with main's `cliResultOutput.ts`: 1 failed (the streaming test), 55 passed. - `npm run check`: typecheck, lint, format and metadata pass. `knip` reports one unused export and three unused types in untouched files (`javascriptStaticAnalysisHelpers.ts`, `androidJniLibraryFacts.ts`, `androidManifestComponents.ts`, `objcSwiftMetadata.ts`), also present on main. - `npm run docs:check`: passed on the previous revision; this revision changes no documentation or contracts. - Earlier full `npx vitest run --project cli-boundary` on this branch: 6 failures. The five in `analyzeFunctionDetail`, `electronCli` and `ghidraJavaDoctor` fail identically with main's `cliResultOutput.ts` on this host; the npx setup case depends on the npm registry. ## Compatibility, safety, and release - Breaking changes or migration steps: none. Only a successful TOON result that currently becomes a capacity failure changes. - Real Hopper/Ghidra, browser, or OS coverage: not applicable; Linux only. - Package or release metadata impact: new direct dependency `@toon-format/toon@2.3.1`, already shipped through `incur`. - Security, privacy, process, or containment review: output formatting only; no new I/O or side effects. ## Author readiness Human owner: @AniketR10 - [ ] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [ ] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [ ] I declare this PR ready for initial maintainer review. ## Before merge - [ ] This PR addresses a concrete problem or an agreed enhancement. - [ ] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff. - [ ] The PR has one focused outcome and the title follows `type(scope): outcome`. - [ ] Related issue is linked, or the reason for not linking one is stated above. - [ ] Tests cover changed observable behavior and meaningful failure paths. - [ ] Owning docs, contracts, and generated metadata are updated where needed. - [ ] User-visible CLI/MCP changes include representative output. - [ ] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
morluto/rea#6419
No description provided.