morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #4938] [Bug] Streamed CLI JSON rejects missing fields in filtered array results #3063

New issue

Closed

opened 2026-10-31 00:35:45 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-31 00:35:45 +00:00

Owner

Originally created by @morluto on GitHub (Oct 18, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/4938

Before submitting

Searched open and closed issues for filter-output (54 results, exhausted); #912/#914 cover empty root selections and #2333 covers selected nulls, not missing entries inside mapped arrays.
Removed credentials and sensitive target data.

Area / interface / earliest failure stage

CLI / CLI / Transport or result projection.

Observed behavior

A legal JSON array result filtered by one key can fail at the streamed JSON boundary. Incur maps missing keys to undefined; REA passes that non-JSON array directly to the strict shared encoder. A controlled result [{selected:1},{other:2},{selected:null}] with --filter-output selected --format json or jsonl rejects with JSON output requires ordinary JSON values, marks output handled/failed, and writes no result for this small input. The same value passes ordinary JSON formatting as [1,null,null].

Reproduction and relevant evidence

Executed against ba98130049 using the built CLI result boundary: construct createCliResultOutput(["--format", "json", "--filter-output", "selected"], writable), then .write([{selected:1},{other:2},{selected:null}], {command:"controlled-array",duration:"0ms",format:"json"}). Repeat for jsonl. Both reject with the message above; handled=true, failed=true, bytes=0. Added CLI-framework controlled-command regressions also failed before the change. No native provider is involved.

Expected behavior

Normalize filter-produced array omissions at their owning CLI boundary, preserving array order/cardinality, explicit nulls and the original command result. JSON should return [1,null,null]; JSONL should preserve all three positions as valid records. Token counting should describe the same normalized selection. Keep strict Evidence/domain JSON validation intact.

ECMAScript SerializeJSONArray preserves otherwise omitted array entries as null. Incur Filter API documents mapped scalar selections and a filtered copy; pinned incur 0.6.1 source and execution establish the behavior used here.

Impact and scope

Confirmed CLI output integration defect with an ordinary controlled array result, both bare/full output and nested array selections. NOT_ENOUGH_INFO: an existing built-in native provider command returning this root shape was not identified or executed. No native-provider production frequency or external target reproduction is claimed. Large-prefix partial-output risk is source-derived at report creation, not measured here.

REA version / runtime / target context

Source ba98130049; Node 24.18.0, macOS arm64; incur 0.6.1. Authored in-memory JSON result; native target not applicable.

Analysis / proposed checks

Normalize only arrays owned by a nonempty Incur filter result before the streamed JSON/JSONL boundary; leave original input and domain serializer validation unchanged. Cover missing keys, inherited method selections, nested arrays, source immutability, full envelopes, token counts, and a bounded multi-chunk JSONL record. The stopped string-limit stress test must remain excluded. Context7 confirms current Filter behavior; DeepWiki's older Incur index lacks Filter and is not used as execution evidence.

Originally created by @morluto on GitHub (Oct 18, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/4938 ### Before submitting - [x] Searched open and closed issues for `filter-output` (54 results, exhausted); #912/#914 cover empty root selections and #2333 covers selected nulls, not missing entries inside mapped arrays. - [x] Removed credentials and sensitive target data. ### Area / interface / earliest failure stage CLI / CLI / Transport or result projection. ### Observed behavior A legal JSON array result filtered by one key can fail at the streamed JSON boundary. Incur maps missing keys to `undefined`; REA passes that non-JSON array directly to the strict shared encoder. A controlled result `[{selected:1},{other:2},{selected:null}]` with `--filter-output selected --format json` or `jsonl` rejects with `JSON output requires ordinary JSON values`, marks output handled/failed, and writes no result for this small input. The same value passes ordinary JSON formatting as `[1,null,null]`. ### Reproduction and relevant evidence Executed against ba98130049fa40ade66b03611ff70c7ddc011a1d using the built CLI result boundary: construct `createCliResultOutput(["--format", "json", "--filter-output", "selected"], writable)`, then `.write([{selected:1},{other:2},{selected:null}], {command:"controlled-array",duration:"0ms",format:"json"})`. Repeat for jsonl. Both reject with the message above; handled=true, failed=true, bytes=0. Added CLI-framework controlled-command regressions also failed before the change. No native provider is involved. ### Expected behavior Normalize filter-produced array omissions at their owning CLI boundary, preserving array order/cardinality, explicit nulls and the original command result. JSON should return `[1,null,null]`; JSONL should preserve all three positions as valid records. Token counting should describe the same normalized selection. Keep strict Evidence/domain JSON validation intact. [ECMAScript SerializeJSONArray](https://tc39.es/ecma262/multipage/structured-data.html#sec-serializejsonarray) preserves otherwise omitted array entries as null. [Incur Filter API](https://github.com/wevm/incur/blob/main/_autodocs/api-reference/filter.md) documents mapped scalar selections and a filtered copy; pinned incur 0.6.1 source and execution establish the behavior used here. ### Impact and scope Confirmed CLI output integration defect with an ordinary controlled array result, both bare/full output and nested array selections. NOT_ENOUGH_INFO: an existing built-in native provider command returning this root shape was not identified or executed. No native-provider production frequency or external target reproduction is claimed. Large-prefix partial-output risk is source-derived at report creation, not measured here. ### REA version / runtime / target context Source ba98130049fa40ade66b03611ff70c7ddc011a1d; Node 24.18.0, macOS arm64; incur 0.6.1. Authored in-memory JSON result; native target not applicable. ### Analysis / proposed checks Normalize only arrays owned by a nonempty Incur filter result before the streamed JSON/JSONL boundary; leave original input and domain serializer validation unchanged. Cover missing keys, inherited method selections, nested arrays, source immutability, full envelopes, token counts, and a bounded multi-chunk JSONL record. The stopped string-limit stress test must remain excluded. Context7 confirms current Filter behavior; DeepWiki's older Incur index lacks Filter and is not used as execution evidence.

bsakaguchi

2026-10-31 00:35:45 +00:00

closed this issue
added the
triage

bug
labels

bsakaguchi commented

2026-10-31 00:35:46 +00:00

Author

Owner

@morluto commented on GitHub (Oct 18, 2026):

Fixed in 2ae4e1608f on main.

The CLI result owner now normalizes only arrays produced by a nonempty Incur filter before streamed JSON/JSONL serialization. Missing/function/symbol entries retain their positions as null. Nested mapped arrays are covered; selected source values beneath objects and the original command result are untouched. The shared strict JSON/Evidence serializer remains unchanged. JSON token counting uses the same normalized selection rather than delegating a second interpretation to Incur.

Validation actually run: 66 passing tests across the new filtered-array controls, formatting recovery, CLI arguments and domain serialization; another 40 existing streamed-output checks passed with the heavy string-limit test explicitly excluded (one skip). Docs checks and five conformance tests, typecheck/lint, test discovery of all 1,571 files, and commit/push hooks passed. Controlled CLI commands cover bare/full JSON and JSONL, token counts, frozen source values, nested arrays, inherited method selections, and an 80 KiB JSONL record with deferred writes followed by an omitted selection. Read-only ownership/consumer review found no remaining blocker.

Bounded before/after comparison: six completed controlled CLI tasks, each selecting 4,096 valid authored rows and validating the entire reply. Warm medians: elapsed 0.062824 → 0.059911 s; process-family CPU 0.067913 → 0.067111 s; sampled family peak RSS 62,971,904 → 63,176,704 bytes. Complete normalized results and serialized cl100k reply estimates (19,481) matched. First process and alternating warm repeats; no cache flush, 10 ms sampling can miss brief/reparented children, elapsed includes parent reply validation. No speedup claim. Actual model/host/schema tokens, JSONL/token-count costs, and the newly successful missing-field task have no comparable successful baseline measurement.

Scope remains a reproduced CLI integration boundary with controlled ordinary JSON results; an existing built-in native-provider command yielding this exact root shape was not identified/executed. No native-provider production frequency is claimed. Context7 and pinned Incur source supported the ownership model; DeepWiki's older index lacked Filter and was not used as behavior evidence. The heavy CPU/string-limit test and profiler stayed stopped.

@morluto commented on GitHub (Oct 18, 2026): Fixed in 2ae4e1608f290c95ff7cff28c4d16a79b301d03d on main. The CLI result owner now normalizes only arrays produced by a nonempty Incur filter before streamed JSON/JSONL serialization. Missing/function/symbol entries retain their positions as null. Nested mapped arrays are covered; selected source values beneath objects and the original command result are untouched. The shared strict JSON/Evidence serializer remains unchanged. JSON token counting uses the same normalized selection rather than delegating a second interpretation to Incur. Validation actually run: 66 passing tests across the new filtered-array controls, formatting recovery, CLI arguments and domain serialization; another 40 existing streamed-output checks passed with the heavy string-limit test explicitly excluded (one skip). Docs checks and five conformance tests, typecheck/lint, test discovery of all 1,571 files, and commit/push hooks passed. Controlled CLI commands cover bare/full JSON and JSONL, token counts, frozen source values, nested arrays, inherited method selections, and an 80 KiB JSONL record with deferred writes followed by an omitted selection. Read-only ownership/consumer review found no remaining blocker. Bounded before/after comparison: six completed controlled CLI tasks, each selecting 4,096 valid authored rows and validating the entire reply. Warm medians: elapsed 0.062824 → 0.059911 s; process-family CPU 0.067913 → 0.067111 s; sampled family peak RSS 62,971,904 → 63,176,704 bytes. Complete normalized results and serialized cl100k reply estimates (19,481) matched. First process and alternating warm repeats; no cache flush, 10 ms sampling can miss brief/reparented children, elapsed includes parent reply validation. No speedup claim. Actual model/host/schema tokens, JSONL/token-count costs, and the newly successful missing-field task have no comparable successful baseline measurement. Scope remains a reproduced CLI integration boundary with controlled ordinary JSON results; an existing built-in native-provider command yielding this exact root shape was not identified/executed. No native-provider production frequency is claimed. Context7 and pinned Incur source supported the ownership model; DeepWiki's older index lacked Filter and was not used as behavior evidence. The heavy CPU/string-limit test and profiler stayed stopped.

bsakaguchi referenced this issue

2026-10-31 01:07:47 +00:00

[PR #3063] [CLOSED] fix(javascript-semantic-calls): evaluate computed method keys in the enclosing callable #5587

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#3063

No description provided.

Rows
Columns