morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #5532] [Bug] Negative and fractional token-window values produce inconsistent pages #3369

New issue

Closed

opened 2026-10-31 00:39:37 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-31 00:39:37 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 20, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/5532

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

CLI

Observed behavior

inspect-plist accepts negative and fractional --token-limit/--token-offset values, then reports inconsistent token windows. With --token-limit -5, a successful inspection returns almost the entire body and a negative endpoint instead of a useful bounded page. With an absent target and --full-output --token-limit -5, the failure envelope includes nextOffset: -5.

These are public CLI observations, independently reproduced. Fractional limits and offsets are accepted too; their markers contain fractional coordinates. The documentation does not establish a specific invalid-number policy, so the issue is inconsistent accepted windows and continuation metadata, rather than a requirement for one particular rejection code.

Reproduction

From the built source checkout below, run this Python 3 probe. It uses a generated XML plist, isolated HOME/XDG directories, and no database or external API. This is the relevant subset of the executed probe; the omitted past-end cases are already tracked in #5110.

python3 - <<'PY'
import os
from pathlib import Path
import plistlib
import re
import subprocess
import tempfile

entry = Path('scripts/rea.mjs').resolve()
with tempfile.TemporaryDirectory() as temporary:
    work = Path(temporary)
    env = os.environ.copy()
    for key in ('HOME', 'XDG_CONFIG_HOME', 'XDG_DATA_HOME', 'XDG_CACHE_HOME', 'XDG_STATE_HOME'):
        directory = work / key.lower()
        directory.mkdir()
        env[key] = str(directory)
    fixture = work / 'small.plist'
    fixture.write_bytes(plistlib.dumps({'Name': 'small', 'Items': [1, 2, 3]}))
    cases = [
        ('full', fixture, []),
        ('positive limit', fixture, ['--token-limit', '5']),
        ('negative limit', fixture, ['--token-limit', '-5']),
        ('fractional limit', fixture, ['--token-limit', '2.5']),
        ('zero limit', fixture, ['--token-limit', '0']),
        ('negative offset', fixture, ['--token-offset', '-100', '--token-limit', '20']),
        ('failure negative limit', work / 'absent.plist', ['--full-output', '--token-limit', '-5']),
    ]
    for label, target, flags in cases:
        result = subprocess.run(['node', str(entry), 'inspect-plist', str(target), '--format', 'toon', *flags], cwd=work, env=env, capture_output=True, text=True)
        markers = re.findall(r'\[truncated: showing tokens [^\]]+\]', result.stdout)
        continuation = re.findall(r'nextOffset: [^\n]+', result.stdout)
        print(label, 'exit=', result.returncode, 'bytes=', len(result.stdout.encode()), 'markers=', markers, 'continuation=', continuation, 'stderr=', repr(result.stderr))
PY

The independent verifier also executed --token-offset 2.5 --token-limit 5 on the same fixture and observed showing tokens 2.5–7.5. Each listed case reproduced in the verifier's probe and its exact-script rerun; the submitter reran the full embedded probe and confirmed the negative-limit, fractional-limit, negative-offset, and negative-continuation results again.

Expected behavior

CLI help advertises limiting output to n tokens and skipping the first n tokens. docs/tool-design.md:46-69 requires preserving meaningful caller choices and reporting truncation; docs/cli.md:437-443 documents the valued output controls.

Accepted windows should report coordinates consistent with the returned slice and provide a continuation that can advance through the result. Values that cannot satisfy that contract should be rejected clearly or receive coherent documented normalization. Zero is a valid empty-window control. No particular error code or normalization implementation is prescribed.

Impact and scope

Negative limits can defeat a caller's intended output budget; negative failure-envelope continuation offsets can send a paging caller backward. Negative offsets select text relative to the end while displaying negative absolute coordinates. Observed scope is local TOON CLI output at the versions below, including successful results and full failure envelopes. No persistent target modification or security boundary crossing was observed. Other formats and production frequency were not measured.

A demonstrated workaround is to use non-negative integer values: limit 5/offset 0 and limit 0 returned coherent windows. Callers can validate computed bounds before invoking REA.

REA version

6.7.0; source commit 074768901f3cdc8205b61773fb1faf6f6c7b3ca8; locked Incur 0.6.1 and tokenx 1.3.0. The latest published REA release is rea-agents-6.7.0.

Runtime and platform

Node.js v26.12.0; macOS arm64 (OS minor version was not recorded by the independent verifier). No optional native analysis provider was involved.

Target and artifact context

Generated XML plist with Name=small and Items=[1,2,3]; an absent local plist for the full failure-envelope case.

Interface

CLI

Earliest failure stage

Request parsing or validation

Relevant output or evidence

Independent verifier's public CLI output, summarized without fixture paths:

full: exit 0; 2014 stdout bytes; 518 estimated tokens
--token-limit 5: exit 0; showing tokens 0–5 of 518
--token-limit -5: exit 0; 2042 stdout bytes; showing tokens 0–-5 of 518
--token-limit 2.5: exit 0; showing tokens 0–2.5 of 518
--token-limit 0: exit 0; showing tokens 0–0 of 518
--token-offset -100 --token-limit 20: exit 0; showing tokens -100–-80 of 518
--token-offset 2.5 --token-limit 5: exit 0; showing tokens 2.5–7.5 of 518
absent target, --full-output --token-limit -5:
  exit 1; showing tokens 0–-5 of 282; nextOffset: -5

Stderr was empty for these cases. Token totals vary because the generated fixture path is embedded in Evidence. The negative-limit body is almost the full body; the appended marker makes total stdout larger than unwindowed stdout. The submitter's repeat produced totals of 509/270 and again nextOffset: -5.

The verifier also confirmed that missing and nonnumeric limit values fail; finite negative and fractional values pass parsing. The failure-envelope case retains the operation's exit status 1.

Typed outcome or recovery expected

A clear input failure or a coherent normalized token window with usable continuation metadata. Retain the underlying operation-failure status when the target is absent.

Analysis or suspected cause

Source-derived attribution: global finite-number parsing lives in the bundled Incur 0.6.1, node_modules/incur/dist/Cli.js:1993-2006. REA mirrors parsing in src/cliOutput.ts:131-140; scripts/rea.mjs and that module pre-validate some output flags. Ordinary successful TOON rendering is delegated to Incur by src/cli/cliResultOutput.ts:222-245. Incur's Cli.js:1047-1063 reports raw indices while token slicing normalizes negative indices. An independently executed standalone Incur CLI reproduced the negative-limit result too.

REA separately repeats the range and continuation arithmetic for full failure envelopes in src/cli/cliResultOutput.ts:353-373, producing the negative nextOffset. The maintainers can choose where to enforce coherent bounds. No issue is being filed against Incur as part of this report.

Additional context

Prepared with AI assistance and independently reproduced before submission. Upstream main was fetched and checked through c374c2fe7290b914d73d9e27af38b498753ec593; changes since the tested baseline do not alter the relevant numeric parsing or window arithmetic.

Related #5110 and its PR #5119 cover exhausted positive offsets on failure envelopes. This report concerns negative/fractional inputs and negative continuation offsets; it does not request another fix for that already reported past-end case. Character corruption with otherwise valid integer windows is a separate defect.

NOT_ENOUGH_INFO: first affected release and last known good revision were not established.

Originally created by @coygeek on GitHub (Oct 20, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/5532 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area CLI ### Observed behavior `inspect-plist` accepts negative and fractional `--token-limit`/`--token-offset` values, then reports inconsistent token windows. With `--token-limit -5`, a successful inspection returns almost the entire body and a negative endpoint instead of a useful bounded page. With an absent target and `--full-output --token-limit -5`, the failure envelope includes `nextOffset: -5`. These are public CLI observations, independently reproduced. Fractional limits and offsets are accepted too; their markers contain fractional coordinates. The documentation does not establish a specific invalid-number policy, so the issue is inconsistent accepted windows and continuation metadata, rather than a requirement for one particular rejection code. ### Reproduction From the built source checkout below, run this Python 3 probe. It uses a generated XML plist, isolated HOME/XDG directories, and no database or external API. This is the relevant subset of the executed probe; the omitted past-end cases are already tracked in #5110. ```sh python3 - <<'PY' import os from pathlib import Path import plistlib import re import subprocess import tempfile entry = Path('scripts/rea.mjs').resolve() with tempfile.TemporaryDirectory() as temporary: work = Path(temporary) env = os.environ.copy() for key in ('HOME', 'XDG_CONFIG_HOME', 'XDG_DATA_HOME', 'XDG_CACHE_HOME', 'XDG_STATE_HOME'): directory = work / key.lower() directory.mkdir() env[key] = str(directory) fixture = work / 'small.plist' fixture.write_bytes(plistlib.dumps({'Name': 'small', 'Items': [1, 2, 3]})) cases = [ ('full', fixture, []), ('positive limit', fixture, ['--token-limit', '5']), ('negative limit', fixture, ['--token-limit', '-5']), ('fractional limit', fixture, ['--token-limit', '2.5']), ('zero limit', fixture, ['--token-limit', '0']), ('negative offset', fixture, ['--token-offset', '-100', '--token-limit', '20']), ('failure negative limit', work / 'absent.plist', ['--full-output', '--token-limit', '-5']), ] for label, target, flags in cases: result = subprocess.run(['node', str(entry), 'inspect-plist', str(target), '--format', 'toon', *flags], cwd=work, env=env, capture_output=True, text=True) markers = re.findall(r'\[truncated: showing tokens [^\]]+\]', result.stdout) continuation = re.findall(r'nextOffset: [^\n]+', result.stdout) print(label, 'exit=', result.returncode, 'bytes=', len(result.stdout.encode()), 'markers=', markers, 'continuation=', continuation, 'stderr=', repr(result.stderr)) PY ``` The independent verifier also executed `--token-offset 2.5 --token-limit 5` on the same fixture and observed `showing tokens 2.5–7.5`. Each listed case reproduced in the verifier's probe and its exact-script rerun; the submitter reran the full embedded probe and confirmed the negative-limit, fractional-limit, negative-offset, and negative-continuation results again. ### Expected behavior CLI help advertises limiting output to n tokens and skipping the first n tokens. `docs/tool-design.md:46-69` requires preserving meaningful caller choices and reporting truncation; `docs/cli.md:437-443` documents the valued output controls. Accepted windows should report coordinates consistent with the returned slice and provide a continuation that can advance through the result. Values that cannot satisfy that contract should be rejected clearly or receive coherent documented normalization. Zero is a valid empty-window control. No particular error code or normalization implementation is prescribed. ### Impact and scope Negative limits can defeat a caller's intended output budget; negative failure-envelope continuation offsets can send a paging caller backward. Negative offsets select text relative to the end while displaying negative absolute coordinates. Observed scope is local TOON CLI output at the versions below, including successful results and full failure envelopes. No persistent target modification or security boundary crossing was observed. Other formats and production frequency were not measured. A demonstrated workaround is to use non-negative integer values: limit 5/offset 0 and limit 0 returned coherent windows. Callers can validate computed bounds before invoking REA. ### REA version 6.7.0; source commit `074768901f3cdc8205b61773fb1faf6f6c7b3ca8`; locked Incur 0.6.1 and tokenx 1.3.0. The latest published REA release is `rea-agents-6.7.0`. ### Runtime and platform Node.js v26.12.0; macOS arm64 (OS minor version was not recorded by the independent verifier). No optional native analysis provider was involved. ### Target and artifact context Generated XML plist with `Name=small` and `Items=[1,2,3]`; an absent local plist for the full failure-envelope case. ### Interface CLI ### Earliest failure stage Request parsing or validation ### Relevant output or evidence Independent verifier's public CLI output, summarized without fixture paths: ```text full: exit 0; 2014 stdout bytes; 518 estimated tokens --token-limit 5: exit 0; showing tokens 0–5 of 518 --token-limit -5: exit 0; 2042 stdout bytes; showing tokens 0–-5 of 518 --token-limit 2.5: exit 0; showing tokens 0–2.5 of 518 --token-limit 0: exit 0; showing tokens 0–0 of 518 --token-offset -100 --token-limit 20: exit 0; showing tokens -100–-80 of 518 --token-offset 2.5 --token-limit 5: exit 0; showing tokens 2.5–7.5 of 518 absent target, --full-output --token-limit -5: exit 1; showing tokens 0–-5 of 282; nextOffset: -5 ``` Stderr was empty for these cases. Token totals vary because the generated fixture path is embedded in Evidence. The negative-limit body is almost the full body; the appended marker makes total stdout larger than unwindowed stdout. The submitter's repeat produced totals of 509/270 and again `nextOffset: -5`. The verifier also confirmed that missing and nonnumeric limit values fail; finite negative and fractional values pass parsing. The failure-envelope case retains the operation's exit status 1. ### Typed outcome or recovery expected A clear input failure or a coherent normalized token window with usable continuation metadata. Retain the underlying operation-failure status when the target is absent. ### Analysis or suspected cause Source-derived attribution: global finite-number parsing lives in the bundled Incur 0.6.1, `node_modules/incur/dist/Cli.js:1993-2006`. REA mirrors parsing in `src/cliOutput.ts:131-140`; `scripts/rea.mjs` and that module pre-validate some output flags. Ordinary successful TOON rendering is delegated to Incur by `src/cli/cliResultOutput.ts:222-245`. Incur's `Cli.js:1047-1063` reports raw indices while token slicing normalizes negative indices. An independently executed standalone Incur CLI reproduced the negative-limit result too. REA separately repeats the range and continuation arithmetic for full failure envelopes in `src/cli/cliResultOutput.ts:353-373`, producing the negative `nextOffset`. The maintainers can choose where to enforce coherent bounds. No issue is being filed against Incur as part of this report. ### Additional context Prepared with AI assistance and independently reproduced before submission. Upstream main was fetched and checked through `c374c2fe7290b914d73d9e27af38b498753ec593`; changes since the tested baseline do not alter the relevant numeric parsing or window arithmetic. Related #5110 and its PR #5119 cover exhausted positive offsets on failure envelopes. This report concerns negative/fractional inputs and negative continuation offsets; it does not request another fix for that already reported past-end case. Character corruption with otherwise valid integer windows is a separate defect. NOT_ENOUGH_INFO: first affected release and last known good revision were not established.

bsakaguchi closed this issue

2026-10-31 00:39:37 +00:00

bsakaguchi commented

2026-10-31 00:39:38 +00:00

Author

Owner

@matbrgz commented on GitHub (Oct 20, 2026):

Cross-referencing with canonical issue #5110 and siblings #5448, #5530, and #5531 regarding negative/fractional token window boundaries and pagination consistency. Cataloged under Cluster AQ in consolidation tracker #4330.

@matbrgz commented on GitHub (Oct 20, 2026): Cross-referencing with canonical issue #5110 and siblings #5448, #5530, and #5531 regarding negative/fractional token window boundaries and pagination consistency. Cataloged under Cluster AQ in consolidation tracker #4330.

bsakaguchi commented

2026-10-31 00:39:38 +00:00

Author

Owner

@coygeek commented on GitHub (Oct 24, 2026):

Checked the reported fix on main at d87e73f2db.

The executable now validates global output controls before command dispatch and rejects negative, fractional, nonfinite, empty, or unsafe-integer coordinates. Both token-limit and token-offset use the same nonnegative-safe-integer check; zero remains accepted. This prevents both successful-output invalid windows and negative failure-envelope continuations from being constructed through the public entrypoint.

Implementation: cliOutput.ts, cliOutput.ts.

The relevant regression checks in cliOutput.test.ts passed in the focused test run on this revision. This closure is based on the current source and those checks; provider fixtures do not establish a real-engine run. Closing as completed on main.

@coygeek commented on GitHub (Oct 24, 2026): Checked the reported fix on `main` at d87e73f2db50be72d33a4b3c7199af35518b441c. The executable now validates global output controls before command dispatch and rejects negative, fractional, nonfinite, empty, or unsafe-integer coordinates. Both token-limit and token-offset use the same nonnegative-safe-integer check; zero remains accepted. This prevents both successful-output invalid windows and negative failure-envelope continuations from being constructed through the public entrypoint. Implementation: [cliOutput.ts](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/src/cliOutput.ts#L57), [cliOutput.ts](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/src/cliOutput.ts#L198). The relevant regression checks in [cliOutput.test.ts](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/tests/boundary/cli/cliOutput.test.ts) passed in the focused test run on this revision. This closure is based on the current source and those checks; provider fixtures do not establish a real-engine run. Closing as completed on main.

bsakaguchi referenced this issue

2026-10-31 00:40:31 +00:00

[GH-ISSUE #5690] [Bug] Apktool version-output overflow is reported as an unusable launcher #3440

bsakaguchi referenced this issue

2026-10-31 00:40:31 +00:00

[GH-ISSUE #5689] [Bug] inspect-adb-client hides execution failures behind a version diagnostic #3441

bsakaguchi referenced this issue

2026-10-31 00:42:16 +00:00

[GH-ISSUE #5924] [Bug] ADB push claims a checksum source when no device digest was parsed #3599

bsakaguchi referenced this issue

2026-10-31 00:42:43 +00:00

[GH-ISSUE #5981] [Bug] ADB version-probe failures name inspect_adb_client during list_adb_devices #3649