morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #5737] [Bug] inspect-plist rejects valid data-bearing plists at depth 129 while deeper integer plists succeed #3484

New issue

Closed

opened 2026-10-31 00:41:01 +00:00 by bsakaguchi · 4 comments

bsakaguchi commented

2026-10-31 00:41:01 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 21, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/5737

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Native analysis or decompilation

Observed behavior

inspect-plist rejects a valid binary plist containing a two-byte data leaf at value depth 129 with unreadable_output. The same command accepts data at depth 128 and integer leaves at depths 129 and 200. Apple's plutil accepts the failing source and its XML conversion. Inspecting that converted XML also fails. The error calls the conversion unreadable and recommends rea doctor instead of identifying the depth restriction.

Reproduction

Run from a built source checkout with locked dependencies installed, on macOS with Python 3, system plutil and jq:

work_dir="$(mktemp -d)"
S="$work_dir"
mkdir -p "$S"/{home,cfg,data,cache,state,tmp}
python3 - "$S" <<'PY'
import pathlib, plistlib, sys
root = pathlib.Path(sys.argv[1])
for name, depth, leaf in [
    ("data128", 128, b"\x00\x01"),
    ("data129", 129, b"\x00\x01"),
    ("int129", 129, 1),
    ("int200", 200, 1),
]:
    value = leaf
    for index in range(depth, 0, -1):
        value = {"level%d" % index: value}
    (root / (name + ".plist")).write_bytes(
        plistlib.dumps(value, fmt=plistlib.FMT_BINARY))
PY
for fixture in data128 data129 int129 int200; do
  env -i PATH="$PATH" HOME="$S/home" XDG_CONFIG_HOME="$S/cfg" XDG_DATA_HOME="$S/data" XDG_CACHE_HOME="$S/cache" XDG_STATE_HOME="$S/state" TMPDIR="$S/tmp" REA_LOG_LEVEL=silent \
    node scripts/rea.mjs inspect-plist "$S/$fixture.plist" --json > "$S/$fixture.json" 2> "$S/$fixture.err"
  printf '%s exit=%s\n' "$fixture" "$?"
  jq -c 'if .error then {code, reason: .details.reason} else {format: .normalized_result.format} end' "$S/$fixture.json"
done
plutil -lint "$S/data129.plist"
plutil -convert xml1 -o "$S/data129.xml" "$S/data129.plist"
plutil -lint "$S/data129.xml"
plutil -convert json -o /dev/null "$S/data129.plist"
printf 'plutil JSON exit=%s\n' "$?"
env -i PATH="$PATH" HOME="$S/home" XDG_CONFIG_HOME="$S/cfg" XDG_DATA_HOME="$S/data" XDG_CACHE_HOME="$S/cache" XDG_STATE_HOME="$S/state" TMPDIR="$S/tmp" REA_LOG_LEVEL=silent \
  node scripts/rea.mjs inspect-plist "$S/data129.xml" --json > "$S/data129-xml.json" 2> "$S/data129-xml.err"
printf 'data129 XML exit=%s\n' "$?"
jq -c '{code, reason: .details.reason}' "$S/data129-xml.json"

The independent verifier executed each CLI fixture once at the baseline below. I reran the script after a fresh npm run build:cached on fetched main; each of its five CLI invocations reproduced the same result. No target code or application was executed.

Expected behavior

Return the complete nested structure with the depth-129 data leaf represented as { "$plist_type": "data", "base64": "AAE=" }, as in the depth-128 control, and retain accurate source format, provenance and limitations. If XML conversion remains the decoding path, retain its applicable limitation. The same depth already succeeds for an integer leaf.

The inspect_plist contract advertises normalized JSON for a selected local plist. Native investigation guidance documents typed scalar projection for XML conversion. Its 128-level documented bound belongs to keyed-archive inspection, rather than ordinary plist inspection. Resource protection and malformed-input validation must remain effective; a genuine resource refusal should identify the bound accurately.

Impact and scope

P2: deterministic rejection of this bounded valid input class returns no normalized facts and gives misleading recovery guidance. Baseline and current-main executions agree. Production frequency is unknown; no data mutation or security-boundary crossing was observed. Converting to XML also fails. Replacing data with an integer changes the selected evidence, so a safe practical workaround preserving the complete result remains unverified. Dates, shipped application fixtures, MCP and other platforms were not tested.

REA version

rea-agents 6.7.0, source baseline 9d8e7803326a382cbbb3ffdb900f5e93e5442d68. Rechecked with a fresh build and public CLI execution on fetched main d0067ba34a39ea47dd7e11bb2e6169bc0938cc3f (also 6.7.0): still reproduces. The lockfile did not change between those revisions.

Runtime and platform

Node.js v26.12.0; macOS 27.0.1; arm64; Python 3 standard-library plistlib and system plutil. No Hopper/Ghidra/IDA involvement.

Target and artifact context

Synthetic binary property lists containing dictionaries nested 128 or 129 levels around data bytes 00 01, plus integer-1 controls at depths 129 and 200. The failing binary is also converted to XML for a second inspection. No private target data or live service is involved.

Interface

CLI

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

Fresh current-main public CLI results, reduced to relevant fields:

data128 exit=0
{"format":"binary"}
data129 exit=1
{"code":"unreadable_output","reason":"plutil XML conversion was not a readable plist"}
int129 exit=0
{"format":"binary"}
int200 exit=0
{"format":"binary"}
plutil JSON exit=1
data129 XML exit=1
{"code":"unreadable_output","reason":"plutil XML conversion was not a readable plist"}

Both plutil -lint calls report OK; XML conversion exits 0. JSON conversion reports Invalid object in plist for JSON format, selecting REA's XML fallback. Both failing CLI responses have category execution_failure, retryable: false, and advice to run rea doctor. I checked the successful controls' complete nested structures and exact leaves: typed data with base64 AAE= at depth 128 and numeric 1 at depths 129 and 200. The data control includes the XML-conversion limitation; both integer controls have no limitations.

Evidence provenance: observed public CLI and native utility checks, independently reproduced at the baseline and rerun on main. The rejecting guard identified below is source-derived reasoning.

Typed outcome or recovery expected

The same depth-129 binary fixture should succeed with the complete nested structure, exact typed data leaf, format: "binary", and accurate provenance and limitations. Its converted XML should also succeed while retaining its own source format. Preserve all three successful controls. Proposed discriminating checks: malformed XML and unsupported non-ASCII data literals must still fail, and keyed-archive and Interface Builder resource contracts must remain intact. These are acceptance checks, not a claim of a tested fix.

Analysis or suspected cause

Source-derived reasoning: NativeMacOSProvider.ts selects XML conversion after JSON refusal. propertyListXmlSource.ts rejects value depth above the shared MAX_PLIST_DEPTH (128) during preflight. parsePlistXml replaces that exception with the unreadable-conversion message. The successful small-integer JSON path bypasses that XML preflight and projection.

The first rejecting guard is REA's preflight, not evidence that its bundled plist decoder failed. The shared depth bound is intentional; the defect is its undocumented, value-type-dependent application in inspect_plist and loss of the diagnostic. This report does not ask to remove resource bounds from other consumers. Current main retains these paths.

Additional context

Related #5701, #5702, #5703 and #5705 concern control-character rejection, Unicode separators, fractional dates and negative zero through XML fallback. This data-only fixture has its own depth-bound restoration condition. Closed #2105 / merged #2161 cover early rejection of over-limit keyed archives; closed #3949 / merged #4114 cover Interface Builder values within its supported depth budget. Their fixes do not restore this ordinary-plist inspection case.

Prepared with AI assistance and independent verification. No fix or successful restoration has been tested. Last known good version: NOT_ENOUGH_INFO; none established.

Originally created by @coygeek on GitHub (Oct 21, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/5737 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Native analysis or decompilation ### Observed behavior `inspect-plist` rejects a valid binary plist containing a two-byte data leaf at value depth 129 with `unreadable_output`. The same command accepts data at depth 128 and integer leaves at depths 129 and 200. Apple's `plutil` accepts the failing source and its XML conversion. Inspecting that converted XML also fails. The error calls the conversion unreadable and recommends `rea doctor` instead of identifying the depth restriction. ### Reproduction Run from a built source checkout with locked dependencies installed, on macOS with Python 3, system plutil and jq: ```sh work_dir="$(mktemp -d)" S="$work_dir" mkdir -p "$S"/{home,cfg,data,cache,state,tmp} python3 - "$S" <<'PY' import pathlib, plistlib, sys root = pathlib.Path(sys.argv[1]) for name, depth, leaf in [ ("data128", 128, b"\x00\x01"), ("data129", 129, b"\x00\x01"), ("int129", 129, 1), ("int200", 200, 1), ]: value = leaf for index in range(depth, 0, -1): value = {"level%d" % index: value} (root / (name + ".plist")).write_bytes( plistlib.dumps(value, fmt=plistlib.FMT_BINARY)) PY for fixture in data128 data129 int129 int200; do env -i PATH="$PATH" HOME="$S/home" XDG_CONFIG_HOME="$S/cfg" XDG_DATA_HOME="$S/data" XDG_CACHE_HOME="$S/cache" XDG_STATE_HOME="$S/state" TMPDIR="$S/tmp" REA_LOG_LEVEL=silent \ node scripts/rea.mjs inspect-plist "$S/$fixture.plist" --json > "$S/$fixture.json" 2> "$S/$fixture.err" printf '%s exit=%s\n' "$fixture" "$?" jq -c 'if .error then {code, reason: .details.reason} else {format: .normalized_result.format} end' "$S/$fixture.json" done plutil -lint "$S/data129.plist" plutil -convert xml1 -o "$S/data129.xml" "$S/data129.plist" plutil -lint "$S/data129.xml" plutil -convert json -o /dev/null "$S/data129.plist" printf 'plutil JSON exit=%s\n' "$?" env -i PATH="$PATH" HOME="$S/home" XDG_CONFIG_HOME="$S/cfg" XDG_DATA_HOME="$S/data" XDG_CACHE_HOME="$S/cache" XDG_STATE_HOME="$S/state" TMPDIR="$S/tmp" REA_LOG_LEVEL=silent \ node scripts/rea.mjs inspect-plist "$S/data129.xml" --json > "$S/data129-xml.json" 2> "$S/data129-xml.err" printf 'data129 XML exit=%s\n' "$?" jq -c '{code, reason: .details.reason}' "$S/data129-xml.json" ``` The independent verifier executed each CLI fixture once at the baseline below. I reran the script after a fresh `npm run build:cached` on fetched main; each of its five CLI invocations reproduced the same result. No target code or application was executed. ### Expected behavior Return the complete nested structure with the depth-129 data leaf represented as `{ "$plist_type": "data", "base64": "AAE=" }`, as in the depth-128 control, and retain accurate source format, provenance and limitations. If XML conversion remains the decoding path, retain its applicable limitation. The same depth already succeeds for an integer leaf. The [inspect_plist contract](https://github.com/morluto/rea/blob/9d8e7803326a382cbbb3ffdb900f5e93e5442d68/src/contracts/native/nativeToolContracts.ts#L112-L117) advertises normalized JSON for a selected local plist. [Native investigation guidance](https://github.com/morluto/rea/blob/9d8e7803326a382cbbb3ffdb900f5e93e5442d68/docs/native-investigation.md#L669-L717) documents typed scalar projection for XML conversion. Its [128-level documented bound](https://github.com/morluto/rea/blob/9d8e7803326a382cbbb3ffdb900f5e93e5442d68/docs/native-investigation.md#L359-L380) belongs to keyed-archive inspection, rather than ordinary plist inspection. Resource protection and malformed-input validation must remain effective; a genuine resource refusal should identify the bound accurately. ### Impact and scope P2: deterministic rejection of this bounded valid input class returns no normalized facts and gives misleading recovery guidance. Baseline and current-main executions agree. Production frequency is unknown; no data mutation or security-boundary crossing was observed. Converting to XML also fails. Replacing data with an integer changes the selected evidence, so a safe practical workaround preserving the complete result remains unverified. Dates, shipped application fixtures, MCP and other platforms were not tested. ### REA version `rea-agents` 6.7.0, source baseline `9d8e7803326a382cbbb3ffdb900f5e93e5442d68`. Rechecked with a fresh build and public CLI execution on fetched main `d0067ba34a39ea47dd7e11bb2e6169bc0938cc3f` (also 6.7.0): still reproduces. The lockfile did not change between those revisions. ### Runtime and platform Node.js v26.12.0; macOS 27.0.1; arm64; Python 3 standard-library plistlib and system plutil. No Hopper/Ghidra/IDA involvement. ### Target and artifact context Synthetic binary property lists containing dictionaries nested 128 or 129 levels around data bytes `00 01`, plus integer-1 controls at depths 129 and 200. The failing binary is also converted to XML for a second inspection. No private target data or live service is involved. ### Interface CLI ### Earliest failure stage Native analysis or reconstruction ### Relevant output or evidence Fresh current-main public CLI results, reduced to relevant fields: ```text data128 exit=0 {"format":"binary"} data129 exit=1 {"code":"unreadable_output","reason":"plutil XML conversion was not a readable plist"} int129 exit=0 {"format":"binary"} int200 exit=0 {"format":"binary"} plutil JSON exit=1 data129 XML exit=1 {"code":"unreadable_output","reason":"plutil XML conversion was not a readable plist"} ``` Both `plutil -lint` calls report `OK`; XML conversion exits 0. JSON conversion reports `Invalid object in plist for JSON format`, selecting REA's XML fallback. Both failing CLI responses have category `execution_failure`, `retryable: false`, and advice to run `rea doctor`. I checked the successful controls' complete nested structures and exact leaves: typed data with base64 `AAE=` at depth 128 and numeric `1` at depths 129 and 200. The data control includes the XML-conversion limitation; both integer controls have no limitations. Evidence provenance: observed public CLI and native utility checks, independently reproduced at the baseline and rerun on main. The rejecting guard identified below is source-derived reasoning. ### Typed outcome or recovery expected The same depth-129 binary fixture should succeed with the complete nested structure, exact typed data leaf, `format: "binary"`, and accurate provenance and limitations. Its converted XML should also succeed while retaining its own source format. Preserve all three successful controls. Proposed discriminating checks: malformed XML and unsupported non-ASCII data literals must still fail, and keyed-archive and Interface Builder resource contracts must remain intact. These are acceptance checks, not a claim of a tested fix. ### Analysis or suspected cause Source-derived reasoning: [NativeMacOSProvider.ts](https://github.com/morluto/rea/blob/9d8e7803326a382cbbb3ffdb900f5e93e5442d68/src/native/NativeMacOSProvider.ts#L797-L875) selects XML conversion after JSON refusal. [propertyListXmlSource.ts](https://github.com/morluto/rea/blob/9d8e7803326a382cbbb3ffdb900f5e93e5442d68/src/domain/propertyListXmlSource.ts#L150-L189) rejects value depth above the shared `MAX_PLIST_DEPTH` (128) during preflight. [parsePlistXml](https://github.com/morluto/rea/blob/9d8e7803326a382cbbb3ffdb900f5e93e5442d68/src/native/parsers/plist.ts#L162-L180) replaces that exception with the unreadable-conversion message. The successful small-integer JSON path bypasses that XML preflight and projection. The first rejecting guard is REA's preflight, not evidence that its bundled plist decoder failed. The shared depth bound is intentional; the defect is its undocumented, value-type-dependent application in `inspect_plist` and loss of the diagnostic. This report does not ask to remove resource bounds from other consumers. Current main retains these paths. ### Additional context Related #5701, #5702, #5703 and #5705 concern control-character rejection, Unicode separators, fractional dates and negative zero through XML fallback. This data-only fixture has its own depth-bound restoration condition. Closed #2105 / merged #2161 cover early rejection of over-limit keyed archives; closed #3949 / merged #4114 cover Interface Builder values within its supported depth budget. Their fixes do not restore this ordinary-plist inspection case. Prepared with AI assistance and independent verification. No fix or successful restoration has been tested. Last known good version: NOT_ENOUGH_INFO; none established.

bsakaguchi closed this issue

2026-10-31 00:41:01 +00:00

bsakaguchi commented

2026-10-31 00:41:02 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 21, 2026):

The interesting part here is the asymmetry: integer leaves sail past depth 129/200 while a two-byte data leaf stops at 129, and the error is misreported as unreadable_output pointing at doctor. That reads like a parser-side bound on data-bearing leaf nodes (the 128-level documented limit) that was never surfaced to the user. The fix direction from #5741 also applies: once the parser knows the input exceeds its structural bound, it should say so explicitly (actual depth vs limit) and treat it as a clear out-of-scope rejection, not an unreadable blob. Worth adding the depth-129 data fixture to the contract tests so the diagnostic message is pinned down alongside the fix.

@XinwangTao commented on GitHub (Oct 21, 2026): The interesting part here is the asymmetry: integer leaves sail past depth 129/200 while a two-byte data leaf stops at 129, and the error is misreported as unreadable_output pointing at doctor. That reads like a parser-side bound on data-bearing leaf nodes (the 128-level documented limit) that was never surfaced to the user. The fix direction from #5741 also applies: once the parser knows the input exceeds its structural bound, it should say so explicitly (actual depth vs limit) and treat it as a clear out-of-scope rejection, not an unreadable blob. Worth adding the depth-129 data fixture to the contract tests so the diagnostic message is pinned down alongside the fix.

bsakaguchi commented

2026-10-31 00:41:02 +00:00

Author

Owner

@kaoru0822-kitauji commented on GitHub (Oct 21, 2026):

Rechecked on current main a5f982de8 (fresh npm run build:cached, macOS arm64, built CLI), using the issue's fixtures. d672cc239 changed the binary path:

Fixture	Result
`data128.plist`	exit 0; `jq` itself rejects the 128-level output (`Exceeds depth limit`), so I did not project the value
`data129.plist`	`resource_constraint`: "binary plist nesting exceeds the native plist decode budget"
`int129.plist`	the same `resource_constraint`
`int200.plist`	the same `resource_constraint`
`data129.xml` (`plutil -convert xml1` of `data129.plist`)	still `unreadable_output` with reason "plutil XML conversion was not a readable plist", plus `rea doctor` advice

So for binary input, the data/integer asymmetry is gone. Both leaf kinds now get the same typed nesting-budget refusal at depth 129. Whether a fixed 128-level budget is the intended contract is the maintainers' call.

The XML-input part still reproduces as reported: a depth-129 XML plist is misreported as unreadable output with doctor advice, instead of a typed nesting refusal or a decode. That appears to belong with the XML-fallback siblings (#5702, and the remaining parts of the #5701 family). Source-derived reasoning only; I did not trace the XML branch. I am not taking this issue.

@kaoru0822-kitauji commented on GitHub (Oct 21, 2026): Rechecked on current main `a5f982de8` (fresh `npm run build:cached`, macOS arm64, built CLI), using the issue's fixtures. `d672cc239` changed the binary path: | Fixture | Result | |---|---| | `data128.plist` | exit 0; `jq` itself rejects the 128-level output (`Exceeds depth limit`), so I did not project the value | | `data129.plist` | `resource_constraint`: "binary plist nesting exceeds the native plist decode budget" | | `int129.plist` | the same `resource_constraint` | | `int200.plist` | the same `resource_constraint` | | `data129.xml` (`plutil -convert xml1` of `data129.plist`) | still `unreadable_output` with reason "plutil XML conversion was not a readable plist", plus `rea doctor` advice | So for binary input, the data/integer asymmetry is gone. Both leaf kinds now get the same typed nesting-budget refusal at depth 129. Whether a fixed 128-level budget is the intended contract is the maintainers' call. The XML-input part still reproduces as reported: a depth-129 XML plist is misreported as unreadable output with doctor advice, instead of a typed nesting refusal or a decode. That appears to belong with the XML-fallback siblings (#5702, and the remaining parts of the #5701 family). Source-derived reasoning only; I did not trace the XML branch. I am not taking this issue.

bsakaguchi commented

2026-10-31 00:41:02 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 21, 2026):

Good to see the binary path now returns resource_constraint with an explicit "nesting exceeds the native plist decode budget" message — that's the right direction and much more actionable than the old unreadable_output.

One remaining inconsistency though: the XML conversion path (data129.xml converted from the same fixture) still fails with unreadable_output and suggests rea doctor, even though the root cause is the same depth-129 data leaf. Since the binary decoder already knows how to classify this as a resource constraint, the XML path should surface the same classification (actual depth vs documented bound) instead of falling back to "unreadable" — otherwise users converting valid plists to XML will keep hitting the misleading error. Also worth noting the error message itself should reference the actual nesting bound rather than a generic doctor hint.

@XinwangTao commented on GitHub (Oct 21, 2026): Good to see the binary path now returns resource_constraint with an explicit "nesting exceeds the native plist decode budget" message — that's the right direction and much more actionable than the old unreadable_output. One remaining inconsistency though: the XML conversion path (data129.xml converted from the same fixture) still fails with unreadable_output and suggests `rea doctor`, even though the root cause is the same depth-129 data leaf. Since the binary decoder already knows how to classify this as a resource constraint, the XML path should surface the same classification (actual depth vs documented bound) instead of falling back to "unreadable" — otherwise users converting valid plists to XML will keep hitting the misleading error. Also worth noting the error message itself should reference the actual nesting bound rather than a generic doctor hint.

bsakaguchi commented

2026-10-31 00:41:02 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 21, 2026):

Thanks for the recheck on main. The binary-path asymmetry being gone (both data and integer leaves now return the same typed resource_constraint at depth 129) matches the parser-side bound theory — good to see it unified. The XML-input path still misreporting depth-129 input as unreadable_output with rea doctor advice remains the real UX problem: users get a misleading recovery hint instead of a precise "nesting exceeds the decode budget" refusal. Suggest pinning a depth-129 XML fixture in the contract tests alongside the binary one, so the XML fallback branch surfaces the same typed refusal as its binary sibling.

@XinwangTao commented on GitHub (Oct 21, 2026): Thanks for the recheck on main. The binary-path asymmetry being gone (both data and integer leaves now return the same typed resource_constraint at depth 129) matches the parser-side bound theory — good to see it unified. The XML-input path still misreporting depth-129 input as unreadable_output with rea doctor advice remains the real UX problem: users get a misleading recovery hint instead of a precise "nesting exceeds the decode budget" refusal. Suggest pinning a depth-129 XML fixture in the contract tests alongside the binary one, so the XML fallback branch surfaces the same typed refusal as its binary sibling.

bsakaguchi referenced this issue

2026-10-31 01:09:19 +00:00

[PR #3484] [MERGED] docs(browser): reconcile HTTP discovery wording with browser sockets #5773

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#3484

No description provided.

Rows
Columns