morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #5701] [Bug] inspect-plist rejects a valid binary plist containing a control character and data #3451

New issue

Closed

opened 2026-10-31 00:40:38 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-31 00:40:38 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 20, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/5701

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Native analysis or decompilation

Observed behavior

inspect-plist exits 1 with unreadable_output and no normalized value for a valid binary plist containing a note string with code points U+0061, U+0001, U+0062 and a payload data value with one zero byte. Removing only the data value succeeds and preserves U+0001. Removing only U+0001 succeeds and returns the typed data value. The failure depends on combining two independently readable values.

Reproduction

From a source checkout with locked dependencies installed and the CLI built, run on macOS:

work_dir="$(mktemp -d)"
mkdir -p "$work_dir/home" "$work_dir/cfg" "$work_dir/data" \
  "$work_dir/cache" "$work_dir/state" "$work_dir/tmp"
python3 -I - "$work_dir" <<'PYTHON'
from pathlib import Path
import plistlib, sys
work = Path(sys.argv[1])
fixtures = {
    "control.bin": {"note": "a\x01b", "payload": b"\x00"},
    "nodata.bin": {"note": "a\x01b"},
    "clean.bin": {"note": "ab", "payload": b"\x00"},
}
for name, value in fixtures.items():
    (work / name).write_bytes(plistlib.dumps(value, fmt=plistlib.FMT_BINARY))
print(repr(plistlib.loads((work / "control.bin").read_bytes())))
PYTHON
for fixture in control nodata clean; do
  env -i PATH="$PATH" HOME="$work_dir/home" \
    XDG_CONFIG_HOME="$work_dir/cfg" XDG_DATA_HOME="$work_dir/data" \
    XDG_CACHE_HOME="$work_dir/cache" XDG_STATE_HOME="$work_dir/state" \
    TMPDIR="$work_dir/tmp" REA_LOG_LEVEL=silent \
    node scripts/rea.mjs inspect-plist "$work_dir/$fixture.bin" --json \
    > "$work_dir/$fixture.result.json"
  printf '%s exit %s\n' "$fixture" "$?"
  cat "$work_dir/$fixture.result.json"
done

This exact script was independently executed at the baseline and rerun against a fresh npm run build:cached build of main below. On main, control exits 1; nodata and clean each exit 0.

Expected behavior

Return the original string and the typed data value in normalized JSON. The inspect_plist contract advertises normalized JSON for a selected local plist. Python plistlib and macOS plutil -p both read the original binary input, and JSON can represent U+0001.

The documented strict XML validation explains why the converted XML fails, but does not declare this binary input unsupported. This shallow fixture has no dates.

Impact and scope

P2: a bounded valid-input combination prevents inspection from returning any normalized facts. Both tested revisions reproduce it. Python plistlib and plutil -p provide demonstrated ways to recover the original values, although they do not produce REA Evidence. Frequency in real artifacts is unknown. No source mutation or security boundary crossing was observed. MCP was not executed.

REA version

rea-agents 6.7.0, source baseline 9d8e7803326a382cbbb3ffdb900f5e93e5442d68. Rechecked by building and executing current main 52c84f6f58fa3cc7b39326654c6d3f261f957957, also 6.7.0; still reproduces.

Runtime and platform

Node.js v26.12.0; macOS 27.0.1; arm64; system plutil. No Hopper/Ghidra/IDA process involved.

Target and artifact context

Self-authored shallow binary property lists generated and independently decoded with Python plistlib. No app bundle or live service required.

Interface

CLI

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

The failing CLI returns this relevant subset:

{"code":"unreadable_output","category":"execution_failure","details":{"operation":"inspect_plist","reason":"plutil XML conversion was not a readable plist"}}

The nodata control returns a note string containing exactly U+0061, U+0001, U+0062. The clean control returns this normalized value:

{"note":"ab","payload":{"$plist_type":"data","base64":"AA=="}}

Additional commands actually executed on the same failing fixture:

/usr/bin/plutil -p "$work_dir/control.bin"
/usr/bin/plutil -convert json -o - -- "$work_dir/control.bin"
/usr/bin/plutil -convert xml1 -o "$work_dir/control.xml" -- "$work_dir/control.bin"

plutil -p reads both values. JSON conversion exits 1 with Invalid object in plist for JSON format. XML conversion exits 0, but a byte check finds raw U+0001 inside <string>a…b</string>. Python ElementTree rejects that conversion with not well-formed (invalid token): line 6, column 10; plistlib.loads reads the original binary as {'note': 'a\x01b', 'payload': b'\x00'}. These observations distinguish the valid binary source from its invalid intermediate XML.

Evidence provenance: observed. Local verification: independently reproduced and rechecked on main. Other control characters and date-bearing triggers were not tested.

Typed outcome or recovery expected

Exit 0 and return note containing exactly U+0061, U+0001, U+0062 and payload equal to {"$plist_type":"data","base64":"AA=="}. Both controls must retain their current values. Dropping or replacing U+0001 would not restore the contract. Preserve the documented rejection of malformed caller-supplied XML.

Analysis or suspected cause

Source-derived reasoning: NativeMacOSProvider.ts selects XML conversion after JSON conversion fails on the data value. parsers/plist.ts maps XML parse failure to the reported error. The shared XML source owner applies strict XML 1.0 validation. The conversion is not a lossless representation for this binary string. This report does not propose relaxing XML validation.

Additional context

Related: #4834 and merged #4839 concern negative-zero identity through the same conversion path, with a different failure and restoration condition. This report concerns complete inspection failure for U+0001 plus data. #4714's malformed-source rejection remains valid; the input here is a readable binary plist. Merged #841 introduced the data/date fallback, but this combination still fails on current main.

Prepared with AI assistance and independent verification. NOT_ENOUGH_INFO: no last known good version or regression range was established. No fix or restoration has been tested.

Originally created by @coygeek on GitHub (Oct 20, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/5701 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Native analysis or decompilation ### Observed behavior `inspect-plist` exits 1 with `unreadable_output` and no normalized value for a valid binary plist containing a `note` string with code points U+0061, U+0001, U+0062 and a `payload` data value with one zero byte. Removing only the data value succeeds and preserves U+0001. Removing only U+0001 succeeds and returns the typed data value. The failure depends on combining two independently readable values. ### Reproduction From a source checkout with locked dependencies installed and the CLI built, run on macOS: ```sh work_dir="$(mktemp -d)" mkdir -p "$work_dir/home" "$work_dir/cfg" "$work_dir/data" \ "$work_dir/cache" "$work_dir/state" "$work_dir/tmp" python3 -I - "$work_dir" <<'PYTHON' from pathlib import Path import plistlib, sys work = Path(sys.argv[1]) fixtures = { "control.bin": {"note": "a\x01b", "payload": b"\x00"}, "nodata.bin": {"note": "a\x01b"}, "clean.bin": {"note": "ab", "payload": b"\x00"}, } for name, value in fixtures.items(): (work / name).write_bytes(plistlib.dumps(value, fmt=plistlib.FMT_BINARY)) print(repr(plistlib.loads((work / "control.bin").read_bytes()))) PYTHON for fixture in control nodata clean; do env -i PATH="$PATH" HOME="$work_dir/home" \ XDG_CONFIG_HOME="$work_dir/cfg" XDG_DATA_HOME="$work_dir/data" \ XDG_CACHE_HOME="$work_dir/cache" XDG_STATE_HOME="$work_dir/state" \ TMPDIR="$work_dir/tmp" REA_LOG_LEVEL=silent \ node scripts/rea.mjs inspect-plist "$work_dir/$fixture.bin" --json \ > "$work_dir/$fixture.result.json" printf '%s exit %s\n' "$fixture" "$?" cat "$work_dir/$fixture.result.json" done ``` This exact script was independently executed at the baseline and rerun against a fresh `npm run build:cached` build of main below. On main, `control` exits 1; `nodata` and `clean` each exit 0. ### Expected behavior Return the original string and the typed data value in normalized JSON. The [inspect_plist contract](https://github.com/morluto/rea/blob/9d8e7803326a382cbbb3ffdb900f5e93e5442d68/src/contracts/native/nativeToolContracts.ts#L111-L117) advertises normalized JSON for a selected local plist. Python plistlib and macOS `plutil -p` both read the original binary input, and JSON can represent U+0001. The [documented strict XML validation](https://github.com/morluto/rea/blob/9d8e7803326a382cbbb3ffdb900f5e93e5442d68/docs/native-investigation.md#L704-L717) explains why the converted XML fails, but does not declare this binary input unsupported. This shallow fixture has no dates. ### Impact and scope P2: a bounded valid-input combination prevents inspection from returning any normalized facts. Both tested revisions reproduce it. Python plistlib and `plutil -p` provide demonstrated ways to recover the original values, although they do not produce REA Evidence. Frequency in real artifacts is unknown. No source mutation or security boundary crossing was observed. MCP was not executed. ### REA version `rea-agents` 6.7.0, source baseline `9d8e7803326a382cbbb3ffdb900f5e93e5442d68`. Rechecked by building and executing current main `52c84f6f58fa3cc7b39326654c6d3f261f957957`, also 6.7.0; still reproduces. ### Runtime and platform Node.js v26.12.0; macOS 27.0.1; arm64; system `plutil`. No Hopper/Ghidra/IDA process involved. ### Target and artifact context Self-authored shallow binary property lists generated and independently decoded with Python plistlib. No app bundle or live service required. ### Interface CLI ### Earliest failure stage Native analysis or reconstruction ### Relevant output or evidence The failing CLI returns this relevant subset: ```json {"code":"unreadable_output","category":"execution_failure","details":{"operation":"inspect_plist","reason":"plutil XML conversion was not a readable plist"}} ``` The `nodata` control returns a `note` string containing exactly U+0061, U+0001, U+0062. The `clean` control returns this normalized `value`: ```json {"note":"ab","payload":{"$plist_type":"data","base64":"AA=="}} ``` Additional commands actually executed on the same failing fixture: ```sh /usr/bin/plutil -p "$work_dir/control.bin" /usr/bin/plutil -convert json -o - -- "$work_dir/control.bin" /usr/bin/plutil -convert xml1 -o "$work_dir/control.xml" -- "$work_dir/control.bin" ``` `plutil -p` reads both values. JSON conversion exits 1 with `Invalid object in plist for JSON format`. XML conversion exits 0, but a byte check finds raw U+0001 inside `<string>a…b</string>`. Python ElementTree rejects that conversion with `not well-formed (invalid token): line 6, column 10`; `plistlib.loads` reads the original binary as `{'note': 'a\x01b', 'payload': b'\x00'}`. These observations distinguish the valid binary source from its invalid intermediate XML. Evidence provenance: observed. Local verification: independently reproduced and rechecked on main. Other control characters and date-bearing triggers were not tested. ### Typed outcome or recovery expected Exit 0 and return `note` containing exactly U+0061, U+0001, U+0062 and `payload` equal to `{"$plist_type":"data","base64":"AA=="}`. Both controls must retain their current values. Dropping or replacing U+0001 would not restore the contract. Preserve the documented rejection of malformed caller-supplied XML. ### Analysis or suspected cause Source-derived reasoning: [NativeMacOSProvider.ts](https://github.com/morluto/rea/blob/9d8e7803326a382cbbb3ffdb900f5e93e5442d68/src/native/NativeMacOSProvider.ts#L828-L875) selects XML conversion after JSON conversion fails on the data value. [parsers/plist.ts](https://github.com/morluto/rea/blob/9d8e7803326a382cbbb3ffdb900f5e93e5442d68/src/native/parsers/plist.ts#L162-L179) maps XML parse failure to the reported error. The shared XML source owner applies strict XML 1.0 validation. The conversion is not a lossless representation for this binary string. This report does not propose relaxing XML validation. ### Additional context Related: #4834 and merged #4839 concern negative-zero identity through the same conversion path, with a different failure and restoration condition. This report concerns complete inspection failure for U+0001 plus data. #4714's malformed-source rejection remains valid; the input here is a readable binary plist. Merged #841 introduced the data/date fallback, but this combination still fails on current main. Prepared with AI assistance and independent verification. NOT_ENOUGH_INFO: no last known good version or regression range was established. No fix or restoration has been tested.

bsakaguchi closed this issue

2026-10-31 00:40:38 +00:00

bsakaguchi commented

2026-10-31 00:40:39 +00:00

Author

Owner

@matbrgz commented on GitHub (Oct 20, 2026):

Cross-referencing with canonical issue #3371 (CDATA in XML keyed-archive keys truncates root and field names) and sibling plist decoding reports #3373, #4042, #4714, #4849, #4852, #5702, #5703, #5705.

Consolidated under Cluster K (XML Keyed-Archive Deserialization & CDATA Handling / Plist Decoder) in tracker #4330.

@matbrgz commented on GitHub (Oct 20, 2026): Cross-referencing with canonical issue #3371 (`CDATA in XML keyed-archive keys truncates root and field names`) and sibling plist decoding reports #3373, #4042, #4714, #4849, #4852, #5702, #5703, #5705. Consolidated under **Cluster K (XML Keyed-Archive Deserialization & CDATA Handling / Plist Decoder)** in tracker #4330.

bsakaguchi referenced this issue

2026-10-31 01:09:36 +00:00

[PR #3534] [CLOSED] fix(javascript): keep unresolved network request targets visible as partial endpoints #5810

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#3451

No description provided.

Rows
Columns