morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #5702] [Bug] inspect-plist rewrites Unicode separators as LF when XML fallback is used #3452

New issue

Closed

opened 2026-10-31 00:40:38 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-31 00:40:38 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 20, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/5702

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Native analysis or decompilation

Observed behavior

On macOS, inspect-plist silently changes U+0085 (NEL), U+2028 (LINE SEPARATOR), and U+2029 (PARAGRAPH SEPARATOR) inside strings to LF when a binary plist contains a data value and inspection uses the XML fallback. The command succeeds without a limitation describing the changed strings. Removing only the data member makes the same strings survive inspection. Independent Python parses of both the binary source and plutil's converted XML preserve all three characters.

Reproduction

Run from a built source checkout with its locked dependencies installed, using Python 3 and system plutil:

work_dir="$(mktemp -d)"
mkdir -p "$work_dir/home" "$work_dir/cfg" "$work_dir/data" "$work_dir/cache" "$work_dir/state" "$work_dir/tmp"
python3 -I - "$work_dir" <<'PYTHON'
from pathlib import Path
import plistlib, sys
work = Path(sys.argv[1])
strings = {"nel": "a" + chr(0x85) + "b", "ls": "a\u2028b", "ps": "a\u2029b", "lf": "a\nb"}
for name, values in [("with-data", dict(strings, payload=b"\x00")), ("without-data", strings)]:
    (work / (name + ".plist")).write_bytes(plistlib.dumps(values, fmt=plistlib.FMT_BINARY))
PYTHON
for name in with-data without-data; do
  env -i PATH="$PATH" HOME="$work_dir/home" XDG_CONFIG_HOME="$work_dir/cfg" \
    XDG_DATA_HOME="$work_dir/data" XDG_CACHE_HOME="$work_dir/cache" \
    XDG_STATE_HOME="$work_dir/state" TMPDIR="$work_dir/tmp" REA_LOG_LEVEL=silent \
    node scripts/rea.mjs inspect-plist "$work_dir/$name.plist" --json > "$work_dir/$name.result.json"
done
plutil -convert xml1 -o "$work_dir/converted.xml" "$work_dir/with-data.plist"
python3 -I - "$work_dir" <<'PYTHON'
from pathlib import Path
import json, plistlib, sys
work = Path(sys.argv[1])
for name in ["with-data", "without-data"]:
    source = plistlib.loads((work / (name + ".plist")).read_bytes())
    result = json.loads((work / (name + ".result.json")).read_text())["normalized_result"]
    print(name, "source:", repr(source))
    print(name, "REA:", json.dumps({"value": result["value"], "limitations": result["limitations"]}, ensure_ascii=True))
print("XML round-trip:", repr(plistlib.loads((work / "converted.xml").read_bytes())))
PYTHON

The verifier ran these commands at the baseline below. I replayed the same reproduction after building current main in an isolated worktree; both CLI results succeeded and the data/no-data comparison reproduced the same failure. These synthetic fixtures require no target execution or network service.

Expected behavior

Preserve the source string values in the normalized JSON result. Adding a sibling data member should not make three distinct Unicode characters become LF. The inspect_plist contract advertises parsing local plists into normalized JSON; the repository's evidence and identity rules require preserving local evidence and making information loss explicit. The independent binary/XML parses and the successful no-data control establish that these are recoverable string values.

Impact and scope

P2: bounded wrong values in inspection evidence for supported binary plists whose data member forces XML fallback. The same trigger reproduced at the verified baseline and current main. Python plistlib directly recovers the source strings, providing an external-inspection workaround; it does not repair REA's result. Frequency in real applications is unknown. MCP, persisted Evidence consumers, other platforms, and date-triggered fallback were not exercised. CR and CRLF are outside this report because an independent XML parse normalizes them too.

REA version

rea-agents 6.7.0, verified source baseline 9d8e7803326a382cbbb3ffdb900f5e93e5442d68. Rechecked after a successful npm run build:cached on fetched main 52c84f6f58fa3cc7b39326654c6d3f261f957957 (also 6.7.0): still reproduces. The lockfile is unchanged between these revisions.

Runtime and platform

Node.js v26.12.0; macOS 27.0.1; arm64; system plutil and Python 3 plistlib. Hopper/Ghidra are not involved.

Target and artifact context

Self-authored binary property lists containing NEL, LINE SEPARATOR, PARAGRAPH SEPARATOR, and an LF control, with/without a one-byte data member. No private target data.

Interface

CLI

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

Current-main CLI result, reduced to the affected value and limitations:

{"value":{"lf":"a\nb","ls":"a\nb","nel":"a\nb","payload":{"$plist_type":"data","base64":"AA=="},"ps":"a\nb"},"limitations":["plutil cannot express this plist as JSON, so it was decoded from plutil's XML conversion; data and date values are typed objects such as { \"$plist_type\": \"data\", \"base64\": ... }."]}

The no-data control returns nel as U+0085 between a and b, ls: "a\u2028b", ps: "a\u2029b", and lf: "a\nb", with limitations: []. Python plistlib reads those same strings from both source binaries and from the data-bearing converted XML. The reproduction prints escaped values so terminal rendering cannot hide the difference. Evidence provenance: observed public CLI and independent source/XML readback; this is not a source-only finding.

Typed outcome or recovery expected

For both fixtures, preserve nel, ls, and ps exactly, keep lf as LF, and retain the one-byte payload as {"$plist_type":"data","base64":"AA=="}. A generic XML-fallback limitation does not disclose changed string values. Merely succeeding or dropping the affected strings would not restore this contract.

Analysis or suspected cause

Source-derived explanation: NativeMacOSProvider.ts falls back from plutil JSON to XML, then parsePlistXml decodes that conversion. propertyListXmlSource.ts explicitly replaces these Unicode separators with LF before DOM parsing; the string branch reads the resulting character content. This points to REA's source normalization, rather than loss in plutil's XML output or JSON serialization. No fix or restoration has been tested.

Additional context

Related closed #4834 concerns negative-zero real identity, including loss during plutil conversion. This report concerns Unicode strings that remain intact in converted XML, so its closure signal is distinct. #4978 concerns data-byte decoding; #4852 concerns invalid scalar literals; open #4047/#4054 concern real CDATA. None covers this valid-string rewrite. Last known good version: NOT_ENOUGH_INFO; none established. Prepared with AI assistance and an independent verifier.

Originally created by @coygeek on GitHub (Oct 20, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/5702 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Native analysis or decompilation ### Observed behavior On macOS, `inspect-plist` silently changes U+0085 (NEL), U+2028 (LINE SEPARATOR), and U+2029 (PARAGRAPH SEPARATOR) inside strings to LF when a binary plist contains a data value and inspection uses the XML fallback. The command succeeds without a limitation describing the changed strings. Removing only the data member makes the same strings survive inspection. Independent Python parses of both the binary source and plutil's converted XML preserve all three characters. ### Reproduction Run from a built source checkout with its locked dependencies installed, using Python 3 and system plutil: ```sh work_dir="$(mktemp -d)" mkdir -p "$work_dir/home" "$work_dir/cfg" "$work_dir/data" "$work_dir/cache" "$work_dir/state" "$work_dir/tmp" python3 -I - "$work_dir" <<'PYTHON' from pathlib import Path import plistlib, sys work = Path(sys.argv[1]) strings = {"nel": "a" + chr(0x85) + "b", "ls": "a\u2028b", "ps": "a\u2029b", "lf": "a\nb"} for name, values in [("with-data", dict(strings, payload=b"\x00")), ("without-data", strings)]: (work / (name + ".plist")).write_bytes(plistlib.dumps(values, fmt=plistlib.FMT_BINARY)) PYTHON for name in with-data without-data; do env -i PATH="$PATH" HOME="$work_dir/home" XDG_CONFIG_HOME="$work_dir/cfg" \ XDG_DATA_HOME="$work_dir/data" XDG_CACHE_HOME="$work_dir/cache" \ XDG_STATE_HOME="$work_dir/state" TMPDIR="$work_dir/tmp" REA_LOG_LEVEL=silent \ node scripts/rea.mjs inspect-plist "$work_dir/$name.plist" --json > "$work_dir/$name.result.json" done plutil -convert xml1 -o "$work_dir/converted.xml" "$work_dir/with-data.plist" python3 -I - "$work_dir" <<'PYTHON' from pathlib import Path import json, plistlib, sys work = Path(sys.argv[1]) for name in ["with-data", "without-data"]: source = plistlib.loads((work / (name + ".plist")).read_bytes()) result = json.loads((work / (name + ".result.json")).read_text())["normalized_result"] print(name, "source:", repr(source)) print(name, "REA:", json.dumps({"value": result["value"], "limitations": result["limitations"]}, ensure_ascii=True)) print("XML round-trip:", repr(plistlib.loads((work / "converted.xml").read_bytes()))) PYTHON ``` The verifier ran these commands at the baseline below. I replayed the same reproduction after building current main in an isolated worktree; both CLI results succeeded and the data/no-data comparison reproduced the same failure. These synthetic fixtures require no target execution or network service. ### Expected behavior Preserve the source string values in the normalized JSON result. Adding a sibling data member should not make three distinct Unicode characters become LF. The [inspect_plist contract](https://github.com/morluto/rea/blob/9d8e7803326a382cbbb3ffdb900f5e93e5442d68/src/contracts/native/nativeToolContracts.ts#L112-L117) advertises parsing local plists into normalized JSON; the repository's [evidence and identity rules](https://github.com/morluto/rea/blob/9d8e7803326a382cbbb3ffdb900f5e93e5442d68/AGENTS.md) require preserving local evidence and making information loss explicit. The independent binary/XML parses and the successful no-data control establish that these are recoverable string values. ### Impact and scope P2: bounded wrong values in inspection evidence for supported binary plists whose data member forces XML fallback. The same trigger reproduced at the verified baseline and current main. Python plistlib directly recovers the source strings, providing an external-inspection workaround; it does not repair REA's result. Frequency in real applications is unknown. MCP, persisted Evidence consumers, other platforms, and date-triggered fallback were not exercised. CR and CRLF are outside this report because an independent XML parse normalizes them too. ### REA version `rea-agents` 6.7.0, verified source baseline `9d8e7803326a382cbbb3ffdb900f5e93e5442d68`. Rechecked after a successful `npm run build:cached` on fetched main `52c84f6f58fa3cc7b39326654c6d3f261f957957` (also 6.7.0): still reproduces. The lockfile is unchanged between these revisions. ### Runtime and platform Node.js v26.12.0; macOS 27.0.1; arm64; system plutil and Python 3 plistlib. Hopper/Ghidra are not involved. ### Target and artifact context Self-authored binary property lists containing NEL, LINE SEPARATOR, PARAGRAPH SEPARATOR, and an LF control, with/without a one-byte data member. No private target data. ### Interface CLI ### Earliest failure stage Native analysis or reconstruction ### Relevant output or evidence Current-main CLI result, reduced to the affected value and limitations: ```json {"value":{"lf":"a\nb","ls":"a\nb","nel":"a\nb","payload":{"$plist_type":"data","base64":"AA=="},"ps":"a\nb"},"limitations":["plutil cannot express this plist as JSON, so it was decoded from plutil's XML conversion; data and date values are typed objects such as { \"$plist_type\": \"data\", \"base64\": ... }."]} ``` The no-data control returns `nel` as U+0085 between `a` and `b`, `ls: "a\u2028b"`, `ps: "a\u2029b"`, and `lf: "a\nb"`, with `limitations: []`. Python plistlib reads those same strings from both source binaries and from the data-bearing converted XML. The reproduction prints escaped values so terminal rendering cannot hide the difference. Evidence provenance: observed public CLI and independent source/XML readback; this is not a source-only finding. ### Typed outcome or recovery expected For both fixtures, preserve `nel`, `ls`, and `ps` exactly, keep `lf` as LF, and retain the one-byte payload as `{"$plist_type":"data","base64":"AA=="}`. A generic XML-fallback limitation does not disclose changed string values. Merely succeeding or dropping the affected strings would not restore this contract. ### Analysis or suspected cause Source-derived explanation: [NativeMacOSProvider.ts](https://github.com/morluto/rea/blob/9d8e7803326a382cbbb3ffdb900f5e93e5442d68/src/native/NativeMacOSProvider.ts#L827-L875) falls back from plutil JSON to XML, then [parsePlistXml](https://github.com/morluto/rea/blob/9d8e7803326a382cbbb3ffdb900f5e93e5442d68/src/native/parsers/plist.ts#L162-L176) decodes that conversion. [propertyListXmlSource.ts](https://github.com/morluto/rea/blob/52c84f6f58fa3cc7b39326654c6d3f261f957957/src/domain/propertyListXmlSource.ts#L10-L32) explicitly replaces these Unicode separators with LF before DOM parsing; [the string branch](https://github.com/morluto/rea/blob/52c84f6f58fa3cc7b39326654c6d3f261f957957/src/domain/propertyListXml.ts#L100-L101) reads the resulting character content. This points to REA's source normalization, rather than loss in plutil's XML output or JSON serialization. No fix or restoration has been tested. ### Additional context Related closed #4834 concerns negative-zero real identity, including loss during plutil conversion. This report concerns Unicode strings that remain intact in converted XML, so its closure signal is distinct. #4978 concerns data-byte decoding; #4852 concerns invalid scalar literals; open #4047/#4054 concern real CDATA. None covers this valid-string rewrite. Last known good version: NOT_ENOUGH_INFO; none established. Prepared with AI assistance and an independent verifier.

bsakaguchi closed this issue

2026-10-31 00:40:38 +00:00

bsakaguchi commented

2026-10-31 00:40:39 +00:00

Author

Owner

@matbrgz commented on GitHub (Oct 20, 2026):

Cross-referencing with canonical issue #3371 (CDATA in XML keyed-archive keys truncates root and field names) and sibling plist decoding reports #3373, #4042, #4714, #4849, #4852, #5701, #5703, #5705.

Consolidated under Cluster K (XML Keyed-Archive Deserialization & CDATA Handling / Plist Decoder) in tracker #4330.

@matbrgz commented on GitHub (Oct 20, 2026): Cross-referencing with canonical issue #3371 (`CDATA in XML keyed-archive keys truncates root and field names`) and sibling plist decoding reports #3373, #4042, #4714, #4849, #4852, #5701, #5703, #5705. Consolidated under **Cluster K (XML Keyed-Archive Deserialization & CDATA Handling / Plist Decoder)** in tracker #4330.

bsakaguchi referenced this issue

2026-10-31 01:09:12 +00:00

[PR #3466] [CLOSED] fix(electron): block app.moveToApplicationsFolder during capture #5757

bsakaguchi referenced this issue

2026-10-31 01:09:14 +00:00

[PR #3471] [MERGED] fix(electron): block moveToApplicationsFolder during active capture #5761