morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #5638] [Bug] inspect-keyed-archive reports ordinary UID dictionaries as resolved references #3416

New issue

Closed

opened 2026-10-31 00:40:08 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-31 00:40:08 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 20, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/5638

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

inspect-keyed-archive exits 0 but reports ordinary dictionaries inside an object value as resolved UID references. The XML fixture below reports 3 references instead of 2; the binary fixture reports 5 instead of 3. The extra edges point from object 1's authored dictionaries to object 3. Their dictionary values remain visible in the object table; the defect is false graph relationships and inflated reference counts.

This still reproduces after #3913's root-context fix and merged PR #870. This report is limited to authored numeric dictionaries inside object values, not root-name maps.

Reproduction

From a source checkout with locked dependencies and a built CLI (npm ci, npm run build:cached), run the following. Python 3 with plistlib.UID, Node.js, jq, and macOS plutil are required. These commands were replayed against a fresh build of current main; both CLI calls succeeded and reproduced the counts below.

work_dir="$(mktemp -d)"
mkdir -p "$work_dir"/{home,cfg,data,cache,state,tmp}
python3 - "$work_dir" <<'PY'
from pathlib import Path
import plistlib
import sys
root = Path(sys.argv[1])
xml = {
    '$archiver': 'NSKeyedArchiver', '$version': 100000,
    '$top': {'root': {'CF$UID': 1}},
    '$objects': [
        '$null',
        {'$class': {'CF$UID': 2}, 'authoredRecord': {'UID': 3}},
        {'$classname': 'T', '$classes': ['T', 'NSObject']},
        'sharedString',
    ],
}
binary = {
    '$archiver': 'NSKeyedArchiver', '$version': 100000,
    '$top': {'root': plistlib.UID(1)},
    '$objects': [
        '$null',
        {'$class': plistlib.UID(2), 'authoredRecord': {'UID': 3},
         'authoredCFRecord': {'CF$UID': 3}, 'realReference': plistlib.UID(3)},
        {'$classname': 'T', '$classes': ['T', 'NSObject']},
        'sharedString',
    ],
}
for name, archive, fmt in [('xml', xml, plistlib.FMT_XML),
                           ('binary', binary, plistlib.FMT_BINARY)]:
    (root / (name + '.plist')).write_bytes(
        plistlib.dumps(archive, fmt=fmt, sort_keys=False))
PY
for format in xml binary; do
  S="$work_dir"
  env -i PATH="$PATH" HOME=$S/home XDG_CONFIG_HOME=$S/cfg \
    XDG_DATA_HOME=$S/data XDG_CACHE_HOME=$S/cache XDG_STATE_HOME=$S/state \
    TMPDIR=$S/tmp REA_LOG_LEVEL=silent \
    node scripts/rea.mjs inspect-keyed-archive "$work_dir/$format.plist" \
    --json > "$work_dir/$format.json"
  jq '.normalized_result | {archive_format,total_references,references}' \
    "$work_dir/$format.json"
  plutil -p "$work_dir/$format.plist"
done

Expected behavior

Keep ordinary dictionaries as values without generating reference edges. XML has 2 real UID references (root → 1, class → 2). Binary has 3 (root → 1, class → 2, realReference → 3). Retain those genuine edges and their resolved statuses. The tool contract promises original object-table nodes and UID edges while preserving shared/cyclic references.

Impact and scope

P2: callers receive false resolved relationships and inflated reference counts for these particular dictionary shapes. The authored values survive. Both synthetic formats reproduced; real-application prevalence, MCP transport behavior, and broader effects were not tested. No automatic workaround preserving the requested graph was validated; filtering all UID-shaped values would also discard genuine binary references.

REA version

rea-agents 6.7.0. Independently verified baseline: 5b676ba9f302228b44c28ab076525abed3e647c1. Rechecked with a fresh npm run build:cached at current origin/main, 17f581ff75d2b6cf643a997275e3e296f6c95760 (also 6.7.0): still reproduces. The cited parser/projection files are unchanged between these revisions.

Runtime and platform

Node.js v26.12.0; macOS 27.0.1; arm64. Source-checkout CLI with isolated HOME/XDG directories. No Hopper/Ghidra provider involved.

Target and artifact context

Synthetic NSKeyedArchiver XML and binary plists, each with four object-table entries. Binary uses actual plistlib.UID markers alongside authored numeric UID and CF$UID dictionaries. These fixtures were authored with plistlib; no Foundation application serialization is claimed.

Interface

CLI

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

Both CLI commands exit 0. Selected output from normalized_result:

{"archive_format":"xml-plist","total_references":3}
{"source":1,"path":["authoredRecord"],"target":3,"status":"resolved","raw":{"UID":3}}

Binary reports total_references: 5, including that same false edge plus:

{"source":1,"path":["authoredCFRecord"],"target":3,"status":"resolved","raw":{"CF$UID":3}}

The real binary realReference control also resolves to 3, and root/class references resolve to 1/2 in both formats. plutil -p displays authoredRecord as a dictionary in both formats and authoredCFRecord as a dictionary in binary, while displaying genuine markers as CFKeyedArchiverUID. XML CF$UID notation is itself interpreted as a UID by Apple's reader; the ordinary CF$UID dictionary claim applies only to binary input.

Evidence provenance: observed; local verification: reproduced; reproduction completeness: complete for these fixtures.

Typed outcome or recovery expected

Return the same object values and genuine resolved edges, with XML total_references: 2 and binary total_references: 3. Emit no edge for either authored dictionary. No fix or restoration has been tested.

Analysis or suspected cause

Source-derived explanation: uid() and the reference scanner recognize any numeric one-key UID/CF$UID dictionary as a marker; the scanner emits an edge from that shape. The reader passes decoded XML/binary values into this projection. The bundled plist binary decoder represents real UID markers as {UID: n}, colliding with authored dictionaries; REA's shape-based interpretation does not retain serialized marker identity. This explanation corroborates the CLI observation rather than establishing a tested repair.

Additional context

Related #3913 is closed with fix 5a590e61b: it scans each root value separately so a scalar root named UID/CF$UID does not turn its enclosing map into a reference. That fix is present in both tested revisions. PR #870 (merged as cfd2b0c75) restricted marker recognition to numeric values to preserve root maps holding nested UID objects. Its fix is also present. Both root-map cases can pass while the object-value trigger here still fails.

Prepared with AI assistance and independently reproduced. No earlier known-good version was established; this is an uncovered case remaining after those fixes.

Originally created by @coygeek on GitHub (Oct 20, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/5638 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior `inspect-keyed-archive` exits 0 but reports ordinary dictionaries inside an object value as resolved UID references. The XML fixture below reports 3 references instead of 2; the binary fixture reports 5 instead of 3. The extra edges point from object 1's authored dictionaries to object 3. Their dictionary values remain visible in the object table; the defect is false graph relationships and inflated reference counts. This still reproduces after #3913's root-context fix and merged PR #870. This report is limited to authored numeric dictionaries inside object values, not root-name maps. ### Reproduction From a source checkout with locked dependencies and a built CLI (`npm ci`, `npm run build:cached`), run the following. Python 3 with `plistlib.UID`, Node.js, jq, and macOS `plutil` are required. These commands were replayed against a fresh build of current main; both CLI calls succeeded and reproduced the counts below. ```sh work_dir="$(mktemp -d)" mkdir -p "$work_dir"/{home,cfg,data,cache,state,tmp} python3 - "$work_dir" <<'PY' from pathlib import Path import plistlib import sys root = Path(sys.argv[1]) xml = { '$archiver': 'NSKeyedArchiver', '$version': 100000, '$top': {'root': {'CF$UID': 1}}, '$objects': [ '$null', {'$class': {'CF$UID': 2}, 'authoredRecord': {'UID': 3}}, {'$classname': 'T', '$classes': ['T', 'NSObject']}, 'sharedString', ], } binary = { '$archiver': 'NSKeyedArchiver', '$version': 100000, '$top': {'root': plistlib.UID(1)}, '$objects': [ '$null', {'$class': plistlib.UID(2), 'authoredRecord': {'UID': 3}, 'authoredCFRecord': {'CF$UID': 3}, 'realReference': plistlib.UID(3)}, {'$classname': 'T', '$classes': ['T', 'NSObject']}, 'sharedString', ], } for name, archive, fmt in [('xml', xml, plistlib.FMT_XML), ('binary', binary, plistlib.FMT_BINARY)]: (root / (name + '.plist')).write_bytes( plistlib.dumps(archive, fmt=fmt, sort_keys=False)) PY for format in xml binary; do S="$work_dir" env -i PATH="$PATH" HOME=$S/home XDG_CONFIG_HOME=$S/cfg \ XDG_DATA_HOME=$S/data XDG_CACHE_HOME=$S/cache XDG_STATE_HOME=$S/state \ TMPDIR=$S/tmp REA_LOG_LEVEL=silent \ node scripts/rea.mjs inspect-keyed-archive "$work_dir/$format.plist" \ --json > "$work_dir/$format.json" jq '.normalized_result | {archive_format,total_references,references}' \ "$work_dir/$format.json" plutil -p "$work_dir/$format.plist" done ``` ### Expected behavior Keep ordinary dictionaries as values without generating reference edges. XML has 2 real UID references (root → 1, class → 2). Binary has 3 (root → 1, class → 2, `realReference` → 3). Retain those genuine edges and their resolved statuses. The [tool contract](https://github.com/morluto/rea/blob/17f581ff75d2b6cf643a997275e3e296f6c95760/src/contracts/artifactToolContracts.ts#L108) promises original object-table nodes and UID edges while preserving shared/cyclic references. ### Impact and scope P2: callers receive false resolved relationships and inflated reference counts for these particular dictionary shapes. The authored values survive. Both synthetic formats reproduced; real-application prevalence, MCP transport behavior, and broader effects were not tested. No automatic workaround preserving the requested graph was validated; filtering all UID-shaped values would also discard genuine binary references. ### REA version `rea-agents` 6.7.0. Independently verified baseline: `5b676ba9f302228b44c28ab076525abed3e647c1`. Rechecked with a fresh `npm run build:cached` at current `origin/main`, `17f581ff75d2b6cf643a997275e3e296f6c95760` (also 6.7.0): still reproduces. The cited parser/projection files are unchanged between these revisions. ### Runtime and platform Node.js v26.12.0; macOS 27.0.1; arm64. Source-checkout CLI with isolated HOME/XDG directories. No Hopper/Ghidra provider involved. ### Target and artifact context Synthetic NSKeyedArchiver XML and binary plists, each with four object-table entries. Binary uses actual `plistlib.UID` markers alongside authored numeric `UID` and `CF$UID` dictionaries. These fixtures were authored with plistlib; no Foundation application serialization is claimed. ### Interface CLI ### Earliest failure stage Native analysis or reconstruction ### Relevant output or evidence Both CLI commands exit 0. Selected output from `normalized_result`: ```json {"archive_format":"xml-plist","total_references":3} {"source":1,"path":["authoredRecord"],"target":3,"status":"resolved","raw":{"UID":3}} ``` Binary reports `total_references: 5`, including that same false edge plus: ```json {"source":1,"path":["authoredCFRecord"],"target":3,"status":"resolved","raw":{"CF$UID":3}} ``` The real binary `realReference` control also resolves to 3, and root/class references resolve to 1/2 in both formats. `plutil -p` displays `authoredRecord` as a dictionary in both formats and `authoredCFRecord` as a dictionary in binary, while displaying genuine markers as `CFKeyedArchiverUID`. XML `CF$UID` notation is itself interpreted as a UID by Apple's reader; the ordinary `CF$UID` dictionary claim applies only to binary input. Evidence provenance: observed; local verification: reproduced; reproduction completeness: complete for these fixtures. ### Typed outcome or recovery expected Return the same object values and genuine resolved edges, with XML `total_references: 2` and binary `total_references: 3`. Emit no edge for either authored dictionary. No fix or restoration has been tested. ### Analysis or suspected cause Source-derived explanation: [`uid()` and the reference scanner](https://github.com/morluto/rea/blob/17f581ff75d2b6cf643a997275e3e296f6c95760/src/domain/apple/keyedArchive.ts#L84-L93) recognize any numeric one-key `UID`/`CF$UID` dictionary as a marker; the scanner emits an edge from that shape. [The reader](https://github.com/morluto/rea/blob/17f581ff75d2b6cf643a997275e3e296f6c95760/src/artifacts/apple/KeyedArchiveReader.ts#L79-L96) passes decoded XML/binary values into this projection. The bundled `plist` binary decoder represents real UID markers as `{UID: n}`, colliding with authored dictionaries; REA's shape-based interpretation does not retain serialized marker identity. This explanation corroborates the CLI observation rather than establishing a tested repair. ### Additional context Related #3913 is closed with fix `5a590e61b`: it scans each root value separately so a scalar root named `UID`/`CF$UID` does not turn its enclosing map into a reference. That fix is present in both tested revisions. PR #870 (merged as `cfd2b0c75`) restricted marker recognition to numeric values to preserve root maps holding nested UID objects. Its fix is also present. Both root-map cases can pass while the object-value trigger here still fails. Prepared with AI assistance and independently reproduced. No earlier known-good version was established; this is an uncovered case remaining after those fixes.

bsakaguchi closed this issue

2026-10-31 00:40:09 +00:00

bsakaguchi commented

2026-10-31 00:40:09 +00:00

Author

Owner

@matbrgz commented on GitHub (Oct 20, 2026):

Cross-referencing with canonical issue #5635 ([Bug] inspect-asset-catalog omits AppKit NSCustomResource image references). Both address reference extraction, UID dictionary decoding, and asset resolution in macOS application resource tools. Cataloged under Cluster BN in consolidation tracker #4330.

@matbrgz commented on GitHub (Oct 20, 2026): Cross-referencing with canonical issue #5635 (`[Bug] inspect-asset-catalog omits AppKit NSCustomResource image references`). Both address reference extraction, UID dictionary decoding, and asset resolution in macOS application resource tools. Cataloged under Cluster BN in consolidation tracker #4330.

bsakaguchi commented

2026-10-31 00:40:10 +00:00

Author

Owner

@morluto commented on GitHub (Oct 23, 2026):

Fixed on main in github.com/morluto/rea@91c28e2735. XML decoding now distinguishes an authored UID dictionary from a genuine CF$UID reference, and parsed reference identity survives key omission and numeric projection. Genuine binary UID records and ordinary binary UID-shaped dictionaries remain distinct; unsafe authored integers retain their exact value.

Validation: the original XML control failed on the baseline (3 references instead of 2). The fixed producer/domain group passed 322 checks; the final reader file passed all 35, including key-omission and genuine-reference controls. Removing identity transfer deliberately fails the new regression. The 12-case compiled CLI/MCP XML acceptance suite also passed. This does not claim real Interface Builder application qualification.

@morluto commented on GitHub (Oct 23, 2026): Fixed on main in https://github.com/morluto/rea/commit/91c28e2735565f56df3c0564884489c17d882fbe. XML decoding now distinguishes an authored `UID` dictionary from a genuine `CF$UID` reference, and parsed reference identity survives key omission and numeric projection. Genuine binary UID records and ordinary binary UID-shaped dictionaries remain distinct; unsafe authored integers retain their exact value. Validation: the original XML control failed on the baseline (3 references instead of 2). The fixed producer/domain group passed 322 checks; the final reader file passed all 35, including key-omission and genuine-reference controls. Removing identity transfer deliberately fails the new regression. The 12-case compiled CLI/MCP XML acceptance suite also passed. This does not claim real Interface Builder application qualification.

bsakaguchi referenced this issue

2026-10-31 01:08:59 +00:00

[PR #3416] [MERGED] test(browser): preserve resource uncertainty through MCP #5729

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#3416

No description provided.

Rows
Columns