morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #8904] [Bug] inspect-keyed-archive loses unrelated integer precision when a string follows the offset table #7785

New issue

Open

opened 2026-11-04 14:13:25 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-11-04 14:13:25 +00:00

Owner

Originally created by @coygeek on GitHub (Nov 4, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/8904

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

rea inspect-keyed-archive <archive> --format json returns a pre-table integer 9007199254740993 as the rounded JSON number 9007199254740992 when a complete string object is placed after the offset table. Object 2 retains status: "observed". The same integer remains an exact decimal wrapper when the string's entry points to its original pre-table location.

Both inputs decode successfully and preserve hello. The failing result adds the generic limitation Property-list number metadata could not be completely observed; decoded numbers may lose integer precision and exact integer/real association is unavailable. All integer bytes remain before the table; the post-table object is only a string.

Observed through the unmodified built source CLI and the separately installed published 6.8.0 CLI. Each pre/post/inert comparison reproduced twice per runtime; all calls exited 0 and input hashes remained unchanged.

Reproduction

From a built checkout of the version below, put Node.js 24.22.1 first on PATH and use Python 3's standard library. Set REA_REPRO_DIR to an empty disposable directory outside the checkout. This script was replayed through the real CLI; HOME and configuration paths stay within that directory.

set -eu
work_dir="${REA_REPRO_DIR:?Set REA_REPRO_DIR to a disposable directory outside the checkout}"
mkdir -p "$work_dir/home"
export HOME="$work_dir/home"
export XDG_CONFIG_HOME="$HOME/config" XDG_CACHE_HOME="$HOME/cache" XDG_STATE_HOME="$HOME/state"
python3 - "$work_dir" <<'PY'
import pathlib, plistlib, struct, sys
root = pathlib.Path(sys.argv[1])
archive = {'$archiver': 'NSKeyedArchiver', '$version': 100000,
           '$objects': ['$null', 'hello', 9007199254740993],
           '$top': {'root': plistlib.UID(2)}}
pre = plistlib.dumps(archive, fmt=plistlib.FMT_BINARY, sort_keys=False)
width, refs, count, top, table = struct.unpack('>6xBBQQQ', pre[-32:])
offsets = [int.from_bytes(pre[table+i*width:table+(i+1)*width], 'big') for i in range(count)]
string_id = next(i for i, offset in enumerate(offsets) if pre[offset:offset+6] == b'\x55hello')
post = bytearray(pre[:-32] + b'\x55hello' + pre[-32:])
post[table+string_id*width:table+(string_id+1)*width] = (len(pre)-32).to_bytes(width, 'big')
inert = pre[:-32] + b'\x55hello' + pre[-32:]
for name, data in [('pre', pre), ('post', bytes(post)), ('inert', inert)]:
    assert plistlib.loads(data) == archive
    (root / (name + '.bplist')).write_bytes(data)
PY
for case in pre post inert; do
  node scripts/rea.mjs inspect-keyed-archive "$work_dir/$case.bplist" --format json > "$work_dir/$case.json"
done
python3 - "$work_dir" <<'PY'
import json, pathlib, sys
root = pathlib.Path(sys.argv[1])
for case in ['pre', 'post', 'inert']:
    result = json.loads((root / (case + '.json')).read_text())['normalized_result']
    print(case, json.dumps(result['objects'][2]['value']))
    print('incomplete:', any('could not be completely observed' in item for item in result['limitations']))
PY

Observed output:

pre {"$plist_type": "integer", "decimal": "9007199254740993"}
incomplete: False
post 9007199254740992
incomplete: True
inert {"$plist_type": "integer", "decimal": "9007199254740993"}
incomplete: False

Python assertions independently check identical archive values, including the exact integer. The post file appends a complete six-byte string after the table and redirects only its offset entry. The inert control appends the same bytes without redirecting that entry, isolating the effect of an addressed post-table object from file length or spare bytes.

Expected behavior

Preserve the exact decimal for the unambiguous pre-table integer, as in both controls. Admitting a complete nonnumeric string should not disable known source-number facts for unrelated integers. Actually truncated or ambiguous numeric records should retain their existing refusal or uncertainty behavior.

Expected source: docs/native-investigation.md:1287-1301 documents original decimal text for unsafe integers whose source values can be associated unambiguously. The post-table string acceptance test at tests/acceptance/applications/keyedArchivePostTable.test.ts:7-59 and merged PR #7820 establish admission of that string layout. This expectation preserves the rule at docs/native-investigation.md:1354-1355 that numeric and date payloads must end before the table.

Impact and scope

P2: bounded loss of exact numeric Evidence, with a generic precision warning. A complete admitted nonnumeric object causes unrelated, fully observed integer facts to be discarded. The original archive is preserved and Python recovers the exact integer. An identity-preserving REA recovery was not verified; the passing controls change artifact bytes.

The demonstrated archive has one unsafe Int64, one string, genuine UID references, distinct offsets, and no integer/real collision. Real-exporter prevalence and other consumer workflows are NOT_ENOUGH_INFO. Separate CLI controls also showed the incomplete warning for post-table UIDs and booleans; the numeric-loss reproduction uses a string. No target execution, mutation, or security-boundary effect is involved.

REA version

Published rea-agents 6.8.0 (--version: 6.8.0), and built source c3647ade1e557c4179a8b825a37e0c2193cd2526 (package 6.8.0). The published package was installed with scripts disabled and executed separately. Freshly fetched main 89317a39fab23df81efa4331311d09d2110377bd has identical affected source blobs; main was not rebuilt or executed.

Runtime and platform

Node.js v24.22.1; Darwin 27.2.0 arm64. Python 3 standard-library plistlib generated and independently checked fixtures. No Hopper or Ghidra provider is required.

Target and artifact context

Inert binary NSKeyedArchiver envelope with $objects: ["$null", "hello", 9007199254740993], $top.root: UID(2), and $version: 100000. The control is 154 bytes; post-table and inert variants are each 160 bytes. The string ends before the trailer; every numeric payload ends before the table.

Interface

CLI

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

Actual normalized_result.objects[2], obtained through the CLI commands above:

{"id":2,"kind":"scalar","class_id":null,"class_name":null,"status":"observed","value":{"$plist_type":"integer","decimal":"9007199254740993"}}

Post-table string case:

{"id":2,"kind":"scalar","class_id":null,"class_name":null,"status":"observed","value":9007199254740992}

The failing result includes the incomplete-number limitation and 1 number(s) beyond the exact integer range remain as decoded because their source integer/real type or reference marker cannot be associated unambiguously; integer precision loss remains possible. Controls instead count one unambiguous integer with exact decimal text.

Fixture SHA-256 values, unchanged after repeated CLI calls:

pre: 62f390b2262c0bd30ff4d3a29d191a55b9a0b8efa51805d2439b43c4c035e0da
post: 0228d5872d9ef7ed2d84681b1badbbfc8a7bd6312e8aab7aa598be8674805f10
inert: f2588bbca623eb06a4816020a4b135e628200e42ef8a8abfaec3a7efafc54847

Source-derived reachability: CLI registration at src/cli/artifactCommands.ts:42-83 reaches src/artifacts/apple/KeyedArchiveReader.ts:242-319. The actual fixture passes selection, stable-read, digest, envelope, and resource guards. Decoder bounds admit the string, but src/artifacts/apple/PlistScalars.ts:277-279 marks metadata incomplete for every offset at or after the table. src/domain/apple/plistNumbers.ts:64 makes the unrelated integer's association unknown. This explains, rather than substitutes for, the observed CLI comparison.

Typed outcome or recovery expected

Object 2 should retain { "$plist_type": "integer", "decimal": "9007199254740993" }. A complete post-table string should not add an incomplete-number limitation or disable exact decimals. Keep archive identity, root UID, object indices, pagination, and source bytes intact.

Analysis or suspected cause

Source-derived: the keyed-archive preflight at src/artifacts/apple/KeyedArchiveReader.ts:95-109 selects decoder bounds and admits the string. The scanner at src/artifacts/apple/PlistScalars.ts:277-279 skips that offset and sets a global incomplete flag before checking its nonnumeric type. The pre-table integer is still collected, but resolvePlistNumberSource at src/domain/apple/plistNumbers.ts:64 discards its association whenever that flag is set. Projection at KeyedArchiveReader.ts:129-140 leaves the rounded decoder number.

This report does not request admitting post-table numeric/date payloads: their refusal is documented and was observed in negative controls. Post-table UID 1 succeeded, so UID rejection is not part of this finding.

Additional context

Closed #7348 and merged #7820 fixed relocation changing a post-table string into null. Here the string is preserved; an unrelated integer still loses exactness.

Closed #5805 and merged #5806 fixed archive-wide precision loss from shared offsets. This reproduction has distinct offsets and a different cause for the global incomplete fallback. Closed #5792 explains intentional refusal of numeric payloads reading table bytes.

Open #7166 concerns tables reaching the trailer and preserving objects in the old trailer. This reproduction has no table/trailer overlap. Searches of open and closed issues and PRs found no current exact duplicate for this trigger.

Native Foundation acceptance, real-producer prevalence, MCP, Python bridge, and Interface Builder behavior were not tested. No current-main runtime execution or full-suite result is claimed.

Originally created by @coygeek on GitHub (Nov 4, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/8904 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior `rea inspect-keyed-archive <archive> --format json` returns a pre-table integer `9007199254740993` as the rounded JSON number `9007199254740992` when a complete string object is placed after the offset table. Object 2 retains `status: "observed"`. The same integer remains an exact decimal wrapper when the string's entry points to its original pre-table location. Both inputs decode successfully and preserve `hello`. The failing result adds the generic limitation `Property-list number metadata could not be completely observed; decoded numbers may lose integer precision and exact integer/real association is unavailable.` All integer bytes remain before the table; the post-table object is only a string. Observed through the unmodified built source CLI and the separately installed published 6.8.0 CLI. Each pre/post/inert comparison reproduced twice per runtime; all calls exited 0 and input hashes remained unchanged. ### Reproduction From a built checkout of the version below, put Node.js 24.22.1 first on PATH and use Python 3's standard library. Set `REA_REPRO_DIR` to an empty disposable directory outside the checkout. This script was replayed through the real CLI; HOME and configuration paths stay within that directory. ```sh set -eu work_dir="${REA_REPRO_DIR:?Set REA_REPRO_DIR to a disposable directory outside the checkout}" mkdir -p "$work_dir/home" export HOME="$work_dir/home" export XDG_CONFIG_HOME="$HOME/config" XDG_CACHE_HOME="$HOME/cache" XDG_STATE_HOME="$HOME/state" python3 - "$work_dir" <<'PY' import pathlib, plistlib, struct, sys root = pathlib.Path(sys.argv[1]) archive = {'$archiver': 'NSKeyedArchiver', '$version': 100000, '$objects': ['$null', 'hello', 9007199254740993], '$top': {'root': plistlib.UID(2)}} pre = plistlib.dumps(archive, fmt=plistlib.FMT_BINARY, sort_keys=False) width, refs, count, top, table = struct.unpack('>6xBBQQQ', pre[-32:]) offsets = [int.from_bytes(pre[table+i*width:table+(i+1)*width], 'big') for i in range(count)] string_id = next(i for i, offset in enumerate(offsets) if pre[offset:offset+6] == b'\x55hello') post = bytearray(pre[:-32] + b'\x55hello' + pre[-32:]) post[table+string_id*width:table+(string_id+1)*width] = (len(pre)-32).to_bytes(width, 'big') inert = pre[:-32] + b'\x55hello' + pre[-32:] for name, data in [('pre', pre), ('post', bytes(post)), ('inert', inert)]: assert plistlib.loads(data) == archive (root / (name + '.bplist')).write_bytes(data) PY for case in pre post inert; do node scripts/rea.mjs inspect-keyed-archive "$work_dir/$case.bplist" --format json > "$work_dir/$case.json" done python3 - "$work_dir" <<'PY' import json, pathlib, sys root = pathlib.Path(sys.argv[1]) for case in ['pre', 'post', 'inert']: result = json.loads((root / (case + '.json')).read_text())['normalized_result'] print(case, json.dumps(result['objects'][2]['value'])) print('incomplete:', any('could not be completely observed' in item for item in result['limitations'])) PY ``` Observed output: ```text pre {"$plist_type": "integer", "decimal": "9007199254740993"} incomplete: False post 9007199254740992 incomplete: True inert {"$plist_type": "integer", "decimal": "9007199254740993"} incomplete: False ``` Python assertions independently check identical archive values, including the exact integer. The post file appends a complete six-byte string after the table and redirects only its offset entry. The inert control appends the same bytes without redirecting that entry, isolating the effect of an addressed post-table object from file length or spare bytes. ### Expected behavior Preserve the exact decimal for the unambiguous pre-table integer, as in both controls. Admitting a complete nonnumeric string should not disable known source-number facts for unrelated integers. Actually truncated or ambiguous numeric records should retain their existing refusal or uncertainty behavior. Expected source: `docs/native-investigation.md:1287-1301` documents original decimal text for unsafe integers whose source values can be associated unambiguously. The post-table string acceptance test at `tests/acceptance/applications/keyedArchivePostTable.test.ts:7-59` and merged PR #7820 establish admission of that string layout. This expectation preserves the rule at `docs/native-investigation.md:1354-1355` that numeric and date payloads must end before the table. ### Impact and scope P2: bounded loss of exact numeric Evidence, with a generic precision warning. A complete admitted nonnumeric object causes unrelated, fully observed integer facts to be discarded. The original archive is preserved and Python recovers the exact integer. An identity-preserving REA recovery was not verified; the passing controls change artifact bytes. The demonstrated archive has one unsafe Int64, one string, genuine UID references, distinct offsets, and no integer/real collision. Real-exporter prevalence and other consumer workflows are `NOT_ENOUGH_INFO`. Separate CLI controls also showed the incomplete warning for post-table UIDs and booleans; the numeric-loss reproduction uses a string. No target execution, mutation, or security-boundary effect is involved. ### REA version Published `rea-agents` 6.8.0 (`--version`: `6.8.0`), and built source `c3647ade1e557c4179a8b825a37e0c2193cd2526` (package 6.8.0). The published package was installed with scripts disabled and executed separately. Freshly fetched main `89317a39fab23df81efa4331311d09d2110377bd` has identical affected source blobs; main was not rebuilt or executed. ### Runtime and platform Node.js v24.22.1; Darwin 27.2.0 arm64. Python 3 standard-library `plistlib` generated and independently checked fixtures. No Hopper or Ghidra provider is required. ### Target and artifact context Inert binary NSKeyedArchiver envelope with `$objects: ["$null", "hello", 9007199254740993]`, `$top.root: UID(2)`, and `$version: 100000`. The control is 154 bytes; post-table and inert variants are each 160 bytes. The string ends before the trailer; every numeric payload ends before the table. ### Interface CLI ### Earliest failure stage Native analysis or reconstruction ### Relevant output or evidence Actual `normalized_result.objects[2]`, obtained through the CLI commands above: ```json {"id":2,"kind":"scalar","class_id":null,"class_name":null,"status":"observed","value":{"$plist_type":"integer","decimal":"9007199254740993"}} ``` Post-table string case: ```json {"id":2,"kind":"scalar","class_id":null,"class_name":null,"status":"observed","value":9007199254740992} ``` The failing result includes the incomplete-number limitation and `1 number(s) beyond the exact integer range remain as decoded because their source integer/real type or reference marker cannot be associated unambiguously; integer precision loss remains possible.` Controls instead count one unambiguous integer with exact decimal text. Fixture SHA-256 values, unchanged after repeated CLI calls: - pre: `62f390b2262c0bd30ff4d3a29d191a55b9a0b8efa51805d2439b43c4c035e0da` - post: `0228d5872d9ef7ed2d84681b1badbbfc8a7bd6312e8aab7aa598be8674805f10` - inert: `f2588bbca623eb06a4816020a4b135e628200e42ef8a8abfaec3a7efafc54847` Source-derived reachability: CLI registration at `src/cli/artifactCommands.ts:42-83` reaches `src/artifacts/apple/KeyedArchiveReader.ts:242-319`. The actual fixture passes selection, stable-read, digest, envelope, and resource guards. Decoder bounds admit the string, but `src/artifacts/apple/PlistScalars.ts:277-279` marks metadata incomplete for every offset at or after the table. `src/domain/apple/plistNumbers.ts:64` makes the unrelated integer's association unknown. This explains, rather than substitutes for, the observed CLI comparison. ### Typed outcome or recovery expected Object 2 should retain `{ "$plist_type": "integer", "decimal": "9007199254740993" }`. A complete post-table string should not add an incomplete-number limitation or disable exact decimals. Keep archive identity, root UID, object indices, pagination, and source bytes intact. ### Analysis or suspected cause Source-derived: the keyed-archive preflight at `src/artifacts/apple/KeyedArchiveReader.ts:95-109` selects decoder bounds and admits the string. The scanner at `src/artifacts/apple/PlistScalars.ts:277-279` skips that offset and sets a global incomplete flag before checking its nonnumeric type. The pre-table integer is still collected, but `resolvePlistNumberSource` at `src/domain/apple/plistNumbers.ts:64` discards its association whenever that flag is set. Projection at `KeyedArchiveReader.ts:129-140` leaves the rounded decoder number. This report does not request admitting post-table numeric/date payloads: their refusal is documented and was observed in negative controls. Post-table UID 1 succeeded, so UID rejection is not part of this finding. ### Additional context Closed [#7348](https://github.com/morluto/rea/issues/7348) and merged [#7820](https://github.com/morluto/rea/pull/7820) fixed relocation changing a post-table string into null. Here the string is preserved; an unrelated integer still loses exactness. Closed [#5805](https://github.com/morluto/rea/issues/5805) and merged [#5806](https://github.com/morluto/rea/pull/5806) fixed archive-wide precision loss from shared offsets. This reproduction has distinct offsets and a different cause for the global incomplete fallback. Closed [#5792](https://github.com/morluto/rea/issues/5792) explains intentional refusal of numeric payloads reading table bytes. Open [#7166](https://github.com/morluto/rea/pull/7166) concerns tables reaching the trailer and preserving objects in the old trailer. This reproduction has no table/trailer overlap. Searches of open and closed issues and PRs found no current exact duplicate for this trigger. Native Foundation acceptance, real-producer prevalence, MCP, Python bridge, and Interface Builder behavior were not tested. No current-main runtime execution or full-suite result is claimed.

bsakaguchi referenced this issue

2026-11-05 01:47:58 +00:00

[PR #7356] [CLOSED] fix(xml): reject ambiguous namespace attribute identities #7828

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#7785

No description provided.

Rows
Columns