morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #5805] [Bug] Keyed archives round every large integer when two object-table entries share one record #3515

New issue

Closed

opened 2026-10-31 00:41:22 +00:00 by bsakaguchi · 3 comments

bsakaguchi commented

2026-10-31 00:41:22 +00:00

Owner

Originally created by @Shreyash021104 on GitHub (Oct 21, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/5805

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

inspect-keyed-archive reports a source Int64 9007199254740993 as 9007199254740992 with status: "observed" when two object-table entries point at the same integer record. An unrelated sibling Int64 1152921504606846979 (2^60+3) in the same archive is also rounded to 1152921504606847000. The only caveat is the generic "Property-list number metadata could not be completely observed" limitation.

The paired control archive, identical except that the second entry points at its own record, returns exact decimals for both. Python plistlib reads the exact integers from both files.

Reproduction

From a built source checkout of main (d672cc2), reproduced on every run:

work_dir="$(mktemp -d)"
python3 -I - "$work_dir" <<'PY_FIXTURE'
import pathlib, plistlib, sys
root=pathlib.Path(sys.argv[1])
big=2**53+1
src=plistlib.dumps({'$archiver':'NSKeyedArchiver','$version':100000,'$top':{'root':plistlib.UID(1)},
  '$objects':['$null',big,7,2**60+3]},fmt=plistlib.FMT_BINARY)
t=len(src)-32;w=src[t+6];n=int.from_bytes(src[t+8:t+16],'big');tab=int.from_bytes(src[t+24:t+32],'big')
offs=[int.from_bytes(src[tab+i*w:tab+(i+1)*w],'big') for i in range(n)]
big_off=next(o for o in offs if src[o:o+9]==b'\x13'+big.to_bytes(8,'big'))
seven=next(i for i,o in enumerate(offs) if src[o:o+2]==b'\x10\x07')
for dup in [False,True]:
  b=bytearray(src)
  if dup: b[tab+seven*w:tab+(seven+1)*w]=big_off.to_bytes(w,'big')
  p=root/('duplicate.plist' if dup else 'control.plist'); p.write_bytes(b)
  print(p.name, plistlib.loads(bytes(b))['$objects'][1:])
PY_FIXTURE
node scripts/rea.mjs inspect-keyed-archive "$work_dir/control.plist" --json
node scripts/rea.mjs inspect-keyed-archive "$work_dir/duplicate.plist" --json

Python prints [9007199254740993, 9007199254740993, 1152921504606846979] for the duplicate file.

REA objects 1 to 3:

control: {"$plist_type":"integer","decimal":"9007199254740993"}, 7, {"$plist_type":"integer","decimal":"1152921504606846979"}
duplicate: 9007199254740992, 9007199254740992, 1152921504606847000, all observed

Both exit 0. Input bytes are unchanged.

Expected behavior

A repeated object-table offset refers to the same source record, so its facts are known. Preserve exact decimals for that record and for unrelated siblings, as for the control. Keep reporting incomplete metadata for entries that cannot be parsed.

Impact and scope

Incorrect numeric Evidence from a static keyed archive, and loss of exactness for every large integer in the file. Real-world frequency is NOT_ENOUGH_INFO: Apple's writers usually unique objects so each offset appears once, and I have not tested plutil acceptance (no macOS host). The same scanner backs Interface Builder decoding; I did not build a separate Interface Builder reproduction.

REA version

6.7.0 source, main at d672cc2

Runtime and platform

Node.js v24.18.0; Linux x64; Python 3 plistlib

Target and artifact context

Generated binary NSKeyedArchiver archives; control and shared-entry variants with identical integer bytes

Interface

CLI

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

Obtained by running the compiled CLI on the generated files above (objects 1 to 3):

control:   [{"$plist_type":"integer","decimal":"9007199254740993"},7,{"$plist_type":"integer","decimal":"1152921504606846979"}]
duplicate: [9007199254740992,9007199254740992,1152921504606847000]
duplicate limitation: "Property-list number metadata could not be completely observed; decoded numbers may lose integer precision and exact integer/real association is unavailable."

Found with a differential comparison against Python plistlib over 1,600 generated keyed archives; every exactness loss outside genuine integer/real collisions traced to this condition.

Typed outcome or recovery expected

Exact { "$plist_type": "integer", "decimal" } values for the shared record and unrelated siblings, as for the control. Entries that cannot be parsed should still report incomplete metadata.

Analysis or suspected cause

Source-derived: scanBinaryPlistScalars in src/artifacts/apple/PlistScalars.ts sets incomplete = true when an offset was already visited (!validOffsets.has(offset) || visitedOffsets.has(offset)). Incomplete literals make createPlistNumberProjection leave every unsafe integer as its decoded double.

Fix ready on Shreyash021104:fix/plist-shared-object-table-entries; PR to follow.

Additional context

No response

Originally created by @Shreyash021104 on GitHub (Oct 21, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/5805 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior `inspect-keyed-archive` reports a source Int64 `9007199254740993` as `9007199254740992` with `status: "observed"` when two object-table entries point at the same integer record. An unrelated sibling Int64 `1152921504606846979` (2^60+3) in the same archive is also rounded to `1152921504606847000`. The only caveat is the generic "Property-list number metadata could not be completely observed" limitation. The paired control archive, identical except that the second entry points at its own record, returns exact decimals for both. Python `plistlib` reads the exact integers from both files. ### Reproduction From a built source checkout of `main` (`d672cc2`), reproduced on every run: ``` work_dir="$(mktemp -d)" python3 -I - "$work_dir" <<'PY_FIXTURE' import pathlib, plistlib, sys root=pathlib.Path(sys.argv[1]) big=2**53+1 src=plistlib.dumps({'$archiver':'NSKeyedArchiver','$version':100000,'$top':{'root':plistlib.UID(1)}, '$objects':['$null',big,7,2**60+3]},fmt=plistlib.FMT_BINARY) t=len(src)-32;w=src[t+6];n=int.from_bytes(src[t+8:t+16],'big');tab=int.from_bytes(src[t+24:t+32],'big') offs=[int.from_bytes(src[tab+i*w:tab+(i+1)*w],'big') for i in range(n)] big_off=next(o for o in offs if src[o:o+9]==b'\x13'+big.to_bytes(8,'big')) seven=next(i for i,o in enumerate(offs) if src[o:o+2]==b'\x10\x07') for dup in [False,True]: b=bytearray(src) if dup: b[tab+seven*w:tab+(seven+1)*w]=big_off.to_bytes(w,'big') p=root/('duplicate.plist' if dup else 'control.plist'); p.write_bytes(b) print(p.name, plistlib.loads(bytes(b))['$objects'][1:]) PY_FIXTURE node scripts/rea.mjs inspect-keyed-archive "$work_dir/control.plist" --json node scripts/rea.mjs inspect-keyed-archive "$work_dir/duplicate.plist" --json ``` Python prints `[9007199254740993, 9007199254740993, 1152921504606846979]` for the duplicate file. REA objects 1 to 3: - control: `{"$plist_type":"integer","decimal":"9007199254740993"}`, `7`, `{"$plist_type":"integer","decimal":"1152921504606846979"}` - duplicate: `9007199254740992`, `9007199254740992`, `1152921504606847000`, all `observed` Both exit 0. Input bytes are unchanged. ### Expected behavior A repeated object-table offset refers to the same source record, so its facts are known. Preserve exact decimals for that record and for unrelated siblings, as for the control. Keep reporting incomplete metadata for entries that cannot be parsed. ### Impact and scope Incorrect numeric Evidence from a static keyed archive, and loss of exactness for every large integer in the file. Real-world frequency is `NOT_ENOUGH_INFO`: Apple's writers usually unique objects so each offset appears once, and I have not tested `plutil` acceptance (no macOS host). The same scanner backs Interface Builder decoding; I did not build a separate Interface Builder reproduction. ### REA version 6.7.0 source, main at d672cc2 ### Runtime and platform Node.js v24.18.0; Linux x64; Python 3 plistlib ### Target and artifact context Generated binary NSKeyedArchiver archives; control and shared-entry variants with identical integer bytes ### Interface CLI ### Earliest failure stage Native analysis or reconstruction ### Relevant output or evidence Obtained by running the compiled CLI on the generated files above (objects 1 to 3): ``` control: [{"$plist_type":"integer","decimal":"9007199254740993"},7,{"$plist_type":"integer","decimal":"1152921504606846979"}] duplicate: [9007199254740992,9007199254740992,1152921504606847000] duplicate limitation: "Property-list number metadata could not be completely observed; decoded numbers may lose integer precision and exact integer/real association is unavailable." ``` Found with a differential comparison against Python `plistlib` over 1,600 generated keyed archives; every exactness loss outside genuine integer/real collisions traced to this condition. ### Typed outcome or recovery expected Exact `{ "$plist_type": "integer", "decimal" }` values for the shared record and unrelated siblings, as for the control. Entries that cannot be parsed should still report incomplete metadata. ### Analysis or suspected cause Source-derived: `scanBinaryPlistScalars` in `src/artifacts/apple/PlistScalars.ts` sets `incomplete = true` when an offset was already visited (`!validOffsets.has(offset) || visitedOffsets.has(offset)`). Incomplete literals make `createPlistNumberProjection` leave every unsafe integer as its decoded double. Fix ready on `Shreyash021104:fix/plist-shared-object-table-entries`; PR to follow. ### Additional context _No response_

bsakaguchi

2026-10-31 00:41:22 +00:00

closed this issue
added the
bug

triage
labels

bsakaguchi commented

2026-10-31 00:41:23 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 21, 2026):

Reproduced the shape from the fixture — the duplicate-entry archive rounding both the shared Int64 and an unrelated sibling (2^60+3 → 1152921504606847000) while the control stays exact is a clear sign the rounding is happening at a shared-path position (likely Number coercion when the entry is looked up), not per-record. The fix in #5806 keeping scalar facts when object-table entries share a record addresses the root cause. One test worth adding beyond the decimal assertions: assert that the unrelated sibling (2^60+3) stays exact in the duplicate archive too, since the report's key observation is that unrelated entries get corrupted as collateral — that guards against a fix that only special-cases the shared record.

@XinwangTao commented on GitHub (Oct 21, 2026): Reproduced the shape from the fixture — the duplicate-entry archive rounding both the shared Int64 and an unrelated sibling (2^60+3 → 1152921504606847000) while the control stays exact is a clear sign the rounding is happening at a shared-path position (likely Number coercion when the entry is looked up), not per-record. The fix in #5806 keeping scalar facts when object-table entries share a record addresses the root cause. One test worth adding beyond the decimal assertions: assert that the unrelated sibling (2^60+3) stays exact in the duplicate archive too, since the report's key observation is that unrelated entries get corrupted as collateral — that guards against a fix that only special-cases the shared record.

bsakaguchi commented

2026-10-31 00:41:23 +00:00

Author

Owner

@Shreyash021104 commented on GitHub (Oct 21, 2026):

Thanks for reproducing it. The sibling case is already covered in #5806: the boundary test in tests/boundary/filesystem/keyedArchiveNumberMetadata.test.ts (archive reader and compiled CLI) asserts that object 3 stays {"$plist_type":"integer","decimal":"1152921504606846979"}, and the unit test keeps every exact integer when two entries share one Int64 record checks the same through the number projection. Both fail when the source change is reverted.

On the cause: the rounding isn't at lookup time. scanBinaryPlistScalars set incomplete = true on any repeated offset, and incomplete literals make createPlistNumberProjection leave every unsafe integer as its decoded double. That's why unrelated siblings lost exactness too. The fix observes a repeated valid offset once instead, and offsets that fail to parse still mark metadata incomplete.

@Shreyash021104 commented on GitHub (Oct 21, 2026): Thanks for reproducing it. The sibling case is already covered in #5806: the boundary test in `tests/boundary/filesystem/keyedArchiveNumberMetadata.test.ts` (archive reader and compiled CLI) asserts that object 3 stays `{"$plist_type":"integer","decimal":"1152921504606846979"}`, and the unit test `keeps every exact integer when two entries share one Int64 record` checks the same through the number projection. Both fail when the source change is reverted. On the cause: the rounding isn't at lookup time. `scanBinaryPlistScalars` set `incomplete = true` on any repeated offset, and incomplete literals make `createPlistNumberProjection` leave every unsafe integer as its decoded double. That's why unrelated siblings lost exactness too. The fix observes a repeated valid offset once instead, and offsets that fail to parse still mark metadata incomplete.

bsakaguchi commented

2026-10-31 00:41:23 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 22, 2026):

The explanation lands: repeated offsets were setting incomplete=true, and incomplete literals make the number projection fall back to decoded doubles for every unsafe integer — hence the unrelated sibling corruption. Observing a repeated valid offset once (while keeping incomplete for unparseable offsets) fixes the shared record without collateral damage. Glad the sibling case is covered in #5806's boundary and unit tests; that directly guards the report's key observation.

@XinwangTao commented on GitHub (Oct 22, 2026): The explanation lands: repeated offsets were setting incomplete=true, and incomplete literals make the number projection fall back to decoded doubles for every unsafe integer — hence the unrelated sibling corruption. Observing a repeated valid offset once (while keeping incomplete for unparseable offsets) fixes the shared record without collateral damage. Glad the sibling case is covered in #5806's boundary and unit tests; that directly guards the report's key observation.

bsakaguchi referenced this issue

2026-10-31 00:51:01 +00:00

[GH-ISSUE #7661] [Bug] MCP input-error normalization replaces rejected field paths with one root issue #4586

bsakaguchi referenced this issue

2026-10-31 01:09:29 +00:00

[PR #3515] [MERGED] fix(browser): reject overflowing scenario timers #5795

bsakaguchi referenced this issue

2026-10-31 01:11:53 +00:00

[PR #4163] feat(analysis-view): page retained browser scenario steps and text #6103