morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #4292] [Bug] Binary keyed archives report integers above Int64.max as 0 #1697

New issue

Closed

opened 2026-10-16 21:09:12 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-16 21:09:12 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 16, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/4292

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

inspect_keyed_archive reports an unsigned integer above Int64.max (for example UInt64.max, 18446744073709551615) stored in a binary keyed archive as value: 0 with status: "observed". Apple's binary property-list writer stores such values as 16-byte integers (marker 0x14). REA's only caveat is the generic note that numbers "may lose integer precision", which does not cover replacing the value with zero. The same 16-byte integer also turns off exact decimal reporting for every other large integer in the archive, so Int64.max in the same file comes back rounded to 9223372036854776000. The XML form of the same archive is decoded exactly.

Observed on the release build. The binary archive (archive_format: "binary-plist", exit status 0) returns:

{"id":1,"kind":"scalar","class_id":null,"class_name":null,"status":"observed","value":0}
{"id":2,"kind":"scalar","class_id":null,"class_name":null,"status":"observed","value":9223372036854776000}

Its numeric limitations are Original archive number metadata could not be completely observed; decoded numbers may lose integer precision and exact integer/real association is unavailable. and 1 number(s) beyond the exact integer range remain as decoded because their source integer/real type or reference marker cannot be associated unambiguously; integer precision loss remains possible. That count of 1 covers only object 2. Object 1, the zeroed value, is not counted.

The XML conversion of the same archive (archive_format: "xml-plist", exit status 0) returns {"$plist_type":"integer","decimal":"18446744073709551615"} and {"$plist_type":"integer","decimal":"9223372036854775807"}, with the limitation 2 unambiguous integer value(s) are reported with exact decimal text.

In the control, a binary archive with only Int64.max returns {"$plist_type":"integer","decimal":"9223372036854775807"}. So the binary path handles 8-byte integers exactly until a 16-byte integer appears in the same archive.

The same 0 came back for an archive that Foundation's NSKeyedArchiver wrote for NSNumber(value: UInt64.max) (bytes 14 0000000000000000 ffffffffffffffff). Both runs were consistent across reruns.

Reproduction

Use the release build stated under Runtime and platform. Each step uses only Python's standard plistlib and the stock plutil:

mkdir -p /tmp/rea-bug01 && cd /tmp/rea-bug01
python3 -I -c "import plistlib; d={'\$archiver':'NSKeyedArchiver','\$version':100000,'\$top':{'root':plistlib.UID(1),'max':plistlib.UID(2)},'\$objects':['\$null',18446744073709551615,9223372036854775807]}; open('archive.bin','wb').write(plistlib.dumps(d,fmt=plistlib.FMT_BINARY))"
plutil -p archive.bin
plutil -convert xml1 -o archive.xml archive.bin
npx -y rea-agents@6.5.0 inspect-keyed-archive ./archive.bin --json
npx -y rea-agents@6.5.0 inspect-keyed-archive ./archive.xml --json

Control, a binary archive that holds only Int64.max:

python3 -I -c "import plistlib; d={'\$archiver':'NSKeyedArchiver','\$version':100000,'\$top':{'root':plistlib.UID(1)},'\$objects':['\$null',9223372036854775807]}; open('control.bin','wb').write(plistlib.dumps(d,fmt=plistlib.FMT_BINARY))"
npx -y rea-agents@6.5.0 inspect-keyed-archive ./control.bin --json

Expected behavior

The "Keyed archive integer precision" section of REA's native investigation guide says integers beyond the exact range of a JSON number "are reported with the original decimal text as { "$plist_type": "integer", "decimal": "<exact digits>" } when that source value can be associated unambiguously", and that otherwise numbers "remain decoded with an explicit precision limitation". The guide's inspect_keyed_archive entry says results retain "raw serialized fields". A 16-byte integer marker in a binary plist is an unambiguous integer. plutil -p reads object 1 as 18446744073709551615 and object 2 as 9223372036854775807. REA should therefore report object 1 as {"$plist_type":"integer","decimal":"18446744073709551615"} and object 2 as {"$plist_type":"integer","decimal":"9223372036854775807"}, as it already does for the XML form of the same archive. At minimum, object 1 must not be reported as an observed 0. If REA cannot decode the value, it should mark the value unknown and add a limitation that names it.

Impact and scope

Any binary keyed archive that holds an unsigned 64-bit value above Int64.max (for example UInt64 counters, bit masks or hashes archived through NSNumber) is reported with a fabricated value of zero marked as observed. Other large integers in the same archive lose their exact decimal form. Analysts who rely on the Evidence get a wrong value and no signal that this value is the one affected. inspect_plist is not affected because it reads values through plutil.

Workaround: Within REA Evidence, rea inspect-plist ./archive.bin --relative-path "$PWD/archive.bin" --json reads the same binary file and returns format: "binary" with exact decimal text for both objects. Converting the archive with plutil -convert xml1 and running inspect-keyed-archive on the XML copy also gives exact values. plutil -p shows the correct values outside REA.

REA version

6.5.0 (tag rea-agents-6.5.0; the relevant code is unchanged on main)

Runtime and platform

REA: rea-agents 6.5.0 from npm (--version prints 6.5.0), release tag rea-agents-6.5.0 at commit 3b79af28e80785afce9f5998d8f03289402a1847.
Host: macOS 27.0.1 (26A434) arm64, Node.js v26.10.0, system plutil.
Main at d52a488 leaves the relevant code unchanged: PlistNumberLiterals.ts is identical, and the only change to KeyedArchiveReader.ts adds date limitations.

Target and artifact context

self-built binary NSKeyedArchiver plist whose $objects holds 18446744073709551615 (16-byte integer) and 9223372036854775807; XML conversion and an Int64.max-only archive as controls

Interface

CLI

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

Expected source: REA docs/native-investigation.md at the release tag, section "Keyed archive integer precision" (L550-L560) and the inspect_keyed_archive entry (L311-L323), plus Apple's plutil -p reading of the fixture (18446744073709551615).
Failure source: rea inspect-keyed-archive ./archive.bin --json on the release build, normalized_result.objects[1] = {"status":"observed","value":0}.
Evidence provenance: observed
Local verification: reproduced
Reproduction completeness: complete

This investigator ran the fixture above on the release build, together with the XML and Int64.max-only controls.

Typed outcome or recovery expected

With the fixture above, inspect-keyed-archive ./archive.bin --json must stop reporting object 1 as an observed 0. It should report {"$plist_type":"integer","decimal":"18446744073709551615"}, as the XML form does. If REA reports any other representation, it must flag the value as not exactly observed in a limitation that counts it. Object 2 must keep its exact decimal text or carry a limitation that counts it. Discriminating controls: the XML form and the Int64.max-only binary archive must keep returning exact decimal text.

Analysis or suspected cause

No response

Additional context

Suspected cause, from reading the source at the release tag:

src/artifacts/apple/KeyedArchiveReader.ts decodes binary archives with parseBinary from the pinned plist 5.0.0 package (L3, L72). For any integer wider than 4 bytes, that decoder reads only the first 8 bytes as hi * 2^32 + lo. For a 16-byte big-endian value those are the high 8 bytes, which are zero for every value below 2^64. Running parseBinary directly on the fixture returns "$objects":["$null",0,9223372036854776000]. plist 5.0.0 is the latest npm release, so this may also be worth reporting upstream to TooTallNate/plist.js. That is a suspected cause, not a verified upstream report.
src/artifacts/apple/PlistNumberLiterals.ts records exact text for 8-byte integers but returns "incomplete" for any wider integer (L162-L164). That explains why one 16-byte integer turns off exact association for the whole archive and why the zeroed value gets only the generic precision note.

Related but different issues:

#3373: XML hexadecimal integers decode as 0. That is a different encoding and parser path.
#4042: XML CDATA reals lose their content.
#3876 / PR #3972: decode_interface_builder rounds integers above 2^53. That is rounding, not replacement with 0, and it happens in a different tool.
The exact-decimal projection this report relies on was proposed in closed PR #1135. Its closing comment says it landed in commit 4034883.

Originally created by @coygeek on GitHub (Oct 16, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/4292 ### Before submitting - [X] I searched the open and closed issues for a related report. - [X] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior `inspect_keyed_archive` reports an unsigned integer above `Int64.max` (for example `UInt64.max`, `18446744073709551615`) stored in a binary keyed archive as `value: 0` with `status: "observed"`. Apple's binary property-list writer stores such values as 16-byte integers (marker `0x14`). REA's only caveat is the generic note that numbers "may lose integer precision", which does not cover replacing the value with zero. The same 16-byte integer also turns off exact decimal reporting for every other large integer in the archive, so `Int64.max` in the same file comes back rounded to `9223372036854776000`. The XML form of the same archive is decoded exactly. Observed on the release build. The binary archive (`archive_format: "binary-plist"`, exit status 0) returns: ```json {"id":1,"kind":"scalar","class_id":null,"class_name":null,"status":"observed","value":0} {"id":2,"kind":"scalar","class_id":null,"class_name":null,"status":"observed","value":9223372036854776000} ``` Its numeric limitations are `Original archive number metadata could not be completely observed; decoded numbers may lose integer precision and exact integer/real association is unavailable.` and `1 number(s) beyond the exact integer range remain as decoded because their source integer/real type or reference marker cannot be associated unambiguously; integer precision loss remains possible.` That count of 1 covers only object 2. Object 1, the zeroed value, is not counted. The XML conversion of the same archive (`archive_format: "xml-plist"`, exit status 0) returns `{"$plist_type":"integer","decimal":"18446744073709551615"}` and `{"$plist_type":"integer","decimal":"9223372036854775807"}`, with the limitation `2 unambiguous integer value(s) are reported with exact decimal text`. In the control, a binary archive with only `Int64.max` returns `{"$plist_type":"integer","decimal":"9223372036854775807"}`. So the binary path handles 8-byte integers exactly until a 16-byte integer appears in the same archive. The same `0` came back for an archive that Foundation's `NSKeyedArchiver` wrote for `NSNumber(value: UInt64.max)` (bytes `14 0000000000000000 ffffffffffffffff`). Both runs were consistent across reruns. ### Reproduction Use the release build stated under Runtime and platform. Each step uses only Python's standard `plistlib` and the stock `plutil`: ```sh mkdir -p /tmp/rea-bug01 && cd /tmp/rea-bug01 python3 -I -c "import plistlib; d={'\$archiver':'NSKeyedArchiver','\$version':100000,'\$top':{'root':plistlib.UID(1),'max':plistlib.UID(2)},'\$objects':['\$null',18446744073709551615,9223372036854775807]}; open('archive.bin','wb').write(plistlib.dumps(d,fmt=plistlib.FMT_BINARY))" plutil -p archive.bin plutil -convert xml1 -o archive.xml archive.bin npx -y rea-agents@6.5.0 inspect-keyed-archive ./archive.bin --json npx -y rea-agents@6.5.0 inspect-keyed-archive ./archive.xml --json ``` Control, a binary archive that holds only `Int64.max`: ```sh python3 -I -c "import plistlib; d={'\$archiver':'NSKeyedArchiver','\$version':100000,'\$top':{'root':plistlib.UID(1)},'\$objects':['\$null',9223372036854775807]}; open('control.bin','wb').write(plistlib.dumps(d,fmt=plistlib.FMT_BINARY))" npx -y rea-agents@6.5.0 inspect-keyed-archive ./control.bin --json ``` ### Expected behavior The "Keyed archive integer precision" section of REA's native investigation guide says integers beyond the exact range of a JSON number "are reported with the original decimal text as `{ "$plist_type": "integer", "decimal": "<exact digits>" }` when that source value can be associated unambiguously", and that otherwise numbers "remain decoded with an explicit precision limitation". The guide's `inspect_keyed_archive` entry says results retain "raw serialized fields". A 16-byte integer marker in a binary plist is an unambiguous integer. `plutil -p` reads object 1 as `18446744073709551615` and object 2 as `9223372036854775807`. REA should therefore report object 1 as `{"$plist_type":"integer","decimal":"18446744073709551615"}` and object 2 as `{"$plist_type":"integer","decimal":"9223372036854775807"}`, as it already does for the XML form of the same archive. At minimum, object 1 must not be reported as an observed `0`. If REA cannot decode the value, it should mark the value unknown and add a limitation that names it. ### Impact and scope Any binary keyed archive that holds an unsigned 64-bit value above `Int64.max` (for example `UInt64` counters, bit masks or hashes archived through `NSNumber`) is reported with a fabricated value of zero marked as observed. Other large integers in the same archive lose their exact decimal form. Analysts who rely on the Evidence get a wrong value and no signal that this value is the one affected. `inspect_plist` is not affected because it reads values through `plutil`. Workaround: Within REA Evidence, `rea inspect-plist ./archive.bin --relative-path "$PWD/archive.bin" --json` reads the same binary file and returns `format: "binary"` with exact decimal text for both objects. Converting the archive with `plutil -convert xml1` and running `inspect-keyed-archive` on the XML copy also gives exact values. `plutil -p` shows the correct values outside REA. ### REA version 6.5.0 (tag `rea-agents-6.5.0`; the relevant code is unchanged on main) ### Runtime and platform - REA: `rea-agents` 6.5.0 from npm (`--version` prints `6.5.0`), release tag `rea-agents-6.5.0` at commit `3b79af28e80785afce9f5998d8f03289402a1847`. - Host: macOS 27.0.1 (26A434) arm64, Node.js v26.10.0, system `plutil`. - Main at `d52a488` leaves the relevant code unchanged: `PlistNumberLiterals.ts` is identical, and the only change to `KeyedArchiveReader.ts` adds date limitations. ### Target and artifact context self-built binary `NSKeyedArchiver` plist whose `$objects` holds `18446744073709551615` (16-byte integer) and `9223372036854775807`; XML conversion and an Int64.max-only archive as controls ### Interface CLI ### Earliest failure stage Native analysis or reconstruction ### Relevant output or evidence - Expected source: REA `docs/native-investigation.md` at the release tag, section "Keyed archive integer precision" ([L550-L560](https://github.com/morluto/rea/blob/3b79af28e80785afce9f5998d8f03289402a1847/docs/native-investigation.md#L550-L560)) and the `inspect_keyed_archive` entry ([L311-L323](https://github.com/morluto/rea/blob/3b79af28e80785afce9f5998d8f03289402a1847/docs/native-investigation.md#L311-L323)), plus Apple's `plutil -p` reading of the fixture (`18446744073709551615`). - Failure source: `rea inspect-keyed-archive ./archive.bin --json` on the release build, `normalized_result.objects[1]` = `{"status":"observed","value":0}`. - Evidence provenance: observed - Local verification: reproduced - Reproduction completeness: complete This investigator ran the fixture above on the release build, together with the XML and Int64.max-only controls. ### Typed outcome or recovery expected With the fixture above, `inspect-keyed-archive ./archive.bin --json` must stop reporting object 1 as an observed `0`. It should report `{"$plist_type":"integer","decimal":"18446744073709551615"}`, as the XML form does. If REA reports any other representation, it must flag the value as not exactly observed in a limitation that counts it. Object 2 must keep its exact decimal text or carry a limitation that counts it. Discriminating controls: the XML form and the Int64.max-only binary archive must keep returning exact decimal text. ### Analysis or suspected cause _No response_ ### Additional context Suspected cause, from reading the source at the release tag: - `src/artifacts/apple/KeyedArchiveReader.ts` decodes binary archives with `parseBinary` from the pinned `plist` 5.0.0 package ([L3](https://github.com/morluto/rea/blob/3b79af28e80785afce9f5998d8f03289402a1847/src/artifacts/apple/KeyedArchiveReader.ts#L3), [L72](https://github.com/morluto/rea/blob/3b79af28e80785afce9f5998d8f03289402a1847/src/artifacts/apple/KeyedArchiveReader.ts#L72)). For any integer wider than 4 bytes, that decoder reads only the first 8 bytes as `hi * 2^32 + lo`. For a 16-byte big-endian value those are the high 8 bytes, which are zero for every value below 2^64. Running `parseBinary` directly on the fixture returns `"$objects":["$null",0,9223372036854776000]`. `plist` 5.0.0 is the latest npm release, so this may also be worth reporting upstream to TooTallNate/plist.js. That is a suspected cause, not a verified upstream report. - `src/artifacts/apple/PlistNumberLiterals.ts` records exact text for 8-byte integers but returns "incomplete" for any wider integer ([L162-L164](https://github.com/morluto/rea/blob/3b79af28e80785afce9f5998d8f03289402a1847/src/artifacts/apple/PlistNumberLiterals.ts#L162-L164)). That explains why one 16-byte integer turns off exact association for the whole archive and why the zeroed value gets only the generic precision note. Related but different issues: - #3373: XML hexadecimal integers decode as 0. That is a different encoding and parser path. - #4042: XML CDATA reals lose their content. - #3876 / PR #3972: `decode_interface_builder` rounds integers above 2^53. That is rounding, not replacement with 0, and it happens in a different tool. - The exact-decimal projection this report relies on was proposed in closed PR #1135. Its closing comment says it landed in commit `4034883`.

bsakaguchi closed this issue

2026-10-17 21:02:59 +00:00

bsakaguchi referenced this issue

2026-10-31 00:54:48 +00:00

[PR #1700] [MERGED] fix(process): observe already-due trailing callbacks before settling the capture journal #4764