morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #3177] [Bug] Apple application projection reverses Unicode code point order for astral paths #1116

New issue

Closed

opened 2026-10-15 18:31:12 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-15 18:31:12 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 14, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3177

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

project-apple-application-graph reverses Unicode code point order in application_roots, bundles[].path and symlinks when application names contain U+10000 and U+FFFD. The authenticated inventory produced by inspect-artifact puts the U+FFFD application's paths first; the projection puts the U+10000 application's paths first.

Both public CLI commands exited 0 with no stderr in the verifier's reproduction. Both application plist components remained present and in code point order, and the projection reported complete-within-inventory with inventory_complete: true. A BMP control using U+E000 and U+FFFD returned the expected order in all three arrays.

Reproduction

Start from a clean checkout of the baseline above, select a supported Node.js runtime, and prepare the CLI:
```
npm ci
npm run build:cached
```

Create the input archive inline. The escaped names make the Unicode values unambiguous:

mkdir repro-data
python3 - <<'PY'
from pathlib import Path
from zipfile import ZipFile, ZipInfo

with ZipFile(Path('repro-data') / 'unicode.zip', 'w') as archive:
    for point in (0x10000, 0xfffd):
        root = f'A{chr(point)}.app'
        archive.writestr(
            f'{root}/Contents/Info.plist',
            '<?xml version="1.0"?><plist version="1.0"><dict/></plist>',
        )
        link = ZipInfo(f'{root}/Contents/Resources/link')
        link.create_system = 3
        link.external_attr = 0o120777 << 16
        archive.writestr(link, '../Info.plist')
PY

Inspect the archive and extract the completed inventory Evidence into a projection input file:

node scripts/rea.mjs inspect-artifact repro-data/unicode.zip --format json > repro-data/inspection.json
python3 - <<'PY'
import json
from pathlib import Path

folder = Path('repro-data')
inspection = json.loads((folder / 'inspection.json').read_text())
inventory = next(
    step['evidence']
    for step in inspection['normalized_result']['substeps']
    if step['operation'] == 'inventory_artifact'
    and step['status'] == 'completed'
)
(folder / 'projection-input.json').write_text(
    json.dumps({'inventory_evidence': [inventory]})
)
print(json.dumps(
    [item['logical_path'] for item in inventory['normalized_result']['occurrences']],
    ensure_ascii=True,
))
PY

Project that authenticated inventory and print the relevant arrays:

node scripts/rea.mjs project-apple-application-graph repro-data/projection-input.json --format json > repro-data/projection.json
python3 - <<'PY'
import json
from pathlib import Path

result = json.loads(Path('repro-data/projection.json').read_text())['normalized_result']
print(json.dumps({
    'application_roots': result['application_roots'],
    'bundle_paths': [bundle['path'] for bundle in result['bundles']],
    'symlinks': result['symlinks'],
    'coverage': result['coverage'],
}, ensure_ascii=True, indent=2))
PY

Expected behavior

The explicit comparator contract in src/domain/mobileApplicationGraphProjection.ts:122-126 specifies Unicode code point ordering for projection arrays. src/domain/unicodeCodePointOrder.ts:1-18 defines that ordering, and src/domain/artifactInventoryEvidence.ts:214-221 applies it to inventory occurrence paths. With the common prefix A, U+FFFD is code point 65533 and U+10000 is code point 65536. Roots, bundle paths and symlink paths should put the U+FFFD application first, consistent with the incoming inventory.

docs/apple-application-analysis.md:49-50 documents locale-independent Unicode ordering for graph and directory identities. That sentence supports the identity context; the source comparator comment supplies the precise code point contract for these arrays.

Impact and scope

The verified impact is P3 presentation ordering for supported Unicode paths. The reproduction retains both applications and their metadata, both commands succeed, and complete coverage remains available. The verifier observed no information loss, consequential interpretation failure or security boundary crossing.

A consumer can display code point sorted copies of roots, bundles and symlinks while retaining the original authenticated Evidence unchanged. The verifier confirmed that this display workaround produces U+FFFD before U+10000. The observed defect concerns these three arrays; the fixture's component metadata already has the correct order.

REA version

6.4.0 (main at 5830088520)

Runtime and platform

Node 26.10.0; macOS 27.0 arm64

Target and artifact context

A local ZIP containing two application plist files and two Unix symlink entries. The application roots are A\ufffd.app and A\ud800\udc00.app, with the escaped surrogate pair representing U+10000. The recipe creates the inputs inline and obtains authenticated nested inventory Evidence through the supported inspect-artifact command. No analysis engine, live application or external service is needed.

Interface

CLI

Earliest failure stage

Transport or result projection

Relevant output or evidence

The projection returned the following escaped paths. U+FFFD should precede U+10000 in each array.

application_roots: ["A\ud800\udc00.app", "A\ufffd.app"]
bundle_paths: ["A\ud800\udc00.app", "A\ufffd.app"]
symlinks: ["A\ud800\udc00.app/Contents/Resources/link", "A\ufffd.app/Contents/Resources/link"]
coverage: {"status":"complete-within-inventory","inventory_complete":true}

Typed outcome or recovery expected

Return a successful projection with roots, bundle paths and symlink paths in Unicode code point order. Preserve both application plist components, original exact path spellings, complete coverage and Evidence provenance. The BMP control must retain its current ordering.

The resulting projection_id must remain consistent with the ordered payload and deterministic for identical inputs. It need not equal the old digest after an affected array changes order; the verifier's read-only digest check confirmed that reordering the arrays changes the projection digest.

Analysis or suspected cause

Source-derived reasoning at the baseline shows two relational string comparators. JavaScript string < and > compare UTF-16 code units, so the leading surrogate for U+10000 sorts before U+FFFD even though its code point is larger.

src/domain/mobileApplicationGraphProjection.ts:122-126 implements compareProjectionStrings with those relational operators despite its code point contract. src/domain/apple/appleApplication.ts:200-203 uses that comparator for symlinks through the binding at line 453. Roots and bundles use a separate comparator at src/domain/apple/appleBundleAnatomy.ts:380-381, applied at lines 147-151 and 173-174. The existing compareUnicodeCodePoints implementation in src/domain/unicodeCodePointOrder.ts:1-18 compares actual code points.

The source-derived public route is src/cli/applicationCommands.ts:112-126 through src/application/apple/AppleApplicationService.ts:17-25. The verifier independently reproduced the ordering through that public CLI route with real ZIP inventory Evidence.

Additional context

Related PR #1867, https://github.com/morluto/rea/pull/1867, replaces locale-sensitive comparisons in other producers; this report concerns locale-independent UTF-16 comparisons in the Apple projection that still reverse the verified astral/BMP pair.

The verifier used the prepared built runtime and replayed the fixture-generation, inspection/extraction and projection/output recipe. npm ci and npm run build:cached are clean-checkout prerequisites and were not rerun during verification. The fetched main at 767fe0f0ccbab3ae4be28e71cc8214c7cd302133 has no changes to the failure-source files since the verified baseline.

Originally created by @coygeek on GitHub (Oct 14, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3177 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior `project-apple-application-graph` reverses Unicode code point order in `application_roots`, `bundles[].path` and `symlinks` when application names contain U+10000 and U+FFFD. The authenticated inventory produced by `inspect-artifact` puts the U+FFFD application's paths first; the projection puts the U+10000 application's paths first. Both public CLI commands exited 0 with no stderr in the verifier's reproduction. Both application plist components remained present and in code point order, and the projection reported `complete-within-inventory` with `inventory_complete: true`. A BMP control using U+E000 and U+FFFD returned the expected order in all three arrays. ### Reproduction 1. Start from a clean checkout of the baseline above, select a supported Node.js runtime, and prepare the CLI: ```sh npm ci npm run build:cached ``` 2. Create the input archive inline. The escaped names make the Unicode values unambiguous: ```sh mkdir repro-data python3 - <<'PY' from pathlib import Path from zipfile import ZipFile, ZipInfo with ZipFile(Path('repro-data') / 'unicode.zip', 'w') as archive: for point in (0x10000, 0xfffd): root = f'A{chr(point)}.app' archive.writestr( f'{root}/Contents/Info.plist', '<?xml version="1.0"?><plist version="1.0"><dict/></plist>', ) link = ZipInfo(f'{root}/Contents/Resources/link') link.create_system = 3 link.external_attr = 0o120777 << 16 archive.writestr(link, '../Info.plist') PY ``` 3. Inspect the archive and extract the completed inventory Evidence into a projection input file: ```sh node scripts/rea.mjs inspect-artifact repro-data/unicode.zip --format json > repro-data/inspection.json python3 - <<'PY' import json from pathlib import Path folder = Path('repro-data') inspection = json.loads((folder / 'inspection.json').read_text()) inventory = next( step['evidence'] for step in inspection['normalized_result']['substeps'] if step['operation'] == 'inventory_artifact' and step['status'] == 'completed' ) (folder / 'projection-input.json').write_text( json.dumps({'inventory_evidence': [inventory]}) ) print(json.dumps( [item['logical_path'] for item in inventory['normalized_result']['occurrences']], ensure_ascii=True, )) PY ``` 4. Project that authenticated inventory and print the relevant arrays: ```sh node scripts/rea.mjs project-apple-application-graph repro-data/projection-input.json --format json > repro-data/projection.json python3 - <<'PY' import json from pathlib import Path result = json.loads(Path('repro-data/projection.json').read_text())['normalized_result'] print(json.dumps({ 'application_roots': result['application_roots'], 'bundle_paths': [bundle['path'] for bundle in result['bundles']], 'symlinks': result['symlinks'], 'coverage': result['coverage'], }, ensure_ascii=True, indent=2)) PY ``` ### Expected behavior The explicit comparator contract in `src/domain/mobileApplicationGraphProjection.ts:122-126` specifies Unicode code point ordering for projection arrays. `src/domain/unicodeCodePointOrder.ts:1-18` defines that ordering, and `src/domain/artifactInventoryEvidence.ts:214-221` applies it to inventory occurrence paths. With the common prefix `A`, U+FFFD is code point 65533 and U+10000 is code point 65536. Roots, bundle paths and symlink paths should put the U+FFFD application first, consistent with the incoming inventory. `docs/apple-application-analysis.md:49-50` documents locale-independent Unicode ordering for graph and directory identities. That sentence supports the identity context; the source comparator comment supplies the precise code point contract for these arrays. ### Impact and scope The verified impact is P3 presentation ordering for supported Unicode paths. The reproduction retains both applications and their metadata, both commands succeed, and complete coverage remains available. The verifier observed no information loss, consequential interpretation failure or security boundary crossing. A consumer can display code point sorted copies of roots, bundles and symlinks while retaining the original authenticated Evidence unchanged. The verifier confirmed that this display workaround produces U+FFFD before U+10000. The observed defect concerns these three arrays; the fixture's component metadata already has the correct order. ### REA version 6.4.0 (main at 58300885207133422af235e0f2b10658cf958b03) ### Runtime and platform Node 26.10.0; macOS 27.0 arm64 ### Target and artifact context A local ZIP containing two application plist files and two Unix symlink entries. The application roots are `A\ufffd.app` and `A\ud800\udc00.app`, with the escaped surrogate pair representing U+10000. The recipe creates the inputs inline and obtains authenticated nested inventory Evidence through the supported `inspect-artifact` command. No analysis engine, live application or external service is needed. ### Interface CLI ### Earliest failure stage Transport or result projection ### Relevant output or evidence The projection returned the following escaped paths. U+FFFD should precede U+10000 in each array. ```text application_roots: ["A\ud800\udc00.app", "A\ufffd.app"] bundle_paths: ["A\ud800\udc00.app", "A\ufffd.app"] symlinks: ["A\ud800\udc00.app/Contents/Resources/link", "A\ufffd.app/Contents/Resources/link"] coverage: {"status":"complete-within-inventory","inventory_complete":true} ``` ### Typed outcome or recovery expected Return a successful projection with roots, bundle paths and symlink paths in Unicode code point order. Preserve both application plist components, original exact path spellings, complete coverage and Evidence provenance. The BMP control must retain its current ordering. The resulting `projection_id` must remain consistent with the ordered payload and deterministic for identical inputs. It need not equal the old digest after an affected array changes order; the verifier's read-only digest check confirmed that reordering the arrays changes the projection digest. ### Analysis or suspected cause Source-derived reasoning at the baseline shows two relational string comparators. JavaScript string `<` and `>` compare UTF-16 code units, so the leading surrogate for U+10000 sorts before U+FFFD even though its code point is larger. `src/domain/mobileApplicationGraphProjection.ts:122-126` implements `compareProjectionStrings` with those relational operators despite its code point contract. `src/domain/apple/appleApplication.ts:200-203` uses that comparator for symlinks through the binding at line 453. Roots and bundles use a separate comparator at `src/domain/apple/appleBundleAnatomy.ts:380-381`, applied at lines 147-151 and 173-174. The existing `compareUnicodeCodePoints` implementation in `src/domain/unicodeCodePointOrder.ts:1-18` compares actual code points. The source-derived public route is `src/cli/applicationCommands.ts:112-126` through `src/application/apple/AppleApplicationService.ts:17-25`. The verifier independently reproduced the ordering through that public CLI route with real ZIP inventory Evidence. ### Additional context Related PR #1867, https://github.com/morluto/rea/pull/1867, replaces locale-sensitive comparisons in other producers; this report concerns locale-independent UTF-16 comparisons in the Apple projection that still reverse the verified astral/BMP pair. The verifier used the prepared built runtime and replayed the fixture-generation, inspection/extraction and projection/output recipe. `npm ci` and `npm run build:cached` are clean-checkout prerequisites and were not rerun during verification. The fetched main at `767fe0f0ccbab3ae4be28e71cc8214c7cd302133` has no changes to the failure-source files since the verified baseline.

bsakaguchi

2026-10-15 18:31:12 +00:00

closed this issue
added the
bug
label

bsakaguchi commented

2026-10-15 18:31:13 +00:00

Author

Owner

@Sriram-PR commented on GitHub (Oct 14, 2026):

I'll take this one, PR shortly.

@Sriram-PR commented on GitHub (Oct 14, 2026): I'll take this one, PR shortly.

bsakaguchi referenced this issue

2026-10-17 21:08:46 +00:00

[PR #1116] [MERGED] fix(build): exclude the test-only generated catalog from packages #2661

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#1116

No description provided.

Rows
Columns