[PR #1013] [MERGED] perf(artifacts): replace quadratic ZIP overlap checks #2581

New issue

Closed

opened 2026-10-17 21:07:51 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-17 21:07:51 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/1013
Author: @morluto
Created: 10/7/2026
Status: ✅ Merged
Merged: 10/7/2026
Merged by: @morluto

Base: main ← Head: codex/zip-overlap-final

📝 Commits (1)

6cde76c perf(artifacts): replace quadratic ZIP overlap checks

📊 Changes

4 files changed (+121 additions, -5 deletions)

View changed files

📝 package-lock.json (+4 -4)
📝 package.json (+1 -1)
📝 src/artifacts/ZipArtifactReader.ts (+3 -0)
➕ tests/boundary/filesystem/artifactZipIntegrity.test.ts (+113 -0)

📄 Description

Large ZIP inventories currently spend quadratic work comparing each entry range with all prior ranges. Upgrade the exact zip.js dependency from 2.8.26 to 2.15.0, which keeps indexed overlap checks and fixes the first indexed release's Buffer-view compatibility issue for REA's file-backed reader. Signature, CRC, and overlap verification remain enabled.

Use tolerant library filename parsing so decoded names reach REA's existing provider-neutral validation unchanged. Unsafe paths still return their typed path constraint rather than a generic library error. Added production inventory regressions cover overlapping entry ranges, encrypted-member observations, corrupt data/CRC, and truncated archives. Existing regressions cover unsafe names and ordering.

Bounded measurements on Node 24.18.0, three alternating pairs each:

File-backed overlap checks only, 4,000 entries: median 114.54 ms → 47.73 ms (58% lower), with identical entry metadata. This excludes enumeration and hashing.
Warmed complete inventory, 1,000 entries, using byte-identical compiled REA code with only the ZIP dependency changed: median 122.69 ms → 133.73 ms (9% slower). All six complete outputs matched, SHA-256 0c91f87b4aa49f03eba20fd680492078ab13b4adaf4a8d3d0f431d713a864675.

The optimization addresses overlap-check scaling; it does not claim a general small-archive throughput gain. Larger whole-inventory runs were intentionally avoided after the earlier worktree CPU load.

Validation: npm run build:cached, npm run check:fast, npm run docs:check, and formatting checks passed. The three focused files artifactZipIntegrity.test.ts, artifactInventoryEntryOrder.test.ts, and artifactProvider.failureDetail.test.ts passed all 20 tests with one worker. Independent Standards and Spec reviews accepted the source. Real Hopper/Ghidra verification was not performed for this archive-only change.

Upstream: indexed overlap checks, 2.15.0 release.

Related to #893.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/1013 **Author:** [@morluto](https://github.com/morluto) **Created:** 10/7/2026 **Status:** ✅ Merged **Merged:** 10/7/2026 **Merged by:** [@morluto](https://github.com/morluto) **Base:** `main` ← **Head:** `codex/zip-overlap-final` --- ### 📝 Commits (1) - [`6cde76c`](https://github.com/morluto/rea/commit/6cde76cf196dda3d36b17e5b93a5bca75826bccf) perf(artifacts): replace quadratic ZIP overlap checks ### 📊 Changes **4 files changed** (+121 additions, -5 deletions) <details> <summary>View changed files</summary> 📝 `package-lock.json` (+4 -4) 📝 `package.json` (+1 -1) 📝 `src/artifacts/ZipArtifactReader.ts` (+3 -0) ➕ `tests/boundary/filesystem/artifactZipIntegrity.test.ts` (+113 -0) </details> ### 📄 Description Large ZIP inventories currently spend quadratic work comparing each entry range with all prior ranges. Upgrade the exact zip.js dependency from 2.8.26 to 2.15.0, which keeps indexed overlap checks and fixes the first indexed release's Buffer-view compatibility issue for REA's file-backed reader. Signature, CRC, and overlap verification remain enabled. Use tolerant library filename parsing so decoded names reach REA's existing provider-neutral validation unchanged. Unsafe paths still return their typed path constraint rather than a generic library error. Added production inventory regressions cover overlapping entry ranges, encrypted-member observations, corrupt data/CRC, and truncated archives. Existing regressions cover unsafe names and ordering. Bounded measurements on Node 24.18.0, three alternating pairs each: - File-backed overlap checks only, 4,000 entries: median 114.54 ms → 47.73 ms (58% lower), with identical entry metadata. This excludes enumeration and hashing. - Warmed complete inventory, 1,000 entries, using byte-identical compiled REA code with only the ZIP dependency changed: median 122.69 ms → 133.73 ms (9% slower). All six complete outputs matched, SHA-256 `0c91f87b4aa49f03eba20fd680492078ab13b4adaf4a8d3d0f431d713a864675`. The optimization addresses overlap-check scaling; it does not claim a general small-archive throughput gain. Larger whole-inventory runs were intentionally avoided after the earlier worktree CPU load. Validation: `npm run build:cached`, `npm run check:fast`, `npm run docs:check`, and formatting checks passed. The three focused files `artifactZipIntegrity.test.ts`, `artifactInventoryEntryOrder.test.ts`, and `artifactProvider.failureDetail.test.ts` passed all 20 tests with one worker. Independent Standards and Spec reviews accepted the source. Real Hopper/Ghidra verification was not performed for this archive-only change. Upstream: [indexed overlap checks](https://github.com/gildas-lormeau/zip.js/releases/tag/v2.14.1), [2.15.0 release](https://github.com/gildas-lormeau/zip.js/releases/tag/v2.15.0). Related to #893. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-17 21:07:51 +00:00

closed this issue
added the
pull-request
label

bsakaguchi referenced this issue

2026-10-31 01:07:23 +00:00

[PR #2979] [CLOSED] fix(install): validate empty version requests and unreported installed versions #5534

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#2581

No description provided.

Rows
Columns