[PR #962] [MERGED] perf(artifacts): stream ASAR members without full buffering #2536

New issue

Closed

opened 2026-10-17 21:07:17 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-17 21:07:17 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/962
Author: @morluto
Created: 10/7/2026
Status: ✅ Merged
Merged: 10/7/2026
Merged by: @morluto

Base: main ← Head: codex/firmware-archive-wave2

📝 Commits (8)

2ed997b fix(artifacts): stream ASAR members without full buffering
69abec0 fix(artifacts): close unopened ASAR member streams
0aeb438 fix(artifacts): preserve unpacked ASAR mismatch digests
efb46f2 fix(artifacts): preserve ASAR read failures
2d4a2b5 fix(artifacts): retain unpacked ASAR stat failures
1cc44e7 Merge remote-tracking branch 'origin/main' into codex/pr962-asar-stat-errors
16ba9db fix(artifacts): validate ASAR entry producer identity
e5b7ff2 Merge remote-tracking branch 'origin/main' into codex/pr962-asar-stat-errors

📊 Changes

4 files changed (+872 additions, -32 deletions)

View changed files

📝 src/artifacts/AsarArtifactReader.ts (+268 -31)
➕ src/artifacts/AsarEntryStream.ts (+111 -0)
➕ tests/boundary/filesystem/asarStreamingAllocation.test.ts (+474 -0)
📝 tests/boundary/filesystem/javascriptArtifactContentIdentity.test.ts (+19 -1)

📄 Description

Summary

Opening an ASAR member for inventory previously called @electron/asar's extractFile, which buffered the complete member before the reader could hash it. A valid packed ASAR with a 64 MiB member showed 67,108,864 additional array-buffer bytes at the open allocation snapshot. The reader now streams the selected member range and continues hashing the complete contents while retaining range, declared-size, and digest checks.

A matching post-change profile showed 0 additional array-buffer bytes at the open snapshot (plus 40 external bytes and 16,384 RSS bytes); this is an allocation snapshot, not a claim of zero peak memory. Packed and unpacked 64 MiB regressions verify the full SHA-256 and byte count.

Unpacked companions are streamed to EOF even when their current size differs from the header's declared size. This preserves the observed byte count and digest while reporting the declared size separately. For compatibility, a no-integrity ASAR fixture with a declared 20-byte unpacked member and a 32-byte companion produced the same full inventory result on pre-PR main and this candidate after normalizing only the obsolete buffering limitation and the manifest/graph hashes derived from that limitation. Both results reported hash_status=verified, declared_size=20, observed size 32, and SHA-256 48b2922be13ce1514be10dba22cf08158591582a814f43a26c41d0f823fef850; the normalized full-result SHA-256 was 7d74c2d71d524cf90d5246f3a59105888ea6095de18b39fde622d4b80e6377f0.

The change also closes the underlying member file handle when the returned stream is destroyed before its first read, checks cancellation after asynchronous open/stat and after closing a zero-length member's handle, and maps packed-member filesystem errors through the artifact I/O failure classification. Missing or replaced containers are rejected even for zero-length members. A close-seam regression closes a real file handle, aborts before the close callback resolves, and verifies the cancellation result and descriptor cleanup using fstat. A stream-iteration EIO regression preserves the io reason, member/container context, and original errno/cause. Tests also retain ENOENT classification and confirm that a same-size replacement ASAR is rejected against the inventoried root digest.

Validation

npm run build:cached
Focused filesystem, artifact-provider, ZIP cancellation, NIB, and keyed-archive tests: 10 files, 42 tests
npm run check:fast
npm run format:check
npm run check:ci

The descriptor-count lifecycle tests use POSIX descriptor tables and skip Windows; packed/unpacked streaming and content-identity regressions remain cross-platform.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/962 **Author:** [@morluto](https://github.com/morluto) **Created:** 10/7/2026 **Status:** ✅ Merged **Merged:** 10/7/2026 **Merged by:** [@morluto](https://github.com/morluto) **Base:** `main` ← **Head:** `codex/firmware-archive-wave2` --- ### 📝 Commits (8) - [`2ed997b`](https://github.com/morluto/rea/commit/2ed997b738f72087369a88d55facd86c5e94b1d1) fix(artifacts): stream ASAR members without full buffering - [`69abec0`](https://github.com/morluto/rea/commit/69abec031eb351db84d5bf52b1a721c2b7155215) fix(artifacts): close unopened ASAR member streams - [`0aeb438`](https://github.com/morluto/rea/commit/0aeb438d77e8dda8f1513c525873b8f40efc80ba) fix(artifacts): preserve unpacked ASAR mismatch digests - [`efb46f2`](https://github.com/morluto/rea/commit/efb46f23c0e8cc1145747fdf9d1663a78a8974d1) fix(artifacts): preserve ASAR read failures - [`2d4a2b5`](https://github.com/morluto/rea/commit/2d4a2b5300e9a1fe5eeaab2fc4830a7be2f0be0f) fix(artifacts): retain unpacked ASAR stat failures - [`1cc44e7`](https://github.com/morluto/rea/commit/1cc44e71c8c41425dbe8cd17eb0eed56f9a73288) Merge remote-tracking branch 'origin/main' into codex/pr962-asar-stat-errors - [`16ba9db`](https://github.com/morluto/rea/commit/16ba9dbe54dec012f7034f68c1e626d209803f6e) fix(artifacts): validate ASAR entry producer identity - [`e5b7ff2`](https://github.com/morluto/rea/commit/e5b7ff2c3f62e7792d1765f0388e001fe3ed13b5) Merge remote-tracking branch 'origin/main' into codex/pr962-asar-stat-errors ### 📊 Changes **4 files changed** (+872 additions, -32 deletions) <details> <summary>View changed files</summary> 📝 `src/artifacts/AsarArtifactReader.ts` (+268 -31) ➕ `src/artifacts/AsarEntryStream.ts` (+111 -0) ➕ `tests/boundary/filesystem/asarStreamingAllocation.test.ts` (+474 -0) 📝 `tests/boundary/filesystem/javascriptArtifactContentIdentity.test.ts` (+19 -1) </details> ### 📄 Description ## Summary Opening an ASAR member for inventory previously called `@electron/asar`'s `extractFile`, which buffered the complete member before the reader could hash it. A valid packed ASAR with a 64 MiB member showed 67,108,864 additional array-buffer bytes at the open allocation snapshot. The reader now streams the selected member range and continues hashing the complete contents while retaining range, declared-size, and digest checks. A matching post-change profile showed 0 additional array-buffer bytes at the open snapshot (plus 40 external bytes and 16,384 RSS bytes); this is an allocation snapshot, not a claim of zero peak memory. Packed and unpacked 64 MiB regressions verify the full SHA-256 and byte count. Unpacked companions are streamed to EOF even when their current size differs from the header's declared size. This preserves the observed byte count and digest while reporting the declared size separately. For compatibility, a no-integrity ASAR fixture with a declared 20-byte unpacked member and a 32-byte companion produced the same full inventory result on pre-PR main and this candidate after normalizing only the obsolete buffering limitation and the manifest/graph hashes derived from that limitation. Both results reported `hash_status=verified`, `declared_size=20`, observed size 32, and SHA-256 `48b2922be13ce1514be10dba22cf08158591582a814f43a26c41d0f823fef850`; the normalized full-result SHA-256 was `7d74c2d71d524cf90d5246f3a59105888ea6095de18b39fde622d4b80e6377f0`. The change also closes the underlying member file handle when the returned stream is destroyed before its first read, checks cancellation after asynchronous open/stat and after closing a zero-length member's handle, and maps packed-member filesystem errors through the artifact I/O failure classification. Missing or replaced containers are rejected even for zero-length members. A close-seam regression closes a real file handle, aborts before the close callback resolves, and verifies the cancellation result and descriptor cleanup using `fstat`. A stream-iteration EIO regression preserves the `io` reason, member/container context, and original errno/cause. Tests also retain `ENOENT` classification and confirm that a same-size replacement ASAR is rejected against the inventoried root digest. ## Validation - `npm run build:cached` - Focused filesystem, artifact-provider, ZIP cancellation, NIB, and keyed-archive tests: 10 files, 42 tests - `npm run check:fast` - `npm run format:check` - `npm run check:ci` The descriptor-count lifecycle tests use POSIX descriptor tables and skip Windows; packed/unpacked streaming and content-identity regressions remain cross-platform. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-17 21:07:17 +00:00

closed this issue
added the
pull-request
label

bsakaguchi referenced this issue

2026-10-31 00:38:15 +00:00

[GH-ISSUE #5333] [Bug] inspect-go-binary returns build_info null when the PE data section has VirtualSize 0 #3242

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#2536

No description provided.

Rows
Columns