[PR #271] [MERGED] perf: enable sha2 asm feature for hardware SHA-256 acceleration #307

New issue

Closed

opened 2026-10-29 14:22:01 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 14:22:01 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/AgentENV/pull/271
Author: @YXalix
Created: 9/10/2026
Status: ✅ Merged
Merged: 9/10/2026
Merged by: @yingdi-shan

Base: main ← Head: perf/sha2-asm-accel

📝 Commits (1)

c255384 perf: enable sha2 asm feature for hardware SHA-256 acceleration

📊 Changes

6 files changed (+15 additions, -5 deletions)

View changed files

📝 Cargo.lock (+10 -0)
📝 Cargo.toml (+1 -1)
📝 crates/benchmarks/Cargo.toml (+1 -1)
📝 crates/e2e-tests/Cargo.toml (+1 -1)
📝 storage/overlaybd/Cargo.toml (+1 -1)
📝 storage/ublk/Cargo.toml (+1 -1)

📄 Description

Motivation

SHA-256 is on the critical path of memory snapshot creation:

storage/overlaybd's AppendDigestTracker hashes every appended byte of the RW layer and the sealed layer at ZFile seal / compact_to time (memory snapshot commits),
registryfs_v2's uploader hashes full layers during registry publication,
src/digest.rs streams whole-file digests for snapshot artifact publication/copy.

The sha2 crate defaults to the portable software implementation. On aarch64, the ARMv8 SHA crypto extensions are only used when the optional asm feature is enabled explicitly — without it, multi-GB snapshot hashing runs entirely in software.

Change

Enable features = ["asm"] on every sha2 declaration in the workspace (runtime deps of agentenv and overlaybd, plus dev-deps of uvm-ublk tests, e2e-tests and benchmarks). Cargo.lock gains sha2-asm v0.6.4; no source code changes.

This is safe across all release targets (linux/darwin × x86_64/aarch64):

aarch64: selects the ARMv8 crypto instruction implementation (SHA1/SHA256 instructions are mandatory from ARMv8.4 and present on all mainstream aarch64 servers).
x86_64: selects SHA-NI with runtime CPU feature detection (via cpufeatures since sha2 0.10.5), falling back to the software implementation on CPUs without SHA extensions — no behavior regression on older x86 hosts, and a free speedup on SHA-NI ones.

Benchmark

aarch64 host; create sandbox → dd 1 GiB into tmpfs → take snapshot; 10 successful rounds per variant:

Step	Before (P50)	After (P50)	Before (Avg)	After (Avg)
Create sandbox	0.067s	0.069s	0.070s	0.078s
dd 1 GiB	5.190s	5.221s	5.181s	5.218s
Create snapshot	9.608s	6.244s	9.733s	6.354s
Total flow	14.871s	11.543s	14.985s	11.651s

Snapshot creation is ~35% faster (avg 9.733s → 6.354s); create and dd steps are unchanged within noise, isolating the win to the hashing-heavy snapshot path.

Verification

cargo tree -i sha2@0.10.9 -e features confirms the asm feature is active through both the agentenv and overlaybd dependency chains.
cargo check --workspace passes on aarch64 (sha2-asm assembly builds).
Hardware instructions confirmed in the compiled artifact (objdump shows sha256h/sha256su in the sha2 CGU).
Digest correctness: cargo test -p agentenv --lib digest:: and cargo test -p overlaybd --lib (338 passed) — hash outputs are unchanged; the only failures are pre-existing mock-HTTP-server environment failures that reproduce identically on unmodified origin/main.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/AgentENV/pull/271 **Author:** [@YXalix](https://github.com/YXalix) **Created:** 9/10/2026 **Status:** ✅ Merged **Merged:** 9/10/2026 **Merged by:** [@yingdi-shan](https://github.com/yingdi-shan) **Base:** `main` ← **Head:** `perf/sha2-asm-accel` --- ### 📝 Commits (1) - [`c255384`](https://github.com/kvcache-ai/AgentENV/commit/c25538476306f3b5083538132ef47129e83c2a76) perf: enable sha2 asm feature for hardware SHA-256 acceleration ### 📊 Changes **6 files changed** (+15 additions, -5 deletions) <details> <summary>View changed files</summary> 📝 `Cargo.lock` (+10 -0) 📝 `Cargo.toml` (+1 -1) 📝 `crates/benchmarks/Cargo.toml` (+1 -1) 📝 `crates/e2e-tests/Cargo.toml` (+1 -1) 📝 `storage/overlaybd/Cargo.toml` (+1 -1) 📝 `storage/ublk/Cargo.toml` (+1 -1) </details> ### 📄 Description ## Motivation SHA-256 is on the critical path of memory snapshot creation: - `storage/overlaybd`'s `AppendDigestTracker` hashes every appended byte of the RW layer and the sealed layer at ZFile seal / `compact_to` time (memory snapshot commits), - `registryfs_v2`'s uploader hashes full layers during registry publication, - `src/digest.rs` streams whole-file digests for snapshot artifact publication/copy. The `sha2` crate defaults to the portable **software** implementation. On aarch64, the ARMv8 SHA crypto extensions are only used when the optional `asm` feature is enabled explicitly — without it, multi-GB snapshot hashing runs entirely in software. ## Change Enable `features = ["asm"]` on every `sha2` declaration in the workspace (runtime deps of `agentenv` and `overlaybd`, plus dev-deps of `uvm-ublk` tests, e2e-tests and benchmarks). `Cargo.lock` gains `sha2-asm v0.6.4`; no source code changes. This is safe across all release targets (linux/darwin × x86_64/aarch64): - **aarch64**: selects the ARMv8 crypto instruction implementation (SHA1/SHA256 instructions are mandatory from ARMv8.4 and present on all mainstream aarch64 servers). - **x86_64**: selects SHA-NI **with runtime CPU feature detection** (via `cpufeatures` since sha2 0.10.5), falling back to the software implementation on CPUs without SHA extensions — no behavior regression on older x86 hosts, and a free speedup on SHA-NI ones. ## Benchmark aarch64 host; create sandbox → `dd` 1 GiB into tmpfs → take snapshot; 10 successful rounds per variant: | Step | Before (P50) | After (P50) | Before (Avg) | After (Avg) | |---|---|---|---|---| | Create sandbox | 0.067s | 0.069s | 0.070s | 0.078s | | dd 1 GiB | 5.190s | 5.221s | 5.181s | 5.218s | | **Create snapshot** | **9.608s** | **6.244s** | **9.733s** | **6.354s** | | Total flow | 14.871s | 11.543s | 14.985s | 11.651s | Snapshot creation is **~35% faster** (avg 9.733s → 6.354s); create and dd steps are unchanged within noise, isolating the win to the hashing-heavy snapshot path. ## Verification - `cargo tree -i sha2@0.10.9 -e features` confirms the `asm` feature is active through both the `agentenv` and `overlaybd` dependency chains. - `cargo check --workspace` passes on aarch64 (sha2-asm assembly builds). - Hardware instructions confirmed in the compiled artifact (`objdump` shows `sha256h`/`sha256su` in the sha2 CGU). - Digest correctness: `cargo test -p agentenv --lib digest::` and `cargo test -p overlaybd --lib` (338 passed) — hash outputs are unchanged; the only failures are pre-existing mock-HTTP-server environment failures that reproduce identically on unmodified `origin/main`. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-29 14:22:01 +00:00