[PR #276] [CLOSED] fix(p2p): restore OSS snapshot layers through peers #311

New issue

Closed

opened 2026-10-29 14:22:04 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 14:22:04 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/AgentENV/pull/276
Author: @WindSongShaoGe
Created: 9/11/2026
Status: ❌ Closed

Base: main ← Head: fix/p2p-snapshot-layer-compatibility

📝 Commits (2)

dd69d8e fix(p2p): restore OSS snapshot layers through peers
1bfe8a1 fix(p2p): retain origin fallback after peer open

📊 Changes

7 files changed (+488 additions, -43 deletions)

View changed files

📝 docs/src/internals/p2p-design.md (+3 -1)
📝 src/overlaybd/p2p/facade.rs (+152 -13)
📝 src/snapshot/manager.rs (+24 -15)
📝 src/snapshot/p2p.rs (+78 -5)
📝 storage/overlaybd/src/backend/registryfs_v2.rs (+6 -1)
📝 storage/overlaybd/src/image/image_file.rs (+216 -7)
📝 storage/overlaybd/src/image/image_service.rs (+9 -1)

📄 Description

What

Fix peer reads of OSS-backed snapshot layers. OverlayBD URL probes now send a bounded Range header, digest-addressed layers can be opened through the P2P facade, and local snapshot layers without digest/size metadata are described before publication. Enable TCP_NODELAY on facade connections used for small range reads during restore.

Why

The actual-URL probe previously sent GET without Range, so the range-only UUID facade returned HTTP 400. Fixing that probe alone still leaves layers without UUIDs on the OSS read path. In a cold-node restore where OSS data reads are blocked, these gaps prevent template or snapshot recovery.

The complete patch passed the strict two-node full-chain suite on main commit 3815a57: template launch and captured disk/memory recovery issued zero OSS data GETs, and OCI conversion reuse issued zero source-layer GETs.

No linked issue. A separate compatibility fix was requested in prior maintainer discussion.

Scope and non-goals

Includes layer publication/read-path fixes, focused regression tests, and P2P design documentation. The standalone full-chain E2E suite and its Compose/configuration files are retained on a separate test branch and are excluded from this PR. No changes to snapshot storage formats or publication compression policy.

Design and behavior changes

Add Range: bytes=0-0 to actual-URL probes, matching the existing authentication probe.
Add /p2p-digest/{digest} using the existing canonical layer identity, descriptor validation, and bounded-range handling shared with the UUID facade.
Try UUID P2P reads when available, then digest P2P reads, then the existing origin path on open failure. Explicit local-file behavior is unchanged.
Hash local layers that lack digest/size metadata before applying the committed-digest publication guard. Bytes whose digest differs from the committed record remain excluded.
Set TCP_NODELAY on accepted facade sockets to avoid Nagle/delayed-ACK stalls for small streamed ranges.

Compatibility and operations

Public API or generated protocol: unchanged; adds an internal loopback facade endpoint.
Configuration or defaults: no schema/default changes; uses existing P2P settings.
Snapshot manifest, artifact layout, or storage format: unchanged; reuses existing layer keys.
Upgrade and rollback: rebuild/deploy server and ublk-daemon together; no data migration. Origin fallback remains available.
Host requirements, permissions, ports, or dependencies: unchanged.

Validation

make fmt
make clippy
make test-unit
Relevant Rust integration tests
make -C services test (required when services/ changes)
Generated clients/server regenerated with the documented make target
Documentation updated
Benchmarks or performance comparison completed

Commands and results:

cargo build --release -p agentenv --bin server -p uvm-ublk-daemon --bin uvm-ublk-daemon
  PASS
cargo test -p agentenv --lib p2p::
  60 passed
cargo test -p overlaybd --lib image::
  42 passed
cargo test -p overlaybd --lib backend::registryfs_v2::tests
  18 passed
make fmt
  PASS
make clippy
  PASS (workspace, all targets/features, -D warnings)

Strict E2E validation used the separately retained test commit 9a7cd21 (main 3815a57 plus the complete patch). The production source files in that commit are byte-identical to this PR. Server/daemon were freshly built from that source and installed in an isolated test image using the previously passing run's dependencies.

env SKIP_BUILD=1 P2P_SNAPSHOT_MODE=strict \
  E2E_P2P_RUNTIME_IMAGE=agentenv-runtime:p2p-main-9a7cd21 \
  AENV_TEST_STATE_DIR=/home/wenhao/AgentENV/target/p2p-latest-main-validation \
  AENV_NODE_A_PORT=18081 AENV_NODE_B_PORT=18082 \
  bash scripts/tests/e2e/run_p2p_full_chain.sh

Result: all 6 stages passed (template P2P launch, source capture, snapshot P2P recovery, OSS fallback, OCI conversion reuse, and OCI fallback). The harness's 12 Python tests also passed. Run ID: 1789115644-117861. Template/snapshot data GETs to OSS and OCI source-layer GETs were zero in the corresponding P2P stages. The test image and suite are local validation artifacts, not public dependencies of this PR.

Skipped checks and reasons:

Aggregate make test-unit and other Rust integration targets were not run; validation focused on the affected modules and the strict two-node VM E2E suite. No Go or generated-code changes, so Go checks and regeneration were not required. No standalone performance benchmark was run.

Risks and reviewer notes

This is broader than the earlier Range-only patch: digest reads, descriptorless-layer publication, and TCP_NODELAY are also included. The validation demonstrates the combined patch, not the individual performance contribution of each change.

Digest probing adds a P2P attempt on remote-layer opens. Computing missing local descriptors adds hashing work at publication. Publication still requires a digest present in the committed record. Publish-time recompression that changes local bytes remains outside the validated strict scenario; the E2E configuration disables publication compression. The facade does not receive OSS credentials, and P2P failures retain the existing origin fallback.

Checklist

The PR contains one coherent change and no unrelated formatting or refactoring.
New behavior is covered by tests, or I explained why testing is impractical.
Logs and examples contain no credentials, tokens, or private registry information.
I did not manually edit generated code without updating its source and regenerating it.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/AgentENV/pull/276 **Author:** [@WindSongShaoGe](https://github.com/WindSongShaoGe) **Created:** 9/11/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `fix/p2p-snapshot-layer-compatibility` --- ### 📝 Commits (2) - [`dd69d8e`](https://github.com/kvcache-ai/AgentENV/commit/dd69d8e389c17af43e155946d7a96218d45ba2fc) fix(p2p): restore OSS snapshot layers through peers - [`1bfe8a1`](https://github.com/kvcache-ai/AgentENV/commit/1bfe8a1d05596ab274cae26732dcd840439a0d31) fix(p2p): retain origin fallback after peer open ### 📊 Changes **7 files changed** (+488 additions, -43 deletions) <details> <summary>View changed files</summary> 📝 `docs/src/internals/p2p-design.md` (+3 -1) 📝 `src/overlaybd/p2p/facade.rs` (+152 -13) 📝 `src/snapshot/manager.rs` (+24 -15) 📝 `src/snapshot/p2p.rs` (+78 -5) 📝 `storage/overlaybd/src/backend/registryfs_v2.rs` (+6 -1) 📝 `storage/overlaybd/src/image/image_file.rs` (+216 -7) 📝 `storage/overlaybd/src/image/image_service.rs` (+9 -1) </details> ### 📄 Description ## What Fix peer reads of OSS-backed snapshot layers. OverlayBD URL probes now send a bounded Range header, digest-addressed layers can be opened through the P2P facade, and local snapshot layers without digest/size metadata are described before publication. Enable TCP_NODELAY on facade connections used for small range reads during restore. ## Why The actual-URL probe previously sent GET without Range, so the range-only UUID facade returned HTTP 400. Fixing that probe alone still leaves layers without UUIDs on the OSS read path. In a cold-node restore where OSS data reads are blocked, these gaps prevent template or snapshot recovery. The complete patch passed the strict two-node full-chain suite on main commit 3815a57: template launch and captured disk/memory recovery issued zero OSS data GETs, and OCI conversion reuse issued zero source-layer GETs. ## Related issue No linked issue. A separate compatibility fix was requested in prior maintainer discussion. ## Scope and non-goals Includes layer publication/read-path fixes, focused regression tests, and P2P design documentation. The standalone full-chain E2E suite and its Compose/configuration files are retained on a separate test branch and are excluded from this PR. No changes to snapshot storage formats or publication compression policy. ## Design and behavior changes - Add `Range: bytes=0-0` to actual-URL probes, matching the existing authentication probe. - Add `/p2p-digest/{digest}` using the existing canonical layer identity, descriptor validation, and bounded-range handling shared with the UUID facade. - Try UUID P2P reads when available, then digest P2P reads, then the existing origin path on open failure. Explicit local-file behavior is unchanged. - Hash local layers that lack digest/size metadata before applying the committed-digest publication guard. Bytes whose digest differs from the committed record remain excluded. - Set TCP_NODELAY on accepted facade sockets to avoid Nagle/delayed-ACK stalls for small streamed ranges. ## Compatibility and operations - Public API or generated protocol: unchanged; adds an internal loopback facade endpoint. - Configuration or defaults: no schema/default changes; uses existing P2P settings. - Snapshot manifest, artifact layout, or storage format: unchanged; reuses existing layer keys. - Upgrade and rollback: rebuild/deploy server and ublk-daemon together; no data migration. Origin fallback remains available. - Host requirements, permissions, ports, or dependencies: unchanged. ## Validation - [x] `make fmt` - [x] `make clippy` - [ ] `make test-unit` - [ ] Relevant Rust integration tests - [ ] `make -C services test` (required when `services/` changes) - [ ] Generated clients/server regenerated with the documented `make` target - [x] Documentation updated - [ ] Benchmarks or performance comparison completed Commands and results: ```text cargo build --release -p agentenv --bin server -p uvm-ublk-daemon --bin uvm-ublk-daemon PASS cargo test -p agentenv --lib p2p:: 60 passed cargo test -p overlaybd --lib image:: 42 passed cargo test -p overlaybd --lib backend::registryfs_v2::tests 18 passed make fmt PASS make clippy PASS (workspace, all targets/features, -D warnings) ``` Strict E2E validation used the separately retained test commit `9a7cd21` (main `3815a57` plus the complete patch). The production source files in that commit are byte-identical to this PR. Server/daemon were freshly built from that source and installed in an isolated test image using the previously passing run's dependencies. ```bash env SKIP_BUILD=1 P2P_SNAPSHOT_MODE=strict \ E2E_P2P_RUNTIME_IMAGE=agentenv-runtime:p2p-main-9a7cd21 \ AENV_TEST_STATE_DIR=/home/wenhao/AgentENV/target/p2p-latest-main-validation \ AENV_NODE_A_PORT=18081 AENV_NODE_B_PORT=18082 \ bash scripts/tests/e2e/run_p2p_full_chain.sh ``` Result: all 6 stages passed (template P2P launch, source capture, snapshot P2P recovery, OSS fallback, OCI conversion reuse, and OCI fallback). The harness's 12 Python tests also passed. Run ID: `1789115644-117861`. Template/snapshot data GETs to OSS and OCI source-layer GETs were zero in the corresponding P2P stages. The test image and suite are local validation artifacts, not public dependencies of this PR. Skipped checks and reasons: Aggregate `make test-unit` and other Rust integration targets were not run; validation focused on the affected modules and the strict two-node VM E2E suite. No Go or generated-code changes, so Go checks and regeneration were not required. No standalone performance benchmark was run. ## Risks and reviewer notes This is broader than the earlier Range-only patch: digest reads, descriptorless-layer publication, and TCP_NODELAY are also included. The validation demonstrates the combined patch, not the individual performance contribution of each change. Digest probing adds a P2P attempt on remote-layer opens. Computing missing local descriptors adds hashing work at publication. Publication still requires a digest present in the committed record. Publish-time recompression that changes local bytes remains outside the validated strict scenario; the E2E configuration disables publication compression. The facade does not receive OSS credentials, and P2P failures retain the existing origin fallback. ## Checklist - [x] The PR contains one coherent change and no unrelated formatting or refactoring. - [x] New behavior is covered by tests, or I explained why testing is impractical. - [x] Logs and examples contain no credentials, tokens, or private registry information. - [x] I did not manually edit generated code without updating its source and regenerating it. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-29 14:22:04 +00:00