[PR #304] [MERGED] fix(p2p): publish committed OSS layer artifacts

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/AgentENV/pull/304 **Author:** [@WindSongShaoGe](https://github.com/WindSongShaoGe) **Created:** 9/19/2026 **Status:** ✅ Merged **Merged:** 9/28/2026 **Merged by:** [@LSX-s-Software](https://github.com/LSX-s-Software) **Base:** `main` ← **Head:** `fix/p2p-publish-committed-layers` --- ### 📝 Commits (1) - [`461b246`](https://github.com/kvcache-ai/AgentENV/commit/461b246e1242819262e4cd5976c32fd1d6415c61) fix(p2p): publish committed OSS layer artifacts ### 📊 Changes **8 files changed** (+373 additions, -50 deletions) <details> <summary>View changed files</summary> 📝 `crates/e2e-tests/tests/snapshot_oss_e2e_test.rs` (+143 -5) 📝 `docs/src/internals/p2p-design.md` (+3 -1) 📝 `src/snapshot/manager.rs` (+68 -11) 📝 `src/snapshot/p2p.rs` (+2 -4) 📝 `src/snapshot/repository/backends/common/acr/publisher.rs` (+22 -7) 📝 `src/snapshot/repository/backends/common/recontainerize.rs` (+39 -1) 📝 `src/snapshot/repository/backends/oss/repository.rs` (+74 -21) 📝 `src/snapshot/repository/interfaces.rs` (+22 -0) </details> ### 📄 Description ## What Publish OSS snapshot layers under their committed digest even when local image configs have no descriptor or upload preparation produces different bytes. Suggested title: `fix(p2p): publish committed OSS layer artifacts` ## Why Post-commit publication currently scans local image configs. Descriptorless memory layers have no digest to advertise, and compressed or dense-exported upload files are dropped before publication. Consumers cannot discover the committed layer by its digest. ## Related issue Follow-up to the P2P snapshot publication investigation. PR #293 addressed URL probe Range compatibility; this PR addresses the separate publication gaps. ## Scope and non-goals Includes committed layer publication and its regression tests. Digest-only OSS reads and the dedicated full-chain E2E harness remain separate changes. ## Design and behavior changes - Return prepared uploads alongside the committed record, keeping transformed temporary files alive through post-commit P2P copies. - Reuse upload descriptors without another SHA256 pass. Preserve UUID publication and deduplicate digest keys. - Retain artifacts from the shared ACR export path when OSS uses source-registry publication. - Keep publication best-effort and strictly after repository success. Preserve startup-recording forwarding and the asynchronous startup-manifest flow. ## Compatibility and operations - Public API or generated protocol: no wire changes; existing repository publish signature remains available, with an additive default trait method for retained uploads. - Configuration or defaults: unchanged. - Snapshot manifest, artifact layout, or storage format: unchanged; temporary paths are not persisted. - Upgrade and rollback: existing snapshots remain compatible; rollback removes the additional digest publication. - Host requirements, permissions, ports, or dependencies: unchanged. ## Validation - [x] `make fmt` - [x] `make clippy` - [ ] `make test-unit` - [x] Relevant Rust integration tests - [ ] `make -C services test` (required when `services/` changes) - [ ] Generated clients/server regenerated with the documented `make` target - [x] Documentation updated - [ ] Benchmarks or performance comparison completed Commands and results: ```text cargo test -p agentenv --lib snapshot:: 149 passed cargo test -p agentenv-e2e-tests --test snapshot_oss_e2e_test -- --ignored --test-threads=1 6 passed make fmt passed make clippy passed (workspace, all targets/features, -D warnings) git diff --check passed ``` On unmodified main 531a002 with only the integration-test changes applied, all three `snapshot_oss_publish*` tests fail because the committed digest is absent from the peer. With the implementation, all pass. Real MinIO/iroh tests compare peer bytes against OSS objects, including compression and sparse dense-export, and verify access after capture cleanup. Skipped checks and reasons: the broader capability/VM suites and E2B compatibility suite were not run for this publication-only change. No Go or generated schema changes. No performance benchmark was run. ## Risks and reviewer notes Prepared temporary files now remain until post-commit P2P publication finishes, extending their disk lifetime. Review the ownership transfer through `SnapshotPublication` and the dense-export temporary guard. The read-side digest route is intentionally a separate PR, so this change alone does not establish end-to-end zero-origin restore. ## Checklist - [x] The PR contains one coherent change and no unrelated formatting or refactoring. - [x] New behavior is covered by tests, or I explained why testing is impractical. - [x] Logs and examples contain no credentials, tokens, or private registry information. - [x] I did not manually edit generated code without updating its source and regenerating it. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi commented

2026-10-29 14:22:17 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/AgentENV/pull/304
Author: @WindSongShaoGe
Created: 9/19/2026
Status: ✅ Merged
Merged: 9/28/2026
Merged by: @LSX-s-Software

Base: main ← Head: fix/p2p-publish-committed-layers

📝 Commits (1)

461b246 fix(p2p): publish committed OSS layer artifacts

📊 Changes

8 files changed (+373 additions, -50 deletions)

View changed files

📝 crates/e2e-tests/tests/snapshot_oss_e2e_test.rs (+143 -5)
📝 docs/src/internals/p2p-design.md (+3 -1)
📝 src/snapshot/manager.rs (+68 -11)
📝 src/snapshot/p2p.rs (+2 -4)
📝 src/snapshot/repository/backends/common/acr/publisher.rs (+22 -7)
📝 src/snapshot/repository/backends/common/recontainerize.rs (+39 -1)
📝 src/snapshot/repository/backends/oss/repository.rs (+74 -21)
📝 src/snapshot/repository/interfaces.rs (+22 -0)

📄 Description

What

Publish OSS snapshot layers under their committed digest even when local image configs have no descriptor or upload preparation produces different bytes.

Suggested title: fix(p2p): publish committed OSS layer artifacts

Why

Post-commit publication currently scans local image configs. Descriptorless memory layers have no digest to advertise, and compressed or dense-exported upload files are dropped before publication. Consumers cannot discover the committed layer by its digest.

Follow-up to the P2P snapshot publication investigation. PR #293 addressed URL probe Range compatibility; this PR addresses the separate publication gaps.

Scope and non-goals

Includes committed layer publication and its regression tests. Digest-only OSS reads and the dedicated full-chain E2E harness remain separate changes.

Design and behavior changes

Return prepared uploads alongside the committed record, keeping transformed temporary files alive through post-commit P2P copies.
Reuse upload descriptors without another SHA256 pass. Preserve UUID publication and deduplicate digest keys.
Retain artifacts from the shared ACR export path when OSS uses source-registry publication.
Keep publication best-effort and strictly after repository success. Preserve startup-recording forwarding and the asynchronous startup-manifest flow.

Compatibility and operations

Public API or generated protocol: no wire changes; existing repository publish signature remains available, with an additive default trait method for retained uploads.
Configuration or defaults: unchanged.
Snapshot manifest, artifact layout, or storage format: unchanged; temporary paths are not persisted.
Upgrade and rollback: existing snapshots remain compatible; rollback removes the additional digest publication.
Host requirements, permissions, ports, or dependencies: unchanged.

Validation

make fmt
make clippy
make test-unit
Relevant Rust integration tests
make -C services test (required when services/ changes)
Generated clients/server regenerated with the documented make target
Documentation updated
Benchmarks or performance comparison completed

Commands and results:

cargo test -p agentenv --lib snapshot::
149 passed

cargo test -p agentenv-e2e-tests --test snapshot_oss_e2e_test -- --ignored --test-threads=1
6 passed

make fmt
passed

make clippy
passed (workspace, all targets/features, -D warnings)

git diff --check
passed

On unmodified main 531a002 with only the integration-test changes applied, all three snapshot_oss_publish* tests fail because the committed digest is absent from the peer. With the implementation, all pass. Real MinIO/iroh tests compare peer bytes against OSS objects, including compression and sparse dense-export, and verify access after capture cleanup.

Skipped checks and reasons: the broader capability/VM suites and E2B compatibility suite were not run for this publication-only change. No Go or generated schema changes. No performance benchmark was run.

Risks and reviewer notes

Prepared temporary files now remain until post-commit P2P publication finishes, extending their disk lifetime. Review the ownership transfer through SnapshotPublication and the dense-export temporary guard. The read-side digest route is intentionally a separate PR, so this change alone does not establish end-to-end zero-origin restore.

Checklist

The PR contains one coherent change and no unrelated formatting or refactoring.
New behavior is covered by tests, or I explained why testing is impractical.
Logs and examples contain no credentials, tokens, or private registry information.
I did not manually edit generated code without updating its source and regenerating it.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

bsakaguchi

closed this issue
added the
pull-request
label

bsakaguchi referenced this issue

2026-10-29 14:22:33 +00:00

[PR #332] fix(oss): use if_not_exists for put-if-absent on S3 backends #354

Rows
Columns

[PR #304] [MERGED] fix(p2p): publish committed OSS layer artifacts #332