mirror of
https://github.com/kvcache-ai/AgentENV.git
synced 2026-11-05 01:17:36 +00:00
[PR #370] [MERGED] refactor(snapshot): move catalog, publish and resolve logic out of repository backends #384
Labels
No labels
bug
documentation
enhancement
enhancement
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
kvcache-ai/AgentENV#384
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
📋 Pull Request Information
Original PR: https://github.com/kvcache-ai/AgentENV/pull/370
Author: @yingdi-shan
Created: 10/12/2026
Status: ✅ Merged
Merged: 10/13/2026
Merged by: @yingdi-shan
Base:
main← Head:refactor/repository-layers📝 Commits (6)
d541eb0fix(repository): serialize OSS document writers with a lease on Alibaba OSSa4d1f36refactor(snapshot): implement the snapshot catalog once on repository documents87c19dcrefactor(snapshot): move registry export and recontainerization out of the repository6d25a4frefactor(snapshot): store snapshot and volume layers through one shared pipelinef8a172erefactor(snapshot): resolve snapshots and volume backings with one resolvercd212f4refactor(repository): name every blob path in the repository layout📊 Changes
50 files changed (+4750 additions, -5320 deletions)
View changed files
📝
Cargo.lock(+1 -0)📝
Cargo.toml(+2 -1)📝
crates/e2e-tests/tests/snapshot_oss_e2e_test.rs(+31 -3)📝
crates/object-store-operator/Cargo.toml(+1 -1)📝
crates/object-store-operator/src/lib.rs(+2 -2)📝
crates/object-store-operator/src/operator.rs(+52 -16)📝
docs/src/internals/persistence-artifact-inventory.md(+4 -2)📝
src/repository/backends/common/mod.rs(+1 -83)📝
src/repository/backends/mod.rs(+1 -1)📝
src/repository/backends/oss/client.rs(+264 -47)📝
src/repository/backends/oss/config.rs(+0 -14)➕
src/repository/backends/oss/documents.rs(+244 -0)➖
src/repository/backends/oss/layout.rs(+0 -32)➕
src/repository/backends/oss/lease.rs(+378 -0)📝
src/repository/backends/oss/mod.rs(+11 -13)📝
src/repository/backends/oss/repository.rs(+107 -1845)➖
src/repository/backends/oss/resolver.rs(+0 -501)📝
src/repository/backends/posixfs/artifacts.rs(+221 -527)📝
src/repository/backends/posixfs/backend.rs(+116 -393)📝
src/repository/backends/posixfs/catalog.rs(+36 -1073)...and 30 more files
📄 Description
What
Moves the remaining snapshot logic out of the repository backends. A backend now only stores documents and blobs. The snapshot module implements each rule once:
fix(repository): serialize OSS document writers with a lease on Alibaba OSSlocks/, claimed by appending at its current length. S3/MinIO keep the ETag compare-and-swap, now retried with jittered backoff. Documents stay single{name}.jsonobjects.refactor(snapshot): implement the snapshot catalog once on repository documentssrc/snapshot/catalog.rs: record first, alias claimed by compare-and-swap, stale-alias removal, one template builder. Publish orchestration and the startup-manifest continuation move tosrc/snapshot/publish.rs.refactor(snapshot): move registry export and recontainerization out of the repositorycommon/acr/→src/snapshot/image_export/acr/,recontainerize.rs→src/snapshot/.refactor(snapshot): store snapshot and volume layers through one shared pipelinesrc/snapshot/store.rshandles, for both snapshots and volumes: layer derivation, descriptor checks, dense export, publish compression, and the source-registry export with its fallback. It writes throughput_blob(BlobKey, BlobBody),delete_snapshot_blobsandstaging_file.refactor(snapshot): resolve snapshots and volume backings with one resolversrc/snapshot/resolver.rsimplementsSnapshotRuntimeResolveronce. It reads throughlocate_blob/has_blob/fetch_blob/read_blob, which take the sameBlobKey.refactor(repository): name every blob path in the repository layoutBlobKeyandSnapshotFilemove tosrc/repository/layout.rsnext toDocumentKey.path()gives the object key andposix_path()the POSIX path. The backendlayout.rsmodules are removed, and the image exporter and the OSS layer URL use these paths. Paths are unchanged.Stacked on #369 (which stacks on #368 and #367). This PR's own commits are the six listed above. The first three commits on the branch belong to those PRs.
Why
Each backend had its own copy of the catalog rules, the publish pipeline and the runtime resolver. The copies had drifted: error messages differed, validation ran in a different order, and only OSS handled aliases. Any rule change had to be made twice.
Separately, #369's ETag compare-and-swap does not work on Alibaba OSS:
PutObjectrejectsIf-Match/If-None-MatchwithNotImplemented. That broke volume reservations and build-cache updates on every write.Related issue
N/A. Follows the discussion on #367–#369.
Scope and non-goals
staging/files left by a crash;SnapshotPublicationstays inrepository::interfaces.write_dense_overlaybd_layer_to_filestays inrepository/backends/common.managed_layers_repo_blob_url()stays: it is therepoBlobUrloverlaybd uses for the whole layer namespace.Design and behavior changes
locks/{document path}, which holds JSON-line frames{"at", "prev", "token"}.prev. This prevents ABA after the lock object is deleted and recreated.ossservice for appends and guarded writes. Everything else keeps the S3 service.BlobKey::{Layer { digest, role }, Snapshot(id, SnapshotFile)}withBlobBody::{File, Bytes}.role(rootfs / attached drive / memory) only labels metrics and logs; layers are stored by digest.locate_blobreturnsBlobLocation::Local(path)on POSIX andBlobLocation::Remote(url)on OSS.put_blob(Snapshot(id, StartupManifest), Bytes); there is no separate method for it.deleting, then releases the alias, rolls back source-registry publications (best-effort), deletes blobs, then the build log, then the record. Because the record goes last, a failed delete can be retried.<root>/staging/, on the repository's filesystem, and hard-linked intolayers/. There is no extra copy when/tmpis on another filesystem. OSS stages in the system temp dir as before.vm_state.bin;firecracker-manifest.jsonis written atomically (temp + rename);ArtifactNotFoundeven when peers have it (new testposix_resolve_never_repairs_missing_files_from_peers)."<label>: managed layer N '<digest>' is missing"on both backends.Compatibility and operations
PublishPolicy::default()is raw / object storage. The OSS backend passes its configured policy as before.{name}.jsonpaths.locks/, which older nodes never list.deletingfield. It is omitted when false and ignored by older nodes.staging/directory under the repository root; commit-marker files from old snapshots are ignored.services-ossfeature.Validation
make fmtmake clippymake test-unitmake -C services test(required whenservices/changes)maketargetCommands and results (on the final commit; fmt, clippy, lib tests and the MinIO e2e suite also passed after each commit):
New tests:
Lease, on a fake append store with per-request latency:
The first test was checked to fail when the acquire logic is mutated.
16 concurrent document updates on MinIO, all landing (
concurrent_document_updates_on_oss_lose_none).Only Alibaba OSS endpoints get the native
ossservice.Shared catalog:
POSIX dense exports hard-linked from
staging/.Capture-lease tests via
SnapshotManager.Blob and document paths pinned to existing repository layouts.
Shared store and resolver unit tests.
POSIX no-peer-repair.
Skipped checks and reasons:
ConditionNotMatch.make -C services test: noservices/changes.Risks and reviewer notes
Most important files:
src/repository/backends/oss/lease.rssrc/repository/backends/oss/documents.rssrc/snapshot/catalog.rssrc/snapshot/store.rssrc/snapshot/resolver.rssrc/repository/interfaces.rssrc/repository/layout.rsLease timing: safety relies on local monotonic time only, not on synchronized clocks. It assumes two things:
If either assumption fails, a taken-over holder's late write can overwrite its successor's.
S3 conditional deletes are not atomic (see above). This is the same race as on
main.#369's description says it fixes Aliyun; that fix only lands with the first commit here.
Checklist
🤖 Generated with Claude Code
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.