[PR #370] [MERGED] refactor(snapshot): move catalog, publish and resolve logic out of repository backends #384

Closed
opened 2026-10-29 14:22:51 +00:00 by bsakaguchi · 0 comments
Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/AgentENV/pull/370
Author: @yingdi-shan
Created: 10/12/2026
Status: ✅ Merged
Merged: 10/13/2026
Merged by: @yingdi-shan

Base: main ← Head: refactor/repository-layers


📝 Commits (6)

  • d541eb0 fix(repository): serialize OSS document writers with a lease on Alibaba OSS
  • a4d1f36 refactor(snapshot): implement the snapshot catalog once on repository documents
  • 87c19dc refactor(snapshot): move registry export and recontainerization out of the repository
  • 6d25a4f refactor(snapshot): store snapshot and volume layers through one shared pipeline
  • f8a172e refactor(snapshot): resolve snapshots and volume backings with one resolver
  • cd212f4 refactor(repository): name every blob path in the repository layout

📊 Changes

50 files changed (+4750 additions, -5320 deletions)

View changed files

📝 Cargo.lock (+1 -0)
📝 Cargo.toml (+2 -1)
📝 crates/e2e-tests/tests/snapshot_oss_e2e_test.rs (+31 -3)
📝 crates/object-store-operator/Cargo.toml (+1 -1)
📝 crates/object-store-operator/src/lib.rs (+2 -2)
📝 crates/object-store-operator/src/operator.rs (+52 -16)
📝 docs/src/internals/persistence-artifact-inventory.md (+4 -2)
📝 src/repository/backends/common/mod.rs (+1 -83)
📝 src/repository/backends/mod.rs (+1 -1)
📝 src/repository/backends/oss/client.rs (+264 -47)
📝 src/repository/backends/oss/config.rs (+0 -14)
➕ src/repository/backends/oss/documents.rs (+244 -0)
➖ src/repository/backends/oss/layout.rs (+0 -32)
➕ src/repository/backends/oss/lease.rs (+378 -0)
📝 src/repository/backends/oss/mod.rs (+11 -13)
📝 src/repository/backends/oss/repository.rs (+107 -1845)
➖ src/repository/backends/oss/resolver.rs (+0 -501)
📝 src/repository/backends/posixfs/artifacts.rs (+221 -527)
📝 src/repository/backends/posixfs/backend.rs (+116 -393)
📝 src/repository/backends/posixfs/catalog.rs (+36 -1073)

...and 30 more files

📄 Description

What

Moves the remaining snapshot logic out of the repository backends. A backend now only stores documents and blobs. The snapshot module implements each rule once:

Commit Change
fix(repository): serialize OSS document writers with a lease on Alibaba OSS On Alibaba OSS, every document write holds a lease: an appendable lock object under locks/, claimed by appending at its current length. S3/MinIO keep the ETag compare-and-swap, now retried with jittered backoff. Documents stay single {name}.json objects.
refactor(snapshot): implement the snapshot catalog once on repository documents Snapshot records and aliases become shared documents. The catalog rules live in src/snapshot/catalog.rs: record first, alias claimed by compare-and-swap, stale-alias removal, one template builder. Publish orchestration and the startup-manifest continuation move to src/snapshot/publish.rs.
refactor(snapshot): move registry export and recontainerization out of the repository Pure move: common/acr/ → src/snapshot/image_export/acr/, recontainerize.rs → src/snapshot/.
refactor(snapshot): store snapshot and volume layers through one shared pipeline src/snapshot/store.rs handles, for both snapshots and volumes: layer derivation, descriptor checks, dense export, publish compression, and the source-registry export with its fallback. It writes through put_blob(BlobKey, BlobBody), delete_snapshot_blobs and staging_file.
refactor(snapshot): resolve snapshots and volume backings with one resolver src/snapshot/resolver.rs implements SnapshotRuntimeResolver once. It reads through locate_blob / has_blob / fetch_blob / read_blob, which take the same BlobKey.
refactor(repository): name every blob path in the repository layout BlobKey and SnapshotFile move to src/repository/layout.rs next to DocumentKey. path() gives the object key and posix_path() the POSIX path. The backend layout.rs modules are removed, and the image exporter and the OSS layer URL use these paths. Paths are unchanged.

Stacked on #369 (which stacks on #368 and #367). This PR's own commits are the six listed above. The first three commits on the branch belong to those PRs.

Why

Each backend had its own copy of the catalog rules, the publish pipeline and the runtime resolver. The copies had drifted: error messages differed, validation ran in a different order, and only OSS handled aliases. Any rule change had to be made twice.

Separately, #369's ETag compare-and-swap does not work on Alibaba OSS: PutObject rejects If-Match/If-None-Match with NotImplemented. That broke volume reservations and build-cache updates on every write.

N/A. Follows the discussion on #367–#369.

Scope and non-goals

  • No behavior change intended for POSIX users, apart from the items under "Design and behavior changes".
  • Not included:
    • managed-layer GC;
    • cleanup of POSIX staging/ files left by a crash;
    • cleanup of a lock object whose holder crashed when no later writer takes it over.
  • SnapshotPublication stays in repository::interfaces. write_dense_overlaybd_layer_to_file stays in repository/backends/common.
  • managed_layers_repo_blob_url() stays: it is the repoBlobUrl overlaybd uses for the whole layer namespace.

Design and behavior changes

  • Document writes on Alibaba OSS (records, aliases, volumes, build logs, build cache) are serialized by a lease. The lease is the append-only object locks/{document path}, which holds JSON-line frames {"at", "prev", "token"}.
    • AppendObject succeeds only at the object's current length. Of several racing appends, only one can land at a given offset.
    • A frame counts only if it landed at the offset it claims and names the current holder as prev. This prevents ABA after the lock object is deleted and recreated.
    • A contender takes over after it has seen the same holder for 30 s. A holder's deadline is send time + 30 s − 10 s. Each guarded write is one attempt of at most 4 s, started only before the deadline. The holder releases with a DELETE bounded by the same deadline. After a failed write, the lease is left to expire.
    • Aliyun endpoints use opendal's native oss service for appends and guarded writes. Everything else keeps the S3 service.
  • Document writes on S3/MinIO: ETag compare-and-swap, with jittered backoff (20 ms → 500 ms) for up to 32 attempts. A conditional delete checks the ETag of a fresh read, then deletes; this is not atomic, because opendal cannot delete conditionally.
  • Blob API:
    • BlobKey::{Layer { digest, role }, Snapshot(id, SnapshotFile)} with BlobBody::{File, Bytes}. role (rootfs / attached drive / memory) only labels metrics and logs; layers are stored by digest.
    • locate_blob returns BlobLocation::Local(path) on POSIX and BlobLocation::Remote(url) on OSS.
    • The startup manifest is now put_blob(Snapshot(id, StartupManifest), Bytes); there is no separate method for it.
  • Snapshot delete marks the record deleting, then releases the alias, rolls back source-registry publications (best-effort), deletes blobs, then the build log, then the record. Because the record goes last, a failed delete can be retried.
    • A startup manifest is attached only to a record that is committed and not deleting. Otherwise the continuation deletes the snapshot's blobs, so a manifest uploaded after the delete removed the files no longer leaks.
  • Volume delete releases its name only if the name still points at that volume.
  • Zero-copy staging on POSIX: dense exports and recontainerized layers are prepared in <root>/staging/, on the repository's filesystem, and hard-linked into layers/. There is no extra copy when /tmp is on another filesystem. OSS stages in the system temp dir as before.
  • Source-registry exporter: now created per publish/delete instead of once per repository. Its registry client and token are no longer reused across publishes.
  • POSIX publish runs the shared pipeline:
    • layers are stored before vm_state.bin;
    • firecracker-manifest.json is written atomically (temp + rename);
    • stored layers are also handed to post-commit P2P publication when a transport is configured.
  • POSIX resolve: blobs are read in place and never fetched. A missing file is ArtifactNotFound even when peers have it (new test posix_resolve_never_repairs_missing_files_from_peers).
  • Error text: some resolver labels and error messages changed. For example, the missing-layer error is now "<label>: managed layer N '<digest>' is missing" on both backends.

Compatibility and operations

  • Public API or generated protocol: no change.
  • Configuration or defaults: no change. PublishPolicy::default() is raw / object storage. The OSS backend passes its configured policy as before.
  • Snapshot manifest, artifact layout, or storage format:
    • OSS documents keep their {name}.json paths.
    • Lock objects live under locks/, which older nodes never list.
    • Snapshot records gain an optional deleting field. It is omitted when false and ignored by older nodes.
    • POSIX gains a staging/ directory under the repository root; commit-marker files from old snapshots are ignored.
  • Upgrade and rollback: nodes can upgrade one at a time. Old and new nodes read and write the same objects.
    • During a mixed rollout on Aliyun, old nodes do not take the lease, so their writes can still race as they do today.
    • On S3, old and new nodes both use ETag compare-and-swap.
    • Rollback needs no data migration.
  • Host requirements, permissions, ports, or dependencies: opendal gains the services-oss feature.

Validation

  • make fmt
  • make clippy
  • make test-unit
  • Relevant Rust integration tests
  • make -C services test (required when services/ changes)
  • Generated clients/server regenerated with the documented make target
  • Documentation updated
  • Benchmarks or performance comparison completed

Commands and results (on the final commit; fmt, clippy, lib tests and the MinIO e2e suite also passed after each commit):

make fmt                                    # clean
make clippy                                 # clean
cargo test -p agentenv --lib                # 976 passed, 4 ignored
make test-unit                              # ok
cargo test -p agentenv-e2e-tests --test snapshot_oss_e2e_test -- --ignored   # 9 passed (MinIO)
make test-agent-integration                 # integration 21 passed / 1 ignored; orchestrator 2 passed; OSS e2e 9 passed

New tests:

  • Lease, on a fake append store with per-request latency:

    • concurrent writers lose no update;
    • a live holder keeps the lock;
    • a dead holder is taken over after the lease;
    • an append whose reply was lost still takes the lock;
    • an expired lease is left to run out;
    • only frames landing where they claim hold the lock.

    The first test was checked to fail when the acquire logic is mutated.

  • 16 concurrent document updates on MinIO, all landing (concurrent_document_updates_on_oss_lose_none).

  • Only Alibaba OSS endpoints get the native oss service.

  • Shared catalog:

    • list filters;
    • alias conflict and stale reclaim;
    • double commit;
    • one builder out of 16;
    • startup manifest after delete;
    • no startup manifest once a delete starts.
  • POSIX dense exports hard-linked from staging/.

  • Capture-lease tests via SnapshotManager.

  • Blob and document paths pinned to existing repository layouts.

  • Shared store and resolver unit tests.

  • POSIX no-peer-repair.

Skipped checks and reasons:

  • Not tested against real Alibaba OSS (no credentials). The lease runs only against the fake store. Its OSS path depends on AppendObject returning 409 when the position does not match, which opendal maps to ConditionNotMatch.
  • make -C services test: no services/ changes.
  • No benchmark. On Aliyun, a document update adds about four requests: append HEAD + APPEND, a lock read when contended, and the release DELETE.

Risks and reviewer notes

  • Most important files:

    • src/repository/backends/oss/lease.rs
    • src/repository/backends/oss/documents.rs
    • src/snapshot/catalog.rs
    • src/snapshot/store.rs
    • src/snapshot/resolver.rs
    • src/repository/interfaces.rs
    • src/repository/layout.rs
  • Lease timing: safety relies on local monotonic time only, not on synchronized clocks. It assumes two things:

    • a guarded write is never applied by the server long after its 4 s client timeout;
    • a holder does not stall for more than the 10 s margin between its deadline check and sending the request.

    If either assumption fails, a taken-over holder's late write can overwrite its successor's.

  • S3 conditional deletes are not atomic (see above). This is the same race as on main.

  • #369's description says it fixes Aliyun; that fix only lands with the first commit here.

Checklist

  • The PR contains one coherent change and no unrelated formatting or refactoring.
  • New behavior is covered by tests, or I explained why testing is impractical.
  • Logs and examples contain no credentials, tokens, or private registry information.
  • I did not manually edit generated code without updating its source and regenerating it.

🤖 Generated with Claude Code


🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/AgentENV/pull/370 **Author:** [@yingdi-shan](https://github.com/yingdi-shan) **Created:** 10/12/2026 **Status:** ✅ Merged **Merged:** 10/13/2026 **Merged by:** [@yingdi-shan](https://github.com/yingdi-shan) **Base:** `main` ← **Head:** `refactor/repository-layers` --- ### 📝 Commits (6) - [`d541eb0`](https://github.com/kvcache-ai/AgentENV/commit/d541eb0a82b7e08d893649207ae6e7e594cc02e4) fix(repository): serialize OSS document writers with a lease on Alibaba OSS - [`a4d1f36`](https://github.com/kvcache-ai/AgentENV/commit/a4d1f36c9e10b79b8a55ee95047956ce195c3ea8) refactor(snapshot): implement the snapshot catalog once on repository documents - [`87c19dc`](https://github.com/kvcache-ai/AgentENV/commit/87c19dc4b76f52ccd896c51a776d1f555616883d) refactor(snapshot): move registry export and recontainerization out of the repository - [`6d25a4f`](https://github.com/kvcache-ai/AgentENV/commit/6d25a4fa27d5915a3fed1a34ea68cf2a5c7a9445) refactor(snapshot): store snapshot and volume layers through one shared pipeline - [`f8a172e`](https://github.com/kvcache-ai/AgentENV/commit/f8a172e92b9b60b347387903ba854730b3d0b37b) refactor(snapshot): resolve snapshots and volume backings with one resolver - [`cd212f4`](https://github.com/kvcache-ai/AgentENV/commit/cd212f4207ee23ca73933b32ee63a0bd7dde213f) refactor(repository): name every blob path in the repository layout ### 📊 Changes **50 files changed** (+4750 additions, -5320 deletions) <details> <summary>View changed files</summary> 📝 `Cargo.lock` (+1 -0) 📝 `Cargo.toml` (+2 -1) 📝 `crates/e2e-tests/tests/snapshot_oss_e2e_test.rs` (+31 -3) 📝 `crates/object-store-operator/Cargo.toml` (+1 -1) 📝 `crates/object-store-operator/src/lib.rs` (+2 -2) 📝 `crates/object-store-operator/src/operator.rs` (+52 -16) 📝 `docs/src/internals/persistence-artifact-inventory.md` (+4 -2) 📝 `src/repository/backends/common/mod.rs` (+1 -83) 📝 `src/repository/backends/mod.rs` (+1 -1) 📝 `src/repository/backends/oss/client.rs` (+264 -47) 📝 `src/repository/backends/oss/config.rs` (+0 -14) ➕ `src/repository/backends/oss/documents.rs` (+244 -0) ➖ `src/repository/backends/oss/layout.rs` (+0 -32) ➕ `src/repository/backends/oss/lease.rs` (+378 -0) 📝 `src/repository/backends/oss/mod.rs` (+11 -13) 📝 `src/repository/backends/oss/repository.rs` (+107 -1845) ➖ `src/repository/backends/oss/resolver.rs` (+0 -501) 📝 `src/repository/backends/posixfs/artifacts.rs` (+221 -527) 📝 `src/repository/backends/posixfs/backend.rs` (+116 -393) 📝 `src/repository/backends/posixfs/catalog.rs` (+36 -1073) _...and 30 more files_ </details> ### 📄 Description ## What Moves the remaining snapshot logic out of the repository backends. A backend now only stores documents and blobs. The snapshot module implements each rule once: | Commit | Change | | --- | --- | | `fix(repository): serialize OSS document writers with a lease on Alibaba OSS` | On Alibaba OSS, every document write holds a lease: an appendable lock object under `locks/`, claimed by appending at its current length. S3/MinIO keep the ETag compare-and-swap, now retried with jittered backoff. Documents stay single `{name}.json` objects. | | `refactor(snapshot): implement the snapshot catalog once on repository documents` | Snapshot records and aliases become shared documents. The catalog rules live in `src/snapshot/catalog.rs`: record first, alias claimed by compare-and-swap, stale-alias removal, one template builder. Publish orchestration and the startup-manifest continuation move to `src/snapshot/publish.rs`. | | `refactor(snapshot): move registry export and recontainerization out of the repository` | Pure move: `common/acr/` → `src/snapshot/image_export/acr/`, `recontainerize.rs` → `src/snapshot/`. | | `refactor(snapshot): store snapshot and volume layers through one shared pipeline` | `src/snapshot/store.rs` handles, for both snapshots and volumes: layer derivation, descriptor checks, dense export, publish compression, and the source-registry export with its fallback. It writes through `put_blob(BlobKey, BlobBody)`, `delete_snapshot_blobs` and `staging_file`. | | `refactor(snapshot): resolve snapshots and volume backings with one resolver` | `src/snapshot/resolver.rs` implements `SnapshotRuntimeResolver` once. It reads through `locate_blob` / `has_blob` / `fetch_blob` / `read_blob`, which take the same `BlobKey`. | | `refactor(repository): name every blob path in the repository layout` | `BlobKey` and `SnapshotFile` move to `src/repository/layout.rs` next to `DocumentKey`. `path()` gives the object key and `posix_path()` the POSIX path. The backend `layout.rs` modules are removed, and the image exporter and the OSS layer URL use these paths. Paths are unchanged. | **Stacked on #369** (which stacks on #368 and #367). This PR's own commits are the six listed above. The first three commits on the branch belong to those PRs. ## Why Each backend had its own copy of the catalog rules, the publish pipeline and the runtime resolver. The copies had drifted: error messages differed, validation ran in a different order, and only OSS handled aliases. Any rule change had to be made twice. Separately, #369's ETag compare-and-swap does not work on Alibaba OSS: `PutObject` rejects `If-Match`/`If-None-Match` with `NotImplemented`. That broke volume reservations and build-cache updates on every write. ## Related issue N/A. Follows the discussion on #367–#369. ## Scope and non-goals - No behavior change intended for POSIX users, apart from the items under "Design and behavior changes". - Not included: - managed-layer GC; - cleanup of POSIX `staging/` files left by a crash; - cleanup of a lock object whose holder crashed when no later writer takes it over. - `SnapshotPublication` stays in `repository::interfaces`. `write_dense_overlaybd_layer_to_file` stays in `repository/backends/common`. - `managed_layers_repo_blob_url()` stays: it is the `repoBlobUrl` overlaybd uses for the whole layer namespace. ## Design and behavior changes - **Document writes on Alibaba OSS** (records, aliases, volumes, build logs, build cache) are serialized by a lease. The lease is the append-only object `locks/{document path}`, which holds JSON-line frames `{"at", "prev", "token"}`. - AppendObject succeeds only at the object's current length. Of several racing appends, only one can land at a given offset. - A frame counts only if it landed at the offset it claims and names the current holder as `prev`. This prevents ABA after the lock object is deleted and recreated. - A contender takes over after it has seen the same holder for 30 s. A holder's deadline is send time + 30 s − 10 s. Each guarded write is one attempt of at most 4 s, started only before the deadline. The holder releases with a DELETE bounded by the same deadline. After a failed write, the lease is left to expire. - Aliyun endpoints use opendal's native `oss` service for appends and guarded writes. Everything else keeps the S3 service. - **Document writes on S3/MinIO:** ETag compare-and-swap, with jittered backoff (20 ms → 500 ms) for up to 32 attempts. A conditional delete checks the ETag of a fresh read, then deletes; this is not atomic, because opendal cannot delete conditionally. - **Blob API:** - `BlobKey::{Layer { digest, role }, Snapshot(id, SnapshotFile)}` with `BlobBody::{File, Bytes}`. `role` (rootfs / attached drive / memory) only labels metrics and logs; layers are stored by digest. - `locate_blob` returns `BlobLocation::Local(path)` on POSIX and `BlobLocation::Remote(url)` on OSS. - The startup manifest is now `put_blob(Snapshot(id, StartupManifest), Bytes)`; there is no separate method for it. - **Snapshot delete** marks the record `deleting`, then releases the alias, rolls back source-registry publications (best-effort), deletes blobs, then the build log, then the record. Because the record goes last, a failed delete can be retried. - A startup manifest is attached only to a record that is committed and not deleting. Otherwise the continuation deletes the snapshot's blobs, so a manifest uploaded after the delete removed the files no longer leaks. - **Volume delete** releases its name only if the name still points at that volume. - **Zero-copy staging on POSIX:** dense exports and recontainerized layers are prepared in `<root>/staging/`, on the repository's filesystem, and hard-linked into `layers/`. There is no extra copy when `/tmp` is on another filesystem. OSS stages in the system temp dir as before. - **Source-registry exporter:** now created per publish/delete instead of once per repository. Its registry client and token are no longer reused across publishes. - **POSIX publish runs the shared pipeline:** - layers are stored before `vm_state.bin`; - `firecracker-manifest.json` is written atomically (temp + rename); - stored layers are also handed to post-commit P2P publication when a transport is configured. - **POSIX resolve:** blobs are read in place and never fetched. A missing file is `ArtifactNotFound` even when peers have it (new test `posix_resolve_never_repairs_missing_files_from_peers`). - **Error text:** some resolver labels and error messages changed. For example, the missing-layer error is now `"<label>: managed layer N '<digest>' is missing"` on both backends. ## Compatibility and operations - Public API or generated protocol: no change. - Configuration or defaults: no change. `PublishPolicy::default()` is raw / object storage. The OSS backend passes its configured policy as before. - Snapshot manifest, artifact layout, or storage format: - OSS documents keep their `{name}.json` paths. - Lock objects live under `locks/`, which older nodes never list. - Snapshot records gain an optional `deleting` field. It is omitted when false and ignored by older nodes. - POSIX gains a `staging/` directory under the repository root; commit-marker files from old snapshots are ignored. - Upgrade and rollback: **nodes can upgrade one at a time.** Old and new nodes read and write the same objects. - During a mixed rollout on Aliyun, old nodes do not take the lease, so their writes can still race as they do today. - On S3, old and new nodes both use ETag compare-and-swap. - Rollback needs no data migration. - Host requirements, permissions, ports, or dependencies: opendal gains the `services-oss` feature. ## Validation - [x] `make fmt` - [x] `make clippy` - [x] `make test-unit` - [x] Relevant Rust integration tests - [ ] `make -C services test` (required when `services/` changes) - [ ] Generated clients/server regenerated with the documented `make` target - [x] Documentation updated - [ ] Benchmarks or performance comparison completed Commands and results (on the final commit; fmt, clippy, lib tests and the MinIO e2e suite also passed after each commit): ```text make fmt # clean make clippy # clean cargo test -p agentenv --lib # 976 passed, 4 ignored make test-unit # ok cargo test -p agentenv-e2e-tests --test snapshot_oss_e2e_test -- --ignored # 9 passed (MinIO) make test-agent-integration # integration 21 passed / 1 ignored; orchestrator 2 passed; OSS e2e 9 passed ``` New tests: - Lease, on a fake append store with per-request latency: - concurrent writers lose no update; - a live holder keeps the lock; - a dead holder is taken over after the lease; - an append whose reply was lost still takes the lock; - an expired lease is left to run out; - only frames landing where they claim hold the lock. The first test was checked to fail when the acquire logic is mutated. - 16 concurrent document updates on MinIO, all landing (`concurrent_document_updates_on_oss_lose_none`). - Only Alibaba OSS endpoints get the native `oss` service. - Shared catalog: - list filters; - alias conflict and stale reclaim; - double commit; - one builder out of 16; - startup manifest after delete; - no startup manifest once a delete starts. - POSIX dense exports hard-linked from `staging/`. - Capture-lease tests via `SnapshotManager`. - Blob and document paths pinned to existing repository layouts. - Shared store and resolver unit tests. - POSIX no-peer-repair. Skipped checks and reasons: - Not tested against real Alibaba OSS (no credentials). The lease runs only against the fake store. Its OSS path depends on AppendObject returning 409 when the position does not match, which opendal maps to `ConditionNotMatch`. - `make -C services test`: no `services/` changes. - No benchmark. On Aliyun, a document update adds about four requests: append HEAD + APPEND, a lock read when contended, and the release DELETE. ## Risks and reviewer notes - Most important files: - `src/repository/backends/oss/lease.rs` - `src/repository/backends/oss/documents.rs` - `src/snapshot/catalog.rs` - `src/snapshot/store.rs` - `src/snapshot/resolver.rs` - `src/repository/interfaces.rs` - `src/repository/layout.rs` - **Lease timing:** safety relies on local monotonic time only, not on synchronized clocks. It assumes two things: - a guarded write is never applied by the server long after its 4 s client timeout; - a holder does not stall for more than the 10 s margin between its deadline check and sending the request. If either assumption fails, a taken-over holder's late write can overwrite its successor's. - S3 conditional deletes are not atomic (see above). This is the same race as on `main`. - #369's description says it fixes Aliyun; that fix only lands with the first commit here. ## Checklist - [x] The PR contains one coherent change and no unrelated formatting or refactoring. - [x] New behavior is covered by tests, or I explained why testing is impractical. - [x] Logs and examples contain no credentials, tokens, or private registry information. - [x] I did not manually edit generated code without updating its source and regenerating it. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
bsakaguchi 2026-10-29 14:22:51 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
kvcache-ai/AgentENV#384
No description provided.