[PR #299] [MERGED] fix(ublk): back off failed idle pool refills

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/AgentENV/pull/299 **Author:** [@yingdi-shan](https://github.com/yingdi-shan) **Created:** 9/16/2026 **Status:** ✅ Merged **Merged:** 9/18/2026 **Merged by:** [@yingdi-shan](https://github.com/yingdi-shan) **Base:** `main` ← **Head:** `fix/ublk-pool-refill-backoff` --- ### 📝 Commits (2) - [`3f6ee43`](https://github.com/kvcache-ai/AgentENV/commit/3f6ee43c34730dd3cce3129ffed935053b4934f2) fix(ublk): back off failed idle pool refills - [`bf8b805`](https://github.com/kvcache-ai/AgentENV/commit/bf8b805d7b1040955c1383431aaba91542c6305a) fix(ublk): raise device limit during host setup ### 📊 Changes **3 files changed** (+51 additions, -2 deletions) <details> <summary>View changed files</summary> 📝 `scripts/docker-setup.sh` (+8 -2) 📝 `scripts/install.sh` (+5 -0) 📝 `storage/ublk-daemon/src/server.rs` (+38 -0) </details> ### 📄 Description ## What Add a one-second backoff when an idle OverlayBD ublk-pool refill fails. ## Why When the kernel has no free ublk slots, `add_dev` fails while the pool remains below its watermark. The refill task immediately reschedules itself, producing a tight retry loop and unbounded warning logs. The backoff keeps automatic retry and recovery while limiting the failure rate. ## Related issue Closes #297 ## Scope and non-goals This changes only the ublk daemon refill retry path and adds a focused regression test. It does not change pool watermarks, ublk allocation, or unrelated working-tree files. ## Design and behavior changes Failed refills hold the existing `refill_inflight` guard during a one-second delay before the next scheduled attempt. Successful refills are unchanged, and retries continue automatically so refilling resumes when a slot becomes available. ## Compatibility and operations - Public API or generated protocol: N/A. - Configuration or defaults: N/A. - Snapshot manifest, artifact layout, or storage format: N/A. - Upgrade and rollback: No migration; rolling back removes the retry delay. - Host requirements, permissions, ports, or dependencies: N/A. ## Validation - [ ] `make fmt` - [ ] `make clippy` - [ ] `make test-unit` - [x] Relevant Rust integration tests - [ ] `make -C services test` (required when `services/` changes) - [ ] Generated clients/server regenerated with the documented `make` target - [ ] Documentation updated - [ ] Benchmarks or performance comparison completed Commands and results: ```text cargo fmt -p uvm-ublk-daemon PASS cargo test -p uvm-ublk-daemon PASS (86 tests) cargo clippy -p uvm-ublk-daemon --all-targets --all-features -- -D warnings PASS ``` Skipped checks and reasons: - `make fmt` could not run because the shell startup configuration references an unset `PS1` under `bash -u`; the focused crate formatter passed. - `make clippy` reached the workspace but failed on existing generated-model documentation warnings in `src/api/generated/src/models.rs`; the changed daemon crate passed Clippy. - `make test-unit` was not run because the focused daemon suite covers the changed path and passed. ## Risks and reviewer notes The delay limits failed refill attempts to at most one per second per in-flight refill task. Once device creation succeeds, the existing scheduling logic continues filling the pool. The regression test forces a prewarm failure before any ublk device allocation and verifies repeated attempts are delayed without being disabled. ## Checklist - [x] The PR contains one coherent change and no unrelated formatting or refactoring. - [x] New behavior is covered by tests, or I explained why testing is impractical. - [x] Logs and examples contain no credentials, tokens, or private registry information. - [x] I did not manually edit generated code without updating its source and regenerating it. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi commented

2026-10-29 14:22:16 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/AgentENV/pull/299
Author: @yingdi-shan
Created: 9/16/2026
Status: ✅ Merged
Merged: 9/18/2026
Merged by: @yingdi-shan

Base: main ← Head: fix/ublk-pool-refill-backoff

📝 Commits (2)

3f6ee43 fix(ublk): back off failed idle pool refills
bf8b805 fix(ublk): raise device limit during host setup

📊 Changes

3 files changed (+51 additions, -2 deletions)

View changed files

📝 scripts/docker-setup.sh (+8 -2)
📝 scripts/install.sh (+5 -0)
📝 storage/ublk-daemon/src/server.rs (+38 -0)

📄 Description

What

Add a one-second backoff when an idle OverlayBD ublk-pool refill fails.

Why

When the kernel has no free ublk slots, add_dev fails while the pool remains below its watermark. The refill task immediately reschedules itself, producing a tight retry loop and unbounded warning logs. The backoff keeps automatic retry and recovery while limiting the failure rate.

Closes #297

Scope and non-goals

This changes only the ublk daemon refill retry path and adds a focused regression test. It does not change pool watermarks, ublk allocation, or unrelated working-tree files.

Design and behavior changes

Failed refills hold the existing refill_inflight guard during a one-second delay before the next scheduled attempt. Successful refills are unchanged, and retries continue automatically so refilling resumes when a slot becomes available.

Compatibility and operations

Public API or generated protocol: N/A.
Configuration or defaults: N/A.
Snapshot manifest, artifact layout, or storage format: N/A.
Upgrade and rollback: No migration; rolling back removes the retry delay.
Host requirements, permissions, ports, or dependencies: N/A.

Validation

make fmt
make clippy
make test-unit
Relevant Rust integration tests
make -C services test (required when services/ changes)
Generated clients/server regenerated with the documented make target
Documentation updated
Benchmarks or performance comparison completed

Commands and results:

cargo fmt -p uvm-ublk-daemon                         PASS
cargo test -p uvm-ublk-daemon                        PASS (86 tests)
cargo clippy -p uvm-ublk-daemon --all-targets --all-features -- -D warnings  PASS

Skipped checks and reasons:

make fmt could not run because the shell startup configuration references an unset PS1 under bash -u; the focused crate formatter passed.
make clippy reached the workspace but failed on existing generated-model documentation warnings in src/api/generated/src/models.rs; the changed daemon crate passed Clippy.
make test-unit was not run because the focused daemon suite covers the changed path and passed.

Risks and reviewer notes

The delay limits failed refill attempts to at most one per second per in-flight refill task. Once device creation succeeds, the existing scheduling logic continues filling the pool. The regression test forces a prewarm failure before any ublk device allocation and verifies repeated attempts are delayed without being disabled.

Checklist

The PR contains one coherent change and no unrelated formatting or refactoring.
New behavior is covered by tests, or I explained why testing is impractical.
Logs and examples contain no credentials, tokens, or private registry information.
I did not manually edit generated code without updating its source and regenerating it.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

bsakaguchi

closed this issue
added the
pull-request
label

bsakaguchi referenced this issue

2026-10-29 14:22:30 +00:00

[PR #328] [MERGED] chore(deps): bump google.golang.org/grpc from 1.69.4 to 1.83.2 in /containerd-plain-snapshotter #351

Rows
Columns

[PR #299] [MERGED] fix(ublk): back off failed idle pool refills #328