[PR #68] feat(scheduler): add resource-bounded grouped round-robin scheduling #154

New issue

Open

opened 2026-10-29 14:20:21 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 14:20:21 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/AgentENV/pull/68
Author: @shudorcl
Created: 7/29/2026
Status: 🔄 Open

Base: main ← Head: feat/grouped-locality-scheduling

📝 Commits (8)

c46bf05 feat(gateway): propagate template scheduling identity
ecce76b refactor(scheduler): derive current scheduling status
41c67ec feat(scheduler): add resource-bounded locality strategy
742aa2d feat(scheduler): configure locality group limits
59e8787 refactor(scheduler): rename locality strategy
f2b8e65 refactor(scheduler): inline group fullness check
bcce052 test(scheduler): cover scheduling snapshot derivation
33ea518 fix(scheduler): bound omitted cold-start resources

📊 Changes

14 files changed (+929 additions, -33 deletions)

View changed files

📝 services/README.md (+25 -2)
📝 services/api/proto/scheduler.pb.go (+15 -3)
📝 services/api/proto/scheduler.proto (+3 -0)
📝 services/gateway/internal/schedule_hint.go (+3 -1)
📝 services/gateway/internal/schedule_hint_test.go (+22 -0)
📝 services/scheduler/cmd/main.go (+8 -1)
➕ services/scheduler/internal/grouped_round_robin_strategy_test.go (+299 -0)
📝 services/scheduler/internal/node_registry.go (+29 -22)
📝 services/scheduler/internal/node_registry_test.go (+109 -0)
📝 services/scheduler/internal/service.go (+2 -2)
📝 services/scheduler/internal/service_test.go (+46 -0)
📝 services/scheduler/internal/strategy.go (+245 -2)
📝 services/shared/config/config.go (+41 -0)
📝 services/shared/config/config_test.go (+82 -0)

📄 Description

What

This PR adds an opt-in grouped_round_robin scheduling strategy as a bounded workload-affinity building block for #15. Requests with the same rootfs image reference or exact template reference stay on one eligible READY node until the group's sandbox, CPU, or memory budget is reached, after which a new group is assigned through global round-robin.

The Gateway now propagates templateID through the internal scheduling hint, scheduling snapshots derive current node status from discovery state and heartbeat TTL, and the new strategy is covered by configuration, documentation, and focused tests.

Why

Plain round-robin can scatter repeated workloads across nodes and repeatedly warm the same image or template artifacts. Grouped placement improves the opportunity for local cache reuse while bounding concentration and preserving the existing node resource filter.

Refs #15

This PR provides request-identity affinity but does not close the broader cache-aware scheduling issue.

Scope and non-goals

In scope: grouped placement by the first rootfs image or exact template reference, configurable group budgets, fresh READY-node enforcement, bounded in-memory group state, gateway hint propagation, and operator documentation.

Non-goals: inspecting or scoring actual node cache or P2P inventory, resolving mutable image tags or template aliases, considering attached-drive identities, adding projected hard-capacity admission, or persisting and reconciling group state across scheduler restarts.

Design and behavior changes

discovered nodes -> current status derivation -> existing node resource filter -> READY-only grouped round-robin -> selected node

Each workload identity has at most one open group. Requests reuse its node while the request fits the configured group budget; otherwise the group closes and the request starts a new group on the next eligible node. New groups for all identities share one stable global round-robin cursor, while missing or oversized identities use a separate round-robin fallback.

Placement accounting is serialized and updated during scheduling so concurrent bursts cannot overfill an open group while heartbeat metrics lag. Open state is capped at 10,000 keys with LRU eviction. Template requests do not carry CPU or memory values, so their groups use the sandbox-count limit only.

Compatibility and operations

Public API or generated protocol: The public HTTP API is unchanged. The internal scheduler protobuf adds optional NewSandboxHint.template_id field 2, which is wire-compatible with mixed Gateway and Scheduler versions.
Configuration or defaults: The default remains round_robin. Selecting grouped_round_robin requires scheduler.grouped_round_robin.max_sandbox_count > 0; CPU and memory group limits are optional. Equivalent environment overrides are documented.
Snapshot manifest, artifact layout, or storage format: N/A; no snapshot or artifact format changes.
Upgrade and rollback: No migration is required. Group state is in-memory and resets on restart; older components or missing hints degrade to ordinary round-robin placement.
Host requirements, permissions, ports, or dependencies: No new host requirements, ports, services, or dependencies.

Validation

make fmt
make clippy
make test-unit
Relevant Rust integration tests
make -C services test (required when services/ changes)
Generated clients/server regenerated with the documented make target
Documentation updated
Benchmarks or performance comparison completed

Commands and results:

make -C services build
make -C services test
make -C services vet
(cd services && go test ./...)
(cd services && go test -race ./...)
git diff --check
# PASS locally

make -C services proto PROTOC=<protoc 3.21.12>
gofmt -w services/api/proto/scheduler.pb.go services/api/proto/scheduler_grpc.pb.go
# PASS in a temporary LF-normalized export; generated files were unchanged

# PASS: Services CI
https://github.com/shudorcl/AgentENV/actions/runs/30477945881

# PASS: AgentENV integration, single-node E2E, and docker-compose E2E
https://github.com/shudorcl/AgentENV/actions/runs/30477945814

Local make -C services fmt-check reports unrelated CRLF files because this WSL checkout uses core.autocrlf=true; the same commit passes fmt-check in Services CI.

Skipped checks and reasons:

Root Rust formatting, Clippy, and unit-test targets were not run because this PR changes only the Go control-plane module and its generated Go protobuf.
No benchmark was run because this PR does not claim a measured performance improvement; group keys and retained state are explicitly bounded.

Risks and reviewer notes

Grouping uses exact, potentially mutable image or template references and is therefore an affinity heuristic rather than proof of cache availability.
Group limits control affinity, not request admission. An individually oversized request is still placed once and immediately closes its group; hard node eligibility remains governed by the existing resource filter.
Accounting happens before sandbox creation completes. A downstream creation failure can consume group budget and cause an earlier rotation, but cannot cause an open group to exceed its configured budget.
Group state is process-local and protected by one mutex. The 10,000-key LRU bound limits memory growth, but scheduling through this strategy is intentionally serialized.
Reviewers should start with services/scheduler/internal/strategy.go, services/gateway/internal/schedule_hint.go, services/scheduler/internal/node_registry.go, and services/shared/config/config.go.

Checklist

The PR contains one coherent change and no unrelated formatting or refactoring.
New behavior is covered by tests, or I explained why testing is impractical.
Logs and examples contain no credentials, tokens, or private registry information.
I did not manually edit generated code without updating its source and regenerating it.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/AgentENV/pull/68 **Author:** [@shudorcl](https://github.com/shudorcl) **Created:** 7/29/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `feat/grouped-locality-scheduling` --- ### 📝 Commits (8) - [`c46bf05`](https://github.com/kvcache-ai/AgentENV/commit/c46bf05f9875b99ad0e9ac69845dd43b67aabd2c) feat(gateway): propagate template scheduling identity - [`ecce76b`](https://github.com/kvcache-ai/AgentENV/commit/ecce76bccadc4c8c9678d6dffb51124437ee27a7) refactor(scheduler): derive current scheduling status - [`41c67ec`](https://github.com/kvcache-ai/AgentENV/commit/41c67ecffd9ee74a276704d100ff60a083a38c57) feat(scheduler): add resource-bounded locality strategy - [`742aa2d`](https://github.com/kvcache-ai/AgentENV/commit/742aa2df888ccb99cda4b437408a9c99ae31b8d0) feat(scheduler): configure locality group limits - [`59e8787`](https://github.com/kvcache-ai/AgentENV/commit/59e8787877c52bc7eedc63bae7baf18d838379fe) refactor(scheduler): rename locality strategy - [`f2b8e65`](https://github.com/kvcache-ai/AgentENV/commit/f2b8e654319e6a11dbc175d1e1c45d91fc6fb424) refactor(scheduler): inline group fullness check - [`bcce052`](https://github.com/kvcache-ai/AgentENV/commit/bcce052159db5fce13c01f4e0ad86d08b9e0e7b0) test(scheduler): cover scheduling snapshot derivation - [`33ea518`](https://github.com/kvcache-ai/AgentENV/commit/33ea51840bf989981fb391f1548e1691fbda0ee1) fix(scheduler): bound omitted cold-start resources ### 📊 Changes **14 files changed** (+929 additions, -33 deletions) <details> <summary>View changed files</summary> 📝 `services/README.md` (+25 -2) 📝 `services/api/proto/scheduler.pb.go` (+15 -3) 📝 `services/api/proto/scheduler.proto` (+3 -0) 📝 `services/gateway/internal/schedule_hint.go` (+3 -1) 📝 `services/gateway/internal/schedule_hint_test.go` (+22 -0) 📝 `services/scheduler/cmd/main.go` (+8 -1) ➕ `services/scheduler/internal/grouped_round_robin_strategy_test.go` (+299 -0) 📝 `services/scheduler/internal/node_registry.go` (+29 -22) 📝 `services/scheduler/internal/node_registry_test.go` (+109 -0) 📝 `services/scheduler/internal/service.go` (+2 -2) 📝 `services/scheduler/internal/service_test.go` (+46 -0) 📝 `services/scheduler/internal/strategy.go` (+245 -2) 📝 `services/shared/config/config.go` (+41 -0) 📝 `services/shared/config/config_test.go` (+82 -0) </details> ### 📄 Description ## What This PR adds an opt-in `grouped_round_robin` scheduling strategy as a bounded workload-affinity building block for #15. Requests with the same rootfs image reference or exact template reference stay on one eligible READY node until the group's sandbox, CPU, or memory budget is reached, after which a new group is assigned through global round-robin. The Gateway now propagates `templateID` through the internal scheduling hint, scheduling snapshots derive current node status from discovery state and heartbeat TTL, and the new strategy is covered by configuration, documentation, and focused tests. ## Why Plain round-robin can scatter repeated workloads across nodes and repeatedly warm the same image or template artifacts. Grouped placement improves the opportunity for local cache reuse while bounding concentration and preserving the existing node resource filter. ## Related issue Refs #15 This PR provides request-identity affinity but does not close the broader cache-aware scheduling issue. ## Scope and non-goals In scope: grouped placement by the first rootfs image or exact template reference, configurable group budgets, fresh READY-node enforcement, bounded in-memory group state, gateway hint propagation, and operator documentation. Non-goals: inspecting or scoring actual node cache or P2P inventory, resolving mutable image tags or template aliases, considering attached-drive identities, adding projected hard-capacity admission, or persisting and reconciling group state across scheduler restarts. ## Design and behavior changes ```text discovered nodes -> current status derivation -> existing node resource filter -> READY-only grouped round-robin -> selected node ``` Each workload identity has at most one open group. Requests reuse its node while the request fits the configured group budget; otherwise the group closes and the request starts a new group on the next eligible node. New groups for all identities share one stable global round-robin cursor, while missing or oversized identities use a separate round-robin fallback. Placement accounting is serialized and updated during scheduling so concurrent bursts cannot overfill an open group while heartbeat metrics lag. Open state is capped at 10,000 keys with LRU eviction. Template requests do not carry CPU or memory values, so their groups use the sandbox-count limit only. ## Compatibility and operations - Public API or generated protocol: The public HTTP API is unchanged. The internal scheduler protobuf adds optional `NewSandboxHint.template_id` field 2, which is wire-compatible with mixed Gateway and Scheduler versions. - Configuration or defaults: The default remains `round_robin`. Selecting `grouped_round_robin` requires `scheduler.grouped_round_robin.max_sandbox_count > 0`; CPU and memory group limits are optional. Equivalent environment overrides are documented. - Snapshot manifest, artifact layout, or storage format: N/A; no snapshot or artifact format changes. - Upgrade and rollback: No migration is required. Group state is in-memory and resets on restart; older components or missing hints degrade to ordinary round-robin placement. - Host requirements, permissions, ports, or dependencies: No new host requirements, ports, services, or dependencies. ## Validation - [ ] `make fmt` - [ ] `make clippy` - [ ] `make test-unit` - [x] Relevant Rust integration tests - [x] `make -C services test` (required when `services/` changes) - [x] Generated clients/server regenerated with the documented `make` target - [x] Documentation updated - [ ] Benchmarks or performance comparison completed Commands and results: ```text make -C services build make -C services test make -C services vet (cd services && go test ./...) (cd services && go test -race ./...) git diff --check # PASS locally make -C services proto PROTOC=<protoc 3.21.12> gofmt -w services/api/proto/scheduler.pb.go services/api/proto/scheduler_grpc.pb.go # PASS in a temporary LF-normalized export; generated files were unchanged # PASS: Services CI https://github.com/shudorcl/AgentENV/actions/runs/30477945881 # PASS: AgentENV integration, single-node E2E, and docker-compose E2E https://github.com/shudorcl/AgentENV/actions/runs/30477945814 ``` Local `make -C services fmt-check` reports unrelated CRLF files because this WSL checkout uses `core.autocrlf=true`; the same commit passes `fmt-check` in Services CI. Skipped checks and reasons: - Root Rust formatting, Clippy, and unit-test targets were not run because this PR changes only the Go control-plane module and its generated Go protobuf. - No benchmark was run because this PR does not claim a measured performance improvement; group keys and retained state are explicitly bounded. ## Risks and reviewer notes - Grouping uses exact, potentially mutable image or template references and is therefore an affinity heuristic rather than proof of cache availability. - Group limits control affinity, not request admission. An individually oversized request is still placed once and immediately closes its group; hard node eligibility remains governed by the existing resource filter. - Accounting happens before sandbox creation completes. A downstream creation failure can consume group budget and cause an earlier rotation, but cannot cause an open group to exceed its configured budget. - Group state is process-local and protected by one mutex. The 10,000-key LRU bound limits memory growth, but scheduling through this strategy is intentionally serialized. - Reviewers should start with `services/scheduler/internal/strategy.go`, `services/gateway/internal/schedule_hint.go`, `services/scheduler/internal/node_registry.go`, and `services/shared/config/config.go`. ## Checklist - [x] The PR contains one coherent change and no unrelated formatting or refactoring. - [x] New behavior is covered by tests, or I explained why testing is impractical. - [x] Logs and examples contain no credentials, tokens, or private registry information. - [x] I did not manually edit generated code without updating its source and regenerating it. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi added the

pull-request

label

2026-10-29 14:20:21 +00:00

bsakaguchi referenced this issue

2026-10-29 14:21:06 +00:00

[PR #154] [MERGED] fix(snapshot): preserve exported image config #215