kvcache-ai/Mooncake

Fork 0

mirror of https://github.com/kvcache-ai/Mooncake.git synced 2026-11-05 01:17:55 +00:00

[GH-ISSUE #2430] [Performance]: segment utilization imbalance with heterogeneous segment sizes #644

New issue

Closed

opened 2026-10-29 21:17:33 +00:00 by bsakaguchi · 4 comments

bsakaguchi commented

2026-10-29 21:17:33 +00:00

Owner

Originally created by @00fish0 on GitHub (Jun 11, 2026).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/2430

Originally assigned to: @00fish0 on GitHub.

Describe your performance question

Summary

When mooncake-store is deployed with segments of different sizes, the larger segments consistently show lower utilization than the smaller ones. After digging into the allocation and eviction code, we believe this is structural: allocation is capacity-blind (uniform per segment), and eviction is segment-blind (global watermark + global LRU-by-lease). Neither side has a feedback mechanism to balance utilization across heterogeneous segments.

We don't have a solution we're fully happy with (see "What we considered" below — including free_ratio_first, which helps but has its own issues), so we'd like to open this up for discussion.

Our deployment & observation

Mooncake Version: v0.3.9

10 memory segments across 5 nodes, two different sizes (node IPs anonymized):

Segment	Allocated	Capacity	Used
node-A:1	366.52 GiB	400 GiB	91.6%
node-A:2	366.33 GiB	400 GiB	91.6%
node-B:1	366.41 GiB	400 GiB	91.6%
node-B:2	366.31 GiB	400 GiB	91.6%
node-C:1	366.38 GiB	400 GiB	91.6%
node-C:2	366.46 GiB	400 GiB	91.6%
node-D:1	366.41 GiB	400 GiB	91.6%
node-D:2	366.30 GiB	400 GiB	91.6%
node-E:1	867.53 GiB	1000 GiB	86.8%
node-E:2	867.54 GiB	1000 GiB	86.8%

Global: 4.56 TiB / 5.08 TiB (90%). Workload is an LLM prefix-KV-cache pool (SGLang HiCache L3 backend), default random allocation strategy, eviction parameters (eviction_high_watermark_ratio=0.9, eviction_ratio=0.05).

The 1000 GiB segments are always ~5 percentage points below the 400 GiB ones, not just transiently. Note also that the eight 400 GiB segments sit at a suspiciously tight 91.6% — this appears to be their practical fill ceiling (allocation failures from fragmentation), not a coincidence of timing.

Why this happens (code analysis)

1. Allocation is capacity-blind

RandomAllocationStrategy::Allocate (mooncake-store/include/allocation_strategy.h) picks a segment uniformly at random over segment names, regardless of capacity or free space. A 1000 GiB segment gets exactly the same write inflow as a 400 GiB one. The only way a large segment absorbs more data is spillover: a random pick lands on a full segment, fails, and the retry loop walks to the next segment that still has space.

The steady-state dynamics that follow: small segments fill up and pin at their ceiling; large segments spend most of the time "catching up" via spillover, so any snapshot shows them at lower utilization.

2. Eviction is segment-blind

MasterService::BatchEvict (mooncake-store/src/master_service.cpp) evicts by lease age across all metadata shards, with no awareness of which segment a replica lives on. There is no per-segment watermark and no way to relieve pressure on a specific segment. The trigger is also global-only:

the global watermark path fires when global_used_ratio > eviction_high_watermark_ratio;
otherwise eviction waits for need_eviction_, which is only set after a PutStart fails to allocate on every segment.

3. The two interact badly

With our topology the small segments cap out at ~91.6%, so the global used ratio mathematically cannot reach the default 0.95 watermark ((8×366 GiB + 2×1000 GiB) / 5200 GiB ≈ 94.8%). The watermark path therefore never fires, and eviction is driven entirely by the allocation-failure fallback. Each cycle ends with a window of client-visible NO_AVAILABLE_HANDLE write failures, followed by a bulk eviction of ~5% of all objects (~250 GiB of cache dropped at once in our case), then refill — a sawtooth in both write success rate and hit rate.

Why this matters

RDMA traffic imbalance / network pressure. Since the small segments are pinned at their ceiling most of the time, nearly all spillover writes concentrate on whichever node hosts the remaining free space. In our deployment both 1000 GiB segments are on the same node, so that single node's NIC absorbs the bulk of cluster write bandwidth for most of each cycle — and since it holds the freshest data, read traffic skews there too.
Lower achievable utilization. The large segments structurally idle below their ceiling, and the unreachable global watermark means the cluster oscillates around 90% instead of its actual allocatable maximum.
Periodic write-failure windows and bulk cache eviction, as described above — particularly painful for a KV-cache workload where hit rate is the whole point.

What we considered

Operational workaround: make all segments the same size (e.g. mount the 1000 GiB nodes as multiple smaller segments). This fixes the inflow share by construction, but it's a per-deployment workaround, not a fix — heterogeneous fleets are common (mixed RAM configs, partial reservations on some nodes).
FreeRatioFirstAllocationStrategy (already in the codebase): sorts sampled candidates by free ratio and greedily picks the emptiest. This does equalize utilization percentages in steady state, but the greedy choice introduces a new problem: the emptiest segment wins almost every allocation until it catches up. A freshly (re)mounted segment — e.g. after a node restart — has free ratio 1.0 and absorbs nearly the entire cluster write bandwidth until it reaches parity, which is a single-node write storm on exactly the node that just came back. The strategy comment frames "new empty segments naturally win" as a feature, but for large segments this is a long, severe transient. It also does nothing for the eviction side (the unreachable watermark and segment-blind eviction remain).
Lowering eviction_high_watermark_ratio / eviction_ratio so the watermark path fires before allocation failures and evicts in smaller steps. This mitigates the sawtooth but requires knowing each deployment's fragmentation ceiling, and still does nothing for the utilization/traffic imbalance.

Possible directions (for discussion)

None of these is obviously right; we'd love input from the community:

Weighted-random allocation — pick segments with probability proportional to free bytes (or power-of-two-choices: sample two, take the one with more free space). Smooths the hotspot problem that greedy free-ratio-first has, while still converging to balanced utilization. Could be a new strategy or a fix to free_ratio_first.
Inflow cap / ramp-up for (re)mounted segments, so an empty segment doesn't absorb the whole cluster's write bandwidth regardless of strategy.
Per-segment eviction watermark — let the eviction thread also react to individual segments above a threshold, and/or make BatchEvict prefer evicting replicas on pressured segments (the candidate selection currently ignores placement entirely).
Make the global watermark meaningful under heterogeneity — e.g. compute used ratio against practically allocatable capacity, or trigger on max(per-segment ratio) instead of the global average, so eviction doesn't degenerate to the allocation-failure path.

Before submitting a new issue...

Make sure you already searched for relevant issues and read the documentation

Originally created by @00fish0 on GitHub (Jun 11, 2026). Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/2430 Originally assigned to: @00fish0 on GitHub. ### Describe your performance question ## Summary When mooncake-store is deployed with segments of different sizes, the larger segments consistently show lower utilization than the smaller ones. After digging into the allocation and eviction code, we believe this is structural: **allocation is capacity-blind (uniform per segment), and eviction is segment-blind (global watermark + global LRU-by-lease)**. Neither side has a feedback mechanism to balance utilization across heterogeneous segments. We don't have a solution we're fully happy with (see "What we considered" below — including `free_ratio_first`, which helps but has its own issues), so we'd like to open this up for discussion. ## Our deployment & observation Mooncake Version: v0.3.9 10 memory segments across 5 nodes, two different sizes (node IPs anonymized): | Segment | Allocated | Capacity | Used | |---|---|---|---| | node-A:1 | 366.52 GiB | 400 GiB | 91.6% | | node-A:2 | 366.33 GiB | 400 GiB | 91.6% | | node-B:1 | 366.41 GiB | 400 GiB | 91.6% | | node-B:2 | 366.31 GiB | 400 GiB | 91.6% | | node-C:1 | 366.38 GiB | 400 GiB | 91.6% | | node-C:2 | 366.46 GiB | 400 GiB | 91.6% | | node-D:1 | 366.41 GiB | 400 GiB | 91.6% | | node-D:2 | 366.30 GiB | 400 GiB | 91.6% | | node-E:1 | 867.53 GiB | 1000 GiB | 86.8% | | node-E:2 | 867.54 GiB | 1000 GiB | 86.8% | Global: 4.56 TiB / 5.08 TiB (90%). Workload is an LLM prefix-KV-cache pool (SGLang HiCache L3 backend), default `random` allocation strategy, eviction parameters (`eviction_high_watermark_ratio=0.9`, `eviction_ratio=0.05`). The 1000 GiB segments are **always** ~5 percentage points below the 400 GiB ones, not just transiently. Note also that the eight 400 GiB segments sit at a suspiciously tight 91.6% — this appears to be their practical fill ceiling (allocation failures from fragmentation), not a coincidence of timing. ## Why this happens (code analysis) ### 1. Allocation is capacity-blind `RandomAllocationStrategy::Allocate` (mooncake-store/include/allocation_strategy.h) picks a segment **uniformly at random over segment names**, regardless of capacity or free space. A 1000 GiB segment gets exactly the same write inflow as a 400 GiB one. The only way a large segment absorbs more data is *spillover*: a random pick lands on a full segment, fails, and the retry loop walks to the next segment that still has space. The steady-state dynamics that follow: small segments fill up and pin at their ceiling; large segments spend most of the time "catching up" via spillover, so any snapshot shows them at lower utilization. ### 2. Eviction is segment-blind `MasterService::BatchEvict` (mooncake-store/src/master_service.cpp) evicts by lease age across all metadata shards, with no awareness of which segment a replica lives on. There is no per-segment watermark and no way to relieve pressure on a specific segment. The trigger is also global-only: - the global watermark path fires when `global_used_ratio > eviction_high_watermark_ratio`; - otherwise eviction waits for `need_eviction_`, which is only set after a `PutStart` fails to allocate on **every** segment. ### 3. The two interact badly With our topology the small segments cap out at ~91.6%, so the global used ratio mathematically cannot reach the default 0.95 watermark (`(8×366 GiB + 2×1000 GiB) / 5200 GiB ≈ 94.8%`). The watermark path therefore **never fires**, and eviction is driven entirely by the allocation-failure fallback. Each cycle ends with a window of client-visible `NO_AVAILABLE_HANDLE` write failures, followed by a bulk eviction of ~5% of all objects (~250 GiB of cache dropped at once in our case), then refill — a sawtooth in both write success rate and hit rate. ## Why this matters 1. **RDMA traffic imbalance / network pressure.** Since the small segments are pinned at their ceiling most of the time, nearly all spillover writes concentrate on whichever node hosts the remaining free space. In our deployment both 1000 GiB segments are on the same node, so that single node's NIC absorbs the bulk of cluster write bandwidth for most of each cycle — and since it holds the freshest data, read traffic skews there too. 2. **Lower achievable utilization.** The large segments structurally idle below their ceiling, and the unreachable global watermark means the cluster oscillates around 90% instead of its actual allocatable maximum. 3. **Periodic write-failure windows and bulk cache eviction**, as described above — particularly painful for a KV-cache workload where hit rate is the whole point. ## What we considered - **Operational workaround: make all segments the same size** (e.g. mount the 1000 GiB nodes as multiple smaller segments). This fixes the inflow share by construction, but it's a per-deployment workaround, not a fix — heterogeneous fleets are common (mixed RAM configs, partial reservations on some nodes). - **`FreeRatioFirstAllocationStrategy`** (already in the codebase): sorts sampled candidates by free ratio and greedily picks the emptiest. This does equalize utilization percentages in steady state, but the greedy choice introduces a new problem: **the emptiest segment wins almost every allocation until it catches up**. A freshly (re)mounted segment — e.g. after a node restart — has free ratio 1.0 and absorbs nearly the entire cluster write bandwidth until it reaches parity, which is a single-node write storm on exactly the node that just came back. The strategy comment frames "new empty segments naturally win" as a feature, but for large segments this is a long, severe transient. It also does nothing for the eviction side (the unreachable watermark and segment-blind eviction remain). - **Lowering `eviction_high_watermark_ratio` / `eviction_ratio`** so the watermark path fires before allocation failures and evicts in smaller steps. This mitigates the sawtooth but requires knowing each deployment's fragmentation ceiling, and still does nothing for the utilization/traffic imbalance. ## Possible directions (for discussion) None of these is obviously right; we'd love input from the community: 1. **Weighted-random allocation** — pick segments with probability proportional to free bytes (or power-of-two-choices: sample two, take the one with more free space). Smooths the hotspot problem that greedy free-ratio-first has, while still converging to balanced utilization. Could be a new strategy or a fix to `free_ratio_first`. 2. **Inflow cap / ramp-up for (re)mounted segments**, so an empty segment doesn't absorb the whole cluster's write bandwidth regardless of strategy. 3. **Per-segment eviction watermark** — let the eviction thread also react to individual segments above a threshold, and/or make `BatchEvict` prefer evicting replicas on pressured segments (the candidate selection currently ignores placement entirely). 4. **Make the global watermark meaningful under heterogeneity** — e.g. compute used ratio against *practically allocatable* capacity, or trigger on `max(per-segment ratio)` instead of the global average, so eviction doesn't degenerate to the allocation-failure path. ### Before submitting a new issue... - [ ] Make sure you already searched for relevant issues and read the [documentation](https://kvcache-ai.github.io/Mooncake/)

bsakaguchi

2026-10-29 21:17:33 +00:00

closed this issue
added the
stale

auto-closed
labels

bsakaguchi commented

2026-10-29 21:17:34 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Jun 11, 2026):

Thanks for opening this issue, @00fish0!

Field	Value
Issue	#2430
GitHub user ID	`50252876`
Reporter	@00fish0

A maintainer will triage this when possible. To help us respond faster, please include:

Mooncake version or commit SHA
Environment (OS, CUDA/driver, RDMA stack if relevant)
Steps to reproduce and expected vs. actual behavior

Useful links: Documentation · Contributing guide

This message was posted automatically by the issue bot.

@github-actions[bot] commented on GitHub (Jun 11, 2026): Thanks for opening this issue, @00fish0! | Field | Value | |-------|-------| | **Issue** | #2430 | | **GitHub user ID** | `50252876` | | **Reporter** | @00fish0 | A maintainer will triage this when possible. To help us respond faster, please include: - Mooncake version or commit SHA - Environment (OS, CUDA/driver, RDMA stack if relevant) - Steps to reproduce and expected vs. actual behavior Useful links: [Documentation](https://kvcache-ai.github.io/Mooncake/) · [Contributing guide](https://github.com/kvcache-ai/Mooncake/blob/main/CONTRIBUTING.md) > This message was posted automatically by the issue bot.

bsakaguchi commented

2026-10-29 21:17:34 +00:00

Author

Owner

@wdlin233 commented on GitHub (Jun 30, 2026):

Hi @00fish0, thanks for the thorough analysis in this issue — it was really helpful for understanding the problem space.

I've been investigating possible approaches and ended up implementing a capacity-aware power-of-two-choices (P2C) allocation strategy as a first step. The code is on the branch issue-2430 on my fork if you want to take a look.

What it does

For each allocation, CapacityAwareP2CAllocationStrategy:

Uniform-randomly samples 2 distinct candidate segments from the eligible pool (not weighted by free space, see "Why this approach").
Compares them: free_ratio descending as the primary key, with free_bytes descending as a tiebreak (within a 1% epsilon). The comparator is the "capacity-aware" part — not the sampler.
Picks the winner. On failure, tries the runner-up before resampling; removes exhausted candidates from the pool. Falls back to the base random walk for any remaining replicas (preserving best-effort semantics).

Preferred-segment and excluded-segment behavior is identical to the existing strategies; replicas of the same slice still land on distinct segments.

Why this approach

We need three things at once:
(a) balanced utilization across heterogeneous segments,
(b) bounded write storm on fresh-segment mount,
(c) capacity-proportional inflow in steady state.

Random ignores capacity entirely. Fails (a) and (c).
FreeRatioFirst nails (a) and (c) but an empty mounted segment wins every selection. Fails (b).
Weighted free-bytes sampling a large empty segment holds a large fraction of total free bytes (if we choose free_bytes or capacity * free_ratio as weights) and would be sampled with that probability. Same storm problem, different mechanism.

P2C samples uniformly (each segment gets at most 2/N selection probability) but makes the choice capacity-aware in the comparator. The 2/N bound is what limits the late-mount storm; the free_ratio → free_bytes (primary key → tiebreaker) comparator is what routes capacity-proportional inflow.

Quantitative results

I ran two scenarios against Random and FreeRatioFirst. Numbers are from allocation_strategy_bench (and a standalone test on the issue topology, but running single-node with in-process allocators. This validates the algorithm-level behavior but does not yet exercise multi-node RDMA paths or real KV-cache workloads).

Scenario A — issue #2430 topology, steady state

8 segments × 8 MiB + 2 segments × 20 MiB (2.5x ratio, matching the issue's 400 GiB / 1000 GiB ratio), driven to 70% cluster utilization.

Strategy	large_used_share	cap_share	util_spread	min util	max util
Random	0.205	0.385	0.571	0.372	0.944
FreeRatioFirst	0.385	0.385	0.000	0.700	0.700
CapacityAwareP2C	0.378	0.385	0.021	0.688	0.708

Random: large segments received only 20.5% of writes despite owning 38.5% of capacity, and per-segment utilization spans 57 pp — exactly the pathology you reported.

CapacityAwareP2C achieves 37.8% large-share (0.7 pp below capacity share, vs. Random's 18 pp gap) and a 2.1 pp utilization spread (vs. Random's 57.1 pp). It was nearly indistinguishable from FreeRatioFirst in steady state.

Scenario B — late-mount write storm

8 segments pre-heated to ~70%, then a 9th fresh segment is mounted (free_ratio = 1.0). Of the next 500 allocations, what fraction lands on the fresh segment? Uniform-random baseline = 1/9 ≈ 11%.

Strategy	post_mount_fresh_share
FreeRatioFirst	0.668
CapacityAwareP2C	0.202

FreeRatioFirst sends 67% of writes to the freshly-mounted node, but CapacityAwareP2C sends 20% which closes to the theoretical 2/N ≈ 22% bound, with a small uplift from the bytes-tiebreak. ~3.3× storm reduction.

Scenario C — in-tree scaleout benchmark (100 segments, 512 KiB)

The existing --workload=scaleout matrix at 100 segments, 2000 allocations, 20% post-scaleout allocation:

Strategy	UtilStdDev	FinalUtil	Succ/Total	Throughput (alloc/s)
Random	0.31	85.0%	441 / 480	245,678
FreeRatioFirst	0.27	86.1%	466 / 480	359,892
CapacityAwareP2C	0.21	86.7%	480 / 480	493,858

P2C has the lowest utilization spread, highest final utilization, and 100% allocation success (vs. 92% and 97% for the other two). Throughput is also highest — fewer fallback retries and fewer failures more than compensate for the small per-allocation overhead.

Summary

Property	Random	FreeRatioFirst	CapacityAwareP2C
Capacity-proportional inflow	❌	✅	✅
Steady-state util spread (heterog.)	57 pp	0 pp	2.1 pp
Bounded late-mount storm	✅	❌ (67%)	✅ (20%)

CapacityAwareP2C is the only strategy currently in the tree that does well on all three.

What this does NOT address

This PR only tackles the allocation side. With P2C, multiple segments may reach their fragmentation ceiling around the same time, which could cause a coordinated eviction spike (sawtooth). If that shows up in practice, a per-segment eviction watermark — triggering BatchEvict when any single segment crosses the threshold rather than waiting for the global average — would be a natural follow-up.

Here's the approach I've implemented so far, along with my reasoning. Would love to hear your thoughts on it.

@wdlin233 commented on GitHub (Jun 30, 2026): Hi @00fish0, thanks for the thorough analysis in this issue — it was really helpful for understanding the problem space. I've been investigating possible approaches and ended up implementing a **capacity-aware power-of-two-choices (P2C) allocation strategy** as a first step. The code is on the [branch issue-2430](https://github.com/wdlin233/Mooncake/tree/issue-2430) on my fork if you want to take a look. ## What it does For each allocation, `CapacityAwareP2CAllocationStrategy`: 1. Uniform-randomly samples **2 distinct candidate segments** from the eligible pool (not weighted by free space, see "Why this approach"). 2. Compares them: **free_ratio descending** as the primary key, with **free_bytes descending** as a tiebreak (within a 1% epsilon). The comparator is the "capacity-aware" part — not the sampler. 3. Picks the winner. On failure, tries the runner-up before resampling; removes exhausted candidates from the pool. Falls back to the base random walk for any remaining replicas (preserving best-effort semantics). Preferred-segment and excluded-segment behavior is identical to the existing strategies; replicas of the same slice still land on distinct segments. ## Why this approach We need three things at once: (a) balanced utilization across heterogeneous segments, (b) bounded write storm on fresh-segment mount, (c) capacity-proportional inflow in steady state. - **`Random`** ignores capacity entirely. Fails (a) and (c). - **`FreeRatioFirst`** nails (a) and (c) but an empty mounted segment wins every selection. Fails (b). - **Weighted free-bytes sampling** a large empty segment holds a large fraction of total free bytes (if we choose `free_bytes` or `capacity * free_ratio` as weights) and would be sampled with that probability. Same storm problem, different mechanism. **P2C samples uniformly (each segment gets at most 2/N selection probability)** but makes the *choice* capacity-aware in the comparator. The `2/N` bound is what limits the late-mount storm; the free_ratio → free_bytes (primary key → tiebreaker) comparator is what routes capacity-proportional inflow. ## Quantitative results I ran two scenarios against `Random` and `FreeRatioFirst`. Numbers are from `allocation_strategy_bench` (and a standalone test on the issue topology, but running single-node with in-process allocators. This validates the algorithm-level behavior but does not yet exercise multi-node RDMA paths or real KV-cache workloads). ### Scenario A — issue #2430 topology, steady state 8 segments × 8 MiB + 2 segments × 20 MiB (2.5x ratio, matching the issue's 400 GiB / 1000 GiB ratio), driven to 70% cluster utilization. | Strategy | large_used_share | cap_share | util_spread | min util | max util | |----------------------|------------------|-----------|-------------|----------|----------| | Random | **0.205** | 0.385 | **0.571** | 0.372 | 0.944 | | FreeRatioFirst | 0.385 | 0.385 | 0.000 | 0.700 | 0.700 | | **CapacityAwareP2C** | **0.378** | 0.385 | **0.021** | 0.688 | 0.708 | `Random`: large segments received only **20.5%** of writes despite owning **38.5%** of capacity, and per-segment utilization spans **57 pp** — exactly the pathology you reported. `CapacityAwareP2C` achieves **37.8%** large-share (0.7 pp below capacity share, vs. Random's 18 pp gap) and a **2.1 pp utilization spread** (vs. Random's 57.1 pp). It was nearly indistinguishable from `FreeRatioFirst` in steady state. ### Scenario B — late-mount write storm 8 segments pre-heated to ~70%, then a 9th fresh segment is mounted (`free_ratio = 1.0`). Of the next 500 allocations, what fraction lands on the fresh segment? Uniform-random baseline = 1/9 ≈ 11%. | Strategy | post_mount_fresh_share | |----------------------|------------------------| | FreeRatioFirst | **0.668** | | **CapacityAwareP2C** | **0.202** | `FreeRatioFirst` sends **67%** of writes to the freshly-mounted node, but `CapacityAwareP2C` sends **20%** which closes to the theoretical `2/N ≈ 22%` bound, with a small uplift from the bytes-tiebreak. **~3.3× storm reduction.** ### Scenario C — in-tree scaleout benchmark (100 segments, 512 KiB) The existing `--workload=scaleout` matrix at 100 segments, 2000 allocations, 20% post-scaleout allocation: | Strategy | UtilStdDev | FinalUtil | Succ/Total | Throughput (alloc/s) | |----------------------|-----------:|----------:|-----------:|---------------------:| | Random | 0.31 | 85.0% | 441 / 480 | 245,678 | | FreeRatioFirst | 0.27 | 86.1% | 466 / 480 | 359,892 | | **CapacityAwareP2C** | **0.21** | **86.7%** | **480 / 480** | **493,858** | P2C has the lowest utilization spread, highest final utilization, and **100% allocation success** (vs. 92% and 97% for the other two). Throughput is also highest — fewer fallback retries and fewer failures more than compensate for the small per-allocation overhead. ### Summary | Property | Random | FreeRatioFirst | CapacityAwareP2C | |-----------------------------------------|--------|----------------|------------------| | Capacity-proportional inflow | ❌ | ✅ | ✅ | | Steady-state util spread (heterog.) | 57 pp | 0 pp | 2.1 pp | | Bounded late-mount storm | ✅ | ❌ (67%) | ✅ (20%) | `CapacityAwareP2C` is the only strategy currently in the tree that does well on all three. ## What this does NOT address This PR only tackles the allocation side. With P2C, multiple segments may reach their fragmentation ceiling around the same time, which could cause a coordinated eviction spike (sawtooth). If that shows up in practice, a per-segment eviction watermark — triggering BatchEvict when any single segment crosses the threshold rather than waiting for the global average — would be a natural follow-up. *** Here's the approach I've implemented so far, along with my reasoning. Would love to hear your thoughts on it.

bsakaguchi commented

2026-10-29 21:17:34 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Sep 29, 2026):

This issue has had no activity for 90 days and will be closed in 7 days if there is no further activity. Please comment or react if it should stay open.

@github-actions[bot] commented on GitHub (Sep 29, 2026): This issue has had no activity for 90 days and will be closed in 7 days if there is no further activity. Please comment or react if it should stay open.

bsakaguchi commented

2026-10-29 21:17:34 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Oct 7, 2026):

Closing due to 3 months of inactivity. If this is still relevant, please comment and we can reopen.

@github-actions[bot] commented on GitHub (Oct 7, 2026): Closing due to 3 months of inactivity. If this is still relevant, please comment and we can reopen.