[PR #6] [MERGED] [Bugfix] Ignore late Mooncake ready events for cancelled transfers #7

New issue

Closed

opened 2026-10-29 21:06:04 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 21:06:04 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/vllm/pull/6
Author: @jiaran-king
Created: 8/27/2026
Status: ✅ Merged
Merged: 8/28/2026
Merged by: @gty111

Base: add-mooncake-ec-connector ← Head: fix/mooncake-ignore-cancelled-ready-events

📝 Commits (1)

d65a282 [Bugfix] Ignore late Mooncake ready events for cancelled transfers

📊 Changes

2 files changed (+38 additions, -0 deletions)

View changed files

📝 tests/v1/ec_connector/unit/test_ec_mooncake_connector.py (+35 -0)
📝 vllm/distributed/ec_transfer/ec_connector/mooncake_ec_connector.py (+3 -0)

📄 Description

Purpose

A late ready event can recreate a pending spec after its transfer_id has
already been cancelled and the Worker reservation removed.

sequenceDiagram
    participant S as Consumer Scheduler
    participant C as Cancel RPC
    participant W as Consumer Worker
    participant E as Ready event channel

    S->>S: remove pending spec and mark A cancelled
    S->>C: submit async cancel(A)
    W->>E: enqueue ready(A)
    C->>W: cancel(A)
    W->>W: remove reservation(A)
    E-->>S: late ready(A)
    S->>S: rebuild pending_spec(A)
    Note over S: Invalid: A is cancelled and pending
    S->>W: later load(A)
    W-->>S: reservation missing: load failed

This change drops ready events for cancelled transfer IDs before TP shard
aggregation. A new transfer with the same mm_hash remains valid.

This is independent of kvcache-ai/vllm#5: duplicate preparation can amplify
the race, but one ready event queued before the Worker processes the
asynchronous cancel is sufficient.

Evidence

The regression test deterministically reproduces the issue on the base: after
all TP4 ready shards are received, a cancelled transfer is recreated in
_pending_specs.

With the fix, the cancelled transfer remains absent from both _pending_specs
and _event_ready_shards.

Duplicate-work Check

I searched open PRs in vllm-project/vllm and kvcache-ai/vllm for this
cancelled-transfer ready-event race. No existing PR addresses it.

AI assistance was used in preparing this change.

Test Plan

.venv/bin/python -m pytest \
  tests/v1/ec_connector/unit/test_ec_mooncake_connector.py \
  -q

uvx pre-commit run --files \
  vllm/distributed/ec_transfer/ec_connector/mooncake_ec_connector.py \
  tests/v1/ec_connector/unit/test_ec_mooncake_connector.py

Test Result

Paired reproduction: base 1 failed; patch 1 passed.
With the fix: 43 passed, 1 warning.
All applicable pre-commit hooks passed.
Model evaluation is not applicable; model execution and numerical output are
unchanged.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/vllm/pull/6 **Author:** [@jiaran-king](https://github.com/jiaran-king) **Created:** 8/27/2026 **Status:** ✅ Merged **Merged:** 8/28/2026 **Merged by:** [@gty111](https://github.com/gty111) **Base:** `add-mooncake-ec-connector` ← **Head:** `fix/mooncake-ignore-cancelled-ready-events` --- ### 📝 Commits (1) - [`d65a282`](https://github.com/kvcache-ai/vllm/commit/d65a282c32364608c7f9641a6621c7b3df4e6cf1) [Bugfix] Ignore late Mooncake ready events for cancelled transfers ### 📊 Changes **2 files changed** (+38 additions, -0 deletions) <details> <summary>View changed files</summary> 📝 `tests/v1/ec_connector/unit/test_ec_mooncake_connector.py` (+35 -0) 📝 `vllm/distributed/ec_transfer/ec_connector/mooncake_ec_connector.py` (+3 -0) </details> ### 📄 Description ## Purpose A late `ready` event can recreate a pending spec after its `transfer_id` has already been cancelled and the Worker reservation removed. ```mermaid sequenceDiagram participant S as Consumer Scheduler participant C as Cancel RPC participant W as Consumer Worker participant E as Ready event channel S->>S: remove pending spec and mark A cancelled S->>C: submit async cancel(A) W->>E: enqueue ready(A) C->>W: cancel(A) W->>W: remove reservation(A) E-->>S: late ready(A) S->>S: rebuild pending_spec(A) Note over S: Invalid: A is cancelled and pending S->>W: later load(A) W-->>S: reservation missing: load failed ``` This change drops ready events for cancelled transfer IDs before TP shard aggregation. A new transfer with the same `mm_hash` remains valid. This is independent of kvcache-ai/vllm#5: duplicate preparation can amplify the race, but one ready event queued before the Worker processes the asynchronous cancel is sufficient. ## Evidence The regression test deterministically reproduces the issue on the base: after all TP4 ready shards are received, a cancelled transfer is recreated in `_pending_specs`. With the fix, the cancelled transfer remains absent from both `_pending_specs` and `_event_ready_shards`. ## Duplicate-work Check I searched open PRs in `vllm-project/vllm` and `kvcache-ai/vllm` for this cancelled-transfer ready-event race. No existing PR addresses it. AI assistance was used in preparing this change. ## Test Plan .venv/bin/python -m pytest \ tests/v1/ec_connector/unit/test_ec_mooncake_connector.py \ -q uvx pre-commit run --files \ vllm/distributed/ec_transfer/ec_connector/mooncake_ec_connector.py \ tests/v1/ec_connector/unit/test_ec_mooncake_connector.py ## Test Result - Paired reproduction: base `1 failed`; patch `1 passed`. - With the fix: `43 passed, 1 warning`. - All applicable pre-commit hooks passed. - Model evaluation is not applicable; model execution and numerical output are unchanged. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-29 21:06:04 +00:00