mirror of
https://github.com/kvcache-ai/Mooncake.git
synced 2026-11-05 01:17:55 +00:00
[GH-ISSUE #5036] [RFC]: Intel XPU CI Coverage and Resource Allocation Proposal #1274
Labels
No labels
ascend
auto-closed
bug
bug
CI/Build
Common
documentation
enhancement
feature request
good first issue
help wanted
high priority
MetaX
nightly-failure
pull-request
PyTorch Backend
question
RFC
Roadmap
stale
Store
Tests
Transfer Engine
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
kvcache-ai/Mooncake#1274
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @VincyZhang on GitHub (Oct 28, 2026).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/5036
Summary
Introduce XPU CI with broad C++ and Python unit/component coverage and two model-level E2E smoke tests. Keep model execution out of PR core testing; run both application scenarios nightly.
Motivation and Goals
Mooncake already has XPU platform, transport, Store, IPC, and native EP tests, but no dedicated XPU Actions workflow at the assessment baseline. Existing tests should become repeatable CI coverage rather than relying primarily on expensive application runs.
The goals are to:
Initial non-goals are RDMA, cross-node validation, GDS, EP with more than two ranks, broad model matrices, and model-quality or performance certification. There is no RDMA hardware available.
This RFC also presents two resource proposals:
kubectl.The resource choice does not change the initial test scope or trigger policy.
Test Coverage and Frequency
GPU counts below describe test requirements, not necessarily scheduler reservations. Existing C++ binaries mix single- and dual-GPU cases, so the core job requests two GPUs and executes its phases sequentially.
C++ and Python identify implementation languages, not test levels. Store roundtrips and multi-process EP checks are component integration tests. Only the two vLLM scenarios load an actual model; these are functional smoke tests, not model accuracy benchmarks.
Existing Entry Points
RealClientTest.XpuDeviceBufferPutGetRoundTripsin the Store client tests.test_xpu_local_copy_pathsin the Store tensor tests and the native EP correctness script.Resource Proposals
Proposal 1: Fixed Runners
Proposal 2: Kubernetes Allocation
kubectl; the Kubernetes scheduler and Intel GPU device plugin allocate devices.Triggers and Execution
Core hardware testing is maintainer-triggered rather than automatic on every PR push. Results apply only to the tested head SHA; a new head needs a fresh trigger and result. Both E2E scenarios are required in an enabled nightly hardware run and may also be rerun manually.
Builds do not reserve GPUs. Hardware jobs consume matching artifacts for the selected SHA, including native C++ test binaries that are not in the wheel. Pin the oneAPI/runtime, PyTorch, vLLM image, and model versions.
Correctness and Acceptance Criteria
Rollout
Comparison with Current ROCm CI
This compares the proposed XPU design with repository behavior at the assessment baseline, not two already implemented CI systems.
BUILD_UNIT_TESTS=OFFand does not execute themcore-4gpuprofile; scripts set GPU indices/TP and serially reuse allocationsrun-e2e-cilabel/manual entry; configurable nightly hardware executionrun-xpu-cilabel/manual core; nightly core and both E2E scenariosRegression Policy
When an application E2E test exposes a defect, add a focused C++ or Python regression test at the lowest practical layer rather than relying only on another expensive model scenario to prevent recurrence.
Before submitting a new issue...
@github-actions[bot] commented on GitHub (Oct 28, 2026):
Thanks for opening this issue, @VincyZhang!
92291705A maintainer will triage this when possible. To help us respond faster, please include:
Useful links: Documentation · Contributing guide
@staryxchen commented on GitHub (Oct 29, 2026):
Could we add a dedicated XPU wheel, similar to the existing EFA/AMD platform-specific wheels, to simplify installation and dependency management?
@VincyZhang commented on GitHub (Oct 29, 2026):
Exactly, that's what we expected too. And we also want to follow the same release workflow after we asure the quality of wheel like
mooncake-transfer-engine-xpu.