[PR #1370] [CLOSED] [TE] feat: zmq communicator based on coro_rpc #2204

New issue

Closed

opened 2026-10-29 21:33:38 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 21:33:38 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/Mooncake/pull/1370
Author: @JasonZhang517
Created: 1/13/2026
Status: ❌ Closed

Base: main ← Head: zmq_communicator

📝 Commits (10+)

4e8c82e [DEV] Add Pre-commit Check before Submit (#1124)
cf63fc8 [Store] Cancel all negative ret val (#1129)
6512006 [Chore] Fix compilation warning in storage backend (#1134)
022a650 [EP] Support multiple torch versions (#1098)
2369fb0 feat(Store): Support Connecting Multiple Dummy Clients to One Real Client (#1122)
635c1a2 [Store] Add support for static labels (host IP/cluster name) in client metrics (#1081)
2700fe1 [Misc] Add Codeowners (#1135)
4f68a76 [Doc] fix MC_MAX_EP_PER_CTX doc (#1142)
7527770 [Bug] fixed bug of master not using glog actually (#1075)
9067264 change cmake for ascend (#1114)

📊 Changes

1724 files changed (+359510 additions, -4539 deletions)

View changed files

📝 .clang-format (+3 -0)
➕ .devcontainer/Dockerfile (+43 -0)
➕ .devcontainer/README.md (+34 -0)
➕ .devcontainer/devcontainer.json (+25 -0)
➕ .github/CODEOWNERS (+20 -0)
➕ .github/ISSUE_TEMPLATE/1.bug_report.yml (+21 -0)
➕ .github/ISSUE_TEMPLATE/2.usage.yml (+25 -0)
➕ .github/ISSUE_TEMPLATE/3.performance_dicussion.yml (+22 -0)
➕ .github/ISSUE_TEMPLATE/4.feature_request.yml (+22 -0)
➕ .github/ISSUE_TEMPLATE/5.RFC.yml (+21 -0)
➕ .github/ISSUE_TEMPLATE/6.documentation.yml (+20 -0)
➕ .github/ISSUE_TEMPLATE/7.installation.yml (+20 -0)
➕ .github/ISSUE_TEMPLATE/config.yaml (+4 -0)
➕ .github/labeler.yml (+24 -0)
➕ .github/pull_request_template.md (+30 -0)
➕ .github/workflows/ci.yml (+575 -0)
➕ .github/workflows/ci_cu13.yml (+130 -0)
➕ .github/workflows/deploy.yml (+61 -0)
➕ .github/workflows/integration-test.yml (+115 -0)
➕ .github/workflows/labeler.yml (+21 -0)

...and 80 more files

📄 Description

Description

This branch aims to provide a communicator similar to ZMQ protocol, supporting 4 kinds of communication pattern:

Pattern	Direction	Sync/Async	Load Balance	Use Case
REQ/REP	Bidirectional	Synchronous	❌	RPC, Command-Response
PUB/SUB	Unidirectional	Asynchronous	❌	Broadcasting, Logging
PUSH/PULL	Unidirectional	Asynchronous	✅ Round-robin	Task Distribution
PAIR	Bidirectional	Asynchronous	❌	1-to-1 Communication

And the bottom layer uses coro rpc to provide data/tensor communications.

┌─────────────────────────────────────────────────────────────┐
│                    ZmqCommunicator                          │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐     │
│  │ Socket Mgmt  │  │ RPC Pools    │  │ RPC Servers  │     │
│  └──────────────┘  └──────────────┘  └──────────────┘     │
└────────────────────┬────────────────────────────────────────┘
                     │
     ┌───────────────┼───────────────┐
     │               │               │               │
┌────▼────┐   ┌─────▼──────┐  ┌────▼────┐    ┌────▼────┐
│ REQ/REP │   │  PUB/SUB   │  │PUSH/PULL│    │  PAIR   │
│ Pattern │   │  Pattern   │  │ Pattern │    │ Pattern │
└─────────┘   └────────────┘  └─────────┘    └─────────┘
     │               │               │               │
     └───────────────┴───────────────┴───────────────┘
                     │
          ┌──────────▼──────────┐
          │   coro_rpc Layer    │
          │  (yalantinglibs)    │
          └──────────┬──────────┘
                     │
          ┌──────────▼──────────┐
          │   async_simple      │
          │   (Coroutines)      │
          └──────────┬──────────┘
                     │
          ┌──────────▼──────────┐
          │  TCP / RDMA         │
          └─────────────────────┘

key features:

Zero-copy Transfer: Uses coro_rpc attachment mechanism for large data
Pattern-based Design: Abstract BasePattern with concrete implementations
Async Coroutines: Non-blocking operations with async_simple
Connection Pooling: Reuses RPC connections for efficiency
RDMA Support: Configurable via MC_RPC_PROTOCOL environment variable

Type of Change

Types
- Bug fix
- New feature
  - Transfer Engine
  - Mooncake Store
  - Mooncake EP
  - Integration
  - P2P Store
  - Python Wheel
- Breaking change
- CI/CD
- Documentation update
- Other

How Has This Been Tested?

this has been integrated to current compilation processes.

Checklist

I have performed a self-review of my own code.
I have updated the documentation.
I have added tests to prove my changes are effective.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/Mooncake/pull/1370 **Author:** [@JasonZhang517](https://github.com/JasonZhang517) **Created:** 1/13/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `zmq_communicator` --- ### 📝 Commits (10+) - [`4e8c82e`](https://github.com/kvcache-ai/Mooncake/commit/4e8c82ef874b2aba202bb5a628d21a1b94cd3905) [DEV] Add Pre-commit Check before Submit (#1124) - [`cf63fc8`](https://github.com/kvcache-ai/Mooncake/commit/cf63fc88a9892c31c38488195af858b38ba8b603) [Store] Cancel all negative ret val (#1129) - [`6512006`](https://github.com/kvcache-ai/Mooncake/commit/6512006aa99516f108bdff7278917df904737648) [Chore] Fix compilation warning in storage backend (#1134) - [`022a650`](https://github.com/kvcache-ai/Mooncake/commit/022a6504d3c1c1cf97e651e07823258ec307787f) [EP] Support multiple torch versions (#1098) - [`2369fb0`](https://github.com/kvcache-ai/Mooncake/commit/2369fb0f13e31a64eadd547549dbad429650c55d) feat(Store): Support Connecting Multiple Dummy Clients to One Real Client (#1122) - [`635c1a2`](https://github.com/kvcache-ai/Mooncake/commit/635c1a25d52d9e33729c2934f5401ddc06c92aa3) [Store] Add support for static labels (host IP/cluster name) in client metrics (#1081) - [`2700fe1`](https://github.com/kvcache-ai/Mooncake/commit/2700fe1338ea89bcad93f6d1d2d50cca871150cc) [Misc] Add Codeowners (#1135) - [`4f68a76`](https://github.com/kvcache-ai/Mooncake/commit/4f68a7616112c0425c0d7e1e593e9511b945a4f0) [Doc] fix MC_MAX_EP_PER_CTX doc (#1142) - [`7527770`](https://github.com/kvcache-ai/Mooncake/commit/7527770059edba88de76bb5514b6350791ac93b5) [Bug] fixed bug of master not using glog actually (#1075) - [`9067264`](https://github.com/kvcache-ai/Mooncake/commit/9067264a916ce407433e76f51055e3f9e63e9456) change cmake for ascend (#1114) ### 📊 Changes **1724 files changed** (+359510 additions, -4539 deletions) <details> <summary>View changed files</summary> 📝 `.clang-format` (+3 -0) ➕ `.devcontainer/Dockerfile` (+43 -0) ➕ `.devcontainer/README.md` (+34 -0) ➕ `.devcontainer/devcontainer.json` (+25 -0) ➕ `.github/CODEOWNERS` (+20 -0) ➕ `.github/ISSUE_TEMPLATE/1.bug_report.yml` (+21 -0) ➕ `.github/ISSUE_TEMPLATE/2.usage.yml` (+25 -0) ➕ `.github/ISSUE_TEMPLATE/3.performance_dicussion.yml` (+22 -0) ➕ `.github/ISSUE_TEMPLATE/4.feature_request.yml` (+22 -0) ➕ `.github/ISSUE_TEMPLATE/5.RFC.yml` (+21 -0) ➕ `.github/ISSUE_TEMPLATE/6.documentation.yml` (+20 -0) ➕ `.github/ISSUE_TEMPLATE/7.installation.yml` (+20 -0) ➕ `.github/ISSUE_TEMPLATE/config.yaml` (+4 -0) ➕ `.github/labeler.yml` (+24 -0) ➕ `.github/pull_request_template.md` (+30 -0) ➕ `.github/workflows/ci.yml` (+575 -0) ➕ `.github/workflows/ci_cu13.yml` (+130 -0) ➕ `.github/workflows/deploy.yml` (+61 -0) ➕ `.github/workflows/integration-test.yml` (+115 -0) ➕ `.github/workflows/labeler.yml` (+21 -0) _...and 80 more files_ </details> ### 📄 Description ## Description This branch aims to provide a communicator similar to ZMQ protocol, supporting 4 kinds of communication pattern: | Pattern | Direction | Sync/Async | Load Balance | Use Case | |---------|-----------|------------|--------------|----------| | REQ/REP | Bidirectional | Synchronous | ❌ | RPC, Command-Response | | PUB/SUB | Unidirectional | Asynchronous | ❌ | Broadcasting, Logging | | PUSH/PULL | Unidirectional | Asynchronous | ✅ Round-robin | Task Distribution | | PAIR | Bidirectional | Asynchronous | ❌ | 1-to-1 Communication | And the bottom layer uses coro rpc to provide data/tensor communications. ``` ┌─────────────────────────────────────────────────────────────┐ │ ZmqCommunicator │ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ │ │ Socket Mgmt │ │ RPC Pools │ │ RPC Servers │ │ │ └──────────────┘ └──────────────┘ └──────────────┘ │ └────────────────────┬────────────────────────────────────────┘ │ ┌───────────────┼───────────────┐ │ │ │ │ ┌────▼────┐ ┌─────▼──────┐ ┌────▼────┐ ┌────▼────┐ │ REQ/REP │ │ PUB/SUB │ │PUSH/PULL│ │ PAIR │ │ Pattern │ │ Pattern │ │ Pattern │ │ Pattern │ └─────────┘ └────────────┘ └─────────┘ └─────────┘ │ │ │ │ └───────────────┴───────────────┴───────────────┘ │ ┌──────────▼──────────┐ │ coro_rpc Layer │ │ (yalantinglibs) │ └──────────┬──────────┘ │ ┌──────────▼──────────┐ │ async_simple │ │ (Coroutines) │ └──────────┬──────────┘ │ ┌──────────▼──────────┐ │ TCP / RDMA │ └─────────────────────┘ ``` key features: 1. **Zero-copy Transfer**: Uses `coro_rpc` attachment mechanism for large data 2. **Pattern-based Design**: Abstract `BasePattern` with concrete implementations 3. **Async Coroutines**: Non-blocking operations with `async_simple` 4. **Connection Pooling**: Reuses RPC connections for efficiency 5. **RDMA Support**: Configurable via `MC_RPC_PROTOCOL` environment variable ## Type of Change * Types - [ ] Bug fix - [ ] New feature - [x] Transfer Engine - [ ] Mooncake Store - [ ] Mooncake EP - [ ] Integration - [ ] P2P Store - [ ] Python Wheel - [ ] Breaking change - [ ] CI/CD - [ ] Documentation update - [ ] Other ## How Has This Been Tested? this has been integrated to current compilation processes. ## Checklist - [ ] I have performed a self-review of my own code. - [ ] I have updated the documentation. - [ ] I have added tests to prove my changes are effective. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-29 21:33:38 +00:00