kvcache-ai/Mooncake

Fork 0

mirror of https://github.com/kvcache-ai/Mooncake.git synced 2026-11-05 01:17:55 +00:00

[GH-ISSUE #594] [Usage]: Questions about transfer engine write #198

New issue

Closed

opened 2026-10-29 21:10:46 +00:00 by bsakaguchi · 7 comments

bsakaguchi commented

2026-10-29 21:10:46 +00:00

Owner

Originally created by @zzt93 on GitHub (Jul 6, 2025).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/594

Describe your usage question

When Node A sends a message to Node B via the transfer engine, can Node B receive an event notification? (not see any api in doc or code)
Are multiple write operations (such as transfer_submit_write, transfer_sync_write, etc.) from Node A to another Node B always ordered?
In the context of managed buffers, why not allocate the buffer and exchange the peer_buffer_address at the time of engine creation? Obtaining the address through ZMQ or other mechanisms feels somewhat awkward to me—after all, it's a communication library, and having to depend on an additional communication channel seems redundant.

Before submitting a new issue...

Make sure you already searched for relevant issues and read the documentation

Originally created by @zzt93 on GitHub (Jul 6, 2025). Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/594 ### Describe your usage question - When Node A sends a message to Node B via the transfer engine, can Node B receive an event notification? (not see any api in doc or code) - Are multiple write operations (such as transfer_submit_write, transfer_sync_write, etc.) from Node A to another Node B always ordered? - In the context of managed buffers, why not allocate the buffer and exchange the peer_buffer_address at the time of engine creation? Obtaining the address through ZMQ or other mechanisms feels somewhat awkward to me—after all, it's a communication library, and having to depend on an additional communication channel seems redundant. ### Before submitting a new issue... - [x] Make sure you already searched for relevant issues and read the [documentation](https://kvcache-ai.github.io/Mooncake/)

bsakaguchi

2026-10-29 21:10:46 +00:00

closed this issue
added the
stale

auto-closed
labels

bsakaguchi commented

2026-10-29 21:10:47 +00:00

Author

Owner

@alogfans commented on GitHub (Jul 7, 2025):

We recently added new APIs submitTransferWithNotify and getNotifies in current Transfer Engine. These APIs use out-of-band communication to achieve notification on receiver side.
They are NOT guaranteed ordered, especially if multiple NICs are used.
Our older implementation used in vLLM doesn't include P2P handshake, so we have to add an extra component to exchange these data. In the next major version we will add support to exchange buffer addresses natively.

@alogfans commented on GitHub (Jul 7, 2025): 1. We recently added new APIs **submitTransferWithNotify** and **getNotifies** in current Transfer Engine. These APIs use out-of-band communication to achieve notification on receiver side. 2. They are NOT guaranteed ordered, especially if multiple NICs are used. 3. Our older implementation used in vLLM doesn't include P2P handshake, so we have to add an extra component to exchange these data. In the next major version we will add support to exchange buffer addresses natively.

bsakaguchi commented

2026-10-29 21:10:47 +00:00

Author

Owner

@zzt93 commented on GitHub (Jul 7, 2025):

We recently added new APIs submitTransferWithNotify and getNotifies in current Transfer Engine. These APIs use out-of-band communication to achieve notification on receiver side.

The getNotifies API seems to require polling on the receiving end. Have you considered using a registration function and a callback mechanism instead?

exchange buffer addresses natively.

Any timeline for this feature?

@zzt93 commented on GitHub (Jul 7, 2025): > We recently added new APIs submitTransferWithNotify and getNotifies in current Transfer Engine. These APIs use out-of-band communication to achieve notification on receiver side. The `getNotifies` API seems to require polling on the receiving end. Have you considered using a registration function and a callback mechanism instead? > exchange buffer addresses natively. Any timeline for this feature?

bsakaguchi commented

2026-10-29 21:10:47 +00:00

Author

Owner

@zzt93 commented on GitHub (Jul 7, 2025):

In the next major version we will add support to exchange buffer addresses natively.

If you haven't developed this feature yet, I can try to implement it and submit a PR to you.

@zzt93 commented on GitHub (Jul 7, 2025): > In the next major version we will add support to exchange buffer addresses natively. If you haven't developed this feature yet, I can try to implement it and submit a PR to you.

bsakaguchi commented

2026-10-29 21:10:47 +00:00

Author

Owner

@zzt93 commented on GitHub (Jul 7, 2025):

They are NOT guaranteed ordered, especially if multiple NICs are used.

If I use sync api like transfer_sync_write to write from Node A to Node B, will it be ordered?

@zzt93 commented on GitHub (Jul 7, 2025): > They are NOT guaranteed ordered, especially if multiple NICs are used. If I use sync api like `transfer_sync_write` to write from Node A to Node B, will it be ordered?

bsakaguchi commented

2026-10-29 21:10:47 +00:00

Author

Owner

@alogfans commented on GitHub (Jul 9, 2025):

The getNotifies API seems to require polling on the receiving end. Have you considered using a registration function and a callback mechanism instead?

The main purpose of getNotifies is to enable the notification support in NIXL. You can alternatively spawn a thread and poll for completions.

Any timeline for the (exchange buffer addresses natively.) feature?

We are now developing and testing Mooncake Transfer Engine V1, which has significant architecture changes and many more features. We expect it to be released during 25Q3.

If I use sync api like transfer_sync_write to write from Node A to Node B, will it be ordered?

Sync operations are ordered.

@alogfans commented on GitHub (Jul 9, 2025): > The getNotifies API seems to require polling on the receiving end. Have you considered using a registration function and a callback mechanism instead? The main purpose of `getNotifies` is to enable the `notification` support in NIXL. You can alternatively spawn a thread and poll for completions. > Any timeline for the (exchange buffer addresses natively.) feature? We are now developing and testing Mooncake Transfer Engine V1, which has significant architecture changes and many more features. We expect it to be released during 25Q3. > If I use sync api like transfer_sync_write to write from Node A to Node B, will it be ordered? Sync operations are ordered.

bsakaguchi commented

2026-10-29 21:10:47 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Jun 30, 2026):

This issue has had no activity for 90 days and will be closed in 7 days if there is no further activity. Please comment or react if it should stay open.

@github-actions[bot] commented on GitHub (Jun 30, 2026): This issue has had no activity for 90 days and will be closed in 7 days if there is no further activity. Please comment or react if it should stay open.

bsakaguchi commented

2026-10-29 21:10:48 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Jul 7, 2026):

Closing due to 3 months of inactivity. If this is still relevant, please comment and we can reopen.

@github-actions[bot] commented on GitHub (Jul 7, 2026): Closing due to 3 months of inactivity. If this is still relevant, please comment and we can reopen.

bsakaguchi referenced this issue

2026-10-29 21:26:59 +00:00

[PR #198] [CLOSED] [Feature] support basic topology-aware in sglang adaptor layer #1388