[GH-ISSUE #3749] [Bug]: [AscendDirect] On 910B A2‑series machines, data cannot be transmitted normally when the protocol is configured as ascend. #952

New issue

Closed

opened 2026-10-29 21:21:16 +00:00 by bsakaguchi · 3 comments

bsakaguchi commented

2026-10-29 21:21:16 +00:00

Owner

Originally created by @HyperionAdonis on GitHub (Aug 28, 2026).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/3749

Originally assigned to: @ascend-direct-dev on GitHub.

Bug Report

running environment
910B Atlas A2 try to run vllm-ascend with mooncake，use Device RoCE（HIXL CS）
mooncake-master（v0.3.12.post1）、mooncake-client（v0.3.12.post1）、vllm-ascend(v0.25.0)
vllm-ascend config:
export ASCEND_GLOBAL_RESOURCE_CONFIG='{comm_resource_config.protocol_desc: ["roce:device"]}'.
{"kv_connector":"AscendStoreConnector","kv_role":"kv_both","kv_connector_extra_config":{"lookup_rpc_port":"0","backend":"mooncake","load_async":false, "register_buffer": true, "use_layer_wise": false, "mooncake_rpc_port": "0"}}
vllm-ascend-mooncake config:
global_segment_size: 0
protocol: ascend
metadata_server: P2PHANDSHAKE
standalone mooncake-client config（server）:
global_segment_size: 32G
protocol: ascend
metadata_server: P2PHANDSHAKE
error log
put_start success、transfer failed

Before submitting...

Ensure you searched for relevant issues and read the [documentation]

Originally created by @HyperionAdonis on GitHub (Aug 28, 2026). Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/3749 Originally assigned to: @ascend-direct-dev on GitHub. ### Bug Report 1. **running environment** 910B Atlas A2 try to run vllm-ascend with mooncake，use Device RoCE（HIXL CS） mooncake-master（v0.3.12.post1）、mooncake-client（v0.3.12.post1）、vllm-ascend(v0.25.0) vllm-ascend config: export ASCEND_GLOBAL_RESOURCE_CONFIG='{comm_resource_config.protocol_desc: ["roce:device"]}'. {"kv_connector":"AscendStoreConnector","kv_role":"kv_both","kv_connector_extra_config":{"lookup_rpc_port":"0","backend":"mooncake","load_async":false, "register_buffer": true, "use_layer_wise": false, "mooncake_rpc_port": "0"}} vllm-ascend-mooncake config: global_segment_size: 0 protocol: ascend metadata_server: P2PHANDSHAKE standalone mooncake-client config（server）: global_segment_size: 32G protocol: ascend metadata_server: P2PHANDSHAKE 2. **error log** put_start success、transfer failed <img width="2728" height="1144" alt="Image" src="https://github.com/user-attachments/assets/58846f3e-15ba-4c03-b094-093cea689585" /> ### Before submitting... - [ ] Ensure you searched for relevant issues and read the [documentation]

bsakaguchi

2026-10-29 21:21:16 +00:00

closed this issue
added the
bug
label

bsakaguchi commented

2026-10-29 21:21:18 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Aug 28, 2026):

Thanks for opening this issue, @HyperionAdonis!

Field	Value
Issue	#3749
GitHub user ID	`20439311`
Reporter	@HyperionAdonis

A maintainer will triage this when possible. To help us respond faster, please include:

Mooncake version or commit SHA
Environment (OS, CUDA/driver, RDMA stack if relevant)
Steps to reproduce and expected vs. actual behavior

Useful links: Documentation · Contributing guide

This message was posted automatically by the issue bot.

@github-actions[bot] commented on GitHub (Aug 28, 2026): Thanks for opening this issue, @HyperionAdonis! | Field | Value | |-------|-------| | **Issue** | #3749 | | **GitHub user ID** | `20439311` | | **Reporter** | @HyperionAdonis | A maintainer will triage this when possible. To help us respond faster, please include: - Mooncake version or commit SHA - Environment (OS, CUDA/driver, RDMA stack if relevant) - Steps to reproduce and expected vs. actual behavior Useful links: [Documentation](https://kvcache-ai.github.io/Mooncake/) · [Contributing guide](https://github.com/kvcache-ai/Mooncake/blob/main/CONTRIBUTING.md) > This message was posted automatically by the issue bot.

bsakaguchi commented

2026-10-29 21:21:18 +00:00

Author

Owner

@HyperionAdonis commented on GitHub (Aug 28, 2026):

with CANN 9.1.0

@HyperionAdonis commented on GitHub (Aug 28, 2026): with CANN 9.1.0

bsakaguchi commented

2026-10-29 21:21:18 +00:00

Author

Owner

@HyperionAdonis commented on GitHub (Aug 31, 2026):

forget set env:
export ASCEND_BUFFER_POOL=0:0
export HCCL_INTRA_ROCE_ENABLE=1

@HyperionAdonis commented on GitHub (Aug 31, 2026): forget set env: export ASCEND_BUFFER_POOL=0:0 export HCCL_INTRA_ROCE_ENABLE=1