[GH-ISSUE #98] run transfer_engine_bench.cpp in NPU device error #54

New issue

Closed

opened 2026-10-29 21:08:27 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-29 21:08:27 +00:00

Owner

Originally created by @zx-ai on GitHub (Feb 12, 2025).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/98

After compiling (without enabling USE_CUDA), I successfully ran a demo on the NPU device (transferring data from A machine's DRAM to B machine's DRAM).
In order to verify if data can be transferred from the NPU memory of machine A to the NPU memory of machine B using the transfer_engine on NPU devices.So, I attempted to modify the allocateMemoryPool() function on machine A, using the aclrtMalloc function to allocate memory on the NPU device.
static void *allocateMemoryPool(size_t size, int socket_id,
bool from_vram = false) {
#ifdef USE_ATLAS
if (from_vram) {
void *d_buf;
aclrtMalloc(&d_buf, size, ACL _MEM_MALLOC_HUGE_FIRST);
return d_buf;
}
#endif
return numa_alloc_onnode(size, socket_id);
}
Upon running the modified code, I encountered an error:
E0212 17:34:25.448876 212704 rdma_context.cpp:227] Fail to register memory 0x14ef5930: Cannot allocate memory [12]
*** Check failure stack trace: ***F0212 17:34:25.448904 212704 transfer_engine_bench.cpp:286] Assert failed: !rc

I am curious as to why GPU memory can be registered using the ibv_reg_mr() function, whereas NPU memory cannot be successfully registered. And Is there any way to solve this problem? Currently, can the main version or the V2.0 tag solve this problem? Will this project support deployment on NPU devices in the future?
I am very much looking forward to your response. Thank you again.

Originally created by @zx-ai on GitHub (Feb 12, 2025). Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/98 After compiling (without enabling USE_CUDA), I successfully ran a demo on the NPU device (transferring data from A machine's DRAM to B machine's DRAM). In order to verify if data can be transferred from the NPU memory of machine A to the NPU memory of machine B using the transfer_engine on NPU devices.So, I attempted to modify the allocateMemoryPool() function on machine A, using the aclrtMalloc function to allocate memory on the NPU device. static void *allocateMemoryPool(size_t size, int socket_id, bool from_vram = false) { #ifdef USE_ATLAS if (from_vram) { void *d_buf; aclrtMalloc(&d_buf, size, ACL _MEM_MALLOC_HUGE_FIRST); return d_buf; } #endif return numa_alloc_onnode(size, socket_id); } Upon running the modified code, I encountered an error: E0212 17:34:25.448876 212704 rdma_context.cpp:227] Fail to register memory 0x14ef5930: Cannot allocate memory [12] *** Check failure stack trace: ***F0212 17:34:25.448904 212704 transfer_engine_bench.cpp:286] Assert failed: !rc I am curious as to why GPU memory can be registered using the ibv_reg_mr() function, whereas NPU memory cannot be successfully registered. And Is there any way to solve this problem? Currently, can the main version or the V2.0 tag solve this problem? Will this project support deployment on NPU devices in the future? I am very much looking forward to your response. Thank you again.

bsakaguchi closed this issue

2026-10-29 21:08:28 +00:00

bsakaguchi commented

2026-10-29 21:08:29 +00:00

Author

Owner

@alogfans commented on GitHub (Feb 13, 2025):

duplicated with #97.

@alogfans commented on GitHub (Feb 13, 2025): duplicated with #97.

bsakaguchi commented

2026-10-29 21:08:29 +00:00

Author

Owner

@stmatengss commented on GitHub (Feb 13, 2025):

Currently, our project does not support NPU devices, and the error you encountered is likely due to differences in memory management between NPU and GPU. If you're interested in contributing NPU support, we'd be happy to collaborate and review your pull requests!

@stmatengss commented on GitHub (Feb 13, 2025): Currently, our project does not support NPU devices, and the error you encountered is likely due to differences in memory management between NPU and GPU. If you're interested in contributing NPU support, we'd be happy to collaborate and review your pull requests!