kvcache-ai/Mooncake

Fork 0

mirror of https://github.com/kvcache-ai/Mooncake.git synced 2026-11-05 01:17:55 +00:00

[GH-ISSUE #1406] [Bug]: Registering self managed memory from mmap to ascend direct transport fails #452

New issue

Closed

opened 2026-10-29 21:14:48 +00:00 by bsakaguchi · 9 comments

bsakaguchi commented

2026-10-29 21:14:48 +00:00

Owner

Originally created by @matthewygf on GitHub (Jan 19, 2026).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/1406

Bug Report

My setup is using LMCache v0.3.12, Mooncake v0.3.7post2, but as far as I know, the current ascend direct transport still have the same code path.

To reproduce the issue:

create a memory buffer using mmap
and then call the mooncakestore python client register memory
the underlying ascend_direct_transport will lookup the type using ret = aclrtPointerGetAttributes(addr, &attributes);

This will fail due to aclrtPointerGetAttributes currently only support aclMallocHost allocated memory.

One potential fix is to allow the python client to pass in the type of memory but require changing the interface.

Second potential fix is to use the slightly lower level api from ascend, which should support the mmap memory.

Before submitting...

Ensure you searched for relevant issues and read the [documentation]

Originally created by @matthewygf on GitHub (Jan 19, 2026). Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/1406 ### Bug Report My setup is using LMCache v0.3.12, Mooncake v0.3.7post2, but as far as I know, the current ascend direct transport still have the same code path. To reproduce the issue: - create a memory buffer using mmap - and then call the mooncakestore python client register memory - the underlying ascend_direct_transport will lookup the type using `ret = aclrtPointerGetAttributes(addr, &attributes);` This will fail due to aclrtPointerGetAttributes currently only support aclMallocHost allocated memory. One potential fix is to allow the python client to pass in the type of memory but require changing the interface. Second potential fix is to use the slightly lower level api from ascend, which should support the mmap memory. ### Before submitting... - [x] Ensure you searched for relevant issues and read the [documentation]

bsakaguchi

2026-10-29 21:14:48 +00:00

closed this issue
added the
stale

auto-closed

bug
labels

bsakaguchi commented

2026-10-29 21:14:49 +00:00

Author

Owner

@matthewygf commented on GitHub (Jan 19, 2026):

@matthewygf commented on GitHub (Jan 19, 2026): Tagging related issue https://github.com/LMCache/LMCache-Ascend/issues/144

bsakaguchi commented

2026-10-29 21:14:49 +00:00

Author

Owner

@ykwd commented on GitHub (Jan 19, 2026):

@ascend-direct-dev could you take a look? Thanks

@ykwd commented on GitHub (Jan 19, 2026): @ascend-direct-dev could you take a look? Thanks

bsakaguchi commented

2026-10-29 21:14:49 +00:00

Author

Owner

@ascend-direct-dev commented on GitHub (Jan 19, 2026):

Hixl only supports memory allocated via aclMallocHost. I will list several reasons why memory allocated via mmap is not recommended:

Under the UB/HCCS transport protocol, the performance of memcpy operations using mmap-allocated host memory is significantly worse compared to memory allocated via aclMallocHost.

In buffer pool mode, when performing D2H or H2D copies, the aclrt interface cannot recognize mmap-allocated memory and will directly report an error.

@ascend-direct-dev commented on GitHub (Jan 19, 2026): Hixl only supports memory allocated via aclMallocHost. I will list several reasons why memory allocated via mmap is not recommended: Under the UB/HCCS transport protocol, the performance of memcpy operations using mmap-allocated host memory is significantly worse compared to memory allocated via aclMallocHost. In buffer pool mode, when performing D2H or H2D copies, the aclrt interface cannot recognize mmap-allocated memory and will directly report an error.

bsakaguchi commented

2026-10-29 21:14:50 +00:00

Author

Owner

@ascend-direct-dev commented on GitHub (Jan 19, 2026):

I recommend adapting lmcache-ascend to use aclMallocHost for allocating host memory.

@ascend-direct-dev commented on GitHub (Jan 19, 2026): I recommend adapting lmcache-ascend to use aclMallocHost for allocating host memory.

bsakaguchi commented

2026-10-29 21:14:50 +00:00

Author

Owner

@matthewygf commented on GitHub (Jan 19, 2026):

@ascend-direct-dev Thanks very much for your reply !

Indeed we usually use aclMallocHost for allocating host memory. However, in some settings where we want to move around the memory, we will use mmap.

What's the real reason being aclMallocHost being faster ? Is that being page locked memory and prepopulating the huge pages? In that case we can also do these via the sys calls etc.

I really don't see why the cannot recognize mmap-allocated memory should not be fixed. This is actually recognizable if we use the rt APIs. Thanks !

@matthewygf commented on GitHub (Jan 19, 2026): @ascend-direct-dev Thanks very much for your reply ! Indeed we usually use `aclMallocHost` for allocating host memory. However, in some settings where we want to move around the memory, we will use `mmap`. What's the real reason being aclMallocHost being faster ? Is that being page locked memory and prepopulating the huge pages? In that case we can also do these via the sys calls etc. I really don't see why the `cannot recognize mmap-allocated memory` should not be fixed. This is actually recognizable if we use the rt APIs. Thanks !

bsakaguchi commented

2026-10-29 21:14:50 +00:00

Author

Owner

@ascend-direct-dev commented on GitHub (Jan 20, 2026):

In buffer pool mode, which use d2rd + d2h_copy to achieve d2rh transfer, the d2h_copy phase will call aclrtMemcpyBatch, which cannot recognize mmap-allocated memory. Additionally, if the memory is allocated with aclMallocHost, it means the memory is definitely pinned, which is safer. Does LMcache-ascend have to use memory allocated via mmap?

@ascend-direct-dev commented on GitHub (Jan 20, 2026): In buffer pool mode, which use d2rd + d2h_copy to achieve d2rh transfer, the d2h_copy phase will call aclrtMemcpyBatch, which cannot recognize mmap-allocated memory. Additionally, if the memory is allocated with aclMallocHost, it means the memory is definitely pinned, which is safer. Does LMcache-ascend have to use memory allocated via mmap?

bsakaguchi commented

2026-10-29 21:14:50 +00:00

Author

Owner

@matthewygf commented on GitHub (Jan 20, 2026):

@ascend-direct-dev thanks again for the reply.

I see.

We find that any h2d if we have a numa-friendly memory allocated on A2 machine, the performance would be better perhaps ~5-8%. We currently achieve this by mmap , mlock, mbind which can achieve pinned numa-friendly memory.

I suppose we can switch back to aclmallochost if needed to turn on bufferpool mode etc.

@matthewygf commented on GitHub (Jan 20, 2026): @ascend-direct-dev thanks again for the reply. I see. We find that any h2d if we have a numa-friendly memory allocated on A2 machine, the performance would be better perhaps ~5-8%. We currently achieve this by mmap , mlock, mbind which can achieve pinned numa-friendly memory. I suppose we can switch back to aclmallochost if needed to turn on bufferpool mode etc.

bsakaguchi commented

2026-10-29 21:14:50 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Jun 9, 2026):

This issue has had no activity for 90 days and will be closed in 7 days if there is no further activity. Please comment or react if it should stay open.

@github-actions[bot] commented on GitHub (Jun 9, 2026): This issue has had no activity for 90 days and will be closed in 7 days if there is no further activity. Please comment or react if it should stay open.

bsakaguchi commented

2026-10-29 21:14:51 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Jun 16, 2026):

Closing due to 3 months of inactivity. If this is still relevant, please comment and we can reopen.

@github-actions[bot] commented on GitHub (Jun 16, 2026): Closing due to 3 months of inactivity. If this is still relevant, please comment and we can reopen.