kvcache-ai/Mooncake

Fork 0

mirror of https://github.com/kvcache-ai/Mooncake.git synced 2026-11-05 01:17:55 +00:00

[GH-ISSUE #12] Failed to set socket affinity #7

New issue

Closed

opened 2026-10-29 21:07:46 +00:00 by bsakaguchi · 6 comments

bsakaguchi commented

2026-10-29 21:07:46 +00:00

Owner

Originally created by @BaiStone2017 on GitHub (Dec 3, 2024).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/12

My mooncake config file as follow：

While I run those commands：
etcd --listen-client-urls http://0.0.0.0:2379 --advertise-client-urls http://192.169.137.233:2379
NCCL_DEBUG=INFO VLLM_LOGGING_LEVEL=DEBUG VLLM_HOST_IP=192.169.137.233 VLLM_PORT="51000" MASTER_ADDR=192.169.137.233 MASTER_PORT="54324" MOONCAKE_CONFIG_PATH=./mooncake_tcp.json VLLM_DISTRIBUTED_KV_ROLE=producer python3 -m vllm.entrypoints.openai.api_server --model qwen2-7b --port 8100 --max-model-len 4096 --gpu-memory-utilization 0.9 --disable-log-stats --disable-log-requests
and
NCCL_DEBUG=INFO VLLM_LOGGING_LEVEL=DEBUG VLLM_HOST_IP=192.169.137.233 VLLM_PORT="51000" MASTER_ADDR=192.169.137.233 MASTER_PORT="54324" MOONCAKE_CONFIG_PATH=./mooncake_tcp.json VLLM_DISTRIBUTED_KV_ROLE=consumer python3 -m vllm.entrypoints.openai.api_server --model qwen2-7b --port 8200 --max-model-len 4096 --gpu-memory-utilization 0.9 --disable-log-stats --disable-log-requests
Only in VLLM_DISTRIBUTED_KV_ROLE=producer instance occur as follow:

Is this normal? Although this does not affect normal inference.

Originally created by @BaiStone2017 on GitHub (Dec 3, 2024). Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/12 My mooncake config file as follow： ![image](https://github.com/user-attachments/assets/ee35c5c5-afa7-41da-9e15-febff1499dce) While I run those commands： `etcd --listen-client-urls http://0.0.0.0:2379 --advertise-client-urls http://192.169.137.233:2379` `NCCL_DEBUG=INFO VLLM_LOGGING_LEVEL=DEBUG VLLM_HOST_IP=192.169.137.233 VLLM_PORT="51000" MASTER_ADDR=192.169.137.233 MASTER_PORT="54324" MOONCAKE_CONFIG_PATH=./mooncake_tcp.json VLLM_DISTRIBUTED_KV_ROLE=producer python3 -m vllm.entrypoints.openai.api_server --model qwen2-7b --port 8100 --max-model-len 4096 --gpu-memory-utilization 0.9 --disable-log-stats --disable-log-requests ` and `NCCL_DEBUG=INFO VLLM_LOGGING_LEVEL=DEBUG VLLM_HOST_IP=192.169.137.233 VLLM_PORT="51000" MASTER_ADDR=192.169.137.233 MASTER_PORT="54324" MOONCAKE_CONFIG_PATH=./mooncake_tcp.json VLLM_DISTRIBUTED_KV_ROLE=consumer python3 -m vllm.entrypoints.openai.api_server --model qwen2-7b --port 8200 --max-model-len 4096 --gpu-memory-utilization 0.9 --disable-log-stats --disable-log-requests ` Only in VLLM_DISTRIBUTED_KV_ROLE=producer instance occur as follow: ![image](https://github.com/user-attachments/assets/5969e0ba-57e2-4e6c-884a-4a2708d38645) Is this normal? Although this does not affect normal inference.

bsakaguchi closed this issue

2026-10-29 21:07:46 +00:00

bsakaguchi commented

2026-10-29 21:07:47 +00:00

Author

Owner

@alogfans commented on GitHub (Dec 4, 2024):

It seems that bindToSocket() in common.h not worked as expected. Can you provide the value of socket_id and cpu_set in bindToSocket(), as well the output of numactl -H?

@alogfans commented on GitHub (Dec 4, 2024): It seems that `bindToSocket()` in `common.h` not worked as expected. Can you provide the value of `socket_id` and `cpu_set` in `bindToSocket()`, as well the output of `numactl -H`?

bsakaguchi commented

2026-10-29 21:07:47 +00:00

Author

Owner

@BaiStone2017 commented on GitHub (Dec 4, 2024):

It seems that bindToSocket() in common.h not worked as expected. Can you provide the value of socket_id and cpu_set in bindToSocket(), as well the output of numactl -H?

和启动容器时分配的数量有关系吗？

@BaiStone2017 commented on GitHub (Dec 4, 2024): > It seems that `bindToSocket()` in `common.h` not worked as expected. Can you provide the value of `socket_id` and `cpu_set` in `bindToSocket()`, as well the output of `numactl -H`? ![C116_2024120409130060109](https://github.com/user-attachments/assets/cc522c66-396b-48f4-93e3-5c5aac60ae96) ![C114_2024120408513904655](https://github.com/user-attachments/assets/55ddf7c3-b8f7-4e56-9cf0-880d6b6ad087) ![C549_2024120408525695659](https://github.com/user-attachments/assets/511b9599-3529-4d43-8334-024d227a7611) 和启动容器时分配的数量有关系吗？

bsakaguchi commented

2026-10-29 21:07:47 +00:00

Author

Owner

@alogfans commented on GitHub (Dec 5, 2024):

Our current implementation uses the numa_* family of interfaces to get a list of threads located at NUMA 0 that matches what numactl -H shows, but only a number of them are available in a container environment, so populating the cpu_set is limited to the ones located at Available CPUs (36/37). You can try this.

@alogfans commented on GitHub (Dec 5, 2024): Our current implementation uses the numa_* family of interfaces to get a list of threads located at NUMA 0 that matches what numactl -H shows, but only a number of them are available in a container environment, so populating the cpu_set is limited to the ones located at Available CPUs (36/37). You can try this.

bsakaguchi commented

2026-10-29 21:07:47 +00:00

Author

Owner

@BaiStone2017 commented on GitHub (Dec 5, 2024):

Our current implementation uses the numa_* family of interfaces to get a list of threads located at NUMA 0 that matches what numactl -H shows, but only a number of them are available in a container environment, so populating the cpu_set is limited to the ones located at Available CPUs (36/37). You can try this.

谢谢。请问下，如果指定线程id，需要怎么样修改代码？

@BaiStone2017 commented on GitHub (Dec 5, 2024): > Our current implementation uses the numa_* family of interfaces to get a list of threads located at NUMA 0 that matches what numactl -H shows, but only a number of them are available in a container environment, so populating the cpu_set is limited to the ones located at Available CPUs (36/37). You can try this. 谢谢。请问下，如果指定线程id，需要怎么样修改代码？

bsakaguchi commented

2026-10-29 21:07:48 +00:00

Author

Owner

@alogfans commented on GitHub (Dec 10, 2024):

You can try out our latest patch. As shown in mooncake-transfer-engine/include/common.h, we use numa_all_cpus_ptr to identify available cpus for current process.

@alogfans commented on GitHub (Dec 10, 2024): You can try out our latest patch. As shown in `mooncake-transfer-engine/include/common.h`, we use `numa_all_cpus_ptr` to identify available cpus for current process.

bsakaguchi commented

2026-10-29 21:07:48 +00:00

Author

Owner

@BaiStone2017 commented on GitHub (Dec 10, 2024):

You can try out our latest patch. As shown in mooncake-transfer-engine/include/common.h, we use numa_all_cpus_ptr to identify available cpus for current process.

thx

@BaiStone2017 commented on GitHub (Dec 10, 2024): > You can try out our latest patch. As shown in `mooncake-transfer-engine/include/common.h`, we use `numa_all_cpus_ptr` to identify available cpus for current process. thx

bsakaguchi referenced this issue

2026-10-29 21:17:35 +00:00

[GH-ISSUE #2440] [Bug]: Execute mooncake+hixl, final destruction phase core #646

bsakaguchi referenced this issue

2026-10-29 21:26:01 +00:00

[PR #7] [MERGED] fix the p2pstore go module name, make it could be imported in other projects. #1280