[PR #685] [MERGED] fix numa cpu distribution 修复numa cpu分布问题 #1458

New issue

Closed

opened 2026-10-29 14:52:37 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 14:52:37 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/ktransformers/pull/685
Author: @wkgcass
Created: 2/26/2025
Status: ✅ Merged
Merged: 2/26/2025
Merged by: @chenht2022

Base: main ← Head: main

📝 Commits (1)

b2bff17 fix numa cpu distribution

📊 Changes

1 file changed (+6 additions, -1 deletions)

View changed files

📝 ktransformers/ktransformers_ext/cpu_backend/backend.cpp (+6 -1)

📄 Description

问题描述

我的平台是双socket，每个socket24核（关闭超线程）。当我配置cpu_infer=41时，性能较高，而配置cpu_infer=42时，性能骤降到1token/s。
htop观察发现socket1上的cpu被全部占满，而socket1上有8个空闲的cpu。

也就是说numa cpu分布不均匀。推测配置42时性能骤降的原因是：socket1上的worker数多于cpu核数。

排查过程

我注意到numa bind位于backend.cpp的Backend::process_tasks里，numa_node会根据如下方式计算：

numa_node = thread_id * numa_num_configured_nodes() / thread_num_;

按理说应该是均匀的才对。但在进行一番print debug后，我发现thread_num_会发生变化：

如下是配置cpu_infer=41时的sort后的输出：

thread_id = 0, nodes = 2, thread_num = 32, numa_node = 0,
thread_id = 1, nodes = 2, thread_num = 32, numa_node = 0,
thread_id = 10, nodes = 2, thread_num = 32, numa_node = 0,
thread_id = 11, nodes = 2, thread_num = 32, numa_node = 0,
thread_id = 12, nodes = 2, thread_num = 32, numa_node = 0,
thread_id = 13, nodes = 2, thread_num = 32, numa_node = 0,
thread_id = 14, nodes = 2, thread_num = 32, numa_node = 0,
thread_id = 15, nodes = 2, thread_num = 32, numa_node = 0,
thread_id = 16, nodes = 2, thread_num = 32, numa_node = 1,
thread_id = 17, nodes = 2, thread_num = 32, numa_node = 1,
thread_id = 18, nodes = 2, thread_num = 32, numa_node = 1,
thread_id = 19, nodes = 2, thread_num = 32, numa_node = 1,
thread_id = 2, nodes = 2, thread_num = 32, numa_node = 0,
thread_id = 20, nodes = 2, thread_num = 32, numa_node = 1,
thread_id = 21, nodes = 2, thread_num = 32, numa_node = 1,
thread_id = 22, nodes = 2, thread_num = 32, numa_node = 1,
thread_id = 23, nodes = 2, thread_num = 32, numa_node = 1,
thread_id = 24, nodes = 2, thread_num = 32, numa_node = 1,
thread_id = 25, nodes = 2, thread_num = 32, numa_node = 1,
thread_id = 26, nodes = 2, thread_num = 32, numa_node = 1,
thread_id = 27, nodes = 2, thread_num = 32, numa_node = 1,
thread_id = 28, nodes = 2, thread_num = 32, numa_node = 1,
thread_id = 29, nodes = 2, thread_num = 32, numa_node = 1,
thread_id = 3, nodes = 2, thread_num = 32, numa_node = 0,
thread_id = 30, nodes = 2, thread_num = 32, numa_node = 1,
thread_id = 31, nodes = 2, thread_num = 32, numa_node = 1,
thread_id = 32, nodes = 2, thread_num = 40, numa_node = 1,
thread_id = 33, nodes = 2, thread_num = 40, numa_node = 1,
thread_id = 34, nodes = 2, thread_num = 40, numa_node = 1,
thread_id = 35, nodes = 2, thread_num = 40, numa_node = 1,
thread_id = 36, nodes = 2, thread_num = 40, numa_node = 1,
thread_id = 37, nodes = 2, thread_num = 40, numa_node = 1,
thread_id = 38, nodes = 2, thread_num = 40, numa_node = 1,
thread_id = 39, nodes = 2, thread_num = 40, numa_node = 1,
thread_id = 4, nodes = 2, thread_num = 32, numa_node = 0,
thread_id = 5, nodes = 2, thread_num = 32, numa_node = 0,
thread_id = 6, nodes = 2, thread_num = 32, numa_node = 0,
thread_id = 7, nodes = 2, thread_num = 32, numa_node = 0,
thread_id = 8, nodes = 2, thread_num = 32, numa_node = 0,
thread_id = 9, nodes = 2, thread_num = 32, numa_node = 0,

很显然thread_num发生了变化，最终导致numa_node = 0更少。

推测是do_work_stealing_job被使用了多次，导致一部分初始化时使用了32一部分使用了40。

问题修复

直接使用传入的线程数初始化thread_num，而不进行min运算。
为了尽可能减少影响，这里加上了USE_NUMA宏判断，开启USE_NUMA才走直接赋值逻辑。

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/ktransformers/pull/685 **Author:** [@wkgcass](https://github.com/wkgcass) **Created:** 2/26/2025 **Status:** ✅ Merged **Merged:** 2/26/2025 **Merged by:** [@chenht2022](https://github.com/chenht2022) **Base:** `main` ← **Head:** `main` --- ### 📝 Commits (1) - [`b2bff17`](https://github.com/kvcache-ai/ktransformers/commit/b2bff1777502519838b8a5ec112d2566635d42e5) fix numa cpu distribution ### 📊 Changes **1 file changed** (+6 additions, -1 deletions) <details> <summary>View changed files</summary> 📝 `ktransformers/ktransformers_ext/cpu_backend/backend.cpp` (+6 -1) </details> ### 📄 Description ## 问题描述我的平台是双socket，每个socket24核（关闭超线程）。当我配置`cpu_infer=41`时，性能较高，而配置`cpu_infer=42`时，性能骤降到`1token/s`。 `htop`观察发现socket1上的cpu被全部占满，而socket1上有8个空闲的cpu。也就是说numa cpu分布不均匀。推测配置`42`时性能骤降的原因是：socket1上的worker数多于cpu核数。 <img width="972" alt="452e53bc3cd0b37dd36dbd27721af6a6" src="https://github.com/user-attachments/assets/3392b6b9-050b-4314-a23c-68ce1c9b6172" /> ## 排查过程我注意到numa bind位于backend.cpp的`Backend::process_tasks`里，numa_node会根据如下方式计算： ``` numa_node = thread_id * numa_num_configured_nodes() / thread_num_; ``` 按理说应该是均匀的才对。但在进行一番print debug后，我发现thread_num_会发生变化：如下是配置`cpu_infer=41`时的sort后的输出： ``` thread_id = 0, nodes = 2, thread_num = 32, numa_node = 0, thread_id = 1, nodes = 2, thread_num = 32, numa_node = 0, thread_id = 10, nodes = 2, thread_num = 32, numa_node = 0, thread_id = 11, nodes = 2, thread_num = 32, numa_node = 0, thread_id = 12, nodes = 2, thread_num = 32, numa_node = 0, thread_id = 13, nodes = 2, thread_num = 32, numa_node = 0, thread_id = 14, nodes = 2, thread_num = 32, numa_node = 0, thread_id = 15, nodes = 2, thread_num = 32, numa_node = 0, thread_id = 16, nodes = 2, thread_num = 32, numa_node = 1, thread_id = 17, nodes = 2, thread_num = 32, numa_node = 1, thread_id = 18, nodes = 2, thread_num = 32, numa_node = 1, thread_id = 19, nodes = 2, thread_num = 32, numa_node = 1, thread_id = 2, nodes = 2, thread_num = 32, numa_node = 0, thread_id = 20, nodes = 2, thread_num = 32, numa_node = 1, thread_id = 21, nodes = 2, thread_num = 32, numa_node = 1, thread_id = 22, nodes = 2, thread_num = 32, numa_node = 1, thread_id = 23, nodes = 2, thread_num = 32, numa_node = 1, thread_id = 24, nodes = 2, thread_num = 32, numa_node = 1, thread_id = 25, nodes = 2, thread_num = 32, numa_node = 1, thread_id = 26, nodes = 2, thread_num = 32, numa_node = 1, thread_id = 27, nodes = 2, thread_num = 32, numa_node = 1, thread_id = 28, nodes = 2, thread_num = 32, numa_node = 1, thread_id = 29, nodes = 2, thread_num = 32, numa_node = 1, thread_id = 3, nodes = 2, thread_num = 32, numa_node = 0, thread_id = 30, nodes = 2, thread_num = 32, numa_node = 1, thread_id = 31, nodes = 2, thread_num = 32, numa_node = 1, thread_id = 32, nodes = 2, thread_num = 40, numa_node = 1, thread_id = 33, nodes = 2, thread_num = 40, numa_node = 1, thread_id = 34, nodes = 2, thread_num = 40, numa_node = 1, thread_id = 35, nodes = 2, thread_num = 40, numa_node = 1, thread_id = 36, nodes = 2, thread_num = 40, numa_node = 1, thread_id = 37, nodes = 2, thread_num = 40, numa_node = 1, thread_id = 38, nodes = 2, thread_num = 40, numa_node = 1, thread_id = 39, nodes = 2, thread_num = 40, numa_node = 1, thread_id = 4, nodes = 2, thread_num = 32, numa_node = 0, thread_id = 5, nodes = 2, thread_num = 32, numa_node = 0, thread_id = 6, nodes = 2, thread_num = 32, numa_node = 0, thread_id = 7, nodes = 2, thread_num = 32, numa_node = 0, thread_id = 8, nodes = 2, thread_num = 32, numa_node = 0, thread_id = 9, nodes = 2, thread_num = 32, numa_node = 0, ``` 很显然thread_num发生了变化，最终导致`numa_node = 0`更少。推测是`do_work_stealing_job`被使用了多次，导致一部分初始化时使用了32一部分使用了40。 ## 问题修复直接使用传入的线程数初始化thread_num，而不进行min运算。为了尽可能减少影响，这里加上了USE_NUMA宏判断，开启USE_NUMA才走直接赋值逻辑。 --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-29 14:52:37 +00:00