[GH-ISSUE #1203] [Feature] Support for Intel Xeon Max where it has 64gb hbm memory to be used in kv offload #825

New issue

Open

opened 2026-10-29 14:45:01 +00:00 by bsakaguchi · 5 comments

bsakaguchi commented

2026-10-29 14:45:01 +00:00

Owner

Originally created by @sprappcom on GitHub (Apr 26, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1203

Checklist

1. If the issue you raised is not a feature but a question, please raise a discussion at https://github.com/kvcache-ai/ktransformers/discussions. Otherwise, it will be closed.
2. To help the community, I will use Chinese/English or attach an Chinese/English translation if using another language. Non-English/Chinese content without translation may be closed.

Motivation

possible to support intel xeon max? 64gb hbm kv offload

No response

Originally created by @sprappcom on GitHub (Apr 26, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1203 ### Checklist - [ ] 1. If the issue you raised is not a feature but a question, please raise a discussion at https://github.com/kvcache-ai/ktransformers/discussions. Otherwise, it will be closed. - [ ] 2. To help the community, I will use Chinese/English or attach an Chinese/English translation if using another language. Non-English/Chinese content without translation may be closed. ### Motivation possible to support intel xeon max? 64gb hbm kv offload ### Related resources _No response_

bsakaguchi commented

2026-10-29 14:45:01 +00:00

Author

Owner

@Atream commented on GitHub (Apr 29, 2025):

64GB of HBM is still a bit small. We need to research how to better utilize this memory. However, we don't currently have access to a Xeon Max machine, so we may not add related optimizations in the near future.

@Atream commented on GitHub (Apr 29, 2025): 64GB of HBM is still a bit small. We need to research how to better utilize this memory. However, we don't currently have access to a Xeon Max machine, so we may not add related optimizations in the near future.

bsakaguchi commented

2026-10-29 14:45:01 +00:00

Author

Owner

@casazaft commented on GitHub (Apr 29, 2025):

我感觉不太现实，这个hbm内存是给cpu使用的，kv缓存是在gpu上MLA需要，二者通信用pcie带宽，估计加速不了。

@casazaft commented on GitHub (Apr 29, 2025): 我感觉不太现实，这个hbm内存是给cpu使用的，kv缓存是在gpu上MLA需要，二者通信用pcie带宽，估计加速不了。

bsakaguchi commented

2026-10-29 14:45:02 +00:00

Author

Owner

@casazaft commented on GitHub (Apr 29, 2025):

I think it is not very realistic. The hbm memory is used for the cpu, and the kv cache is needed for MLA on the gpu. The two communicate using pcie bandwidth, so it is estimated that it cannot be accelerated.

@casazaft commented on GitHub (Apr 29, 2025): I think it is not very realistic. The hbm memory is used for the cpu, and the kv cache is needed for MLA on the gpu. The two communicate using pcie bandwidth, so it is estimated that it cannot be accelerated.

bsakaguchi commented

2026-10-29 14:45:02 +00:00

Author

Owner

@sprappcom commented on GitHub (May 4, 2025):

@casazaft @Atream purely without GPU to offload to the HBM as "GPU" vram.
it's 1.5tb/s hbm memory and fast enough to replace 4090 with more ram (64GB).

possible to do so?

@sprappcom commented on GitHub (May 4, 2025): @casazaft @Atream purely without GPU to offload to the HBM as "GPU" vram. it's 1.5tb/s hbm memory and fast enough to replace 4090 with more ram (64GB). possible to do so?

bsakaguchi commented

2026-10-29 14:45:02 +00:00

Author

Owner

@casazaft commented on GitHub (May 5, 2025):

intel cpu就算启用amx，矩阵算力也是比不上GPU的，别说4090，3090都比不上。MLA的计算量很大，都移动到CPU上算，速度很慢。

@casazaft commented on GitHub (May 5, 2025): intel cpu就算启用amx，矩阵算力也是比不上GPU的，别说4090，3090都比不上。MLA的计算量很大，都移动到CPU上算，速度很慢。