kvcache-ai/ktransformers

Fork 0

mirror of https://github.com/kvcache-ai/ktransformers.git synced 2026-11-05 01:07:28 +00:00

[GH-ISSUE #554] 如何完整编译 #404

New issue

Closed

opened 2026-10-29 14:39:09 +00:00 by bsakaguchi · 6 comments

bsakaguchi commented

2026-10-29 14:39:09 +00:00

Owner

Originally created by @cnsdzbhm on GitHub (Feb 21, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/554

我没有avx2，pip dev install . 不能安装，请给个整体编译的说明或创建一个完整的makefile，就像llama.cpp或linux或openwrt一样，编译环境可以宽松些，毕竟编译和运行不是一个概念。有了这完整的编译环境将来就有可能会运行在不同的边缘设备上。

Originally created by @cnsdzbhm on GitHub (Feb 21, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/554 我没有avx2，pip dev install . 不能安装，请给个整体编译的说明或创建一个完整的makefile，就像llama.cpp或linux或openwrt一样，编译环境可以宽松些，毕竟编译和运行不是一个概念。有了这完整的编译环境将来就有可能会运行在不同的边缘设备上。

bsakaguchi

2026-10-29 14:39:09 +00:00

closed this issue
added the
stale
label

bsakaguchi commented

2026-10-29 14:39:11 +00:00

Author

Owner

@fungaren commented on GitHub (Feb 21, 2025):

mkdir kt
cd kt
git clone --branch v0.2.1.post1 https://github.com/kvcache-ai/ktransformers.git
cd ktransformers
git submodule init && git submodule update
curl -LO https://github.com/kvcache-ai/ktransformers/releases/download/v0.2.1.post1/ktransformers-0.2.1.post1+cu121torch23avx512-cp310-cp310-linux_x86_64.whl
curl -LO https://github.com/Dao-AILab/flash-attention/releases/download/v2.7.4.post1/flash_attn-2.7.4.post1+cu12torch2.3cxx11abiFALSE-cp310-cp310-linux_x86_64.whl
cd -
cat > Dockerfile <<EOF
FROM nvcr.io/nvidia/cuda:12.4.1-base-ubuntu22.04
RUN sed -i 's|archive.ubuntu.com|mirrors.volces.com|g' /etc/apt/sources.list && \
    apt-get update && apt-get install -y git python3 python3-pip build-essential && \
    apt clean && rm -f /var/lib/apt/lists/*_*
ADD ktransformers/ /ktransformers
WORKDIR /ktransformers
ENV USE_NUMA=1
RUN pip install --no-cache-dir -i http://mirrors.volces.com/pypi/simple \
    --trusted-host=mirrors.volces.com ktransformers-0.2.1.post1+cu121torch23avx512-cp310-cp310-linux_x86_64.whl \
    flash_attn-2.7.4.post1+cu12torch2.3cxx11abiFALSE-cp310-cp310-linux_x86_64.whl
EOF
docker build -t ktransformers:v0.2.1-cuda-12.4 .

@fungaren commented on GitHub (Feb 21, 2025): ```bash mkdir kt cd kt git clone --branch v0.2.1.post1 https://github.com/kvcache-ai/ktransformers.git cd ktransformers git submodule init && git submodule update curl -LO https://github.com/kvcache-ai/ktransformers/releases/download/v0.2.1.post1/ktransformers-0.2.1.post1+cu121torch23avx512-cp310-cp310-linux_x86_64.whl curl -LO https://github.com/Dao-AILab/flash-attention/releases/download/v2.7.4.post1/flash_attn-2.7.4.post1+cu12torch2.3cxx11abiFALSE-cp310-cp310-linux_x86_64.whl cd - cat > Dockerfile <<EOF FROM nvcr.io/nvidia/cuda:12.4.1-base-ubuntu22.04 RUN sed -i 's|archive.ubuntu.com|mirrors.volces.com|g' /etc/apt/sources.list && \ apt-get update && apt-get install -y git python3 python3-pip build-essential && \ apt clean && rm -f /var/lib/apt/lists/*_* ADD ktransformers/ /ktransformers WORKDIR /ktransformers ENV USE_NUMA=1 RUN pip install --no-cache-dir -i http://mirrors.volces.com/pypi/simple \ --trusted-host=mirrors.volces.com ktransformers-0.2.1.post1+cu121torch23avx512-cp310-cp310-linux_x86_64.whl \ flash_attn-2.7.4.post1+cu12torch2.3cxx11abiFALSE-cp310-cp310-linux_x86_64.whl EOF docker build -t ktransformers:v0.2.1-cuda-12.4 . ```

bsakaguchi commented

2026-10-29 14:39:11 +00:00

Author

Owner

@cnsdzbhm commented on GitHub (Feb 23, 2025):

谢谢您的答复。我想知道ktransformers-0.2.1.post1+cu121torch23avx512-cp310-cp310-linux_x86_64.whl是通过那几个开关编译出来的，我没有AVX2,我想编译个ktransformers-0.2.1.post1+cu121torch23avx-cp310-cp310-linux_x86_64.whl，这个avx whl可以调整哪几个编译开关编译出来。能讲一下ktransformers-0.2.1.post1+cu121torch23avx512-cp310-cp310-linux_x86_64.whl完整的编译过程？

@cnsdzbhm commented on GitHub (Feb 23, 2025): 谢谢您的答复。我想知道ktransformers-0.2.1.post1+cu121torch23avx512-cp310-cp310-linux_x86_64.whl是通过那几个开关编译出来的，我没有AVX2,我想编译个ktransformers-0.2.1.post1+cu121torch23avx-cp310-cp310-linux_x86_64.whl，这个avx whl可以调整哪几个编译开关编译出来。能讲一下ktransformers-0.2.1.post1+cu121torch23avx512-cp310-cp310-linux_x86_64.whl完整的编译过程？

bsakaguchi commented

2026-10-29 14:39:12 +00:00

Author

Owner

@fungaren commented on GitHub (Feb 24, 2025):

依赖一个 cpufeature 的库，搜一下这个关键字看下源码。另外，目前不支持 ARM

https://github.com/kvcache-ai/ktransformers/issues/447

@fungaren commented on GitHub (Feb 24, 2025): 依赖一个 cpufeature 的库，搜一下这个关键字看下源码。另外，目前不支持 ARM https://github.com/kvcache-ai/ktransformers/issues/447

bsakaguchi commented

2026-10-29 14:39:12 +00:00

Author

Owner

@cnsdzbhm commented on GitHub (Feb 28, 2025):

关于avx cpu指令集的问题，在 ./ktransformers/ktransformers/ktransformers_ext/CMakeLists.txt 有option(LLAMA_AVX "llama: enable AVX" OFF) 开关，打开这个开关能行？或者说如何打开这个开关。./ktransformers/setup.py, class CpuInstructInfo: 函数没有AVX的实现，在这里或者后续要做什么工作。

@cnsdzbhm commented on GitHub (Feb 28, 2025): 关于avx cpu指令集的问题，在 ./ktransformers/ktransformers/ktransformers_ext/CMakeLists.txt 有option(LLAMA_AVX "llama: enable AVX" OFF) 开关，打开这个开关能行？或者说如何打开这个开关。./ktransformers/setup.py, class CpuInstructInfo: 函数没有AVX的实现，在这里或者后续要做什么工作。

bsakaguchi commented

2026-10-29 14:39:12 +00:00

Author

Owner

@ErvinXie commented on GitHub (Dec 11, 2025):

This issue has been automatically closed due to inactivity for 60 days.

If you believe this issue is still relevant, please feel free to reopen it with additional information or context.

Thank you for your contribution to KTransformers!

此 issue 因超过 60 天无活动而被自动关闭。

如果您认为此问题仍然存在，请随时重新打开并提供更多信息或上下文。

感谢您对 KTransformers 的贡献！

@ErvinXie commented on GitHub (Dec 11, 2025): This issue has been automatically closed due to inactivity for 60 days. If you believe this issue is still relevant, please feel free to reopen it with additional information or context. Thank you for your contribution to KTransformers! --- 此 issue 因超过 60 天无活动而被自动关闭。如果您认为此问题仍然存在，请随时重新打开并提供更多信息或上下文。感谢您对 KTransformers 的贡献！

bsakaguchi commented

2026-10-29 14:39:13 +00:00

Author

Owner

@ErvinXie commented on GitHub (Dec 11, 2025):

This issue has been automatically closed due to inactivity for 60 days.

If you believe this issue is still relevant, please feel free to reopen it with additional information or context.

Thank you for your contribution to KTransformers!

此 issue 因超过 60 天无活动而被自动关闭。

如果您认为此问题仍然存在，请随时重新打开并提供更多信息或上下文。

感谢您对 KTransformers 的贡献！

@ErvinXie commented on GitHub (Dec 11, 2025): This issue has been automatically closed due to inactivity for 60 days. If you believe this issue is still relevant, please feel free to reopen it with additional information or context. Thank you for your contribution to KTransformers! --- 此 issue 因超过 60 天无活动而被自动关闭。如果您认为此问题仍然存在，请随时重新打开并提供更多信息或上下文。感谢您对 KTransformers 的贡献！