kvcache-ai/ktransformers

Fork 0

mirror of https://github.com/kvcache-ai/ktransformers.git synced 2026-11-05 01:07:28 +00:00

[GH-ISSUE #1818] Convert MiniMax M2.1 or DeepSeek V3.2 to CPU weight failed #1137

New issue

Closed

opened 2026-10-29 14:49:08 +00:00 by bsakaguchi · 4 comments

bsakaguchi commented

2026-10-29 14:49:08 +00:00

Owner

Originally created by @merries711 on GitHub (Jan 28, 2026).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1818

Reminder

I have read the above rules and searched the existing issues.

System Info

------- System Info --------
CPU: INTEL(R) XEON(R) GOLD 6530 x2
RAM: 1T DDR5
GPU: NVIDIA GeForce RTX 3090 24G

------- KTransformers CLI v0.1.0 -------
Python 3.13.11
平台 Debian 13.3 ( Linux 6.12.63+deb13-amd64）
CUDA 13.1
------- Packages: -------
kt-kernel 0.5.1
sglang 0.5.8 (PyPI)

installed from source .

Reproduction

convert command :

$ python scripts/convert_cpu_weights.py --input-path $HOME/MiniMax/MiniMax-M2.1/ --input-type bf16 --output $HOME/MiniMax/MiniMax-M2.1-INT8 --quant-method int8

Loading model configuration...
Error during conversion: Missing required config fields: ['num_experts']
Traceback (most recent call last):
  File "/home/u01/ktransformers/kt-kernel/scripts/convert_cpu_weights.py", line 974, in main
    model_config = load_model_config(args.input_path, args.input_type)
  File "/home/u01/ktransformers/kt-kernel/scripts/convert_cpu_weights.py", line 92, in load_model_config
    raise ValueError(f"Missing required config fields: {missing_fields}")
ValueError: Missing required config fields: ['num_experts']

by manually modifying config.json file，add "num_experts: 256" :

Total tensors loaded: 96103
Starting conversion...
Input: /home/u01/MiniMax/MiniMax-M2.1/
Output: /home/u01/MiniMax/MiniMax-M2.1-INT8
Found 0 layers with MoE experts:
No MoE expert layers found in input!

but in the original config.json file exists "num_local_experts : 256," , can the script use the it ?
https://huggingface.co/MiniMaxAI/MiniMax-M2.1/resolve/main/config.json

Others

No response

Originally created by @merries711 on GitHub (Jan 28, 2026). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1818 ### Reminder - [x] I have read the above rules and searched the existing issues. ### System Info ------- System Info -------- CPU: INTEL(R) XEON(R) GOLD 6530 x2 RAM: 1T DDR5 GPU: NVIDIA GeForce RTX 3090 24G ------- KTransformers CLI v0.1.0 ------- Python 3.13.11 平台 Debian 13.3 ( Linux 6.12.63+deb13-amd64） CUDA 13.1 ------- Packages: ------- kt-kernel 0.5.1 sglang 0.5.8 (PyPI) installed from source . ### Reproduction convert command : ```shell $ python scripts/convert_cpu_weights.py --input-path $HOME/MiniMax/MiniMax-M2.1/ --input-type bf16 --output $HOME/MiniMax/MiniMax-M2.1-INT8 --quant-method int8 Loading model configuration... Error during conversion: Missing required config fields: ['num_experts'] Traceback (most recent call last): File "/home/u01/ktransformers/kt-kernel/scripts/convert_cpu_weights.py", line 974, in main model_config = load_model_config(args.input_path, args.input_type) File "/home/u01/ktransformers/kt-kernel/scripts/convert_cpu_weights.py", line 92, in load_model_config raise ValueError(f"Missing required config fields: {missing_fields}") ValueError: Missing required config fields: ['num_experts'] ``` by manually modifying config.json file，add "num_experts: 256" : ```shell Total tensors loaded: 96103 Starting conversion... Input: /home/u01/MiniMax/MiniMax-M2.1/ Output: /home/u01/MiniMax/MiniMax-M2.1-INT8 Found 0 layers with MoE experts: No MoE expert layers found in input! ``` **but in the original config.json file exists "num_local_experts : 256," , can the script use the it ?** [https://huggingface.co/MiniMaxAI/MiniMax-M2.1/resolve/main/config.json](https://huggingface.co/MiniMaxAI/MiniMax-M2.1/resolve/main/config.json) ### Others _No response_

bsakaguchi closed this issue

2026-10-29 14:49:08 +00:00

bsakaguchi commented

2026-10-29 14:49:09 +00:00

Author

Owner

@SkqLiao commented on GitHub (Jan 28, 2026):

Hi, MiniMax M2.1's weights are already in FP8 format, so you can run FP8 inference directly without converting the weights. See the official tutorial for details.
Additionally, MiniMax’s weight format differs from DeepSeek’s, and the current conversion scripts do not support it—so we recommend using native FP8 inference instead of attempting to convert the weights.

@SkqLiao commented on GitHub (Jan 28, 2026): Hi, MiniMax M2.1's weights are already in FP8 format, so you can run FP8 inference directly without converting the weights. See the official [tutorial](https://github.com/kvcache-ai/ktransformers/blob/main/doc/en/kt-kernel/MiniMax-M2.1-Tutorial.md) for details. Additionally, MiniMax’s weight format differs from DeepSeek’s, and the current conversion scripts do not support it—so we recommend using native FP8 inference instead of attempting to convert the weights.

bsakaguchi commented

2026-10-29 14:49:09 +00:00

Author

Owner

@CodeZ-Hao commented on GitHub (Jan 29, 2026):

Hi, MiniMax M2.1's weights are already in FP8 format, so you can run FP8 inference directly without converting the weights. See the official tutorial for details. Additionally, MiniMax’s weight format differs from DeepSeek’s, and the current conversion scripts do not support it—so we recommend using native FP8 inference instead of attempting to convert the weights.

@SkqLiao I tried MiniMax M2.1 in FP8 with 4080, it's prefill has 700+ tokens, it work well, but it tooks more gpu memory than AMX. In this model size, amx performance would more than 500+ tps prefill, it can be an alternative in less gpu mem time.
And in this case, 3090 has not support FP8, so the amx support is the only chooise.

@CodeZ-Hao commented on GitHub (Jan 29, 2026): > Hi, MiniMax M2.1's weights are already in FP8 format, so you can run FP8 inference directly without converting the weights. See the official [tutorial](https://github.com/kvcache-ai/ktransformers/blob/main/doc/en/kt-kernel/MiniMax-M2.1-Tutorial.md) for details. Additionally, MiniMax’s weight format differs from DeepSeek’s, and the current conversion scripts do not support it—so we recommend using native FP8 inference instead of attempting to convert the weights. @SkqLiao I tried MiniMax M2.1 in FP8 with 4080, it's prefill has 700+ tokens, it work well, but it tooks more gpu memory than AMX. In this model size, amx performance would more than 500+ tps prefill, it can be an alternative in less gpu mem time. And in this case, 3090 has not support FP8, so the amx support is the only chooise.

bsakaguchi commented

2026-10-29 14:49:09 +00:00

Author

Owner

@merries711 commented on GitHub (Feb 2, 2026):

triton.compiler.errors.CompilationError: at 1:0:
def fused_moe_kernel(
^
ValueError("type fp8e4nv not supported in this architecture. The supported fp8 dtypes are ('fp8e4b15', 'fp8e5')")

[2026-02-02 08:51:10] Received sigquit from a child process. It usually means the child failed.

3090 do not support FP8 ?

@merries711 commented on GitHub (Feb 2, 2026): ```python triton.compiler.errors.CompilationError: at 1:0: def fused_moe_kernel( ^ ValueError("type fp8e4nv not supported in this architecture. The supported fp8 dtypes are ('fp8e4b15', 'fp8e5')") [2026-02-02 08:51:10] Received sigquit from a child process. It usually means the child failed. ``` 3090 do not support FP8 ?

bsakaguchi commented

2026-10-29 14:49:09 +00:00

Author

Owner

@ErvinXie commented on GitHub (Feb 28, 2026):

MiniMax M2.1 weights are already in FP8 format and can be used directly without conversion. See the reply from SkqLiao above. Closing.

@ErvinXie commented on GitHub (Feb 28, 2026): MiniMax M2.1 weights are already in FP8 format and can be used directly without conversion. See the reply from SkqLiao above. Closing.

bsakaguchi referenced this issue

2026-10-29 14:53:34 +00:00

[PR #1137] [CLOSED] Refactor the chat interface to support tool calling and parameter pro… #1569