[GH-ISSUE #1195 ] [Bug] 0.2.4Post1 Cannot Load Unsloth Dynamic v2.0 UD-Q4_K_XL #817

Owner

Originally created by @Ning18 on GitHub (Apr 25, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1195

Originally assigned to: @Azure-Tang on GitHub.

Checklist

1. I have searched related issues but cannot get the expected help.
2. The bug has not been fixed in the latest version.
3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.
4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/kvcache-ai/ktransformers/discussions. Otherwise, it will be closed.
5. To help the community, I will use Chinese/English or attach an Chinese/English translation if using another language. Non-Chinese/English content without translation may be closed.

Describe the bug

The Unsloth Dynamic v2.0 UD-Q4_K_XL released a day ago, downloaded from https://huggingface.co/unsloth/DeepSeek-R1-GGUF-UD/tree/main/UD-Q4_K_XL, cannot be loaded by 0.2.4Post1.
[Note]

Error message is
Exception: can't find blk.0.attn_kv_b.weight in GGUF file!
The system can run the old ggufs from unsloth/DeepSeek-R1-GGUF/tree/main/DeepSeek-R1-UD-Q2_K_XL successfully. Command line parameters are the same except for --gguf_path.
The downloaded ggufs are checked with sha256sum, which produced the same values as on the huggingface.

[log]
Injecting model.layers.60.mlp.shared_experts.down_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.60.mlp.shared_experts.act_fn as default
Injecting model.layers.60.input_layernorm as ktransformers.operators.layernorm . RMSNorm
Injecting model.layers.60.post_attention_layernorm as ktransformers.operators.layernorm . RMSNorm
Injecting model.norm as ktransformers.operators.layernorm . RMSNorm
Injecting cache as default
Injecting lm_head as ktransformers.operators.linear . KTransformersLinear
loading token_embd.weight to cpu
loading token_embd.weight with CPU
loading blk.0.attn_q_a_norm.weight to cuda
loading blk.0.attn_kv_a_norm.weight to cuda
Process SpawnProcess-1:
...
...
File "... .../ktransformers/util/utils.py", line 118, in load_cur_state_dict
raise Exception(f"can't find {translated_key} in GGUF file!")
Exception: can't find blk.0.attn_kv_b.weight in GGUF file!

Reproduction

#!/bin/bash

#gguf_path="/models/R1-Q2KXL"
gguf_path="/models/R1-UD2-Q4KXL"
optimize_config_path="ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat-serve.yaml"

python ktransformers/server/main.py
--port 10002
--model_name R1
--model_path /models/R1
--gguf_path "$gguf_path"
--optimize_config_path "$optimize_config_path"
--max_new_tokens 32000
--cache_lens 260000
--chunk_size 256
--max_batch_size 4
--backend_type balance_serve
--force_think
--log_file R1.log
--temperature 0.6

Environment

OS: ubuntu 22.04
GPU: RTX 5000ada 32GB
CPU: EPYC9654x1
DDR5 768GB

Originally created by @Ning18 on GitHub (Apr 25, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1195 Originally assigned to: @Azure-Tang on GitHub. ### Checklist - [x] 1. I have searched related issues but cannot get the expected help. - [x] 2. The bug has not been fixed in the latest version. - [x] 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback. - [x] 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/kvcache-ai/ktransformers/discussions. Otherwise, it will be closed. - [x] 5. To help the community, I will use Chinese/English or attach an Chinese/English translation if using another language. Non-Chinese/English content without translation may be closed. ### Describe the bug The Unsloth Dynamic v2.0 UD-Q4_K_XL released a day ago, downloaded from https://huggingface.co/unsloth/DeepSeek-R1-GGUF-UD/tree/main/UD-Q4_K_XL, cannot be loaded by 0.2.4Post1. [Note] 1. Error message is Exception: can't find blk.0.attn_kv_b.weight in GGUF file! 2. The system can run the old ggufs from unsloth/DeepSeek-R1-GGUF/tree/main/DeepSeek-R1-UD-Q2_K_XL successfully. Command line parameters are the same except for --gguf_path. 3. The downloaded ggufs are checked with sha256sum, which produced the same values as on the huggingface. [log] Injecting model.layers.60.mlp.shared_experts.down_proj as ktransformers.operators.linear . KTransformersLinear Injecting model.layers.60.mlp.shared_experts.act_fn as default Injecting model.layers.60.input_layernorm as ktransformers.operators.layernorm . RMSNorm Injecting model.layers.60.post_attention_layernorm as ktransformers.operators.layernorm . RMSNorm Injecting model.norm as ktransformers.operators.layernorm . RMSNorm Injecting cache as default Injecting lm_head as ktransformers.operators.linear . KTransformersLinear loading token_embd.weight to cpu loading token_embd.weight with CPU loading blk.0.attn_q_a_norm.weight to cuda loading blk.0.attn_kv_a_norm.weight to cuda Process SpawnProcess-1: ... ... File "... .../ktransformers/util/utils.py", line 118, in load_cur_state_dict raise Exception(f"can't find {translated_key} in GGUF file!") Exception: can't find blk.0.attn_kv_b.weight in GGUF file! ### Reproduction #!/bin/bash #gguf_path="/models/R1-Q2KXL" gguf_path="/models/R1-UD2-Q4KXL" optimize_config_path="ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat-serve.yaml" python ktransformers/server/main.py \ --port 10002 \ --model_name R1 \ --model_path /models/R1 \ --gguf_path "$gguf_path" \ --optimize_config_path "$optimize_config_path"\ --max_new_tokens 32000 \ --cache_lens 260000 \ --chunk_size 256 \ --max_batch_size 4 \ --backend_type balance_serve \ --force_think \ --log_file R1.log \ --temperature 0.6 ### Environment OS: ubuntu 22.04 GPU: RTX 5000ada 32GB CPU: EPYC9654x1 DDR5 768GB

bsakaguchi closed this issue

bsakaguchi commented

Author

Owner

@Azure-Tang commented on GitHub (Apr 25, 2025):

Can you check gguf sha256sum is corresponding with huggingface.

@Azure-Tang commented on GitHub (Apr 25, 2025): Can you check gguf sha256sum is corresponding with huggingface.

bsakaguchi commented

Author

Owner

@Ning18 commented on GitHub (Apr 25, 2025):

Can you check gguf sha256sum is corresponding with huggingface.

Yes, as I said in the [Note] section of the original post. I checked sha256sum of all the ggufs, and produced the same values as huggingface's.

@Ning18 commented on GitHub (Apr 25, 2025): > Can you check gguf sha256sum is corresponding with huggingface. Yes, as I said in the [Note] section of the original post. I checked sha256sum of all the ggufs, and produced the same values as huggingface's.

bsakaguchi commented

Author

Owner

@Ning18 commented on GitHub (Apr 25, 2025):

I think I have found the direct reason for the problem. The new UD 2.0 ggufs changed its weight's name and tensor size.
Just as the error message suggests: "Exception: can't find blk.0.attn_kv_b.weight in GGUF file!"

[new gguf, cannot load]

blk.0.attn_k_b.weight [128, 512, 128]
blk.0.attn_v_b.weight [512, 128, 128]

[old, can load]

blk.0.attn_kv_b.weight [512, 32 768]

They removed blk.0.attn_kv_b.weight, and added blk.0.attn_k_b.weight and blk.0.attn_v_b.weight.
Can the problem be overcome by modifying optimize_config file?

@Ning18 commented on GitHub (Apr 25, 2025): I think I have found the direct reason for the problem. The new UD 2.0 ggufs changed its weight's name and tensor size. Just as the error message suggests: "Exception: can't find blk.0.attn_kv_b.weight in GGUF file!" [new gguf, cannot load] - blk.0.attn_k_b.weight [128, 512, 128] - blk.0.attn_v_b.weight [512, 128, 128] [old, can load] - blk.0.attn_kv_b.weight [512, 32 768] They removed blk.0.attn_kv_b.weight, and added blk.0.attn_k_b.weight and blk.0.attn_v_b.weight. Can the problem be overcome by modifying optimize_config file?

bsakaguchi commented

Author

Owner

@Azure-Tang commented on GitHub (Apr 26, 2025):

No, I think we need to adjust the logic for how we process attn_kv_b. In our implementation, part of attn_kv_b is merged into o_projection during the inference stage to achieve better performance.

I will work on finding a solution that can support both formats.

@Azure-Tang commented on GitHub (Apr 26, 2025): No, I think we need to adjust the logic for how we process attn_kv_b. In our implementation, part of attn_kv_b is merged into o_projection during the inference stage to achieve better performance. I will work on finding a solution that can support both formats.

bsakaguchi commented

Author

Owner

@Azure-Tang commented on GitHub (Apr 26, 2025):

If you're eager to try the latest Unsloth, I suggest modifying the code as follows:

Update [this line](github.com/kvcache-ai/ktransformers@a7b995365e/ktransformers/models/modeling_deepseek_v3.py (L676)) by replacing self.kv_b_proj with:

self.attn_k_b = nn.Linear(
    config.kv_lora_rank,
    self.num_heads * (self.q_head_dim - self.qk_rope_head_dim),
    bias=False,
)
self.attn_v_b = nn.Linear(
    config.kv_lora_rank,
    self.num_heads * self.v_head_dim,
    bias=False,
)

Disable the absorb function [here](github.com/kvcache-ai/ktransformers@a7b995365e/ktransformers/operators/attention.py (L405)).
Then, replace the two matmul operations related to q_absorb and out_absorb with self.attn_k_b and self.attn_v_b respectively.

I believe this approach should work.

@Azure-Tang commented on GitHub (Apr 26, 2025): If you're eager to try the latest Unsloth, I suggest modifying the code as follows: 1. Update [[this line](https://github.com/kvcache-ai/ktransformers/blob/a7b995365ef477bc77d7e5588a0d4e988404eec4/ktransformers/models/modeling_deepseek_v3.py#L676)](https://github.com/kvcache-ai/ktransformers/blob/a7b995365ef477bc77d7e5588a0d4e988404eec4/ktransformers/models/modeling_deepseek_v3.py#L676) by replacing `self.kv_b_proj` with: ```python self.attn_k_b = nn.Linear( config.kv_lora_rank, self.num_heads * (self.q_head_dim - self.qk_rope_head_dim), bias=False, ) self.attn_v_b = nn.Linear( config.kv_lora_rank, self.num_heads * self.v_head_dim, bias=False, ) ``` 2. Disable the absorb function [[here](https://github.com/kvcache-ai/ktransformers/blob/a7b995365ef477bc77d7e5588a0d4e988404eec4/ktransformers/operators/attention.py#L405)](https://github.com/kvcache-ai/ktransformers/blob/a7b995365ef477bc77d7e5588a0d4e988404eec4/ktransformers/operators/attention.py#L405). Then, replace the two matmul operations related to `q_absorb` and `out_absorb` with `self.attn_k_b` and `self.attn_v_b` respectively. I believe this approach should work.

bsakaguchi commented

Author

Owner

@Ning18 commented on GitHub (Apr 26, 2025):

Thanks for the instant response! I'll have a try during the May-Day holiday.

@Ning18 commented on GitHub (Apr 26, 2025): Thanks for the instant response! I'll have a try during the May-Day holiday.

bsakaguchi commented

Author

Owner

@adonispujols commented on GitHub (May 6, 2025):

Fixing this would be greatly appreciated!

@adonispujols commented on GitHub (May 6, 2025): Fixing this would be greatly appreciated!

bsakaguchi commented

Author

Owner

@ltcs11 commented on GitHub (May 22, 2025):

still have some problems with：

    raise FileNotFoundError(f"Weight file not found for key {key}")
FileNotFoundError: Weight file not found for key blk.0.self_attn.attn_k_b

or can we specific a llama.cpp commit id that create old style attn_kv_b

i found is 06bb53ad9b6e6d92b6ab6979927530080b1c990c (before 25.04.15 this commit github.com/ggml-org/llama.cpp@daa422881a)

@ltcs11 commented on GitHub (May 22, 2025): still have some problems with： ``` raise FileNotFoundError(f"Weight file not found for key {key}") FileNotFoundError: Weight file not found for key blk.0.self_attn.attn_k_b ``` or can we specific a llama.cpp commit id that create old style attn_kv_b i found is 06bb53ad9b6e6d92b6ab6979927530080b1c990c (before 25.04.15 this commit https://github.com/ggml-org/llama.cpp/commit/daa422881a0ec7944771bcc8ff8de34d11f5bd3b)

bsakaguchi commented

Author

Owner

@trilog-inc commented on GitHub (May 29, 2025):

Hi ,

I am running into this problem with the new R1-0528 quants from unsloth. Tried the fix mentionned above but didn't work. Has anyone gotten it to work with the new attn_kv split?

@trilog-inc commented on GitHub (May 29, 2025): Hi , I am running into this problem with the new R1-0528 quants from unsloth. Tried the fix mentionned above but didn't work. Has anyone gotten it to work with the new attn_kv split?

bsakaguchi commented

Author

Owner

@Martossien commented on GitHub (May 29, 2025):

Same here with R1-0528 ...

@Martossien commented on GitHub (May 29, 2025): Same here with R1-0528 ...

bsakaguchi commented

Author

Owner

@344303947 commented on GitHub (May 30, 2025):

Same here with R1-0528 ...
UD-Q2_K_XL
find blk.0.attn_kv_b.weight in GGUF

@344303947 commented on GitHub (May 30, 2025): Same here with R1-0528 ... UD-Q2_K_XL find blk.0.attn_kv_b.weight in GGUF

bsakaguchi commented

Author

Owner

@344303947 commented on GitHub (May 30, 2025):

loading blk.0.attn_q_a_norm.weight to cuda
loading blk.0.attn_kv_a_norm.weight to cuda
Process SpawnProcess-1:
Traceback (most recent call last):
File "/home/sean/miniconda3/envs/kt/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/home/sean/miniconda3/envs/kt/lib/python3.10/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/server/backend/interfaces/balance_serve.py", line 244, in run_engine
engine = Engine(args, token_queue, broadcast_endpoint)
File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/server/backend/interfaces/balance_serve.py", line 162, in init
optimize_and_load_gguf(self.model, optimize_config_path, gguf_path, config)
File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/optimize/optimize.py", line 131, in optimize_and_load_gguf
load_weights(module, gguf_loader)
File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/util/utils.py", line 125, in load_weights
load_weights(child, gguf_loader, prefix+name+".")
File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/util/utils.py", line 127, in load_weights
module.load()
File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/operators/base_operator.py", line 63, in load
utils.load_weights(child, self.gguf_loader, self.key+".")
File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/util/utils.py", line 125, in load_weights
load_weights(child, gguf_loader, prefix+name+".")
File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/util/utils.py", line 125, in load_weights
load_weights(child, gguf_loader, prefix+name+".")
File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/util/utils.py", line 125, in load_weights
load_weights(child, gguf_loader, prefix+name+".")
File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/util/utils.py", line 127, in load_weights
module.load()
File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/operators/base_operator.py", line 63, in load
utils.load_weights(child, self.gguf_loader, self.key+".")
File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/util/utils.py", line 125, in load_weights
load_weights(child, gguf_loader, prefix+name+".")
File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/util/utils.py", line 123, in load_weights
load_cur_state_dict(module, gguf_loader, prefix)
File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/util/utils.py", line 118, in load_cur_state_dict
raise Exception(f"can't find {translated_key} in GGUF file!")
Exception: can't find blk.0.attn_kv_b.weight in GGUF file!
^CTraceback (most recent call last):
File "/home/sean/ubuntu/ktransformers/ktransformers/server/main.py", line 145, in
main()
File "/home/sean/ubuntu/ktransformers/ktransformers/server/main.py", line 132, in main
create_interface(config=cfg, default_args=cfg)
File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/server/utils/create_interface.py", line 30, in create_interface
GlobalInterface.interface = BackendInterface(default_args)
File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/server/backend/interfaces/balance_serve.py", line 284, in init
start_event.wait()
File "/home/sean/miniconda3/envs/kt/lib/python3.10/multiprocessing/synchronize.py", line 349, in wait
self._cond.wait(timeout)
File "/home/sean/miniconda3/envs/kt/lib/python3.10/multiprocessing/synchronize.py", line 261, in wait
return self._wait_semaphore.acquire(True, timeout)
KeyboardInterrupt

@344303947 commented on GitHub (May 30, 2025): loading blk.0.attn_q_a_norm.weight to cuda loading blk.0.attn_kv_a_norm.weight to cuda Process SpawnProcess-1: Traceback (most recent call last): File "/home/sean/miniconda3/envs/kt/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap self.run() File "/home/sean/miniconda3/envs/kt/lib/python3.10/multiprocessing/process.py", line 108, in run self._target(*self._args, **self._kwargs) File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/server/backend/interfaces/balance_serve.py", line 244, in run_engine engine = Engine(args, token_queue, broadcast_endpoint) File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/server/backend/interfaces/balance_serve.py", line 162, in __init__ optimize_and_load_gguf(self.model, optimize_config_path, gguf_path, config) File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/optimize/optimize.py", line 131, in optimize_and_load_gguf load_weights(module, gguf_loader) File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/util/utils.py", line 125, in load_weights load_weights(child, gguf_loader, prefix+name+".") File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/util/utils.py", line 127, in load_weights module.load() File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/operators/base_operator.py", line 63, in load utils.load_weights(child, self.gguf_loader, self.key+".") File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/util/utils.py", line 125, in load_weights load_weights(child, gguf_loader, prefix+name+".") File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/util/utils.py", line 125, in load_weights load_weights(child, gguf_loader, prefix+name+".") File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/util/utils.py", line 125, in load_weights load_weights(child, gguf_loader, prefix+name+".") File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/util/utils.py", line 127, in load_weights module.load() File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/operators/base_operator.py", line 63, in load utils.load_weights(child, self.gguf_loader, self.key+".") File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/util/utils.py", line 125, in load_weights load_weights(child, gguf_loader, prefix+name+".") File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/util/utils.py", line 123, in load_weights load_cur_state_dict(module, gguf_loader, prefix) File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/util/utils.py", line 118, in load_cur_state_dict raise Exception(f"can't find {translated_key} in GGUF file!") Exception: can't find blk.0.attn_kv_b.weight in GGUF file! ^CTraceback (most recent call last): File "/home/sean/ubuntu/ktransformers/ktransformers/server/main.py", line 145, in <module> main() File "/home/sean/ubuntu/ktransformers/ktransformers/server/main.py", line 132, in main create_interface(config=cfg, default_args=cfg) File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/server/utils/create_interface.py", line 30, in create_interface GlobalInterface.interface = BackendInterface(default_args) File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/server/backend/interfaces/balance_serve.py", line 284, in __init__ start_event.wait() File "/home/sean/miniconda3/envs/kt/lib/python3.10/multiprocessing/synchronize.py", line 349, in wait self._cond.wait(timeout) File "/home/sean/miniconda3/envs/kt/lib/python3.10/multiprocessing/synchronize.py", line 261, in wait return self._wait_semaphore.acquire(True, timeout) KeyboardInterrupt

bsakaguchi commented

Author

Owner

@Azure-Tang commented on GitHub (May 30, 2025):

We are downloading latest gguf to test this issue.

@Azure-Tang commented on GitHub (May 30, 2025): We are downloading latest gguf to test this issue.

bsakaguchi commented

Author

Owner

@apalesnail commented on GitHub (May 30, 2025):

Same issue observed in Q4_K_M deepseek r1 0528

@apalesnail commented on GitHub (May 30, 2025): Same issue observed in Q4_K_M deepseek r1 0528

bsakaguchi commented

Author

Owner

@trilog-inc commented on GitHub (May 30, 2025):

Thanks @Azure-Tang ! I may try to quantize R1 0528 using an old version of llama.cpp, gonna have to fetch a new drive for storage though ...

Otherwise it seems that this needs to be addressed. The gguf quantization scripts of llama.cpp are all configured to output this new type of GGUF from now on.

@trilog-inc commented on GitHub (May 30, 2025): Thanks @Azure-Tang ! I may try to quantize R1 0528 using an old version of llama.cpp, gonna have to fetch a new drive for storage though ... Otherwise it seems that this needs to be addressed. The gguf quantization scripts of llama.cpp are all configured to output this new type of GGUF from now on.

bsakaguchi commented

Author

Owner

@344303947 commented on GitHub (Jun 2, 2025):

deepseek r1 0528 main分支代码已解决问题。

@344303947 commented on GitHub (Jun 2, 2025): deepseek r1 0528 main分支代码已解决问题。

bsakaguchi commented

Author

Owner

@Ning18 commented on GitHub (Jun 2, 2025):

The issue is solved by [Patch] lload DeepSeek-R1-0528.

I confirmed that the main branch right now (0.3.1) can successfully load UD-Q4_K_XL under unsloth/DeepSeek-V3-0324-GGUF-UD.

@Ning18 commented on GitHub (Jun 2, 2025): The issue is solved by [[Patch] lload DeepSeek-R1-0528](https://github.com/kvcache-ai/ktransformers/commit/a6b3243a562b05e7bd93a4bca2e3a745e46b7fd0). I confirmed that the main branch right now (0.3.1) can successfully load UD-Q4_K_XL under unsloth/DeepSeek-V3-0324-GGUF-UD.

bsakaguchi commented