[PR #645] [MERGED] Ensure backward compatibility with PyTorch 2.2 #1444

New issue

Closed

opened 2026-10-29 14:52:29 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 14:52:29 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/ktransformers/pull/645
Author: @yeahdongcn
Created: 2/24/2025
Status: ✅ Merged
Merged: 2/24/2025
Merged by: @Atream

Base: main ← Head: torch2.2

📝 Commits (1)

f88c05a Ensure backward compatibility with Torch 2.2

📊 Changes

2 files changed (+32 additions, -25 deletions)

View changed files

📝 ktransformers/ktransformers_ext/cuda/binding.cpp (+23 -16)
📝 ktransformers/ktransformers_ext/cuda/custom_gguf/ops.h (+9 -9)

📄 Description

While testing with torch_musa from PyTorch 2.2, I encountered an issue when loading weights to the GPU:

  File "/ws/./ktransformers/util/custom_gguf.py", line 674, in dequantize_q6_k_gpu
    return KTransformersOps.dequantize_q6_k(c_pointer, data.size, block_size, ele_per_blk, device, target_dtype)
TypeError: dequantize_q6_k(): incompatible function arguments. The following argument types are supported:
    1. (data: int, num_bytes: int, blk_size: int, ele_per_blk: int, device: torch.device, target_dtype: torch._C._te.ScalarType) -> torch.Tensor

This PR ensures backward compatibility with Torch 2.2 and fixes a mismatched parameter type in the header file.

Testing Done

make dev_install
Verified weight loading to CPU/GPU using python ./ktransformers/local_chat.py ...

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/ktransformers/pull/645 **Author:** [@yeahdongcn](https://github.com/yeahdongcn) **Created:** 2/24/2025 **Status:** ✅ Merged **Merged:** 2/24/2025 **Merged by:** [@Atream](https://github.com/Atream) **Base:** `main` ← **Head:** `torch2.2` --- ### 📝 Commits (1) - [`f88c05a`](https://github.com/kvcache-ai/ktransformers/commit/f88c05a6f18c349639aa58823cc8072d52b2c349) Ensure backward compatibility with Torch 2.2 ### 📊 Changes **2 files changed** (+32 additions, -25 deletions) <details> <summary>View changed files</summary> 📝 `ktransformers/ktransformers_ext/cuda/binding.cpp` (+23 -16) 📝 `ktransformers/ktransformers_ext/cuda/custom_gguf/ops.h` (+9 -9) </details> ### 📄 Description While testing with torch_musa from PyTorch 2.2, I encountered an issue when loading weights to the GPU: ```bash File "/ws/./ktransformers/util/custom_gguf.py", line 674, in dequantize_q6_k_gpu return KTransformersOps.dequantize_q6_k(c_pointer, data.size, block_size, ele_per_blk, device, target_dtype) TypeError: dequantize_q6_k(): incompatible function arguments. The following argument types are supported: 1. (data: int, num_bytes: int, blk_size: int, ele_per_blk: int, device: torch.device, target_dtype: torch._C._te.ScalarType) -> torch.Tensor ``` This PR ensures backward compatibility with Torch 2.2 and fixes a mismatched parameter type in the header file. ### Testing Done - [x] `make dev_install` - [x] Verified weight loading to CPU/GPU using `python ./ktransformers/local_chat.py ...` --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-29 14:52:29 +00:00