[GH-ISSUE #15] GPU support without fp16. Multi gpu support #9

New issue

Closed

opened 2026-10-29 14:32:36 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-29 14:32:36 +00:00

Owner

Originally created by @SlerpE on GitHub (Jul 29, 2024).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/15

Hello, I am very excited about this project. I wanted to ask if ktransformers supports or will support video cards that do not work with fp16? Is there multi gpu support like llama cpp?

Originally created by @SlerpE on GitHub (Jul 29, 2024). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/15 Hello, I am very excited about this project. I wanted to ask if ktransformers supports or will support video cards that do not work with fp16? Is there multi gpu support like llama cpp?

bsakaguchi closed this issue

2026-10-29 14:32:36 +00:00

bsakaguchi commented

2026-10-29 14:32:37 +00:00

Author

Owner

@james0zan commented on GitHub (Jul 29, 2024):

We are working on multi-gpu suppport. But the support for non-fp16 card may not come soon, because we do not have such devices for experiemnting and testing.

@james0zan commented on GitHub (Jul 29, 2024): We are working on multi-gpu suppport. But the support for non-fp16 card may not come soon, because we do not have such devices for experiemnting and testing.

bsakaguchi referenced this issue

2026-10-29 14:37:56 +00:00

[GH-ISSUE #465] /pytorch/aten/src/ATen/native/cuda/TensorCompare.cu:110: _assert_async_cuda_kernel: block: [0,0,0], thread: [0,0,0] Assertion probability tensor contains either inf, nan or element < 0 failed. #327

bsakaguchi referenced this issue

2026-10-29 14:42:22 +00:00

[GH-ISSUE #903] Rocm run deepseek r1 671b q4 failed #646

bsakaguchi referenced this issue

2026-10-29 14:45:29 +00:00

[GH-ISSUE #1269] [Question] benchmark with evalscope failed #859

bsakaguchi referenced this issue

2026-10-29 14:45:38 +00:00

[GH-ISSUE #1286] [Bug] inference_context.v_cache[0][i] IndexError: list index out of range #871