[GH-ISSUE #1233] [Feature] flashinfer.get_batch_indices_positions( TypeError: get_batch_indices_positions() takes 3 positional arguments but 4 were given #836

New issue

Closed

opened 2026-10-29 14:45:08 +00:00 by bsakaguchi · 4 comments

bsakaguchi commented

2026-10-29 14:45:08 +00:00

Owner

Originally created by @cc8476 on GitHub (Apr 29, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1233

Originally assigned to: @Atream on GitHub.

Checklist

1. If the issue you raised is not a feature but a question, please raise a discussion at https://github.com/kvcache-ai/ktransformers/discussions. Otherwise, it will be closed.
2. To help the community, I will use Chinese/English or attach an Chinese/English translation if using another language. Non-English/Chinese content without translation may be closed.

Motivation

我查了一下，似乎没有错，下载的flashinfer是
https://github.com/kvcache-ai/custom_flashinfer/tree/fix-precision-mla-merge-main
然后进入目录后，pip install -e .
但是我看了下page.py下的get_batch_indices_positions ，确实是3个参数，为什么调用的时候是4个呢？
求助。。

报错信息如下：

Process SpawnProcess-1:
Traceback (most recent call last):
File "/usr/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib/python3.10/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/home/dsr1_server/ktransformers/ktransformers/server/backend/interfaces/balance_serve.py", line 285, in run_engine
engine.model_runner.warmup()
File "/home/dsr1_server/ktransformers/ktransformers/server/balance_serve/inference/model_runner.py", line 137, in warmup
self.outputs_buf[i] = self.model(self.input[i], self.features_buf[i], self.bsz_tensor_buf, self.num_tokens_tensor_buf, self.page_idx_buf[i], self.page_offset_buf[i], cuda_graph_idx=i)
File "/home/dsr1_server/myenv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/dsr1_server/myenv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
return forward_call(*args, **kwargs)
File "/home/dsr1_server/ktransformers/ktransformers/models/custom_modeling_qwen3_moe.py", line 78, in forward
self.attn[cuda_graph_idx].calc_batch_indices(hidden_states.shape[0])
File "/home/dsr1_server/ktransformers/ktransformers/operators/flashinfer_batch_prefill_wrapper.py", line 101, in calc_batch_indices
self.batch_indices, self.positions = flashinfer.get_batch_indices_positions(
TypeError: get_batch_indices_positions() takes 3 positional arguments but 4 were given

No response

Originally created by @cc8476 on GitHub (Apr 29, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1233 Originally assigned to: @Atream on GitHub. ### Checklist - [ ] 1. If the issue you raised is not a feature but a question, please raise a discussion at https://github.com/kvcache-ai/ktransformers/discussions. Otherwise, it will be closed. - [ ] 2. To help the community, I will use Chinese/English or attach an Chinese/English translation if using another language. Non-English/Chinese content without translation may be closed. ### Motivation 我查了一下，似乎没有错，下载的flashinfer是 https://github.com/kvcache-ai/custom_flashinfer/tree/fix-precision-mla-merge-main 然后进入目录后，pip install -e . 但是我看了下page.py下的get_batch_indices_positions ，确实是3个参数，为什么调用的时候是4个呢？求助。。报错信息如下： Process SpawnProcess-1: Traceback (most recent call last): File "/usr/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap self.run() File "/usr/lib/python3.10/multiprocessing/process.py", line 108, in run self._target(*self._args, **self._kwargs) File "/home/dsr1_server/ktransformers/ktransformers/server/backend/interfaces/balance_serve.py", line 285, in run_engine engine.model_runner.warmup() File "/home/dsr1_server/ktransformers/ktransformers/server/balance_serve/inference/model_runner.py", line 137, in warmup self.outputs_buf[i] = self.model(self.input[i], self.features_buf[i], self.bsz_tensor_buf, self.num_tokens_tensor_buf, self.page_idx_buf[i], self.page_offset_buf[i], cuda_graph_idx=i) File "/home/dsr1_server/myenv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/home/dsr1_server/myenv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl return forward_call(*args, **kwargs) File "/home/dsr1_server/ktransformers/ktransformers/models/custom_modeling_qwen3_moe.py", line 78, in forward self.attn[cuda_graph_idx].calc_batch_indices(hidden_states.shape[0]) File "/home/dsr1_server/ktransformers/ktransformers/operators/flashinfer_batch_prefill_wrapper.py", line 101, in calc_batch_indices self.batch_indices, self.positions = flashinfer.get_batch_indices_positions( TypeError: get_batch_indices_positions() takes 3 positional arguments but 4 were given ### Related resources _No response_

bsakaguchi closed this issue

2026-10-29 14:45:08 +00:00

bsakaguchi commented

2026-10-29 14:45:08 +00:00

Author

Owner

@Atream commented on GitHub (Apr 29, 2025):

使用custom flashinfer中的GQA_var_batch分支

@Atream commented on GitHub (Apr 29, 2025): 使用custom flashinfer中的GQA_var_batch分支

bsakaguchi commented

2026-10-29 14:45:08 +00:00

Author

Owner

@cc8476 commented on GitHub (Apr 30, 2025):

@Atream 您好，改用这个分支后，用碰到2个问题
1.custom_flashinfer-GQA_var_batch/flashinfer/init.py 需要
from ._build_meta import version as version 但是没有_build_meta文件，这个我注释掉正常了

2.请问下ktransformers/third_party/custom_flashinfer/flashinfer/data/csrc/batch_prefill_customize_config.jinja
这个文件从哪里来呢

File "/home/dsr1_server/ktransformers/third_party/custom_flashinfer/flashinfer/prefill.py", line 1405, in plan
self._cached_module = get_batch_prefill_module(self._backend)(
File "/home/dsr1_server/ktransformers/third_party/custom_flashinfer/flashinfer/prefill.py", line 199, in backend_module
module = gen_batch_prefill_module(backend, *args)
File "/home/dsr1_server/ktransformers/third_party/custom_flashinfer/flashinfer/jit/attention/pytorch.py", line 763, in gen_batch_prefill_module
return gen_customize_batch_prefill_module(
File "/home/dsr1_server/ktransformers/third_party/custom_flashinfer/flashinfer/jit/attention/pytorch.py", line 1152, in gen_customize_batch_prefill_module
with open(FLASHINFER_CSRC_DIR / "batch_prefill_customize_config.jinja") as f:
FileNotFoundError: [Errno 2] No such file or directory: '/home/dsr1_server/ktransformers/third_party/custom_flashinfer/flashinfer/data/csrc/batch_prefill_customize_config.jinja'

@cc8476 commented on GitHub (Apr 30, 2025): @Atream 您好，改用这个分支后，用碰到2个问题 1.custom_flashinfer-GQA_var_batch/flashinfer/__init__.py 需要 from ._build_meta import __version__ as __version__ 但是没有_build_meta文件，这个我注释掉正常了 2.请问下ktransformers/third_party/custom_flashinfer/flashinfer/data/csrc/batch_prefill_customize_config.jinja 这个文件从哪里来呢 File "/home/dsr1_server/ktransformers/third_party/custom_flashinfer/flashinfer/prefill.py", line 1405, in plan self._cached_module = get_batch_prefill_module(self._backend)( File "/home/dsr1_server/ktransformers/third_party/custom_flashinfer/flashinfer/prefill.py", line 199, in backend_module module = gen_batch_prefill_module(backend, *args) File "/home/dsr1_server/ktransformers/third_party/custom_flashinfer/flashinfer/jit/attention/pytorch.py", line 763, in gen_batch_prefill_module return gen_customize_batch_prefill_module( File "/home/dsr1_server/ktransformers/third_party/custom_flashinfer/flashinfer/jit/attention/pytorch.py", line 1152, in gen_customize_batch_prefill_module with open(FLASHINFER_CSRC_DIR / "batch_prefill_customize_config.jinja") as f: FileNotFoundError: [Errno 2] No such file or directory: '/home/dsr1_server/ktransformers/third_party/custom_flashinfer/flashinfer/data/csrc/batch_prefill_customize_config.jinja'

bsakaguchi commented

2026-10-29 14:45:09 +00:00

Author

Owner

@cc8476 commented on GitHub (Apr 30, 2025):

是不是新的禁用了flashinfer...

我刚刚拉了最新的ktransformers仓库的master分支，然后显示跑起来了

后面再试试请求api是否正常

感谢感谢！

@cc8476 commented on GitHub (Apr 30, 2025): 是不是新的禁用了flashinfer... 我刚刚拉了最新的ktransformers仓库的master分支，然后显示跑起来了后面再试试请求api是否正常感谢感谢！

bsakaguchi commented

2026-10-29 14:45:09 +00:00

Author

Owner

@Atream commented on GitHub (Apr 30, 2025):

localchat和ktransformers引擎不刚需flashinfer，但多并发的balance serve刚需flashinfer

如果觉得flashinfer不对，最好清除下~/.cache/flashinfer，再重新pip install -e . -v安装custom_flashinfer，或者直接让install.sh自动安装

@Atream commented on GitHub (Apr 30, 2025): localchat和ktransformers引擎不刚需flashinfer，但多并发的balance serve刚需flashinfer 如果觉得flashinfer不对，最好清除下~/.cache/flashinfer，再重新pip install -e . -v安装custom_flashinfer，或者直接让install.sh自动安装