mirror of
https://github.com/kvcache-ai/ktransformers.git
synced 2026-11-05 01:07:28 +00:00
[GH-ISSUE #1083] [Feature] deepseek-0324 function call支持 #764
Labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
kvcache-ai/ktransformers#764
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @1556900941lizerui on GitHub (Apr 8, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1083
检查清单
需求背景
目前在部署完deepseek-0324后，发现并不支持function call，能否提供相关支持
相关资源
No response
@KMSorSMS commented on GitHub (Apr 11, 2025):
目前不支持，后续准备添加
@344303947 commented on GitHub (Apr 13, 2025):
https://www.bilibili.com/video/BV12LdrYuEXt/?vd_source=1c35b1bd94e0a83fee53178c65018f35 Ktransformers支持function calling以及MCP调用教程
@KMSorSMS commented on GitHub (Apr 13, 2025):
挺好的，如果可以的话，能整理一个 pr 贡献给社区吗？我们将会非常感谢您的贡献 ✨
@344303947 commented on GitHub (Apr 14, 2025):
https://github.com/kvcache-ai/ktransformers/pull/1134
@wishfay commented on GitHub (Apr 22, 2025):
running build_ext
/opt/conda/lib/python3.11/site-packages/torch/utils/cpp_extension.py:426: UserWarning: There are no g++ version bounds defined for CUDA version 12.1
warnings.warn(f'There are no {compiler_name} version bounds defined for CUDA version {cuda_str_version}')
CMake args: ['-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/workspace/ktransformers/build/lib.linux-x86_64-cpython-311/', '-DPYTHON_EXECUTABLE=/opt/conda/bin/python3.11', '-DCMAKE_BUILD_TYPE=Release', '-DKTRANSFORMERS_USE_CUDA=ON', '-D_GLIBCXX_USE_CXX11_ABI=0']
CMake args: ['-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/workspace/ktransformers/build/lib.linux-x86_64-cpython-311/', '-DPYTHON_EXECUTABLE=/opt/conda/bin/python3.11', '-DCMAKE_BUILD_TYPE=Release', '-DKTRANSFORMERS_USE_CUDA=ON', '-D_GLIBCXX_USE_CXX11_ABI=0', '-DLLAMA_NATIVE=ON', '-DEXAMPLE_VERSION_INFO=0.2.4.post1+cu121torch25avx2']
build_temp: /workspace/ktransformers/csrc/ktransformers_ext/build
Traceback (most recent call last):
File "/opt/conda/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 389, in
main()
File "/opt/conda/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 373, in main
json_out["return_val"] = hook(**hook_input["kwargs"])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 280, in build_wheel
return _build_backend().build_wheel(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/setuptools/build_meta.py", line 415, in build_wheel
return self._build_with_temp_dir(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/setuptools/build_meta.py", line 397, in _build_with_temp_dir
self.run_setup()
File "/opt/conda/lib/python3.11/site-packages/setuptools/build_meta.py", line 313, in run_setup
exec(code, locals())
File "", line 646, in
File "/opt/conda/lib/python3.11/site-packages/setuptools/init.py", line 108, in setup
return distutils.core.setup(**attrs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 184, in setup
return run_commands(dist)
^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 200, in run_commands
dist.run_commands()
File "/opt/conda/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 970, in run_commands
self.run_command(cmd)
File "/opt/conda/lib/python3.11/site-packages/setuptools/dist.py", line 945, in run_command
super().run_command(command)
File "/opt/conda/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 989, in run_command
cmd_obj.run()
File "", line 250, in run
File "/opt/conda/lib/python3.11/site-packages/wheel/_bdist_wheel.py", line 378, in run
self.run_command("build")
File "/opt/conda/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
self.distribution.run_command(command)
File "/opt/conda/lib/python3.11/site-packages/setuptools/dist.py", line 945, in run_command
super().run_command(command)
File "/opt/conda/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 989, in run_command
cmd_obj.run()
File "/opt/conda/lib/python3.11/site-packages/setuptools/_distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
File "/opt/conda/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
self.distribution.run_command(command)
File "/opt/conda/lib/python3.11/site-packages/setuptools/dist.py", line 945, in run_command
super().run_command(command)
File "/opt/conda/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 989, in run_command
cmd_obj.run()
File "/opt/conda/lib/python3.11/site-packages/setuptools/command/build_ext.py", line 93, in run
_build_ext.run(self)
File "/opt/conda/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 359, in run
self.build_extensions()
File "/opt/conda/lib/python3.11/site-packages/torch/utils/cpp_extension.py", line 868, in build_extensions
build_ext.build_extensions(self)
File "/opt/conda/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 479, in build_extensions
self._build_extensions_serial()
File "/opt/conda/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 505, in _build_extensions_serial
self.build_extension(ext)
File "", line 576, in build_extension
File "", line 357, in run_command_with_live_tail
File "/opt/conda/lib/python3.11/subprocess.py", line 571, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['cmake', '/workspace/ktransformers/csrc/ktransformers_ext', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/workspace/ktransformers/build/lib.linux-x86_64-cpython-311/', '-DPYTHON_EXECUTABLE=/opt/conda/bin/python3.11', '-DCMAKE_BUILD_TYPE=Release', '-DKTRANSFORMERS_USE_CUDA=ON', '-D_GLIBCXX_USE_CXX11_ABI=0', '-DLLAMA_NATIVE=ON', '-DEXAMPLE_VERSION_INFO=0.2.4.post1+cu121torch25avx2']' died with <Signals.SIGSEGV: 11>.
error: subprocess-exited-with-error
× Building wheel for ktransformers (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
full command: /opt/conda/bin/python3.11 /opt/conda/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py build_wheel /tmp/tmpnssqqo6h
cwd: /workspace/ktransformers
Building wheel for ktransformers (pyproject.toml) ... error
ERROR: Failed building wheel for ktransformers
Failed to build ktransformers
ERROR: Failed to build installable wheels for some pyproject.toml based projects (ktransformers)
重新编译
env USE_BALANCE_SERVE=1 USE_NUMA=1 PYTHONPATH="$(which python)" PATH="$(dirname $(which python)):$PATH" bash ./install.sh
更新了那四个文件，尝试在docker中重新编译，但是报上述错误，如何解决呢？