kevin
c8db24d5eb
Update config.py
...
Update config.py
2025-04-16 17:32:08 +08:00
kevin
badf7a1bb1
Merge branch 'kvcache-ai:main' into features/add-function-calling
2025-04-16 17:21:27 +08:00
Chengyu Qiu
d2cf81423f
Merge pull request #1135 from Creeper-MZ/function_call
...
Feat: Add Function call support
2025-04-16 09:57:22 +08:00
ZiWei Yuan
fcbd41e175
Merge pull request #1143 from jizhilong/improve-cmake-subprocess-output
...
feat(build): display limited tail of subprocesses in real time
2025-04-15 17:37:44 +08:00
jizhilong
0638ea298d
feat(build): display limited tail of subprocesses in real time
...
this is a followup on #1108
2025-04-15 16:40:38 +08:00
ZiWei Yuan
8dc1ab9e04
Merge pull request #1108 from jizhilong/expose-cmake-logs
...
chore: show cmake output in real time during build_ext
2025-04-14 17:07:00 +08:00
sean.su
8699109129
Refactor the chat interface to support tool calling and parameter processing
...
Defined new data structures in chat.py to replace OpenAI's original implementation, adding support for tool calling.
Implemented logic for extracting and processing tool calls, enabling dynamic function invocation during conversations.
Added methods in balance_serve.py to retrieve sampling parameters, handling default values and edge cases.
Updated ktransformers.py and transformers.py to support the passing of tool parameters.
Modified the default value of top_p in config.py to 1.0 to increase generation diversity.
Extended the message model in chat.py to support the transmission of tool call information.
These changes enhance the system's flexibility and functionality, enabling more complex interaction patterns.
2025-04-14 15:23:37 +08:00
Creeper-MZ
a7e8d7c1af
updata function_call
2025-04-13 23:48:51 -04:00
wang jiahao
038db30ec9
Merge pull request #1132 from wangkuigang-yewu-cmss/long-prompt-crash
...
使用长prompt时，避免rpc进程挂掉
2025-04-13 22:06:11 +08:00
wangkuigang-yewu-cmss
4538bdae97
prevent rpc process from crashing on long prompt
...
当prompt超过cache_len的时候，rpc进程会crash掉，导致整体不可用。
这里增加一个检查，让过长的prompt在请求早期就被提前过滤掉
2025-04-13 16:13:16 +08:00
ErvinXie
797dac7e31
Merge pull request #1109 from aubreyli/libxxhash-fPIC
...
xxHash: fix link error due to non-position-independent code
2025-04-13 14:15:31 +08:00
ZiWei Yuan
77956822ce
Merge pull request #1116 from ikawrakow/ik/add_copyright
...
Add missing references to ik_llama.cpp
2025-04-13 11:53:12 +08:00
Iwan Kawrakow
99a247e167
Spelling
2025-04-11 10:15:42 +03:00
Iwan Kawrakow
c46b0c59d0
Add missing references to ik_llama.cpp
2025-04-11 09:39:57 +03:00
Aubrey Li
63ca2fa84d
xxHash: fix link error due to non-position-independent code
...
Add PROPERTIES POSITION_INDEPENDENT_CODE option to fix the
following error:
/usr/bin/ld: ../../third_party/xxHash/libxxhash.a(xxhash.c.o):
relocation R_X86_64_32S against `.rodata' can not be used when
making a shared object; recompile with -fPIC
Trying to link a non-PIC static library libxxhash.a into a
.so shared library, which is not allowed. The object file
xxhash.c.o must be recompiled with explicit -fPIC support.
2025-04-10 21:50:23 +08:00
jizhilong
690d4d42f9
chore: show cmake output in real time during build_ext
...
otherwise cmake error messages may be suppressed, making debugging
difficult
2025-04-10 21:33:04 +08:00
Atream
35ba63e259
Merge pull request #1103 from kvcache-ai/Atream-patch-6
...
Create SECURITY.md
2025-04-09 19:50:57 +08:00
Atream
5f8cdc7640
Create SECURITY.md
2025-04-09 19:50:38 +08:00
Atream
92a67ab549
Merge pull request #1101 from kvcache-ai/Atream-patch-5
...
Update llama4.md
2025-04-09 19:23:46 +08:00
Atream
98dbdcd66c
Update llama4.md
2025-04-09 19:23:35 +08:00
Atream
9fad782c1a
Merge pull request #1100 from kvcache-ai/Atream-patch-4
...
Update llama4.md
2025-04-09 19:10:03 +08:00
Atream
346d202297
Update llama4.md
2025-04-09 19:09:44 +08:00
Atream
a46c43b2db
Merge pull request #1099 from kvcache-ai/Atream-patch-3
...
Update llama4.md
2025-04-09 18:01:46 +08:00
Atream
d1fcb208cc
Update llama4.md
2025-04-09 18:01:13 +08:00
Atream
0774fe4d62
Merge pull request #1098 from kvcache-ai/Atream-patch-2
...
Update llama4.md
2025-04-09 17:58:44 +08:00
Atream
ed2b971e02
Update llama4.md
2025-04-09 17:57:37 +08:00
Jianwei Dong
c689b23364
Merge pull request #1097 from kvcache-ai/update-llama4-tutorial
...
update llama4 tutorial
2025-04-09 17:40:58 +08:00
djw
26798500bd
update llama4 tutorial
2025-04-09 09:40:08 +00:00
Jianwei Dong
1e0be68e51
Merge pull request #1096 from kvcache-ai/update-llama4-tutorial
...
update llama4 tutorial
2025-04-09 17:37:33 +08:00
djw
f73b4ca706
update llama4 tutorial
2025-04-09 09:36:30 +00:00
Jianwei Dong
2de96a1f05
Merge pull request #1095 from kvcache-ai/update-llama4-tutorial
...
update llama4 tutorial
2025-04-09 17:35:14 +08:00
djw
ecc3028c13
update llama4 tutorial
2025-04-09 09:34:04 +00:00
Azure
a74a58d864
Merge pull request #1091 from aubreyli/add_g++
...
balance_serve: Add g++ to compiler list
2025-04-09 14:40:30 +08:00
Aubrey Li
45d20fa87b
balance_serve: Add g++ to compiler list
...
In some OS distributions, g++ exists in the following form:
# ls -l /usr/bin/g++*
-rwxr-xr-x 4 root root 985784 Dec 9 12:51 /usr/bin/g++
So make sure to add g++ to the compiler list as well.
2025-04-09 11:25:35 +08:00
Atream
9037bf30d5
Merge pull request #1090 from kvcache-ai/Atream-patch-1
...
Update attention.py
2025-04-09 10:54:37 +08:00
Atream
3b9e16cec7
Update attention.py
2025-04-09 10:54:00 +08:00
wang jiahao
94476ce5cc
Merge pull request #1085 from kvcache-ai/qiyuxinlin-patch-5
...
Update balance-serve.md
2025-04-08 19:19:37 +08:00
wang jiahao
23ceb1c049
Update balance-serve.md
2025-04-08 19:19:00 +08:00
wang jiahao
41ce92bb22
Merge pull request #1084 from kvcache-ai/fix-config
...
format kvc2, delete quant_configs, move model_configs to ~/.ktransfor…
2025-04-08 19:14:07 +08:00
qiyuxinlin
64de784328
format kvc2, delete quant_configs, move model_configs to ~/.ktransformers
2025-04-08 10:06:07 +00:00
Atream
10fd2e281f
Merge pull request #1079 from kvcache-ai/fix-compile
...
fix compile, add abi check to setup.py
2025-04-08 14:36:31 +08:00
Atream
9dd24ecd72
fix compile, add abi check to setup.py
2025-04-08 06:18:30 +00:00
wang jiahao
f4ae7c85ed
Merge pull request #1069 from kvcache-ai/qiyuxinlin-patch-4
...
Update balance-serve.md
2025-04-07 19:20:00 +08:00
wang jiahao
2fcdbee769
Update balance-serve.md
2025-04-07 19:19:49 +08:00
Azure
77c6cc82ac
Merge pull request #1063 from aubreyli/KLinearCPUInfer.forward-fix
...
Fix TypeError when invoke KLinearCPUInfer.forward()
2025-04-07 15:10:46 +08:00
wang jiahao
6463070b16
Merge pull request #1064 from kvcache-ai/fix-temperature
...
fix temperature=0, flashinfer sample error
2025-04-07 12:32:28 +08:00
dongjw
ec03bcbd7f
fix temperature=0, flashinfer sample error
2025-04-07 12:30:47 +08:00
Atream
aac0c91d02
Merge pull request #1060 from kvcache-ai/fix-compile
...
Fix compile
2025-04-07 12:10:31 +08:00
Aubrey Li
12a4c631df
Fix TypeError when invoke KLinearCPUInfer.forward()
...
Fix the following error:
File "/home/aubrey/work/ktransformers/ktransformers/operators/linear.py", line 825, in forward
y = self.generate_linear.forward(x, bsz_tensor)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: KLinearCPUInfer.forward() takes 2 positional arguments but 3 were given
2025-04-07 12:03:35 +08:00
Atream
fa87c1aeea
Update CMakeLists.txt
2025-04-07 10:32:30 +08:00
