mirror of
https://github.com/kvcache-ai/ktransformers.git
synced 2026-11-05 01:07:28 +00:00
[GH-ISSUE #607] raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 7 column 1 (char 6) 报错了 #448
Labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
kvcache-ai/ktransformers#448
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @otsuka88888 on GitHub (Feb 23, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/607
附上模型路径 cpu信息 和运行完整报错
哪个好心大佬能帮忙看看错在哪了吗
(ktransformers) root@ubuntu:/models/DeepSeek-R1-GGUF# ll
total 10220
drwxr-xr-x 4 root root 4096 Feb 23 09:45 ./
drwxr-xr-x 4 root root 4096 Feb 22 23:11 ../
-rw-r--r-- 1 root root 127945 Feb 22 23:38 config.json
-rw-r--r-- 1 root root 354000 Feb 22 13:51 configuration_deepseek.py
drwxr-xr-x 3 root root 4096 Feb 22 23:16 DeepSeek-R1-Q4_K_M/
-rw-r--r-- 1 root root 235556 Feb 22 13:51 generation_config.json
-rw-r--r-- 1 root root 256556 Feb 22 23:39 .gitattributes
-rw-r--r-- 1 root root 75769 Feb 23 09:44 modeling_deepseek.py
-rw-r--r-- 1 root root 8898324 Feb 23 09:45 model.safetensors.index.json
drwxr-xr-x 3 root root 4096 Feb 21 22:00 ._____temp/
-rw-r--r-- 1 root root 254847 Feb 22 13:52 tokenizer_config.json
-rw-r--r-- 1 root root 226836 Feb 22 13:51 tokenizer.json
(ktransformers) root@ubuntu:/models/DeepSeek-R1-GGUF/DeepSeek-R1-Q4_K_M# ll
total 408635988
drwxr-xr-x 3 root root 4096 Feb 22 23:16 ./
drwxr-xr-x 4 root root 4096 Feb 23 09:45 ../
drwxr-xr-x 2 root root 4096 Feb 22 23:16 DeepSeek-R1-Q4_K_M/
-rw-r--r-- 1 root root 48339779936 Jan 26 08:34 DeepSeek-R1-Q4_K_M-00001-of-00009.gguf
-rw-r--r-- 1 root root 49429396320 Jan 26 08:40 DeepSeek-R1-Q4_K_M-00002-of-00009.gguf
-rw-r--r-- 1 root root 49527312640 Jan 26 08:41 DeepSeek-R1-Q4_K_M-00003-of-00009.gguf
-rw-r--r-- 1 root root 47615522305 Feb 22 01:39 DeepSeek-R1-Q4_K_M-00004-of-00009.gguf
-rw-r--r-- 1 root root 49422027488 Jan 26 08:36 DeepSeek-R1-Q4_K_M-00005-of-00009.gguf
-rw-r--r-- 1 root root 48272509536 Jan 26 08:39 DeepSeek-R1-Q4_K_M-00006-of-00009.gguf
-rw-r--r-- 1 root root 49429396320 Jan 26 08:42 DeepSeek-R1-Q4_K_M-00007-of-00009.gguf
-rw-r--r-- 1 root root 46938773696 Jan 26 08:44 DeepSeek-R1-Q4_K_M-00008-of-00009.gguf
-rw-r--r-- 1 root root 29468415345 Jan 26 08:31 DeepSeek-R1-Q4_K_M-00009-of-00009.gguf
ktransformers --model_path /models/DeepSeek-R1-GGUF/ --gguf_path /models/DeepSeek-R1-GGUF/DeepSeek-R1-Q4/
(ktransformers) root@ubuntu:~/ktransformers# lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 46 bits physical, 48 bits virtua
l
Byte Order: Little Endian
CPU(s): 96
On-line CPU(s) list: 0-95
Vendor ID: GenuineIntel
BIOS Vendor ID: Intel(R) Corporation
Model name: Intel(R) Xeon(R) Gold 6248R CPU
@ 3.00GHz
BIOS Model name: Intel(R) Xeon(R) Gold 6248R CPU
@ 3.00GHz CPU @ 3.0GHz
BIOS CPU family: 179
CPU family: 6
Model: 85
Thread(s) per core: 2
Core(s) per socket: 24
Socket(s): 2
Stepping: 7
CPU(s) scaling MHz: 31%
CPU max MHz: 4000.0000
CPU min MHz: 1200.0000
BogoMIPS: 6000.00
Flags: fpu vme de pse tsc msr pae mce c
x8 apic sep mtrr pge mca cmov pa
t pse36 clflush dts acpi mmx fxs
r sse sse2 ss ht tm pbe syscall
nx pdpe1gb rdtscp lm constant_ts
c art arch_perfmon pebs bts rep_
good nopl xtopology nonstop_tsc
cpuid aperfmperf pni pclmulqdq d
tes64 ds_cpl vmx smx est tm2 sss
e3 sdbg fma cx16 xtpr pdcm pcid
dca sse4_1 sse4_2 x2apic movbe p
opcnt tsc_deadline_timer aes xsa
ve avx f16c rdrand lahf_lm abm 3
dnowprefetch cpuid_fault epb cat
_l3 cdp_l3 ssbd mba ibrs ibpb st
ibp ibrs_enhanced tpr_shadow fle
xpriority ept vpid ept_ad fsgsba
se tsc_adjust bmi1 avx2 smep bmi
2 erms invpcid cqm mpx rdt_a avx
512f avx512dq rdseed adx smap cl
flushopt clwb intel_pt avx512cd
avx512bw avx512vl xsaveopt xsave
c xgetbv1 xsaves cqm_llc cqm_occ
up_llc cqm_mbm_total cqm_mbm_loc
al dtherm ida arat pln pts vnmi
pku ospke avx512_vnni md_clear f
lush_l1d arch_capabilities
Virtualization features:
Virtualization: VT-x
Caches (sum of all):
L1d: 1.5 MiB (48 instances)
L1i: 1.5 MiB (48 instances)
L2: 48 MiB (48 instances)
L3: 71.5 MiB (2 instances)
NUMA:
NUMA node(s): 2
NUMA node0 CPU(s): 0-23,48-71
NUMA node1 CPU(s): 24-47,72-95
Vulnerabilities:
Gather data sampling: Mitigation; Microcode
Itlb multihit: KVM: Mitigation: VMX disabled
L1tf: Not affected
Mds: Not affected
Meltdown: Not affected
Mmio stale data: Mitigation; Clear CPU buffers; S
MT vulnerable
Reg file data sampling: Not affected
Retbleed: Mitigation; Enhanced IBRS
Spec rstack overflow: Not affected
Spec store bypass: Mitigation; Speculative Store By
pass disabled via prctl
Spectre v1: Mitigation; usercopy/swapgs barr
iers and __user pointer sanitiza
tion
Spectre v2: Mitigation; Enhanced / Automatic
IBRS; IBPB conditional; RSB fil
ling; PBRSB-eIBRS SW sequence; B
HI Syscall hardening, KVM SW loo
p
Srbds: Not affected
Tsx async abort: Mitigation; TSX disabled
ktransformers) root@ubuntu:~/ktransformers# ktransformers --model_path /models/DeepSeek-R1-GGUF/ --gguf_path /models/DeepSeek-R1-GGUF/DeepSeek-R1-Q4/
flashinfer not found, use triton for linux
2025-02-23 09:58:30,224 INFO /root/anaconda3/envs/ktransformers/lib/python3.11/site-packages/ktransformers/server/main.py[29]: Creating SQL tables
2025-02-23 09:58:30,225 INFO /root/anaconda3/envs/ktransformers/lib/python3.11/site-packages/ktransformers/server/api/openai/assistants/assistants.py[75]: Creating default assistant
Traceback (most recent call last):
File "/root/anaconda3/envs/ktransformers/bin/ktransformers", line 8, in
sys.exit(main())
^^^^^^
File "/root/anaconda3/envs/ktransformers/lib/python3.11/site-packages/ktransformers/server/main.py", line 115, in main
create_interface(config=cfg, default_args=cfg)
File "/root/anaconda3/envs/ktransformers/lib/python3.11/site-packages/ktransformers/server/utils/create_interface.py", line 27, in create_interface
GlobalInterface.interface = BackendInterface(default_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/anaconda3/envs/ktransformers/lib/python3.11/site-packages/ktransformers/server/backend/interfaces/ktransformers.py", line 30, in init
self.tokenizer = AutoTokenizer.from_pretrained(args.model_dir, device=args.device, trust_remote_code=args.trust_remote_code)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/anaconda3/envs/ktransformers/lib/python3.11/site-packages/transformers/models/auto/tokenization_auto.py", line 833, in from_pretrained
tokenizer_config = get_tokenizer_config(pretrained_model_name_or_path, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/anaconda3/envs/ktransformers/lib/python3.11/site-packages/transformers/models/auto/tokenization_auto.py", line 687, in get_tokenizer_config
result = json.load(reader)
^^^^^^^^^^^^^^^^^
File "/root/anaconda3/envs/ktransformers/lib/python3.11/json/init.py", line 293, in load
return loads(fp.read(),
^^^^^^^^^^^^^^^^
File "/root/anaconda3/envs/ktransformers/lib/python3.11/json/init.py", line 346, in loads
return _default_decoder.decode(s)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/anaconda3/envs/ktransformers/lib/python3.11/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/anaconda3/envs/ktransformers/lib/python3.11/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 7 column 1 (char 6)
@otsuka88888 commented on GitHub (Feb 23, 2025):
修改启动命令后报错RuntimeError: CUDA error: invalid device function
Compile with
TORCH_USE_CUDA_DSAto enable device-side assertions.