turboderp-org/exllamav2
1
0
Fork 0
mirror of https://github.com/turboderp-org/exllamav2.git synced 2026-04-20 14:29:28 +00:00
Code Issues Packages Projects Releases Wiki Activity

Kernel optimizations WIP

Browse Source
This commit is contained in:
turboderp
2023-09-02 05:37:00 +02:00
parent c5cf3956dc
commit 92ce76dec1
9 changed files with 835 additions and 198 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
test_inference.py
View File

@@ -52,7 +52,7 @@ if args.prompt:
print(f" -- Warmup...")
model.forward(ids[:, :])
model.forward(ids[:, -1:])
print(f" -- Generating (greedy sampling)...")
print()