turboderp 23fc4737ae Fast safetensors mode with direct IO and pinned buffer

2024-01-18 20:11:53 +01:00

..

test_alloc.py

34B testing

2023-09-10 06:15:33 +02:00

test_autosplit.py

Fix unhandled OoM condition when loading GPTQ model with auto split

2023-10-28 20:08:39 +02:00

test_batch_latency.py

Batch latency test script

2023-12-23 22:04:40 +01:00

test_fasttensors.py

Fast safetensors mode with direct IO and pinned buffer

2024-01-18 20:11:53 +01:00

test_gemv.py

Instrumentation etc.

2023-12-10 17:36:40 +01:00

test_hgemm.py

Tests for half GEMM kernels

2023-11-25 11:54:53 +01:00

test_humaneval.py

Add script to compare quantized and unquantized model

2023-12-23 02:57:13 +01:00

test_mmlu.py

Add script to compare quantized and unquantized model

2023-12-23 02:57:13 +01:00

test_tokenizer.py

Fix some tokenization edge cases

2023-12-03 22:03:23 +01:00

test.py

Optimizer batched sampling

2023-12-23 22:04:10 +01:00