turboderp c2aac982e4 Globally set Torch number of threads to 1

2024-06-17 00:39:16 +02:00

..

exllamav2_ext

Q GEMM: Initialize with bias when possible

2024-06-17 00:37:36 +02:00

generator

Only convert blocked_tokens set to list once

2024-06-16 16:41:17 +02:00

hadamard

Add hadamard functions

2024-04-13 14:24:48 +02:00

server

custom bos for cat

2024-05-01 00:07:50 -07:00

tokenizer

Update TODO: items

2024-05-18 06:43:27 +02:00

__init__.py

Add Q6 cache mode

2024-06-09 01:23:50 +02:00

architecture.py

Support MiniCPM architecture

2024-05-31 22:23:35 +02:00

attn.py

Paged attn: Skip some flash-attn wrapper code

2024-06-17 00:34:52 +02:00

cache.py

Q cache: Dequant after QKV projection to increase L2 cache hits in attn

2024-06-10 13:03:14 +02:00

compat.py

Non-blocking host-device copies in forward pass

2024-06-16 19:18:01 +02:00

config.py

Add env variable overrides for config defaults

2024-06-03 21:35:52 +02:00

embedding.py

Support MiniCPM architecture

2024-05-31 22:23:35 +02:00

ext.py

Use -Ofast for gcc

2024-06-16 14:08:19 +02:00

fasttensors.py

Add fasttensors fallback

2024-06-02 11:43:16 +02:00

headnorm.py

Load all modules in inference_mode

2024-05-31 22:23:35 +02:00

layernorm.py

Load all modules in inference_mode

2024-05-31 22:23:35 +02:00

linear.py

Load all modules in inference_mode

2024-05-31 22:23:35 +02:00

lora.py

Add indexed embeddings (unfinished)

2024-04-04 20:26:37 +02:00

mlp.py

QMLP: Skip .view

2024-06-16 19:14:47 +02:00

model_init.py

Add fasttensors fallback

2024-06-02 11:43:16 +02:00

model.py

Globally set Torch number of threads to 1

2024-06-17 00:39:16 +02:00

module.py

Add intervention API

2024-06-03 19:18:15 +02:00

moe_mlp.py

Load all modules in inference_mode

2024-05-31 22:23:35 +02:00

parallel_decoder.py

Load all modules in inference_mode

2024-05-31 22:23:35 +02:00

pos_embedding.py

Load all modules in inference_mode

2024-05-31 22:23:35 +02:00

rmsnorm.py

Support MiniCPM architecture

2024-05-31 22:23:35 +02:00

util.py

Dynamic gen: add banned strings

2024-05-22 22:21:49 +02:00

version.py

Bump to v0.1.5

2024-06-09 02:15:09 +02:00