tabbyAPI

theroyallab/tabbyAPI

Fork 0

mirror of https://github.com/theroyallab/tabbyAPI.git synced 2026-07-13 02:27:11 +00:00

Commit Graph

Select branches

Hide Pull Requests

breaking

main

main_tools

tool-calls

#100

#101

#102

#103

#104

#105

#108

#109

#11

#111

#112

#113

#114

#12

#121

#122

#125

#126

#128

#129

#13

#130

#132

#133

#134

#136

#138

#139

#141

#141

#146

#147

#150

#154

#155

#158

#159

#161

#163

#171

#173

#174

#175

#178

#18

#180

#181

#183

#184

#185

#189

#19

#190

#190

#191

#192

#193

#194

#195

#197

#199

#20

#201

#204

#206

#208

#213

#213

#214

#214

#22

#220

#222

#222

#224

#225

#228

#23

#230

#232

#233

#233

#24

#243

#244

#246

#249

#252

#253

#254

#26

#264

#265

#266

#27

#271

#28

#281

#289

#29

#292

#293

#295

#30

#302

#303

#305

#307

#308

#31

#312

#313

#313

#320

#322

#326

#327

#328

#329

#33

#330

#331

#336

#339

#34

#340

#341

#342

#346

#349

#35

#351

#358

#36

#362

#364

#365

#37

#370

#370

#371

#372

#373

#374

#375

#376

#377

#378

#378

#380

#382

#39

#393

#394

#394

#395

#395

#396

#396

#397

#4

#40

#402

#403

#403

#407

#411

#411

#413

#413

#415

#415

#418

#420

#421

#422

#423

#424

#424

#425

#427

#429

#43

#430

#431

#431

#44

#45

#46

#47

#50

#54

#56

#58

#59

#61

#63

#66

#71

#72

#75

#76

#78

#80

#82

#85

#88

#9

#93

#94

#99

#99

7fded4f183 Tree: Switch to async generators kingbri 2024-03-14 10:27:39 -04:00
33e2df50b7 API: Disable SSE ping chunks kingbri 2024-03-14 20:47:05 -04:00
7006fa4cc8 Tree: Format kingbri 2024-03-13 23:33:18 -04:00
efc01d947b API + Model: Add speculative ngram decoding kingbri 2024-03-13 23:32:11 -04:00
2ebefe8258 Logging: Move metrics to gen logging kingbri 2024-03-13 23:13:55 -04:00
1ec8eb9620 Tree: Format kingbri 2024-03-13 00:02:31 -04:00
8e4745920c Requirements: Update Ruff kingbri 2024-03-13 00:02:19 -04:00
6f03be9523 API: Split functions into their own files kingbri 2024-03-12 01:31:59 -04:00
104a6121cb API: Split into separate folder kingbri 2024-03-11 22:45:30 -04:00
5a2de30066 Tree: Update to cleanup globals kingbri 2024-03-09 22:31:47 -05:00
b373b25235 API: Move to ModelManager kingbri 2024-03-08 23:57:07 -05:00
8b46282aef Model: Fix state flag sets on unload kingbri 2024-03-11 23:42:52 -04:00
894be4a818 Startup: Check if the port is available and fallback kingbri 2024-03-11 21:55:16 -04:00
7c6fd7ac60 Main: Cleanup kingbri 2024-03-11 18:10:35 -04:00
53d889e0f0 Logging: Fix legacy warn statement kingbri 2024-03-11 01:31:43 -04:00
ba3da6d92f Logging: Escape rich markup sequences kingbri 2024-03-11 00:28:48 -04:00
4cc0b59bdc Requirements: Add sse-starlette kingbri 2024-03-10 19:41:08 -04:00
42c0dbe795 Generation: Explicitly release semaphore on disconnect kingbri 2024-03-10 17:53:55 -04:00
2025a1c857 Requirements: Unpin uvicorn kingbri 2024-03-10 17:48:43 -04:00
bbb1a4ec20 Tree: Format kingbri 2024-03-10 17:45:09 -04:00
045262f51f Logging: Loglevel INFO kingbri 2024-03-10 17:33:37 -04:00
d45e847c7a API: Fix disconnect handling on streaming responses kingbri 2024-03-10 17:31:47 -04:00
6b4f100db2 Logger: Escape tags kingbri 2024-03-10 01:24:50 -05:00
e33971859b Requirements: Pin uvicorn kingbri 2024-03-10 01:18:16 -05:00
a69ee976f0 API: Let the user know if a disconnect occurred kingbri 2024-03-09 15:47:46 -05:00
c77259bfbb Logger: Fix reformatting of message kingbri 2024-03-09 15:40:37 -05:00
4d09226364 Logging: Fix Uvicorn hook kingbri 2024-03-08 17:56:48 -05:00
2295b12643 Progress: Fix bar with draft models kingbri 2024-03-08 01:48:06 -05:00
c9b4b7c509 Tree: Format kingbri 2024-03-08 01:00:04 -05:00
cad72315f4 Init: Switch to display redoc endpoint kingbri 2024-03-08 00:53:26 -05:00
ef2dc326f5 Logging: Fix inconsistent formatting kingbri 2024-03-08 00:51:07 -05:00
228c227c1e Logging: Switch to loguru kingbri 2024-03-07 23:20:17 -05:00
fe0ff240e7 Progress: Switch to Rich kingbri 2024-03-06 00:37:31 -05:00
39617adb65 Requirements: Update Exllamav2 kingbri 2024-03-06 22:29:55 -05:00
47c42a23d4 Merge pull request #72 from djmaze/patch-1 Brian Dashore 2024-03-06 01:13:37 -05:00
9a007c4707 Model: Add support for Q4 cache kingbri 2024-03-06 00:57:41 -05:00
0b25c208d6 API: Fix error reporting kingbri 2024-03-05 18:16:02 -05:00
165cc6fc2d API: Remove unnecessary endpoint kingbri 2024-03-04 23:02:12 -05:00
d2c6ae2d35 API: Back to async kingbri 2024-03-04 22:59:10 -05:00
b0c295dd2f API: Add more methods to semaphore kingbri 2024-03-03 01:22:34 -05:00
c82697fef2 API: Fix issues with concurrent requests and queueing kingbri 2024-02-26 22:52:28 -05:00
de91eade4b Merge pull request #75 from DocShotgun/main Brian Dashore 2024-03-03 01:30:45 -05:00
8245488926 Additional clarification for override_base_seq_len DocShotgun 2024-03-02 09:29:50 -08:00
4afb4137f7 Remove explicit pytorch & exllamav2 in Dockerfile Martin Honermeyer 2024-02-25 18:03:01 +01:00
fc857893ee Model: Remove Exllamav2 patches kingbri 2024-02-24 01:19:57 -05:00
73a1d9ef78 Model: Fix imports kingbri 2024-02-24 00:46:02 -05:00
f6d749c771 Model: Add EBNF grammar support kingbri 2024-02-23 01:36:10 -05:00
57b3d69949 API + Model: Add support for JSON schema constraints kingbri 2024-02-22 01:36:46 -05:00
ccd41d720d Requirements: Bump ExllamaV2 kingbri 2024-02-24 01:19:40 -05:00
360802762c Model: Fix logit bias token checks kingbri 2024-02-22 21:44:15 -05:00
5a23b9ebc9 Tree: Format kingbri 2024-02-22 01:28:30 -05:00
bee26a2f2c API: Auto-unload on a load request kingbri 2024-02-21 23:00:11 -05:00
368eb2e2d9 Update README kingbri 2024-02-20 00:18:36 -05:00
a19a4eb1be Model: Format kingbri 2024-02-18 18:31:31 -05:00
7def32e4de Model: Fix logit bias handling kingbri 2024-02-18 18:30:58 -05:00
aa34b2e5fd Model: Prefer auto over manual GPU split kingbri 2024-02-17 00:21:48 -05:00
ea00a6bd45 Requirements: Update Exllamav2 kingbri 2024-02-14 21:47:40 -05:00
cce97deea5 Model: Switch logprobs to use post-sampling kingbri 2024-02-11 00:34:51 -05:00
949248fb94 Config: Add experimental torch cuda malloc backend kingbri 2024-02-14 21:44:04 -05:00
664e2c417e Model: Fix GPU split args loading kingbri 2024-02-14 17:42:20 -05:00
a79c42ff4c Sampling: Make validators simpler kingbri 2024-02-11 15:22:43 -05:00
f627485534 OAI: Fix completion token fetching kingbri 2024-02-11 01:12:13 -05:00
7e730e3507 Sampling: Add universal validation system kingbri 2024-02-10 01:47:13 -05:00
9f1d891490 Packages: Fix exllamav2 version check kingbri 2024-02-10 14:00:26 -05:00
8d8cf5dc69 Model: Fix dynatemp fallback kingbri 2024-02-10 12:02:31 -05:00
17636ed899 Create pull request template Brian Dashore 2024-02-09 14:53:08 -05:00
c3601bdd18 Issues: Disable blank issues Brian Dashore 2024-02-09 14:48:03 -05:00
aa56ff829f Add issue templates Brian Dashore 2024-02-09 14:43:33 -05:00
2f568ff573 Config: Expose auto GPU split reserve config kingbri 2024-02-08 22:08:51 -05:00
43bba526bf Model: Fix logprobs unwrapping kingbri 2024-02-08 19:28:19 -05:00
c7428f0bcd API: Add logprobs for chat completions kingbri 2024-02-08 01:16:38 -05:00
c02fe4d1db API: Fix response creation kingbri 2024-02-07 23:36:08 -05:00
0af6a38af3 Model: Add logprobs support kingbri 2024-02-07 21:41:15 -05:00
2642ef7156 OAI: Update logprobs type kingbri 2024-02-03 01:06:43 -05:00
284f20263f API: Clean up tokenizing endpoint kingbri 2024-02-05 00:20:10 -05:00
bb48f77ca1 Neutralize samplers (#59) AliCat 2024-02-07 22:23:09 -07:00
321c9a1ea9 Requirements: Fix FA2 version number kingbri 2024-02-07 21:37:30 -05:00
58590a6c57 Config: Add option to force streaming off kingbri 2024-02-07 21:08:21 -05:00
d0027bce32 Requirements: Update flash attention 2 for Windows kingbri 2024-02-07 20:44:23 -05:00
c0ad647fa7 Model: Auto-detect a one GPU setup and fix gpu_split_auto kingbri 2024-02-06 22:58:55 -05:00
849179df17 Model: Make loading use less VRAM kingbri 2024-02-06 22:29:56 -05:00
fedebadc81 Model: Fix generate window fallback kingbri 2024-02-06 14:48:42 -05:00
543a9b68c8 Requirements: Update Exllamav2 to 0.0.13.post1 kingbri 2024-02-04 21:25:57 -05:00
f10a5cfee6 Auth: Create keys on different exception kingbri 2024-02-04 01:56:42 -05:00
fa2acb2828 Adds aliases for min_temp and max_temp (#58) erinmaybe 2024-02-03 21:51:29 -05:00
a769d90bad Args: Fix developer group kingbri 2024-02-03 00:16:47 -05:00
f1ea15d77e Model: Remove backwards compatability hacks kingbri 2024-02-02 23:40:10 -05:00
6eeb62b82c Requirements: Update exllamav2, torch, and FA2 kingbri 2024-02-02 22:24:44 -05:00
1919bf7705 Launch: Make exllamav2 requirement more friendly kingbri 2024-02-02 22:04:35 -05:00
b827bcbb44 Sampling: Cleanup and update kingbri 2024-02-01 12:58:55 -05:00
2ea063cea9 Tree: Require exllamav2 version for startup kingbri 2024-02-01 12:40:24 -05:00
d3781920b3 OAI: Split up utility functions kingbri 2024-02-01 00:26:42 -05:00
634d299fd9 Sampling: Fix smoothing factor default fallback kingbri 2024-02-02 23:35:15 -05:00
d7c18855e7 added quadratic sampling (#56) Alexander Abushady 2024-02-02 22:12:59 -05:00
4a7b8b1b7a Samplers: Add dynamic temperature kingbri 2024-01-31 01:20:59 -05:00
3605067898 Requirements: Don't use torch 2.2 kingbri 2024-01-29 23:30:10 -05:00
751627e571 OAI: Add fasttensors to model load endpoint kingbri 2024-01-25 01:01:29 -05:00
fc4570220c API + Model: Add new parameters and clean up documentation kingbri 2024-01-25 00:11:30 -05:00
90fb41a77a Model: Fix prompt template initialization kingbri 2024-01-24 23:36:35 -05:00
740b0215dd Model: Dynamically scale generate_window kingbri 2024-01-24 01:26:38 -05:00

... 8 9 10 11 12 ...