theroyallab/tabbyAPI
1
0
Fork 0
mirror of https://github.com/theroyallab/tabbyAPI.git synced 2026-07-13 02:27:11 +00:00
Code Issues Packages Projects Releases Wiki Activity

Commit Graph

  • 9fc3fc4c54 OAI: Amend comments kingbri 2024-07-11 14:10:24 -04:00
  • 1f46a1130c OAI: Restrict list permissions for API keys kingbri 2024-07-11 14:06:03 -04:00
  • 10890913b8 Auth: Revert x-admin-key allowance in API key check kingbri 2024-07-11 12:33:06 -04:00
  • dfb4c51d5f OAI: Fix function idioms kingbri 2024-07-11 11:26:50 -04:00
  • b9a58ff01b Auth: Make key permission check work on Requests kingbri 2024-07-11 11:16:24 -04:00
  • 279e900ea5 Add on the fly model loading to requests Colin Kealty 2024-06-04 13:35:48 -04:00
  • ff15eed85d Update README.md Brian Dashore 2024-07-08 21:26:11 +00:00
  • 5c293499bd OAI: Reorder functions kingbri 2024-07-08 15:27:08 -04:00
  • 521d21b9f2 OAI: Add return types for docs kingbri 2024-07-08 15:23:41 -04:00
  • 62e495fc13 Model: Grammar: Fix lru_cache clear function kingbri 2024-07-08 15:10:15 -04:00
  • 17438288c7 Merge pull request #146 from theroyallab/tokenizer_data_fix Brian Dashore 2024-07-08 15:08:29 -04:00
  • c7ce97f119 Tree: Ruff lint kingbri 2024-07-08 15:06:28 -04:00
  • 8a81fe2eb4 Actions: Add Github Pages deploy kingbri 2024-07-08 14:13:49 -04:00
  • 6613e38436 Main: Make openapi export store locally kingbri 2024-07-08 14:52:17 -04:00
  • ae66e8f9ba Ruff: Lint kingbri 2024-07-08 13:44:12 -04:00
  • b907421285 Main: Fix launch if EXPORT_OPENAPI is unset kingbri 2024-07-08 13:41:44 -04:00
  • a59e8ef9e7 Main: Make EXPORT_OPENAPI only work if true or 1 kingbri 2024-07-08 12:50:49 -04:00
  • e58e197f0b Ruff: Remove deprecated rule E999 kingbri 2024-07-08 12:36:15 -04:00
  • 933268f7e2 API: Integrate OpenAPI export script kingbri 2024-07-08 12:34:32 -04:00
  • e97ad9cb27 RUFF turboderp 2024-07-08 03:51:14 +02:00
  • 8bbce3455c RUFF turboderp 2024-07-08 03:49:26 +02:00
  • 5e82b7eb69 API: Add standalone method to fetch OpenAPI docs kingbri 2024-07-07 21:34:18 -04:00
  • 4cf79c5ae1 Clear tokenizer_data cache when unloading model turboderp 2024-07-08 03:31:05 +02:00
  • b7e7df1220 Move tokenizer_data cache to global scope turboderp 2024-07-08 02:54:49 +02:00
  • 4d0bb1ffc3 Cache creation tokenizer_data in LMFE turboderp 2024-07-08 00:51:59 +02:00
  • bb8b02a60a Wrap arch_compat_overrides in try block turboderp 2024-07-07 07:54:05 +02:00
  • 773639ea89 Model: Fix flash-attn checks kingbri 2024-07-06 20:57:10 -04:00
  • 27d2d5f3d2 Config + Model: Allow for default fallbacks from config for model loads kingbri 2024-07-06 17:29:59 -04:00
  • d03752e31b Issues: Fix template kingbri 2024-06-23 21:52:01 -04:00
  • 45fae89af6 Update README kingbri 2024-06-23 21:49:47 -04:00
  • c5ea2abe24 Dependencies: Update ExllamaV2 kingbri 2024-06-23 21:45:04 -04:00
  • d85b526644 Dependencies: Pin numpy kingbri 2024-06-23 21:40:09 -04:00
  • 107436f601 Dependencies: Fix AMD triton (#139) DocShotgun 2024-06-18 06:19:27 -07:00
  • 06ee610a97 Update README Brian Dashore 2024-06-17 03:56:47 +00:00
  • c575105e41 ExllamaV2: Cleanup log placements kingbri 2024-06-16 00:15:05 -04:00
  • 8da7644571 Fix exception unloading models. (#138) Glenn Maynard 2024-06-15 16:44:29 -05:00
  • 85387d97ad Fix disabling flash attention in exl2 config (#136) DocShotgun 2024-06-12 11:00:46 -07:00
  • 156b74f3f0 Revision to paged attention checks (#133) DocShotgun 2024-06-09 08:28:11 -07:00
  • 55d979b7a5 Update dependencies, support Python 3.12, update for exl2 0.1.5 (#134) DocShotgun 2024-06-09 08:27:39 -07:00
  • dcd9428325 Model: Warn if cache size is too small for CFG (#132) DocShotgun 2024-06-05 10:40:14 -07:00
  • e391d84e40 More extensive checks for paged mode support (#121) DocShotgun 2024-06-05 00:33:21 -07:00
  • dbdcb38ad7 Allow either "[" or "{" prefix to support JSON grammar with top level arrays (#129) turboderp 2024-06-04 02:32:39 +02:00
  • e889fa3efe Bump exllamav2 to v0.1.4 (#128) turboderp 2024-06-04 02:32:08 +02:00
  • 6cc3bd9752 feat: list support in message.content (#122) Orion 2024-06-04 01:57:15 +08:00
  • 1951f7521c Forward exceptions from _stream_collector to stream_generate_(chat)_completion (#126) turboderp 2024-06-03 19:42:45 +02:00
  • 0eb8fa5d1e [fix] Bring draft progress and model progress in sync with model loader (#125) turboderp 2024-06-03 19:41:02 +02:00
  • a011c17488 Revert "Forward exceptions from _stream_collector to stream_generate_chat_completion" turboderp 2024-06-02 15:37:37 +02:00
  • 1bb8d1a312 Forward exceptions from _stream_collector to stream_generate_chat_completion turboderp 2024-06-02 15:13:30 +02:00
  • e95e67a000 OAI: Add validation to "n" kingbri 2024-05-28 00:34:32 -04:00
  • e2a8b6e8ae OAI: Add "n" support for streaming generations kingbri 2024-05-27 23:53:25 -04:00
  • c8371e0f50 OAI: Copy gen params for "n" kingbri 2024-05-26 22:54:34 -04:00
  • b944f8d756 OAI: Add "n" for non-streaming generations kingbri 2024-05-26 22:22:51 -04:00
  • 8d31a5aed1 Dependencies: Update Flash Attention 2 kingbri 2024-05-28 00:45:35 -04:00
  • 516b52b341 Merge pull request #112 from DocShotgun/main Brian Dashore 2024-05-27 18:04:43 -04:00
  • 19961f4126 Dependencies: Update ExllamaV2 kingbri 2024-05-27 13:38:07 -04:00
  • 04cbed16e8 Update README kingbri 2024-05-27 13:37:57 -04:00
  • 4087586449 Start: Create config.yml if it doesn't exist kingbri 2024-05-26 21:37:52 -04:00
  • 7084081b1f Tree: Lint DocShotgun 2024-05-26 18:27:30 -07:00
  • 116cf56c87 Model: Auto-round cache size on init kingbri 2024-05-26 21:24:54 -04:00
  • ce5e2ec8de Logging: Clarify new vs cached tokens in prompt processing DocShotgun 2024-05-26 18:21:17 -07:00
  • 3dcae8b023 Merge pull request #111 from DocShotgun/main Brian Dashore 2024-05-26 20:52:21 -04:00
  • bec919e202 Config: Change cache_size description and location kingbri 2024-05-26 20:50:56 -04:00
  • 7ab7ffd562 Tree: Format DocShotgun 2024-05-26 15:48:18 -07:00
  • 767e6a798a API + Model: Add support for specifying k/v cache size DocShotgun 2024-05-26 14:17:01 -07:00
  • d710a1b441 OAI: Switch to background task for disconnect checks kingbri 2024-05-26 13:52:20 -04:00
  • 660f9b8432 OAI: Fix request cancellation behavior kingbri 2024-05-26 13:00:33 -04:00
  • 094c7b1734 Model: Fix paged and FA2 checks kingbri 2024-05-26 11:29:31 -04:00
  • 9fbbc5afca Tree: Swap from map to list comprehensions kingbri 2024-05-25 21:13:58 -04:00
  • 46d0d13914 Model/Grammar: Fix filter append call kingbri 2024-05-25 20:59:45 -04:00
  • a46ee62d03 Model: Clarify warning and device check on load kingbri 2024-05-25 20:56:07 -04:00
  • 47582c2440 Dependencies: Update ExllamaV2 kingbri 2024-05-25 18:32:22 -04:00
  • 43cd7f57e8 API + Model: Add blocks and checks for various load requests kingbri 2024-05-25 18:24:11 -04:00
  • 408c66a1f2 Model: Change FA2 and paged attention checks kingbri 2024-05-24 22:33:47 -04:00
  • c2d3675408 Model: Add min_tokens support kingbri 2024-05-24 21:04:08 -04:00
  • 5f0fb9c4ff Model: Add CFG support kingbri 2024-05-24 21:03:33 -04:00
  • 06ff47e2b4 Model: Use true async jobs and add logprobs kingbri 2024-05-23 21:37:50 -04:00
  • 32ae62feac Model: Add filter support to dynamic gen kingbri 2024-05-23 00:55:22 -04:00
  • 8ccd8fe5f8 Model: Initial dynamic generator support kingbri 2024-05-23 00:13:31 -04:00
  • c474076b22 Concurrency: Remove release_semaphore method kingbri 2024-05-19 10:42:26 -04:00
  • b9fd8555fe Sampling: Copy over iterable overrides kingbri 2024-05-17 21:27:56 -04:00
  • 0e9385e023 API: Fix usage reporting for chat completions kingbri 2024-05-17 00:03:15 -04:00
  • e4bb709305 Model: Fix usage stats in non-streaming gens kingbri 2024-05-12 22:44:50 -04:00
  • 213430a122 Model/Grammar: Remove lmfe checks kingbri 2024-05-12 22:24:28 -04:00
  • b255847c2a Merge pull request #105 from DocShotgun/main Brian Dashore 2024-05-12 22:22:12 -04:00
  • abe411c6fb API + Model: Add support for regex pattern constraints Adds the ability to constrain generation via regex pattern using lm-format-enforcer. DocShotgun 2024-05-12 19:10:43 -07:00
  • 57525219d0 Fix: Properly handle banned_strings and decode_special tokens (#104) Ycros 2024-05-13 06:47:45 +10:00
  • 611f00818b Merge pull request #103 from DocShotgun/main Brian Dashore 2024-05-12 16:47:12 -04:00
  • dad34237ba Samplers: Add example override for generate_window DocShotgun 2024-05-12 00:39:01 -07:00
  • 9463ecfa40 Samplers: Minor fixes for sampler override * Add missing settings to sample_preset.yml * Fix override for skip_special_tokens DocShotgun 2024-05-12 00:31:31 -07:00
  • c8ec742be9 Samplers: Expose skew sampling kingbri 2024-05-12 01:41:01 -04:00
  • 6f4012d20d API: Add preset listing for sampler overrides kingbri 2024-05-12 01:34:51 -04:00
  • b4bc941cbe Tree: Lint kingbri 2024-05-11 22:42:39 -04:00
  • 2da3fb2caf Start: Bump ROCm error version kingbri 2024-05-11 21:57:51 -04:00
  • 7bebc085ec Model: Remove legacy checks kingbri 2024-05-11 19:26:23 -04:00
  • cd78728a77 Dependencies: Update ExllamaV2 kingbri 2024-05-11 19:26:03 -04:00
  • 5432f523cb Merge pull request #102 from DocShotgun/main Brian Dashore 2024-05-10 21:21:57 -04:00
  • 366d57cf45 Tree: Format kingbri 2024-05-10 21:20:41 -04:00
  • 7eee936a3f Model: Remove old code and fix API handling kingbri 2024-05-10 21:20:00 -04:00
  • c0b631ba92 API: Add banned_strings From exllamav2: List of strings that the generator will refuse to output. As soon as a partial match happens, a checkpoint is saved that the generator can rewind to if need be. Subsequent tokens are then held until the full string is resolved (match or no match) and either emitted or discarded, accordingly. DocShotgun 2024-05-10 13:53:55 -07:00
  • a1df22668b API: Add min_tokens Bans the EOS token until the generation reaches a minimum length. This will not prevent the model from otherwise ending the generation early by outputting other stop conditions. DocShotgun 2024-05-10 12:30:17 -07:00