turboderp-org/exllamav2
1
0
Fork 0
mirror of https://github.com/turboderp-org/exllamav2.git synced 2026-07-03 13:57:15 +00:00
Code Issues Packages Projects Releases Wiki Activity

Commit Graph

  • 7dc12af3a8 Update README.md master turboderp 2026-03-04 14:12:19 +01:00
  • da07facd10 Actions: Update for torch 2.9 kingbri 2025-12-09 15:12:18 -05:00
  • 60d02227af Actions: Attempt amdgpu repo list kingbri 2025-08-16 17:40:38 -04:00
  • e7cc61cce7 Actions: Attempt amdgpu repo list actions kingbri 2025-08-16 17:40:38 -04:00
  • c322d0ed2e Actions: Replace focal with jammy for ROCm kingbri 2025-08-16 16:04:35 -04:00
  • cece58d925 Actions: Update for torch 2.8 kingbri 2025-08-16 15:47:26 -04:00
  • 6a2d831140 Actions: Remove sentencepiece from other workflows v0.3.2 kingbri 2025-07-13 18:11:53 -04:00
  • 586bfc6744 ExllamaV2: Bump version kingbri 2025-07-13 18:06:03 -04:00
  • 186171e077 Merge branch 'dev' kingbri 2025-07-13 18:05:44 -04:00
  • 0f2eae558e Actions: Remove sentencepiece from build steps kingbri 2025-07-09 11:34:16 -04:00
  • d2a810f2ae Fix rounding error in Pixtral image preprocessor dev turboderp 2025-06-05 01:32:36 +02:00
  • 2ca8281c31 Merge branch 'dev' turboderp 2025-05-29 00:33:35 +02:00
  • 0efb999c24 Merge remote-tracking branch 'origin/dev' into dev turboderp 2025-05-29 00:33:05 +02:00
  • b311d0aca4 Remove sentencepiece dep from setup.py turboderp 2025-05-29 00:32:51 +02:00
  • 2b20c24dcd Merge branch 'dev' v0.3.1 kingbri 2025-05-27 11:13:29 -04:00
  • a08ef4f1ed ExllamaV2: Bump version kingbri 2025-05-27 11:13:06 -04:00
  • 97e4fd90f1 Fail if tokenizer.json not found turboderp 2025-05-27 17:10:35 +02:00
  • a811641c3b Optimize paged cache defrag turboderp 2025-05-27 00:52:44 +02:00
  • 1adff7d827 Remove SentencePiece support turboderp 2025-05-14 15:30:57 +02:00
  • a87ea02830 Remove SentencePiece support turboderp 2025-05-14 13:41:09 +02:00
  • 0a3d4200e1 Actions: Build rocm only kingbri 2025-05-12 14:17:24 -04:00
  • bb4206d5bc Ext: Fix register call for float kingbri 2025-05-12 14:15:56 -04:00
  • 0a7733110e Ext: Fix CUDA type cast kingbri 2025-05-12 12:36:40 -04:00
  • 9d621509ab Project: Bump version v0.3.0 kingbri 2025-05-12 10:11:57 -04:00
  • aa2d5aa471 Merge pull request #788 from turboderp-org/dev Brian 2025-05-12 10:10:06 -04:00
  • c820539b79 Actions: Add redirects to CUDA downloads kingbri 2025-05-09 14:25:10 -04:00
  • b4c6b39590 Actions: Update CUDA install commands kingbri 2025-05-09 12:34:25 -04:00
  • 89d17b7ba3 Actions: Migrate to temp windows build action kingbri 2025-05-07 13:58:52 -04:00
  • 3d9fde2dd0 Actions: Go back to VS 17.9 kingbri 2025-05-07 13:57:25 -04:00
  • e312b74f15 Fix unload() for vision tower turboderp 2025-05-07 19:08:29 +02:00
  • 747fbadca9 Merge branch 'master' into dev turboderp 2025-05-03 18:29:14 +02:00
  • 68976a07d7 Add basic support for Qwen3MoE turboderp 2025-05-01 20:23:33 +02:00
  • b422a85c47 Merge branch 'master' into dev turboderp 2025-04-29 20:44:36 +02:00
  • a3440098a4 Add Qwen3ForCausalLM turboderp 2025-04-29 20:44:10 +02:00
  • af04f9a393 Merge pull request #776 from kingbri1/master Brian 2025-04-26 13:30:47 -04:00
  • be765fab7a Actions: Fix CUDA build kingbri 2025-04-24 21:44:10 -04:00
  • abbece178e Temporary build action for Torch 2.7 turboderp 2025-04-24 18:39:46 +02:00
  • 263c758ae5 Actions: Update install methods (#775) Brian 2025-04-24 12:35:36 -04:00
  • 0374e367aa Upgrade large runner actions to 22.04 turboderp 2025-04-24 02:34:40 +02:00
  • 569dcb2ec6 Change wheels to CUDA 12.8.1 turboderp 2025-04-24 01:58:31 +02:00
  • 68e2b92d79 Build on ubuntu-22.04, enable Hopper and Blackwell on Torch 2.7.0 wheels v0.2.9 turboderp 2025-04-24 00:52:11 +02:00
  • 7b2e4d8ddc Actions: Add Torch 2.7 (#773) Brian 2025-04-23 18:39:31 -04:00
  • ea6cc68ac1 Added Fix for Loading unquantized Huggingface models. (#771) RaahimSiddiqi 2025-04-24 03:38:10 +05:00
  • 3a90264940 Bump to v0.2.9 turboderp 2025-04-24 00:36:39 +02:00
  • 2c170bb6c6 Gemma3 local RoPE fixes turboderp 2025-04-24 00:31:40 +02:00
  • 2d48ccd23e Merge branch 'dev' turboderp 2025-04-18 23:29:41 +02:00
  • 9244003a40 Add support for Mistral 3.1 VLM turboderp 2025-04-18 22:47:47 +02:00
  • 68f7461985 Optional attn bias for GLM4 turboderp 2025-04-16 01:24:45 +02:00
  • 6a5d303355 Merge remote-tracking branch 'origin/dev' into dev turboderp 2025-04-15 18:57:47 +02:00
  • de19cbcc59 Add GLM4 architecture turboderp 2025-04-15 18:57:29 +02:00
  • 09c18e9c47 Added banned_strings parameter to the generator. (#756) RaahimSiddiqi 2025-04-12 01:12:17 +05:00
  • 61450b4860 concatenate the sin and cos tensors (#758) MikeRoz47 2025-04-11 16:11:13 -04:00
  • b148bb42b8 Fix Gemma3 head norm (RMS) turboderp 2025-04-11 00:18:06 +02:00
  • d471d44f01 Gemma3 local RoPE fixes turboderp 2025-04-10 22:15:20 +02:00
  • a03db457ef Fix: Prioritize default head_dim when provided by architecture (Gemma3) over computed head_dim turboderp 2025-03-15 11:52:51 +01:00
  • 385a5162ba Fix: Correctly read query_pre_attn_scalar from text_config (Gemma3) turboderp 2025-03-15 11:01:33 +01:00
  • 17762c177f Merge remote-tracking branch 'origin/dev' into dev turboderp 2025-03-15 01:37:43 +01:00
  • 6f7623ff0e Update examples turboderp 2025-03-15 00:15:08 +01:00
  • 77a1e2cb0c Warn instead of failing for unsupported vision model turboderp 2025-03-15 00:13:52 +01:00
  • 578fd4234f Support Gemma3 (vision) turboderp 2025-03-15 00:10:31 +01:00
  • c0267e37fe Support Gemma3 (text) turboderp 2025-03-14 23:45:48 +01:00
  • 565339101b Allow text model to use Q/K norm while vision model doesn't turboderp 2025-03-14 23:44:05 +01:00
  • 07afc90788 Tensor renaming kludge (Gemma3 has one _weight tensor) turboderp 2025-03-14 23:37:57 +01:00
  • e2fa480595 Auto expand Q/K norm weight to match number of heads turboderp 2025-03-14 23:21:18 +01:00
  • a88c18cac1 Add architecture-specific config defaults (Gemma3 config.json is incomplete) turboderp 2025-03-14 23:20:30 +01:00
  • b6c1912f29 Respect norm_constant_bias in Q/K norms (Gemma3) turboderp 2025-03-14 23:17:50 +01:00
  • 4b5dbecdc1 Allow key prefix for lm_head (Gemma3) turboderp 2025-03-14 23:16:51 +01:00
  • 4844f3873c Upcast MM embeddings when residual is FP32 turboderp 2025-03-14 23:16:00 +01:00
  • fe51a8f4b5 Correctly include Q/K norms when compiling model turboderp 2025-03-14 23:15:15 +01:00
  • 38f4d7c87d Allow loading transposed unquantized linear layer turboderp 2025-03-14 23:14:40 +01:00
  • 9669fa33c9 Allow component models to use learned pos embeddings without regarding LLM max_seq_len turboderp 2025-03-14 23:13:59 +01:00
  • 7b05acd233 Allow per-layer RoPE theta turboderp 2025-03-14 23:12:50 +01:00
  • 23395dfa42 Fix FP32 residual for paged attn turboderp 2025-03-14 23:09:31 +01:00
  • eaf8ad1041 Update chat.py, include multi-line input support and context clearing through input (#738) Thomas 2025-03-10 15:28:33 +01:00
  • d8fa1a8250 Support partial_rotary_factor (Phi-4 mini) turboderp 2025-02-28 08:51:11 +01:00
  • 2e630aefdd Fix alt pos embeddings and block diagonal mask when flash-attn is disabled turboderp 2025-02-13 22:13:48 +01:00
  • 1a80d38891 Update build actions turboderp 2025-02-08 03:29:05 +01:00
  • f1c4126045 Update build actions turboderp 2025-02-08 03:14:32 +01:00
  • f98a7b7099 Update build actions turboderp 2025-02-08 03:08:45 +01:00
  • 096076b3fd Update build actions turboderp 2025-02-08 02:49:21 +01:00
  • 0f4a9f0042 Update build actions turboderp 2025-02-08 01:36:51 +01:00
  • f3de3cbd34 Update build actions turboderp 2025-02-08 01:05:22 +01:00
  • 94e57904bc Update build actions v0.2.8 turboderp 2025-02-08 00:57:29 +01:00
  • 3a9618d471 Update build actions turboderp 2025-02-08 00:44:44 +01:00
  • 3486f9eb71 Merge branch 'refs/heads/dev' turboderp 2025-02-08 00:26:52 +01:00
  • 6e4a84a1e3 Bump to 0.2.8 turboderp 2025-02-08 00:26:30 +01:00
  • d05fbcc854 Fix Pixtral regression turboderp 2025-02-04 21:01:23 +01:00
  • 96b2f9df77 Add Qwen2.5 mode to grounding demo turboderp 2025-01-29 22:39:38 +01:00
  • cce6f95cd3 Initial support for Qwen2.5-VL turboderp 2025-01-29 02:55:44 +01:00
  • d0413b06f8 Check length of gpu_split in model_init turboderp 2025-01-09 11:36:25 +01:00
  • c8fa853c89 Test script: Allow --eval_rows in wiki2 ppl test turboderp 2025-01-09 11:14:48 +01:00
  • 318435db81 Sampler: Remove superfluous pre-sort pass turboderp 2025-01-09 11:14:19 +01:00
  • d302fa3d37 Optimizer: Ensure weight budget is fully used up turboderp 2025-01-09 11:14:03 +01:00
  • b400394f06 Update build actions turboderp 2025-01-09 11:13:03 +01:00
  • b9c025b4b1 Enable large runner turboderp 2024-12-30 05:47:11 +01:00
  • c41acd5c11 Extra ROCm 6.2 actions turboderp 2024-12-30 04:31:44 +01:00
  • 7c08c6df71 Deactivate mamba turboderp 2024-12-30 04:07:50 +01:00
  • c8075cabf4 Update conda-incubator turboderp 2024-12-30 03:58:55 +01:00
  • ae241a9af5 Fix video example v0.2.7 turboderp 2024-12-30 02:24:49 +01:00
  • 1ef618389b Bump to v0.2.7 turboderp 2024-12-30 02:19:19 +01:00