turboderp-org/exllamav2
1
0
Fork 0
mirror of https://github.com/turboderp-org/exllamav2.git synced 2026-04-20 14:29:28 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,378 Commits 3 Branches 43 Tags
94e57904bc02388edb7bec7ad35b88db9759d032
Commit Graph

1378 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
turboderp
94e57904bc Update build actions v0.2.8 2025-02-08 00:57:29 +01:00
turboderp
3a9618d471 Update build actions 2025-02-08 00:44:44 +01:00
turboderp
3486f9eb71 Merge branch 'refs/heads/dev' 2025-02-08 00:26:52 +01:00
turboderp
6e4a84a1e3 Bump to 0.2.8 2025-02-08 00:26:30 +01:00
turboderp
d05fbcc854 Fix Pixtral regression 2025-02-04 21:01:23 +01:00
turboderp
96b2f9df77 Add Qwen2.5 mode to grounding demo 2025-01-29 22:41:36 +01:00
turboderp
cce6f95cd3 Initial support for Qwen2.5-VL 2025-01-29 03:03:36 +01:00
turboderp
d0413b06f8 Check length of gpu_split in model_init 2025-01-09 11:36:25 +01:00
turboderp
c8fa853c89 Test script: Allow --eval_rows in wiki2 ppl test 2025-01-09 11:14:48 +01:00
turboderp
318435db81 Sampler: Remove superfluous pre-sort pass 2025-01-09 11:14:19 +01:00
turboderp
d302fa3d37 Optimizer: Ensure weight budget is fully used up 2025-01-09 11:14:03 +01:00
turboderp
b400394f06 Update build actions 2025-01-09 11:13:03 +01:00
turboderp
b9c025b4b1 Enable large runner 2024-12-30 05:47:11 +01:00
turboderp
c41acd5c11 Extra ROCm 6.2 actions 2024-12-30 04:31:44 +01:00
turboderp
7c08c6df71 Deactivate mamba 2024-12-30 04:07:50 +01:00
turboderp
c8075cabf4 Update conda-incubator 2024-12-30 04:00:15 +01:00
turboderp
ae241a9af5 Fix video example v0.2.7 2024-12-30 02:24:49 +01:00
turboderp
1ef618389b Bump to v0.2.7 2024-12-30 02:19:19 +01:00
turboderp
b010cb950f Fix compilation errors on aarch64 2024-12-29 20:30:59 +01:00
turboderp
fb5000ac62 Don't compile AVX2 functions when building without AVX2 support 2024-12-29 19:05:54 +01:00
turboderp
82bb648517 Fix Granite3 logit scaling 2024-12-27 19:54:19 +01:00
turboderp
bee449d116 Support Granite 3.x arch 2024-12-27 19:11:21 +01:00
turboderp
ab4d9e15eb Chat example Granite3 template 2024-12-27 18:32:46 +01:00
turboderp
ebfefc4bed Support Cohere2 architecture 2024-12-25 20:14:45 +01:00
turboderp
d815f5f9e1 Fix RoPE alpha after refactor in #4d25874 2024-12-25 18:09:11 +01:00
nintwentydo
b2dd5a7e06 Modify handling for Pixtral Large model params (#701) 
* Modify handling for Pixtral Large model params.

* Fix multimodal_projector_bias to default to True if not in model config.json
 2024-12-21 19:58:41 +01:00
turboderp
cf7fcd18d2 Fix chat example system prompt 2024-12-18 07:52:09 +01:00
turboderp
f76bc8537a Read number of vision tower layers from config for Pixtral (fix Pixtral-Large) 2024-12-18 01:29:20 +01:00
turboderp
4061c24373 Qwen2-VL: Basic video support 2024-12-15 23:32:41 +01:00
turboderp
c78d9027aa Fix ChatML template in multimodal example 2024-12-15 21:38:40 +01:00
turboderp
9934f06442 Refactoring 2024-12-15 21:37:29 +01:00
turboderp
edf1a3575a Util function to get byte size of MM embeddings object 2024-12-09 23:22:42 +01:00
turboderp
254e76b178 Merge remote-tracking branch 'origin/dev' into dev 2024-12-09 20:15:23 +01:00
turboderp
8bb283d319 Cleanup build actions 2024-12-09 20:14:35 +01:00
turboderp
f4119aec5b Fix background filter eval when draft model used 2024-12-09 20:12:50 +01:00
DocShotgun
af69ce9458 Prevent UnboundLocalError when loading with yarn/su with short ctx len (#694) 
* scaling_factor is left unbound when the requested max_seq_len < the model's original unscaled max_seq_len
 2024-12-08 21:24:45 +01:00
turboderp
4f83f52d7d Merge branch 'refs/heads/dev' v0.2.6 2024-12-07 15:56:16 +01:00
turboderp
15b5df784a Cleanup build actions 2024-12-07 15:55:53 +01:00
turboderp
ebaf819bc0 Merge remote-tracking branch 'origin/dev' into dev 2024-12-07 15:55:33 +01:00
turboderp
83a57c74ed Bump to v0.2.6 2024-12-07 15:55:11 +01:00
turboderp
ba9774f1c8 Enable noise tokens for Qwen2-VL quantizatino 2024-12-07 15:53:52 +01:00
turboderp
c55656cc0c Fix system RAM consumption while quantizing, fixes #692 2024-12-05 21:16:36 +01:00
turboderp
c86f62c3b8 Ensure MRoPE ID tensor is contiguous 2024-12-05 18:02:02 +01:00
Philipp Emanuel Weidmann
db78601226 Prevent NPE in deallocate_pages (#688) 
Prevent NPE in `deallocate_pages`

If `deallocate_pages` is called on a job for which `allocate_pages`
has not been called (see `iterate_start_jobs` for conditions under
which this is true), `allocated_pages` is `None`, raising a NPE
when attempting to iterate.

In particular, this prevents `clear_queue` from working. In
practice, this problem readily occurs when starting a few jobs
and then calling `clear_queue`.
 2024-12-01 22:02:32 +01:00
turboderp
663eea1b53 Fix 64-bit dtype for MSVC 2024-12-01 20:09:40 +01:00
turboderp
bc7db9395d Merge remote-tracking branch 'origin/master' v0.2.5 2024-12-01 14:29:44 +01:00
turboderp
e3b5549e0b Bump to v0.2.5 2024-12-01 14:21:59 +01:00
turboderp
fa7e89c197 Update example 2024-12-01 14:20:33 +01:00
turboderp
48e6306193 Update chat example, prompt formats 2024-11-30 13:31:35 +01:00
turboderp
1f685bd8d3 Update grounding demo 2024-11-23 14:46:51 +01:00