rattus
d531e3fb2a
model_patcher: Improve dynamic offload heuristic ( #12759 )
...
Define a threshold below which a weight loading takes priority. This
actually makes the offload consistent with non-dynamic, because what
happens, is when non-dynamic fills ints to_load list, it will fill-up
any left-over pieces that could fix large weights with small weights
and load them, even though they were lower priority. This actually
improves performance because the timy weights dont cost any VRAM and
arent worth the control overhead of the DMA etc.
2026-03-04 07:47:44 -08:00
..
2026-02-01 01:01:11 -05:00
2025-11-23 04:55:22 -05:00
2026-02-20 22:52:32 -08:00
2025-01-18 05:27:58 -05:00
2025-09-04 20:36:20 -04:00
2026-02-11 14:53:42 -05:00
2026-03-02 19:43:47 -05:00
2025-07-16 14:42:07 -04:00
2024-06-27 18:43:11 -04:00
2026-01-21 23:03:51 -05:00
2026-02-28 01:18:40 -05:00
2026-02-15 20:30:09 -05:00
2026-03-01 19:18:56 -08:00
2024-07-30 14:41:13 -04:00
2026-01-12 17:05:54 -05:00
2023-08-18 11:13:29 -04:00
2025-03-06 00:24:43 -05:00
2024-07-17 13:12:50 -04:00
2026-01-12 17:05:54 -05:00
2024-11-21 08:38:23 -05:00
2025-04-05 07:01:01 -04:00
2026-02-01 01:01:11 -05:00
2026-02-26 01:30:31 -05:00
2026-03-01 09:38:30 -08:00
2026-02-13 22:23:52 -05:00
2025-01-24 06:15:54 -05:00
2024-08-12 23:20:57 -04:00
2026-01-14 00:49:38 -05:00
2025-07-06 07:07:39 -04:00
2026-01-01 22:06:14 -05:00
2026-03-02 19:43:47 -05:00
2026-02-13 15:35:13 -05:00
2026-02-26 18:19:19 -05:00
2026-02-21 10:52:57 -08:00
2026-03-02 19:43:47 -05:00
2026-03-02 19:43:47 -05:00
2026-03-03 18:19:40 -08:00
2026-03-04 07:47:44 -08:00
2026-02-25 23:38:46 -05:00
2025-10-23 21:21:14 -04:00
2026-02-27 19:05:51 -05:00
2023-09-13 11:38:20 -04:00
2025-10-15 16:47:26 -07:00
2026-02-01 08:42:32 -08:00
2025-09-13 18:03:34 -04:00
2026-01-14 00:49:38 -05:00
2026-02-16 20:09:24 -05:00
2026-01-23 19:50:48 -05:00
2026-02-28 16:50:18 -05:00
2026-02-28 16:50:18 -05:00
2024-07-30 14:41:13 -04:00
2026-02-19 22:57:44 -05:00
2026-02-28 16:50:18 -05:00
2025-04-25 19:36:00 -04:00
2025-12-05 18:25:31 -05:00
2026-03-03 17:35:22 -05:00
2026-03-03 18:19:40 -08:00
2026-02-01 01:01:11 -05:00