rattus
d297a749a2
dynamic_vram: Fix windows Aimdo crash + Fix LLM performance ( #12408 )
...
* model_management: lazy-cache aimdo_tensor
These tensors cosntructed from aimdo-allocations are CPU expensive to
make on the pytorch side. Add a cache version that will be valid with
signature match to fast path past whatever torch is doing.
* dynamic_vram: Minimize fast path CPU work
Move as much as possible inside the not resident if block and cache
the formed weight and bias rather than the flat intermediates. In
extreme layer weight rates this adds up.
2026-02-11 14:50:16 -05:00
..
2026-02-01 01:01:11 -05:00
2025-11-23 04:55:22 -05:00
2026-01-27 13:03:29 -08:00
2025-01-18 05:27:58 -05:00
2025-09-04 20:36:20 -04:00
2026-02-01 01:01:11 -05:00
2026-02-10 22:04:32 -05:00
2025-07-16 14:42:07 -04:00
2024-06-27 18:43:11 -04:00
2026-01-21 23:03:51 -05:00
2026-02-09 19:45:56 -05:00
2026-02-10 21:45:19 -05:00
2023-06-13 10:11:33 -04:00
2026-02-01 01:01:11 -05:00
2024-07-30 14:41:13 -04:00
2026-01-12 17:05:54 -05:00
2023-08-18 11:13:29 -04:00
2023-04-01 23:19:15 -04:00
2025-03-06 00:24:43 -05:00
2024-07-17 13:12:50 -04:00
2023-04-01 23:19:15 -04:00
2026-01-12 17:05:54 -05:00
2024-11-21 08:38:23 -05:00
2025-04-05 07:01:01 -04:00
2026-02-01 01:01:11 -05:00
2025-08-04 04:20:12 -04:00
2025-12-30 14:40:42 -08:00
2026-02-01 01:01:11 -05:00
2025-01-24 06:15:54 -05:00
2024-08-12 23:20:57 -04:00
2026-01-14 00:49:38 -05:00
2025-07-06 07:07:39 -04:00
2026-01-01 22:06:14 -05:00
2026-02-03 00:06:18 -05:00
2025-09-01 18:54:02 -04:00
2026-02-03 13:54:23 -05:00
2026-02-01 01:01:11 -05:00
2026-02-10 22:04:32 -05:00
2026-02-03 00:06:18 -05:00
2026-02-11 14:50:16 -05:00
2026-02-11 14:50:16 -05:00
2025-10-08 17:49:02 -04:00
2025-10-23 21:21:14 -04:00
2026-02-11 14:50:16 -05:00
2023-09-13 11:38:20 -04:00
2025-10-15 16:47:26 -07:00
2026-02-01 08:42:32 -08:00
2025-09-13 18:03:34 -04:00
2026-01-14 00:49:38 -05:00
2025-08-15 00:22:26 -04:00
2026-01-23 19:50:48 -05:00
2026-02-10 21:45:19 -05:00
2026-02-01 01:01:11 -05:00
2024-07-30 14:41:13 -04:00
2026-02-03 00:06:18 -05:00
2026-02-10 13:37:46 -05:00
2025-04-25 19:36:00 -04:00
2025-12-05 18:25:31 -05:00
2026-02-06 20:14:52 -05:00
2026-02-10 13:37:17 -05:00
2026-02-01 01:01:11 -05:00