Jaret Burkett c78b1404e3 Deepen offload prefetch pipeline with per-slot events ...

Replace the 2-slot ping-pong + single global "compute-started" event
with a depth-N ring buffer where each transfer waits only on the slot
it's reusing (D layers back) instead of the most-recent compute. Applies
to forward and backward, Linear and Conv. Depth is tunable via
AI_TOOLKIT_OFFLOAD_DEPTH (default 4).

Bit-exact vs non-offload (output, grad_input, weight grads). No speedup
on a bandwidth-bound PCIe link (already saturated at depth 2), but the
cleaner per-slot design removes the fragile shared-event serialization
and lets deeper prefetch help on faster buses.

2026-06-07 16:07:13 -06:00

..

__init__.py

git status

2025-10-01 14:12:17 -06:00

manager_modules.py

Deepen offload prefetch pipeline with per-slot events

2026-06-07 16:07:13 -06:00

manager.py

Fix text encoder offload bug when caching embeddings (#868)

2026-06-07 14:09:32 -06:00

test_memory_manager.py

Deepen offload prefetch pipeline with per-slot events

2026-06-07 16:07:13 -06:00