Offload only activated experts to the GPU (#698)

Browse Source

* Offload only activated experts

* This seems to do the trick for -fmoe

* Do not recalculate activated expers for fused up/gate

* Log out of bounds access details

* Add a command line argument

---------

Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>

...

This commit is contained in:

Kawrakow

2025-09-04 12:22:30 +02:00

committed by GitHub

parent 144d456717

commit 13c3b6412e

8 changed files with 155 additions and 45 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

2439

ggml/src/ggml-backend.cpp Normal file

File diff suppressed because it is too large Load Diff