deepseek-ai/DeepGEMM
1
0
Fork 0
mirror of https://github.com/deepseek-ai/DeepGEMM.git synced 2026-11-05 00:47:26 +00:00
Code Issues 162 Projects Packages Wiki Activity

[PR #68] [MERGED] Correctly flush L2 (+performance impact & upcoming optimization fork) #229

New issue
Closed
opened 2026-10-26 21:15:06 +00:00 by bsakaguchi · 0 comments
bsakaguchi commented 2026-10-26 21:15:06 +00:00
Owner
Copy link

📋 Pull Request Information

Original PR: https://github.com/deepseek-ai/DeepGEMM/pull/68
Author: @ademeure
Created: 3/15/2025
Status: ✅ Merged
Merged: 3/16/2025
Merged by: @soundOfDestiny

Base: main ← Head: flush_l2_pr

📝 Commits (1)

  • 6cbff57 Correctly flush L2, as reconstructing the tensors on every iteration effectively put them in the L2, and gave the GPU enough idle time to avoid thermal throttling in a potentially unrealistic way.

📊 Changes

2 files changed (+35 additions, -11 deletions)

View changed files

📝 deep_gemm/utils.py (+23 -2)
📝 tests/test_core.py (+12 -9)

📄 Description

Thank you for open sourcing this under a MIT license! I was working on optimized PTX matrix multiplications for H100 as you released this, and decided to continue working on it as a fork of your repo instead (not yet public). I've achieved very large performance gains (see below) and I plan to open source it very soon (+ maybe try to get a modded nanogpt speedrun record with it). I hope it can be useful to the wider community worldwide.

However as part of this work I have discovered that your benchmarking code does not properly flush the L2 cache. This comment is misleading "# Construct new tensors every time to avoid L2 cache acceleration"!

Constructing new tensors actually puts them in the L2 cache, as all DRAM writes on NVIDIA GPUs always go through the L2 cache first, including host-to-device cudaMemcpy. The "flush_l2" parameter in bench_kineto is disabled by default, but enabling it won't work because construction happens after the L2 flush. Another interesting side-effect is this gives the GPU enough time to "cool down" and you avoid thermal throttling completely (the GPU is idle some of the time and the kernels that do run are very low power compared to matmuls).

This might be representative of some use cases (e.g. previous kernel converts A & B to FP8 putting them in L2 + low enough power that the GPU cools down) and thermal throttling might be avoided with liquid cooling or if you reserve SMs for communication etc... But that's clearly not the intended effect of the code based on your comments and it's not standard benchmarking practice, so I think it makes sense to fix this.

I've also added options to maximise throttling (2000 runs) or minimise it (long sleep between tests) compared to the new default (huge 8GB memset to give the GPU time to cool down while staying at maximum clocks with DVFS, this gave me the best performance on GH200 interestingly enough - you could just replace it with a short flush and a sleep if you prefer, it's not a big difference).

Unfortunately it significantly reduces performance. Here's my GH200 96GiB baseline (you'd need to retest H800 yourself):

Testing GEMM:
Performance (m= 64, n= 2112, k= 7168): 10 us | throughput: 195 TFLOPS, 1595 GB/s
Performance (m= 64, n=24576, k= 1536): 15 us | throughput: 323 TFLOPS, 2740 GB/s
Performance (m= 64, n=32768, k= 512): 9 us | throughput: 235 TFLOPS, 2297 GB/s
Performance (m= 64, n= 7168, k=16384): 39 us | throughput: 381 TFLOPS, 3024 GB/s
Performance (m= 64, n= 4096, k= 7168): 13 us | throughput: 297 TFLOPS, 2397 GB/s
Performance (m= 64, n= 7168, k= 2048): 6 us | throughput: 295 TFLOPS, 2465 GB/s
Performance (m= 128, n= 2112, k= 7168): 11 us | throughput: 340 TFLOPS, 1457 GB/s
Performance (m= 128, n=24576, k= 1536): 16 us | throughput: 587 TFLOPS, 2686 GB/s
Performance (m= 128, n=32768, k= 512): 11 us | throughput: 378 TFLOPS, 2222 GB/s
Performance (m= 128, n= 7168, k=16384): 42 us | throughput: 722 TFLOPS, 2913 GB/s
Performance (m= 128, n= 4096, k= 7168): 14 us | throughput: 538 TFLOPS, 2244 GB/s
Performance (m= 128, n= 7168, k= 2048): 7 us | throughput: 514 TFLOPS, 2295 GB/s
Performance (m= 4096, n= 2112, k= 7168): 114 us | throughput: 1088 TFLOPS, 542 GB/s
Performance (m= 4096, n=24576, k= 1536): 319 us | throughput: 970 TFLOPS, 770 GB/s
Performance (m= 4096, n=32768, k= 512): 235 us | throughput: 585 TFLOPS, 1223 GB/s
Performance (m= 4096, n= 7168, k=16384): 662 us | throughput: 1453 TFLOPS, 367 GB/s
Performance (m= 4096, n= 4096, k= 7168): 174 us | throughput: 1381 TFLOPS, 530 GB/s
Performance (m= 4096, n= 7168, k= 2048): 116 us | throughput: 1034 TFLOPS, 703 GB/s

Testing grouped contiguous GEMM:
Performance (num_groups=4, m_per_group=8192, n=4096, k=7168): 1354 us | throughput: 1421 TFLOPS, 458 GB/s
Performance (num_groups=4, m_per_group=8192, n=7168, k=2048): 880 us | throughput: 1094 TFLOPS, 677 GB/s
Performance (num_groups=8, m_per_group=4096, n=4096, k=7168): 1336 us | throughput: 1440 TFLOPS, 553 GB/s
Performance (num_groups=8, m_per_group=4096, n=7168, k=2048): 854 us | throughput: 1127 TFLOPS, 766 GB/s

Testing grouped masked GEMM:
Performance (num_groups=1, m_per_group=1024, n=4096, k=7168): 46 us | throughput: 1295 TFLOPS, 971 GB/s
Performance (num_groups=1, m_per_group=1024, n=7168, k=2048): 32 us | throughput: 926 TFLOPS, 969 GB/s
Performance (num_groups=2, m_per_group= 512, n=4096, k=7168): 51 us | throughput: 1180 TFLOPS, 1461 GB/s
Performance (num_groups=2, m_per_group= 512, n=7168, k=2048): 32 us | throughput: 933 TFLOPS, 1431 GB/s
Performance (num_groups=4, m_per_group= 256, n=4096, k=7168): 57 us | throughput: 1054 TFLOPS, 2335 GB/s
Performance (num_groups=4, m_per_group= 256, n=7168, k=2048): 35 us | throughput: 849 TFLOPS, 2131 GB/s

And with fixed L2 flush on same GH200 96GiB (faster tests might be because it's even better at avoiding thermal throttling):

Testing GEMM:
Performance (m= 64, n= 2112, k= 7168): 12 us | throughput: 167 TFLOPS, 1364 GB/s
Performance (m= 64, n=24576, k= 1536): 19 us | throughput: 258 TFLOPS, 2193 GB/s
Performance (m= 64, n=32768, k= 512): 11 us | throughput: 188 TFLOPS, 1838 GB/s
Performance (m= 64, n= 7168, k=16384): 46 us | throughput: 326 TFLOPS, 2588 GB/s
Performance (m= 64, n= 4096, k= 7168): 17 us | throughput: 224 TFLOPS, 1807 GB/s
Performance (m= 64, n= 7168, k= 2048): 9 us | throughput: 213 TFLOPS, 1781 GB/s
Performance (m= 128, n= 2112, k= 7168): 13 us | throughput: 302 TFLOPS, 1291 GB/s
Performance (m= 128, n=24576, k= 1536): 20 us | throughput: 479 TFLOPS, 2191 GB/s
Performance (m= 128, n=32768, k= 512): 13 us | throughput: 322 TFLOPS, 1893 GB/s
Performance (m= 128, n= 7168, k=16384): 47 us | throughput: 635 TFLOPS, 2563 GB/s
Performance (m= 128, n= 4096, k= 7168): 18 us | throughput: 419 TFLOPS, 1745 GB/s
Performance (m= 128, n= 7168, k= 2048): 10 us | throughput: 394 TFLOPS, 1760 GB/s
Performance (m= 4096, n= 2112, k= 7168): 114 us | throughput: 1087 TFLOPS, 542 GB/s
Performance (m= 4096, n=24576, k= 1536): 312 us | throughput: 991 TFLOPS, 786 GB/s
Performance (m= 4096, n=32768, k= 512): 233 us | throughput: 589 TFLOPS, 1231 GB/s
Performance (m= 4096, n= 7168, k=16384): 638 us | throughput: 1508 TFLOPS, 381 GB/s
Performance (m= 4096, n= 4096, k= 7168): 175 us | throughput: 1372 TFLOPS, 526 GB/s
Performance (m= 4096, n= 7168, k= 2048): 116 us | throughput: 1035 TFLOPS, 704 GB/s

Testing grouped contiguous GEMM:
Performance (num_groups=4, m_per_group=8192, n=4096, k=7168): 1335 us | throughput: 1441 TFLOPS, 465 GB/s
Performance (num_groups=4, m_per_group=8192, n=7168, k=2048): 854 us | throughput: 1127 TFLOPS, 698 GB/s
Performance (num_groups=8, m_per_group=4096, n=4096, k=7168): 1350 us | throughput: 1425 TFLOPS, 547 GB/s
Performance (num_groups=8, m_per_group=4096, n=7168, k=2048): 857 us | throughput: 1123 TFLOPS, 764 GB/s

Testing grouped masked GEMM:
Performance (num_groups=1, m_per_group=1024, n=4096, k=7168): 48 us | throughput: 1253 TFLOPS, 940 GB/s
Performance (num_groups=1, m_per_group=1024, n=7168, k=2048): 33 us | throughput: 903 TFLOPS, 945 GB/s
Performance (num_groups=2, m_per_group= 512, n=4096, k=7168): 52 us | throughput: 1149 TFLOPS, 1423 GB/s
Performance (num_groups=2, m_per_group= 512, n=7168, k=2048): 34 us | throughput: 891 TFLOPS, 1367 GB/s
Performance (num_groups=4, m_per_group= 256, n=4096, k=7168): 59 us | throughput: 1018 TFLOPS, 2255 GB/s
Performance (num_groups=4, m_per_group= 256, n=7168, k=2048): 37 us | throughput: 812 TFLOPS, 2040 GB/s

Thankfully I have managed to more than recover that performance in my upcoming fork (which by now changes nearly all of fp8_gemm.cuh "Ship of Theseus" style).

Here's a small teaser (this is before I fixed unaligned block sizes so n=2048 instead of 2112, and I might lose a bit of the performance once I fix it properly, to be determined...) - looking forward to releasing this soon, it involves a lot of crazy tricks (different setup than numbers above so not 1:1 comparable):

Test Type Dimensions Base + L2 flush fix Optimized 128x Optimized 256x Optimized 128x % Optimized 256x %
GEMM (64, 2048, 7168) 167 201 202 20.36% 20.96%
GEMM (64, 24576, 1536) 264 284 285 7.58% 7.95%
GEMM (64, 32768, 512) 200 213 212 6.50% 6.00%
GEMM (64, 7168, 16384) 329 345 346 4.86% 5.17%
GEMM (64, 4096, 7168) 229 274 276 19.65% 20.52%
GEMM (64, 7168, 2048) 219 223 225 1.83% 2.74%
GEMM (128, 2048, 7168) 308 328 338 6.49% 9.74%
GEMM (128, 24576, 1536) 490 494 503 0.82% 2.65%
GEMM (128, 32768, 512) 335 386 386 15.22% 15.22%
GEMM (128, 7168, 16384) 640 657 658 2.66% 2.81%
GEMM (128, 4096, 7168) 433 487 487 12.47% 12.47%
GEMM (128, 7168, 2048) 405 406 411 0.25% 1.48%
GEMM (4096, 2048, 7168) 1090 1412 1497 29.54% 37.34%
GEMM (4096, 24576, 1536) 992 1274 1303 28.43% 31.35%
GEMM (4096, 32768, 512) 585 1015 1022 73.50% 74.70%
GEMM (4096, 7168, 16384) 1512 1526 1625 0.93% 7.47%
GEMM (4096, 4096, 7168) 1382 1442 1526 4.34% 10.42%
GEMM (4096, 7168, 2048) 1046 1282 1326 22.56% 26.77%
GEMM Grouped (4, 8192, 4096, 7168) 1453 1520 1606 4.61% 10.53%
GEMM Grouped (4, 8192, 7168, 2048) 1139 1385 1429 21.60% 25.46%
GEMM Grouped (8, 4096, 4096, 7168) 1443 1515 1610 4.99% 11.57%
GEMM Grouped (8, 4096, 7168, 2048) 1132 1386 1428 22.44% 26.15%

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/deepseek-ai/DeepGEMM/pull/68 **Author:** [@ademeure](https://github.com/ademeure) **Created:** 3/15/2025 **Status:** ✅ Merged **Merged:** 3/16/2025 **Merged by:** [@soundOfDestiny](https://github.com/soundOfDestiny) **Base:** `main` ← **Head:** `flush_l2_pr` --- ### 📝 Commits (1) - [`6cbff57`](https://github.com/deepseek-ai/DeepGEMM/commit/6cbff5778fabab426a2ac2f66dabe76f44c6bef3) Correctly flush L2, as reconstructing the tensors on every iteration effectively put them in the L2, and gave the GPU enough idle time to avoid thermal throttling in a potentially unrealistic way. ### 📊 Changes **2 files changed** (+35 additions, -11 deletions) <details> <summary>View changed files</summary> 📝 `deep_gemm/utils.py` (+23 -2) 📝 `tests/test_core.py` (+12 -9) </details> ### 📄 Description Thank you for open sourcing this under a MIT license! I was working on optimized PTX matrix multiplications for H100 as you released this, and decided to continue working on it as a fork of your repo instead (not yet public). I've achieved very large performance gains (see below) and I plan to open source it very soon (+ maybe try to get a modded nanogpt speedrun record with it). I hope it can be useful to the wider community worldwide. However as part of this work I have discovered that your benchmarking code does not properly flush the L2 cache. This comment is misleading "# Construct new tensors every time to avoid L2 cache acceleration"! Constructing new tensors actually puts them *in* the L2 cache, as all DRAM writes on NVIDIA GPUs always go through the L2 cache first, including host-to-device cudaMemcpy. The "flush_l2" parameter in bench_kineto is disabled by default, but enabling it won't work because construction happens *after* the L2 flush. Another interesting side-effect is this gives the GPU enough time to "cool down" and you avoid thermal throttling completely (the GPU is idle some of the time and the kernels that do run are very low power compared to matmuls). This might be representative of some use cases (e.g. previous kernel converts A & B to FP8 putting them in L2 + low enough power that the GPU cools down) and thermal throttling might be avoided with liquid cooling or if you reserve SMs for communication etc... But that's clearly not the intended effect of the code based on your comments and it's not standard benchmarking practice, so I think it makes sense to fix this. I've also added options to maximise throttling (2000 runs) or minimise it (long sleep between tests) compared to the new default (huge 8GB memset to give the GPU time to cool down while staying at maximum clocks with DVFS, this gave me the best performance on GH200 interestingly enough - you could just replace it with a short flush and a sleep if you prefer, it's not a big difference). Unfortunately it significantly reduces performance. Here's my GH200 96GiB baseline (you'd need to retest H800 yourself): > Testing GEMM: > Performance (m= 64, n= 2112, k= 7168): 10 us | throughput: 195 TFLOPS, 1595 GB/s > Performance (m= 64, n=24576, k= 1536): 15 us | throughput: 323 TFLOPS, 2740 GB/s > Performance (m= 64, n=32768, k= 512): 9 us | throughput: 235 TFLOPS, 2297 GB/s > Performance (m= 64, n= 7168, k=16384): 39 us | throughput: 381 TFLOPS, 3024 GB/s > Performance (m= 64, n= 4096, k= 7168): 13 us | throughput: 297 TFLOPS, 2397 GB/s > Performance (m= 64, n= 7168, k= 2048): 6 us | throughput: 295 TFLOPS, 2465 GB/s > Performance (m= 128, n= 2112, k= 7168): 11 us | throughput: 340 TFLOPS, 1457 GB/s > Performance (m= 128, n=24576, k= 1536): 16 us | throughput: 587 TFLOPS, 2686 GB/s > Performance (m= 128, n=32768, k= 512): 11 us | throughput: 378 TFLOPS, 2222 GB/s > Performance (m= 128, n= 7168, k=16384): 42 us | throughput: 722 TFLOPS, 2913 GB/s > Performance (m= 128, n= 4096, k= 7168): 14 us | throughput: 538 TFLOPS, 2244 GB/s > Performance (m= 128, n= 7168, k= 2048): 7 us | throughput: 514 TFLOPS, 2295 GB/s > Performance (m= 4096, n= 2112, k= 7168): 114 us | throughput: 1088 TFLOPS, 542 GB/s > Performance (m= 4096, n=24576, k= 1536): 319 us | throughput: 970 TFLOPS, 770 GB/s > Performance (m= 4096, n=32768, k= 512): 235 us | throughput: 585 TFLOPS, 1223 GB/s > Performance (m= 4096, n= 7168, k=16384): 662 us | throughput: 1453 TFLOPS, 367 GB/s > Performance (m= 4096, n= 4096, k= 7168): 174 us | throughput: 1381 TFLOPS, 530 GB/s > Performance (m= 4096, n= 7168, k= 2048): 116 us | throughput: 1034 TFLOPS, 703 GB/s > > Testing grouped contiguous GEMM: > Performance (num_groups=4, m_per_group=8192, n=4096, k=7168): 1354 us | throughput: 1421 TFLOPS, 458 GB/s > Performance (num_groups=4, m_per_group=8192, n=7168, k=2048): 880 us | throughput: 1094 TFLOPS, 677 GB/s > Performance (num_groups=8, m_per_group=4096, n=4096, k=7168): 1336 us | throughput: 1440 TFLOPS, 553 GB/s > Performance (num_groups=8, m_per_group=4096, n=7168, k=2048): 854 us | throughput: 1127 TFLOPS, 766 GB/s > > Testing grouped masked GEMM: > Performance (num_groups=1, m_per_group=1024, n=4096, k=7168): 46 us | throughput: 1295 TFLOPS, 971 GB/s > Performance (num_groups=1, m_per_group=1024, n=7168, k=2048): 32 us | throughput: 926 TFLOPS, 969 GB/s > Performance (num_groups=2, m_per_group= 512, n=4096, k=7168): 51 us | throughput: 1180 TFLOPS, 1461 GB/s > Performance (num_groups=2, m_per_group= 512, n=7168, k=2048): 32 us | throughput: 933 TFLOPS, 1431 GB/s > Performance (num_groups=4, m_per_group= 256, n=4096, k=7168): 57 us | throughput: 1054 TFLOPS, 2335 GB/s > Performance (num_groups=4, m_per_group= 256, n=7168, k=2048): 35 us | throughput: 849 TFLOPS, 2131 GB/s And with fixed L2 flush on same GH200 96GiB (faster tests might be because it's even better at avoiding thermal throttling): > Testing GEMM: > Performance (m= 64, n= 2112, k= 7168): 12 us | throughput: 167 TFLOPS, 1364 GB/s > Performance (m= 64, n=24576, k= 1536): 19 us | throughput: 258 TFLOPS, 2193 GB/s > Performance (m= 64, n=32768, k= 512): 11 us | throughput: 188 TFLOPS, 1838 GB/s > Performance (m= 64, n= 7168, k=16384): 46 us | throughput: 326 TFLOPS, 2588 GB/s > Performance (m= 64, n= 4096, k= 7168): 17 us | throughput: 224 TFLOPS, 1807 GB/s > Performance (m= 64, n= 7168, k= 2048): 9 us | throughput: 213 TFLOPS, 1781 GB/s > Performance (m= 128, n= 2112, k= 7168): 13 us | throughput: 302 TFLOPS, 1291 GB/s > Performance (m= 128, n=24576, k= 1536): 20 us | throughput: 479 TFLOPS, 2191 GB/s > Performance (m= 128, n=32768, k= 512): 13 us | throughput: 322 TFLOPS, 1893 GB/s > Performance (m= 128, n= 7168, k=16384): 47 us | throughput: 635 TFLOPS, 2563 GB/s > Performance (m= 128, n= 4096, k= 7168): 18 us | throughput: 419 TFLOPS, 1745 GB/s > Performance (m= 128, n= 7168, k= 2048): 10 us | throughput: 394 TFLOPS, 1760 GB/s > Performance (m= 4096, n= 2112, k= 7168): 114 us | throughput: 1087 TFLOPS, 542 GB/s > Performance (m= 4096, n=24576, k= 1536): 312 us | throughput: 991 TFLOPS, 786 GB/s > Performance (m= 4096, n=32768, k= 512): 233 us | throughput: 589 TFLOPS, 1231 GB/s > Performance (m= 4096, n= 7168, k=16384): 638 us | throughput: 1508 TFLOPS, 381 GB/s > Performance (m= 4096, n= 4096, k= 7168): 175 us | throughput: 1372 TFLOPS, 526 GB/s > Performance (m= 4096, n= 7168, k= 2048): 116 us | throughput: 1035 TFLOPS, 704 GB/s > > Testing grouped contiguous GEMM: > Performance (num_groups=4, m_per_group=8192, n=4096, k=7168): 1335 us | throughput: 1441 TFLOPS, 465 GB/s > Performance (num_groups=4, m_per_group=8192, n=7168, k=2048): 854 us | throughput: 1127 TFLOPS, 698 GB/s > Performance (num_groups=8, m_per_group=4096, n=4096, k=7168): 1350 us | throughput: 1425 TFLOPS, 547 GB/s > Performance (num_groups=8, m_per_group=4096, n=7168, k=2048): 857 us | throughput: 1123 TFLOPS, 764 GB/s > > Testing grouped masked GEMM: > Performance (num_groups=1, m_per_group=1024, n=4096, k=7168): 48 us | throughput: 1253 TFLOPS, 940 GB/s > Performance (num_groups=1, m_per_group=1024, n=7168, k=2048): 33 us | throughput: 903 TFLOPS, 945 GB/s > Performance (num_groups=2, m_per_group= 512, n=4096, k=7168): 52 us | throughput: 1149 TFLOPS, 1423 GB/s > Performance (num_groups=2, m_per_group= 512, n=7168, k=2048): 34 us | throughput: 891 TFLOPS, 1367 GB/s > Performance (num_groups=4, m_per_group= 256, n=4096, k=7168): 59 us | throughput: 1018 TFLOPS, 2255 GB/s > Performance (num_groups=4, m_per_group= 256, n=7168, k=2048): 37 us | throughput: 812 TFLOPS, 2040 GB/s Thankfully I have managed to more than recover that performance in my upcoming fork (which by now changes nearly all of fp8_gemm.cuh "Ship of Theseus" style). Here's a small teaser (this is before I fixed unaligned block sizes so n=2048 instead of 2112, and I might lose a bit of the performance once I fix it properly, to be determined...) - looking forward to releasing this soon, it involves a lot of crazy tricks (different setup than numbers above so not 1:1 comparable): Test Type | Dimensions | Base + L2 flush fix | Optimized 128x | Optimized 256x | Optimized 128x % | Optimized 256x % -- | -- | -- | -- | -- | -- | -- GEMM | (64, 2048, 7168) | 167 | 201 | 202 | 20.36% | 20.96% GEMM | (64, 24576, 1536) | 264 | 284 | 285 | 7.58% | 7.95% GEMM | (64, 32768, 512) | 200 | 213 | 212 | 6.50% | 6.00% GEMM | (64, 7168, 16384) | 329 | 345 | 346 | 4.86% | 5.17% GEMM | (64, 4096, 7168) | 229 | 274 | 276 | 19.65% | 20.52% GEMM | (64, 7168, 2048) | 219 | 223 | 225 | 1.83% | 2.74% GEMM | (128, 2048, 7168) | 308 | 328 | 338 | 6.49% | 9.74% GEMM | (128, 24576, 1536) | 490 | 494 | 503 | 0.82% | 2.65% GEMM | (128, 32768, 512) | 335 | 386 | 386 | 15.22% | 15.22% GEMM | (128, 7168, 16384) | 640 | 657 | 658 | 2.66% | 2.81% GEMM | (128, 4096, 7168) | 433 | 487 | 487 | 12.47% | 12.47% GEMM | (128, 7168, 2048) | 405 | 406 | 411 | 0.25% | 1.48% GEMM | (4096, 2048, 7168) | 1090 | 1412 | 1497 | 29.54% | 37.34% GEMM | (4096, 24576, 1536) | 992 | 1274 | 1303 | 28.43% | 31.35% GEMM | (4096, 32768, 512) | 585 | 1015 | 1022 | 73.50% | 74.70% GEMM | (4096, 7168, 16384) | 1512 | 1526 | 1625 | 0.93% | 7.47% GEMM | (4096, 4096, 7168) | 1382 | 1442 | 1526 | 4.34% | 10.42% GEMM | (4096, 7168, 2048) | 1046 | 1282 | 1326 | 22.56% | 26.77% GEMM Grouped | (4, 8192, 4096, 7168) | 1453 | 1520 | 1606 | 4.61% | 10.53% GEMM Grouped | (4, 8192, 7168, 2048) | 1139 | 1385 | 1429 | 21.60% | 25.46% GEMM Grouped | (8, 4096, 4096, 7168) | 1443 | 1515 | 1610 | 4.99% | 11.57% GEMM Grouped | (8, 4096, 7168, 2048) | 1132 | 1386 | 1428 | 22.44% | 26.15% --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
bsakaguchi 2026-10-26 21:15:06 +00:00
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
public-sync-20261014
0930-release
nv_dev
fix-mega-moe-fence
public-release-2609
public-release-2607
public-release-2606
mega-update
public-release-260416
ko3n1g/ci/build-wheels
nv_dev_f8e8fb5
nv_dev_950d31a
nv_dev_67fc648
nv_dev_c491439
nv_dev_4ff3f54
v2.1.1.post3
v2.1.1.post2
v2.1.1.post1
v2.1.1
v2.1.0
Labels
Clear labels   
pull-request
Mirrored from GitHub Pull Request

No labels
pull-request
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
deepseek-ai/DeepGEMM#229
No description provided.