[GH-ISSUE #69] questions #42

New issue

Closed

opened 2026-10-26 21:11:05 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-26 21:11:05 +00:00

Owner

Originally created by @shenyichen on GitHub (Mar 18, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepGEMM/issues/69

We are currently exploring the possibility of optimizing several of the projects you have open-sourced. We are particularly interested in understanding the potential performance improvements that could be achieved on a single H800. Specifically, we would like to know how much enhancement in throughput and reduction in latency we could expect after the optimization process.
Furthermore, we would be extremely grateful if you could provide us with some benchmark data or tools that we could use to evaluate the performance before and after the optimizations. This would be immensely helpful in guiding our efforts and ensuring that we achieve the best possible results.

Originally created by @shenyichen on GitHub (Mar 18, 2025). Original GitHub issue: https://github.com/deepseek-ai/DeepGEMM/issues/69 We are currently exploring the possibility of optimizing several of the projects you have open-sourced. We are particularly interested in understanding the potential performance improvements that could be achieved on a single H800. Specifically, we would like to know how much enhancement in throughput and reduction in latency we could expect after the optimization process. Furthermore, we would be extremely grateful if you could provide us with some benchmark data or tools that we could use to evaluate the performance before and after the optimizations. This would be immensely helpful in guiding our efforts and ensuring that we achieve the best possible results.

bsakaguchi closed this issue

2026-10-26 21:11:05 +00:00

bsakaguchi commented

2026-10-26 21:11:07 +00:00

Author

Owner

@LyricZhao commented on GitHub (Mar 24, 2025):

Hi, the optimization of FlashMLA and DeepGEMM (this repo) can be done with a single H800. But the current performance of these kernels are close to physical limit, leaving only several cases may have ~10% optimizating space (1-5% improvement for end-to-end inference improvement, I guess).

For benchmarking of DeepGEMM, you can simply run tests/test_core.py for all common shapes and their performance (you can compare the TFLOPS or memory bandwidth with the physical limits as an optimization guideline).

@LyricZhao commented on GitHub (Mar 24, 2025): Hi, the optimization of [FlashMLA](https://github.com/deepseek-ai/FlashMLA/) and DeepGEMM (this repo) can be done with a single H800. But the current performance of these kernels are close to physical limit, leaving only several cases may have ~10% optimizating space (1-5% improvement for end-to-end inference improvement, I guess). For benchmarking of DeepGEMM, you can simply run `tests/test_core.py` for all common shapes and their performance (you can compare the TFLOPS or memory bandwidth with the physical limits as an optimization guideline).

bsakaguchi referenced this issue

2026-10-26 21:14:57 +00:00

[PR #42] [MERGED] Performance: reducing the percentage of FFMA interleaving yields a sight performance gain, roughly 0.5% #216