Commit Graph

5 Commits

Author SHA1 Message Date
Binyang Li
325f79f9dc Add configurable FP8 low-latency dispatch
Quantize BF16 dispatch payloads to FP8 E4M3 with format-defined block scales while preserving BF16 expert outputs for combine. Clean up the sender structure, payload metadata, vector conversions, Python API, and multi-rank coverage.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: efbacae6-f679-430b-bc16-b45ae162fc76
2026-07-13 18:47:21 +00:00
Binyang Li
8841cdc765 WIP 2026-07-13 05:02:43 +00:00
Binyang Li
152f2ab02d code optimization 2026-07-13 03:07:12 +00:00
Binyang Li
b1d0893da9 Update ep test. Enable cuda graph for ep testing (#829) 2026-07-07 16:48:43 -07:00
Binyang Li
8e34326d7a Binyli/ep revise (#828)
This pull request makes significant improvements to the MoE (Mixture of
Experts) Python API and documentation, focusing on clarifying and
expanding the Expert Parallel (EP) interface, especially around
quantization, dispatch/combine handles, and overlap configuration. The
changes introduce new data structures, update function signatures, and
improve documentation to better reflect the current and planned
capabilities of the system. Additionally, the base development container
is updated to CUDA 13.0, and minor corrections are made to extension
naming.
2026-07-06 21:14:29 -07:00