mirror of
https://github.com/Comfy-Org/comfy-kitchen.git
synced 2026-11-04 23:37:42 +00:00
[GH-ISSUE #38] FP8 E4M3 quantized models inference is slower than same model quantized in MXFP8 #18
Labels
No labels
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
Comfy-Org/comfy-kitchen#18
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @matalama80td3l on GitHub (May 18, 2026).
Original GitHub issue: https://github.com/Comfy-Org/comfy-kitchen/issues/38
It's quite weird. I think mxfp8 inference is a bit more complex than the legacy fp8e4m3. So does it mean we still have some potential in legacy fp8 inference?
@matalama80td3l commented on GitHub (May 18, 2026):
Hardware: 5060ti
sageattn2
qwen-edit in mxfp8
qwen-edit in fp8
@contentis commented on GitHub (May 20, 2026):
The speed of FP8 and MXFP8 should be more or less the same - for the FP8 run, did you use a PTQ checkpoint or does this use some sort of dynamic scaling as this is where FP8 can get slow.
@matalama80td3l commented on GitHub (May 22, 2026):
I'm using this qwen-image-edit checkpoint, and it used fp8 without per-tensor scale factor. I wrote a script to transfer all diffusion fp8 tensors to mxfp8 by making all E8M0 block-scale factors equals 0 so I get a same mxfp8 model.
@contentis commented on GitHub (Jun 2, 2026):
If you don't use a calibrated checkpoint (no scales) I would assume that it will do dynamic quantization which results in an all-reduce operations that can be very costly.
Btw, for MXFP8 there is a sample here https://github.com/Comfy-Org/comfy-kitchen/blob/main/samples/mxfp8_model_patcher.py that can patch the model on the fly.