[PR #34] [CLOSED] Fix FP8 dequantization on MPS by falling back to CPU #91

New issue

Closed

opened 2026-10-26 22:20:17 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 22:20:17 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/comfy-kitchen/pull/34
Author: @snwchd71
Created: 4/4/2026
Status: ❌ Closed

Base: main ← Head: fix/mps-fp8-dequantize

📝 Commits (1)

2d427c4 Fix FP8 dequantization on MPS by falling back to CPU

📊 Changes

2 files changed (+67 additions, -0 deletions)

View changed files

📝 comfy_kitchen/backends/eager/quantization.py (+18 -0)
📝 tests/test_qdq.py (+49 -0)

📄 Description

Summary

MPS (Apple Silicon) does not support FP8 dtype conversion, causing dequantize_per_tensor_fp8() to crash with:

TypeError: Trying to convert Float8_e4m3fn to the MPS backend but it does not have support for that dtype.

This adds an explicit device + dtype guard to dequantize on CPU and transfer only the final result back to MPS. Includes a one-time logger.warning for visibility.

No hot-path overhead: the check is a string compare + frozenset membership, only evaluated when the tensor is on MPS with an FP8 dtype
Bit-identical results: every FP8 value is exactly representable in bfloat16/float16, verified with torch.equal
Auto-disables: the dtype guard means this becomes a no-op if MPS ever gains FP8 support

Prior work

#23 by @AoiYamada identified the same fix. This PR adds tests, a one-time warning, and the dtype guard.
Comfy-Org/ComfyUI#12378 by @tashiscool fixes the complementary quantization path in ComfyUI core (stochastic_rounding, quant_ops.py). Both fixes are needed for end-to-end FP8 on MPS, confirmed by @vSnake87 on M3 Max.

Test plan

test_dequantize_fp8_cpu_fallback_correctness — verifies fallback math is bit-identical to standard path (runs in CI, parametrized float16/bfloat16)
test_dequantize_fp8_on_mps_device — end-to-end on actual MPS hardware (skipped in CI, parametrized float16/bfloat16)
Full test suite: 103 passed, 203 skipped (CUDA/Triton), 0 failures
ruff check clean

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/comfy-kitchen/pull/34 **Author:** [@snwchd71](https://github.com/snwchd71) **Created:** 4/4/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `fix/mps-fp8-dequantize` --- ### 📝 Commits (1) - [`2d427c4`](https://github.com/Comfy-Org/comfy-kitchen/commit/2d427c494c641bc92779533593287abf4bf82e2f) Fix FP8 dequantization on MPS by falling back to CPU ### 📊 Changes **2 files changed** (+67 additions, -0 deletions) <details> <summary>View changed files</summary> 📝 `comfy_kitchen/backends/eager/quantization.py` (+18 -0) 📝 `tests/test_qdq.py` (+49 -0) </details> ### 📄 Description ## Summary MPS (Apple Silicon) does not support FP8 dtype conversion, causing `dequantize_per_tensor_fp8()` to crash with: ``` TypeError: Trying to convert Float8_e4m3fn to the MPS backend but it does not have support for that dtype. ``` This adds an explicit device + dtype guard to dequantize on CPU and transfer only the final result back to MPS. Includes a one-time `logger.warning` for visibility. - **No hot-path overhead**: the check is a string compare + frozenset membership, only evaluated when the tensor is on MPS with an FP8 dtype - **Bit-identical results**: every FP8 value is exactly representable in bfloat16/float16, verified with `torch.equal` - **Auto-disables**: the dtype guard means this becomes a no-op if MPS ever gains FP8 support ## Prior work - #23 by @AoiYamada identified the same fix. This PR adds tests, a one-time warning, and the dtype guard. - Comfy-Org/ComfyUI#12378 by @tashiscool fixes the complementary **quantization** path in ComfyUI core (`stochastic_rounding`, `quant_ops.py`). Both fixes are needed for end-to-end FP8 on MPS, confirmed by @vSnake87 on M3 Max. ## Test plan - [x] `test_dequantize_fp8_cpu_fallback_correctness` — verifies fallback math is bit-identical to standard path (runs in CI, parametrized float16/bfloat16) - [x] `test_dequantize_fp8_on_mps_device` — end-to-end on actual MPS hardware (skipped in CI, parametrized float16/bfloat16) - [x] Full test suite: 103 passed, 203 skipped (CUDA/Triton), 0 failures - [x] `ruff check` clean --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-26 22:20:17 +00:00