[PR #23] [CLOSED] Fix FP8 tensor support on MPS backend for Apple Silicon Macs #85

New issue

Closed

opened 2026-10-26 22:20:13 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 22:20:13 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/comfy-kitchen/pull/23
Author: @AoiYamada
Created: 1/17/2026
Status: ❌ Closed

Base: main ← Head: fix/convert-float8-e4m3fn-to-the-mps-backend-is-unsupported-error

📝 Commits (1)

d99790e fix: convert float8 e4m3fn to the mps backend is unsupported error

📊 Changes

1 file changed (+9 additions, -0 deletions)

View changed files

📝 comfy_kitchen/backends/eager/quantization.py (+9 -0)

📄 Description

Problem:
Users on Apple Silicon Macs (MPS backend) encounter TypeError: Trying to convert Float8_e4m3fn to the MPS backend but it does not have support for that dtype when running FP8-quantized models through the sampler. This occurs because MPS does not natively support FP8 data type conversions.

Related Issues:

(all report similar FP8/MPS compatibility issues)

Previous Attempts & Limitations:

Force CPU execution (--cpu flag): Makes generation prohibitively slow
GGUF format: Same performance issue as CPU execution
Model conversion to FP16: Even after converting all WAN model to FP16 and using FP16 vae, text encoder, the sampler still produces FP8 tensors internally

The conversion script:

from safetensors.torch import load_file, save_file
import torch
import sys

if len(sys.argv) < 2:
    print("Usage: python3 convert-fp16.py <input_safetensors_file>")
    sys.exit(1)

path = sys.argv[1] # input file

def convert_safetensors_to_fp16(input_path):
    state_dict = load_file(input_path)
    fp16_state_dict = {}
    for key, value in state_dict.items():
        if value.dtype == torch.float8_e4m3fn:
            fp16_state_dict[key] = value.to(torch.float16)
        else:
            fp16_state_dict[key] = value
    
    # Save the new file
    output_path = path.replace(".safetensors", "") + "-fp16.safetensors"
    save_file(fp16_state_dict, output_path, metadata={"format": "pt"})
    print(f"Converted file saved to {output_path}")

convert_safetensors_to_fp16(path)

Root Cause:
The comfy_kitchen quantization library attempts direct FP8 tensor operations on MPS, which lacks FP8 support. The error occurs in dequantize_per_tensor_fp8() when trying to convert FP8 tensors to other formats on MPS.

Solution:
Add a device-aware fallback in dequantize_per_tensor_fp8():

Detect FP8 tensors on MPS devices
Temporarily move tensors to CPU for FP8 conversion (which CPU supports)
Return results to MPS for continued processing

This minimal fix enables FP8-quantized models to work on Apple Silicon while:

Maintaining MPS acceleration for the majority of operations
Avoiding performance penalties of full CPU execution

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/comfy-kitchen/pull/23 **Author:** [@AoiYamada](https://github.com/AoiYamada) **Created:** 1/17/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `fix/convert-float8-e4m3fn-to-the-mps-backend-is-unsupported-error` --- ### 📝 Commits (1) - [`d99790e`](https://github.com/Comfy-Org/comfy-kitchen/commit/d99790ec4dc09c27ad46a9cddce6192159a89840) fix: convert float8 e4m3fn to the mps backend is unsupported error ### 📊 Changes **1 file changed** (+9 additions, -0 deletions) <details> <summary>View changed files</summary> 📝 `comfy_kitchen/backends/eager/quantization.py` (+9 -0) </details> ### 📄 Description **Problem**: Users on Apple Silicon Macs (MPS backend) encounter `TypeError: Trying to convert Float8_e4m3fn to the MPS backend but it does not have support for that dtype` when running FP8-quantized models through the sampler. This occurs because MPS does not natively support FP8 data type conversions. **Related Issues**: - https://github.com/Comfy-Org/ComfyUI/issues/9255 - https://github.com/Comfy-Org/ComfyUI/issues/6995 - https://github.com/Comfy-Org/ComfyUI/issues/11817 - https://github.com/Comfy-Org/ComfyUI/issues/11626 (all report similar FP8/MPS compatibility issues) **Previous Attempts & Limitations**: 1. **Force CPU execution** (`--cpu` flag): Makes generation prohibitively slow 2. **GGUF format**: Same performance issue as CPU execution 3. **Model conversion to FP16**: Even after converting all WAN model to FP16 and using FP16 vae, text encoder, the sampler still produces FP8 tensors internally The conversion script: ```python3 from safetensors.torch import load_file, save_file import torch import sys if len(sys.argv) < 2: print("Usage: python3 convert-fp16.py <input_safetensors_file>") sys.exit(1) path = sys.argv[1] # input file def convert_safetensors_to_fp16(input_path): state_dict = load_file(input_path) fp16_state_dict = {} for key, value in state_dict.items(): if value.dtype == torch.float8_e4m3fn: fp16_state_dict[key] = value.to(torch.float16) else: fp16_state_dict[key] = value # Save the new file output_path = path.replace(".safetensors", "") + "-fp16.safetensors" save_file(fp16_state_dict, output_path, metadata={"format": "pt"}) print(f"Converted file saved to {output_path}") convert_safetensors_to_fp16(path) ``` **Root Cause**: The `comfy_kitchen` quantization library attempts direct FP8 tensor operations on MPS, which lacks FP8 support. The error occurs in `dequantize_per_tensor_fp8()` when trying to convert FP8 tensors to other formats on MPS. **Solution**: Add a device-aware fallback in `dequantize_per_tensor_fp8()`: - Detect FP8 tensors on MPS devices - Temporarily move tensors to CPU for FP8 conversion (which CPU supports) - Return results to MPS for continued processing This minimal fix enables FP8-quantized models to work on Apple Silicon while: - Maintaining MPS acceleration for the majority of operations - Avoiding performance penalties of full CPU execution --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-26 22:20:13 +00:00