[PR #2766] [MERGED] FLUXNATION FLUX.1 Fused Neuromorphic SPIKE Attention & Step Cache Cuda Kernel #2753

New issue

Closed

opened 2026-10-28 21:12:20 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-28 21:12:20 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/ComfyUI-Manager/pull/2766
Author: @ULT7RA
Created: 4/6/2026
Status: ✅ Merged
Merged: 4/13/2026
Merged by: @ltdrdata

Base: main ← Head: add-fluxnation

📝 Commits (1)

4ca2aa4 Add FLUXNATION fused spike attention CUDA kernel for FLUX.1

📊 Changes

1 file changed (+11 additions, -0 deletions)

View changed files

📝 custom-node-list.json (+11 -0)

📄 Description

Adding FLUXNATION to the custom node list. This is a fused FP8 CUDA kernel for FLUX.1 that replaces the entire SingleStreamBlock forward pass with a single torch.ops call. Modulation, dual GEMMs, QKV projection, RoPE, attention, gating and residual are all fused into one kernel call with zero Python overhead. On top of that it ships with neuromorphic spike attention which scores every attention block by real dot product similarity and drops the ones that dont matter once image structure has formed keeping only the top 45 percent. Step caching then replays the cached output on alternating spike steps so you get zero attention compute every other step. The result is 30 percent faster generation than stock ComfyUI with no quality loss. Works with all 38 ComfyUI samplers at any step count. A must have for anyone running a 40 series Ada Lovelace card including the 4070 4080 4090 and the Ti variants. FP16 ports are included for 10 series Pascal cards like the 1080 Ti, 20 series Turing cards like the 2080 Ti, and 30 series Ampere cards like the 3090 so older hardware is covered too. Repo is at https://github.com/ULT7RA/FLUXNATION

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/ComfyUI-Manager/pull/2766 **Author:** [@ULT7RA](https://github.com/ULT7RA) **Created:** 4/6/2026 **Status:** ✅ Merged **Merged:** 4/13/2026 **Merged by:** [@ltdrdata](https://github.com/ltdrdata) **Base:** `main` ← **Head:** `add-fluxnation` --- ### 📝 Commits (1) - [`4ca2aa4`](https://github.com/Comfy-Org/ComfyUI-Manager/commit/4ca2aa42483f65d6ec1ebcd8d9fe2cc37d7c8415) Add FLUXNATION fused spike attention CUDA kernel for FLUX.1 ### 📊 Changes **1 file changed** (+11 additions, -0 deletions) <details> <summary>View changed files</summary> 📝 `custom-node-list.json` (+11 -0) </details> ### 📄 Description Adding FLUXNATION to the custom node list. This is a fused FP8 CUDA kernel for FLUX.1 that replaces the entire SingleStreamBlock forward pass with a single torch.ops call. Modulation, dual GEMMs, QKV projection, RoPE, attention, gating and residual are all fused into one kernel call with zero Python overhead. On top of that it ships with neuromorphic spike attention which scores every attention block by real dot product similarity and drops the ones that dont matter once image structure has formed keeping only the top 45 percent. Step caching then replays the cached output on alternating spike steps so you get zero attention compute every other step. The result is 30 percent faster generation than stock ComfyUI with no quality loss. Works with all 38 ComfyUI samplers at any step count. A must have for anyone running a 40 series Ada Lovelace card including the 4070 4080 4090 and the Ti variants. FP16 ports are included for 10 series Pascal cards like the 1080 Ti, 20 series Turing cards like the 2080 Ti, and 30 series Ampere cards like the 3090 so older hardware is covered too. Repo is at https://github.com/ULT7RA/FLUXNATION --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-28 21:12:20 +00:00