[PR #83] [MERGED] Support int8 convrot embedding lookup #132

New issue

Closed

opened 2026-10-26 22:20:41 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 22:20:41 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/comfy-kitchen/pull/83
Author: @kijai
Created: 7/22/2026
Status: ✅ Merged
Merged: 7/25/2026
Merged by: @comfyanonymous

Base: main ← Head: int8_embed

📝 Commits (2)

acdeb6d Support int8 convrot embeddings
3fd6629 linting

📊 Changes

4 files changed (+80 additions, -1 deletions)

View changed files

📝 comfy_kitchen/backends/eager/__init__.py (+12 -0)
📝 comfy_kitchen/backends/eager/quantization.py (+53 -0)
📝 comfy_kitchen/backends/triton/__init__.py (+1 -1)
📝 comfy_kitchen/tensor/int8.py (+14 -0)

📄 Description

Adds support for int8 embedding lookup, useful for text generation models with lm_head or embed_tokens layers such as used in Gemma4.

Compared to the fp8 scaled (the only previously supported quant format for embed lookup), int8 with convrot reaches much lower relative error at ~0.8% vs ~2.6% of fp8_scaled.

Since the embedding lookup is only ~0.10% of the token time, speed is not an issue and I only included the eager backend.

Paired core PR: https://github.com/Comfy-Org/ComfyUI/pull/15035

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/comfy-kitchen/pull/83 **Author:** [@kijai](https://github.com/kijai) **Created:** 7/22/2026 **Status:** ✅ Merged **Merged:** 7/25/2026 **Merged by:** [@comfyanonymous](https://github.com/comfyanonymous) **Base:** `main` ← **Head:** `int8_embed` --- ### 📝 Commits (2) - [`acdeb6d`](https://github.com/Comfy-Org/comfy-kitchen/commit/acdeb6dc61a28dc399fdca2159b81e2ef5c698c1) Support int8 convrot embeddings - [`3fd6629`](https://github.com/Comfy-Org/comfy-kitchen/commit/3fd6629f410c99bf5cf4250e2752710bd1233ae8) linting ### 📊 Changes **4 files changed** (+80 additions, -1 deletions) <details> <summary>View changed files</summary> 📝 `comfy_kitchen/backends/eager/__init__.py` (+12 -0) 📝 `comfy_kitchen/backends/eager/quantization.py` (+53 -0) 📝 `comfy_kitchen/backends/triton/__init__.py` (+1 -1) 📝 `comfy_kitchen/tensor/int8.py` (+14 -0) </details> ### 📄 Description Adds support for int8 embedding lookup, useful for text generation models with lm_head or embed_tokens layers such as used in Gemma4. Compared to the fp8 scaled (the only previously supported quant format for embed lookup), int8 with convrot reaches much lower relative error at ~0.8% vs ~2.6% of fp8_scaled. Since the embedding lookup is only ~0.10% of the token time, speed is not an issue and I only included the eager backend. Paired core PR: https://github.com/Comfy-Org/ComfyUI/pull/15035 --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-26 22:20:41 +00:00

closed this issue
added the
pull-request
label

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

Comfy-Org/comfy-kitchen#132

No description provided.

Rows
Columns