[GH-ISSUE #125] Considering Q4 introduces high PPL, does ktransfomers support official claimed F8 for 671B? #68

New issue

Closed

opened 2026-10-29 14:33:33 +00:00 by bsakaguchi · 4 comments

bsakaguchi commented

2026-10-29 14:33:33 +00:00

Owner

Originally created by @ghostplant on GitHub (Feb 10, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/125

Considering Q4 introduces high PPL, does ktransfomers support official claimed F8 for 671B? What's the token generation speed?

Originally created by @ghostplant on GitHub (Feb 10, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/125 Considering Q4 introduces high PPL, does ktransfomers support official claimed F8 for 671B? What's the token generation speed?

bsakaguchi

2026-10-29 14:33:33 +00:00

closed this issue
added the
stale
label

bsakaguchi commented

2026-10-29 14:33:34 +00:00

Author

Owner

@chenht2022 commented on GitHub (Feb 10, 2025):

Good Insight! But at this point, we only support and test Q4 and Q8. We will consider to add your recommended feature in the future.

@chenht2022 commented on GitHub (Feb 10, 2025): Good Insight! But at this point, we only support and test Q4 and Q8. We will consider to add your recommended feature in the future.

bsakaguchi commented

2026-10-29 14:33:34 +00:00

Author

Owner

@ghostplant commented on GitHub (Feb 10, 2025):

OK, by the way, may I know what the GPU type it is used by? https://github.com/kvcache-ai/ktransformers?tab=readme-ov-file#-show-cases

@ghostplant commented on GitHub (Feb 10, 2025): OK, by the way, may I know what the GPU type it is used by? https://github.com/kvcache-ai/ktransformers?tab=readme-ov-file#-show-cases

bsakaguchi commented

2026-10-29 14:33:34 +00:00

Author

Owner

@Azure-Tang commented on GitHub (Feb 11, 2025):

OK, by the way, may I know what the GPU type it is used by? https://github.com/kvcache-ai/ktransformers?tab=readme-ov-file#-show-cases

4090d

@Azure-Tang commented on GitHub (Feb 11, 2025): > OK, by the way, may I know what the GPU type it is used by? https://github.com/kvcache-ai/ktransformers?tab=readme-ov-file#-show-cases 4090d

bsakaguchi commented

2026-10-29 14:33:34 +00:00

Author

Owner

@ErvinXie commented on GitHub (Dec 11, 2025):

This issue has been automatically closed due to inactivity for 60 days.

If you believe this issue is still relevant, please feel free to reopen it with additional information or context.

Thank you for your contribution to KTransformers!

此 issue 因超过 60 天无活动而被自动关闭。

如果您认为此问题仍然存在，请随时重新打开并提供更多信息或上下文。

感谢您对 KTransformers 的贡献！

@ErvinXie commented on GitHub (Dec 11, 2025): This issue has been automatically closed due to inactivity for 60 days. If you believe this issue is still relevant, please feel free to reopen it with additional information or context. Thank you for your contribution to KTransformers! --- 此 issue 因超过 60 天无活动而被自动关闭。如果您认为此问题仍然存在，请随时重新打开并提供更多信息或上下文。感谢您对 KTransformers 的贡献！