[GH-ISSUE #72] Support for gpt fast. #59

New issue

Closed

opened 2026-10-26 21:02:54 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-26 21:02:54 +00:00

Owner

Originally created by @briandw on GitHub (Dec 7, 2023).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-Coder/issues/72

I'm trying to get DeepSeek to work with gpt-fast.
https://github.com/pytorch-labs/gpt-fast/issues/35

I have it converted and running but the output is gibberish.
Something is going wrong in conversion.
Anyone interested in helping out?

Originally created by @briandw on GitHub (Dec 7, 2023). Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-Coder/issues/72 I'm trying to get DeepSeek to work with gpt-fast. https://github.com/pytorch-labs/gpt-fast/issues/35 I have it converted and running but the output is gibberish. Something is going wrong in conversion. Anyone interested in helping out?

bsakaguchi closed this issue

2026-10-26 21:02:54 +00:00

bsakaguchi commented

2026-10-26 21:02:55 +00:00

Author

Owner

@briandw commented on GitHub (Dec 12, 2023):

Looks like the only difference is in the scaled rotary embeddings.
If you replace the tokenizer and fix the sampling and use this PR https://github.com/pytorch-labs/gpt-fast/pull/44
and the correct model params. gpt fast works!

@briandw commented on GitHub (Dec 12, 2023): Looks like the only difference is in the scaled rotary embeddings. If you replace the tokenizer and fix the sampling and use this PR https://github.com/pytorch-labs/gpt-fast/pull/44 and the correct model params. gpt fast works!