mirror of
https://github.com/deepseek-ai/ESFT.git
synced 2026-11-05 00:57:35 +00:00
[GH-ISSUE #22] Unable to reproduce reported performance #17
Labels
No labels
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
deepseek-ai/ESFT#17
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @jensen-Shen-Li on GitHub (May 14, 2026).
Original GitHub issue: https://github.com/deepseek-ai/ESFT/issues/22
Hi, thank you for releasing the code and models.
Hi, I’m trying to reproduce the paper results. When I directly evaluate the released adapter, I can get results close to those reported in the paper. However, when I run the whole pipeline from scratch — computing expert scores, generating the expert config, training the adapter, and then evaluating it — the final adapter performs much worse. I used 8 GPUs, the same base.yaml, the same base model, and did not modify the code. Could you please let me know if there are any extra details needed to reproduce the released adapter, such as the exact expert config, random seed, dataset version, score-computation settings, or checkpoint selection strategy?