[GH-ISSUE #25] Device-Level Balance Loss and Communication Balance Loss #24

New issue

Closed

opened 2026-10-26 21:06:53 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-26 21:06:53 +00:00

Owner

Originally created by @hashming on GitHub (May 13, 2024).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-V2/issues/25

What's the main difference?
As I see from your paper, pi' == pi'', and fi' = some_coeff * fi''
maybe fi'' should be:
... (Token t is sent to Device i from Device j where j!=i)

Originally created by @hashming on GitHub (May 13, 2024). Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-V2/issues/25 What's the main difference? As I see from your paper, pi' == pi'', and fi' = some_coeff * fi'' maybe fi'' should be: ... (Token t is sent to Device i from Device j where j!=i)

bsakaguchi closed this issue

2026-10-26 21:06:53 +00:00

bsakaguchi commented

2026-10-26 21:06:54 +00:00

Author

Owner

@hashming commented on GitHub (May 13, 2024):

maybe the authors already meant this by using the word "sent"...

@hashming commented on GitHub (May 13, 2024): maybe the authors already meant this by using the word "sent"...

bsakaguchi commented

2026-10-26 21:06:54 +00:00

Author

Owner

@tianzhangwu commented on GitHub (Dec 30, 2024):

solved my puzzle

@tianzhangwu commented on GitHub (Dec 30, 2024): solved my puzzle

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

deepseek-ai/DeepSeek-V2#24

No description provided.

Rows
Columns