[GH-ISSUE #175] Why minimum Byte Size Forced to 2 when calculation hidden bytes for FP8 in get_hidden_bytes() #149

New issue

Closed

opened 2026-10-26 09:55:15 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-26 09:55:15 +00:00

Owner

Originally created by @000945 on GitHub (May 23, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/175

in https://github.com/deepseek-ai/DeepEP/blob/main/README.md Interfaces and examples section,

def get_hidden_bytes(x: torch.Tensor) -> int:
t = x[0] if isinstance(x, tuple) else x
return t.size(1) * max(t.element_size(), 2)

FP8 tensors use 1 byte per element, but the function forces a 2-byte minimum，Is there a memory/performance trade-off?

Originally created by @000945 on GitHub (May 23, 2025). Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/175 in https://github.com/deepseek-ai/DeepEP/blob/main/README.md Interfaces and examples section, def get_hidden_bytes(x: torch.Tensor) -> int: t = x[0] if isinstance(x, tuple) else x return t.size(1) * max(t.element_size(), 2) FP8 tensors use 1 byte per element, but the function forces a 2-byte minimum，Is there a memory/performance trade-off?

bsakaguchi closed this issue

2026-10-26 09:55:15 +00:00

bsakaguchi commented

2026-10-26 09:55:16 +00:00

Author

Owner

@LyricZhao commented on GitHub (May 27, 2025):

Dispatch can use FP8, but combine always uses BF16 (2 bytes). Using BF16 for combine is important for model performance/precision.

@LyricZhao commented on GitHub (May 27, 2025): Dispatch can use FP8, but combine always uses BF16 (2 bytes). Using BF16 for combine is important for model performance/precision.

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

deepseek-ai/DeepEP#149

No description provided.

Rows
Columns