[GH-ISSUE #309] fix: operator precedence bug in pack_ue8m0_to_int assertion (mantissa check always passes) #154

New issue

Open

opened 2026-10-26 21:13:49 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-26 21:13:49 +00:00

Owner

Originally created by @kuishou68 on GitHub (Apr 19, 2026).
Original GitHub issue: https://github.com/deepseek-ai/DeepGEMM/issues/309

In deep_gemm/utils/math.py, the pack_ue8m0_to_int function has an operator precedence bug that makes the validation assertion ineffective.

Bug

def pack_ue8m0_to_int(x: torch.Tensor):
    assert x.dtype == torch.float and x.size(-1) % 4 == 0
    assert (x.view(torch.int) & ((1 << 23) - 1) == 0).all()  # BUG: wrong precedence

In Python, == has higher precedence than &. So the expression is parsed as:

x.view(torch.int) & (((1 << 23) - 1) == 0)

((1 << 23) - 1) == 0 evaluates to False (which is 0 numerically), then x.view(torch.int) & 0 gives all zeros, and .all() on all zeros returns True.

This means the assertion always passes regardless of whether the mantissa bits are actually zero, so it silently accepts invalid inputs instead of catching them.

Fix

Add parentheses to ensure the AND operation happens before the equality check:

assert ((x.view(torch.int) & ((1 << 23) - 1)) == 0).all()

This correctly checks that all mantissa bits (lower 23 bits) are zero, which is the intended behavior for UE8M0 format values.

Originally created by @kuishou68 on GitHub (Apr 19, 2026). Original GitHub issue: https://github.com/deepseek-ai/DeepGEMM/issues/309 In `deep_gemm/utils/math.py`, the `pack_ue8m0_to_int` function has an operator precedence bug that makes the validation assertion ineffective. ## Bug ```python def pack_ue8m0_to_int(x: torch.Tensor): assert x.dtype == torch.float and x.size(-1) % 4 == 0 assert (x.view(torch.int) & ((1 << 23) - 1) == 0).all() # BUG: wrong precedence ``` In Python, `==` has higher precedence than `&`. So the expression is parsed as: ``` x.view(torch.int) & (((1 << 23) - 1) == 0) ``` `((1 << 23) - 1) == 0` evaluates to `False` (which is `0` numerically), then `x.view(torch.int) & 0` gives all zeros, and `.all()` on all zeros returns `True`. This means the assertion **always passes** regardless of whether the mantissa bits are actually zero, so it silently accepts invalid inputs instead of catching them. ## Fix Add parentheses to ensure the AND operation happens before the equality check: ```python assert ((x.view(torch.int) & ((1 << 23) - 1)) == 0).all() ``` This correctly checks that all mantissa bits (lower 23 bits) are zero, which is the intended behavior for UE8M0 format values.

bsakaguchi commented

2026-10-26 21:13:50 +00:00

Author

Owner

@kuishou68 commented on GitHub (Apr 19, 2026):

I've opened PR #310 to fix this issue.

@kuishou68 commented on GitHub (Apr 19, 2026): I've opened PR #310 to fix this issue.

bsakaguchi referenced this issue

2026-10-26 21:15:32 +00:00

[PR #154] [MERGED] Fix smxx layout assertion #267