[GH-ISSUE #96] fp8_wgrad_gemm Support unaligned n,k and gmem stride? #54

New issue

Open

opened 2026-10-26 21:11:20 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-26 21:11:20 +00:00

Owner

Originally created by @John-tsy on GitHub (May 12, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepGEMM/issues/96

in this commit
github.com/deepseek-ai/DeepGEMM@919f55be9c

I found that it can only pass when the stride(0) of tensor lhs and tensor rhs is aligned to 64.

I would like to know how you ensure the alignment requirement of stride(0)？

extremely grateful

Originally created by @John-tsy on GitHub (May 12, 2025). Original GitHub issue: https://github.com/deepseek-ai/DeepGEMM/issues/96 in this commit https://github.com/deepseek-ai/DeepGEMM/commit/919f55be9c77afdd593a0207c4bb42d7d7b99023 I found that it can only pass when the stride(0) of tensor lhs and tensor rhs is aligned to 64. I would like to know how you ensure the alignment requirement of stride(0)？ extremely grateful

bsakaguchi commented

2026-10-26 21:11:21 +00:00

Author

Owner

@zheanxu commented on GitHub (May 13, 2025):

Both lhs and rhs must have a stride(0) that is a multiple of 16. This is a requirement for asynchronous data transfers using the Tensor Memory Accelerator (TMA).

As documented in the CUDA Driver API:

globalStrides array, which specifies tensor stride of each of the lower tensorRank - 1 dimensions in bytes, must be a multiple of 16 and less than 2^40.

To comply with this alignment requirement, you can add padding to ensure stride(0) is a multiple of 16.

@zheanxu commented on GitHub (May 13, 2025): Both `lhs` and `rhs` must have a `stride(0)` that is a multiple of 16. This is a requirement for asynchronous data transfers using the Tensor Memory Accelerator (TMA). As documented in the [CUDA Driver API](https://docs.nvidia.com/cuda/cuda-driver-api/group__CUDA__TENSOR__MEMORY.html#group__CUDA__TENSOR__MEMORY_1ga7c7d2aaac9e49294304e755e6f341d7): > `globalStrides` array, which specifies tensor stride of each of the lower tensorRank - 1 dimensions in bytes, must be a multiple of 16 and less than 2^40. To comply with this alignment requirement, you can add padding to ensure `stride(0)` is a multiple of 16.

bsakaguchi referenced this issue

2026-10-26 21:15:01 +00:00

[PR #54] [CLOSED] Fix 2D passed in TMA scale #221