[PR #692] Fix JIT failure on SM100+ with CUDA <= 13.0: use 64-bit size operand for st.bulk #710

New issue

Open

opened 2026-10-27 21:05:28 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-27 21:05:28 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/deepseek-ai/DeepEP/pull/692
Author: @MengYu10151
Created: 7/20/2026
Status: 🔄 Open

Base: main ← Head: fix-st-bulk-size-operand

📝 Commits (1)

747ea8e Fix JIT failure on SM100+ with CUDA <= 13.0: use 64-bit size operand for st.bulk

📊 Changes

1 file changed (+1 additions, -1 deletions)

View changed files

📝 deep_ep/include/deep_ep/common/ptx.cuh (+1 -1)

📄 Description

Summary

st_bulk() in deep_ep/include/deep_ep/common/ptx.cuh passes the st.bulk size operand in a 32-bit register ("r" constraint). The PTX ISA defines this operand as .u64 — the 32-bit form only became legal in PTX ISA 9.0 and is first implemented in ptxas 13.1 (its own diagnostic states: Feature 'st.bulk with 32-bit size argument' requires PTX ISA .version 9.0 or later).

As a result, on CUDA 12.8–13.0 toolchains, runtime JIT compilation of dispatch_copy_epilogue fails on SM100-family GPUs whenever dispatch(..., do_expand=True, do_zero_padding=True) is used. This is not a rare parameter combination: the standard test suite tests/elastic/test_ep.py exercises it in its default flow, so on a clean SM100-family environment (no JIT cache) with CUDA <= 13.0 the first test run fails:

ptxas /tmp/xxx.ptx, line 401; error   : Arguments mismatch for instruction 'mov'
ptxas fatal   : Ptx assembly aborted due to errors

This is likely invisible in environments already on CUDA 13.1+.

Changes

One line: pass the size as a 64-bit operand ("l" constraint), which is accepted by every toolkit version.

Verification

Verified on 8x B300 (sm_103) with tests/elastic/test_ep.py --test-first-only (EP_DISABLE_GIN=1), and by directly compiling the JIT-generated dispatch_copy_epilogue_impl<true, true, true, ...> kernel across toolkits:

ptxas version	32-bit `"r"` (before)	64-bit `"l"` (after)
CUDA 12.9	❌ `Arguments mismatch for instruction 'st.bulk'`	✅ compiles
CUDA 13.0	❌ `Arguments mismatch for instruction 'mov'` — runtime JIT failure on all ranks, test aborts	✅ compiles, full test passes
CUDA 13.1	✅ compiles	✅ compiles

No behavior or performance change where both forms compile: with CUDA 13.1 the generated SASS for the affected kernel is byte-identical before/after (cuobjdump -sass diff is empty, identical register/shared-memory usage), and an 8-GPU A/B run shows identical dispatch bandwidth and passing correctness checks.

To pin down the exact trigger condition, all 8 <kDoExpand, kCachedMode, kDoZeroPadding> template combinations of dispatch_copy_epilogue_impl were compiled with CUDA 13.0 against the unfixed header: only the two instantiations with kDoExpand && kDoZeroPadding (<true, false, true> and <true, true, true>) fail; every other combination compiles fine, since the st_bulk() call sits behind if constexpr (kDoZeroPadding and kDoExpand).

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/deepseek-ai/DeepEP/pull/692 **Author:** [@MengYu10151](https://github.com/MengYu10151) **Created:** 7/20/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `fix-st-bulk-size-operand` --- ### 📝 Commits (1) - [`747ea8e`](https://github.com/deepseek-ai/DeepEP/commit/747ea8edb68fa30f6e6fd664c754bbeb24b79dc6) Fix JIT failure on SM100+ with CUDA <= 13.0: use 64-bit size operand for st.bulk ### 📊 Changes **1 file changed** (+1 additions, -1 deletions) <details> <summary>View changed files</summary> 📝 `deep_ep/include/deep_ep/common/ptx.cuh` (+1 -1) </details> ### 📄 Description # Summary `st_bulk()` in `deep_ep/include/deep_ep/common/ptx.cuh` passes the `st.bulk` size operand in a 32-bit register (`"r"` constraint). The PTX ISA defines this operand as `.u64` — the 32-bit form only became legal in PTX ISA 9.0 and is first implemented in ptxas 13.1 (its own diagnostic states: `Feature 'st.bulk with 32-bit size argument' requires PTX ISA .version 9.0 or later`). As a result, on CUDA 12.8–13.0 toolchains, runtime JIT compilation of `dispatch_copy_epilogue` fails on SM100-family GPUs whenever `dispatch(..., do_expand=True, do_zero_padding=True)` is used. This is not a rare parameter combination: the standard test suite `tests/elastic/test_ep.py` exercises it in its default flow, so on a clean SM100-family environment (no JIT cache) with CUDA <= 13.0 the first test run fails: ``` ptxas /tmp/xxx.ptx, line 401; error : Arguments mismatch for instruction 'mov' ptxas fatal : Ptx assembly aborted due to errors ``` This is likely invisible in environments already on CUDA 13.1+. # Changes One line: pass the size as a 64-bit operand (`"l"` constraint), which is accepted by every toolkit version. # Verification Verified on 8x B300 (sm_103) with `tests/elastic/test_ep.py --test-first-only` (`EP_DISABLE_GIN=1`), and by directly compiling the JIT-generated `dispatch_copy_epilogue_impl<true, true, true, ...>` kernel across toolkits: | ptxas version | 32-bit `"r"` (before) | 64-bit `"l"` (after) | |---|---|---| | CUDA 12.9 | ❌ `Arguments mismatch for instruction 'st.bulk'` | ✅ compiles | | CUDA 13.0 | ❌ `Arguments mismatch for instruction 'mov'` — runtime JIT failure on all ranks, test aborts | ✅ compiles, full test passes | | CUDA 13.1 | ✅ compiles | ✅ compiles | No behavior or performance change where both forms compile: with CUDA 13.1 the generated SASS for the affected kernel is byte-identical before/after (`cuobjdump -sass` diff is empty, identical register/shared-memory usage), and an 8-GPU A/B run shows identical dispatch bandwidth and passing correctness checks. To pin down the exact trigger condition, all 8 `<kDoExpand, kCachedMode, kDoZeroPadding>` template combinations of `dispatch_copy_epilogue_impl` were compiled with CUDA 13.0 against the unfixed header: only the two instantiations with `kDoExpand && kDoZeroPadding` (`<true, false, true>` and `<true, true, true>`) fail; every other combination compiles fine, since the `st_bulk()` call sits behind `if constexpr (kDoZeroPadding and kDoExpand)`. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi added the

pull-request

label

2026-10-27 21:05:28 +00:00

bsakaguchi referenced this issue

2026-10-27 21:05:38 +00:00

[PR #712] Reject NCCL runtimes older than the build #725