[GH-ISSUE #486] Questions about environment setups #314

New issue

Closed

opened 2026-10-26 09:58:28 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-26 09:58:28 +00:00

Owner

Originally created by @CUHKSZzxy on GitHub (Nov 10, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/486

Thanks for such great work !

Currently I'm trying to use DeepEP for multi-node MoE model deployment, though I have read through related issues and docs, there are still several confusions that require official confirmations, to make sure I do not misunderstand.

How to validate the DeepEP environment

For now, i am using the official testcases (intra / inter / low latency). And I suppose that if all those tests succeed, DeepEP env should be good to go, is such assumption right ?

About NVSHMEM

According to NVSHMEM install guide, it involves two stages 1. install NVSHMEM binary, 2. Enable NVSHMEM IBGDA support. And for IBGDA suppport step, we choose either 2.1 Configure NVIDIA driver or 2.2 Install GDRCopy and load the gdrdrv kernel module.

Is the second step Enable NVSHMEM IBGDA support a necessary steps for multi-node deployment?

For 2.1 Configure NVIDIA driver, we need to modify driver config and reboot on host machine. But if we choose 2.2 Install GDRCopy and load the gdrdrv kernel module approach.

Do we still need to reboot the host machine?
Can we wrap all NVSHMEM / GDRCopy related things into a docker image, without having to perform anything special on the host machine?
How can I verify IBGDA support setups? Use the official testcases (intra / inter / low latency) ?

Thanks in advance, any help would be appreciated.

Originally created by @CUHKSZzxy on GitHub (Nov 10, 2025). Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/486 Thanks for such great work ! Currently I'm trying to use DeepEP for multi-node MoE model deployment, though I have read through related issues and docs, there are still several confusions that require official confirmations, to make sure I do not misunderstand. 1. How to validate the DeepEP environment For now, i am using the [official testcases (intra / inter / low latency)](https://github.com/deepseek-ai/DeepEP/tree/main/tests). And I suppose that if all those tests succeed, DeepEP env should be good to go, is such assumption right ? 2. About NVSHMEM According to [NVSHMEM install guide](https://github.com/deepseek-ai/DeepEP/blob/main/third-party/README.md), it involves two stages `1. install NVSHMEM binary`, `2. Enable NVSHMEM IBGDA support`. And for IBGDA suppport step, we choose either `2.1 Configure NVIDIA driver` or `2.2 Install GDRCopy and load the gdrdrv kernel module`. - Is the second step `Enable NVSHMEM IBGDA support` a necessary steps for multi-node deployment? For `2.1 Configure NVIDIA driver`, we need to modify driver config and reboot on host machine. But if we choose `2.2 Install GDRCopy and load the gdrdrv kernel module` approach. - Do we still need to reboot the host machine? - Can we wrap all NVSHMEM / GDRCopy related things into a docker image, without having to perform anything special on the host machine? - How can I verify IBGDA support setups? Use the official testcases (intra / inter / low latency) ? Thanks in advance, any help would be appreciated.

bsakaguchi closed this issue

2026-10-26 09:58:28 +00:00

bsakaguchi commented

2026-10-26 09:58:30 +00:00

Author

Owner

@tinuademargaret commented on GitHub (Nov 21, 2025):

Not an official confirmation, but afaik:

IBGDA support is required for multi-node DeepEP deployments. IBGDA provides the efficient inter-node GPU-to-GPU communication over InfiniBand that DeepEP relies on.

If you enable IBGDA via the GDRCopy + gdrdrv kernel module route, you don’t need to reboot the host after loading the kernel module. However, the kernel modules cannot be built or loaded inside docker, even with --privileged since docker does not have the ability to modify the host kernel. The gdrdrv module has to be built and loaded on the host machine itself.

Once the host has the kernel module loaded, the container only needs the GDRCopy userspace library, NVSHMEM, and DeepEP installed. These parts are safe to install inside Docker, and you can run the container with --gpus all --privileged so DeepEP and NVSHMEM can access the host’s GPUs and the already loaded module.

@tinuademargaret commented on GitHub (Nov 21, 2025): Not an official confirmation, but afaik: IBGDA support is required for multi-node DeepEP deployments. IBGDA provides the efficient inter-node GPU-to-GPU communication over InfiniBand that DeepEP relies on. If you enable IBGDA via the GDRCopy + gdrdrv kernel module route, you don’t need to reboot the host after loading the kernel module. However, the kernel modules cannot be built or loaded inside docker, even with `--privileged` since docker does not have the ability to modify the host kernel. The gdrdrv module has to be built and loaded on the host machine itself. Once the host has the kernel module loaded, the container only needs the GDRCopy userspace library, NVSHMEM, and DeepEP installed. These parts are safe to install inside Docker, and you can run the container with `--gpus all --privileged` so DeepEP and NVSHMEM can access the host’s GPUs and the already loaded module.

bsakaguchi commented

2026-10-26 09:58:30 +00:00

Author

Owner

@CUHKSZzxy commented on GitHub (Nov 24, 2025):

Not an official confirmation, but afaik:

IBGDA support is required for multi-node DeepEP deployments. IBGDA provides the efficient inter-node GPU-to-GPU communication over InfiniBand that DeepEP relies on.

If you enable IBGDA via the GDRCopy + gdrdrv kernel module route, you don’t need to reboot the host after loading the kernel module. However, the kernel modules cannot be built or loaded inside docker, even with --privileged since docker does not have the ability to modify the host kernel. The gdrdrv module has to be built and loaded on the host machine itself.

Once the host has the kernel module loaded, the container only needs the GDRCopy userspace library, NVSHMEM, and DeepEP installed. These parts are safe to install inside Docker, and you can run the container with --gpus all --privileged so DeepEP and NVSHMEM can access the host’s GPUs and the already loaded module.

Really appreciate your detailed explanations! I now have a clearer understanding of those concerns.

@CUHKSZzxy commented on GitHub (Nov 24, 2025): > Not an official confirmation, but afaik: > > IBGDA support is required for multi-node DeepEP deployments. IBGDA provides the efficient inter-node GPU-to-GPU communication over InfiniBand that DeepEP relies on. > > If you enable IBGDA via the GDRCopy + gdrdrv kernel module route, you don’t need to reboot the host after loading the kernel module. However, the kernel modules cannot be built or loaded inside docker, even with `--privileged` since docker does not have the ability to modify the host kernel. The gdrdrv module has to be built and loaded on the host machine itself. > > Once the host has the kernel module loaded, the container only needs the GDRCopy userspace library, NVSHMEM, and DeepEP installed. These parts are safe to install inside Docker, and you can run the container with `--gpus all --privileged` so DeepEP and NVSHMEM can access the host’s GPUs and the already loaded module. Really appreciate your detailed explanations! I now have a clearer understanding of those concerns.

bsakaguchi referenced this issue

2026-10-27 21:03:44 +00:00

[PR #353] [MERGED] fix combine timeout due to delayed forwarder min head update #533