deepseek-ai/DeepEP
1
0
Fork 0
mirror of https://github.com/deepseek-ai/DeepEP.git synced 2026-11-05 00:47:32 +00:00
Code Issues 308 Projects Packages Wiki Activity

[GH-ISSUE #383] Build fails with nvshmem installed as RPM in /usr/lib64 — unable to create wheel #259

New issue
Open
opened 2026-10-26 09:57:34 +00:00 by bsakaguchi · 2 comments
bsakaguchi commented 2026-10-26 09:57:34 +00:00
Owner
Copy link

Originally created by @vshawrh on GitHub (Aug 28, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/383

Description

When trying to build this package against nvshmem provided as an RPM (installed under /usr/lib64 and /usr/include/nvshmem_${CUDA_MAJOR_VERSION}), the wheel build fails due to incorrect linking flags and library discovery issues.

Currently, the setup.py assumes local paths like ${nvshmem_dir}/lib and ${nvshmem_dir}/include. This breaks in RPM-based environments where nvshmem is installed system-wide.

Issues observed

  1. Link flags not generic

    • Existing extra_link_args used explicit -l:libnvshmem_host.so and -l:libnvshmem_device.a.
    • These are brittle because they rely on filenames instead of sonames (-lnvshmem_host, -lnvshmem_device).
    • Also, nvshmem_bootstrap_uid.so was incorrectly linked, but not actually required.

  2. RPATH handling

    • Previously only ${nvshmem_dir}/lib was added to -rpath.
    • On RPM-based installs, libraries are under /usr/lib64/nvshmem/${CUDA_MAJOR_VERSION} and /usr/lib64, requiring explicit rpath.

  3. Device linking

    • nvcc_dlink was missing system paths for nvshmem_device, causing unresolved references during device code linking.

  4. Wheel creation fails

    • Since the build cannot resolve nvshmem_host and nvshmem_device properly, pip wheel . fails to produce a wheel on systems where nvshmem is installed as an RPM.
    • Error messages include missing libnvshmem_host.so.3 and unresolved device symbols.

Changes needed

  • Use system include and library directories:

    include_dirs.extend(['/usr/include', f'/usr/include/nvshmem_{os.getenv("CUDA_MAJOR_VERSION")}'])
library_dirs.extend(['/usr/lib64', f'/usr/lib64/nvshmem/{os.getenv("CUDA_MAJOR_VERSION")}'])

  • Update linker flags to use sonames instead of filenames:

    extra_link_args.extend([
    '-lnvshmem',
    '-Wl,--no-as-needed',
    '-lnvshmem_host',
    '-lnvshmem_device',
    f'-Wl,-rpath,/usr/lib64/nvshmem/{os.getenv("CUDA_MAJOR_VERSION")}:/usr/lib64'
])

  • Update device linking with nvcc:

    nvcc_dlink.extend([
    '-dlink',
    '-L/usr/lib64',
    f'-L/usr/lib64/nvshmem/{os.getenv("CUDA_MAJOR_VERSION")}',
    '-lnvshmem_device'
])

Expected outcome

  • Build succeeds when nvshmem is installed from RPM.
  • Wheel (.whl) can be created and installed in a clean environment.
  • Linker resolves libnvshmem_host.so.3 and libnvshmem_device dynamically without hardcoding filenames.
Originally created by @vshawrh on GitHub (Aug 28, 2025). Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/383 ### Description When trying to build this package against `nvshmem` provided as an RPM (installed under `/usr/lib64` and `/usr/include/nvshmem_${CUDA_MAJOR_VERSION}`), the wheel build fails due to incorrect linking flags and library discovery issues. Currently, the `setup.py` assumes local paths like `${nvshmem_dir}/lib` and `${nvshmem_dir}/include`. This breaks in RPM-based environments where `nvshmem` is installed system-wide. #### Issues observed 1. **Link flags not generic** * Existing `extra_link_args` used explicit `-l:libnvshmem_host.so` and `-l:libnvshmem_device.a`. * These are brittle because they rely on filenames instead of sonames (`-lnvshmem_host`, `-lnvshmem_device`). * Also, `nvshmem_bootstrap_uid.so` was incorrectly linked, but not actually required. 2. **RPATH handling** * Previously only `${nvshmem_dir}/lib` was added to `-rpath`. * On RPM-based installs, libraries are under `/usr/lib64/nvshmem/${CUDA_MAJOR_VERSION}` and `/usr/lib64`, requiring explicit rpath. 3. **Device linking** * `nvcc_dlink` was missing system paths for `nvshmem_device`, causing unresolved references during device code linking. 4. **Wheel creation fails** * Since the build cannot resolve `nvshmem_host` and `nvshmem_device` properly, `pip wheel .` fails to produce a wheel on systems where `nvshmem` is installed as an RPM. * Error messages include missing `libnvshmem_host.so.3` and unresolved device symbols. #### Changes needed * Use **system include and library directories**: ```python include_dirs.extend(['/usr/include', f'/usr/include/nvshmem_{os.getenv("CUDA_MAJOR_VERSION")}']) library_dirs.extend(['/usr/lib64', f'/usr/lib64/nvshmem/{os.getenv("CUDA_MAJOR_VERSION")}']) ``` * Update **linker flags** to use sonames instead of filenames: ```python extra_link_args.extend([ '-lnvshmem', '-Wl,--no-as-needed', '-lnvshmem_host', '-lnvshmem_device', f'-Wl,-rpath,/usr/lib64/nvshmem/{os.getenv("CUDA_MAJOR_VERSION")}:/usr/lib64' ]) ``` * Update **device linking** with `nvcc`: ```python nvcc_dlink.extend([ '-dlink', '-L/usr/lib64', f'-L/usr/lib64/nvshmem/{os.getenv("CUDA_MAJOR_VERSION")}', '-lnvshmem_device' ]) ``` #### Expected outcome * Build succeeds when `nvshmem` is installed from RPM. * Wheel (`.whl`) can be created and installed in a clean environment. * Linker resolves `libnvshmem_host.so.3` and `libnvshmem_device` dynamically without hardcoding filenames.
bsakaguchi commented 2026-10-26 09:57:35 +00:00
Author
Owner
Copy link

@EmilienM commented on GitHub (Aug 29, 2025):

To make the build process more portable and user-configurable, I propose that we update setup.py to use environment variables for discovering NVSHMEM paths and linker flags.
We could adopt an approach that prioritizes explicit environment variables, with a fallback mechanism for backward compatibility.

New Environment Variables:

  • NVSHMEM_INCLUDE_PATH: An os.pathsep-separated list of include directories for the NVSHMEM headers.
  • NVSHMEM_LIBRARY_PATH: An os.pathsep-separated list of library directories for the NVSHMEM libraries.
  • NVSHMEM_LDFLAGS: A string containing all necessary linker flags (e.g., -lnvshmem -Wl,--no-as-needed -lnvshmem_host -lnvshmem_device -Wl,-rpath,...).

Implement Fallback Logic:

  • To maintain compatibility with existing setups, if NVSHMEM_INCLUDE_PATH is not defined but NVSHMEM_DIR is, set NVSHMEM_INCLUDE_PATH = f"{NVSHMEM_DIR}/include".
  • Similarly, if NVSHMEM_LIBRARY_PATH is not defined but NVSHMEM_DIR is, set NVSHMEM_LIBRARY_PATH = f"{NVSHMEM_DIR}/lib".

Conditional Compilation:

The NVSHMEM extension should only be built if both NVSHMEM_INCLUDE_PATH and NVSHMEM_LIBRARY_PATH are successfully resolved (either directly or via the NVSHMEM_DIR fallback). This prevents build failures when NVSHMEM is not available or configured.

Note

I've done similar work in flashinfer-python: github.com/flashinfer-ai/flashinfer@ce68e1d0cc

<!-- gh-comment-id:3237603517 --> @EmilienM commented on GitHub (Aug 29, 2025): To make the build process more portable and user-configurable, I propose that we update `setup.py` to use environment variables for discovering NVSHMEM paths and linker flags. We could adopt an approach that prioritizes explicit environment variables, with a fallback mechanism for backward compatibility. ### New Environment Variables: * NVSHMEM_INCLUDE_PATH: An os.pathsep-separated list of include directories for the NVSHMEM headers. * NVSHMEM_LIBRARY_PATH: An os.pathsep-separated list of library directories for the NVSHMEM libraries. * NVSHMEM_LDFLAGS: A string containing all necessary linker flags (e.g., -lnvshmem -Wl,--no-as-needed -lnvshmem_host -lnvshmem_device -Wl,-rpath,...). ### Implement Fallback Logic: * To maintain compatibility with existing setups, if NVSHMEM_INCLUDE_PATH is not defined but NVSHMEM_DIR is, set NVSHMEM_INCLUDE_PATH = f"{NVSHMEM_DIR}/include". * Similarly, if NVSHMEM_LIBRARY_PATH is not defined but NVSHMEM_DIR is, set NVSHMEM_LIBRARY_PATH = f"{NVSHMEM_DIR}/lib". ### Conditional Compilation: The NVSHMEM extension should only be built if both NVSHMEM_INCLUDE_PATH and NVSHMEM_LIBRARY_PATH are successfully resolved (either directly or via the NVSHMEM_DIR fallback). This prevents build failures when NVSHMEM is not available or configured. ### Note I've done similar work in flashinfer-python: https://github.com/flashinfer-ai/flashinfer/commit/ce68e1d0cc8a69da4ead85a5280a183f3e2a5a00
bsakaguchi commented 2026-10-26 09:57:35 +00:00
Author
Owner
Copy link

@YidaWeng commented on GitHub (Oct 1, 2026):

V2.5 removed the NVSHMEM backend entirely (README news: "Fully remove V1, including its APIs, NVSHMEM backend, and legacy documentation. NVSHMEM is no longer a dependency").

setup.py on current main only links NCCL (no libnvshmem_* paths), so the RPM /usr/lib64 discovery issue described here no longer applies. This issue looks safe to close as obsolete for V2.5+.

<!-- gh-comment-id:5929484479 --> @YidaWeng commented on GitHub (Oct 1, 2026): V2.5 removed the NVSHMEM backend entirely (`README` news: "Fully remove V1, including its APIs, NVSHMEM backend, and legacy documentation. NVSHMEM is no longer a dependency"). `setup.py` on current `main` only links NCCL (no `libnvshmem_*` paths), so the RPM `/usr/lib64` discovery issue described here no longer applies. This issue looks safe to close as obsolete for V2.5+.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
hybrid-ep
chore/remove-clang-format-yapf
v2.5
nvDev
try-fix-cu128
mori-ep
antgroup-opt
compat-old-torch
add-linter
eager-rdma
v1.2.1
Labels
Clear labels   
pull-request
Mirrored from GitHub Pull Request

No labels
pull-request
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
deepseek-ai/DeepEP#259
No description provided.