[GH-ISSUE #571] Unexpected Performance Drop Caused by Explicit deep_ep.Buffer Memory Release #352

New issue

Open

opened 2026-10-26 09:59:00 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 09:59:00 +00:00

Owner

Originally created by @Eddie-Wang1120 on GitHub (Jan 26, 2026).
Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/571

Problem description

Currently, we explicitly release deep_ep.Buffer by calling del buffer and setting buffer = None in order to free device memory:

buffer = deep_ep.Buffer(
    group,
    int(2e9),
    int(1e9),
    low_latency_mode=args.test_ll_compatibility,
    num_qps_per_rank=num_qps_per_rank,
    explicitly_destroy=False,
)
assert num_local_ranks == 8 and num_ranks > 8

del buffer
buffer = None

However, this approach forces the buffer to be fully re-created every time it is used again. After upgrading to a newer NVSHMEM version, this repeated creation and destruction introduces a noticeable performance regression, likely due to the increased overhead in buffer initialization and NVSHMEM internal setup.

We would like to know:

Is there a recommended way to explicitly release or recycle deep_ep.Buffer memory without destroying the entire buffer object?
Are there any best practices when using deep_ep.Buffer with newer NVSHMEM versions to mitigate this performance drop?

Any guidance or suggestions would be greatly appreciated.

Originally created by @Eddie-Wang1120 on GitHub (Jan 26, 2026). Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/571 ## Problem description Currently, we explicitly release `deep_ep.Buffer` by calling `del buffer` and setting `buffer = None` in order to free device memory: ```python buffer = deep_ep.Buffer( group, int(2e9), int(1e9), low_latency_mode=args.test_ll_compatibility, num_qps_per_rank=num_qps_per_rank, explicitly_destroy=False, ) assert num_local_ranks == 8 and num_ranks > 8 del buffer buffer = None ``` However, this approach forces the buffer to be fully re-created every time it is used again. After upgrading to a newer NVSHMEM version, this repeated creation and destruction introduces a noticeable performance regression, likely due to the increased overhead in buffer initialization and NVSHMEM internal setup. We would like to know: - Is there a recommended way to explicitly release or recycle deep_ep.Buffer memory without destroying the entire buffer object? - Are there any best practices when using deep_ep.Buffer with newer NVSHMEM versions to mitigate this performance drop? Any guidance or suggestions would be greatly appreciated.

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

deepseek-ai/DeepEP#352

No description provided.

Rows
Columns