[GH-ISSUE #68] Why the log in buffer.py can't print?? #61

New issue

Closed

opened 2026-10-26 09:53:34 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 09:53:34 +00:00

Owner

Originally created by @Huixxi on GitHub (Mar 11, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/68

class Buffer:
"""
The core expert-parallel (EP) communication buffers for Mixture of Experts (MoE) model, which supports:
- high-throughput intranode all-to-all (dispatch and combine, using NVLink)
- high-throughput internode all-to-all (dispatch and combine, using RDMA without AR)
- low-latency all-to-all (dispatch and combine, using RDMA, AR supported)

Attributes:
    num_sms: the SMs used in high-throughput kernels.
    rank: the local rank number.
    group_size: the number of ranks in the group.
    group: the communication group.
    num_nvl_bytes: the buffer size for intranode NVLink communication.
    num_rdma_bytes: the buffer size for internode (also for intranode with low-latency mode) RDMA communication.
    runtime: the C++ runtime.
"""

num_sms: int = 20

def __init__(self, group: dist.ProcessGroup,
             num_nvl_bytes: int = 0, num_rdma_bytes: int = 0,
             low_latency_mode: bool = False, num_qps_per_rank: int = 1) -> None:
    """
    Initialize the communication buffer.

    Arguments:
        group: the communication group.
        num_nvl_bytes: the buffer size for intranode NVLink communication.
        num_rdma_bytes: the buffer size for internode (also for intranode with low-latency mode) RDMA communication.
        low_latency_mode: whether to enable low-latency mode.
        num_qps_per_rank: the number of QPs for RDMA, the low-latency mode requires that this number equals
            to the number of local experts.
    """

    # Initialize the CPP runtime
    print("=============================================", flush=True)
    self.rank = group.rank()
    self.group_size = group.size()
    self.group = group

Originally created by @Huixxi on GitHub (Mar 11, 2025). Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/68 class Buffer: """ The core expert-parallel (EP) communication buffers for Mixture of Experts (MoE) model, which supports: - high-throughput intranode all-to-all (dispatch and combine, using NVLink) - high-throughput internode all-to-all (dispatch and combine, using RDMA without AR) - low-latency all-to-all (dispatch and combine, using RDMA, AR supported) Attributes: num_sms: the SMs used in high-throughput kernels. rank: the local rank number. group_size: the number of ranks in the group. group: the communication group. num_nvl_bytes: the buffer size for intranode NVLink communication. num_rdma_bytes: the buffer size for internode (also for intranode with low-latency mode) RDMA communication. runtime: the C++ runtime. """ num_sms: int = 20 def __init__(self, group: dist.ProcessGroup, num_nvl_bytes: int = 0, num_rdma_bytes: int = 0, low_latency_mode: bool = False, num_qps_per_rank: int = 1) -> None: """ Initialize the communication buffer. Arguments: group: the communication group. num_nvl_bytes: the buffer size for intranode NVLink communication. num_rdma_bytes: the buffer size for internode (also for intranode with low-latency mode) RDMA communication. low_latency_mode: whether to enable low-latency mode. num_qps_per_rank: the number of QPs for RDMA, the low-latency mode requires that this number equals to the number of local experts. """ # Initialize the CPP runtime print("=============================================", flush=True) self.rank = group.rank() self.group_size = group.size() self.group = group

bsakaguchi closed this issue

2026-10-26 09:53:34 +00:00

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

deepseek-ai/DeepEP#61

No description provided.

Rows
Columns