[GH-ISSUE #503] When running test_internode.py on two nodes (with 8 GPUs per node), an error occurs. #324

New issue

Open

opened 2026-10-26 09:58:35 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 09:58:35 +00:00

Owner

Originally created by @SaltFish11 on GitHub (Nov 24, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/503

When running test_internode.py on two nodes (with 8 GPUs per node), an error occurs.
Execution Command:：

MASTER_ADDR=${MASTER_ADDR:-"10.124.173.185"}
MASTER_PORT=${MASTER_PORT:-10556}
WORLD_SIZE=${WORLD_SIZE:-2}
RANK=${RANK:-${ARNOLD_ID:-0}}

MASTER_ADDR="$MASTER_ADDR" MASTER_PORT=$MASTER_PORT WORLD_SIZE=$WORLD_SIZE RANK=$RANK
python ./tests/test_internode.py 2>&1 | tee debug_${RANK}.log

Error Logs:
rank0:

https://github.com/SaltFish11/DeepEP/blob/develop/debug_0.log

rank1:

https://github.com/SaltFish11/DeepEP/blob/develop/debug_1.log

Originally created by @SaltFish11 on GitHub (Nov 24, 2025). Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/503 When running test_internode.py on two nodes (with 8 GPUs per node), an error occurs. Execution Command:： > MASTER_ADDR=${MASTER_ADDR:-"10.124.173.185"} > MASTER_PORT=${MASTER_PORT:-10556} > WORLD_SIZE=${WORLD_SIZE:-2} > RANK=${RANK:-${ARNOLD_ID:-0}} > > MASTER_ADDR="$MASTER_ADDR" MASTER_PORT=$MASTER_PORT WORLD_SIZE=$WORLD_SIZE RANK=$RANK \ > python ./tests/test_internode.py 2>&1 | tee debug_${RANK}.log Error Logs: rank0: https://github.com/SaltFish11/DeepEP/blob/develop/debug_0.log rank1: https://github.com/SaltFish11/DeepEP/blob/develop/debug_1.log

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

deepseek-ai/DeepEP#324

No description provided.

Rows
Columns