deepseek-ai/DeepEP

Fork 0

mirror of https://github.com/deepseek-ai/DeepEP.git synced 2026-11-05 00:47:32 +00:00

[PR #539] Add delay before trap() to allow printf buffer flush #622

New issue

Open

opened 2026-10-27 21:04:37 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-27 21:04:37 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/deepseek-ai/DeepEP/pull/539
Author: @yurekami
Created: 12/25/2025
Status: 🔄 Open

Base: main ← Head: fix-trap-log-flush

📝 Commits (1)

90ebb9e Add delay before trap() to allow printf buffer flush

📊 Changes

2 files changed (+8 additions, -0 deletions)

View changed files

📝 csrc/kernels/configs.cuh (+1 -0)
📝 csrc/kernels/utils.cuh (+7 -0)

📄 Description

Summary

Adds a configurable delay before trap() to allow printf buffers to flush
Prevents loss of critical error logs when one warp times out before others

Problem

When trap() is called by one warp during timeout, the kernel execution aborts immediately. Other warps may still have unflushed printf buffers, causing their diagnostic messages (from Forwarder, Sender, Coordinator roles) to be lost.

Solution

Added NUM_TRAP_FLUSH_ITERATIONS constant (1000 iterations)
Modified trap() to delay ~500μs (1000 × 500ns) before executing the trap instruction
Uses #pragma unroll 1 to prevent compiler optimization from removing the delay loop

Changes

csrc/kernels/configs.cuh - Added NUM_TRAP_FLUSH_ITERATIONS constant
csrc/kernels/utils.cuh - Modified trap() function with delay loop

Test plan

Trigger a timeout condition and verify all warp roles output their diagnostic messages
Verify the delay doesn't significantly impact normal operation (only affects error paths)

Fixes #480

🤖 Generated with Claude Code

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/deepseek-ai/DeepEP/pull/539 **Author:** [@yurekami](https://github.com/yurekami) **Created:** 12/25/2025 **Status:** 🔄 Open **Base:** `main` ← **Head:** `fix-trap-log-flush` --- ### 📝 Commits (1) - [`90ebb9e`](https://github.com/deepseek-ai/DeepEP/commit/90ebb9ecc723685fc4c1c75e264b0cd74a16787d) Add delay before trap() to allow printf buffer flush ### 📊 Changes **2 files changed** (+8 additions, -0 deletions) <details> <summary>View changed files</summary> 📝 `csrc/kernels/configs.cuh` (+1 -0) 📝 `csrc/kernels/utils.cuh` (+7 -0) </details> ### 📄 Description ## Summary - Adds a configurable delay before `trap()` to allow printf buffers to flush - Prevents loss of critical error logs when one warp times out before others ## Problem When `trap()` is called by one warp during timeout, the kernel execution aborts immediately. Other warps may still have unflushed printf buffers, causing their diagnostic messages (from Forwarder, Sender, Coordinator roles) to be lost. ## Solution - Added `NUM_TRAP_FLUSH_ITERATIONS` constant (1000 iterations) - Modified `trap()` to delay ~500μs (1000 × 500ns) before executing the trap instruction - Uses `#pragma unroll 1` to prevent compiler optimization from removing the delay loop ## Changes - `csrc/kernels/configs.cuh` - Added `NUM_TRAP_FLUSH_ITERATIONS` constant - `csrc/kernels/utils.cuh` - Modified `trap()` function with delay loop ## Test plan - [ ] Trigger a timeout condition and verify all warp roles output their diagnostic messages - [ ] Verify the delay doesn't significantly impact normal operation (only affects error paths) Fixes #480 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi added the

pull-request

label

2026-10-27 21:04:37 +00:00

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

deepseek-ai/DeepEP#622

No description provided.

Rows
Columns