[PR #1218] [MERGED] [TE/HIP] Fix HIP Shareable POSIX File Descriptor Handles

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

kvcache-ai/Mooncake#2082

bsakaguchi commented

2026-10-29 21:32:39 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/Mooncake/pull/1218
Author: @amd-arozanov
Created: 12/16/2025
Status: ✅ Merged
Merged: 12/17/2025
Merged by: @alogfans

Base: main ← Head: posix_fd4

📝 Commits (6)

ef557fc [TE/HIP] Fix HIP shareable handles implementation
bde45fe [TE/HIP] Set proper device context for each transfer
515b2fd [TE/HIP] Extract common logic into processTransferRequest
4eb7327 [TE/HIP] Add stream synchronization
8c9208b [TE/HIP] Add RAII wrapper for file descriptor management
25d983e [TE/HIP] Simplify device context setting for host memory

📊 Changes

2 files changed (+188 additions, -77 deletions)

View changed files

📝 mooncake-transfer-engine/include/transport/hip_transport/hip_transport.h (+4 -1)
📝 mooncake-transfer-engine/src/transport/hip_transport/hip_transport.cpp (+184 -76)

📄 Description

Description

(AI Assist)

This PR is focusing on fixes for proper POSIX file descriptor-based shareable handle implementation.

Changes:

Implement proper shareable handle support for HIP

Added open_fd() function that uses pidfd_open and pidfd_getfd syscalls to correctly open POSIX file descriptors from hipxFabricHandle in remote processes

Add proper device context management

Extracted device ID from request.source to set correct device context for each transfer operation
Ensures memory operations execute on the appropriate GPU device

Add stream synchronization for memory transfers

Added hipStreamSynchronize() call after hipMemcpy to ensure memory transfer is fully complete before marking operation as successful
Without synchronization, hipMemcpy may return immediately while the actual data transfer is still in progress

Refactor transfer request processing

Eliminated code duplication between submitTransfer() and submitTransferTask() by extracting shared logic into a helper method: processTransferRequest().

Type of Change

Types
- Bug fix
- New feature
  - Transfer Engine
  - Mooncake Store
  - Mooncake EP
  - Integration
  - P2P Store
  - Python Wheel
- Breaking change
- CI/CD
- Documentation update
- Other

How Has This Been Tested?

Ran transfer_engine_bench and transfer_engine_validator on nodes with AMG GPU

Checklist

I have performed a self-review of my own code.
I have updated the documentation.
I have added tests to prove my changes are effective.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/Mooncake/pull/1218 **Author:** [@amd-arozanov](https://github.com/amd-arozanov) **Created:** 12/16/2025 **Status:** ✅ Merged **Merged:** 12/17/2025 **Merged by:** [@alogfans](https://github.com/alogfans) **Base:** `main` ← **Head:** `posix_fd4` --- ### 📝 Commits (6) - [`ef557fc`](https://github.com/kvcache-ai/Mooncake/commit/ef557fc4081d228e24aeef50dcef9a25407f98f5) [TE/HIP] Fix HIP shareable handles implementation - [`bde45fe`](https://github.com/kvcache-ai/Mooncake/commit/bde45fe411c1fcb6d658e350b8671478f246fe8a) [TE/HIP] Set proper device context for each transfer - [`515b2fd`](https://github.com/kvcache-ai/Mooncake/commit/515b2fd7b4c84215ae7add60aa4b8e0d41e0cfb6) [TE/HIP] Extract common logic into processTransferRequest - [`4eb7327`](https://github.com/kvcache-ai/Mooncake/commit/4eb73276299c6b9921e6b9b2bb1921a5f90b5109) [TE/HIP] Add stream synchronization - [`8c9208b`](https://github.com/kvcache-ai/Mooncake/commit/8c9208b30eb8a4231efc4b138cf9f192d00ae4c5) [TE/HIP] Add RAII wrapper for file descriptor management - [`25d983e`](https://github.com/kvcache-ai/Mooncake/commit/25d983e5468020277c3667f00715b24038350e75) [TE/HIP] Simplify device context setting for host memory ### 📊 Changes **2 files changed** (+188 additions, -77 deletions) <details> <summary>View changed files</summary> 📝 `mooncake-transfer-engine/include/transport/hip_transport/hip_transport.h` (+4 -1) 📝 `mooncake-transfer-engine/src/transport/hip_transport/hip_transport.cpp` (+184 -76) </details> ### 📄 Description ## Description (AI Assist) This PR is focusing on fixes for proper POSIX file descriptor-based shareable handle implementation. ### Changes: #### Implement proper shareable handle support for HIP - Added `open_fd()` function that uses `pidfd_open` and `pidfd_getfd` syscalls to correctly open POSIX file descriptors from `hipxFabricHandle` in remote processes #### Add proper device context management - Extracted device ID from `request.source` to set correct device context for each transfer operation - Ensures memory operations execute on the appropriate GPU device #### Add stream synchronization for memory transfers - Added `hipStreamSynchronize()` call after `hipMemcpy` to ensure memory transfer is fully complete before marking operation as successful - Without synchronization, `hipMemcpy` may return immediately while the actual data transfer is still in progress #### Refactor transfer request processing - Eliminated code duplication between `submitTransfer()` and `submitTransferTask()` by extracting shared logic into a helper method: `processTransferRequest()`. ## Type of Change * Types - [x] Bug fix - [ ] New feature - [x] Transfer Engine - [ ] Mooncake Store - [ ] Mooncake EP - [ ] Integration - [ ] P2P Store - [ ] Python Wheel - [ ] Breaking change - [ ] CI/CD - [ ] Documentation update - [ ] Other ## How Has This Been Tested? Ran transfer_engine_bench and transfer_engine_validator on nodes with AMG GPU ## Checklist - [x] I have performed a self-review of my own code. - [x] I have updated the documentation. - [x] I have added tests to prove my changes are effective. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

closed this issue
added the
pull-request
label

Rows
Columns

[PR #1218] [MERGED] [TE/HIP] Fix HIP Shareable POSIX File Descriptor Handles #2082