[PR #2] [CLOSED] WIP Add Robust Multi-GPU and Cross-Device Support for PyIsolate Benchmarks #3

New issue

Closed

opened 2026-10-26 22:11:28 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 22:11:28 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/pyisolate/pull/2
Author: @billybasass
Created: 7/14/2025
Status: ❌ Closed

Base: main ← Head: main

📝 Commits (10+)

6ed3376 Add multi-GPU, Intel/AMD support, robust device selection, and linter fixes
a9ebed1 main change is device type was a integer and changed it to a string some you can still pick cpu only testing
ea59404 fix redundant torch requirements and correct wheels
60afae5 set python to 3.12 because 3.13 does not have all wheels and make the array writable before converting to a tensor
a01c1f4 ruff and lint checks
14c2605 fix: venv isolation for extension imports, ensure only venv site-packages are used; codebase ruff clean
7de4668 intel testing added
ca3ced1 venv isolation has been applied
42728ed added GPU share results for intel GPUs
1b55f5b The memory benchmark will now loop over all available backends (including xpu for Intel GPUs)

📊 Changes

12 files changed (+996 additions, -533 deletions)

View changed files

📝 benchmarks/benchmark.py (+219 -36)
📝 benchmarks/memory_benchmark.py (+460 -322)
➕ cleanup_pyisolate.ps1 (+32 -0)
📝 pyisolate/_internal/client.py (+12 -3)
➕ pyisolate/_internal/gpu_utils.py (+67 -0)
📝 pyisolate/_internal/host.py (+39 -10)
📝 pyisolate/_internal/shared.py (+50 -33)
📝 pyisolate/shared.py (+6 -0)
📝 pyproject.toml (+0 -1)
📝 run_benchmarks_windows.ps1 (+50 -13)
📝 tests/test_benchmarks.py (+46 -114)
📝 tests/test_torch_tensor_integration.py (+15 -1)

📄 Description

This PR adds, cross-vendor GPU support to the PyIsolate benchmarking suite, enabling seamless benchmarking on NVIDIA (CUDA), AMD (ROCm), and Intel (XPU/oneAPI) GPUs, as well as CPU. It also improves device selection, serialization, and reporting for all supported platforms.
Key Features & Changes
Multi-GPU Device Selection:
Detects all available GPUs (NVIDIA, AMD, Intel) in the system.
Prompts the user to select which GPU to use if more than one is present.
Prints the name of the selected device at the start of each benchmark run.
Device-Aware Tensor Serialization:
CPU and CUDA: Uses PyTorch’s native (zero-copy) serialization for inter-process communication.
Intel XPU: Implements a fallback serialization for XPU tensors:
XPU tensors are moved to CPU, serialized as a buffer (with shape and dtype), sent through the multiprocessing queue, and reconstructed as XPU tensors on the receiving side.
This avoids PyCapsule/pickle errors and ensures all tests run on Intel GPUs, even though the transfer is not zero-copy.
All benchmarks (including shared-torch and RPC) now run on Intel GPUs.
Benchmark Script Enhancements:
Benchmarks are run on all available backends (CPU, CUDA, XPU).
Status messages and results include the backend and device name for clarity and reproducibility.
Extension Venv and Dependency Handling:
Extension venvs now install the correct PyTorch wheel and index for each backend.
Ensures correct torch version and index for Intel GPUs, and removes invalid version specifiers.
Linter and Syntax Fixes:
Fixed all linter and syntax errors, including stray else: statements and indentation issues.
Codebase is now linter-clean and CI-ready.
Motivation
Enable robust benchmarking and extension support across all major GPU vendors.
Ensure that all tests run on all supported hardware, with clear device selection and reporting.
Provide a future-proof serialization mechanism for device tensors, especially for Intel XPU, until native support is available in PyTorch.
Testing
Benchmarks were run on systems with NVIDIA, AMD, and Intel GPUs.
All tests pass, and results are correctly reported for each device.
No PyCapsule/pickle errors on Intel GPUs.
Closes Issues
Fixes device selection and reporting for multi-GPU systems.
Fixes XPU tensor serialization for multiprocessing.
Fixes linter and syntax errors in the codebase.
Ready for review!

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/pyisolate/pull/2 **Author:** [@billybasass](https://github.com/billybasass) **Created:** 7/14/2025 **Status:** ❌ Closed **Base:** `main` ← **Head:** `main` --- ### 📝 Commits (10+) - [`6ed3376`](https://github.com/Comfy-Org/pyisolate/commit/6ed3376668a02420dfcf66385a9a03980e50e207) Add multi-GPU, Intel/AMD support, robust device selection, and linter fixes - [`a9ebed1`](https://github.com/Comfy-Org/pyisolate/commit/a9ebed18d75f0193ec70b1f0b4eb244a87f7f300) main change is device type was a integer and changed it to a string some you can still pick cpu only testing - [`ea59404`](https://github.com/Comfy-Org/pyisolate/commit/ea59404c07a391eaa3872e9f3d022968673c0b4e) fix redundant torch requirements and correct wheels - [`60afae5`](https://github.com/Comfy-Org/pyisolate/commit/60afae5e196e74c9cd6d8ee7d2f0bc57bd246c81) set python to 3.12 because 3.13 does not have all wheels and make the array writable before converting to a tensor - [`a01c1f4`](https://github.com/Comfy-Org/pyisolate/commit/a01c1f49b574e8d9eb9ea80884005dec021482e2) ruff and lint checks - [`14c2605`](https://github.com/Comfy-Org/pyisolate/commit/14c26051bd8b2fe9e8ae04cb37cbc25085811c06) fix: venv isolation for extension imports, ensure only venv site-packages are used; codebase ruff clean - [`7de4668`](https://github.com/Comfy-Org/pyisolate/commit/7de46684e93921dc86711e6065be7434f7511e8c) intel testing added - [`ca3ced1`](https://github.com/Comfy-Org/pyisolate/commit/ca3ced13698ef7752a7c10427a31693315de6b86) venv isolation has been applied - [`42728ed`](https://github.com/Comfy-Org/pyisolate/commit/42728ed0aa10f9db667895f9812ecadf2a19d237) added GPU share results for intel GPUs - [`1b55f5b`](https://github.com/Comfy-Org/pyisolate/commit/1b55f5b64c45e540e9b5d586676722871b165363) The memory benchmark will now loop over all available backends (including xpu for Intel GPUs) ### 📊 Changes **12 files changed** (+996 additions, -533 deletions) <details> <summary>View changed files</summary> 📝 `benchmarks/benchmark.py` (+219 -36) 📝 `benchmarks/memory_benchmark.py` (+460 -322) ➕ `cleanup_pyisolate.ps1` (+32 -0) 📝 `pyisolate/_internal/client.py` (+12 -3) ➕ `pyisolate/_internal/gpu_utils.py` (+67 -0) 📝 `pyisolate/_internal/host.py` (+39 -10) 📝 `pyisolate/_internal/shared.py` (+50 -33) 📝 `pyisolate/shared.py` (+6 -0) 📝 `pyproject.toml` (+0 -1) 📝 `run_benchmarks_windows.ps1` (+50 -13) 📝 `tests/test_benchmarks.py` (+46 -114) 📝 `tests/test_torch_tensor_integration.py` (+15 -1) </details> ### 📄 Description This PR adds, cross-vendor GPU support to the PyIsolate benchmarking suite, enabling seamless benchmarking on NVIDIA (CUDA), AMD (ROCm), and Intel (XPU/oneAPI) GPUs, as well as CPU. It also improves device selection, serialization, and reporting for all supported platforms. Key Features & Changes Multi-GPU Device Selection: Detects all available GPUs (NVIDIA, AMD, Intel) in the system. Prompts the user to select which GPU to use if more than one is present. Prints the name of the selected device at the start of each benchmark run. Device-Aware Tensor Serialization: CPU and CUDA: Uses PyTorch’s native (zero-copy) serialization for inter-process communication. Intel XPU: Implements a fallback serialization for XPU tensors: XPU tensors are moved to CPU, serialized as a buffer (with shape and dtype), sent through the multiprocessing queue, and reconstructed as XPU tensors on the receiving side. This avoids PyCapsule/pickle errors and ensures all tests run on Intel GPUs, even though the transfer is not zero-copy. All benchmarks (including shared-torch and RPC) now run on Intel GPUs. Benchmark Script Enhancements: Benchmarks are run on all available backends (CPU, CUDA, XPU). Status messages and results include the backend and device name for clarity and reproducibility. Extension Venv and Dependency Handling: Extension venvs now install the correct PyTorch wheel and index for each backend. Ensures correct torch version and index for Intel GPUs, and removes invalid version specifiers. Linter and Syntax Fixes: Fixed all linter and syntax errors, including stray else: statements and indentation issues. Codebase is now linter-clean and CI-ready. Motivation Enable robust benchmarking and extension support across all major GPU vendors. Ensure that all tests run on all supported hardware, with clear device selection and reporting. Provide a future-proof serialization mechanism for device tensors, especially for Intel XPU, until native support is available in PyTorch. Testing Benchmarks were run on systems with NVIDIA, AMD, and Intel GPUs. All tests pass, and results are correctly reported for each device. No PyCapsule/pickle errors on Intel GPUs. Closes Issues Fixes device selection and reporting for multi-GPU systems. Fixes XPU tensor serialization for multiprocessing. Fixes linter and syntax errors in the codebase. Ready for review! --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-26 22:11:28 +00:00