[PR #398] Make the JIT cache key independent of the install include path #417

New issue

Open

opened 2026-10-26 21:17:32 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 21:17:32 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/deepseek-ai/DeepGEMM/pull/398
Author: @matteso1
Created: 8/5/2026
Status: 🔄 Open

Base: main ← Head: jit-cache-key-relocatable

📝 Commits (3)

865b0ce Make the JIT cache key independent of the install include path
06a6ae2 Include bundled headers in the JIT cache key
4f85910 Fix header hashing through directory symlinks

📊 Changes

5 files changed (+142 additions, -2 deletions)

View changed files

📝 csrc/jit/compiler.hpp (+24 -1)
📝 csrc/jit/include_parser.hpp (+26 -0)
📝 csrc/utils/system.hpp (+1 -1)
➕ tests/test_include_parser.cpp (+58 -0)
➕ tests/test_include_parser.py (+33 -0)

📄 Description

This resubmits #388 so the repository's current automated review runs.

The bug

The kernel cache key embeds the absolute -I{library_include_path} token through flags. Byte-identical kernels therefore re-JIT and create duplicate cache entries when the install prefix changes, such as a renamed virtual environment or a container build path that differs from the runtime path. That prevents a shared DG_JIT_CACHE_DIR from being portable across those layouts.

This follows simon-mo's request on vllm-project/vllm#48190 to propose the relocation fix upstream rather than carry it as a vendored patch. Related but distinct work includes #301/#302 for JIT cache concurrency and #333 for wheel portability.

The fix

Strip only the exact -I{library_include_path} token when computing the cache key. The actual compile command and DG_JIT_DEBUG output are unchanged.

To keep reuse safe across different installed bundles, the full installed include tree is hashed once per process by relative path and file content. That tree hash covers the packaged DeepGEMM, CUTE, and CUTLASS headers without depending on the install prefix. The compiler-version signature, every behavioral flag, and every other include path also remain keyed. Existing entries re-JIT once under the new key.

Measured

The broader vendored variant in vllm-project/vllm#48190 also drops the compiler-version field. On one H100 NVL with Qwen3-30B-A3B-FP8, TP1, and DeepGEMM a6b593d, relocating a planted cache caused the stock key to recompile 41/41 kernels and start in 170.6 seconds. The tested portable key reused 41/41 and started in 76.9 seconds, a 93.7-second difference, n=1 per arm.

This PR is intentionally narrower because it retains compiler-version identity. Its exact diff has not been independently timed on GPU hardware.

Test plan

git diff --check
A local CPU regression verified that identical installed header trees produce the same hash across two roots, while changing a bundled CUTE header changes the hash.
Source review against current DeepGEMM main at 559d79fb6994a58b8a15b4b93bf13ccc16edf247

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/deepseek-ai/DeepGEMM/pull/398 **Author:** [@matteso1](https://github.com/matteso1) **Created:** 8/5/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `jit-cache-key-relocatable` --- ### 📝 Commits (3) - [`865b0ce`](https://github.com/deepseek-ai/DeepGEMM/commit/865b0ce23d689801cd2f48ef9c5316c6c7e9fa87) Make the JIT cache key independent of the install include path - [`06a6ae2`](https://github.com/deepseek-ai/DeepGEMM/commit/06a6ae2e81f66097ebab27bf2ee9d6bc902b3bc0) Include bundled headers in the JIT cache key - [`4f85910`](https://github.com/deepseek-ai/DeepGEMM/commit/4f85910306ce4ccf15cb0818061e9ed58bc5bb34) Fix header hashing through directory symlinks ### 📊 Changes **5 files changed** (+142 additions, -2 deletions) <details> <summary>View changed files</summary> 📝 `csrc/jit/compiler.hpp` (+24 -1) 📝 `csrc/jit/include_parser.hpp` (+26 -0) 📝 `csrc/utils/system.hpp` (+1 -1) ➕ `tests/test_include_parser.cpp` (+58 -0) ➕ `tests/test_include_parser.py` (+33 -0) </details> ### 📄 Description This resubmits #388 so the repository's current automated review runs. ## The bug The kernel cache key embeds the absolute `-I{library_include_path}` token through `flags`. Byte-identical kernels therefore re-JIT and create duplicate cache entries when the install prefix changes, such as a renamed virtual environment or a container build path that differs from the runtime path. That prevents a shared `DG_JIT_CACHE_DIR` from being portable across those layouts. This follows simon-mo's request on vllm-project/vllm#48190 to propose the relocation fix upstream rather than carry it as a vendored patch. Related but distinct work includes #301/#302 for JIT cache concurrency and #333 for wheel portability. ## The fix Strip only the exact `-I{library_include_path}` token when computing the cache key. The actual compile command and `DG_JIT_DEBUG` output are unchanged. To keep reuse safe across different installed bundles, the full installed include tree is hashed once per process by relative path and file content. That tree hash covers the packaged DeepGEMM, CUTE, and CUTLASS headers without depending on the install prefix. The compiler-version `signature`, every behavioral flag, and every other include path also remain keyed. Existing entries re-JIT once under the new key. ## Measured The broader vendored variant in vllm-project/vllm#48190 also drops the compiler-version field. On one H100 NVL with Qwen3-30B-A3B-FP8, TP1, and DeepGEMM `a6b593d`, relocating a planted cache caused the stock key to recompile 41/41 kernels and start in 170.6 seconds. The tested portable key reused 41/41 and started in 76.9 seconds, a 93.7-second difference, n=1 per arm. This PR is intentionally narrower because it retains compiler-version identity. Its exact diff has not been independently timed on GPU hardware. ## Test plan - `git diff --check` - A local CPU regression verified that identical installed header trees produce the same hash across two roots, while changing a bundled CUTE header changes the hash. - Source review against current DeepGEMM `main` at `559d79fb6994a58b8a15b4b93bf13ccc16edf247` --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi added the

pull-request

label

2026-10-26 21:17:32 +00:00

bsakaguchi referenced this issue

2026-10-26 21:17:42 +00:00

[PR #419] [SM12x] Restore pure-fp8 1d1d path (removed in 8b1392b); fix kernel TU preamble #431