[PR #388] [CLOSED] Make the JIT cache key independent of the install include path #409

New issue

Closed

opened 2026-10-26 21:17:23 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 21:17:23 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/deepseek-ai/DeepGEMM/pull/388
Author: @matteso1
Created: 7/20/2026
Status: ❌ Closed

Base: main ← Head: jit-cache-key-relocatable

📝 Commits (1)

865b0ce Make the JIT cache key independent of the install include path

📊 Changes

1 file changed (+24 additions, -1 deletions)

View changed files

📝 csrc/jit/compiler.hpp (+24 -1)

📄 Description

The kernel cache key embeds the absolute -I{library_include_path} token via flags, so a byte-identical kernel re-JITs and duplicates cache entries whenever the install prefix differs (venv rename, container build-layer vs runtime layout, moved NFS mount). That defeats sharing a DG_JIT_CACHE_DIR across layouts.

This strips that one token from the key (strip_flag_token, exact match, key computation only; the compile command and DG_JIT_DEBUG output are unchanged). It stays safe because that directory's header content is already keyed through the include hash embedded in code (IncludeParser::get_hash_value); everything else stays keyed: the compiler-version signature, every behavioral flag, and every other -I (NVRTC's cuda_home/include headers are not content-hashed, so their path stays). Since the key changes, existing entries re-JIT once under it.

This PR follows simon-mo's request on the open vllm-project/vllm#48190 to propose the relocation fix upstream instead of carrying it as a vendored patch. The vendored variant proposed there also drops the version field; on H100, relocating a prebuilt cache across install layouts turned a guaranteed miss into a hit and cut 93.7s from a Qwen3-30B-A3B-FP8 cold start (n=1). This PR deliberately keeps the version field keyed.

Related but distinct: #301/#302 (JIT cache concurrency), #333 (wheel portability).

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/deepseek-ai/DeepGEMM/pull/388 **Author:** [@matteso1](https://github.com/matteso1) **Created:** 7/20/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `jit-cache-key-relocatable` --- ### 📝 Commits (1) - [`865b0ce`](https://github.com/deepseek-ai/DeepGEMM/commit/865b0ce23d689801cd2f48ef9c5316c6c7e9fa87) Make the JIT cache key independent of the install include path ### 📊 Changes **1 file changed** (+24 additions, -1 deletions) <details> <summary>View changed files</summary> 📝 `csrc/jit/compiler.hpp` (+24 -1) </details> ### 📄 Description The kernel cache key embeds the absolute `-I{library_include_path}` token via `flags`, so a byte-identical kernel re-JITs and duplicates cache entries whenever the install prefix differs (venv rename, container build-layer vs runtime layout, moved NFS mount). That defeats sharing a `DG_JIT_CACHE_DIR` across layouts. This strips that one token from the key (`strip_flag_token`, exact match, key computation only; the compile command and `DG_JIT_DEBUG` output are unchanged). It stays safe because that directory's header content is already keyed through the include hash embedded in `code` (`IncludeParser::get_hash_value`); everything else stays keyed: the compiler-version `signature`, every behavioral flag, and every other `-I` (NVRTC's `cuda_home/include` headers are not content-hashed, so their path stays). Since the key changes, existing entries re-JIT once under it. This PR follows simon-mo's request on the open vllm-project/vllm#48190 to propose the relocation fix upstream instead of carrying it as a vendored patch. The vendored variant proposed there also drops the version field; on H100, relocating a prebuilt cache across install layouts turned a guaranteed miss into a hit and cut 93.7s from a Qwen3-30B-A3B-FP8 cold start (n=1). This PR deliberately keeps the version field keyed. Related but distinct: #301/#302 (JIT cache concurrency), #333 (wheel portability). --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-26 21:17:23 +00:00