kvcache-ai/kvcache-blog

Fork 0

mirror of https://github.com/kvcache-ai/kvcache-blog.git synced 2026-11-05 01:07:35 +00:00

[PR #14] [MERGED] Add KV Cache Workload Analyzer #15

New issue

Closed

opened 2026-10-29 14:31:57 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 14:31:57 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/kvcache-blog/pull/14
Author: @yokinoshitayoki
Created: 6/10/2026
Status: ✅ Merged
Merged: 6/12/2026
Merged by: @ykwd

Base: main ← Head: main

📝 Commits (10+)

dd641ec Add KV Cache Lab tool
7c8a176 Enhance KV Cache Lab trace simulation
5801a08 Address KV Cache Lab review feedback
43cdc4b Compress KV Cache Lab precomputed data
34b0273 Validate uploaded traces before simulation
b5d5f86 Rename KV cache workload tool
aa5d746 Validate uploaded trace schema and JS hash ids
a762eee Merge branch 'main' of github.com:yokinoshitayoki/kvcache-blog
6068096 Fix upload calculation max event constant
d04381b Clarify upload progress phases

📊 Changes

36 files changed (+226909 additions, -6 deletions)

View changed files

➕ .gitattributes (+2 -0)
📝 .gitignore (+8 -0)
📝 README.md (+1 -1)
➕ assets/js/kv-cache-lab-worker.js (+443 -0)
➕ assets/js/kv-cache-lab.js (+4329 -0)
➕ assets/wasm/kvcache-sim.wasm (+0 -0)
📝 config/_default/hugo.yaml (+1 -1)
📝 config/_default/params.yaml (+1 -1)
📝 content/_index.md (+1 -1)
➕ content/tools/kv-cache-analyzer/index.md (+13 -0)
📝 content/tools/kv-cache-size-calculator/index.md (+0 -0)
➕ data/kv_cache_lab/precomputed.json (+214630 -0)
➕ data/kv_cache_lab/presets.yaml (+92 -0)
📝 layouts/partials/blox/hero.html (+2 -2)
➕ layouts/shortcodes/kv-cache-lab.html (+766 -0)
➕ scripts/build-wasm.sh (+17 -0)
➕ scripts/kv-cache-lab-extend-precomputed.mjs (+114 -0)
➕ scripts/kv-cache-lab-fetch-traces.mjs (+6 -0)
➕ scripts/kv-cache-lab-full-precompute.mjs (+1458 -0)
➕ scripts/kv-cache-lab-merge-precomputed.mjs (+78 -0)

...and 16 more files

📄 Description

Summary

This PR adds KV Cache Workload Analyzer, a new interactive tool for estimating KV cache behavior across models, traces, memory budgets, and eviction policies.

KV Cache Lab helps users answer: given a model, a request trace, and a KV cache memory budget, what steady-state prefill token hit rate can be expected under FIFO, LRU, and theoretical Optimal eviction?

What Changed

Added a new Hugo tool page at /tools/kv-cache-lab/.
Added model-aware KV cache capacity accounting using the existing KV cache calculator model definitions.
Added real trace based simulation presets, including Mooncake FAST25 Tool&Agent, Qwen Bailian, RAGPulse, Weka Claude Code, and other agent/coding traces.
Added offline precomputed curves for public traces, so the browser does not replay large traces on page load.
Added three charts:
- Prefill token hit rate
- Theoretical prefill throughput
- Useful cache occupancy
Added support for FIFO, LRU, and Belady/MIN Optimal eviction.
Added model-specific controls such as KV precision, indexer precision, and draft KV cache where applicable.
Added clear UI explanations for trace semantics, chart calculations, eviction behavior, and theoretical limitations.

Customize Trace

This PR also adds Customize trace support, proposed by fatSheep (@00fish0).

Users can upload their own .jsonl or .jsonl.gz trace files directly in the browser. Files are processed locally and are not uploaded to any server. Uploaded traces are simulated with the same FIFO/LRU/Optimal policies and shown in the same charts as public traces.

Each JSONL record declares block identity through hash_ids, input token length through input_length, and native trace granularity through block_size.

Notes

KV Cache Lab is a theoretical simulator. It models block-level cache admission/eviction and token-weighted prefill hit rate, but it does not model all production system effects such as batching, scheduling, network transfer, KV lookup overhead, memory bandwidth, high-watermark batch eviction, or engine-specific implementation details.

Validation

node --test tests/kv-cache-calculator.test.mjs tests/kv-cache-lab.test.mjs tests/kv-cache-lab-pool.test.mjs tests/kv-cache-lab-wasm.test.mjs tests/kv-cache-lab-traces.test.mjs
/private/tmp/hugo-0.126.3/hugo --minify
Browser smoke test on /tools/kv-cache-lab/

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/kvcache-blog/pull/14 **Author:** [@yokinoshitayoki](https://github.com/yokinoshitayoki) **Created:** 6/10/2026 **Status:** ✅ Merged **Merged:** 6/12/2026 **Merged by:** [@ykwd](https://github.com/ykwd) **Base:** `main` ← **Head:** `main` --- ### 📝 Commits (10+) - [`dd641ec`](https://github.com/kvcache-ai/kvcache-blog/commit/dd641ecbf9e58402357e19f1a5b757c5bb531435) Add KV Cache Lab tool - [`7c8a176`](https://github.com/kvcache-ai/kvcache-blog/commit/7c8a176f1296329d2476a8151e9ce4ba1e40e1b8) Enhance KV Cache Lab trace simulation - [`5801a08`](https://github.com/kvcache-ai/kvcache-blog/commit/5801a08af031ae2954bbe1b786bb8a98b09c91ea) Address KV Cache Lab review feedback - [`43cdc4b`](https://github.com/kvcache-ai/kvcache-blog/commit/43cdc4b54994b8075512eb1cc4d97b45839a6ab1) Compress KV Cache Lab precomputed data - [`34b0273`](https://github.com/kvcache-ai/kvcache-blog/commit/34b0273d44e160e444aaa6d5383ca9588940da24) Validate uploaded traces before simulation - [`b5d5f86`](https://github.com/kvcache-ai/kvcache-blog/commit/b5d5f865e865513a04e7149bf24af3538165ce0b) Rename KV cache workload tool - [`aa5d746`](https://github.com/kvcache-ai/kvcache-blog/commit/aa5d746b25de00796b18c594a090b242826695d4) Validate uploaded trace schema and JS hash ids - [`a762eee`](https://github.com/kvcache-ai/kvcache-blog/commit/a762eeeec35754879f2d7fda6f6fcc19e8aed62c) Merge branch 'main' of github.com:yokinoshitayoki/kvcache-blog - [`6068096`](https://github.com/kvcache-ai/kvcache-blog/commit/6068096b3ec8e72602a6b628f8a9fd482c1a3f8c) Fix upload calculation max event constant - [`d04381b`](https://github.com/kvcache-ai/kvcache-blog/commit/d04381b015e2a64ad04b19d3ec50dafa9a70da0f) Clarify upload progress phases ### 📊 Changes **36 files changed** (+226909 additions, -6 deletions) <details> <summary>View changed files</summary> ➕ `.gitattributes` (+2 -0) 📝 `.gitignore` (+8 -0) 📝 `README.md` (+1 -1) ➕ `assets/js/kv-cache-lab-worker.js` (+443 -0) ➕ `assets/js/kv-cache-lab.js` (+4329 -0) ➕ `assets/wasm/kvcache-sim.wasm` (+0 -0) 📝 `config/_default/hugo.yaml` (+1 -1) 📝 `config/_default/params.yaml` (+1 -1) 📝 `content/_index.md` (+1 -1) ➕ `content/tools/kv-cache-analyzer/index.md` (+13 -0) 📝 `content/tools/kv-cache-size-calculator/index.md` (+0 -0) ➕ `data/kv_cache_lab/precomputed.json` (+214630 -0) ➕ `data/kv_cache_lab/presets.yaml` (+92 -0) 📝 `layouts/partials/blox/hero.html` (+2 -2) ➕ `layouts/shortcodes/kv-cache-lab.html` (+766 -0) ➕ `scripts/build-wasm.sh` (+17 -0) ➕ `scripts/kv-cache-lab-extend-precomputed.mjs` (+114 -0) ➕ `scripts/kv-cache-lab-fetch-traces.mjs` (+6 -0) ➕ `scripts/kv-cache-lab-full-precompute.mjs` (+1458 -0) ➕ `scripts/kv-cache-lab-merge-precomputed.mjs` (+78 -0) _...and 16 more files_ </details> ### 📄 Description ## Summary This PR adds **KV Cache Workload Analyzer**, a new interactive tool for estimating KV cache behavior across models, traces, memory budgets, and eviction policies. KV Cache Lab helps users answer: given a model, a request trace, and a KV cache memory budget, what steady-state prefill token hit rate can be expected under FIFO, LRU, and theoretical Optimal eviction? ## What Changed - Added a new Hugo tool page at `/tools/kv-cache-lab/`. - Added model-aware KV cache capacity accounting using the existing KV cache calculator model definitions. - Added real trace based simulation presets, including Mooncake FAST25 Tool&Agent, Qwen Bailian, RAGPulse, Weka Claude Code, and other agent/coding traces. - Added offline precomputed curves for public traces, so the browser does not replay large traces on page load. - Added three charts: - **Prefill token hit rate** - **Theoretical prefill throughput** - **Useful cache occupancy** - Added support for FIFO, LRU, and Belady/MIN Optimal eviction. - Added model-specific controls such as KV precision, indexer precision, and draft KV cache where applicable. - Added clear UI explanations for trace semantics, chart calculations, eviction behavior, and theoretical limitations. ## Customize Trace This PR also adds **Customize trace** support, proposed by fatSheep (@00fish0). Users can upload their own `.jsonl` or `.jsonl.gz` trace files directly in the browser. Files are processed locally and are not uploaded to any server. Uploaded traces are simulated with the same FIFO/LRU/Optimal policies and shown in the same charts as public traces. Each JSONL record declares block identity through `hash_ids`, input token length through `input_length`, and native trace granularity through `block_size`. ## Notes KV Cache Lab is a theoretical simulator. It models block-level cache admission/eviction and token-weighted prefill hit rate, but it does not model all production system effects such as batching, scheduling, network transfer, KV lookup overhead, memory bandwidth, high-watermark batch eviction, or engine-specific implementation details. ## Validation - `node --test tests/kv-cache-calculator.test.mjs tests/kv-cache-lab.test.mjs tests/kv-cache-lab-pool.test.mjs tests/kv-cache-lab-wasm.test.mjs tests/kv-cache-lab-traces.test.mjs` - `/private/tmp/hugo-0.126.3/hugo --minify` - Browser smoke test on `/tools/kv-cache-lab/` --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-29 14:31:57 +00:00

closed this issue
added the
pull-request
label

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

kvcache-ai/kvcache-blog#15

No description provided.

Rows
Columns