kvcache-ai/sglang
1
0
Fork 0
mirror of https://github.com/kvcache-ai/sglang.git synced 2026-11-05 01:07:39 +00:00
Code Issues 26 Projects Packages Wiki Activity

[PR #51] feat(kse): port unified-sparse-kv KVCache Sparsity Engine from magicYang1573/sglang #51

New issue
Open
opened 2026-10-29 21:06:47 +00:00 by bsakaguchi · 0 comments
bsakaguchi commented 2026-10-29 21:06:47 +00:00
Owner
Copy link

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/sglang/pull/51
Author: @Copilot
Created: 5/26/2026
Status: 🔄 Open

Base: main ← Head: copilot/move-unified-sparse-kv

📝 Commits (1)

  • 6467cf5 feat(kse): port unified-sparse-kv KVCache Sparsity Engine from magicYang1573/sglang

📊 Changes

31 files changed (+5215 additions, -1 deletions)

View changed files

➕ docs/advanced_features/kv_cache_sparsity_engine-zh.md (+909 -0)
➕ docs/advanced_features/kv_cache_sparsity_engine.md (+867 -0)
📝 python/sglang/srt/layers/attention/base_attn_backend.py (+14 -1)
➕ python/sglang/srt/layers/kse/README.md (+313 -0)
➕ python/sglang/srt/layers/kse/__init__.py (+35 -0)
➕ python/sglang/srt/layers/kse/adapters/__init__.py (+8 -0)
➕ python/sglang/srt/layers/kse/adapters/flashattention_adapter.py (+130 -0)
➕ python/sglang/srt/layers/kse/adapters/flashinfer_adapter.py (+167 -0)
➕ python/sglang/srt/layers/kse/adapters/triton_adapter.py (+113 -0)
➕ python/sglang/srt/layers/kse/base_adapter.py (+50 -0)
➕ python/sglang/srt/layers/kse/base_policy.py (+93 -0)
➕ python/sglang/srt/layers/kse/config.py (+18 -0)
➕ python/sglang/srt/layers/kse/controller.py (+251 -0)
➕ python/sglang/srt/layers/kse/policies/__init__.py (+7 -0)
➕ python/sglang/srt/layers/kse/policies/quest.py (+258 -0)
➕ python/sglang/srt/layers/kse/policies/streaming_llm.py (+107 -0)
➕ python/sglang/srt/layers/kse/registry.py (+71 -0)
➕ python/sglang/srt/layers/kse/types.py (+48 -0)
📝 python/sglang/srt/model_executor/forward_batch_info.py (+5 -0)
📝 python/sglang/srt/model_executor/model_runner.py (+65 -0)

...and 11 more files

📄 Description

  • Explore source branch (magicYang1573/sglang unified-sparse-kv)
  • Identify 11 unique KSE commits and changed files
  • Copy all new KSE files (docs, python module, tests)
  • Apply modifications to existing files:
    • python/sglang/srt/layers/attention/base_attn_backend.py
    • python/sglang/srt/model_executor/forward_batch_info.py
    • python/sglang/srt/model_executor/model_runner.py
    • python/sglang/srt/server_args.py
  • Run validation (CodeQL: 0 alerts)
  • Create PR #51

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/sglang/pull/51 **Author:** [@Copilot](https://github.com/apps/copilot-swe-agent) **Created:** 5/26/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `copilot/move-unified-sparse-kv` --- ### 📝 Commits (1) - [`6467cf5`](https://github.com/kvcache-ai/sglang/commit/6467cf5b244b46751629a2dea6595c3a200518ee) feat(kse): port unified-sparse-kv KVCache Sparsity Engine from magicYang1573/sglang ### 📊 Changes **31 files changed** (+5215 additions, -1 deletions) <details> <summary>View changed files</summary> ➕ `docs/advanced_features/kv_cache_sparsity_engine-zh.md` (+909 -0) ➕ `docs/advanced_features/kv_cache_sparsity_engine.md` (+867 -0) 📝 `python/sglang/srt/layers/attention/base_attn_backend.py` (+14 -1) ➕ `python/sglang/srt/layers/kse/README.md` (+313 -0) ➕ `python/sglang/srt/layers/kse/__init__.py` (+35 -0) ➕ `python/sglang/srt/layers/kse/adapters/__init__.py` (+8 -0) ➕ `python/sglang/srt/layers/kse/adapters/flashattention_adapter.py` (+130 -0) ➕ `python/sglang/srt/layers/kse/adapters/flashinfer_adapter.py` (+167 -0) ➕ `python/sglang/srt/layers/kse/adapters/triton_adapter.py` (+113 -0) ➕ `python/sglang/srt/layers/kse/base_adapter.py` (+50 -0) ➕ `python/sglang/srt/layers/kse/base_policy.py` (+93 -0) ➕ `python/sglang/srt/layers/kse/config.py` (+18 -0) ➕ `python/sglang/srt/layers/kse/controller.py` (+251 -0) ➕ `python/sglang/srt/layers/kse/policies/__init__.py` (+7 -0) ➕ `python/sglang/srt/layers/kse/policies/quest.py` (+258 -0) ➕ `python/sglang/srt/layers/kse/policies/streaming_llm.py` (+107 -0) ➕ `python/sglang/srt/layers/kse/registry.py` (+71 -0) ➕ `python/sglang/srt/layers/kse/types.py` (+48 -0) 📝 `python/sglang/srt/model_executor/forward_batch_info.py` (+5 -0) 📝 `python/sglang/srt/model_executor/model_runner.py` (+65 -0) _...and 11 more files_ </details> ### 📄 Description - [x] Explore source branch (magicYang1573/sglang unified-sparse-kv) - [x] Identify 11 unique KSE commits and changed files - [x] Copy all new KSE files (docs, python module, tests) - [x] Apply modifications to existing files: - [x] `python/sglang/srt/layers/attention/base_attn_backend.py` - [x] `python/sglang/srt/model_executor/forward_batch_info.py` - [x] `python/sglang/srt/model_executor/model_runner.py` - [x] `python/sglang/srt/server_args.py` - [x] Run validation (CodeQL: 0 alerts) - [x] Create PR #51 --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
test/register-nixl-ep-ci
test/33111-recovery-ci
fix/pr33638-mooncake-gather
sunxun/mooncake-mxfp4-fused
fix/guard-compressed-kv-pool-init
tjq/mooncake-first-load-handover-20260906
exp/k3-7dcaf119-fi0617
release/converge-post3-runtime-main
release/0.7.0.post3-frozen-pypi
fix/glm53-index-share-config
hotfix/0.7.0.post2-pypi-runtime
feat/glm5-next-video-tools-0808
sunxun/weight-cache-elastic-ep-pr2-pr36198
sunxun/weight-cache-elastic-ep-pr1-pr36198
sunxun/weight-cache-static-dp-ep-spawn
sunxun/weight-cache-elastic-ep-pr1
sunxun/weight-cache-elastic-ep-pr2
release/transformers-kt-post2
fix/pr33684-rebase-35026-merge
feat/glm5-next-0808
unidy2002/weight-cache-static-dp-ep
copilot/move-unified-sparse-kv
fix/nan-4090
pr50
fix/scheduler-req-pool-regression
support_multi_protocol
mateng/agent-aware-kvcache-phase1-origin-main
mateng/mooncake-ib-device-routing
feature/agent-aware-streaming-session
merge-deepseek-v4
sunxun/dpc-fix
fix-kt-061-torch-range
copilot/pr20535-hicache-tests
refact_ckpt_engine
chore/kt-layerwise-prefill-main
chore/kt-layerwise-prefill-label
feat/kt-numa-nodes
add_encoder_cache_doc
pr-20129-test
fix/sglang-kt-self-referencing-extras
rename-to-sglang-kt
support_dont_store_kvcache
layerwise-prefill-opt-1
fix-cross-instance-prefetch
fix/layerwise-prefill-quant-detect
fix/mistral-kt-loader-remap
qwen3.5
oql_test_mtp
add_hicache_pp_test
add_nvlink_doc
oql-merge
prefill-limit
kimi_k2.5
layerwise-prefill-opt
experts_sched
lora-refactor-bench
lora-refactor-debug
lora-refactor
glm4.7-eplb
merge_main_oql
add_mooncake_connector2
refactor
glm47
add_hicache_pp_support
oql_fix_k2
layerwise_prefill
load_format
add_dynamic_connector_v2
add_dynamic_connect
kimi_k2
glm-46v
dev_kimi_k2
dev-djw-dsv32
support_json_topo_for_pd
kt-qwen3vl
dev-djw-ci
tmp-1
sglang-kt-ci-merge
fix_mooncake_store_test_bug
sglang-kt-ci
add_mooncake_connector
add_hicache_mooncake_gpu_ib_topo
add_gpu_id_device_topo_support
sglang-kt-unify
dev-djw
temp-fix-scaling-factor
ljq-dev
sglang-kt-fork
fix_doc_miss_index
kt-fork-tmp
develop-cht-defer
fix_nightly
support_hybrid_pd
fix_dummy_attn_backend
fix_ci_time
bench_refactor
add_multi_tenant_with_prefix_tag
refactor_pd_suite
fix_nvlink_doc
refactor_mc_ci
update_pd_abort_log
fix_mooncake_store_batch_set_v1_bug
add_mooncake_backend_extra_config
ship_pp_calculate_logic
pd_spec_improve
fix_main_ci_2
fix_main_ci
fix_hicache_mooncake_test
fix_flaky_logprobs_test
fix_gb200
upgrade_mc_0.3.6
evict_policy
add_test_hicache_storage_mooncake_backend
refactor_mooncake_conn
Xuc/ckpt-engine
add_dynamic_import_storage_backend
fix_ci_deps
fix_main_lint
support_ckpt_engine_as_model_load
fix_mooncake_storage_backend_per_tp_config
update_mooncake_store_doc
fix_layernorm_import
mooncake_bench_integrate
fix_storage_backend_clear
refactor_pd_tests
Xuc/evict
fix_multi_tokenizer
add_fastapi_for_hicache
mini-lb
fix_nvlink_accuracy
fix_failure_test
fix_router_ci
add_batch_reg_support
fix_nvlink_accuracy_issue
fixing_pd_pp
add_batch_transfer_note
support_pd_pp
update_owner
fix_pd_abort
bump_mc
fix_test_disaggregation
fix_dp_non_mla_different_tp
prevent_grammar_crash_pd
fix_prefill_timeout
fix_tbo_missing
fix_pd_router_test
fix_pd_code_format
fix_mooncake_ci
fix_mc_nvlink
tiny_fix_kvpoll
add_local_ip_nic_option
add_custom_memory_pool
improve_pd_ut
fix_transfer_infos
fix_tracker_gc
add_pd_env_var_doc
Xuc/pd-fallback
add_completion_endpoint
fix_8_gpu_ci
fix_abort_is_single
fix_abort_finish_reason
fix_pp_single_node_test
fix_prefill_unbootstrap_failure
handle_prefill_failure
add_pd_test
revert_thread_pool
optimize_ib_device_help_info
optimize_mooncake
pd_support_dp_attn
add_raise_mooncake_exception
sgl_different_tp_size
auto-detect-net
mooncake_transfer_engine_refactor
sgl_multi_prefill
mateng/sglang
Xuc/sglang
dev/mateng
leipi/devel
No results found.
Labels
Clear labels   
pull-request
Mirrored from GitHub Pull Request

No labels
pull-request
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
kvcache-ai/sglang#51
No description provided.