kvcache-ai
Mirrored from GitHub organization: kvcache-ai
A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations
Updated 2026-10-28 06:08:08 +00:00
🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support
Updated 2026-10-10 03:10:21 +00:00
🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
Updated 2026-09-30 13:28:50 +00:00
AFD-Ledger is an analytical toolkit for provisioning and evaluating attention–FFN disaggregated LLM inference deployments.
Updated 2026-08-07 11:22:33 +00:00
A unified library of SOTA model optimization techniques like quantization, pruning, distillation, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed.
Updated 2026-06-17 09:29:07 +00:00
SGLang is a fast serving framework for large language models and vision language models.
Updated 2026-04-22 05:25:53 +00:00
SGLang is a fast serving framework for large language models and vision language models.
Updated 2025-08-12 08:21:02 +00:00