kvcache-ai
Mirrored from GitHub organization: kvcache-ai
Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.
Updated 2026-11-04 12:43:02 +00:00
A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations
Updated 2026-10-28 06:08:08 +00:00
Updated 2026-10-20 07:05:33 +00:00
Updated 2026-10-19 00:32:26 +00:00
🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support
Updated 2026-10-10 03:10:21 +00:00
🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
Updated 2026-09-30 13:28:50 +00:00
AFD-Ledger is an analytical toolkit for provisioning and evaluating attention–FFN disaggregated LLM inference deployments.
Updated 2026-08-07 11:22:33 +00:00
Secure and fast lightweight microVMs for serverless computing. This is a fork from loopholelabs that adds live migration and optionally PMV support. (No KVM or Nested Virtualization Needed)
Updated 2026-07-30 03:44:54 +00:00
Updated 2026-07-25 17:50:13 +00:00
A unified library of SOTA model optimization techniques like quantization, pruning, distillation, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed.
Updated 2026-06-17 09:29:07 +00:00