mirror of
https://github.com/kvcache-ai/custom_flashinfer.git
synced 2026-07-02 21:28:23 +00:00
29 lines
525 B
ReStructuredText
29 lines
525 B
ReStructuredText
.. _apidecode:
|
|
|
|
flashinfer.decode
|
|
=================
|
|
|
|
.. currentmodule:: flashinfer.decode
|
|
|
|
Single Request Decoding
|
|
-----------------------
|
|
|
|
.. autosummary::
|
|
:toctree: ../generated
|
|
|
|
single_decode_with_kv_cache
|
|
|
|
Batch Decoding
|
|
--------------
|
|
|
|
.. autoclass:: BatchDecodeWithPagedKVCacheWrapper
|
|
:members:
|
|
:exclude-members: begin_forward, end_forward, forward, forward_return_lse
|
|
|
|
.. automethod:: __init__
|
|
|
|
.. autoclass:: CUDAGraphBatchDecodeWithPagedKVCacheWrapper
|
|
:members:
|
|
|
|
.. automethod:: __init__
|