[GH-ISSUE #4822] [Feature Request]: Ship PEP 561 typing metadata and stubs for mooncake.engine #1225

New issue

Open

opened 2026-10-29 21:24:09 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-29 21:24:09 +00:00

Owner

Originally created by @MrZ20 on GitHub (Oct 21, 2026).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/4822

Originally assigned to: @stmatengss on GitHub.

Describe your feature request

Please consider shipping PEP 561 typing metadata and type stubs for the public
mooncake.engine Python API.

The published wheels expose mooncake.engine as a compiled pybind11 extension,
but they do not include an engine.pyi stub or a mooncake/py.typed marker.
Consequently, downstream projects using mypy cannot use TransferEngine as a
typed dependency and receive:

Skipping analyzing "mooncake.engine": module is installed, but missing library
stubs or py.typed marker  [import-untyped]

Evidence

I inspected the PyPI wheel
mooncake-transfer-engine-npu==0.3.13.post1 for CPython 3.12 on x86_64. It
contains mooncake/engine.so, but no .pyi files and no
mooncake/py.typed marker. The current main tree also has no .pyi or
py.typed file, and mooncake-wheel/pyproject.toml currently packages the
compiled *.so without a corresponding stub.

Minimal reproducer:

from mooncake.engine import TransferEngine

engine: TransferEngine | None = None

Running mypy check.py reports import-untyped.

This recently affected vLLM Ascend after Mooncake became an installed lint
dependency:

Failing mypy job:
https://github.com/vllm-project/vllm-ascend/actions/runs/39557520821/job/119080661493
Downstream suppression required as a workaround:
https://github.com/vllm-project/vllm-ascend/pull/19358

Suggested implementation

Provide a reviewed mooncake/engine.pyi stub for the public pybind11 API,
especially TransferEngine.
Add mooncake/py.typed once the distributed package is intended to be
PEP 561 typed.
Include the stub and marker in every wheel variant (CUDA, non-CUDA, NPU,
MUSA, EFA, and others built from the shared packaging flow).
Add a wheel-level CI smoke test that installs the built wheel and runs mypy
or Pyright against a small public-API example.

Adding only py.typed would silence the package-level import-untyped
diagnostic, but an engine.pyi file is still needed to expose useful type
information for the compiled extension.

Before submitting a new issue...

I searched for existing issues and documentation related to py.typed,
PEP 561, mypy, and Python type stubs.

Originally created by @MrZ20 on GitHub (Oct 21, 2026). Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/4822 Originally assigned to: @stmatengss on GitHub. ### Describe your feature request Please consider shipping PEP 561 typing metadata and type stubs for the public `mooncake.engine` Python API. The published wheels expose `mooncake.engine` as a compiled pybind11 extension, but they do not include an `engine.pyi` stub or a `mooncake/py.typed` marker. Consequently, downstream projects using mypy cannot use `TransferEngine` as a typed dependency and receive: ```text Skipping analyzing "mooncake.engine": module is installed, but missing library stubs or py.typed marker [import-untyped] ``` #### Evidence I inspected the PyPI wheel `mooncake-transfer-engine-npu==0.3.13.post1` for CPython 3.12 on x86_64. It contains `mooncake/engine.so`, but no `.pyi` files and no `mooncake/py.typed` marker. The current `main` tree also has no `.pyi` or `py.typed` file, and `mooncake-wheel/pyproject.toml` currently packages the compiled `*.so` without a corresponding stub. Minimal reproducer: ```python from mooncake.engine import TransferEngine engine: TransferEngine | None = None ``` Running `mypy check.py` reports `import-untyped`. This recently affected vLLM Ascend after Mooncake became an installed lint dependency: - Failing mypy job: https://github.com/vllm-project/vllm-ascend/actions/runs/39557520821/job/119080661493 - Downstream suppression required as a workaround: https://github.com/vllm-project/vllm-ascend/pull/19358 #### Suggested implementation 1. Provide a reviewed `mooncake/engine.pyi` stub for the public pybind11 API, especially `TransferEngine`. 2. Add `mooncake/py.typed` once the distributed package is intended to be PEP 561 typed. 3. Include the stub and marker in every wheel variant (CUDA, non-CUDA, NPU, MUSA, EFA, and others built from the shared packaging flow). 4. Add a wheel-level CI smoke test that installs the built wheel and runs mypy or Pyright against a small public-API example. Adding only `py.typed` would silence the package-level `import-untyped` diagnostic, but an `engine.pyi` file is still needed to expose useful type information for the compiled extension. ### Before submitting a new issue... - [x] I searched for existing issues and documentation related to `py.typed`, PEP 561, mypy, and Python type stubs.

bsakaguchi commented

2026-10-29 21:24:10 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Oct 21, 2026):

Thanks for opening this issue, @MrZ20!

Field	Value
Issue	#4822
GitHub user ID	`121143079`
Reporter	@MrZ20

A maintainer will triage this when possible. To help us respond faster, please include:

Mooncake version or commit SHA
Environment (OS, CUDA/driver, RDMA stack if relevant)
Steps to reproduce and expected vs. actual behavior

Useful links: Documentation · Contributing guide

This message was posted automatically by the issue bot.

@github-actions[bot] commented on GitHub (Oct 21, 2026): Thanks for opening this issue, @MrZ20! | Field | Value | |-------|-------| | **Issue** | #4822 | | **GitHub user ID** | `121143079` | | **Reporter** | @MrZ20 | A maintainer will triage this when possible. To help us respond faster, please include: - Mooncake version or commit SHA - Environment (OS, CUDA/driver, RDMA stack if relevant) - Steps to reproduce and expected vs. actual behavior Useful links: [Documentation](https://kvcache-ai.github.io/Mooncake/) · [Contributing guide](https://github.com/kvcache-ai/Mooncake/blob/main/CONTRIBUTING.md) > This message was posted automatically by the issue bot.

bsakaguchi commented

2026-10-29 21:24:11 +00:00

Author

Owner

@stmatengss commented on GitHub (Oct 22, 2026):

@MrZ20 Please make sure https://github.com/kvcache-ai/Mooncake/pull/4836 resolve your problems

@stmatengss commented on GitHub (Oct 22, 2026): @MrZ20 Please make sure https://github.com/kvcache-ai/Mooncake/pull/4836 resolve your problems

bsakaguchi referenced this issue

2026-10-29 21:32:44 +00:00

[PR #1236] [MERGED] [EP] Implement send/recv #2094

bsakaguchi referenced this issue

2026-10-29 21:33:09 +00:00

[PR #1308] [MERGED] [EP] Use NVLink in EP if possible #2149