[GH-ISSUE #4660] [Bug]: TENT never releases per-peer state of departed peers (RDMA endpoints, 32 MiB pinned each, and idle RPC client connections) #1183

New issue

Open

opened 2026-10-29 21:23:43 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-29 21:23:43 +00:00

Owner

Originally created by @xiaodouzi666 on GitHub (Oct 14, 2026).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/4660

Bug Report

What happens

A TENT engine keeps per-peer state keyed by the peer's segment name (ip:port in P2P mode). When a peer goes away and is never contacted again — it exits, or restarts and comes back under a new ip:port (rpc_server_port=0 picks a random port) — that state is not released. Line numbers below are as of 342c8aee.

RDMA endpoints. Each local NIC has one SIEVEEndpointStore (src/transport/rdma/context.cpp:320), and an RdmaEndPoint is created lazily for every (local NIC, peer NIC) pair that carries traffic or that the peer bootstraps. Every endpoint's construct() allocates and registers two notification buffers of kNotifyBufferSize (64 KiB) × kNotifyMaxPendingSends (256) = 16 MiB each (include/tent/transport/rdma/endpoint.h:303-304, src/transport/rdma/endpoint.cpp:187-210): 32 MiB of pinned memory per endpoint, plus 1 + num_lanes QPs, whether or not notifications are ever used. closeSegment() only drops the segment handle mapping (src/runtime/transfer_engine_impl.cpp:735-738 → SegmentManager::closeRemote, src/runtime/segment_manager.cpp:62); no transport is told. An endpoint is retired only on a QP error or fatal event on it (workers.cpp:671-677, rdma_transport.cpp:1130-1131), a local port event (context.cpp:520), a new bootstrap from the same NIC path (rdma_transport.cpp:860-866; the comment there reads "Endpoints are never reset"), engine shutdown, or capacity eviction, which runs only when a new key is inserted after a successful construct() (endpoint_store.cpp:263) with a cap of endpoint_store_cap = 65536 per local NIC (params.h:55). An idle endpoint of a peer that left under a name that never comes back hits none of these.
RPC client connections. CoroRpcAgent keeps one ClientPool per server address (include/tent/rpc/rpc.h:126). A client goes back to the pool after every successful call (src/rpc/rpc.cpp:104), so the pool keeps as many idle connections as the peak concurrency towards that peer; it is cleared only when a later call to the same address fails or on stop() (rpc.cpp:167-168). A departed peer is never called again, so its connections stay open in CLOSE-WAIT.

Measured (two hosts, 4 RoCE NICs each; one long-lived engine serving 400 short sessions, each from a freshly started peer engine on a new ip:port):

pinned memory grows by ~181 MiB per departed peer (≈ 5.7 endpoints × 32 MiB): 70.6 GiB after 400 peers;
descriptors grow by ~6.6 per peer (2666 after 400). Classified after 20 peers: +140 TCP sockets, all CLOSE-WAIT outbound connections to the departed peers' RPC ports (7 per peer), and ~15 more each of eventpoll/timerfd/eventfd; the uverbs descriptors stay constant;
with a finite RLIMIT_MEMLOCK, once the limit is reached new peers are refused (Cannot allocate endpoint; the notification-buffer ibv_reg_mr fails with ENOMEM), and capacity eviction does not help because it only runs after a successful construct();
exiting the long-lived process then takes ~30 ms per departed peer (~12 s after 400) to release the registrations.

Why it matters

Any long-running TENT process whose peers churn — prefill/decode workers restarted by an orchestrator, P2P segments with rpc_server_port=0 — accumulates pinned memory and descriptors until RLIMIT_MEMLOCK, memory or RLIMIT_NOFILE runs out. Lowering MC_MAX_EP_PER_CTX / endpoint_store_cap bounds the endpoint part today, at the cost of also evicting live endpoints; nothing bounds the RPC pools.

Possible directions

Give the transports and the RPC agent a hook when a segment is closed or when the control plane learns that a peer is gone, then retire that peer's endpoints with the existing RdmaEndPoint::resetConnection() (endpoint.cpp:700, already used by Workers::disableEndpoint and by the notify path's RetireEndpoint) and drop its ClientPool; or an idle-time eviction in the endpoint store and the RPC pools, independent of the capacity cap; or allocate the notification buffers lazily, on the first notification. Happy to help with a patch if one of these is the preferred shape.

Reproduction

Engine A stays up; in a loop, start engine B (P2P, rpc_server_port=0), openSegment(A), WRITE one buffer to A and send A one notification, exit. Watch VmPin in /proc/<A>/status and ss -tnp for A. We measured it with the diagnostics tool we are preparing for #2864 (on top of #4154); its reports can be attached.

Environment: commit 342c8aee (first seen on an earlier main, e4229791f3); Linux x86_64 hosts with 400 Gb/s mlx5 RoCE v2 NICs; no GPU involved.

Before submitting...

Ensure you searched for relevant issues and read the [documentation]

Originally created by @xiaodouzi666 on GitHub (Oct 14, 2026). Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/4660 ### Bug Report **What happens** A TENT engine keeps per-peer state keyed by the peer's segment name (`ip:port` in P2P mode). When a peer goes away and is never contacted again — it exits, or restarts and comes back under a new `ip:port` (`rpc_server_port=0` picks a random port) — that state is not released. Line numbers below are as of `342c8aee`. 1. **RDMA endpoints.** Each local NIC has one `SIEVEEndpointStore` (`src/transport/rdma/context.cpp:320`), and an `RdmaEndPoint` is created lazily for every (local NIC, peer NIC) pair that carries traffic or that the peer bootstraps. Every endpoint's `construct()` allocates and registers two notification buffers of `kNotifyBufferSize` (64 KiB) × `kNotifyMaxPendingSends` (256) = 16 MiB each (`include/tent/transport/rdma/endpoint.h:303-304`, `src/transport/rdma/endpoint.cpp:187-210`): 32 MiB of pinned memory per endpoint, plus `1 + num_lanes` QPs, whether or not notifications are ever used. `closeSegment()` only drops the segment handle mapping (`src/runtime/transfer_engine_impl.cpp:735-738` → `SegmentManager::closeRemote`, `src/runtime/segment_manager.cpp:62`); no transport is told. An endpoint is retired only on a QP error or fatal event on it (`workers.cpp:671-677`, `rdma_transport.cpp:1130-1131`), a local port event (`context.cpp:520`), a new bootstrap from the same NIC path (`rdma_transport.cpp:860-866`; the comment there reads "Endpoints are never reset"), engine shutdown, or capacity eviction, which runs only when a new key is inserted after a successful `construct()` (`endpoint_store.cpp:263`) with a cap of `endpoint_store_cap` = 65536 per local NIC (`params.h:55`). An idle endpoint of a peer that left under a name that never comes back hits none of these. 2. **RPC client connections.** `CoroRpcAgent` keeps one `ClientPool` per server address (`include/tent/rpc/rpc.h:126`). A client goes back to the pool after every successful call (`src/rpc/rpc.cpp:104`), so the pool keeps as many idle connections as the peak concurrency towards that peer; it is cleared only when a later call to the same address fails or on `stop()` (`rpc.cpp:167-168`). A departed peer is never called again, so its connections stay open in `CLOSE-WAIT`. **Measured** (two hosts, 4 RoCE NICs each; one long-lived engine serving 400 short sessions, each from a freshly started peer engine on a new `ip:port`): - pinned memory grows by ~181 MiB per departed peer (≈ 5.7 endpoints × 32 MiB): 70.6 GiB after 400 peers; - descriptors grow by ~6.6 per peer (2666 after 400). Classified after 20 peers: +140 TCP sockets, all `CLOSE-WAIT` outbound connections to the departed peers' RPC ports (7 per peer), and ~15 more each of eventpoll/timerfd/eventfd; the uverbs descriptors stay constant; - with a finite `RLIMIT_MEMLOCK`, once the limit is reached new peers are refused (`Cannot allocate endpoint`; the notification-buffer `ibv_reg_mr` fails with `ENOMEM`), and capacity eviction does not help because it only runs after a successful `construct()`; - exiting the long-lived process then takes ~30 ms per departed peer (~12 s after 400) to release the registrations. **Why it matters** Any long-running TENT process whose peers churn — prefill/decode workers restarted by an orchestrator, P2P segments with `rpc_server_port=0` — accumulates pinned memory and descriptors until `RLIMIT_MEMLOCK`, memory or `RLIMIT_NOFILE` runs out. Lowering `MC_MAX_EP_PER_CTX` / `endpoint_store_cap` bounds the endpoint part today, at the cost of also evicting live endpoints; nothing bounds the RPC pools. **Possible directions** Give the transports and the RPC agent a hook when a segment is closed or when the control plane learns that a peer is gone, then retire that peer's endpoints with the existing `RdmaEndPoint::resetConnection()` (`endpoint.cpp:700`, already used by `Workers::disableEndpoint` and by the notify path's `RetireEndpoint`) and drop its `ClientPool`; or an idle-time eviction in the endpoint store and the RPC pools, independent of the capacity cap; or allocate the notification buffers lazily, on the first notification. Happy to help with a patch if one of these is the preferred shape. **Reproduction** Engine A stays up; in a loop, start engine B (P2P, `rpc_server_port=0`), `openSegment(A)`, WRITE one buffer to A and send A one notification, exit. Watch `VmPin` in `/proc/<A>/status` and `ss -tnp` for A. We measured it with the diagnostics tool we are preparing for #2864 (on top of #4154); its reports can be attached. **Environment**: commit `342c8aee` (first seen on an earlier main, `e4229791f3`); Linux x86_64 hosts with 400 Gb/s mlx5 RoCE v2 NICs; no GPU involved. ### Before submitting... - [ ] Ensure you searched for relevant issues and read the [documentation]

bsakaguchi added the

bug

label

2026-10-29 21:23:43 +00:00

bsakaguchi commented

2026-10-29 21:23:43 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Oct 14, 2026):

Thanks for opening this issue, @xiaodouzi666!

Field	Value
Issue	#4660
GitHub user ID	`77219630`
Reporter	@xiaodouzi666

A maintainer will triage this when possible. To help us respond faster, please include:

Mooncake version or commit SHA
Environment (OS, CUDA/driver, RDMA stack if relevant)
Steps to reproduce and expected vs. actual behavior

Useful links: Documentation · Contributing guide

This message was posted automatically by the issue bot.

@github-actions[bot] commented on GitHub (Oct 14, 2026): Thanks for opening this issue, @xiaodouzi666! | Field | Value | |-------|-------| | **Issue** | #4660 | | **GitHub user ID** | `77219630` | | **Reporter** | @xiaodouzi666 | A maintainer will triage this when possible. To help us respond faster, please include: - Mooncake version or commit SHA - Environment (OS, CUDA/driver, RDMA stack if relevant) - Steps to reproduce and expected vs. actual behavior Useful links: [Documentation](https://kvcache-ai.github.io/Mooncake/) · [Contributing guide](https://github.com/kvcache-ai/Mooncake/blob/main/CONTRIBUTING.md) > This message was posted automatically by the issue bot.