[PR #1310] [MERGED] [TE] Add Task Completion Latency Tracking and Detailed Metrics Reporting

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

kvcache-ai/Mooncake#2151

bsakaguchi commented

2026-10-29 21:33:10 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/Mooncake/pull/1310
Author: @staryxchen
Created: 12/30/2025
Status: ✅ Merged
Merged: 12/31/2025
Merged by: @chestnut-Q

Base: main ← Head: refactor/metrics-tracking-tent

📝 Commits (6)

b34d9ff feat(metrics): add task completion latency tracking and detailed metrics reporting
72463bf feat: add metrics support for submitTransferWithNotify
25299be refactor(metrics): improve metrics recording logic with early returns
c054f85 refactor(metrics): simplify bucket count tracking logic
e10a613 fix(metrics): conditionally enable metrics collection
5df05ea docs: clarify Transfer Engine metrics limitation with TENT

📊 Changes

6 files changed (+191 additions, -19 deletions)

View changed files

📝 docs/source/deployment/mooncake-store-deployment-guide.md (+1 -1)
📝 docs/source/python-api-reference/transfer-engine.md (+1 -1)
📝 mooncake-transfer-engine/include/transfer_engine_impl.h (+84 -7)
📝 mooncake-transfer-engine/include/transport/transport.h (+4 -0)
📝 mooncake-transfer-engine/src/transfer_engine.cpp (+4 -1)
📝 mooncake-transfer-engine/src/transfer_engine_impl.cpp (+97 -9)

📄 Description

Description

This PR enhances the metrics system in the Transfer Engine by adding task completion latency tracking using histograms and improving the metrics reporting output.

Also fix two bug:

Add skip_metrics parameter to getBatchTransferStatus() to prevent double-counting when called internally
Remove acquiring the lock on critical data paths to prevent performance degradation, reported in #1303

Type of Change

Types
- Bug fix
- New feature
  - Transfer Engine
  - Mooncake Store
  - Mooncake EP
  - Integration
  - P2P Store
  - Python Wheel
- Breaking change
- CI/CD
- Documentation update
- Other

How Has This Been Tested?

Checklist

I have performed a self-review of my own code.
I have updated the documentation.
I have added tests to prove my changes are effective.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/Mooncake/pull/1310 **Author:** [@staryxchen](https://github.com/staryxchen) **Created:** 12/30/2025 **Status:** ✅ Merged **Merged:** 12/31/2025 **Merged by:** [@chestnut-Q](https://github.com/chestnut-Q) **Base:** `main` ← **Head:** `refactor/metrics-tracking-tent` --- ### 📝 Commits (6) - [`b34d9ff`](https://github.com/kvcache-ai/Mooncake/commit/b34d9ff4684e3e91255c58f9065cfc5e6ae7d637) feat(metrics): add task completion latency tracking and detailed metrics reporting - [`72463bf`](https://github.com/kvcache-ai/Mooncake/commit/72463bf955b651acd47c74c060f476e7c75ae2f1) feat: add metrics support for submitTransferWithNotify - [`25299be`](https://github.com/kvcache-ai/Mooncake/commit/25299be57216add0b880af61ad7595dccac77c67) refactor(metrics): improve metrics recording logic with early returns - [`c054f85`](https://github.com/kvcache-ai/Mooncake/commit/c054f854d36d35b1994d40714c5db7db976c6589) refactor(metrics): simplify bucket count tracking logic - [`e10a613`](https://github.com/kvcache-ai/Mooncake/commit/e10a613480e7e4a95a5839f79f0efe0dde51cc4b) fix(metrics): conditionally enable metrics collection - [`5df05ea`](https://github.com/kvcache-ai/Mooncake/commit/5df05ea2f40f86135c4f2c362c661ad4ababc29e) docs: clarify Transfer Engine metrics limitation with TENT ### 📊 Changes **6 files changed** (+191 additions, -19 deletions) <details> <summary>View changed files</summary> 📝 `docs/source/deployment/mooncake-store-deployment-guide.md` (+1 -1) 📝 `docs/source/python-api-reference/transfer-engine.md` (+1 -1) 📝 `mooncake-transfer-engine/include/transfer_engine_impl.h` (+84 -7) 📝 `mooncake-transfer-engine/include/transport/transport.h` (+4 -0) 📝 `mooncake-transfer-engine/src/transfer_engine.cpp` (+4 -1) 📝 `mooncake-transfer-engine/src/transfer_engine_impl.cpp` (+97 -9) </details> ### 📄 Description ## Description This PR enhances the metrics system in the Transfer Engine by adding task completion latency tracking using histograms and improving the metrics reporting output. Also fix two bug: - Add skip_metrics parameter to getBatchTransferStatus() to prevent double-counting when called internally - Remove acquiring the lock on critical data paths to prevent performance degradation, reported in #1303 ## Type of Change * Types - [ ] Bug fix - [x] New feature - [x] Transfer Engine - [ ] Mooncake Store - [ ] Mooncake EP - [ ] Integration - [ ] P2P Store - [ ] Python Wheel - [ ] Breaking change - [ ] CI/CD - [ ] Documentation update - [ ] Other ## How Has This Been Tested? <img width="882" height="354" alt="Clipboard_Screenshot_1767098059" src="https://github.com/user-attachments/assets/2e6274d6-6e8f-4753-bc70-e59da25daf0f" /> ## Checklist - [x] I have performed a self-review of my own code. - [x] I have updated the documentation. - [x] I have added tests to prove my changes are effective. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

closed this issue
added the
pull-request
label

Rows
Columns

[PR #1310] [MERGED] [TE] Add Task Completion Latency Tracking and Detailed Metrics Reporting #2151