kvcache-ai/Mooncake

Fork 0

mirror of https://github.com/kvcache-ai/Mooncake.git synced 2026-11-05 01:17:55 +00:00

[PR #1355] [MERGED] [TENT] Add Metrics System with HTTP Server and Prometheus Integration #2188

New issue

Closed

opened 2026-10-29 21:33:26 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 21:33:26 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/Mooncake/pull/1355
Author: @staryxchen
Created: 1/11/2026
Status: ✅ Merged
Merged: 1/12/2026
Merged by: @alogfans

Base: main ← Head: dev/metric-tent

📝 Commits (10+)

7295832 feat(metrics): add TENT metrics system with HTTP server and Prometheus
bf57aa2 Update docs/source/design/tent/metrics.md
aeac251 refactor(metrics): simplify config loading with explicit priority
58589f7 refactor(transfer_engine): extract metrics recording logic into dedicated method
b2c6f6c build(metrics): improve yalantinglibs dependency handling
acb368e refactor(metrics): replace manual JSON construction with nlohmann/json library
7f8a5c6 style: reformat code with clang-format
48a5a0d refactor(config): centralize parsing utilities in ConfigHelper
3d9a540 test: add unit tests for metrics config loader and reorganize test structure
c01191d style: reformat code lines for better readability

📊 Changes

17 files changed (+2319 additions, -2 deletions)

View changed files

➕ docs/source/design/tent/metrics.md (+388 -0)
📝 mooncake-transfer-engine/tent/CMakeLists.txt (+2 -1)
📝 mooncake-transfer-engine/tent/config/transfer-engine.json (+18 -0)
📝 mooncake-transfer-engine/tent/include/tent/common/config.h (+9 -0)
➕ mooncake-transfer-engine/tent/include/tent/metrics/config_loader.h (+100 -0)
➕ mooncake-transfer-engine/tent/include/tent/metrics/tent_metrics.h (+276 -0)
📝 mooncake-transfer-engine/tent/include/tent/runtime/transfer_engine_impl.h (+6 -0)
📝 mooncake-transfer-engine/tent/src/CMakeLists.txt (+1 -0)
📝 mooncake-transfer-engine/tent/src/common/config.cpp (+80 -0)
➕ mooncake-transfer-engine/tent/src/metrics/CMakeLists.txt (+29 -0)
➕ mooncake-transfer-engine/tent/src/metrics/config_loader.cpp (+239 -0)
➕ mooncake-transfer-engine/tent/src/metrics/tent_metrics.cpp (+391 -0)
📝 mooncake-transfer-engine/tent/src/runtime/CMakeLists.txt (+1 -1)
📝 mooncake-transfer-engine/tent/src/runtime/transfer_engine_impl.cpp (+74 -0)
➕ mooncake-transfer-engine/tent/tests/CMakeLists.txt (+12 -0)
➕ mooncake-transfer-engine/tent/tests/metrics_config_loader_test.cpp (+433 -0)
➕ mooncake-transfer-engine/tent/tests/tent_metrics_example.cpp (+260 -0)

📄 Description

Summary

This PR introduces a comprehensive metrics system for TENT, providing Prometheus-compatible metrics collection and HTTP server endpoints for monitoring data transfer performance.

Key Changes

New Features

Metrics Collection System: Implemented counter and histogram metrics for monitoring read/write operations, including:
- Total bytes transferred (read/write)
- Request counts (read/write)
- Failure counts (read/write)
- Latency distribution histograms
- Request size distribution histograms
HTTP Server with Multiple Endpoints:
- /metrics - Prometheus format for scraping
- /metrics/json - JSON format for programmatic access
- /metrics/summary - Human-readable summary
- /health - Health check endpoint
Configuration System: Flexible configuration via:
- JSON config file (transfer-engine.json)
- Environment variables
- Programmatic API
Performance Optimizations:
- Compile-time disable option (-DTENT_METRICS_ENABLED=OFF) for zero runtime overhead
- Runtime disable option via TentMetrics::setEnabled(false) for minimal overhead (~1ns atomic load)
- Pre-allocated buffers for reduced memory allocation in hot paths

Integration with TransferEngine

Metrics are automatically integrated into TransferEngine with RAII-style latency tracking:

Start time recorded on submitTransfer
Latency calculated and metrics recorded when getTransferStatus detects task completion

Important Design Decisions

Metrics Disabled by Default

This is a significant change that adds a new subsystem to the codebase. To ensure stability and avoid any unexpected performance impact on existing deployments, metrics are disabled by default at compile time (TENT_METRICS_ENABLED=OFF).

Users who want to enable metrics must explicitly build with:

cmake -DTENT_METRICS_ENABLED=ON ..

This conservative approach allows teams to:

Evaluate the metrics system in non-production environments first
Gradually roll out metrics collection
Maintain zero overhead for performance-critical deployments

Current Metrics Scope

The current implementation provides metrics that are consistent with the classic Transfer Engine metrics, focusing on:

Transfer throughput (bytes read/written)
Request rates
Failure rates
Latency distributions

This implementation is designed to be extensible. Future PRs can easily add more metrics by following the documented pattern in the metrics documentation.

Documentation and Examples

This PR includes comprehensive documentation and examples:

Documentation (docs/source/design/tent/metrics.md):
- Configuration guide (config file, environment variables)
- Quick start guide
- API reference
- Prometheus/Grafana integration examples
- Guide for adding new metrics
Example Application (mooncake-transfer-engine/tent/examples/tent_metrics_example.cpp):
- Demonstrates initialization and configuration
- Shows manual and RAII-style latency recording
- Demonstrates runtime enable/disable switch
- Includes HTTP server endpoint testing

Files Changed

File	Description
`docs/source/design/tent/metrics.md`	Comprehensive documentation
`tent/include/tent/metrics/tent_metrics.h`	Core metrics API and macros
`tent/include/tent/metrics/config_loader.h`	Configuration loading API
`tent/src/metrics/tent_metrics.cpp`	Metrics implementation
`tent/src/metrics/config_loader.cpp`	Configuration loader implementation
`tent/src/runtime/transfer_engine_impl.cpp`	TransferEngine integration
`tent/examples/tent_metrics_example.cpp`	Example application
`tent/config/transfer-engine.json`	Example configuration file

Testing

Example application demonstrates all features

HTTP endpoints can be tested with curl:

curl http://localhost:9100/metrics
curl http://localhost:9100/metrics/json
curl http://localhost:9100/metrics/summary
curl http://localhost:9100/health

Future Work

This implementation lays the foundation for extended metrics support. Potential future additions include:

Per-transport metrics (RDMA, TCP, NVLink, etc.)
Connection pool statistics
Memory buffer utilization
Queue depth metrics
Custom user-defined metrics

Type of Change

Types
- Bug fix
- New feature
  - Transfer Engine
  - Mooncake Store
  - Mooncake EP
  - Integration
  - P2P Store
  - Python Wheel
- Breaking change
- CI/CD
- Documentation update
- Other

How Has This Been Tested?

Checklist

I have performed a self-review of my own code.
I have updated the documentation.
I have added tests to prove my changes are effective.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/Mooncake/pull/1355 **Author:** [@staryxchen](https://github.com/staryxchen) **Created:** 1/11/2026 **Status:** ✅ Merged **Merged:** 1/12/2026 **Merged by:** [@alogfans](https://github.com/alogfans) **Base:** `main` ← **Head:** `dev/metric-tent` --- ### 📝 Commits (10+) - [`7295832`](https://github.com/kvcache-ai/Mooncake/commit/72958328da13ec4c33605ff10cd418f37668451a) feat(metrics): add TENT metrics system with HTTP server and Prometheus - [`bf57aa2`](https://github.com/kvcache-ai/Mooncake/commit/bf57aa245652e3d72a9eb64896e46502e18addbf) Update docs/source/design/tent/metrics.md - [`aeac251`](https://github.com/kvcache-ai/Mooncake/commit/aeac2514061d479f2a8119f53942b5d280f8fedd) refactor(metrics): simplify config loading with explicit priority - [`58589f7`](https://github.com/kvcache-ai/Mooncake/commit/58589f7750ae68eedd02b268fc3dcc420089a183) refactor(transfer_engine): extract metrics recording logic into dedicated method - [`b2c6f6c`](https://github.com/kvcache-ai/Mooncake/commit/b2c6f6c94d919d89e90b528feaf24946b7992cb3) build(metrics): improve yalantinglibs dependency handling - [`acb368e`](https://github.com/kvcache-ai/Mooncake/commit/acb368ed4bd99b0b26e2e305ef4f0c883b37cf09) refactor(metrics): replace manual JSON construction with nlohmann/json library - [`7f8a5c6`](https://github.com/kvcache-ai/Mooncake/commit/7f8a5c67ca7db4c9c7be3c0841c3ba477fca179b) style: reformat code with clang-format - [`48a5a0d`](https://github.com/kvcache-ai/Mooncake/commit/48a5a0de2dc692b0e2c19e4b02a78fd4094e7225) refactor(config): centralize parsing utilities in ConfigHelper - [`3d9a540`](https://github.com/kvcache-ai/Mooncake/commit/3d9a54039257618cf02ba098416884c99c0bf4fb) test: add unit tests for metrics config loader and reorganize test structure - [`c01191d`](https://github.com/kvcache-ai/Mooncake/commit/c01191d3d4fafd9d7f70a5455bbb5ccc19512209) style: reformat code lines for better readability ### 📊 Changes **17 files changed** (+2319 additions, -2 deletions) <details> <summary>View changed files</summary> ➕ `docs/source/design/tent/metrics.md` (+388 -0) 📝 `mooncake-transfer-engine/tent/CMakeLists.txt` (+2 -1) 📝 `mooncake-transfer-engine/tent/config/transfer-engine.json` (+18 -0) 📝 `mooncake-transfer-engine/tent/include/tent/common/config.h` (+9 -0) ➕ `mooncake-transfer-engine/tent/include/tent/metrics/config_loader.h` (+100 -0) ➕ `mooncake-transfer-engine/tent/include/tent/metrics/tent_metrics.h` (+276 -0) 📝 `mooncake-transfer-engine/tent/include/tent/runtime/transfer_engine_impl.h` (+6 -0) 📝 `mooncake-transfer-engine/tent/src/CMakeLists.txt` (+1 -0) 📝 `mooncake-transfer-engine/tent/src/common/config.cpp` (+80 -0) ➕ `mooncake-transfer-engine/tent/src/metrics/CMakeLists.txt` (+29 -0) ➕ `mooncake-transfer-engine/tent/src/metrics/config_loader.cpp` (+239 -0) ➕ `mooncake-transfer-engine/tent/src/metrics/tent_metrics.cpp` (+391 -0) 📝 `mooncake-transfer-engine/tent/src/runtime/CMakeLists.txt` (+1 -1) 📝 `mooncake-transfer-engine/tent/src/runtime/transfer_engine_impl.cpp` (+74 -0) ➕ `mooncake-transfer-engine/tent/tests/CMakeLists.txt` (+12 -0) ➕ `mooncake-transfer-engine/tent/tests/metrics_config_loader_test.cpp` (+433 -0) ➕ `mooncake-transfer-engine/tent/tests/tent_metrics_example.cpp` (+260 -0) </details> ### 📄 Description ## Summary This PR introduces a comprehensive metrics system for TENT, providing Prometheus-compatible metrics collection and HTTP server endpoints for monitoring data transfer performance. ## Key Changes ### New Features - **Metrics Collection System**: Implemented counter and histogram metrics for monitoring read/write operations, including: - Total bytes transferred (read/write) - Request counts (read/write) - Failure counts (read/write) - Latency distribution histograms - Request size distribution histograms - **HTTP Server with Multiple Endpoints**: - `/metrics` - Prometheus format for scraping - `/metrics/json` - JSON format for programmatic access - `/metrics/summary` - Human-readable summary - `/health` - Health check endpoint - **Configuration System**: Flexible configuration via: - JSON config file (`transfer-engine.json`) - Environment variables - Programmatic API - **Performance Optimizations**: - Compile-time disable option (`-DTENT_METRICS_ENABLED=OFF`) for **zero runtime overhead** - Runtime disable option via `TentMetrics::setEnabled(false)` for minimal overhead (~1ns atomic load) - Pre-allocated buffers for reduced memory allocation in hot paths ### Integration with TransferEngine Metrics are automatically integrated into TransferEngine with RAII-style latency tracking: - Start time recorded on `submitTransfer` - Latency calculated and metrics recorded when `getTransferStatus` detects task completion ## Important Design Decisions ### Metrics Disabled by Default **This is a significant change that adds a new subsystem to the codebase.** To ensure stability and avoid any unexpected performance impact on existing deployments, **metrics are disabled by default at compile time** (`TENT_METRICS_ENABLED=OFF`). Users who want to enable metrics must explicitly build with: ```bash cmake -DTENT_METRICS_ENABLED=ON .. ``` This conservative approach allows teams to: 1. Evaluate the metrics system in non-production environments first 2. Gradually roll out metrics collection 3. Maintain zero overhead for performance-critical deployments ### Current Metrics Scope The current implementation provides metrics that are **consistent with the classic Transfer Engine** metrics, focusing on: - Transfer throughput (bytes read/written) - Request rates - Failure rates - Latency distributions **This implementation is designed to be extensible.** Future PRs can easily add more metrics by following the documented pattern in the metrics documentation. ## Documentation and Examples This PR includes comprehensive documentation and examples: - **Documentation** (`docs/source/design/tent/metrics.md`): - Configuration guide (config file, environment variables) - Quick start guide - API reference - Prometheus/Grafana integration examples - Guide for adding new metrics - **Example Application** (`mooncake-transfer-engine/tent/examples/tent_metrics_example.cpp`): - Demonstrates initialization and configuration - Shows manual and RAII-style latency recording - Demonstrates runtime enable/disable switch - Includes HTTP server endpoint testing ## Files Changed | File | Description | |------|-------------| | `docs/source/design/tent/metrics.md` | Comprehensive documentation | | `tent/include/tent/metrics/tent_metrics.h` | Core metrics API and macros | | `tent/include/tent/metrics/config_loader.h` | Configuration loading API | | `tent/src/metrics/tent_metrics.cpp` | Metrics implementation | | `tent/src/metrics/config_loader.cpp` | Configuration loader implementation | | `tent/src/runtime/transfer_engine_impl.cpp` | TransferEngine integration | | `tent/examples/tent_metrics_example.cpp` | Example application | | `tent/config/transfer-engine.json` | Example configuration file | ## Testing - Example application demonstrates all features - HTTP endpoints can be tested with `curl`: ```bash curl http://localhost:9100/metrics curl http://localhost:9100/metrics/json curl http://localhost:9100/metrics/summary curl http://localhost:9100/health ``` ## Future Work This implementation lays the foundation for extended metrics support. Potential future additions include: - Per-transport metrics (RDMA, TCP, NVLink, etc.) - Connection pool statistics - Memory buffer utilization - Queue depth metrics - Custom user-defined metrics ## Type of Change * Types - [ ] Bug fix - [x] New feature - [x] Transfer Engine - [ ] Mooncake Store - [ ] Mooncake EP - [ ] Integration - [ ] P2P Store - [ ] Python Wheel - [ ] Breaking change - [ ] CI/CD - [x] Documentation update - [ ] Other ## How Has This Been Tested? <img width="872" height="803" alt="Clipboard_Screenshot_1768135129" src="https://github.com/user-attachments/assets/8f972f5d-0be7-40b6-b213-0c1ce9925193" /> ## Checklist - [x] I have performed a self-review of my own code. - [x] I have updated the documentation. - [x] I have added tests to prove my changes are effective. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-29 21:33:26 +00:00