[GH-ISSUE #1078] [Feature] How to measue the concurrency performence #757

New issue

Closed

opened 2026-10-29 14:44:05 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-29 14:44:05 +00:00

Owner

Originally created by @yydxlv on GitHub (Apr 8, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1078

Thanks for the wonderful work,
As the demo shows, How can i measure the concurrency performence ,such as the prefill and decode performence ?

Originally created by @yydxlv on GitHub (Apr 8, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1078 Thanks for the wonderful work, As the demo shows, How can i measure the concurrency performence ,such as the prefill and decode performence ?

bsakaguchi closed this issue

2026-10-29 14:44:05 +00:00

bsakaguchi commented

2026-10-29 14:44:06 +00:00

Author

Owner

@qiyuxinlin commented on GitHub (Apr 8, 2025):

Hi @yydxlv, when a request is completed, the terminal will print the model's prefill and decode speeds.
You can add up the prefill and decode speeds of all requests.

@qiyuxinlin commented on GitHub (Apr 8, 2025): Hi @yydxlv, when a request is completed, the terminal will print the model's prefill and decode speeds. You can add up the prefill and decode speeds of all requests.