NVIDIA/nvbench
1
0
Fork 0
mirror of https://github.com/NVIDIA/nvbench.git synced 2026-05-12 09:15:47 +00:00
Code Issues Packages Projects Releases Wiki Activity

Use median and IR/relative as cmp_time/ref_time and cmp_noise/ref_noise

Browse Source 
These measures are less sensitive to outliers
This commit is contained in:
Oleksandr Pavlyk
2026-05-04 16:11:22 -05:00
parent ea592b6444
commit e53a1a2654
1 changed files with 4 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
python/scripts/nvbench_compare.py
View File

@@ -385,16 +385,16 @@ def compare_benches(
return next(filter(lambda s: s["tag"] == tag, summaries), None)
cmp_time_summary = lookup_summary(
cmp_summaries, "nv/cold/time/gpu/mean"
cmp_summaries, "nv/cold/time/gpu/median"
)
ref_time_summary = lookup_summary(
ref_summaries, "nv/cold/time/gpu/mean"
ref_summaries, "nv/cold/time/gpu/median"
)
cmp_noise_summary = lookup_summary(
cmp_summaries, "nv/cold/time/gpu/stdev/relative"
cmp_summaries, "nv/cold/time/gpu/ir/relative"
)
ref_noise_summary = lookup_summary(
ref_summaries, "nv/cold/time/gpu/stdev/relative"
ref_summaries, "nv/cold/time/gpu/ir/relative"
)
# TODO: Use other timings, too. Maybe multiple rows, with a