NVIDIA/nvbench
1
0
Fork 0
mirror of https://github.com/NVIDIA/nvbench.git synced 2026-04-20 14:58:54 +00:00
Code Issues Packages Projects Releases Wiki Activity
219 Commits 8 Branches 5 Tags
237363784afb67a604533ee261d723ea33a2b893
Commit Graph

219 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Robert Maynard
237363784a Patch nlohmann json to avoid a compiler bug in nvcc 11.0 
Fixes #18
 2021-07-08 16:52:52 -04:00
Allison Vacanti
e84e13ed51 Simplify benchmark_base::set_printer API. 
Forcing the caller to use `std::ref` is gross.
 2021-06-23 17:04:41 -04:00
Allison Vacanti
5fab2536e5 Merge pull request #14 from vyasr/enhanced_compare 
Improve compare output
 2021-06-22 16:27:03 -04:00
Allison Vacanti
861f66c161 Update comparison script. 
- Handle multiple benchmarks.
- Split output to resemble nvbench markdown output:
  - Headings for benchmarks / devices
  - Compact tables with axis values / stats
- Handle `null` stdev (these are infinite, e.g. too few samples)
- Format numbers, times, percentages similar to nvbench.
- Add summary of total/pass/unknown/failure stats
- Add new test.json files (with multiple benchmarks in each)
 2021-06-22 16:15:22 -04:00
Allison Vacanti
668a98ef16 Encode generic numerics as strings in JSON format. 
JSON encodes all numerics as `double`s, which would truncate int64_t.
Write all generic numbers from `named_values` objects as strings.
These are still distinguishable by their `type` properties.
 2021-06-22 16:10:52 -04:00
Vyas Ramasubramani
f1715786fd Add bracketing newlines. 2021-06-22 12:30:05 -04:00
Vyas Ramasubramani
1002082817 Apply black formatting. 2021-06-22 12:30:05 -04:00
Vyas Ramasubramani
dae1f16426 Add some intermeidate variables and simplify code to make it self-documenting. 2021-06-22 12:30:05 -04:00
Vyas Ramasubramani
053eb493c7 Add nonzero error code on exit if any tests fail and change output format to Github-style Markdown. 2021-06-22 12:30:05 -04:00
Vyas Ramasubramani
ab39c924b2 Add comment regarding better metrics of failure. 2021-06-22 12:30:05 -04:00
Ashwin Srinath
52775574a3 Add colorama to requirements 2021-06-22 12:30:05 -04:00
Ashwin Srinath
f3658c2b77 Add a requirements.txt 2021-06-22 12:30:05 -04:00
Ashwin Srinath
0f63f86e9f Report pass/fail status 2021-06-22 12:30:05 -04:00
Vyas Ramasubramani
38ec66dc0f Add small comment. 2021-06-22 12:30:05 -04:00
Vyas Ramasubramani
5db95906f8 Convert named values to a key-value mapping from a list. 2021-06-22 12:30:05 -04:00
Vyas Ramasubramani
c2868dfa35 Convert states into a key-value mapping. 2021-06-22 12:30:05 -04:00
Vyas Ramasubramani
0491e9c17f Store axes and summaries as dictionaries rather than lists. 2021-06-22 12:30:05 -04:00
Vyas Ramasubramani
8fcad4908f Improve table formatting, add calculated performance difference. 2021-06-22 12:30:05 -04:00
Ashwin Srinath
d9704f0db1 Add pretty printing of table 2021-06-22 12:30:05 -04:00
Vyas Ramasubramani
ed4feade63 Use f-string with explicit format width. 2021-06-22 12:30:05 -04:00
Allison Vacanti
1fc4576ceb Initial draft of python comparison script. 2021-06-22 12:29:53 -04:00
Allison Vacanti
ff507596bf Fix typo in docs. 2021-04-12 14:48:45 -04:00
Allison Vacanti
4ab0f5de7c Change CPU timer start point. 2021-04-10 20:00:48 -04:00
Allison Vacanti
34479b3116 Add path for json develop branch to simplify upstream patch testing. 2021-03-24 10:00:59 -04:00
Allison Vacanti
ebd79caa39 Add state.add_buffer_size(...) method. 2021-03-18 18:09:04 -04:00
Allison Vacanti
926c950419 Update item rate formatting. 
The number of items per second can go very high when using thrust
fancy iterators -- some of the reduction benchmarks are handling
>10^15 items/sec.

Updated the table to go to petaitems/sec, and switched from k,m,b,t
abbreviations to metric K, M, G, T, P.
 2021-03-18 18:05:42 -04:00
Allison Vacanti
95b3c459b8 Add some columns to an example. 2021-03-18 16:08:30 -04:00
Allison Vacanti
790160d6e6 Rename .h -> .cuh. 2021-03-18 16:08:16 -04:00
Allison Vacanti
0dcd915ea6 Fix test failure. 2021-03-18 16:07:50 -04:00
Allison Vacanti
8b7a2e86b8 Avoid recompiling option_parser every time cmake runs. 
Switch to `configure_file`, which won't touch the output file unless
the contents change.
 2021-03-18 16:07:40 -04:00
Allison Vacanti
52d3dfec8c Add size columns to an axes example. 2021-03-18 13:43:27 -04:00
Allison Vacanti
4e83e048ba Store percentages as ratios. 
Human-readable outputs (md) and CLI inputs still use percentages.
In-memory and machine-readable outputs (csv, json) use ratios.

This is the convention that spreadsheet apps expect. Fixes #2.
 2021-03-18 13:42:43 -04:00
Allison Vacanti
e62c786498 Set small timeout when running examples with ctest. 2021-03-18 13:33:50 -04:00
Allison Vacanti
ea53972af8 Add nvbench.all metatarget. 
This builds all NVBench tests and examples without building targets in
any parent projects.
 2021-03-18 13:33:23 -04:00
Allison Vacanti
94bf208701 Formatting and doc update. 2021-03-18 10:46:28 -04:00
Allison Vacanti
ab8fc13aac Simplify enum_type_list usage. 
Using C++17 auto template parameters enables two improvements:

1. Enum type no longer needed in `enum_type_list`:

```
// Old:
nvbench::enum_type_list<my_enum, my_enum::value1, my_enum::value2, ...>
// New:
nvbench::enum_type_list<my_enum::value1, my_enum::value2, ...>
```

2. New `nvbench::enum_type` helper cuts down on verbosity when writing
benchmarks

```
// Old
template <typename T, my_enum_a MyEnumA, my_enum_b MyEnumB>
void my_bench(nvbench::state &state,
              nvbench::type_list<T,
                                 std::integral_constant<my_enum_a,
                                                        MyEnumA>,
                                 std::integral_constant<my_enum_b,
                                                        MyEnumB>>)
{ ... }

// New
template <typename T, my_enum_a MyEnumA, my_enum_b MyEnumB>
void my_bench(nvbench::state &state,
              nvbench::type_list<T,
                                 nvbench::enum_type<MyEnumA>,
                                 nvbench::enum_type<MyEnumB>>)
{ ... }
```
 2021-03-18 02:59:51 -04:00
Allison Vacanti
01cb22cad0 Move demangle<T>() out of detail namespace. 2021-03-17 19:22:28 -04:00
Allison Vacanti
d928b9a50f Make column name optional for non-templated throughput setters. 2021-03-17 19:21:42 -04:00
Allison Vacanti
9f6404bac6 Pad range max for floating point types. 
This avoids an annoying case where the max value is dropped due to
rounding errors.

Adds a few other missing test cases for `nvbench::range`, too.

Fixes #3.
 2021-03-16 15:12:53 -04:00
Allison Vacanti
56414078ea Added state::get_XXXX_or_default(axis_name, default_value) 2021-03-16 14:36:20 -04:00
Allison Vacanti
60c94d9ed6 Add enum_type_axis and examples/enums.cu. 
- `enum_type_axis` simplifies using integral_constants with type axes.
- `examples/enums.cu` demonstrates various ways of implementing parameter
  sweeps with enum types.
 2021-03-16 13:57:52 -04:00
Allison Vacanti
90f55a32ee Add missing format string. 
MSVC didn't catch this, but GCC won't compile it.
 2021-03-12 11:10:20 -05:00
Allison Vacanti
aa8141f9fb Encapsulate state usage in implementation. 2021-03-12 11:00:38 -05:00
Allison Vacanti
b6d6b238fc Add timeout to blocking_kernel. 
This adds a diagnostic when a KernelLauncher syncs but does not pass
`nvbench::exec_tag_sync`.
 2021-03-12 10:03:06 -05:00
Allison Vacanti
6f1cafeda5 [clang-format] Formatting. 2021-03-12 09:33:25 -05:00
Allison Vacanti
1e5f858d99 Merge pull request #1 from PointKernel/doc-update 
Minor corrections in doc
 2021-03-11 18:32:00 -05:00
Yunsong Wang
a097e6d90d Minor corrections in doc 2021-03-11 16:47:03 -05:00
Allison Vacanti
ca8e2d6391 Reset on correct stream in timer example. 2021-03-09 20:18:13 -05:00
Allison Vacanti
270ddcbbbf Fix link to timer example. 2021-03-09 20:07:58 -05:00
Allison Vacanti
f15b668b03 Add nvbench.test.all and nvbench.example.all metatargets. 2021-03-09 16:03:14 -05:00