NVIDIA/nvbench
1
0
Fork 0
mirror of https://github.com/NVIDIA/nvbench.git synced 2026-03-14 20:27:24 +00:00
Code Issues Packages Projects Releases Wiki Activity
571 Commits 10 Branches 4 Tags
pull-request/80
Commit Graph

113 Commits

Author SHA1 Message Date
Allison Piper
c8909c7d1b Refactoring / renaming. 2025-05-02 20:54:55 +00:00
Allison Piper
4defa02f51 Precommit reformatting. 2025-05-01 16:53:52 +00:00
Allison Piper
250d755bd6 Update new test to support device-init changes. 2025-05-01 16:40:09 +00:00
Allison Piper
edefcd0f6a Update fmt usage for new version. 2025-05-01 16:38:31 +00:00
Allison Piper
5016bec99e Merge remote-tracking branch 'origin/main' into fea/axes_iteration_space 2025-05-01 16:36:56 +00:00
Allison Piper
9d189280de Fix get_config_count for CPU-only benchmarks. (#218) 2025-05-01 12:34:35 -04:00
Allison Piper
50e764a308 Merge remote-tracking branch 'origin/main' into fea/axes_iteration_space 2025-05-01 15:14:58 +00:00
Sergey Pavlov
433376fd83 Restrict stopping criterion parameter usage in command line (#174) 
* restrict stopping criterion parameter usage in command line
* Update docs for stopping criterion.
* Add convenience benchmark_base API for criterion params.
* Add more test cases for stopping criterion parsing.

---------

Co-authored-by: Sergey Pavlov <psvvsp89@gmail.com>
Co-authored-by: Allison Piper <alliepiper16@gmail.com>
 2025-04-30 15:53:45 -04:00
Elias Stehle
ca0e795b46 Merge pull request #113 from elstehle/fix/per-device-stream 
Fixes cudaErrorInvalidValue when running on nvbench-created cuda stream
 2025-04-30 15:40:33 -04:00
Allison Piper
3440855dbd Formatting updates. 2025-04-14 17:26:12 +00:00
Allison Piper
93ea533fd3 Drop support for MSVC. 2025-04-04 22:17:03 +00:00
Allison Piper
4d7b3e8100 Add missing header to test. 2025-04-04 17:44:33 -04:00
Sergey Pavlov
a171514056 Added cudaGetLastError() calls to reset benchmarking kernel errors (issue 88). (#173) 
* Create and use NVBENCH_CUDA_CALL_RESET_ERROR.

* Moved cudaGetLastError() call to NVBENCH_CUDA_CALL macro

---------

Co-authored-by: Sergey Pavlov <psvvsp89@gmail.com>
 2024-05-31 11:32:01 -04:00
Allison Piper
555d628e9b Use a reproducible seed in test rng. (#164) 2024-04-12 11:55:05 -04:00
Allison Piper
5ee8811a1a Fix and test using RAII global state in main. (#168) 2024-04-09 17:27:49 -04:00
Allison Piper
165cf924c5 Refactor main implementation to improve reusability and customization. (#165) 
* Refactor main implementation to improve reusability and customization.

Move the implementation of `main` out of macros and into separate
functions. This allows for easier reuse and customization of the macros.
Existing macro usage should still work as expected, and new
customization points will simplify common tasks like argument parsing
going forward.

* Add tests that validate common main customizations.
 2024-04-09 12:45:58 -04:00
Allison Piper
a0f2fab72b Squashed commit of the following: 
commit c5b2fc0a8b
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Sat Apr 6 21:48:20 2024 +0000

    Add supported compilers and tools in README.md.

commit 92fe366da5
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Sat Apr 6 20:45:30 2024 +0000

    Fix issues discovered by header tests.

commit f7f6c92143
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Sat Apr 6 20:45:06 2024 +0000

    Setup header tests, add C++20 header tests + examples.

    The core library will always be built with C++17, but
    we test our headers / examples under 17 and 20.

commit 4b24f26b66
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Sat Apr 6 16:21:42 2024 +0000

    Pass CUDA FLAGS to install tests.

commit 4fb672ae91
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Sat Apr 6 15:43:41 2024 +0000

    Add newer GCC (13) and Clang (17, 18).
 2024-04-06 22:05:40 +00:00
Allison Piper
e8c8877d36 Squashed commit of the following: 
commit 4b309e6ad8
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Sat Apr 6 13:19:14 2024 +0000

    Minor cleanups

commit 476ed2ceae
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Sat Apr 6 12:53:37 2024 +0000

    WAR compiler ice in nlohmann json.

    Only seeing this on GCC 9 + CTK 11.1. Seems to be
    having trouble with the `[[no_unique_address]]` optimization.

commit a9bf1d3e42
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Sat Apr 6 00:24:47 2024 +0000

    Bump nlohmann json.

commit 80980fe373
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Sat Apr 6 00:22:07 2024 +0000

    Fix llvm filesystem support

commit f6099e6311
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 23:18:44 2024 +0000

    Drop MSVC 2017 testing.

commit 5ae50a8ef5
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 23:02:32 2024 +0000

    Add mroe missing headers.

commit b2a9ae04d9
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 22:37:56 2024 +0000

    Remove old CUDA+MSVC builds and make windows build-only.

commit 5b18c26a28
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 22:37:07 2024 +0000

    Fix header for std::min/max.

    Why do I always think it's utility instead of algorithm....

commit 6a409efa2d
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 22:18:18 2024 +0000

    Temporarily disable CUPTI on all windows builds.

commit f432f88866
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 21:42:52 2024 +0000

    Fix warnings on MSVC.

commit 829787649b
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 21:03:16 2024 +0000

    More flailing about in powershell.

commit 21742e6bea
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 20:36:08 2024 +0000

    Cleanup filesystem header handling.

commit de3d202635
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 20:09:00 2024 +0000

    Windows CI debugging.

commit a4151667ff
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 19:45:40 2024 +0000

    Quotation mark madness

commit dd04f3befe
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 19:27:27 2024 +0000

    Temporarily disable NVML on windows CI until new containers are ready.

commit f3952848c4
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 19:25:22 2024 +0000

    WAR issues on gcc-7.

commit 198986875e
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 19:25:04 2024 +0000

    More matrix/devcontainer updates.

commit b9712f8696
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 18:30:35 2024 +0000

    Fix windows build scripts.

commit 943f268280
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 18:18:33 2024 +0000

    Fix warnings with clang host compiler.

commit 7063e1d60a
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 18:14:28 2024 +0000

    More devcontainer hijinks.

commit 06532fde81
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 17:51:25 2024 +0000

    More matrix updates.

commit 78a265ea55
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 17:34:00 2024 +0000

    Support CLI CMake options for windows ci scripts.

commit 670895c867
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 17:31:59 2024 +0000

    Add missing devcontainers.

commit b121823e74
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 17:22:54 2024 +0000

    Build for `all-major` architectures in presets.

    We can get away with this because we require CMake 3.23.1.
    This was added in 3.23.

commit fccfd44685
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 17:22:08 2024 +0000

    Update matrix file.

commit e7d43ba90e
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 16:23:48 2024 +0000

    Consolidate build/test jobs.

commit c4044056ec
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 16:04:11 2024 +0000

    Add missing build script.
 2024-04-06 13:56:10 +00:00
Georgy Evtushenko
4be0c5bdcd API convention 2024-01-11 10:48:52 -08:00
Georgy Evtushenko
dacbee127c Base method naming convention 2024-01-11 10:41:11 -08:00
Georgy Evtushenko
182c77e4f4 Got rid of the params description API 2024-01-10 12:30:17 -08:00
Georgy Evtushenko
42c6bdea70 Handle empty input in mean 2024-01-10 09:52:14 -08:00
Georgy Evtushenko
fade52fa2e Different singleton convention 2024-01-08 14:08:12 -08:00
Georgy Evtushenko
85ed6f007c Rename criterion registry to criterion manager 2024-01-08 13:15:46 -08:00
Georgy Evtushenko
de724a21f1 Rename get_params to get_params_description 2024-01-08 13:06:48 -08:00
Georgy Evtushenko
b789240c76 Entropy-based stopping criterion 2024-01-05 14:59:48 -08:00
Vyas Ramasubramani
a3b729bca8 fmt::memory_buffer is no longer an iterator. 2022-11-03 10:04:02 -07:00
Robert Maynard
910b5cc759 Simplified user iterators 2022-09-01 16:18:08 -04:00
Robert Maynard
c2bfc99f12 remove need for output_indices 2022-09-01 14:34:06 -04:00
Robert Maynard
5708e6cd92 remove need for make_space_iterator 2022-08-31 11:08:44 -04:00
Robert Maynard
dc7e2b789d Drop ability to zip axii after construction 2022-08-29 10:24:45 -04:00
Robert Maynard
ba8356f811 Refactor names 2022-04-12 12:35:09 -04:00
Robert Maynard
26467f3855 More cleanup 2022-04-12 10:54:53 -04:00
Robert Maynard
c3c86e1f26 implement easier API to add axis and zip/user iteration at the same time 2022-02-28 16:44:08 -05:00
Robert Maynard
a25f578891 Rename tie_axes to zip_axes 2022-02-28 14:23:35 -05:00
Robert Maynard
344878e9dc Allow users to control iteration via the concept of iteration spaces. 
Changes in the work include:
- [x] Internally use linear_space for iterating
- [x] Simplify type and value iteration in `state_iterator::build_axis_configs`
- [x] Store the iteration space in `axes_metadata`
- [x] Expose `tie` and `user` spaces to user
- [x] Add tests for `linear`, `tie`, and `user`
- [x] Add examples for `tie` and `user`
 2022-02-25 15:09:51 -05:00
Yunsong Wang
af4c35d78b Fix a bug in config count unit test: count number of devices as well 2022-02-11 18:24:58 -05:00
Yunsong Wang
6159d9c6cb Minor correction in unit test 2022-02-06 20:19:21 -05:00
Yunsong Wang
33a896f99e Update copyright year 2022-02-04 17:25:50 -05:00
Yunsong Wang
470beda9f0 Add nvbench::state stream tests 2022-02-04 16:55:29 -05:00
Allison Vacanti
a72f248af6 Require the NVBench package in test_export testing. 2022-01-19 15:42:26 -05:00
Allison Vacanti
6dee1eec3b Refactor summary API and update nvbench/summary.cuh docs. 
The string used when constructing a summary is no longer a human
readable name, but rather a tag string (e.g. "nv/cold/time/gpu/mean").
These will make lookup easier and more stable going forward.

name vs. short_name no longer exists. Now there is just "name", which
is used for column headings. The "description" string may still be
used for detailed information.

Updated the json tests and compare script to reflect these changes.
 2022-01-11 15:06:26 -05:00
Allison Vacanti
2f8bb28c52 Merge pull request #64 from allisonvacanti/noise_convergence 
New convergence check
 2021-12-21 21:30:39 -05:00
Allison Vacanti
178dd0eb68 Implement new convergence check for noisy kernels. 
Previously, convergence was tested by waiting for the relative stdev
of cuda timings ("noise") to drop below a certain percentage
(`max_noise`).

This assumed that all benchmarks would eventually see their noise drop
to some threshold, but this is not the case. In practice, many benchmarks
never converge to the default 0.5% relative stdev and instead will always
run to the 15s timeout -- even if the means have converged in a second
or two.

Added a new check that tests when the noise itself stabilizes and ends
the benchmark, even if noise > max_noise.

After testing, this patch alone significantly reduces the runtime of the
Thrust+CUB benchmark suite (from 30 hours to 5 hours) and produces similar
timing results.

The parameters used to tune this feature are not exposed -- if this
approach works long-term and there's a strong motivation to let users
tweak them, then we can worry about names/APIs/CLI/docs later.
 2021-12-21 21:24:02 -05:00
Allison Vacanti
8e56a7bd94 Add noisy_bench with some benchmarks that currently always time-out. 2021-12-21 21:05:13 -05:00
Allison Vacanti
c9ab8e2eb3 Fix progress display for inactive type axis values. 
When type axis values were disabled they were still counted towards
a benchmark's total number of configs.
 2021-12-21 20:36:52 -05:00
Allison Vacanti
20522c807d Add an nvbench-ctl executable. 
This will provide functionality such as clock locking (--lgm),
persistance mode (--pm), device querying (--list), version checking
(--version), and documentation (--help).

This is possible already with any nvbench executable, but having
one with a reliable name will be helpful for scripting and writing
documentation.
 2021-12-21 12:02:07 -05:00
Allison Vacanti
5d70492714 Enable more warning flags. 
- /W4 on MSVC
- -Wall -Wextra + others on gcc/clang
- New NVBench_ENABLE_WERROR option to toggle "warnings as errors"
- Mark the nlohmann_json library as IMPORTED to switch to system includes
- Rename nvbench_main -> nvbench.main to follow target name conventions
- Explicitly suppress some cudafe warnings when compiling templates in
  nlohmann_json headers.
- Explicitly suppress some warnings from Thrust headers.
- Various fixes for warnings exposed by new flags.
- Disable CUPTI on CTK < 11.3 (See #52).
 2021-12-18 20:13:25 -05:00
Georgy Evtushenko
1bc715267c CUPTI support 2021-12-18 12:03:52 +03:00
Allison Vacanti
b2d37c21fd Add export tests. 2021-10-20 14:02:16 -04:00