NVIDIA/nvbench
1
0
Fork 0
mirror of https://github.com/NVIDIA/nvbench.git synced 2026-03-14 20:27:24 +00:00
Code Issues Packages Projects Releases Wiki Activity
480 Commits 10 Branches 4 Tags
a6df59a9b5fcdd2e19e61b3ab3e2ba5f436f221d
Commit Graph

480 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Allison Piper
a6df59a9b5 Add support for CPU-only benchmarking. 
Fixes #95.

CPU-only mode is enabled by setting the `is_cpu_only` property while
defining a benchmark, e.g. `NVBENCH_BENCH(foo).set_is_cpu_only(true)`.

An optional `nvbench::exec_tag::no_gpu` hint can also be passed to
`state.exec` to avoid instantiating GPU benchmarking backends. Note that
a CUDA compiler and CUDA runtime are always required, even if all benchmarks
in a translation unit are CPU-only.

Similarly, a new `nvbench::exec_tag::gpu` hint can be used to avoid
compiling CPU-only backends for GPU benchmarks.
 2025-04-08 11:17:23 -04:00
Allison Piper
1efed5f8e1 Merge pull request #200 from alliepiper/update_deps 
Update dependencies, drop support for old compilers and MSVC.
 2025-04-04 18:45:59 -04:00
Allison Piper
93ea533fd3 Drop support for MSVC. 2025-04-04 22:17:03 +00:00
Allison Piper
1d0daa52ae Add skip-vdc option to CI. 2025-04-04 17:44:33 -04:00
Allison Piper
7d210614f5 Attempt to suppress system include warnings on MSVC. 2025-04-04 17:44:33 -04:00
Allison Piper
2a25b351ab Bump required cmake version to 3.30.4 for rapids-cmake. 2025-04-04 17:44:33 -04:00
Allison Piper
a3fb3ce610 Migrate CI to l4 queue. 2025-04-04 17:44:33 -04:00
Allison Piper
15d34106d4 Disable unicode in fmtlib on nvcc + msvc. 
This doesn't appear to be supported.
 2025-04-04 17:44:33 -04:00
Allison Piper
435df5220f Regenerate devcontainers. 2025-04-04 17:44:33 -04:00
Allison Piper
1a5fa2277e Drop support for CTK < 11.8 and clang < 14. 
Newer versions of fmt have a ton of issues building on CTK 11.1, and 11.8 is the next available container we have built for CI. We may still work with some earlier versions, but we do not test them.

We no longer have CI images available for clang < 14, so drop official support.
 2025-04-04 17:44:33 -04:00
Allison Piper
8478f7d0bf Guard fmt def behind nvcc check. 2025-04-04 17:44:33 -04:00
Allison Piper
9d9a30fbd6 Bump devcontainers to 25.06 branch. 2025-04-04 17:44:33 -04:00
Allison Piper
5f6f8a65ee Enable /utf-8 on MSVC. 2025-04-04 17:44:33 -04:00
Allison Piper
a1acb3e8b2 Update CI matrix and devcontainers. 2025-04-04 17:44:33 -04:00
Allison Piper
4d7b3e8100 Add missing header to test. 2025-04-04 17:44:33 -04:00
Allison Piper
0e8089a246 Disable fmtlib's use of llvm _BitInt, as it is not supported when using nvcc. 2025-04-04 17:44:33 -04:00
Allison Piper
e6705e3114 Update fmtlib/fmt to 11.1.4. 
Switched away from the rapids-cmake provided version and manually CPM'd it.
rapids-cmake will stop providing fmtlib later this year, and the version currently supported is rather old.
Included the same logic that rapids-cmake currently uses to hopefully provide a smooth transition for edge cases (external fmt, etc).

Added `FMT_SYSTEM_HEADERS=ON` to mark fmt headers as system includes, suppressing any internal warnings.
 2025-04-04 17:44:33 -04:00
Allison Piper
5aa5a3c225 Update rapids-cmake to 25.04. 2025-04-04 17:44:33 -04:00
Allison Piper
2d9eafc765 Merge pull request #202 from alliepiper/misc-fixes 
Misc fixes
pre_msvc_drop 		2025-04-04 16:47:25 -04:00
Allison Piper
497eaed1d9 Use correct timer when computing cpu stats in measure_cold. 2025-04-04 20:16:04 +00:00
Allison Piper
618e1f048c Fix typo in docstring. 2025-04-04 20:14:44 +00:00
Allison Piper
f6af8b9769 Whitespace cleanup. 2025-04-04 20:14:21 +00:00
Allison Piper
c03033b50e Fix error message when #TypeAxisNames != #TypeAxes. (#192) 
The error message was being generated after moving strings out of `names`, so some of the axis names were blank.
This moves the check + error before any strings are moved.
 2024-11-20 13:11:03 -05:00
Bernhard Manfred Gruber
f52aa4b0aa Distinguish slower, same and faster comparisons (#190) 
Fixes: #178
 2024-11-15 12:41:47 -05:00
Georgii Evtushenko
0ce45af043 Plot comparison results (#90) 2024-11-13 14:28:04 -05:00
Jordan Jacobelli
92286e1d4a devcontainer: replace VAULT_HOST with AWS_ROLE_ARN (#187) 
* devcontainer: replace VAULT_HOST with AWS_ROLE_ARN

Signed-off-by: Jordan Jacobelli <jjacobelli@nvidia.com>

* Update devcontainers base image to support AWS_ROLE_ARN

Signed-off-by: Jordan Jacobelli <jjacobelli@nvidia.com>

* Bump cuda latest version to 12.6

Signed-off-by: Jordan Jacobelli <jjacobelli@nvidia.com>

* Replace ubuntu18.04 with ubuntu20.04

Ubuntu 18.04 is not supported anymore

Signed-off-by: Jordan Jacobelli <jjacobelli@nvidia.com>

* Use DOOD stategy to keep supporting ubuntu18.04

See https://github.com/NVIDIA/cccl/pull/1779

Signed-off-by: Jordan Jacobelli <jjacobelli@nvidia.com>

---------

Signed-off-by: Jordan Jacobelli <jjacobelli@nvidia.com>
 2024-10-25 11:49:02 -04:00
Sergey Pavlov
a171514056 Added cudaGetLastError() calls to reset benchmarking kernel errors (issue 88). (#173) 
* Create and use NVBENCH_CUDA_CALL_RESET_ERROR.

* Moved cudaGetLastError() call to NVBENCH_CUDA_CALL macro

---------

Co-authored-by: Sergey Pavlov <psvvsp89@gmail.com>
 2024-05-31 11:32:01 -04:00
Cliff Burdick
088c9ee658 Removed extraneous break statements that caused nvc++ to give warnings (#171) 2024-05-17 16:35:11 -04:00
Allison Piper
961fa91258 Don't fail CI when ninja_summary fails. (#172) 2024-05-17 14:35:12 -04:00
Allison Piper
555d628e9b Use a reproducible seed in test rng. (#164) 2024-04-12 11:55:05 -04:00
Allison Piper
60761e0946 Enable extra NVBench features in windows build. (#169) 
* Enable extra NVBench features in windows build.

These were delayed as they required changes to the devcontainers.

* Revamp nvml.dll logic.
 2024-04-10 13:45:53 -04:00
Allison Piper
5ee8811a1a Fix and test using RAII global state in main. (#168) 2024-04-09 17:27:49 -04:00
Allison Piper
165cf924c5 Refactor main implementation to improve reusability and customization. (#165) 
* Refactor main implementation to improve reusability and customization.

Move the implementation of `main` out of macros and into separate
functions. This allows for easier reuse and customization of the macros.
Existing macro usage should still work as expected, and new
customization points will simplify common tasks like argument parsing
going forward.

* Add tests that validate common main customizations.
 2024-04-09 12:45:58 -04:00
Allison Piper
9e8efa2c88 Preserve .devcontainers/img/ when cleaning. (#166) 2024-04-08 18:24:07 -04:00
Allison Piper
044e22411e Move CUPTI disablement to build script. (#167) 2024-04-08 18:23:38 -04:00
Allison Piper
a0f2fab72b Squashed commit of the following: 
commit c5b2fc0a8b
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Sat Apr 6 21:48:20 2024 +0000

    Add supported compilers and tools in README.md.

commit 92fe366da5
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Sat Apr 6 20:45:30 2024 +0000

    Fix issues discovered by header tests.

commit f7f6c92143
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Sat Apr 6 20:45:06 2024 +0000

    Setup header tests, add C++20 header tests + examples.

    The core library will always be built with C++17, but
    we test our headers / examples under 17 and 20.

commit 4b24f26b66
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Sat Apr 6 16:21:42 2024 +0000

    Pass CUDA FLAGS to install tests.

commit 4fb672ae91
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Sat Apr 6 15:43:41 2024 +0000

    Add newer GCC (13) and Clang (17, 18).
 2024-04-06 22:05:40 +00:00
Allison Piper
a2f88ff790 Set CUDA_MODULE_LOADING=EAGER before main. (#157) 
* Set `CUDA_MODULE_LOADING=EAGER` before `main`.

Fixes #136

* Portability for `setenv`.

* Remove pre-main CUDART usage and setup env in main.

* Fail examples if they deadlock.

This is the best way we have to diagnose a regression for
NVIDIA/nvbench#136.

* Add an initialize method to benchmark_manager for CUDA-related setup.

Benchmarks are created statically, so their constructors cannot call the CUDA APIs without breaking the CUDA_MODULE_LOAD setup.

This method is called from `main` after the environment has been configured.
 2024-04-06 11:03:42 -04:00
Allison Piper
e8c8877d36 Squashed commit of the following: 
commit 4b309e6ad8
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Sat Apr 6 13:19:14 2024 +0000

    Minor cleanups

commit 476ed2ceae
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Sat Apr 6 12:53:37 2024 +0000

    WAR compiler ice in nlohmann json.

    Only seeing this on GCC 9 + CTK 11.1. Seems to be
    having trouble with the `[[no_unique_address]]` optimization.

commit a9bf1d3e42
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Sat Apr 6 00:24:47 2024 +0000

    Bump nlohmann json.

commit 80980fe373
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Sat Apr 6 00:22:07 2024 +0000

    Fix llvm filesystem support

commit f6099e6311
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 23:18:44 2024 +0000

    Drop MSVC 2017 testing.

commit 5ae50a8ef5
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 23:02:32 2024 +0000

    Add mroe missing headers.

commit b2a9ae04d9
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 22:37:56 2024 +0000

    Remove old CUDA+MSVC builds and make windows build-only.

commit 5b18c26a28
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 22:37:07 2024 +0000

    Fix header for std::min/max.

    Why do I always think it's utility instead of algorithm....

commit 6a409efa2d
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 22:18:18 2024 +0000

    Temporarily disable CUPTI on all windows builds.

commit f432f88866
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 21:42:52 2024 +0000

    Fix warnings on MSVC.

commit 829787649b
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 21:03:16 2024 +0000

    More flailing about in powershell.

commit 21742e6bea
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 20:36:08 2024 +0000

    Cleanup filesystem header handling.

commit de3d202635
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 20:09:00 2024 +0000

    Windows CI debugging.

commit a4151667ff
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 19:45:40 2024 +0000

    Quotation mark madness

commit dd04f3befe
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 19:27:27 2024 +0000

    Temporarily disable NVML on windows CI until new containers are ready.

commit f3952848c4
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 19:25:22 2024 +0000

    WAR issues on gcc-7.

commit 198986875e
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 19:25:04 2024 +0000

    More matrix/devcontainer updates.

commit b9712f8696
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 18:30:35 2024 +0000

    Fix windows build scripts.

commit 943f268280
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 18:18:33 2024 +0000

    Fix warnings with clang host compiler.

commit 7063e1d60a
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 18:14:28 2024 +0000

    More devcontainer hijinks.

commit 06532fde81
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 17:51:25 2024 +0000

    More matrix updates.

commit 78a265ea55
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 17:34:00 2024 +0000

    Support CLI CMake options for windows ci scripts.

commit 670895c867
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 17:31:59 2024 +0000

    Add missing devcontainers.

commit b121823e74
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 17:22:54 2024 +0000

    Build for `all-major` architectures in presets.

    We can get away with this because we require CMake 3.23.1.
    This was added in 3.23.

commit fccfd44685
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 17:22:08 2024 +0000

    Update matrix file.

commit e7d43ba90e
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 16:23:48 2024 +0000

    Consolidate build/test jobs.

commit c4044056ec
Author: Allison Piper <alliepiper16@gmail.com>
Date:   Fri Apr 5 16:04:11 2024 +0000

    Add missing build script.
 2024-04-06 13:56:10 +00:00
Allison Piper
04b70059b8 Setup clangd compile commands output. 2024-04-05 15:08:04 +00:00
Allison Piper
eb5940c64f Add initial CI, presets, and devcontainers. 2024-04-04 21:42:43 +00:00
Georgii Evtushenko
42d99a5753 Fix MSVC (#158) 
* Fix MSVC

* Revert change of source list name
 2024-04-04 09:53:30 -07:00
Georgii Evtushenko
d8dced8a64 Merge pull request #151 from gevtushenko/entropy 
Entropy-based stopping criterion
 2024-01-12 23:19:25 +04:00
Georgy Evtushenko
aebbeb371b Check if criterion options start with -- 2024-01-11 11:24:59 -08:00
Georgy Evtushenko
4be0c5bdcd API convention 2024-01-11 10:48:52 -08:00
Georgy Evtushenko
34c57965a2 Move params 2024-01-11 10:42:43 -08:00
Georgy Evtushenko
dacbee127c Base method naming convention 2024-01-11 10:41:11 -08:00
Georgy Evtushenko
182c77e4f4 Got rid of the params description API 2024-01-10 12:30:17 -08:00
Georgy Evtushenko
bcd5c7d885 Remove extra member variable 2024-01-10 10:15:00 -08:00
Georgy Evtushenko
b59c251e3b Making stopping criterion constructor explicit 2024-01-10 10:13:03 -08:00
Georgy Evtushenko
6b5315fc8f Readable names 2024-01-10 10:12:19 -08:00