Commit Graph

630 Commits

Author SHA1 Message Date
Oleksandr Pavlyk
c2a2acc9b6 Change float64_t arg-type for set_throttle_threshold to float32_t
The C++ method signature of set_throttle_threshold/set_trottle_recovery_delay,
which uses nvbench::float32_t
2025-08-04 12:14:52 -05:00
Oleksandr Pavlyk
584f48ac97 Remove warm-up invocations outside of launcher in examples/throughout and auto_throughput 2025-08-04 12:14:44 -05:00
Oleksandr Pavlyk
d8b0acc8d4 Export exception to nvbench namespace 2025-08-04 12:00:42 -05:00
Oleksandr Pavlyk
9dfdd8af89 Minimal test file 2025-08-04 11:59:17 -05:00
Oleksandr Pavlyk
6aff4712f8 Change permissions of test/run_1.py 2025-08-04 10:13:08 -05:00
Oleksandr Pavlyk
73e18419b2 Stub of __cuda_stream__ special method declare tuple[int, int] as return type
This is to indicate that special method always returns a pair of integers
2025-08-04 10:11:33 -05:00
Oleksandr Pavlyk
a5e0a48f80 Add test test functions for cpp/python exceptions 2025-08-04 10:09:10 -05:00
Oleksandr Pavlyk
40a2337a6b Review fix: make nvbenhch_run_error constructable
Allow `throw nvbench_run_error("Msg");` to compile.

Add comment around definition of nvbench_run_error
2025-08-04 10:09:04 -05:00
Oleksandr Pavlyk
4fc628c4d7 Python native extension to use CXX/CUDA standard of NVBench library
This fixes cryptic build failure with GNU compiler 14
2025-08-01 15:33:39 -05:00
Oleksandr Pavlyk
3fea652d16 Fix type in stub declaration for Benchmark.add_string_axis 2025-08-01 15:03:06 -05:00
Oleksandr Pavlyk
f1fbfd85b4 Renamed src/README.md to src/.BUILD_LOCALLY.md
Provided more context to the command stated in the readme, and
changed so as to not hard-code installation paths of NVBench,
and checkout path of pybind11.
2025-07-31 16:27:54 -05:00
Oleksandr Pavlyk
453a1648aa Improvements to readability of examples per PR review 2025-07-31 16:20:52 -05:00
Oleksandr Pavlyk
c91204f259 Improved docstrings per PR review suggestions 2025-07-31 15:48:49 -05:00
Oleksandr Pavlyk
fb23591aef Fixed missing space in README 2025-07-31 15:42:30 -05:00
Oleksandr Pavlyk
add539a0c1 Replaced argument type annotation: int -> typing.SupportsInt
Same for float->typing.SupportsFloat. Result types remain int/float
2025-07-30 16:54:56 -05:00
Oleksandr Pavlyk
88a3ad0138 Add test/stub.py
The following static analysis run should run green

```
mypy --ignore-missing-imports test/stub.py
```
2025-07-30 13:54:37 -05:00
Oleksandr Pavlyk
9c01f229a6 Add Benchmark set methods, such as set_stopping_criterion, set_timeout, etc
Add
   - State.get_stopping_criterion() -> str
   - Benchmark.set_stopping_criterion(criterion: str) -> Self
   - Benchmark.set_criterion_param_int64(name: str, value: int) -> Self
   - Benchmark.set_criterion_param_float64(name: str, value: float) -> Self
   - Benchmark.set_criterion_param_string(name: str, value: str) -> Self
   - Benchmark.set_timeout(duration: float) -> Self
   - Benchmark.set_skip_time(skip_time: float) -> Self
   - Benchmark.set_throttle_threshold(frac: float) -> Self
   - Benchmark.set_throttle_recovery_delay(duration: float) -> Self
   - Benchmark.set_min_samples(count: int) -> Self
2025-07-30 13:37:17 -05:00
Oleksandr Pavlyk
6b9050e404 Add example of benchmarking pytorch code 2025-07-28 15:57:11 -05:00
Oleksandr Pavlyk
413c4a114b Support nvbench.State.set_throttle_threshold 2025-07-28 15:37:05 -05:00
Oleksandr Pavlyk
afb9951ed8 Enable building of NVBench as part of buildign extension
1. Download and include CPM.cmake, version 0.42.0
2. Use CPM.make to get Pybind11
3. Update to use pybind11=3.0.0
4. Also use CPM to configure/build nvbench
2025-07-28 15:37:05 -05:00
Oleksandr Pavlyk
b6821b7624 Rename NVBenchRuntimeException to NVBenchRuntimeError
Added exception to __init__.pyi
2025-07-28 15:37:05 -05:00
Oleksandr Pavlyk
b97e27cbf2 Add use of add_axis_values and add_axis_values_as_string to test/run_1.py 2025-07-28 15:37:05 -05:00
Oleksandr Pavlyk
eb614ac52f Add State.get_axis_values and State.get_axis_values_as_string
Add nvbench.State methods to get Python dictionary representing
axis values of benchmark configuration state represents.

get_axis_values_as_string gives a string of space-separated
name=values pairs.
2025-07-28 15:37:05 -05:00
Oleksandr Pavlyk
5613281c2e nvbench.State.exec validates arg to be a callable
Add names to method arguments to make it more self-descriptive.
2025-07-28 15:37:05 -05:00
Oleksandr Pavlyk
c747a19b98 Remove code setting up CUDA_MODULE_LOADING=EAGER in Python extension 2025-07-28 15:37:05 -05:00
Oleksandr Pavlyk
985db4f144 Add examples/cccl_cooperative_block_reduce.py 2025-07-28 15:37:05 -05:00
Oleksandr Pavlyk
445d881eda Expand README
Make it explicit in README that we build and locally install NVBench first,
and then build Python package use the library as a dependency.

The nvbench library is installed into Python layout alongside the native
extension.
2025-07-28 15:37:05 -05:00
Oleksandr Pavlyk
5e8c17c740 Fix mypy error in import statement used in cutlass_gemm example 2025-07-28 15:37:05 -05:00
Oleksandr Pavlyk
b9554d7980 Fix method name typo in stub file discovered by mypy 2025-07-28 15:37:05 -05:00
Oleksandr Pavlyk
5c01c34793 Fix mypy error in cutlass_gemm example 2025-07-28 15:37:05 -05:00
Oleksandr Pavlyk
5428534124 Add license header to __init__.py 2025-07-28 15:37:05 -05:00
Oleksandr Pavlyk
480614e847 Add license to stub fuile, add comment about syncing impl and stubs
Add comments stating the need to keep implementation and Python stub
file in sync to both files. In the stub file to comment documents
use of mypy's stubgen to generate stubs and calls to compare that against
current stubs. It also calls out the need to keep docstrings and
doctring examples in sync with implementation.
2025-07-28 15:37:05 -05:00
Oleksandr Pavlyk
a69a3647b2 CUTLASS example added, license headers added, fixes
- Add license header to each example file.
- Fixed broken runs caused by type declarations.
- Fixed hang in throughput.py when --run-once by doing a
  manual warm-up step, like in auto_throughput.py
2025-07-28 15:37:05 -05:00
Oleksandr Pavlyk
c136efab65 Use absolute imports in cuda/nvbench/__init__.py 2025-07-28 15:37:05 -05:00
Oleksandr Pavlyk
fc0249d188 Updated examples/axes.py to use get_float64_or_default 2025-07-28 15:37:05 -05:00
Oleksandr Pavlyk
361c0337be Use cuda-pathfinder instead of cuda-bindings for Pathfinder
Removed use of __all__ per PR feedback. Emit warnings.warn if
version information could not be retrieved from the package metadata,
e.g., package has been renamed by source code was not updated.
2025-07-28 15:37:05 -05:00
Oleksandr Pavlyk
51fa07fab8 Avoid overloading get_int64_or_default as get_int64
Introduce get_int64_or_default method, and counterparts for
float64 and string.

Provided names for Python arguments.

Tried generating Python stubs automatically with

```
stubgen -m cuda.nvbench._nvbench
```

Gave up on this, since it does not include doc-strings.
It would be nice to compare auto-generated _nvbench.pyi with
__init__.pyi for discrepancies though.
2025-07-28 15:37:05 -05:00
Oleksandr Pavlyk
dc7f9edfd4 Support nvbench.Benchmark.add_int64_power_of_two_axis 2025-07-28 15:37:05 -05:00
Oleksandr Pavlyk
526856db4e Fix typo in the method spelling 2025-07-28 15:37:05 -05:00
Oleksandr Pavlyk
893cefb400 Fix the need to set PYTHONPATH, edited README
Edit wheel.packages metadata to include namespace package "cuda".
Updated README to remove the work-around of setting PYTHONPATH,
as it is no longer necessary.
2025-07-28 15:37:05 -05:00
Oleksandr Pavlyk
a535a1d173 Fix type annotations in cuda.nvbench, and in examples 2025-07-28 15:37:05 -05:00
Oleksandr Pavlyk
13ad115ca3 Add nvbench.Benchmark.set_run_once method 2025-07-28 15:37:05 -05:00
Oleksandr Pavlyk
bd2b536ab4 cpu_only -> cpu_activity
Change example to illustrate timing CPU work.

First example does only CPU work (sleeps), use CPU-only timer.
Second examples does both CPU and GPU work (sleeps in either case).
Use cold-run timer with/without sync tag to measure both CPU and GPU times.
2025-07-28 15:37:05 -05:00
Oleksandr Pavlyk
d09df0f754 Expand examples/cpu_only.py
Benchmark function that sleeps for 1 seconda on the host using CPU-only
timer, as well as CPU/GPU timer that does/doesn't use blocking kernel.

All three methods must report consistent values close to 1 second.
2025-07-28 15:37:05 -05:00
Oleksandr Pavlyk
e426368485 Correct propagating nvbench_main exceptions to Python
python examples/cpu_only.py --run-once -d 0 --output foo.md

used to trip SystemError, returned a result with an exception set.

It now returns a clean NVBenchmarkError exception.
2025-07-28 15:37:05 -05:00
Oleksandr Pavlyk
9ab642cf69 Add suggestion to create conda environment with recent CMake to build nvbench 2025-07-28 15:37:05 -05:00
Oleksandr Pavlyk
e589518376 Change test and examples from using camelCase to using snake_case as implementation changed 2025-07-28 15:37:05 -05:00
Oleksandr Pavlyk
81fff085b9 Change method nameing from camelCase to snake_case
This ensures names of Python API methods are consistent with those of C++
counterparts.
2025-07-28 15:37:05 -05:00
Oleksandr Pavlyk
11ae98389d Replace use of py::object copy constructor with use of move constructor
Change explicit constructor of benchmark_wrapper_t to use move-constructor
of py::object instead of copy constructor by replacing `py::object(o)` with
`py::object(std::move(o))`.
2025-07-28 15:37:05 -05:00
Oleksandr Pavlyk
d3071fb038 Addressed PR feedback re: definition of benchmark_wrapper_t
See https://github.com/NVIDIA/nvbench/pull/237#discussion_r2183749750
2025-07-28 15:37:05 -05:00