Commit Graph

8 Commits

Author SHA1 Message Date
Oleksandr Pavlyk
d63a2761eb Implement Timer, and support State.exec(fn, timer=True) (#364)
* Add type annotations for future functionality

```python
class Timer:
    def start(self) -> None: ...
    def stop(self) -> None: ...
```

and overloaded `State.exec` so:

  - normal mode accepts `Callable[[Launch], None]`
  - `timer=True` accepts `Callable[[Launch, Timer], None]`

No implementation yet. Type annotation checked with

```
(py313) :~/repos/nvbench/python$ python -m mypy --ignore-missing-imports /tmp/check_timer.py
/tmp/check_timer.py:24: error: No overload variant of "exec" of "State" matches argument types "Callable[[Launch], None]", "bool"  [call-overload]
/tmp/check_timer.py:24: note: Possible overload variants:
/tmp/check_timer.py:24: note:     def exec(self, Callable[[Launch], None], /, *, batched: bool | None = ..., sync: bool | None = ..., timer: Literal[False] = ...) -> None
/tmp/check_timer.py:24: note:     def exec(self, Callable[[Launch, Timer], None], /, *, timer: Literal[True], sync: bool | None = ...) -> None
/tmp/check_timer.py:25: error: Argument 1 to "exec" of "State" has incompatible type "Callable[[Launch, Timer], None]"; expected "Callable[[Launch], None]"  [arg-type]
/tmp/check_timer.py:26: error: No overload variant of "exec" of "State" matches argument types "Callable[[Launch, int], None]", "bool"  [call-overload]
/tmp/check_timer.py:26: note: Possible overload variants:
/tmp/check_timer.py:26: note:     def exec(self, Callable[[Launch], None], /, *, batched: bool | None = ..., sync: bool | None = ..., timer: Literal[False] = ...) -> None
/tmp/check_timer.py:26: note:     def exec(self, Callable[[Launch, Timer], None], /, *, timer: Literal[True], sync: bool | None = ...) -> None
Found 3 errors in 1 file (checked 1 source file)

(py313) :~/repos/nvbench/python$ nl -ba /tmp/check_timer.py
     1  # /tmp/check_nvbench_timer.py
     2  import cuda.bench as bench
     3
     4  def normal_ok(launch: bench.Launch) -> None:
     5      pass
     6
     7  def timer_ok(launch: bench.Launch, timer: bench.Timer) -> None:
     8      timer.start()
     9      timer.stop()
    10
    11  def missing_timer(launch: bench.Launch) -> None:
    12      pass
    13
    14  def extra_timer(launch: bench.Launch, timer: bench.Timer) -> None:
    15      pass
    16
    17  def wrong_timer_type(launch: bench.Launch, timer: int) -> None:
    18      pass
    19
    20  def state_bench(state: bench.State) -> None:
    21      state.exec(normal_ok)
    22      state.exec(normal_ok, timer=False)
    23      state.exec(timer_ok, timer=True)
    24      state.exec(missing_timer, timer=True)       # should fail
    25      state.exec(extra_timer)                     # should fail
    26      state.exec(wrong_timer_type, timer=True)    # should fail
```

* Implement cuda.bench.Timer object

The Timer class is not user-constructible. It exposes two nullary
methods timer.start() and timer.stop().

The instance of Timer class would be provided to launchable object
passed to State.exec with timer=True.

* Implement support for State.exec( launch_fn, timer=True)

* Change type annotation for batch to default to None

None is interpreted as `not timer`, i.e., it effectively
defaults to True (as before) for usage without timer set,
but starts defaulting to `False` is `timer=True` is set.

The batched keyword type is `bool | None`.

* Implement default batched=None behavior

API allows one to specify all 3 keywords, sync, batched,
and timer. batched is None by default, run-time interpreted
as `(not timer)`.

* Update tests for new behavior of batched/time combination

* Add python/examples/exec_tag_timer.py

* Expand Timer class and methods docstrings

* Reworked python/example/exec_tag_timer.py to align with C++ example.

* Replace ::cuda::std::name with cuda::std::name

* Resolve review feedback
2026-05-15 10:19:40 -05:00
Oleksandr Pavlyk
44ec7de6bd Implement decorators to register benchmarks add axis and options (#347)
* Add decorators for registering benchmarks and adding axis

cuda.bench.register(fn) continues returning Benchmark, and supports
legacy use.

New signature added:
   cuda.bench.register():
      Returns a decorator

```
@bench.register()
@bench.axis.float64("Duration (s)", [7e-5, 1e-4, 5e-4])
@bench.option.min_samples(120)
def single_float64_axis(state: bench.State):
   ...
```

* Remove example/auto_throughput.py

The C++ counterpart's purpose is to demonstrate use of CUPTI
metrics, but these are not supported in Python bindings, so
this example is a duplicate of example/throughput.py

* Add wrong decorator order test for bench.axis.*

* Strengthen type annotation for register function

Acting on code rabbit nit-pick require that function being
registered take cuda.bench.State object as an argument.

Verified the fix as

```
(py313) :~/repos/nvbench/python$ python -m mypy --ignore-missing-import /tmp/t.py
/tmp/t.py:8: error: Argument 1 has incompatible type "Callable[[], None]"; expected "Callable[[State], None]"  [arg-type]
Found 1 error in 1 file (checked 1 source file)
(py313) :~/repos/nvbench/python$ nl -ba /tmp/t.py
     1  # /tmp/check_nvbench_register.py
     2  import cuda.bench as bench
     3
     4  @bench.register()
     5  def good(state: bench.State) -> None:
     6      pass
     7
     8  @bench.register()
     9  def bad() -> None:
    10      pass
```

* Replace use of global variable with thread-safe lru_cache

This improves thread-safety of module initialization.

* Abide by RUF005 linting rule

* Expand docstrings regarding cuda.bench.register() decorator

It explains to the user what the decorator does and provides
a concise usage example.

* Sharpen wording on exception maybe-thrown by decorator
2026-05-14 15:41:30 -05:00
Oleksandr Pavlyk
7c60edcc0a cuda.core.experimental -> cuda.core 2026-04-01 08:16:04 -05:00
Oleksandr Pavlyk
b5e4b4ba31 cuda.nvbench -> cuda.bench
Per PR review suggestion:
   - `cuda.parallel`    - device-wide algorithms/Thrust
   - `cuda.cooperative` - Cooperative algorithsm/CUB
   - `cuda.bench`       - Benchmarking/NVBench
2025-08-04 13:42:43 -05:00
Oleksandr Pavlyk
a69a3647b2 CUTLASS example added, license headers added, fixes
- Add license header to each example file.
- Fixed broken runs caused by type declarations.
- Fixed hang in throughput.py when --run-once by doing a
  manual warm-up step, like in auto_throughput.py
2025-07-28 15:37:05 -05:00
Oleksandr Pavlyk
e589518376 Change test and examples from using camelCase to using snake_case as implementation changed 2025-07-28 15:37:05 -05:00
Oleksandr Pavlyk
883e5819b6 Use cuda.Stream.from_handle to create core.Stream from nvbench.CudaStream 2025-07-28 15:37:04 -05:00
Oleksandr Pavlyk
b357af0092 Add examples/skip.py 2025-07-28 15:37:04 -05:00