* Limit busy-wait loops in per-subinterpreter GIL test
Add explicit timeouts to the busy-wait coordination loops in the
Per-Subinterpreter GIL test in tests/test_with_catch/test_subinterpreter.cpp.
Previously those loops spun indefinitely waiting for shared atomics like
`started` and `sync` to change, which is fine when CPython's free-threading
and per-interpreter GIL behavior matches the test's expectations but becomes
pathologically bad when that behavior regresses: the `test_with_catch`
executable can then hang forever, causing our 3.14t CI jobs to time out
after 90 minutes.
This change keeps the structure and intent of the test but adds a
std::chrono::steady_clock deadline to each of the coordination loops,
using a conservative 10 second bound. Worker threads record a failure and
return if they hit the timeout, while the main thread fails the test via
Catch2 instead of hanging. That way, if future CPython free-threading
patches change the semantics again, the test will fail quickly and
produced a diagnosable error instead of wedging the CI job.
* Revert "Limit busy-wait loops in per-subinterpreter GIL test"
This reverts commit 7847adacda.
* Add progress reporter for test_with_catch Catch runner
Introduce a custom Catch2 reporter for tests/test_with_catch that prints a
simple one-line status for each test case as it starts and ends, and wire the
cpptest CMake target to invoke test_with_catch with -r progress. This makes
it much easier to see where the embedded/interpreter test binary is spending
its time in CI logs, and in particular to pinpoint which test case is stuck
when the free-threading builds hang.
Compared to adding ad hoc timeouts around potentially infinite busy-wait
loops in individual tests, a progress reporter is a more general and robust
approach: it gives visibility into all tests (including future ones) without
changing their behavior, and turns otherwise opaque 90-minute timeouts into
locatable issues in the Catch output.
* Temporarily limit CI to Python 3.14t free-threading jobs
* Temporarily remove non-CI GitHub workflow files
* Temporarily disable AppVeyor builds via skip_commits
* Add DEBUG_LOOK in TEST_CASE("Move Subinterpreter")
* Add Python version banner to Catch progress reporter
Print the CPython version once at the start of the Catch-based
interpreter tests using Py_GetVersion(). This makes it trivial to
confirm which free-threaded build a failing run is using when
inspecting CI or local logs.
* Revert "Add DEBUG_LOOK in TEST_CASE("Move Subinterpreter")"
This reverts commit ad3e1c34ce.
* Pin CI free-threaded runs to CPython 3.14.0t
Update the standard-small and standard-large GitHub Actions jobs to
request python-version 3.14.0t instead of 3.14t. This forces setup-python
to use the last-known-good 3.14.0 free-threaded build rather than the
newer 3.14.1+ builds where subinterpreter finalization regressed.
* Revert "Pin CI free-threaded runs to CPython 3.14.0t"
This reverts commit 5281e1c20c.
* Revert "Temporarily disable AppVeyor builds via skip_commits"
This reverts commit ed11292636.
* Revert "Temporarily remove non-CI GitHub workflow files"
This reverts commit 0fe6a42a04.
* Revert "Temporarily limit CI to Python 3.14t free-threading jobs"
This reverts commit 60ae0e8f74.
* Pin CI free-threaded runs to CPython 3.14.0t
Update the standard-small and standard-large GitHub Actions jobs to
request python-version 3.14.0t instead of 3.14t. This forces setup-python
to use the last-known-good 3.14.0 free-threaded build rather than the
newer 3.14.1+ builds where subinterpreter finalization regressed.
* Switch NVHPC job to ubuntu-24.04 and disable AppVeyor
* Temporarily trim workflows to focus on NVHPC job
* First restore ci.yml from test-with-catch-timeouts branch, then delete all jobs except ubuntu-nvhpc7
* Change runner to ubuntu-24.04
* Use nvhpc-25-11
* Undo ALL changes relative to master (i.e. this branch is now an exact copy of master)
* Change runner to ubuntu-24.04
* Use nvhpc-25-11
* Remove misleading 7 from job name (i.e. ubuntu-nvhpc7 → ubuntu-nvhpc)
* adding windows arm test
- excluding numpy 2.2.0 for arm64 builds
* adding windows arm msys2 test
* testing mingw python
* unnamed namespace test on windows arm with clang and mingw
* Revert "unnamed namespace test on windows arm with clang and mingw"
This reverts commit 08abf889ae.
* bumping c++ version
* commenting out other tests
* Ignore unnmaed namespace on arm windows with mingw
- Updatig XFAIL condition to expect a failure on windows arm with
mingw and clang
- setting python home and path variables in c++ tests
* Revert "commenting out other tests"
This reverts commit dc75243963.
* removing windows-11-arm from big test, push
* removing redundant shell
* removing trailing whitespace
* Clarify Windows ARM clang job naming
Rename the Windows ARM clang jobs to windows_arm_clang_msvc and windows_arm_clang_msys2 and adjust their display names to clang-msvc / clang-msys2. The first runs clang against the MSVC/Windows SDK toolchain and python.org CPython for Windows ARM, while the second runs clang inside the MSYS2/MinGW-w64 CLANGARM64 environment with MSYS2 Python.
Using clearly distinguished job names makes it easier to discuss failures and behavior in each environment without ambiguity, both in logs and in PR review discussions.
* Reduce Windows ARM clang matrix size
Limit the windows_arm_clang_msvc job to Python 3.13 and the windows_arm_clang_msys2 job to Python 3.12 to stay within our constrained GitHub Actions resources. Keep both jobs using a matrix over os and python so their structure stays aligned and it remains easy to expand coverage when needed.
* Remove matrix.python from windows_arm_clang_msys2 job: the Python version is determined by the msys2/setup-msys2 action and cannot be changed
---------
Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
* Add fast_type_map, use it authoritatively for local types and as a hint for global types
nanobind has a similar two-level lookup strategy, added and explained
by
b515b1f7f2
In this PR I've ported this approach to pybind11. To avoid an ABI
break, I've kept the fast maps to the `local_internals`. I think this
should be safe because any particular module should see its
`local_internals` reset at least as often as the global `internals`,
and misses in the fast "hint" map for global types fall back to the
global `internals`.
Performance seems to have improved. Using my patched fork of
pybind11_benchmark
(https://github.com/swolchok/pybind11_benchmark/tree/benchmark-updates,
specifically commit hash b6613d12607104d547b1c10a8145d1b3e9937266), I
run bench.py and observe the MyInt case. Each time, I do 3 runs and
just report all 3.
master, Mac: 75.9, 76.9, 75.3 nsec/loop
this PR, Mac: 73.8, 73.8, 73.6 nsec/loop
master, Linux box: 188, 187, 188 nsec/loop
this PR, Linux box: 164, 165, 164 nsec/loop
Note that the "real" percentage improvement is larger than implied by the
above because master does not yet include #5824.
* simplify unsafe_reset_local_internals in test
* pre-implement PYBIND11_INTERNALS_VERSION 12
* use PYBIND11_INTERNALS_VERSION 12 on Python 3.14 per suggestion
* Implement reviewer comments: revert PY_VERSION_HEX change, fix REVIEW comment, add two-level lookup comments. ci.yml coming separately
* Use the inplace build to smoke test ABI bump?
* [skip ci] Remove "smoke" from comment. This is full testing, just only on a few platforms.
---------
Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Occasionally a test will get stuck and run for 6 hours until Github cancels the workflow.
This reduces the timeout to 90 minutes to not waste resources.
Pybind11's tests seem to run in 30 minutes so this should be plenty of time.
* Revert "s/windows-2022/windows-latest/ in .github/workflows/{ci,pip}.yml (#5826)"
This reverts commit 852a4b5010.
* Add module-level skip for Windows build >= 26100 in test_iostream.py
* Changes suggested by at-henryiii
* test: Added test case for visibility of common symbols across shared libraries
* style: pre-commit fixes
* tests: cmake target name fix
* tests: Added visibility test to ci
* tests: set the default visibility to hidden
* prototype/proof-of-concept fix: PYBIND11_EXPORT_GUARDED_DELETE
* Fix silly oversight: actually use PYBIND11_EXPORT_GUARDED_DELETE
* Update struct_smart_holder.h
* style: pre-commit fixes
* Update include/pybind11/detail/struct_smart_holder.h
* Update struct_smart_holder.h
* ci: fix addition to reusable-standard.yml
* Update CMakeLists.txt
* refactor: rename tests to test_cross_module_rtti
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
---------
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
* First draft a subinterpreter embedding API
* Move subinterpreter tests to their own file
* Migrate subinterpreter tests to use the new embedded class.
* Add a test for moving subinterpreters across threads for destruction
And find a better way to make that work.
* Code organization
* Add a test which shows demostrates how gil_scoped interacts with sub-interpreters
* Add documentation for embeded sub-interpreters
* Some additional docs work
* Add some convenience accessors
* Add some docs cross references
* Sync some things that were split out into #5665
* Update subinterpreter docs example to not use the CPython api
* Fix pip test
* style: pre-commit fixes
* Fix MSVC warnings
I am surprised other compilers allowed this code with a deleted move ctor.
* Add some sub-headings to the docs
* Oops, make_unique is C++14 so remove it from the tests.
* I think this fixes the EndInterpreter issues on all versions.
It just has to be ifdef'd because it is slightly broken on 3.12, working well on 3.13, and kind of crashy on 3.14beta. These two verion ifdefs solve all the issues.
* Add a note about exceptions.
They contain Python object references and acquire the GIL, that means they are a danger with subinterpreters!
* style: pre-commit fixes
* Add try/catch to docs examples to match the tips
* Python 3.12 is very picky about this first PyThreadState
Try special casing the destruction on the same thread.
* style: pre-commit fixes
* Missed a rename in a ifdef block
* I think this test is causing problems in 3.12, so try ifdefing it to see if the problems go away.
* style: pre-commit fixes
* Document the 3.12 constraints with a warning
* Apply suggestions from code review
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* ci: add cpptest to the clang-tidy job
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* noexcept move operations
* Update include/pybind11/subinterpreter.h
std::memset
Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com>
---------
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com>
* Move embedded modules to multiphase init
So that they too can support multi-interpreter and nogil tags
* Update the multiple interpreter test for embedded module changes
* Add a note to embedded module docs about the new tags
* Oops, missed a warning pop
* Remove unused variable
* Update ci.yml
* Fix this embedded GIL test for free-threading
* Oops, need to use ptr() here
* This test created a subinterpreter when PYBIND11_SUBINTERPRETER_SUPPORT was off
So the fix is really this test should not be run in these older versions at all.
The hang was a GIL issue between the subinterpreters during pybind11::exception::what().
* fix: standard mutex for 3.13t
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
---------
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
* ci: support Python 3.14
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* fix: Python 3.14 name change
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* tests: fix expected output to handle Python 3.14
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* fix: tighten CLI and add color on 3.14+
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* tests: ignore failure on 3.14.0b1
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* fix: support Python 3.14.0b1 with interperters
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* Update test_multiple_interpreters.py
* Update test_multiple_interpreters.py
* fix: new breakage for 3.14 fixed
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* fix: handle empty annotations 3.14
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* fix: Python 3.14 may not create the annotations dict
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* fix: use PyUnstable_IsImmortal
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* fix: use sys._is_immortal
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* tests: ignore large values for refcount too
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* style: pre-commit fixes
* ci: enable all free-threaded builds
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* fix: patch for embed
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* Revert "fix: patch for embed"
This reverts commit c4226a0671.
* ci: drop new 3.xt additions
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* fix: logic issue, also add some comments
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* Update include/pybind11/pytypes.h
---------
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* CI: Fail on any warnings with clang 18
* CI: Fail on any warnings with gcc 13
* Fix cmake's try_compile warning
* Guard redundant declarations
* ci.yml: fix syntax error
* Use PYBIND11_WARNING macros
* Fix more redundant declarations
... introduced with merge of smart_holder branch
* ci: speed up ci a bit by thinning out matrix
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* Update configure.yml
* Update configure.yml
* Update configure.yml
* ci: drop third build
* ci: fix minor issues
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* ci: trim a bit more
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
---------
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
