* Allow per-interpreter internals/local_internals
* Significant rewrite to avoid using thread_locals as much as possible.
Since we can avoid them by checking this atomic, the cmake config conditional shouldn't be necessary.
The slower path (with thread_locals and extra checks) only comes in when a second interpreter is actually instanciated.
* Add a test for per-interpreter GIL
Uses two extra threads to demonstrate that neither shares a GIL.
* Fix for nonconforming std::atomic constructors on some compilers
* style: pre-commit fixes
* Fix initializer to make MSVC happy.
* Switch to gil_scoped_acquire_simple, get rid of old copy of it from internals.h
* Use the PyThreadState's interp member rather than the thread state itself.
* Be more explicit about the type of the internalspp
* Suggested renamings and rewordings
* Rename find_internals_pp and change it to take in the state dict reference
* Use the old raise_from instead of pybind11_fail
* Move most of the internals initialization into its constructor.
* Move round_up_to_next_pow2 function upwards
* Remove redundant forward decl
* Add a python-driven subinterpreter test
* Disable the python subinterpreter test on emscripten
Can't load the native-built cpp modules.
* Switch the internals pointer pointer to a unique_ptr pointer
* Spelling
* Fix clang-tidy warning, compare pointer to nullptr
* Rename get_interpreter_counter to get_num_interpreters_seen
* Try simplifying the test's cmake set_target_properties
* Replace mod_* tags with a single tag w/enum
Update tests accordingly
* Add a test for shared-GIL (legacy) subinterpreters
* Update test to work around differences in the various versions of interpreters modules
* Fix unused parameter
* Rename tests and associated test modules.
* Switch get_internals_pp to a template function
* Rename curtstate to cur_tstate
* refactor: use simpler names
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* style: pre-commit fixes
* fix: return class, not enum
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgkio@gmail.com>
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* Have to join these threads to make sure they are totally done before the test returns.
* Wrap module_def initialization in a static so it only happens once.
If it happens concurrently in multiple threads, badness ensues....
* style: pre-commit fixes
---------
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgkio@gmail.com>
* Support free-threaded CPython (PEP 703)
Some additional locking is added in the free-threaded build when
`Py_GIL_DISABLED` is defined:
- Most accesses to internals are protected by a single mutex
- The registered_instances uses a striped lock to improve concurrency
Pybind11 modules can indicate support for running with the GIL disabled
by calling `set_gil_not_used()`.
* refactor: use PYBIND11_MODULE (#11)
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* Address code review
* Suppress MSVC warning
* Changes from review
* style: pre-commit fixes
* `py::mod_gil_not_used()` suggestion.
* Update include/pybind11/pybind11.h
---------
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
* Fix segfault when reloading interpreter with external modules
When embedding the interpreter and loading external modules in that
embedded interpreter, the external module correctly shares its
internals_ptr with the one in the embedded interpreter. When the
interpreter is shut down, however, only the `internals_ptr` local to
the embedded code is actually reset to nullptr: the external module
remains set.
The result is that loading an external pybind11 module, letting the
interpreter go through a finalize/initialize, then attempting to use
something in the external module fails because this external module is
still trying to use the old (destroyed) internals. This causes
undefined behaviour (typically a segfault).
This commit fixes it by adding a level of indirection in the internals
path, converting the local internals variable to `internals **` instead
of `internals *`. With this change, we can detect a stale internals
pointer and reload the internals pointer (either from a capsule or by
creating a new internals instance).
(No issue number: this was reported on gitter by @henryiii and @aoloe).
