Files
pybind11/.github/workflows/upstream.yml
Ralf W. Grosse-Kunstleve 78381e5e28 Improve C++ test infrastructure: progress reporter, timeouts, and skip hanging Move Subinterpreter test (#5942)
* Improve C++ test infrastructure and disable hanging test

This commit improves the C++ test infrastructure to ensure test output
is visible in CI logs, and disables a test that hangs on free-threaded
Python 3.14+.

Changes:

## CI/test infrastructure improvements

- .github/workflows: Added `timeout-minutes: 3` to all C++ test steps
  to prevent indefinite hangs.

- tests/**/CMakeLists.txt: Added `USES_TERMINAL` to C++ test targets
  (cpptest, test_cross_module_rtti, test_pure_cpp) to ensure output is
  shown immediately rather than buffered and possibly lost on crash/timeout.

- tests/test_with_catch/catch.cpp: Added a custom Catch2 progress reporter
  with timestamps, Python version info, and a SIGTERM handler to make test
  execution and failures clearly visible in CI logs.

## Disabled hanging test

- The "Move Subinterpreter" test is disabled on free-threaded Python 3.14+
  due to a hang in Py_EndInterpreter() when the subinterpreter is destroyed
  from a different thread than it was created on. Work on fixing the
  underlying issue will continue under PR #5940.

Context: We were in the dark for months (since we started testing with
Python 3.14t) because CI logs gave no clue about the root cause of hangs.
This led to ignoring intermittent hangs (mostly on macOS). Our hand was
forced only with the Python 3.14.1 release, when hangs became predictable
on all platforms.

For the full development history of these changes, see PR #5933.

* Add test summary to progress reporter

Print the total number of test cases and assertions at the end of the
test run, making it easy to spot if tests are disabled or added.

Example output:
  [  PASSED  ] 20 test cases, 1589 assertions.

* Add PYBIND11_CATCH2_SKIP_IF macro to skip tests at runtime

Catch2 v2 doesn't have native skip support (v3 does with SKIP()).
This macro allows tests to be skipped with a visible message while
still appearing in the test list.

Use this for the Move Subinterpreter test on free-threaded Python 3.14+
so it shows as skipped rather than being conditionally compiled out.

Example output:
  [ RUN      ] Move Subinterpreter
  [ SKIPPED ] Skipped on free-threaded Python 3.14+ (see PR #5940)
  [       OK ] Move Subinterpreter

* Fix clang-tidy bugprone-macro-parentheses warning in PYBIND11_CATCH2_SKIP_IF
2025-12-21 21:25:06 -08:00

119 lines
2.9 KiB
YAML

name: Upstream
on:
workflow_dispatch:
pull_request:
permissions:
contents: read
concurrency:
group: upstream-${{ github.ref }}
cancel-in-progress: true
env:
PIP_BREAK_SYSTEM_PACKAGES: 1
# For cmake:
VERBOSE: 1
jobs:
standard:
name: "🐍 3.13 latest • ubuntu-latest • x64"
runs-on: ubuntu-latest
# Only runs when the 'python dev' label is selected
if: "contains(github.event.pull_request.labels.*.name, 'python dev')"
steps:
- uses: actions/checkout@v6
- name: Setup Python 3.13
uses: actions/setup-python@v6
with:
python-version: "3.13"
allow-prereleases: true
- name: Setup Boost
run: sudo apt-get install libboost-dev
- name: Update CMake
uses: jwlawson/actions-setup-cmake@v2.0
- name: Run pip installs
run: |
python -m pip install --upgrade pip
python -m pip install -r tests/requirements.txt
- name: Show platform info
run: |
python -m platform
cmake --version
pip list
# First build - C++11 mode and inplace
- name: Configure C++11
run: >
cmake -S . -B build11
-DPYBIND11_WERROR=ON
-DDOWNLOAD_CATCH=ON
-DDOWNLOAD_EIGEN=ON
-DCMAKE_CXX_STANDARD=11
-DCMAKE_BUILD_TYPE=Debug
- name: Build C++11
run: cmake --build build11 -j 2
- name: Python tests C++11
run: cmake --build build11 --target pytest -j 2
- name: C++11 tests
timeout-minutes: 3
run: cmake --build build11 --target cpptest -j 2
- name: Interface test C++11
run: cmake --build build11 --target test_cmake_build
# Second build - C++17 mode and in a build directory
- name: Configure C++17
run: >
cmake -S . -B build17
-DPYBIND11_WERROR=ON
-DDOWNLOAD_CATCH=ON
-DDOWNLOAD_EIGEN=ON
-DCMAKE_CXX_STANDARD=17
- name: Build C++17
run: cmake --build build17 -j 2
- name: Python tests C++17
run: cmake --build build17 --target pytest
- name: C++17 tests
timeout-minutes: 3
run: cmake --build build17 --target cpptest
# Third build - C++17 mode with unstable ABI
- name: Configure (unstable ABI)
run: >
cmake -S . -B build17max
-DPYBIND11_WERROR=ON
-DDOWNLOAD_CATCH=ON
-DDOWNLOAD_EIGEN=ON
-DCMAKE_CXX_STANDARD=17
-DPYBIND11_INTERNALS_VERSION=10000000
- name: Build (unstable ABI)
run: cmake --build build17max -j 2
- name: Python tests (unstable ABI)
run: cmake --build build17max --target pytest
- name: Interface test (unstable ABI)
run: cmake --build build17max --target test_cmake_build
# This makes sure the setup_helpers module can build packages using
# setuptools
- name: Setuptools helpers test
run: |
pip install setuptools
pytest tests/extra_setuptools