X Tutup
Skip to content

gh-108867: Add PyThreadState_GetUnchecked() function#108870

Merged
vstinner merged 2 commits intopython:mainfrom
vstinner:tstate_getunsafe
Oct 3, 2023
Merged

gh-108867: Add PyThreadState_GetUnchecked() function#108870
vstinner merged 2 commits intopython:mainfrom
vstinner:tstate_getunsafe

Conversation

@vstinner
Copy link
Member

@vstinner vstinner commented Sep 4, 2023 •
edited
Loading

Add PyThreadState_GetUnchecked() function: similar to
PyThreadState_Get(), but don't issue a fatal error if it is NULL. The
caller is responsible to check if the result is NULL. Previously,
this function was private and known as _PyThreadState_UncheckedGet().

📚 Documentation preview 📚: https://cpython-previews--108870.org.readthedocs.build/

@bedevere-bot bedevere-bot mentioned this pull request Sep 4, 2023
Closed
@vstinner vstinner force-pushed the tstate_getunsafe branch 2 times, most recently from c180b5b to 572f4a4 Compare September 6, 2023 02:36
@vstinner
Copy link
Member Author

vstinner commented Sep 6, 2023

I was worried about Py_TRASHCAN_BEGIN_CONDITION() macro which currently calls _PyThreadState_UncheckedGet(). This macro uses private functions:

/* Python 3.9 private API, invoked by the macros below. */
PyAPI_FUNC(int) _PyTrash_begin(PyThreadState *tstate, PyObject *op);
PyAPI_FUNC(void) _PyTrash_end(PyThreadState *tstate);

/* Python 3.10 private API, invoked by the Py_TRASHCAN_BEGIN(). */
PyAPI_FUNC(int) _PyTrash_cond(PyObject *op, destructor dealloc);

I was worried that we still provide Python 3.9 functions. Is it the a stable ABI? Nope, these functions are not part of Misc/stable_abi.toml.

Moreover, I explicitly removed the following macros from the limited C API in Python 3.9:

  • PyTrash_UNWIND_LEVEL
  • Py_TRASHCAN_BEGIN_CONDITION()
  • Py_TRASHCAN_BEGIN()
  • Py_TRASHCAN_END()
  • Py_TRASHCAN_SAFE_BEGIN()
  • Py_TRASHCAN_SAFE_END()

So in fact, we keep Python 3.9 functions for the ABI backward compatibility, whereas we do not support these APIs in the stable ABI.

In short, it's ok to rename _PyThreadState_UncheckedGet() to PyThreadState_GetUnsafe(), it's not used by the stable ABI.

@vstinner
Copy link
Member Author

vstinner commented Sep 9, 2023

A bunch of projects currently uses _PyThreadState_UncheckedGet(), whereas I removed the function from the public C API in Python 3.13.

Affected projects (18):

  • Cython (0.29.36)
  • catboost (1.2)
  • cffi (1.15.1)
  • debugpy (1.6.7)
  • dlib (19.24.2)
  • fastobo (0.12.2)
  • frozendict (2.3.8)
  • onnx (1.14.0)
  • onnxoptimizer (0.3.13)
  • onnxsim (0.4.33)
  • orjson (3.9.1)
  • osmium (3.6.0)
  • praat-parselmouth (0.4.3)
  • ptvsd (4.3.2)
  • pybind11 (2.10.4)
  • pydevd (2.9.6)
  • pydevd-pycharm (232.8296.19)
  • pythonnet (3.0.1)

In this list, IMO the interesting one is Cython:

$ git grep -E '(_PyThreadState_UncheckedGet|__Pyx_PyThreadState_Current)'
Cython/Utility/AsyncGen.c:    tstate = __Pyx_PyThreadState_Current;
Cython/Utility/AsyncGen.c:    tstate = __Pyx_PyThreadState_Current;
Cython/Utility/Coroutine.c:    __Pyx_PyGen__FetchStopIterationValue(__Pyx_PyThreadState_Current, pvalue)
Cython/Utility/Coroutine.c:    tstate = __Pyx_PyThreadState_Current;
Cython/Utility/Coroutine.c:    __Pyx_PyGen__FetchStopIterationValue(__Pyx_PyThreadState_Current, &val);
Cython/Utility/Exceptions.c:    __pyx_assertions_enabled_flag = ! _PyInterpreterState_GetConfig(__Pyx_PyThreadState_Current->interp)->optimization_level;
Cython/Utility/Exceptions.c:#define __Pyx_PyThreadState_assign  $local_tstate_cname = __Pyx_PyThreadState_Current;
Cython/Utility/Exceptions.c:        PyThreadState *tstate = __Pyx_PyThreadState_Current;
Cython/Utility/Exceptions.c:        (void) __Pyx_CLineForTraceback(__Pyx_PyThreadState_Current, c_line);
Cython/Utility/Exceptions.c:    PyThreadState *tstate = __Pyx_PyThreadState_Current;
Cython/Utility/ModuleSetupCode.c:    // Py3<3.5.2 does not support _PyThreadState_UncheckedGet().
Cython/Utility/ModuleSetupCode.c:  #define __Pyx_PyThreadState_Current PyThreadState_Get()
Cython/Utility/ModuleSetupCode.c:  #define __Pyx_PyThreadState_Current PyThreadState_GET()
Cython/Utility/ModuleSetupCode.c:  #define __Pyx_PyThreadState_Current _PyThreadState_UncheckedGet()
Cython/Utility/ModuleSetupCode.c:  #define __Pyx_PyThreadState_Current PyThreadState_GET()
Cython/Utility/ModuleSetupCode.c:  #define __Pyx_PyThreadState_Current _PyThreadState_Current
Cython/Utility/ModuleSetupCode.c:  current = tcur == __Pyx_PyThreadState_Current;
Cython/Utility/ObjectHandling.c:    PyThreadState *tstate = __Pyx_PyThreadState_Current;
Cython/Utility/Profile.c:          tstate = __Pyx_PyThreadState_Current;                                          \
Cython/Utility/Profile.c:      PyThreadState* tstate = __Pyx_PyThreadState_Current;                                 \
Cython/Utility/Profile.c:              tstate = __Pyx_PyThreadState_Current;                                       \
Cython/Utility/Profile.c:          PyThreadState* tstate = __Pyx_PyThreadState_Current;                            \
Cython/Utility/Profile.c:      PyThreadState* tstate = __Pyx_PyThreadState_Current;                                \
Cython/Utility/Profile.c:              tstate = __Pyx_PyThreadState_Current;                                        \
Cython/Utility/Profile.c:          PyThreadState* tstate = __Pyx_PyThreadState_Current;                             \
Cython/Utility/Profile.c:      PyThreadState* tstate = __Pyx_PyThreadState_Current;                                 \

I can easily add PyThreadState_GetUnsafe() to pythoncapi-compat by calling _PyThreadState_UncheckedGet() on Python 3.12 and older. This private function is available since Python 3.5. It seems to be available on PyPy:

$ cd /usr/include/pypy3.9/
$ grep _PyThreadState_UncheckedGet -R .
./pylifecycle.h:    (_Py_IsFinalizing() ? _PyThreadState_UncheckedGet() : NULL)
./pypy_decl.h:#define _PyThreadState_UncheckedGet _PyPyThreadState_UncheckedGet
./pypy_decl.h:PyAPI_FUNC(PyThreadState *) _PyThreadState_UncheckedGet(void);

@vstinner
Copy link
Member Author

vstinner commented Sep 9, 2023

I asked in the C API Working Group which function name is better: use Unchecked suffix or Unsafe suffix?

@vstinner vstinner mentioned this pull request Sep 9, 2023
Closed
encukou
encukou reviewed Sep 11, 2023
View reviewed changes
@vstinner
Copy link
Member Author

vstinner commented Sep 11, 2023

I renamed the function to PyThreadState_GetUnchecked() and addressed @encukou's suggestion on the documentation.

vstinner
vstinner commented Sep 18, 2023
View reviewed changes
@vstinner
pythongh-108867: Add PyThreadState_GetUnchecked() function
2372091 
Add PyThreadState_GetUnchecked() function: similar to
PyThreadState_Get(), but don't issue a fatal error if it is NULL. The
caller is responsible to check if the result is NULL. Previously,
this function was private and known as _PyThreadState_UncheckedGet().
@vstinner vstinner changed the title gh-108867: Add PyThreadState_GetUnsafe() function gh-108867: Add PyThreadState_GetUnchecked() function Oct 3, 2023
@vstinner
Copy link
Member Author

vstinner commented Oct 3, 2023

I renamed the function to PyThreadState_GetUnchecked()

Oops, my PR still used PyThreadState_GetUnsafe() name. I messed up something.

Anyway, I updated it again, and now it uses PyThreadState_GetUnchecked() name everywhere.

@vstinner vstinner enabled auto-merge (squash) October 3, 2023 16:23
@vstinner vstinner merged commit d735016 into python:main Oct 3, 2023
@vstinner vstinner deleted the tstate_getunsafe branch October 3, 2023 16:53
@vstinner
Copy link
Member Author

vstinner commented Oct 3, 2023

I added PyThreadState_GetUnchecked() to pythoncapi-compat: python/pythoncapi-compat@f78c780

Glyphack pushed a commit to Glyphack/cpython that referenced this pull request Sep 2, 2024
@vstinner @Glyphack
pythongh-108867: Add PyThreadState_GetUnchecked() function (python#10…
5fd59fe 
…8870)

Add PyThreadState_GetUnchecked() function: similar to
PyThreadState_Get(), but don't issue a fatal error if it is NULL. The
caller is responsible to check if the result is NULL. Previously,
this function was private and known as _PyThreadState_UncheckedGet().
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@encukou encukou encukou left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@vstinner @encukou @bedevere-bot
X Tutup