GH-140061: Use _PyObject_IsUniquelyReferenced to check if object is uniquely referenced by sergey-miryanov · Pull Request #140062 · python/cpython

sergey-miryanov · 2025-10-13T19:48:41Z

There are a lot of places where Py_REFCNT(..)==1 used. IIUC it is not correct for no-gil version.

Issue: Use _PyObject_IsUniquelyReferenced instead of Py_REFCNT(x) == 1 for free-threaded compatibility #140061

…erenced

sergey-miryanov · 2025-10-13T19:49:13Z

I think it should skip news, but not sure about backporting.

ZeroIntensity · 2025-10-14T13:56:47Z

skip news isn't right -- Py_REFCNT(op) == 1 is an actual bug on the free-threaded build (and sometimes the GILful build), at least when using it for optimizations.

ZeroIntensity

When the reference is owned by the eval loop, we need to use PyUnstable_Object_IsUniqueReferencedTemporary instead. I think there are a few cases of that here.

cc @colesbury

colesbury

You need to be careful about changing assert(), especially when the code paths can be called by extensions.
There are a number of conditions like Py_REFCNT(result) > 1 that need to be changed, such as in itertooslmodule.c

Modules/_functoolsmodule.c

Modules/_testcapi/object.c

Objects/dictobject.c

Objects/unicodeobject.c

Objects/longobject.c

Objects/listobject.c

sergey-miryanov · 2025-10-14T20:40:31Z

@ZeroIntensity @colesbury Thanks for review! I have fixed your suggestions, but it fails in TSAN(free-threading). Can you take a look?

ZeroIntensity · 2025-10-15T00:20:12Z

It seems that me and Kumar broke that earlier (see #140138), sorry! You can ignore it for now and we'll rebase this once we get it fixed on main.

sergey-miryanov · 2025-10-15T05:30:25Z

@ZeroIntensity Thanks!

vstinner · 2025-10-15T11:35:38Z

Is this PR ready for a review?

sergey-miryanov · 2025-10-15T12:31:08Z

@vstinner Yes, it is ready to review. If @colesbury don't have any comments, of course.

vstinner · 2025-10-15T12:33:16Z

Ok, I click on [Ready for review] button in this case :-)

sergey-miryanov · 2025-10-15T12:36:03Z

Ough, I forgot that I turned it to draft :(

vstinner · 2025-10-15T12:36:42Z

Remaining code using Py_REFCNT() with 1:

$ git grep 'Py_REFCNT.*\(==\|<\|<=\|>\|>=\) *1'
Doc/c-api/object.rst:   :c:expr:`Py_REFCNT(op) == 1`.
Doc/c-api/object.rst:   is only used by this thread. :c:expr:`Py_REFCNT(op) == 1` is **not**
Doc/whatsnew/3.14.rst:  a replacement for ``Py_REFCNT(op) == 1`` on :term:`free threaded
Include/internal/pycore_object.h:    return Py_REFCNT(ob) == 1;
Modules/_ctypes/_ctypes.c:   assert (Py_REFCNT(obj) == 1);
Modules/_functoolsmodule.c:            assert(Py_REFCNT(args) == 1);
Modules/_io/bytesio.c:  * Py_REFCNT(buf) == 1, exports == 0.
Modules/_io/bytesio.c:  * Py_REFCNT(buf) > 1.  exports == 0,
Modules/_io/bytesio.c:  * exports > 0.  Py_REFCNT(buf) == 1, any modifications are forbidden.
Modules/_testcapi/object.c:    assert(Py_REFCNT(obj) == 1);
Modules/_testcapi/object.c:    assert(Py_REFCNT(obj) == 1);
Modules/_testcapi/object.c:        assert(Py_REFCNT(obj) == 1); \
Modules/_testcapimodule.c:    assert(Py_REFCNT(obj) == 1);
Modules/itertoolsmodule.c:        assert (npools==0 || Py_REFCNT(result) == 1);
Modules/itertoolsmodule.c:        assert(r == 0 || Py_REFCNT(result) == 1);
Modules/itertoolsmodule.c:        assert(r == 0 || Py_REFCNT(result) == 1);
Modules/itertoolsmodule.c:        assert(r == 0 || Py_REFCNT(result) == 1);
Objects/listobject.c:    assert(Py_REFCNT(self) == 1 || PyMutex_IsLocked(&_PyObject_CAST(self)->ob_mutex));
Objects/longobject.c:                assert(Py_REFCNT(z) == 1);
Objects/longobject.c:                assert(_PyLong_IsZero(z) || Py_REFCNT(z) == 1);
Objects/longobject.c:        assert(Py_REFCNT(z) == 1);
Objects/longobject.c:    assert(Py_REFCNT(obj) == 1);
Objects/longobject.c:    assert(Py_REFCNT(obj) == 1);
Objects/object.c:    CHECK(Py_REFCNT(op) >= 1);
Objects/object.c:    _PyObject_ASSERT(name, Py_REFCNT(name) >= 1);
Objects/setobject.c:    assert(Py_REFCNT(so) == 1);
Objects/typeobject.c:    CHECK(Py_REFCNT(type) >= 1);
Objects/unicodeobject.c:    assert(Py_REFCNT(unicode) == 1);
Objects/unicodeobject.c:            assert(Py_REFCNT(unicode) == 1);
Python/ceval.c:    assert(Py_REFCNT(locals) > 1);
Python/gc_free_threading.c:                _PyObject_ASSERT(op, Py_REFCNT(op) > 1);

There are documentation and assertions. Assertions should continue to use Py_REFCNT(), right?

vstinner

LGTM

Objects/longobject.c

Modules/_elementtree.c

sergey-miryanov · 2025-10-15T12:49:34Z

Assertions should continue to use Py_REFCNT(), right?

As I understand @colesbury - yes.

Co-authored-by: Victor Stinner <vstinner@python.org>

colesbury · 2025-10-15T13:27:00Z

Thanks @sergey-miryanov!

miss-islington-app · 2025-10-15T13:48:25Z

Thanks @sergey-miryanov for the PR, and @colesbury for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖

…bjects are uniquely referenced (pythongh-140062) The previous `Py_REFCNT(x) == 1` checks can have data races in the free threaded build. `_PyObject_IsUniquelyReferenced(x)` is a more conservative check that is safe in the free threaded build and is identical to `Py_REFCNT(x) == 1` in the default GIL-enabled build. (cherry picked from commit 32c2649) Co-authored-by: Sergey Miryanov <sergey.miryanov@gmail.com>

bedevere-app · 2025-10-15T13:48:40Z

GH-140157 is a backport of this pull request to the 3.14 branch.

…objects are uniquely referenced (gh-140062) (gh-140157) The previous `Py_REFCNT(x) == 1` checks can have data races in the free threaded build. `_PyObject_IsUniquelyReferenced(x)` is a more conservative check that is safe in the free threaded build and is identical to `Py_REFCNT(x) == 1` in the default GIL-enabled build. (cherry picked from commit 32c2649) Co-authored-by: Sergey Miryanov <sergey.miryanov@gmail.com>

sergey-miryanov · 2025-10-15T15:43:05Z

Thanks everyone for review!

…bjects are uniquely referenced (pythongh-140062) The previous `Py_REFCNT(x) == 1` checks can have data races in the free threaded build. `_PyObject_IsUniquelyReferenced(x)` is a more conservative check that is safe in the free threaded build and is identical to `Py_REFCNT(x) == 1` in the default GIL-enabled build.

Use _PyObject_IsUniquelyReferenced to check if object is uniquely ref…

311f629

…erenced

sergey-miryanov requested review from markshannon, methane and rhettinger as code owners October 13, 2025 19:48

bedevere-app bot added the awaiting review label Oct 13, 2025

bedevere-app bot mentioned this pull request Oct 13, 2025

Use _PyObject_IsUniquelyReferenced instead of Py_REFCNT(x) == 1 for free-threaded compatibility #140061

Closed

Fix build on linux

d381a91

picnixz added the skip news label Oct 13, 2025

sergey-miryanov added 2 commits October 14, 2025 10:53

Return Py_REFCNT to test_weakref_capi

d416ac6

Fix warning in ensure_shared_on_resize

45593e8

ZeroIntensity removed the skip news label Oct 14, 2025

ZeroIntensity reviewed Oct 14, 2025

View reviewed changes

colesbury self-requested a review October 14, 2025 15:59

colesbury reviewed Oct 14, 2025

View reviewed changes

Objects/listobject.c Outdated Show resolved Hide resolved

sergey-miryanov marked this pull request as draft October 14, 2025 18:39

bedevere-app bot removed the awaiting review label Oct 14, 2025

sergey-miryanov added 7 commits October 14, 2025 23:40

Address review suggestions

2989918

Revert for asserts

d3b0e31

Revert for ctypes

375bf2c

Fix functools

86079c4

Fix Py_REFCNT > 1

b8b54c7

Fix compilation

307be2c

Add news entry

83f4b84

Merge branch 'main' into 140061-fix-checking-object-uniq-ref

7074ae0

vstinner marked this pull request as ready for review October 15, 2025 12:33

bedevere-app bot added the awaiting review label Oct 15, 2025

vstinner approved these changes Oct 15, 2025

View reviewed changes

Objects/longobject.c Outdated Show resolved Hide resolved

Modules/_elementtree.c Outdated Show resolved Hide resolved

bedevere-app bot added awaiting merge and removed awaiting review labels Oct 15, 2025

Apply suggestions from code review

1b2193f

Co-authored-by: Victor Stinner <vstinner@python.org>

colesbury added the needs backport to 3.14 bugs and security fixes label Oct 15, 2025

colesbury approved these changes Oct 15, 2025

View reviewed changes

colesbury merged commit 32c2649 into python:main Oct 15, 2025
47 checks passed

bedevere-app bot removed the awaiting merge label Oct 15, 2025

bedevere-app bot removed the needs backport to 3.14 bugs and security fixes label Oct 15, 2025

Uh oh!

Conversation

sergey-miryanov commented Oct 13, 2025 • edited by bedevere-app bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sergey-miryanov commented Oct 13, 2025

Uh oh!

ZeroIntensity commented Oct 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ZeroIntensity left a comment

Choose a reason for hiding this comment

Uh oh!

colesbury left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sergey-miryanov commented Oct 14, 2025

Uh oh!

ZeroIntensity commented Oct 15, 2025

Uh oh!

sergey-miryanov commented Oct 15, 2025

Uh oh!

vstinner commented Oct 15, 2025

Uh oh!

sergey-miryanov commented Oct 15, 2025

Uh oh!

vstinner commented Oct 15, 2025

Uh oh!

sergey-miryanov commented Oct 15, 2025

Uh oh!

vstinner commented Oct 15, 2025

Uh oh!

vstinner left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

sergey-miryanov commented Oct 15, 2025

Uh oh!

colesbury commented Oct 15, 2025

Uh oh!

Uh oh!

miss-islington-app bot commented Oct 15, 2025

Uh oh!

bedevere-app bot commented Oct 15, 2025

Uh oh!

sergey-miryanov commented Oct 15, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

sergey-miryanov commented Oct 13, 2025 •

edited by bedevere-app bot

Loading

ZeroIntensity commented Oct 14, 2025 •

edited

Loading