Can we have some benchmarks?

Yes :)

Benchmark (on an M2 Macbook Air):

Script:

import timeit

for n in (1, 10, 100, 1000, 10000):
    setup = f'''
a = set(range({n}))
b = {{frozenset(a)}}
'''
    print(timeit.timeit("a in b", setup), f'N = {n}')

main branch result:

0.3688333749996673 N = 1
0.48004929200033075 N = 10
1.781020708000142 N = 100
11.256919416999153 N = 1000
152.03752795800028 N = 10000

my branch result:

0.30569541699878755 N = 1
0.3746081659992342 N = 10
1.2535620420021587 N = 100
7.2957435000025725 N = 1000
86.63279329200304 N = 10000

There will be no performance regression on the normal case.

This is a neat performance improvement, that contrary to most other such attempts, does not add to the complexity of the code; nice. AFAICS, this is good to go. I'll leave the landing of the PR to the code owner.

Thank you so much for your review. I have made changes to resolve the issues.

We used to do something like this, but it caused bugs and had to be removed. See https://bugs.python.org/issue8757 and checkin 4d45c10 . IIRC there was also a reason that the set had to be made temporarily immutable during the lookup. I don't remember all the details now. Perhaps @serhiy-storchaka does.

The original set_swap_bodies variants were very fast, so it was a bummer to have to use a set copy instead. The good news is that set copies are now much faster than they were. Also, we realized that the implicit frozenset conversion promised in the docs was an almost never used feature.

It was before I started contributing to CPython, so I have no memories about this case.

The approach of this PR is better than the code used before bpo-8757. It leaves the original set key unmodified, so other threads are not affected if they only read it. The original reproducer should pass this test.

But what if the set key is modified in other thread? I think this is safe if set comparison is thread-safe (and AFAIK it is).

There is a new race condition: the set key can be changed during calculating its hash (currently creating a frozenset from a set is atomic, or it can be made atomic). But I do not think this is bad. We should not guarantee the correct result in such case.

I think this change is safe. LGTM.