gh-136380: Fix import behavior for concurrent.futures.InterpreterPoolExecutor by aisk · Pull Request #136381 · python/cpython

aisk · 2025-07-07T13:17:23Z

It's hard to write the test, but on my local machine, when remove the _interpreters module or raise a ModuleNotFoundError in the try branch, the behavior is correct compare to a normal non-exist field:

>>> from concurrent.futures import InterpreterPoolExecutor
Traceback (most recent call last):
  File "<python-input-0>", line 1, in <module>
    from concurrent.futures import InterpreterPoolExecutor
ImportError: cannot import name 'InterpreterPoolExecutor' from 'concurrent.futures' (C:\Users\xxxxx\Source\cpython\Lib\concurrent\futures\__init__.py)
>>> from concurrent import futures
>>> futures.InterpreterPoolExecutor
Traceback (most recent call last):
  File "<python-input-2>", line 1, in <module>
    futures.InterpreterPoolExecutor
  File "C:\Users\xxxxx\Source\cpython\Lib\concurrent\futures\__init__.py", line 69, in __getattr__      
    raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
AttributeError: module 'concurrent.futures' has no attribute 'InterpreterPoolExecutor'

>>> from concurrent.futures import NotExist
Traceback (most recent call last):
  File "<python-input-0>", line 1, in <module>
    from concurrent.futures import NotExist
ImportError: cannot import name 'NotExist' from 'concurrent.futures' (C:\Users\xxxxx\Source\cpython\Lib\concurrent\futures\__init__.py)
>>> from concurrent import futures
>>> futures.NotExist
Traceback (most recent call last):
  File "<python-input-2>", line 1, in <module>
    futures.NotExist
  File "C:\Users\xxxxx\Source\cpython\Lib\concurrent\futures\__init__.py", line 69, in __getattr__      
    raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
AttributeError: module 'concurrent.futures' has no attribute 'NotExist'

We must import _interpreters in __init__.py to determine whether to append the InterpreterPoolExecutor to __all__. This may have a negative impact on importing performance.

Issue: Inconsistent import behavior when concurrent.futures.InterpreterPoolExecutor not exist #136380

serhiy-storchaka

The downside of this is that _interpreters is now always imported.

Could you add tests? It is not easy, the tests should remove concurrent and all its submodules recursively from sys.module, and restore sys.module at the end. There are helpers for this somewhere, they can be private.

Lib/concurrent/futures/__init__.py

ZeroIntensity

cc @ericsnowcurrently

Misc/NEWS.d/next/Library/2025-07-07-22-12-32.gh-issue-136380.1b_nXl.rst

Lib/concurrent/futures/__init__.py

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

…b_nXl.rst Co-authored-by: Peter Bierma <zintensitydev@gmail.com>

aisk · 2025-07-07T16:59:00Z

The downside of this is that _interpreters is now always imported.

Could you add tests? It is not easy, the tests should remove concurrent and all its submodules recursively from sys.module, and restore sys.module at the end. There are helpers for this somewhere, they can be private.

Found a way to test it by running the test within a subprocess.

serhiy-storchaka

LGTM. 👍

Lib/test/test_concurrent_futures/test_interpreter_pool.py

Lib/concurrent/futures/__init__.py

Lib/test/test_concurrent_futures/test_interpreter_pool.py

ZeroIntensity · 2025-07-08T08:12:22Z

I didn't mean to hit the update button, sorry about that.

Lib/test/test_concurrent_futures/test_interpreter_pool.py

Co-authored-by: sobolevn <mail@sobolevn.me>

sobolevn

Thank you!

miss-islington-app · 2025-07-08T13:32:18Z

Thanks @aisk for the PR, and @sobolevn for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖

…terPoolExecutor` (pythonGH-136381) (cherry picked from commit 490eea0) Co-authored-by: AN Long <aisk@users.noreply.github.com> Co-authored-by: Serhiy Storchaka <storchaka@gmail.com> Co-authored-by: Peter Bierma <zintensitydev@gmail.com> Co-authored-by: sobolevn <mail@sobolevn.me>

bedevere-app · 2025-07-08T13:32:28Z

GH-136420 is a backport of this pull request to the 3.14 branch.

…eterPoolExecutor` (GH-136381) (#136420) gh-136380: Fix import behavior for `concurrent.futures.InterpreterPoolExecutor` (GH-136381) (cherry picked from commit 490eea0) Co-authored-by: AN Long <aisk@users.noreply.github.com> Co-authored-by: Serhiy Storchaka <storchaka@gmail.com> Co-authored-by: Peter Bierma <zintensitydev@gmail.com> Co-authored-by: sobolevn <mail@sobolevn.me>

…terPoolExecutor` (python#136381) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com> Co-authored-by: Peter Bierma <zintensitydev@gmail.com> Co-authored-by: sobolevn <mail@sobolevn.me>

henryiii · 2025-07-29T18:09:18Z

This broke tab completion and dir(concurrent.futures), I've opened a fix in #137214.

…terPoolExecutor` (python#136381) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com> Co-authored-by: Peter Bierma <zintensitydev@gmail.com> Co-authored-by: sobolevn <mail@sobolevn.me>

…nterpreterPoolExecutor` (pythonGH-136381) (python#136420) pythongh-136380: Fix import behavior for `concurrent.futures.InterpreterPoolExecutor` (pythonGH-136381) (cherry picked from commit 490eea0) Co-authored-by: AN Long <aisk@users.noreply.github.com> Co-authored-by: Serhiy Storchaka <storchaka@gmail.com> Co-authored-by: Peter Bierma <zintensitydev@gmail.com> Co-authored-by: sobolevn <mail@sobolevn.me>

Fix import behavior for concurrent.futures.InterpreterPoolExecutor

6ed4774

bedevere-app bot added the awaiting review label Jul 7, 2025

aisk changed the title ~~Fix import behavior for concurrent.futures.InterpreterPoolExecutor~~ gh-136380: Fix import behavior for concurrent.futures.InterpreterPoolExecutor Jul 7, 2025

bedevere-app bot mentioned this pull request Jul 7, 2025

Inconsistent import behavior when concurrent.futures.InterpreterPoolExecutor not exist #136380

Closed

aisk added 2 commits July 7, 2025 22:26

Fix rst marker

cc1a90a

Fix import start

7508c73

serhiy-storchaka reviewed Jul 7, 2025

View reviewed changes

Lib/concurrent/futures/__init__.py Outdated Show resolved Hide resolved

ZeroIntensity reviewed Jul 7, 2025

View reviewed changes

Misc/NEWS.d/next/Library/2025-07-07-22-12-32.gh-issue-136380.1b_nXl.rst Outdated Show resolved Hide resolved

Lib/concurrent/futures/__init__.py Outdated Show resolved Hide resolved

aisk and others added 4 commits July 8, 2025 01:20

Update Lib/concurrent/futures/__init__.py

fe2c921

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

Update Misc/NEWS.d/next/Library/2025-07-07-22-12-32.gh-issue-136380.1…

0f1c7b2

…b_nXl.rst Co-authored-by: Peter Bierma <zintensitydev@gmail.com>

Simplify the try import codes

30a34dd

Add test

1946abd

aisk added 2 commits July 8, 2025 02:01

Fix test

f0c58ae

Fix test

6a17c55

serhiy-storchaka approved these changes Jul 7, 2025

View reviewed changes

Lib/test/test_concurrent_futures/test_interpreter_pool.py Outdated Show resolved Hide resolved

bedevere-app bot added awaiting merge and removed awaiting review labels Jul 7, 2025

serhiy-storchaka reviewed Jul 7, 2025

View reviewed changes

Lib/test/test_concurrent_futures/test_interpreter_pool.py Outdated Show resolved Hide resolved

aisk added 2 commits July 8, 2025 02:17

Better test

9336630

Merge branch 'main' into import-interpreter-pool-executor

7907833

serhiy-storchaka approved these changes Jul 8, 2025

View reviewed changes

sobolevn reviewed Jul 8, 2025

View reviewed changes

Lib/concurrent/futures/__init__.py Outdated Show resolved Hide resolved

Lib/test/test_concurrent_futures/test_interpreter_pool.py Show resolved Hide resolved

Lib/test/test_concurrent_futures/test_interpreter_pool.py Show resolved Hide resolved

Merge branch 'main' into import-interpreter-pool-executor

462426a

sobolevn reviewed Jul 8, 2025

View reviewed changes

Lib/test/test_concurrent_futures/test_interpreter_pool.py Outdated Show resolved Hide resolved

aisk and others added 3 commits July 8, 2025 22:02

Update Lib/concurrent/futures/__init__.py

36f0a0c

Co-authored-by: sobolevn <mail@sobolevn.me>

Update Lib/test/test_concurrent_futures/test_interpreter_pool.py

0a9d5c8

Co-authored-by: sobolevn <mail@sobolevn.me>

Update test

b2e6e69

serhiy-storchaka approved these changes Jul 8, 2025

View reviewed changes

sobolevn approved these changes Jul 8, 2025

View reviewed changes

sobolevn enabled auto-merge (squash) July 8, 2025 13:18

sobolevn added the needs backport to 3.14 bugs and security fixes label Jul 8, 2025

sobolevn merged commit 490eea0 into python:main Jul 8, 2025
42 checks passed

bedevere-app bot removed the awaiting merge label Jul 8, 2025

bedevere-app bot removed the needs backport to 3.14 bugs and security fixes label Jul 8, 2025

aisk deleted the import-interpreter-pool-executor branch July 8, 2025 13:36

henryiii mentioned this pull request Jul 29, 2025

gh-137282: Fix tab completion and dir() on concurrent.futures #137214

Merged

henryiii mentioned this pull request Jul 31, 2025

Tab completion / dir broken on concurrent.futures #137282

Closed

Uh oh!

Conversation

aisk commented Jul 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

serhiy-storchaka left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ZeroIntensity left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

aisk commented Jul 7, 2025

Uh oh!

serhiy-storchaka left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ZeroIntensity commented Jul 8, 2025

Uh oh!

Uh oh!

sobolevn left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

miss-islington-app bot commented Jul 8, 2025

Uh oh!

bedevere-app bot commented Jul 8, 2025

Uh oh!

henryiii commented Jul 29, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

aisk commented Jul 7, 2025 •

edited

Loading