gh-142737: Handle lost `io.open` in `_Py_FindSourceFile` by johnslavik · Pull Request #142747 · python/cpython

johnslavik · 2025-12-15T12:55:12Z

This fixes the problem in the issue.
WIP: a regrtest is needed.

Issue: assertion failure at Objects/call.c:618: PyObject *callmethod(PyThreadState *, PyObject *, const char *, struct __va_list_tag *): Assertion 'callable != NULL' failed. #142737

johnslavik · 2025-12-15T13:30:09Z

Testing this will be harder because we're in a scenario where display_source_line is fully broken, and the C impl of traceback doesn't use linecache. So need to fix up tests to be able to not expect lines displayed.

johnslavik · 2025-12-15T13:53:49Z

Not sure about whether reusing TracebackFormatMixin is a good idea. These tests are specifically built on an assumption that lines will appear in the tracebacks.

Objects/call.c

Python/traceback.c

Lib/test/test_traceback.py

Misc/NEWS.d/next/Core_and_Builtins/2025-12-15-15-01-21.gh-issue-142737.xYXzeB.rst

johnslavik · 2025-12-15T14:51:29Z

For some reason the test doesn't crash for a build without the fix (rc is 1, not 134; stderr is as expected after fix).
Outside the test, the snippet from the test does crash for the old build.

I'll need to tweak the test and we're ready.

johnslavik · 2025-12-15T15:33:52Z

Oh, I think I know why. It doesn't look for source when we pass the code via -c.

ZeroIntensity · 2025-12-15T16:13:15Z

And perhaps we should put an assert(callable != NULL) in _PyObject_CallMethodFormat, what do you think? It relies on that assumption.

I was going to suggest that, go for it.

johnslavik · 2025-12-15T20:41:19Z

@ZeroIntensity ready

Lib/test/test_traceback.py

ZeroIntensity

LGTM!

miss-islington-app · 2025-12-15T22:58:54Z

Thanks @johnslavik for the PR, and @ZeroIntensity for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13, 3.14.
🐍🍒⛏🤖

…nGH-142747) (cherry picked from commit f277781) Co-authored-by: Bartosz Sławecki <bartosz@ilikepython.com>

miss-islington-app · 2025-12-15T22:59:02Z

Sorry, @johnslavik and @ZeroIntensity, I could not cleanly backport this to 3.13 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker f277781bba684322dffffe45cd878f4652ccf7e4 3.13

bedevere-bot · 2025-12-15T23:00:55Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot wasm32-wasi 3.x (tier-2) has failed when building commit f277781.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1046/builds/10487) and take a look at the build logs.
Check if the failure is related to this commit (f277781) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1046/builds/10487

Summary of the results of the build (if available):

Click to see traceback logs

remote: Enumerating objects: 14, done.        
remote: Counting objects:   9% (1/11)        
remote: Counting objects:  18% (2/11)        
remote: Counting objects:  27% (3/11)        
remote: Counting objects:  36% (4/11)        
remote: Counting objects:  45% (5/11)        
remote: Counting objects:  54% (6/11)        
remote: Counting objects:  63% (7/11)        
remote: Counting objects:  72% (8/11)        
remote: Counting objects:  81% (9/11)        
remote: Counting objects:  90% (10/11)        
remote: Counting objects: 100% (11/11)        
remote: Counting objects: 100% (11/11), done.        
remote: Compressing objects:  12% (1/8)        
remote: Compressing objects:  25% (2/8)        
remote: Compressing objects:  37% (3/8)        
remote: Compressing objects:  50% (4/8)        
remote: Compressing objects:  62% (5/8)        
remote: Compressing objects:  75% (6/8)        
remote: Compressing objects:  87% (7/8)        
remote: Compressing objects: 100% (8/8)        
remote: Compressing objects: 100% (8/8), done.        
remote: Total 14 (delta 3), reused 3 (delta 3), pack-reused 3 (from 2)        
From https://github.com/python/cpython
 * branch                    main       -> FETCH_HEAD
Note: switching to 'f277781bba684322dffffe45cd878f4652ccf7e4'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at f277781bba6 gh-142737: Handle lost `io.open` in `_Py_FindSourceFile` (GH-142747)
Switched to and reset branch 'main'

⚠️ WARNING: This script is deprecated and slated for removal in Python 3.20; execute the `wasi/` directory instead (i.e. `python Tools/wasm/wasi`)

configure: WARNING: no system libmpdec found; falling back to pure-Python version for the decimal module

⚠️ WARNING: This script is deprecated and slated for removal in Python 3.20; execute the `wasi/` directory instead (i.e. `python Tools/wasm/wasi`)


⚠️ WARNING: This script is deprecated and slated for removal in Python 3.20; execute the `wasi/` directory instead (i.e. `python Tools/wasm/wasi`)

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.bcannon-wasi.wasi.debug/build/Tools/wasm/wasi.py", line 12, in <module>
    runpy.run_path(pathlib.Path(__file__).parent / "wasi", run_name="__main__")
  File "<frozen runpy>", line 304, in run_path
  File "<frozen runpy>", line 88, in _run_code
  File "/home/buildbot/buildarea/3.x.bcannon-wasi.wasi.debug/build/Tools/wasm/wasi/__main__.py", line 554, in <module>
    main()
  File "/home/buildbot/buildarea/3.x.bcannon-wasi.wasi.debug/build/Tools/wasm/wasi/__main__.py", line 550, in main
    dispatch[context.subcommand](context)
  File "/home/buildbot/buildarea/3.x.bcannon-wasi.wasi.debug/build/Tools/wasm/wasi/__main__.py", line 123, in wrapper
    return func(context, working_dir)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.bcannon-wasi.wasi.debug/build/Tools/wasm/wasi/__main__.py", line 340, in configure_wasi_python
    raise FileNotFoundError(
FileNotFoundError: wasmtime not found; download from https://github.com/bytecodealliance/wasmtime

⚠️ WARNING: This script is deprecated and slated for removal in Python 3.20; execute the `wasi/` directory instead (i.e. `python Tools/wasm/wasi`)

bedevere-app · 2025-12-15T23:01:55Z

GH-142773 is a backport of this pull request to the 3.14 branch.

…nGH-142747) (cherry picked from commit f277781)

…pythonGH-142747) (cherry picked from commit f277781) Co-authored-by: Bartosz Sławecki <bartosz@ilikepython.com>

bedevere-app · 2025-12-15T23:04:51Z

GH-142774 is a backport of this pull request to the 3.13 branch.

…42747) (GH-142774) (cherry picked from commit f277781)

…42747) (GH-142773) gh-142737: Handle lost `io.open` in `_Py_FindSourceFile` (GH-142747) (cherry picked from commit f277781) Co-authored-by: Bartosz Sławecki <bartosz@ilikepython.com>

…nGH-142747)

Check callable isn't NULL in _PyObject_CallMethodFormat

cf8b9c3

bedevere-app bot mentioned this pull request Dec 15, 2025

assertion failure at Objects/call.c:618: PyObject *callmethod(PyThreadState *, PyObject *, const char *, struct __va_list_tag *): Assertion 'callable != NULL' failed. #142737

Closed

Add initial test idea

1cb2a7e

johnslavik force-pushed the check-callable-callmethodformat branch from c74ce79 to 1cb2a7e Compare December 15, 2025 13:33

Give test cases meaningful names

e5c77a0

johnslavik force-pushed the check-callable-callmethodformat branch from a95f248 to e5c77a0 Compare December 15, 2025 13:37

Skip tests that require lines to be captured

173fd5c

ZeroIntensity reviewed Dec 15, 2025

View reviewed changes

Objects/call.c Outdated Show resolved Hide resolved

johnslavik added 2 commits December 15, 2025 15:02

Add news entry

6b81561

Goto error immediately if we can't get handle to io.open

323385d

johnslavik requested a review from ZeroIntensity December 15, 2025 14:09

johnslavik changed the title ~~gh-142737: Check callable isn't NULL in _PyObject_CallMethodFormat~~ gh-142737: Handle lost io.open in _Py_FindSourceFile Dec 15, 2025

johnslavik commented Dec 15, 2025

View reviewed changes

Python/traceback.c Show resolved Hide resolved

Clarify skip reason

092356b

ZeroIntensity reviewed Dec 15, 2025

View reviewed changes

Lib/test/test_traceback.py Outdated Show resolved Hide resolved

Lib/test/test_traceback.py Outdated Show resolved Hide resolved

Misc/NEWS.d/next/Core_and_Builtins/2025-12-15-15-01-21.gh-issue-142737.xYXzeB.rst Show resolved Hide resolved

ZeroIntensity added needs backport to 3.13 bugs and security fixes needs backport to 3.14 bugs and security fixes labels Dec 15, 2025

johnslavik added 3 commits December 15, 2025 15:24

Add some docs & comments to tests

1a971f4

Elaborate in the news entry

206633c

Use a subprocess-based test

d95a3d4

johnslavik force-pushed the check-callable-callmethodformat branch from fbeeba3 to d95a3d4 Compare December 15, 2025 14:43

Examine rc to make sure it's not a crash

36a91ad

Simplify minimal reproducer

b3d1296

johnslavik added 2 commits December 15, 2025 16:45

Fiat lux

e0933bb

Fix formatting

6589e9f

johnslavik force-pushed the check-callable-callmethodformat branch from d5700c9 to 6589e9f Compare December 15, 2025 15:48

Apply suggestion from @johnslavik

77effbc

Add assert(callable != NULL) in _PyObject_CallMethodFormat

45774d2

johnslavik requested a review from markshannon as a code owner December 15, 2025 20:00

ZeroIntensity reviewed Dec 15, 2025

View reviewed changes

Lib/test/test_traceback.py Show resolved Hide resolved

Mark as CPython-only and mention _Py_FindSourceFile

ae292f5

johnslavik commented Dec 15, 2025

View reviewed changes

Lib/test/test_traceback.py Outdated Show resolved Hide resolved

Better wording in the comment

f9aae29

johnslavik requested a review from ZeroIntensity December 15, 2025 22:30

ZeroIntensity approved these changes Dec 15, 2025

View reviewed changes

bedevere-app bot added awaiting merge and removed awaiting review labels Dec 15, 2025

ZeroIntensity enabled auto-merge (squash) December 15, 2025 22:35

ZeroIntensity merged commit f277781 into python:main Dec 15, 2025
50 checks passed

bedevere-app bot removed the awaiting merge label Dec 15, 2025

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Dec 15, 2025

pythongh-142737: Handle lost io.open in _Py_FindSourceFile (pytho…

fa92604

…nGH-142747) (cherry picked from commit f277781) Co-authored-by: Bartosz Sławecki <bartosz@ilikepython.com>

miss-islington-app bot assigned ZeroIntensity Dec 15, 2025

bedevere-app bot removed the needs backport to 3.14 bugs and security fixes label Dec 15, 2025

johnslavik added a commit to johnslavik/cpython that referenced this pull request Dec 15, 2025

pythongh-142737: Handle lost io.open in _Py_FindSourceFile (pytho…

cb7317e

…nGH-142747) (cherry picked from commit f277781)

johnslavik added a commit to johnslavik/cpython that referenced this pull request Dec 15, 2025

[3.13] pythongh-142737: Handle lost io.open in _Py_FindSourceFile (…

4f409f8

…pythonGH-142747) (cherry picked from commit f277781) Co-authored-by: Bartosz Sławecki <bartosz@ilikepython.com>

bedevere-app bot removed the needs backport to 3.13 bugs and security fixes label Dec 15, 2025

ZeroIntensity pushed a commit that referenced this pull request Dec 15, 2025

[3.13] gh-142737: Handle lost io.open in _Py_FindSourceFile (GH-1…

3e45440

…42747) (GH-142774) (cherry picked from commit f277781)

fatelei pushed a commit to fatelei/cpython that referenced this pull request Dec 16, 2025

pythongh-142737: Handle lost io.open in _Py_FindSourceFile (pytho…

63ba8e6

…nGH-142747)

Uh oh!

Conversation

johnslavik commented Dec 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

johnslavik commented Dec 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

johnslavik commented Dec 15, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

johnslavik commented Dec 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

johnslavik commented Dec 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ZeroIntensity commented Dec 15, 2025

Uh oh!

johnslavik commented Dec 15, 2025

Uh oh!

Uh oh!

Uh oh!

ZeroIntensity left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

miss-islington-app bot commented Dec 15, 2025

Uh oh!

miss-islington-app bot commented Dec 15, 2025

Uh oh!

bedevere-bot commented Dec 15, 2025

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

bedevere-app bot commented Dec 15, 2025

Uh oh!

bedevere-app bot commented Dec 15, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

johnslavik commented Dec 15, 2025 •

edited

Loading

johnslavik commented Dec 15, 2025 •

edited

Loading

johnslavik commented Dec 15, 2025 •

edited

Loading

johnslavik commented Dec 15, 2025 •

edited

Loading