gh-127371 Avoid unbounded growth SpooledTempfile.writelines by bertptrs · Pull Request #127372 · python/cpython

bertptrs · 2024-11-28T18:17:10Z

By checking after every line whether we've rolled over, we ensure that internal buffer doesn't grow too far beyond the expected bounds (at most one line).

To avoid a performance regression, I opted to introduce several fast-paths: if the SpooledTemplFile has already been rolled over to disk, or if it never will because the size is unbounded, the implementation delegates directly to the underlying file object.

Issue: Unbounded growth in SpooledTemporaryFile.writelines() #127371

ghost · 2024-11-28T18:17:13Z

All commit authors signed the Contributor License Agreement.

bedevere-app · 2024-11-28T18:17:14Z

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

tomasr8

Thanks! I think this also needs a test :)

Lib/tempfile.py

Misc/NEWS.d/next/Security/2024-11-28-20-29-21.gh-issue-127371.PeEhUd.rst

bertptrs · 2024-11-29T08:03:56Z

Thanks for the review! Addressed your comments and added a simple test.

tomasr8 · 2024-11-29T08:07:13Z

Thanks! Please avoid force pushing if possible, it makes it more difficult to see what the latest changes are 🙂

As the bot said, you'll also need to sign the Contributor License Agreement.

Lib/test/test_tempfile.py

Misc/NEWS.d/next/Security/2024-11-28-20-29-21.gh-issue-127371.PeEhUd.rst

Lib/tempfile.py

bertptrs · 2024-11-30T11:48:08Z

I think it should be good like this, then. Note: I'm aware of the failing CI test for my news entry, but I don't know what's wrong about the way I linked the method. Could you help me with that?

tomasr8 · 2024-11-30T18:19:34Z

I think it should be good like this, then. Note: I'm aware of the failing CI test for my news entry, but I don't know what's wrong about the way I linked the method. Could you help me with that?

I think that's because the method is not documented so there is nothing to link to. IIRC ! should fix it, i.e.:

:meth:`!tempfile.SpooledTemporaryFile.writelines`

tomasr8

LGTM! :)

bertptrs · 2025-01-18T11:16:28Z

Hey, it's been a while, is there anything I need to do to get a core review, or any changes I could make to make it easier to merge?

tomasr8 · 2025-01-18T12:55:54Z

Serhiy might have time :) cc @serhiy-storchaka

bertptrs · 2025-03-01T09:22:00Z

Hey, it's been another month and a bit, could this still get a review?

Misc/NEWS.d/next/Security/2024-11-28-20-29-21.gh-issue-127371.PeEhUd.rst

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>

picnixz · 2025-03-01T09:32:00Z

You should merge main into this branch because of some new CI configuration (just hit the "Update branch" button)

bertptrs · 2025-03-01T09:35:56Z

@picnixz do you prefer I rebase it (squashing some fixups along the way) or merge main into this branch? I was previously requested to not force-push, but merging main will also make the history unclear.

picnixz · 2025-03-01T09:38:49Z

Just merge main by hitting the "Update branch" button. We generally want to avoid force pushes because it makes incremental reviews much harder. We anyway squash-merge at the end, so there's no need for a clean commit history on the PR in general. What's important is to make reviews simpler and easier to follow.

Note that hitting "Update branch" only merges main with a single commit so it's fine.
Afterwards, just commit your next fixes

picnixz · 2025-03-01T09:43:11Z

FTR, there are situations when force-pushing is necessary. I've listed them in some PR, but I'll list them here again:

If you need to change the commit information (author, email, etc) for the CLA bot to be happy.
If you incorrectly merged something locally and did something wrong.
When you've opened an unreviewed draft PR. You can do whatever you want in general, reviews are not meant to be yet submitted, and if they are, it's the reviewer's responsibility to know that the PR was not yet ready.
When you really want to squash commits together for some reasons, or if a commit itself is messed up, or if a commit should be dropped because it wouldn't ease reviews. Or to reword a commit message because it's wrong.

In general, only the two first situations warrant a force-push. I force-push a lot in a draft PR if no review has been made yet, but otherwise, once a review was done, I generally avoid force-pushing. I however rebase a lot my local PRs before they're submitted upstream.

bertptrs · 2025-03-01T10:25:35Z

Thank you for explaining. My employer and other projects I work on have a radically different view on the same, so I'm just trying to understand how the commit culture is here and adhere to it. Please don't take my questions as criticism.

picnixz · 2025-03-01T10:33:49Z

My employer and other projects I work on have a radically different view on the same, so I'm just trying to understand how the commit culture is here and adhere to it. Please don't take my questions as criticism.

No worries. It's always good to ask when one's unsure! Every project has different guidelines. We should probably expand the https://devguide.python.org/ to indicate when force-pushing is allowed and when/why it's not good otherwise.

EDIT: we do have a small note but it's kind of burried:

In order to keep the commit history intact, please avoid squashing or amending history and then force-pushing to the PR. Reviewers often want to look at individual commits.

serhiy-storchaka

LGTM.

Small suggestion to reduce overhead and avoid behavior change for weird iterators.

Lib/tempfile.py

serhiy-storchaka

LGTM.

miss-islington-app · 2025-03-05T15:42:19Z

Thanks @bertptrs for the PR, and @serhiy-storchaka for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖

miss-islington-app · 2025-03-05T15:42:19Z

Thanks @bertptrs for the PR, and @serhiy-storchaka for merging it 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖

…thonGH-127372) (cherry picked from commit cb67b44) Co-authored-by: Bert Peters <bert@bertptrs.nl>

bedevere-app · 2025-03-05T15:42:33Z

GH-130885 is a backport of this pull request to the 3.12 branch.

…thonGH-127372) (cherry picked from commit cb67b44) Co-authored-by: Bert Peters <bert@bertptrs.nl>

bedevere-app · 2025-03-05T15:42:41Z

GH-130886 is a backport of this pull request to the 3.13 branch.

…H-127372) (#130885) gh-127371 Avoid unbounded growth SpooledTempfile.writelines (GH-127372) (cherry picked from commit cb67b44) Co-authored-by: Bert Peters <bert@bertptrs.nl>

…H-127372) (#130886) gh-127371 Avoid unbounded growth SpooledTempfile.writelines (GH-127372) (cherry picked from commit cb67b44) Co-authored-by: Bert Peters <bert@bertptrs.nl>

bedevere-bot · 2025-03-05T21:52:31Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x RHEL9 Refleaks 3.13 (tier-3) has failed when building commit 39f7b06.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1575/builds/562) and take a look at the build logs.
Check if the failure is related to this commit (39f7b06) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1575/builds/562

Failed tests:

test.test_multiprocessing_spawn.test_manager

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/managers.py", line 265, in serve_client
    raise ke
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/managers.py", line 259, in serve_client
    obj, exposed, gettypeid = id_to_obj[ident]
                              ~~~~~~~~~^^^^^^^
KeyError: '3ff9c8a4ed0'
---------------------------------------------------------------------------
Timeout (0:45:00)!
Thread 0x000003ffb8373740 (most recent call first):
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/popen_fork.py", line 28 in poll
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/popen_fork.py", line 44 in wait
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/process.py", line 149 in join
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/unittest/case.py", line 614 in _callCleanup
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/unittest/case.py", line 688 in doCleanups
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/unittest/case.py", line 655 in run
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/unittest/case.py", line 707 in __call__
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/unittest/suite.py", line 122 in run
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/unittest/suite.py", line 84 in __call__
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/unittest/suite.py", line 122 in run
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/unittest/suite.py", line 84 in __call__
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/unittest/runner.py", line 240 in run
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/test/libregrtest/single.py", line 57 in _run_suite
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/test/libregrtest/single.py", line 37 in run_unittest
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/test/libregrtest/single.py", line 135 in test_func
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/test/libregrtest/refleak.py", line 132 in runtest_refleak
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/test/libregrtest/single.py", line 87 in regrtest_runner
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/test/libregrtest/single.py", line 138 in _load_run_test
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/test/libregrtest/single.py", line 181 in _runtest_env_changed_exc
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/test/libregrtest/single.py", line 281 in _runtest
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/test/libregrtest/single.py", line 310 in run_single_test
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/test/libregrtest/worker.py", line 77 in worker_process
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/test/libregrtest/worker.py", line 112 in main
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/test/libregrtest/worker.py", line 116 in <module>
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/runpy.py", line 88 in _run_code
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/runpy.py", line 198 in _run_module_as_main


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/process.py", line 313, in _bootstrap
    self.run()
    ~~~~~~~~^^
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/test/_test_multiprocessing.py", line 1581, in f
    cond.release()
    ~~~~~~~~~~~~^^
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/managers.py", line 1066, in release
    return self._callmethod('release')
           ~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/managers.py", line 847, in _callmethod
    raise convert_to_error(kind, result)
multiprocessing.managers.RemoteError: 
---------------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/managers.py", line 263, in serve_client
    self.id_to_local_proxy_obj[ident]
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^
KeyError: '3ff74338420'


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/process.py", line 313, in _bootstrap
    self.run()
    ~~~~~~~~^^
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/test/_test_multiprocessing.py", line 1580, in f
    woken.release()
    ~~~~~~~~~~~~~^^
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/managers.py", line 1066, in release
    return self._callmethod('release')
           ~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/managers.py", line 847, in _callmethod
    raise convert_to_error(kind, result)
multiprocessing.managers.RemoteError: 
---------------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/managers.py", line 263, in serve_client
    self.id_to_local_proxy_obj[ident]
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^
KeyError: '3ff988a4ed0'


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/process.py", line 313, in _bootstrap
    self.run()
    ~~~~~~~~^^
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/test/_test_multiprocessing.py", line 1580, in f
    woken.release()
    ~~~~~~~~~~~~~^^
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/managers.py", line 1066, in release
    return self._callmethod('release')
           ~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/managers.py", line 847, in _callmethod
    raise convert_to_error(kind, result)
multiprocessing.managers.RemoteError: 
---------------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/managers.py", line 263, in serve_client
    self.id_to_local_proxy_obj[ident]
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^
KeyError: '3ff9c8a4ed0'


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/process.py", line 313, in _bootstrap
    self.run()
    ~~~~~~~~^^
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/test/_test_multiprocessing.py", line 1580, in f
    woken.release()
    ~~~~~~~~~~~~~^^
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/managers.py", line 1066, in release
    return self._callmethod('release')
           ~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/managers.py", line 847, in _callmethod
    raise convert_to_error(kind, result)
multiprocessing.managers.RemoteError: 
---------------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/managers.py", line 263, in serve_client
    self.id_to_local_proxy_obj[ident]
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^
KeyError: '3ff8dca4ed0'


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/managers.py", line 265, in serve_client
    raise ke
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/managers.py", line 259, in serve_client
    obj, exposed, gettypeid = id_to_obj[ident]
                              ~~~~~~~~~^^^^^^^
KeyError: '3ff988a4ed0'
---------------------------------------------------------------------------
k


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/process.py", line 313, in _bootstrap
    self.run()
    ~~~~~~~~^^
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/test/_test_multiprocessing.py", line 1581, in f
    cond.release()
    ~~~~~~~~~~~~^^
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/managers.py", line 1066, in release
    return self._callmethod('release')
           ~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/managers.py", line 847, in _callmethod
    raise convert_to_error(kind, result)
multiprocessing.managers.RemoteError: 
---------------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/managers.py", line 263, in serve_client
    self.id_to_local_proxy_obj[ident]
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^
KeyError: '3ff74c934e0'


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/managers.py", line 265, in serve_client
    raise ke
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/managers.py", line 259, in serve_client
    obj, exposed, gettypeid = id_to_obj[ident]
                              ~~~~~~~~~^^^^^^^
KeyError: '3ff74338420'
---------------------------------------------------------------------------
k


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/managers.py", line 265, in serve_client
    raise ke
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/managers.py", line 259, in serve_client
    obj, exposed, gettypeid = id_to_obj[ident]
                              ~~~~~~~~~^^^^^^^
KeyError: '3ff74c934e0'
---------------------------------------------------------------------------
k


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/managers.py", line 265, in serve_client
    raise ke
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/managers.py", line 259, in serve_client
    obj, exposed, gettypeid = id_to_obj[ident]
                              ~~~~~~~~~^^^^^^^
KeyError: '3ff8dca4ed0'
---------------------------------------------------------------------------
Timeout (0:45:00)!
Thread 0x000003ffaf1f3740 (most recent call first):
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/popen_fork.py", line 28 in poll
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/popen_fork.py", line 44 in wait
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/process.py", line 149 in join
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/unittest/case.py", line 614 in _callCleanup
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/unittest/case.py", line 688 in doCleanups
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/unittest/case.py", line 655 in run
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/unittest/case.py", line 707 in __call__
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/unittest/suite.py", line 122 in run
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/unittest/suite.py", line 84 in __call__
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/unittest/suite.py", line 122 in run
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/unittest/suite.py", line 84 in __call__
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/unittest/runner.py", line 240 in run
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/test/libregrtest/single.py", line 57 in _run_suite
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/test/libregrtest/single.py", line 37 in run_unittest
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/test/libregrtest/single.py", line 135 in test_func
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/test/libregrtest/refleak.py", line 132 in runtest_refleak
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/test/libregrtest/single.py", line 87 in regrtest_runner
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/test/libregrtest/single.py", line 138 in _load_run_test
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/test/libregrtest/single.py", line 181 in _runtest_env_changed_exc
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/test/libregrtest/single.py", line 281 in _runtest
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/test/libregrtest/single.py", line 310 in run_single_test
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/test/libregrtest/worker.py", line 77 in worker_process
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/test/libregrtest/worker.py", line 112 in main
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/test/libregrtest/worker.py", line 116 in <module>
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/runpy.py", line 88 in _run_code
  File "/home/buildbot/buildarea/3.13.cstratak-rhel9-s390x.refleak/build/Lib/runpy.py", line 198 in _run_module_as_main

bedevere-app bot mentioned this pull request Nov 28, 2024

Unbounded growth in SpooledTemporaryFile.writelines() #127371

Closed

bedevere-app bot added the awaiting review label Nov 28, 2024

tomasr8 reviewed Nov 28, 2024

View reviewed changes

Lib/tempfile.py Outdated Show resolved Hide resolved

Misc/NEWS.d/next/Security/2024-11-28-20-29-21.gh-issue-127371.PeEhUd.rst Outdated Show resolved Hide resolved

bertptrs and others added 3 commits November 29, 2024 09:03

Avoid unbounded growth SpooledTempfile.writelines

917df7d

📜🤖 Added by blurb_it.

5ad82d4

Ensure that SpooledTemporaryFile rolls over in writelines

b4dd28c

bertptrs force-pushed the bounded-spooled-writelines branch from 751c78f to b4dd28c Compare November 29, 2024 08:03

bertptrs requested a review from tomasr8 November 29, 2024 08:03

tomasr8 reviewed Nov 29, 2024

View reviewed changes

Lib/test/test_tempfile.py Show resolved Hide resolved

Misc/NEWS.d/next/Security/2024-11-28-20-29-21.gh-issue-127371.PeEhUd.rst Outdated Show resolved Hide resolved

Lib/tempfile.py Show resolved Hide resolved

bertptrs mentioned this pull request Nov 29, 2024

gh-127371: Fix high memory usage in SpooledTemporaryFile.writelines() when handling large iterators #127378

Closed

bertptrs added 2 commits November 30, 2024 12:28

fixup! 📜🤖 Added by blurb_it.

07d2a66

fixup! Ensure that SpooledTemporaryFile rolls over in writelines

9177f34

fixup! fixup! 📜🤖 Added by blurb_it.

d685dfb

tomasr8 approved these changes Dec 1, 2024

View reviewed changes

bedevere-app bot added awaiting core review and removed awaiting review labels Dec 1, 2024

picnixz requested a review from serhiy-storchaka March 1, 2025 09:27

picnixz reviewed Mar 1, 2025

View reviewed changes

Misc/NEWS.d/next/Security/2024-11-28-20-29-21.gh-issue-127371.PeEhUd.rst Outdated Show resolved Hide resolved

fixup! fixup! fixup! 📜🤖 Added by blurb_it.

b50e034

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>

Merge branch 'main' into bounded-spooled-writelines

e00b96a

serhiy-storchaka reviewed Mar 3, 2025

View reviewed changes

Lib/tempfile.py Outdated Show resolved Hide resolved

Lib/tempfile.py Outdated Show resolved Hide resolved

fixup! fixup! Ensure that SpooledTemporaryFile rolls over in writelines

ee6db45

bertptrs requested a review from serhiy-storchaka March 4, 2025 20:45

serhiy-storchaka approved these changes Mar 4, 2025

View reviewed changes

bedevere-app bot added awaiting merge and removed awaiting core review labels Mar 4, 2025

serhiy-storchaka merged commit cb67b44 into python:main Mar 4, 2025
43 checks passed

bedevere-app bot removed the awaiting merge label Mar 4, 2025

Yhg1s added 3.12 only security fixes 3.13 bugs and security fixes needs backport to 3.12 only security fixes needs backport to 3.13 bugs and security fixes and removed 3.12 only security fixes 3.13 bugs and security fixes labels Mar 5, 2025

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Mar 5, 2025

pythongh-127371 Avoid unbounded growth SpooledTempfile.writelines (py…

05290bc

…thonGH-127372) (cherry picked from commit cb67b44) Co-authored-by: Bert Peters <bert@bertptrs.nl>

bedevere-app bot removed the needs backport to 3.12 only security fixes label Mar 5, 2025

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Mar 5, 2025

pythongh-127371 Avoid unbounded growth SpooledTempfile.writelines (py…

54b4f9d

…thonGH-127372) (cherry picked from commit cb67b44) Co-authored-by: Bert Peters <bert@bertptrs.nl>

bedevere-app bot removed the needs backport to 3.13 bugs and security fixes label Mar 5, 2025

Uh oh!

Conversation

bertptrs commented Nov 28, 2024 • edited by bedevere-app bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ghost commented Nov 28, 2024 • edited by ghost Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bedevere-app bot commented Nov 28, 2024

Uh oh!

tomasr8 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

bertptrs commented Nov 29, 2024

Uh oh!

tomasr8 commented Nov 29, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bertptrs commented Nov 30, 2024

Uh oh!

tomasr8 commented Nov 30, 2024

Uh oh!

tomasr8 left a comment

Choose a reason for hiding this comment

Uh oh!

bertptrs commented Jan 18, 2025

Uh oh!

tomasr8 commented Jan 18, 2025

Uh oh!

bertptrs commented Mar 1, 2025

Uh oh!

Uh oh!

picnixz commented Mar 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bertptrs commented Mar 1, 2025

Uh oh!

picnixz commented Mar 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

picnixz commented Mar 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bertptrs commented Mar 1, 2025

Uh oh!

picnixz commented Mar 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

serhiy-storchaka left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

serhiy-storchaka left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

miss-islington-app bot commented Mar 5, 2025

Uh oh!

miss-islington-app bot commented Mar 5, 2025

Uh oh!

bedevere-app bot commented Mar 5, 2025

Uh oh!

bedevere-app bot commented Mar 5, 2025

Uh oh!

bedevere-bot commented Mar 5, 2025

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

bertptrs commented Nov 28, 2024 •

edited by bedevere-app bot

Loading

ghost commented Nov 28, 2024 •

edited by ghost

Loading

picnixz commented Mar 1, 2025 •

edited

Loading

picnixz commented Mar 1, 2025 •

edited

Loading

picnixz commented Mar 1, 2025 •

edited

Loading

picnixz commented Mar 1, 2025 •

edited

Loading