X Tutup
The Wayback Machine - https://web.archive.org/web/20221223155659/https://github.com/python/cpython/pull/31913/files
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bpo-47029: Fix BrokenPipeError in multiprocessing.Queue at garbage collection and explicit close #31913

Merged
merged 4 commits into from May 3, 2022
+15 −12
Merged

bpo-47029: Fix BrokenPipeError in multiprocessing.Queue at garbage collection and explicit close #31913

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
d227300
Fix a BrokenPipeError when a multiprocessing.Queue is garbage collected
maggyero Mar 15, 2022
9950aaa
Close self._reader from the queue thread instead of the main thread
maggyero Apr 26, 2022
cf7d8ec
Update queues.py
maggyero Apr 26, 2022
d8e8957
Create 2022-04-26-19-01-13.bpo-47029.qkT42X.rst
maggyero Apr 26, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
23 Lib/multiprocessing/queues.py
View file
Open in desktop
@@ -139,13 +139,10 @@ def put_nowait(self, obj):

def close(self):
self._closed = True
try:
self._reader.close()
finally:
close = self._close
if close:
self._close = None
close()
close = self._close
if close:
self._close = None
close()
maggyero marked this conversation as resolved.
Show resolved Hide resolved

def join_thread(self):
debug('Queue.join_thread()')
@@ -169,8 +166,9 @@ def _start_thread(self):
self._thread = threading.Thread(
target=Queue._feed,
args=(self._buffer, self._notempty, self._send_bytes,
self._wlock, self._writer.close, self._ignore_epipe,
self._on_queue_feeder_error, self._sem),
self._wlock, self._reader.close, self._writer.close,
self._ignore_epipe, self._on_queue_feeder_error,
self._sem),
name='QueueFeederThread'
)
self._thread.daemon = True
@@ -211,8 +209,8 @@ def _finalize_close(buffer, notempty):
notempty.notify()

@staticmethod
def _feed(buffer, notempty, send_bytes, writelock, close, ignore_epipe,
onerror, queue_sem):
def _feed(buffer, notempty, send_bytes, writelock, reader_close,
writer_close, ignore_epipe, onerror, queue_sem):
debug('starting thread to feed data to pipe')
nacquire = notempty.acquire
nrelease = notempty.release
@@ -238,7 +236,8 @@ def _feed(buffer, notempty, send_bytes, writelock, close, ignore_epipe,
obj = bpopleft()
if obj is sentinel:
debug('feeder thread got sentinel -- exiting')
close()
reader_close()
writer_close()
return

# serialize the data before acquiring the lock
4 Misc/NEWS.d/next/Library/2022-04-26-19-01-13.bpo-47029.qkT42X.rst
View file
Open in desktop
@@ -0,0 +1,4 @@
Always close the read end of the pipe used by :class:`multiprocessing.Queue`
*after* the last write of buffered data to the write end of the pipe to avoid
:exc:`BrokenPipeError` at garbage collection and at
:meth:`multiprocessing.Queue.close` calls. Patch by Géry Ogam.
X Tutup