X Tutup
The Wayback Machine - https://web.archive.org/web/20250519072433/https://github.com/python/cpython/issues/68720
Skip to content

asyncio.sock_recv() blocks normal ioloop actions. #68720

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
socketpair mannequin opened this issue Jun 29, 2015 · 7 comments
Closed

asyncio.sock_recv() blocks normal ioloop actions. #68720

socketpair mannequin opened this issue Jun 29, 2015 · 7 comments
Labels
topic-asyncio type-bug An unexpected behavior, bug, or error

Comments

@socketpair
Copy link
Mannequin

socketpair mannequin commented Jun 29, 2015

BPO 24532
Nosy @gvanrossum, @vstinner, @socketpair, @1st1

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2015-06-29.17:25:51.536>
labels = ['type-bug', 'expert-asyncio']
title = 'asyncio.sock_recv() blocks normal ioloop actions.'
updated_at = <Date 2015-06-30.06:17:49.320>
user = 'https://github.com/socketpair'

bugs.python.org fields:

activity = <Date 2015-06-30.06:17:49.320>
actor = 'vstinner'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['asyncio']
creation = <Date 2015-06-29.17:25:51.536>
creator = 'socketpair'
dependencies = []
files = []
hgrepos = []
issue_num = 24532
keywords = []
message_count = 6.0
messages = ['245952', '245957', '245958', '245973', '245974', '245978']
nosy_count = 4.0
nosy_names = ['gvanrossum', 'vstinner', 'socketpair', 'yselivanov']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue24532'
versions = ['Python 3.4']

@socketpair
Copy link
Mannequin Author

socketpair mannequin commented Jun 29, 2015

Suppose that program:

====================================

import asyncio
import socket
def receiver(loop):
    (a, b) = socket.socketpair()
    loop.call_later(1, lambda: print('Should be called inside the loop'))
    end = loop.time() + 3
    print('Starting busy receiver')
    while loop.time() < end:
        a.send(b'test')
        yield from loop.sock_recv(b, 65536)
        # yield from asyncio.sleep(0) # <=====================
    print('Busy receiver complete')
    # just not to stop ioloop immediatelly
    yield from asyncio.sleep(0.5)
def main():
    loop = asyncio.get_event_loop()
    loop.run_until_complete(receiver(loop))
    loop.close()
if __name__ == '__main__':
    main()

====================================

Without asyncio.sleep(0) it will not fire time-delayed calls! If I add asyncio.sleep(0), everything work right. As I think, It is because recv() syscall is always succeeded, and we never return to ioloop (to epoll() I mean).

In other words, it is classical reader starvation as mentioned in "man epoll".

It is not documented, that this function may block event loop, in spite of it returns coroutine! I thought that this function will setup EPOLLIN event for socket's FD + call recv() after that. I spent many time to debug program.

@socketpair socketpair mannequin added topic-asyncio type-bug An unexpected behavior, bug, or error labels Jun 29, 2015
@vstinner
Copy link
Member

You should develop using asyncio debug mode:

https://docs.python.org/dev/library/asyncio-dev.html#asyncio-dev

haypo@selma$ PYTHONASYNCIODEBUG=1 ./python x.py 
Starting busy receiver
Traceback (most recent call last):
  File "x.py", line 21, in <module>
    main()
  File "x.py", line 18, in main
    loop.run_until_complete(receiver(loop))
  File "/home/haypo/prog/python/default/Lib/asyncio/base_events.py", line 341, in run_until_complete
    return future.result()
  File "/home/haypo/prog/python/default/Lib/asyncio/futures.py", line 276, in result
    raise self._exception
  File "/home/haypo/prog/python/default/Lib/asyncio/tasks.py", line 238, in _step
    result = coro.send(value)
  File "x.py", line 11, in receiver
    yield from loop.sock_recv(b, 65536)
  File "/home/haypo/prog/python/default/Lib/asyncio/selector_events.py", line 316, in sock_recv
    raise ValueError("the socket must be non-blocking")
ValueError: the socket must be non-blocking
/home/haypo/prog/python/default/Lib/asyncio/base_events.py:384: ResourceWarning: unclosed event loop <_UnixSelectorEventLoop running=False closed=False debug=True>
sys:1: ResourceWarning: unclosed <socket object at 0x7f0b03d7d688>
sys:1: ResourceWarning: unclosed <socket object at 0x7f0b03d7d5f8>

@1st1
Copy link
Member

1st1 commented Jun 29, 2015

You should develop using asyncio debug mode:

Maybe we should promote this check to the production mode?

@socketpair
Copy link
Mannequin Author

socketpair mannequin commented Jun 30, 2015

Adding of b.setblocking(0) after socketpair() does not help.

@socketpair
Copy link
Mannequin Author

socketpair mannequin commented Jun 30, 2015

$ PYTHONASYNCIODEBUG=1 ./bug.py
Starting busy receiver
Busy receiver complete
Executing <Task pending coro=<receiver() running at ./bug.py:16> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.4/asyncio/tasks.py:490> cb=[_run_until_complete_cb() at /usr/lib/python3.4/asyncio/base_events.py:123] created at /usr/lib/python3.4/asyncio/base_events.py:296> took 3.001 seconds
Should be called inside the loop

@vstinner
Copy link
Member

Le lundi 29 juin 2015, Yury Selivanov <report@bugs.python.org> a écrit :

> You should develop using asyncio debug mode:

Maybe we should promote this check to the production mode?

asyncio must be slow. The check has a cost, I prefer to keep it only in
debug mode. It's probably more a doc issue. We should maybe repeat the info
at the beginning of the asyncio doc.

@ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
@ezio-melotti ezio-melotti moved this to Todo in asyncio Jul 17, 2022
@kumaraditya303
Copy link
Contributor

kumaraditya303 commented Sep 11, 2022

The issue is in your code as it is trying to use blocking sockets in asyncio.

@kumaraditya303 kumaraditya303 closed this as not planned Won't fix, can't repro, duplicate, stale Sep 11, 2022
Repository owner moved this from Todo to Done in asyncio Sep 11, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic-asyncio type-bug An unexpected behavior, bug, or error
Projects
Status: Done
Development

No branches or pull requests

3 participants
X Tutup