X Tutup
The Wayback Machine - https://web.archive.org/web/20240116104349/https://github.com/python/cpython/issues/114104
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Not all generators containing await are compiled to async generators #114104

Open
mniip opened this issue Jan 16, 2024 · 1 comment
Open

Not all generators containing await are compiled to async generators #114104

mniip opened this issue Jan 16, 2024 · 1 comment
Labels
docs Documentation in the Doc dir

Comments

@mniip
Copy link

mniip commented Jan 16, 2024

Documentation

A generator of the form f(x) for x in await a first awaits a and then creates a regular sync generator, as can be seen in the following minimal reproducer:

import asyncio
async def foo():
    print("Entered foo")
    return [1, 2, 3]
async def main():
    gen = (x for x in await foo())
    print(gen)
    print(list(gen))
asyncio.run(main())
Entered foo
<generator object main.<locals>.<genexpr> at 0x7f8fce7f8110>
[1, 2, 3]

However the python language reference 6.2.8. Generator expressions states:

If a generator expression contains either async for clauses or await expressions it is called an asynchronous generator expression. An asynchronous generator expression returns a new asynchronous generator object, which is an asynchronous iterator (see Asynchronous Iterators).

This seems to suggest that the generator f(x) for x in await a does contain an await expression, so it should become an async_generator? However there is also:

However, the iterable expression in the leftmost for clause is immediately evaluated, so that an error produced by it will be emitted at the point where the generator expression is defined, rather than at the point where the first value is retrieved.

This seems to hint at an implementation detail that the leftmost for clause has special treatment and is always evaluated before the generator is constructed, but the interactions with await are unclear from this, especially whether this disqualifies the generator from becoming async.

I don't know whether this is an undocumented feature or a bug.

@serhiy-storchaka
Copy link
Member

cc @gvanrossum

I think that it is easier to make the documentation more explicit. The iterable expression in the leftmost for clause is not included in a generator expression. It is evaluated (and implicit iter() is called) before entering a generator function, so it is not a part of the generator function and does not make it asynchronous.

We can, of cause, change the implementation, but what is the use case for this? If you want to make an asynchronous generator and await foo() in it, you can use the following workaround:

gen = (x for _ in [1] for x in await foo())

But it is more clear to write an inner function:

async def gen():
    for x in await foo():
        yield x

In this way you can even make an asynchronous generator that does not contain any await.

async def gen():
    for x in sync_foo():
        yield x

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs Documentation in the Doc dir
Projects
None yet
Development

No branches or pull requests

2 participants
X Tutup