X Tutup
The Wayback Machine - https://web.archive.org/web/20221229210056/https://github.com/python/cpython/pull/97027
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GH-75720: use time.perf_counter in asyncio loop.time() #97027

Closed
wants to merge 2 commits into from

Conversation

kumaraditya303
Copy link
Contributor

@kumaraditya303 kumaraditya303 commented Sep 22, 2022

@kumaraditya303 kumaraditya303 added type-feature A feature request or enhancement expert-asyncio labels Sep 22, 2022
@gvanrossum gvanrossum added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Sep 23, 2022
@bedevere-bot
Copy link

bedevere-bot commented Sep 23, 2022

🤖 New build scheduled with the buildbot fleet by @gvanrossum for commit 1c641ef 🤖

If you want to schedule another build, you need to add the "🔨 test-with-buildbots" label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Sep 23, 2022
@gvanrossum
Copy link
Member

gvanrossum commented Sep 23, 2022

I’m asking for a buildbot test so we’ll know if any exotic systems have weird or broken perf timers.

@gvanrossum
Copy link
Member

gvanrossum commented Sep 23, 2022

A concern I have is, are there systems whose perf timer wraps around within the lifetime of a typical server? That would be a problem…

@graingert
Copy link
Contributor

graingert commented Sep 23, 2022

See also #88494

@vstinner
Copy link
Member

vstinner commented Sep 23, 2022

I’m asking for a buildbot test so we’ll know if any exotic systems have weird or broken perf timers.

Changing time.monotonic() on Windows to use a clock with a better resolution is discussed in #88494 which explains why it was not done yet.

@kumaraditya303
Copy link
Contributor Author

kumaraditya303 commented Sep 23, 2022

I’m asking for a buildbot test so we’ll know if any exotic systems have weird or broken perf timers.

Okay so all buildbots have passed so that's good.

A concern I have is, are there systems whose perf timer wraps around within the lifetime of a typical server? That would be a problem…

As explained on discord, this isn't a problem as there are no wrapping around issues.

@kumaraditya303
Copy link
Contributor Author

kumaraditya303 commented Sep 23, 2022

Changing time.monotonic() on Windows to use a clock with a better resolution is discussed in #88494 which explains why it was not done yet.

This PR is about changing asyncio to use high precision clock, not about changing time.monotonic to use higher precision clock. I prefer to keep these two issues separate, someone else would have to investigate for that issue.

@kumaraditya303
Copy link
Contributor Author

kumaraditya303 commented Sep 23, 2022

@gvanrossum: Trio also always uses time.perf_counter for high precision monotonic clock.

@gvanrossum
Copy link
Member

gvanrossum commented Sep 23, 2022

@kumaraditya303 and @vstinner, unfortunately I don't have time to read #88494, but maybe you can? I'd like an approval from @vstinner if he thinks this is okay for asyncio before I merge this.

@kumaraditya303
Copy link
Contributor Author

kumaraditya303 commented Sep 23, 2022

unfortunately I don't have time to read #88494, but maybe you can? I'd like an approval from @vstinner if he thinks this is okay for asyncio before I merge this.

I don't have time to read the entire #88494, but one thing which stood out from issue is that #85471 was resolved by switching to perf counter on Windows. @vstinner would know more since he worked on the code.
And FWIW Microsoft recommends QueryPerformanceCounter which is what this PR uses.

@kumaraditya303 kumaraditya303 requested a review from vstinner Sep 23, 2022
@graingert
Copy link
Contributor

graingert commented Sep 23, 2022

My understanding of #88494 is that it would fix this and all other uses of time.monotonic and the main thing blocking it is getting a suitable windows 11 development environment available for @vstinner and or @njsmith

This PR looks good to me, but might be worth a docstring change?

@vstinner
Copy link
Member

vstinner commented Sep 23, 2022

I have some worries about using QueryPerformanceCounter() as time.monotonic() on Windows. So far, I'm not convinced that it's safe. The details are in #88494. This issue has a long history and is complicated because it's a complicated problem, and Windows doesn't provide any clock which fits all requirements. Moreover, there are also questions about how clocks behave on Windows 7 and Windows 8, and if Python 3.12 still support these operating systems.

I don't have the bandwidth to dig into this complicated issue. So I cannot review this asyncio change.

@kumaraditya303
Copy link
Contributor Author

kumaraditya303 commented Sep 23, 2022

Moreover, there are also questions about how clocks behave on Windows 7 and Windows 8, and if Python 3.12 still support these operating systems.

All things aside, CPython 3.9+ does not support Windows 7 so it is irrelevant.

@kumaraditya303
Copy link
Contributor Author

kumaraditya303 commented Sep 23, 2022

@zooba Do you have any opinion on this?

@zooba
Copy link
Member

zooba commented Sep 23, 2022

@zooba Do you have any opinion on this?

No, sorry. If you need one from me, I'll have to schedule some research time next week to figure out what my opinion would be.

@kumaraditya303
Copy link
Contributor Author

kumaraditya303 commented Sep 23, 2022

No, sorry. If you need one from me, I'll have to schedule some research time next week to figure out what my opinion would be.

Take your time, there is no rush for this. Thank you!

@kumaraditya303 kumaraditya303 marked this pull request as draft Sep 23, 2022
@kumaraditya303 kumaraditya303 removed the request for review from vstinner Sep 23, 2022
@gvanrossum
Copy link
Member

gvanrossum commented Sep 26, 2022

Had a long chat with Steve Dower about this. We found that there's one outright bug in the PR, it doesn't change how loop._clock_resolution is set.

It looks that since at least Windows 7, perf_counter() is monotonic, so no worries about that.

But there's a possibility that _run_once() and run_forever() may end up spinning when the selector rounds the timeout down. We think that the selector on Windows always uses the monotonic clock with a 15.7 msec resolution, but we don't know yet whether it rounds down.

If it does, maybe it's actually correct, on Windows, to use the resolution of the monotonic clock, but only coincidentally.

I think @zooba will add more color.

@zooba
Copy link
Member

zooba commented Sep 26, 2022

I had a look at this with Guido and have the following suggestions:

  • keep asyncio using time.monotonic by default
  • change time.monotonic on Windows to use QueryPerformanceCounter (which has been monotonic and cheap since Windows 8)
  • add a MINIMUM_SELECT_TIMEOUT constant set to time.get_clock_info("time").resolution to asyncio/base_events.py and use it here to ensure we don't spinlock

The only weirdness is using the time() resolution instead of monotonic() for the minimum timeout, but it's the best available value (and should handle Windows systems where the clock has been modified). We can pass 0 through without rounding up to the minimum, but if we're going to pass anything that rounds down to 0 (i.e. <1ms) we'd better be sure to execute the waiting task at the other end (in the usual case). Right now, this bit will make sure we do that, but that's also leading to tasks being released early (though likely not in real-world cases), so we could drop the extra constant and rely on the minimum time.

If someone uses await sleep(0.010) in a loop, they should get >=10ms worth of delay rather than consuming 100% CPU in a spinlock.

@kumaraditya303
Copy link
Contributor Author

kumaraditya303 commented Sep 27, 2022

keep asyncio using time.monotonic by default

Okay, closing. Thanks for the info.

@kumaraditya303 kumaraditya303 deleted the perf-counter branch Sep 27, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants
X Tutup