New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SSLContext.set_default_verify_paths holds GIL for > 1s #94637
Comments
|
Yes, it is safe. I wonder, why You can also optimize the code and use a single SSLContext object for all network code. It is safe to share a single instance across threads if you don't modify the object (load more certs, change verification parameters, options, etc.). |
Good to know; thanks! Even with the GIL-related hitches fixed, that would probably be worth it for me to do to shave that time off of individual network round trips. I'm assuming this would be done by explicitly passing in the SSLContext for each particular use? Or is there any higher level way to get things to share a single context by default? And on a related note, is there any clean way to point all contexts at a root cert file by default? Since Android doesn't seem to have certs in a consistent place I'm bundling ones from the Python certifi package and using the SSL_CERT_FILE env var to point SSL at them, which gets them picked up by all contexts, but I didn't know if there's a more 'correct' way to do that sort of thing. |
|
All stdlib network modules accept a I recommend that users always create a context with |
|
Ok; sounds good. I'll just pass in an explicit shared context created from One obscure note in case its helpful: on my Android OpenSSL build, SSL_CERT_FILE wasn't actually getting read by default. OpenSSL does a |
…ythonGH-94658) (cherry picked from commit 78307c7) Co-authored-by: Christian Heimes <christian@python.org>
…ythonGH-94658) (cherry picked from commit 78307c7) Co-authored-by: Christian Heimes <christian@python.org>


Hi; I'm the author of a game engine which makes heavy use of Python. The engine sends various network requests in background threads for various purposes. Recently I switched these to use https instead of http, and soon after noticed my Android builds in particular started hitching badly.
I tracked this down to my main logic thread occasionally spending upwards of 1 second waiting for a GIL lock, and further tracked that down to SSLContext.set_default_verify_paths(). That gets called by the urllib requests I'm firing off in my bg thread and winds up starving all other threads as it holds on to the GIL for the entirety of the SSL_CTX_set_default_verify_paths() call, which for some reason is taking quite a while on some Android devices.
I can separately look into why the underlying call is taking so long in the Android case, but regardless I'm able to completely eliminate the hitches by releasing the GIL for that call (enclosing the SSL_CTX_set_default_verify_paths() call with PySSL_BEGIN_ALLOW_THREADS/PySSL_END_ALLOW_THREADS).
Is that safe and reasonable to do for that call? I'd be happy to make a PR if so.
This was all tested on Python 3.10.5.
Thanks
-Eric
cpython/Modules/_ssl.c
Lines 4301 to 4310 in 760b8cf
The text was updated successfully, but these errors were encountered: