bpo-43564: preserve original exception in args of FTP URLError by carljm · Pull Request #24938 · python/cpython

carljm · 2021-03-19T21:26:51Z

This makes this case consistent with the other cases in this module where an OSError is caught and wrapped and re-raised as a URLError, in that the original exception is preserved in the args of the URLError for introspection by the caller.

In particular, this means that test.support.transient_internet works with these wrapped errors, so that the FTP urllib2net tests correctly skip instead of failing when the network is not reachable.

https://bugs.python.org/issue43564

github-actions · 2021-04-20T00:10:39Z

This PR is stale because it has been open for 30 days with no activity.

* main: (5519 commits) Minor edits to the Descriptor HowTo Guide (pythonGH-24901) Fix link to Lifecycle of a Pull Request in CONTRIBUTING (python#98102) pythonGH-94597: deprecate `SafeChildWatcher`, `FastChildWatcher` and `MultiLoopChildWatcher` child watchers (python#98089) Auto-cancel old builds when new commit pushed to branch (python#98009) pythongh-95011: Migrate syslog module to Argument Clinic (pythonGH-95012) pythongh-68686: Retire eptag ptag scripts (python#98064) pythongh-97922: Run the GC only on eval breaker (python#97920) GitHub Workflows security hardening (python#96492) Add `@ezio-melotti` as codeowner for `.github/`. (python#98079) pythongh-97913 Docs: Add walrus operator to the index (python#97921) [doc] Fix broken links to C extensions accelerating stdlib modules (python#96914) pythongh-97822: Fix http.server documentation reference to test() function (python#98027) pythongh-91052: Add PyDict_Unwatch for unwatching a dictionary (python#98055) pythonGH-98023: Change default child watcher to PidfdChildWatcher on supported systems (python#98024) pythonGH-94182: Run the PidfdChildWatcher on the running loop (python#94184) pythongh-92886: make test_ast pass with -O (assertions off) (pythonGH-98058) pythongh-92886: make test_coroutines pass with -O (assertions off) (pythonGH-98060) pythongh-57179: Add note on symlinks for os.walk (python#94799) pythongh-94808: Fix regex on exotic platforms (python#98036) pythongh-90085: Remove vestigial -t and -c timeit options (python#94941) ...

carljm · 2022-10-09T10:20:55Z

I re-checked this change and I believe it is still correct/useful. The conditions for this to cause a test suite failure are a bit more subtle than described in the issue. If the network is totally unavailable, then the gethostbyname call will fail at

cpython/Lib/urllib/request.py

Line 1558 in aeb28f5

host = socket.gethostbyname(host)

and this error handler already wraps the full network error the way most places in urllib do, which test.support.transient_internet handles correctly.

In order to reproduce this problem, you need a host where there is a network and DNS available (so gethostbyname succeeds) but the network does not have access to the public internet, so the connect_ftp call a bit later at

cpython/Lib/urllib/request.py

Line 1568 in aeb28f5

fw = self.connect_ftp(user, passwd, host, port, dirs, req.timeout)

fails. And it is the error handling around the latter that is inconsistent with the rest of urllib, breaks test.support.transient_internet, and is fixed in this PR.

Without such a host, you can verify this fix by manually adding raise OSError(61, "foo") to the top of the connect_ftp method at

cpython/Lib/urllib/request.py

Line 1588 in aeb28f5

return ftpwrapper(user, passwd, host, port, dirs, timeout,

. Without this fix, ./python.exe -m test test_urllib2net -u network will then fail. With this fix, it will pass and skip the FTP tests due to test.support.transient_internet.

carljm · 2022-10-09T10:22:38Z

Requesting review from @orsenthil, listed in experts index as expert for urllib.

* main: pythongh-97841: Add methoddef for _filters_mutated (pythongh-98115)

orsenthil · 2022-10-10T01:58:54Z

Looks like a right candidate to back port to other supported branches.

important change for CI

miss-islington · 2022-10-10T01:59:10Z

Thanks @carljm for the PR, and @orsenthil for merging it 🌮🎉.. I'm working now to backport this PR to: 3.10, 3.11.
🐍🍒⛏🤖

miss-islington · 2022-10-10T01:59:12Z

Sorry @carljm and @orsenthil, I had trouble checking out the 3.11 backport branch.
Please backport using cherry_picker on command line.
cherry_picker ad817cd5c44416da3752ebf9baf16d650703275c 3.11

miss-islington · 2022-10-10T01:59:14Z

Sorry, @carljm and @orsenthil, I could not cleanly backport this to 3.10 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker ad817cd5c44416da3752ebf9baf16d650703275c 3.10

miss-islington · 2022-10-10T02:37:27Z

Thanks @carljm for the PR, and @orsenthil for merging it 🌮🎉.. I'm working now to backport this PR to: 3.11.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

miss-islington · 2022-10-10T02:37:28Z

Sorry @carljm and @orsenthil, I had trouble checking out the 3.11 backport branch.
Please backport using cherry_picker on command line.
cherry_picker ad817cd5c44416da3752ebf9baf16d650703275c 3.11

miss-islington · 2022-10-10T02:38:06Z

Thanks @carljm for the PR, and @orsenthil for merging it 🌮🎉.. I'm working now to backport this PR to: 3.11.
🐍🍒⛏🤖

bedevere-bot · 2022-10-10T02:38:14Z

GH-98136 is a backport of this pull request to the 3.11 branch.

…nGH-24938) * bpo-43564: preserve original error in args of FTP URLError * Add NEWS blurb Co-authored-by: Carl Meyer <carljm@instagram.com> (cherry picked from commit ad817cd) Co-authored-by: Carl Meyer <carl@oddbird.net>

) * bpo-43564: preserve original error in args of FTP URLError * Add NEWS blurb Co-authored-by: Carl Meyer <carljm@instagram.com> (cherry picked from commit ad817cd) Co-authored-by: Carl Meyer <carl@oddbird.net>

miss-islington · 2022-10-10T05:10:04Z

Thanks @carljm for the PR, and @orsenthil for merging it 🌮🎉.. I'm working now to backport this PR to: 3.10.
🐍🍒⛏🤖

miss-islington · 2022-10-10T05:10:18Z

Sorry, @carljm and @orsenthil, I could not cleanly backport this to 3.10 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker ad817cd5c44416da3752ebf9baf16d650703275c 3.10

…GH-24938) * bpo-43564: preserve original error in args of FTP URLError * Add NEWS blurb Co-authored-by: Carl Meyer <carljm@instagram.com>. (cherry picked from commit ad817cd) Co-authored-by: Carl Meyer <carl@oddbird.net>

bedevere-bot · 2022-10-10T05:13:48Z

GH-98138 is a backport of this pull request to the 3.10 branch.

…GH-24938) (#98138) * bpo-43564: preserve original error in args of FTP URLError * Add NEWS blurb Co-authored-by: Carl Meyer <carljm@instagram.com>. (cherry picked from commit ad817cd) Co-authored-by: Carl Meyer <carl@oddbird.net> Co-authored-by: Carl Meyer <carl@oddbird.net>

…n#24938) * bpo-43564: preserve original error in args of FTP URLError * Add NEWS blurb Co-authored-by: Carl Meyer <carljm@instagram.com>

bpo-43564: preserve original error in args of FTP URLError

6e00df6

the-knights-who-say-ni added the CLA signed label Mar 19, 2021

bedevere-bot added the awaiting review label Mar 19, 2021

github-actions bot added the stale Stale PR or inactive for long period of time. label Apr 20, 2021

ezio-melotti removed the CLA signed label Jul 13, 2022

github-actions bot removed the stale Stale PR or inactive for long period of time. label Aug 5, 2022

carljm added 2 commits October 9, 2022 11:24

Add NEWS blurb

c188cff

carljm requested a review from orsenthil October 9, 2022 10:22

Merge branch 'main' into urllibnonet

5d2c846

* main: pythongh-97841: Add methoddef for _filters_mutated (pythongh-98115)

orsenthil approved these changes Oct 10, 2022

View reviewed changes

bedevere-bot added awaiting merge and removed awaiting review labels Oct 10, 2022

orsenthil added needs backport to 3.10 only security fixes needs backport to 3.11 only security fixes labels Oct 10, 2022

orsenthil merged commit ad817cd into python:main Oct 10, 2022

bedevere-bot removed the awaiting merge label Oct 10, 2022

miss-islington assigned orsenthil Oct 10, 2022

AlexWaygood added needs backport to 3.11 only security fixes and removed needs backport to 3.11 only security fixes labels Oct 10, 2022

AlexWaygood removed the needs backport to 3.11 only security fixes label Oct 10, 2022

AlexWaygood added the needs backport to 3.11 only security fixes label Oct 10, 2022

bedevere-bot removed the needs backport to 3.11 only security fixes label Oct 10, 2022

orsenthil added needs backport to 3.10 only security fixes and removed needs backport to 3.10 only security fixes labels Oct 10, 2022

bedevere-bot removed the needs backport to 3.10 only security fixes label Oct 10, 2022

carljm mentioned this pull request Apr 10, 2022

ftp tests in test_urllib2net should skip on unreachable network #87730

Closed

carljm deleted the urllibnonet branch October 10, 2022 06:51

dhomeier mentioned this pull request Dec 2, 2022

v5.2rc1 _download_file_from_source does not catch URLError on Python 3.10.8 + 3.11.0 astropy/astropy#14091

Closed

Uh oh!

Conversation

carljm commented Mar 19, 2021 • edited by bedevere-bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Apr 20, 2021

Uh oh!

carljm commented Oct 9, 2022

Uh oh!

carljm commented Oct 9, 2022

Uh oh!

orsenthil commented Oct 10, 2022

Uh oh!

miss-islington commented Oct 10, 2022

Uh oh!

miss-islington commented Oct 10, 2022

Uh oh!

miss-islington commented Oct 10, 2022

Uh oh!

miss-islington commented Oct 10, 2022

Uh oh!

miss-islington commented Oct 10, 2022

Uh oh!

miss-islington commented Oct 10, 2022

Uh oh!

bedevere-bot commented Oct 10, 2022

Uh oh!

miss-islington commented Oct 10, 2022

Uh oh!

miss-islington commented Oct 10, 2022

Uh oh!

bedevere-bot commented Oct 10, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

carljm commented Mar 19, 2021 •

edited by bedevere-bot

Loading