iDer reported a vulnerability in the HTTP server.

(1) Start a local HTTP server (listen to tcp/8000):

python3 -m http.server 8000

(2) Open a web browser and to go:

http://localhost:8000//www.python.org/%2f..

=> the browser is redirected to http://www.python.org/%2f../

(on this example, the python.org web server redirects to https://www.python.org/%2f../ )


Raw HTTP to see the HTTP redirection using netcat:
---
$ echo -ne "GET //www.python.org/%2f.. HTTP/1.0\n\n" | nc localhost 8000
HTTP/1.0 301 Moved Permanently
Server: SimpleHTTP/0.6 Python/3.6.2
Date: Mon, 20 Nov 2017 13:31:42 GMT
Location: //www.python.org/%2f../
---

The problem is in the SimpleHTTPRequestHandler.send_head() function:

* self.path = '//www.python.org/%2f..'
* translate_path() translates '//www.python.org//..' path to self.directory (the current directory by default).
* isdir(self.directory) is True but self.path doesn't send with '/', so send_head() creates a HTTP redirection (HTTP 301)
* The redirection URL is '//www.python.org/%2f../'. Extract of the raw HTTP: "Location: //www.python.org/%2f../"

The web browsers translates the URL '//www.python.org/%2f../' to "http://www.python.org/%2f../"... It surprised me, but ok, it's a fact.

I'm not sure what is the best way to fix this vulnerability without rejecting valid HTTP requests.

IMHO the root issue is the redirection URL starting with "//". I would expect something like "localhost//". The problem is that I'm not sure that the HTTP server knows its own "external" hostname. "localhost" is wrong is the server is accessed from the outside. Maybe the server must just fail on that case?


This vulnerabilility was reported to the Python Security Response Team (PSRT) at October 18, 2017 (one month ago). Since no obvious fix was found, it was decided to make the vulnerability public to get more eyes on it to find a quick fix.

Note: I'm not sure that this vulnerability is important, since the redirected URL ends with "/%2f../" which should be rejected by any correct HTTP Server (say, not the Python builtin "simple" HTTP server...).

Extract of send_head():

        path = self.translate_path(self.path)
        f = None
        if os.path.isdir(path):
            parts = urllib.parse.urlsplit(self.path)
            ...

urllib.parse.urlsplit('//www.python.org/%2f..') returns:

SplitResult(scheme='', netloc='www.python.org', path='/%2f..', query='', fragment='')

Is urlsplit() the correct function to call here? www.python.org is part of the path, not of the netloc.

I wrote this patch, but I'm not sure that it's ok to always reject redirection URLs starting with //:

diff --git a/Lib/http/server.py b/Lib/http/server.py
index 502bce0c7a..494031b8c2 100644
--- a/Lib/http/server.py
+++ b/Lib/http/server.py
@@ -673,10 +673,18 @@ class SimpleHTTPRequestHandler(BaseHTTPRequestHandler):
             parts = urllib.parse.urlsplit(self.path)
             if not parts.path.endswith('/'):
                 # redirect browser - doing basically what apache does
-                self.send_response(HTTPStatus.MOVED_PERMANENTLY)
                 new_parts = (parts[0], parts[1], parts[2] + '/',
                              parts[3], parts[4])
                 new_url = urllib.parse.urlunsplit(new_parts)
+
+                # Browsers interpret "Location: //uri" as an absolute URI
+                # like "http://URI"
+                if new_url.startswith('//'):
+                    self.send_error(HTTPStatus.BAD_REQUEST,
+                                    "URI must not start with //")
+                    return None
+
+                self.send_response(HTTPStatus.MOVED_PERMANENTLY)
                 self.send_header("Location", new_url)
                 self.end_headers()
                 return None

Maybe a good fix would be to “escape” the double slash with “/.”:

if os.path.isdir(path):
    url = self.path
    if url.startswith('//'):  # E.g. "//www.python.org/%2f.."
        url = "/." + url  # Becomes "/.//www.python.org/%2f.."
    parts = urllib.parse.urlsplit(url)
    ...

When this “escaped” URL is resolved with the base URL, it should give the right result:

>>> base = "http://localhost:8000//www.python.org/%2f.."
>>> redirect = "/.//www.python.org/%2f../"
>>> urljoin(base, redirect)
'http://localhost:8000//www.python.org/%2f../'

A simpler idea is to strip off all but one of the leading slashes, so you end up with "/www.python.org/%2f..". That would technically be a different URL, but would access the same file through the default SimpleHTTPRequestHandler behaviour, so most people wouldn’t notice.

In Issue 34276 I suggested a fix to “urlunsplit”. In this case it would send “Location: ////www.python.org/%2f../", with an extra pair of slashes denoting an empty host name. This should stop a browser from seeing “www.python.org” as a host name.

This looks like a duplicate of Issue43223 which has a PR in progress.