Bug report

the parser mishandles lines containing null bytes when parsing source -- this allows the code to be misleadingly different from what it looks like.

I've been told by security@ that it is ok to post this publicly.

in the below example, <NUL> is an actual null byte:

x = '<NUL>' nothing to see here
';import os;os.system('echo pwnd')

and the execution and appearance in the terminal:

$ cat t.py
x = '' nothing to see here
';import os;os.system('echo pwnd')
$ python3 t.py
pwnd

it appears that after splitting the source into lines, the individual lines are treated as c strings and so the null terminator is misinterpreted, jamming the string contents together and it executes similar to this:

x = '';import os;os.system('echo pwnd')

note that if you want to write out a file like this here's a simple bit of code you can paste into an interactive prompt:

open('t.py', 'w').write("x = '\0' nothing to see here\n';import os;os.system('echo pwnd')\n")

here is perhaps a shorter example:

open('t.py', 'w').write("x = 1\0 + 1\n+2\nprint(x)\n")

I originally found this due to a bug report where the ast parser rejects code containing null bytes:

>>> import ast
>>> ast.parse("x = '\0'")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.8/ast.py", line 47, in parse
    return compile(source, filename, mode, flags,
ValueError: source code string cannot contain null bytes
>>> ast.parse(b"x = '\0'")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.8/ast.py", line 47, in parse
    return compile(source, filename, mode, flags,
ValueError: source code string cannot contain null bytes

ideally I would want the interpreter to reject files containing null bytes as a SyntaxError (and update the ast.parse error to a SyntaxError as well) -- though it appears there are some of these files in the wild -- such as https://github.com/univention/univention-corporate-server/blob/5.0-2/services/univention-ldb-modules/buildtools/bin/waf-svn

Your environment

• CPython versions tested on: 3.7 ... 3.11rc1 (though pretty sure this reproduces on all versions)
• Operating system and architecture: ubuntu 22.04, linux, x86_64