gh-133516: Raise `ValueError` when constants `True`, `False` or `None` are used as an identifier after NFKC normalization by TERESH1 · Pull Request #133523 · python/cpython

TERESH1 · 2025-05-06T15:14:09Z

#133516

python-cla-bot · 2025-05-06T15:14:15Z

All commit authors signed the Contributor License Agreement.

Lib/test/test_ast/test_ast.py

Misc/NEWS.d/next/Core_and_Builtins/2025-05-06-15-01-41.gh-issue-133516.RqWVf2.rst

Parser/pegen.c

Add dot to end Co-authored-by: Stan Ulbrych <89152624+StanFromIreland@users.noreply.github.com>

pablogsal

LGTM

ZeroIntensity

Formatting silliness

Parser/pegen.c

ZeroIntensity · 2025-05-06T20:51:09Z

Actually, wait a minute--@pablogsal, could you reproduce this on main or 3.13? I can't with either the test case or the reproducer provided in the issue. I don't see how this could possibly be platform-dependent.

pablogsal · 2025-05-06T21:54:34Z

Actually, wait a minute--@pablogsal, could you reproduce this on main or 3.13? I can't with either the test case or the reproducer provided in the issue. I don't see how this could possibly be platform-dependent.

Humm good point. In macOS I am seeing this:

Python 3.14.0a7+ (heads/main-dirty:3dfed230928, May  6 2025, 16:15:52) [Clang 17.0.0 (clang-1700.0.13.3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> compile(source=b'Fal\xc5\xbfe', filename="<test>", mode="exec")
Traceback (most recent call last):
  File "<python-input-0>", line 1, in <module>
    compile(source=b'Fal\xc5\xbfe', filename="<test>", mode="exec")
    ~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: identifier field can't represent 'False' constant

pablogsal · 2025-05-06T21:56:32Z

Is hitting this assert:

cpython/Python/codegen.c

Line 3187 in a4be3bc

assert(!_PyUnicode_EqualToASCIIString(name, "None") &&

maybe we don't have asserts compiled in? But debug mode should not optimize them away....

TERESH1 · 2025-05-07T06:16:38Z

Actually, wait a minute--@pablogsal, could you reproduce this on main or 3.13? I can't with either the test case or the reproducer provided in the issue. I don't see how this could possibly be platform-dependent.

Humm good point. In macOS I am seeing this:
Python 3.14.0a7+ (heads/main-dirty:3dfed230928, May  6 2025, 16:15:52) [Clang 17.0.0 (clang-1700.0.13.3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> compile(source=b'Fal\xc5\xbfe', filename="<test>", mode="exec")
Traceback (most recent call last):
  File "<python-input-0>", line 1, in <module>
    compile(source=b'Fal\xc5\xbfe', filename="<test>", mode="exec")
    ~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: identifier field can't represent 'False' constant

Using Py_DEBUG, _PyAST_Validate is called after parsing AST

cpython/Parser/pegen.c

Lines 976 to 986 in 3dfed23

    
               // test_peg_generator defines _Py_TEST_PEGEN to not call PyAST_Validate() 
        
           #if defined(Py_DEBUG) && !defined(_Py_TEST_PEGEN) 
        
               if (p->start_rule == Py_single_input || 
        
                   p->start_rule == Py_file_input || 
        
                   p->start_rule == Py_eval_input) 
        
               { 
        
                   if (!_PyAST_Validate(res)) { 
        
                       return NULL; 
        
                   } 
        
               } 
        
           #endif

But if only asserts are enabled, then there is no AST validation

Parser/pegen.c

ZeroIntensity

LGTM as well now

miss-islington-app · 2025-05-07T18:11:29Z

Thanks @TERESH1 for the PR, and @pablogsal for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖

miss-islington-app · 2025-05-07T18:11:46Z

Sorry, @TERESH1 and @pablogsal, I could not cleanly backport this to 3.13 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker d9b0b07098ec4bdb47151948c77bcbbf3e796a5d 3.13

miss-islington-app · 2025-05-07T18:11:50Z

Thanks @TERESH1 for the PR, and @pablogsal for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖

bedevere-app · 2025-05-07T18:12:01Z

GH-133596 is a backport of this pull request to the 3.14 branch.

pablogsal · 2025-05-07T18:12:15Z

Sorry, @TERESH1 and @pablogsal, I could not cleanly backport this to 3.13 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker d9b0b07098ec4bdb47151948c77bcbbf3e796a5d 3.13

@TERESH1 can you do the backpory following this instruction?

bedevere-app · 2025-05-07T20:50:43Z

GH-133615 is a backport of this pull request to the 3.13 branch.

…r `None` are used as an identifier after NFKC normalization (GH-133523) (#133596)

…lse` or `None` are used as an identifier after NFKC normalization (pythonGH-133523) (cherry picked from commit d9b0b07)

…r `None` are used as an identifier after NFKC normalization (GH-133523) (#133615)

… `None` are used as an identifier after NFKC normalization (python#133523)

Fix pythongh-133516

17aae76

TERESH1 requested review from Eclips4, lysnikolaou and pablogsal as code owners May 6, 2025 15:14

bedevere-app bot mentioned this pull request May 6, 2025

Failed assertion in codegen_nameop #133516

Closed

bedevere-app bot added the awaiting review label May 6, 2025

StanFromIreland reviewed May 6, 2025

View reviewed changes

Lib/test/test_ast/test_ast.py Outdated Show resolved Hide resolved

Misc/NEWS.d/next/Core_and_Builtins/2025-05-06-15-01-41.gh-issue-133516.RqWVf2.rst Outdated Show resolved Hide resolved

Parser/pegen.c Outdated Show resolved Hide resolved

TERESH1 and others added 2 commits May 6, 2025 19:25

Fix mistake at news

cd5e276

Add dot to end Co-authored-by: Stan Ulbrych <89152624+StanFromIreland@users.noreply.github.com>

Fix assert in test and make error message more friendly

8f737e3

pablogsal approved these changes May 6, 2025

View reviewed changes

bedevere-app bot added awaiting merge and removed awaiting review labels May 6, 2025

pablogsal added needs backport to 3.12 only security fixes needs backport to 3.13 bugs and security fixes and removed needs backport to 3.12 only security fixes labels May 6, 2025

ZeroIntensity reviewed May 6, 2025

View reviewed changes

Parser/pegen.c Outdated Show resolved Hide resolved

Parser/pegen.c Outdated Show resolved Hide resolved

Reformat for PEP-7 style

c71edcf

ZeroIntensity reviewed May 7, 2025

View reviewed changes

Parser/pegen.c Outdated Show resolved Hide resolved

Keep the error message the same as in the validate_name

81b6c61

ZeroIntensity approved these changes May 7, 2025

View reviewed changes

pablogsal merged commit d9b0b07 into python:main May 7, 2025
39 checks passed

bedevere-app bot removed the awaiting merge label May 7, 2025

pablogsal added the needs backport to 3.14 bugs and security fixes label May 7, 2025

miss-islington-app bot assigned pablogsal May 7, 2025

bedevere-app bot removed the needs backport to 3.14 bugs and security fixes label May 7, 2025

TERESH1 mentioned this pull request May 7, 2025

[3.13] gh-133516: Raise ValueError when constants True, False or None are used as an identifier after NFKC normalization (GH-133523) #133615

Merged

bedevere-app bot removed the needs backport to 3.13 bugs and security fixes label May 7, 2025

pablogsal pushed a commit that referenced this pull request May 7, 2025

[3.14] gh-133516: Raise ValueError when constants True, False o…

54c3aa1

…r `None` are used as an identifier after NFKC normalization (GH-133523) (#133596)

TERESH1 added a commit to TERESH1/cpython that referenced this pull request May 8, 2025

[3.13] pythongh-133516: Raise ValueError when constants True, `Fa…

daadf7c

…lse` or `None` are used as an identifier after NFKC normalization (pythonGH-133523) (cherry picked from commit d9b0b07)

TERESH1 added a commit to TERESH1/cpython that referenced this pull request May 8, 2025

[3.13] pythongh-133516: Raise ValueError when constants True, `Fa…

28bdaf2

…lse` or `None` are used as an identifier after NFKC normalization (pythonGH-133523) (cherry picked from commit d9b0b07)

pablogsal pushed a commit that referenced this pull request May 8, 2025

[3.13] gh-133516: Raise ValueError when constants True, False o…

a719e5f

…r `None` are used as an identifier after NFKC normalization (GH-133523) (#133615)

Pranjal095 pushed a commit to Pranjal095/cpython that referenced this pull request Jul 12, 2025

pythongh-133516: Raise ValueError when constants True, False or…

1b7cec2

… `None` are used as an identifier after NFKC normalization (python#133523)

Uh oh!

Conversation

TERESH1 commented May 6, 2025

Uh oh!

python-cla-bot bot commented May 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pablogsal left a comment

Choose a reason for hiding this comment

Uh oh!

ZeroIntensity left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

ZeroIntensity commented May 6, 2025

Uh oh!

pablogsal commented May 6, 2025

Uh oh!

pablogsal commented May 6, 2025

Uh oh!

TERESH1 commented May 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

ZeroIntensity left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

miss-islington-app bot commented May 7, 2025

Uh oh!

miss-islington-app bot commented May 7, 2025

Uh oh!

miss-islington-app bot commented May 7, 2025

Uh oh!

bedevere-app bot commented May 7, 2025

Uh oh!

pablogsal commented May 7, 2025

Uh oh!

bedevere-app bot commented May 7, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

python-cla-bot bot commented May 6, 2025 •

edited

Loading

TERESH1 commented May 7, 2025 •

edited

Loading