X Tutup
The Wayback Machine - https://web.archive.org/web/20221007210420/https://github.com/python/cpython/tree/main/Parser
Skip to content
main
Switch branches/tags
cpython/Parser/
Go to file
cpython/Parser/

Latest commit

@lysnikolaou
lysnikolaou gh-97973: Return all necessary information from the tokenizer (GH-97984)
cbf0afd Oct 6, 2022
gh-97973: Return all necessary information from the tokenizer (GH-97984) 
Right now, the tokenizer only returns type and two pointers to the start and end of the token.
This PR modifies the tokenizer to return the type and set all of the necessary information,
so that the parser does not have to this.
cbf0afd

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
. .
Python.asdl
 
 
action_helpers.c
 
 
asdl.py
 
 
asdl_c.py
 
 
myreadline.c
 
 
parser.c
 
 
peg_api.c
 
 
pegen.c
 
 
pegen.h
 
 
pegen_errors.c
 
 
string_parser.c
 
 
string_parser.h
 
 
token.c
 
 
tokenizer.c
 
 
tokenizer.h
 
 
X Tutup