bpo-45494: Fix parser crash when reporting errors involving invalid continuation characters #28993
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
pablogsal
changed the title
pablogsal
added
needs backport to 3.9
pablogsal
force-pushed
the
bpo-45494
branch
3 times, most recently
from
60578dd
to
4023afe
Compare
…ontinuation characters
There are two errors that this commit fixes:
* The parser was not correctly computing the offset and the string
source for E_LINECONT errors due to the incorrect usage of strtok().
* The parser was not correctly unwinding the call stack when a tokenizer
exception happened in rules involving optionals ('?', [...]) as we
always make them return valid results by using the comma operator. We
need to check first if we don't have an error before continuing.
|
Man, |
|
Thanks @pablogsal for the PR, and @ambv for merging it 🌮🎉.. I'm working now to backport this PR to: 3.9, 3.10. |
|
Sorry @pablogsal and @ambv, I had trouble checking out the |
|
Sorry, @pablogsal and @ambv, I could not cleanly backport this to |
|
regen-pegen fun is endless |
ambv
pushed a commit
to ambv/cpython
that referenced
this pull request
Oct 19, 2021
…valid continuation characters (pythonGH-28993) There are two errors that this commit fixes: * The parser was not correctly computing the offset and the string source for E_LINECONT errors due to the incorrect usage of strtok(). * The parser was not correctly unwinding the call stack when a tokenizer exception happened in rules involving optionals ('?', [...]) as we always make them return valid results by using the comma operator. We need to check first if we don't have an error before continuing.. (cherry picked from commit a106343) Co-authored-by: Pablo Galindo Salgado <[email protected]>
|
GH-29070 is a backport of this pull request to the 3.10 branch. |
Wait, this pr should not need any regen-pegen |
ambv
pushed a commit
to ambv/cpython
that referenced
this pull request
Oct 19, 2021
…alid continuation characters (pythonGH-28993) There are two errors that this commit fixes: * The parser was not correctly computing the offset and the string source for E_LINECONT errors due to the incorrect usage of strtok(). * The parser was not correctly unwinding the call stack when a tokenizer exception happened in rules involving optionals ('?', [...]) as we always make them return valid results by using the comma operator. We need to check first if we don't have an error before continuing.. (cherry picked from commit a106343) Co-authored-by: Pablo Galindo Salgado <[email protected]>
|
GH-29071 is a backport of this pull request to the 3.9 branch. |
|
| Wait, this pr should not need any regen-pegen Yeah, it doesn't. I just tend to always run |
ambv
added a commit
that referenced
this pull request
Oct 19, 2021
…valid continuation characters (GH-28993) (GH-29070) There are two errors that this commit fixes: * The parser was not correctly computing the offset and the string source for E_LINECONT errors due to the incorrect usage of strtok(). * The parser was not correctly unwinding the call stack when a tokenizer exception happened in rules involving optionals ('?', [...]) as we always make them return valid results by using the comma operator. We need to check first if we don't have an error before continuing.. (cherry picked from commit a106343) Co-authored-by: Pablo Galindo Salgado <[email protected]>
ambv
added a commit
that referenced
this pull request
Oct 20, 2021
…alid continuation characters (GH-28993) (#29071) There are two errors that this commit fixes: * The parser was not correctly computing the offset and the string source for E_LINECONT errors due to the incorrect usage of strtok(). * The parser was not correctly unwinding the call stack when a tokenizer exception happened in rules involving optionals ('?', [...]) as we always make them return valid results by using the comma operator. We need to check first if we don't have an error before continuing.. (cherry picked from commit a106343) Co-authored-by: Pablo Galindo Salgado <[email protected]> NOTE: unlike the cherry-picked original, this commit points at a crazy location due to a bug in the tokenizer that required a big refactor in 3.10 to fix. We are leaving as-is for 3.9.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There are two errors that this commit fixes:
source for E_LINECONT errors due to the incorrect usage of strtok().
exception happened in rules involving optionals ('?', [...]) as we
always make them return valid results by using the comma operator. We
need to check first if we don't have an error before continuing.
https://bugs.python.org/issue45494