Remove the 2to3 program and the lib2to3 module

[vstinner]

The 2to3 program is implemented with the lib2to3 module which implements a parser of the Python language. Problem: this parser is unable to parse Python 3.10 grammar :-( This issue was already known in Python 3.9 when PEP 617 – New PEG parser for CPython was approved.

The 2to3 program and the lib2to3 were marked as "pending" deprecated since Python 3.9, and then officially deprecated in Python 3.11 (import lib2to3 emits a DeprecationWarning).

One of a famous user of lib2to3 was the black project but they decided to fork lib2to3 and their fork was made compatible with Python 3.10 grammar (maybe also Python 3.11).

I propose to remove 2to3 and lib2to3 at the beginning of the Python 3.13 development cycle to give users more time to be prepared for this incompatible change before Python 3.13 final release (scheduled in October 2024).

cc @isidentical @lysnikolaou @pablogsal @ambv

---

Example of valid Python script (script.py:

with (open(__file__) as fp): data=fp.read()
print(len(data))

It works well on Python 3.11:

$ python3.11 script.py 
61

But lib2to3 is unable to parse it:

$ python3.11 -m lib2to3 script.py 
(...)
RefactoringTool: No files need to be modified.
RefactoringTool: There was 1 error:
RefactoringTool: Can't parse script.py: ParseError: bad input: type=1, value='as', context=(' ', (1, 21))

Linked PRs

• gh-104780: Remove 2to3 program and lib2to3 module #104781

[vstinner]

Done by ae00b81

[hugovk]

Wow!

(I think this issue can be closed, please re-open otherwise :)

[vstinner]

lib2to3 was deprecated in Python 3.11 by issue #84540.