python3.9 get-pip.py results in "AttributeError: 'HTMLParser' object has no attribute 'unescape'"

[msoultan]

• pip version:
  version 20.2.4

• Python version:
  Python 3.9

• Operating system:
  Ubuntu 18.04.5 LTS

When I try an install pip I get the following error:

$ curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1842k  100 1842k    0     0  4168k      0 --:--:-- --:--:-- --:--:-- 4159k
$ python3.9 get-pip.py --user
Traceback (most recent call last):
  File "/tmp/get-pip.py", line 23704, in <module>
    main()
  File "/tmp/get-pip.py", line 198, in main
    bootstrap(tmpdir=tmpdir)
  File "/tmp/get-pip.py", line 120, in bootstrap
    import setuptools  # noqa
  File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 14, in <module>
    from setuptools.dist import Distribution, Feature
  File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 24, in <module>
    from setuptools.depends import Require
  File "/usr/lib/python3/dist-packages/setuptools/depends.py", line 7, in <module>
    from .py33compat import Bytecode
  File "/usr/lib/python3/dist-packages/setuptools/py33compat.py", line 54, in <module>
    unescape = getattr(html, 'unescape', html_parser.HTMLParser().unescape)
AttributeError: 'HTMLParser' object has no attribute 'unescape'

[uranusjr]

Do you have a file named html.py or directory html in your current directory? You need to (re)move it since it is clashing with the built-in html module.

[uranusjr]

Ah, I think I know what’s going on. PR soon.

[msoultan]

And in case it helps, here's exactly what I'm doing:

********@*************:/tmp$ mkdir test

********@*************:/tmp$ cd test

********@*************:/tmp/test$ ls -l
total 0

********@*************:/tmp/test$ curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1842k  100 1842k    0     0  2606k      0 --:--:-- --:--:-- --:--:-- 2606k

********@*************:/tmp/test$ ls -l
total 1844
-rw-rw-r-- 1 ******** ******** 1886796 Nov 13 10:55 get-pip.py

********@*************:/tmp/test$ python3.9 get-pip.py --user
Traceback (most recent call last):
  File "/tmp/test/get-pip.py", line 23704, in <module>
    main()
  File "/tmp/test/get-pip.py", line 198, in main
    bootstrap(tmpdir=tmpdir)
  File "/tmp/test/get-pip.py", line 120, in bootstrap
    import setuptools  # noqa
  File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 14, in <module>
    from setuptools.dist import Distribution, Feature
  File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 24, in <module>
    from setuptools.depends import Require
  File "/usr/lib/python3/dist-packages/setuptools/depends.py", line 7, in <module>
    from .py33compat import Bytecode
  File "/usr/lib/python3/dist-packages/setuptools/py33compat.py", line 54, in <module>
    unescape = getattr(html, 'unescape', html_parser.HTMLParser().unescape)
AttributeError: 'HTMLParser' object has no attribute 'unescape'

[uranusjr]

BTW why do you need get-pip on 3.9 in the first place? Modern Python versions have a built-in ensurepip module that covers the same functionality: python3.9 -m ensurepip --user.

[msoultan]

I'm just following the instructions to install pip for Python 3.9 as described here:

https://pip.pypa.io/en/stable/installing/

[uranusjr]

Ah, I think I know what’s going on. PR soon.

Turns out this has been fixed for at long time: pypa/setuptools@53b8db3

But your setuptools package is ancient and does not contain it.

Judging from your paths, you seem to be on a Debian-derived system. How did you install Python 3.9? You should use apt to install pip if you installed Python from apt, instead of using get-pip.py, which may break your system (as described in the warning).

[msoultan]

So to install Python 3.9 on Ubuntu 16 I use the following commands:

sudo apt install software-properties-common
sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt update
sudo apt install python3.9

Is this not the correct way to install python3.9? Is there a better way to do it? I used to use apt to install pip, but it was always installing an older version - I haven't tried it recently.

[uranusjr]

It is a correct way to install Python 3.9, and should already contain pip. Does python3.9 -m pip --version prints the installation information?

It seems like someone reported a similar issue to deadsnakes recently: deadsnakes/issues#117. You probably want to join the discussion there since it would be much easier to reach deadsnakes maintainers.

[msoultan]

Hmm... doesn't seem so:

$ python3.9 -m pip --version
/usr/bin/python3.9: No module named pip

[pfmoore]

Then that's a bug in how Ubuntu package Python. I have a feeling it's a known problem - I vaguely recall that Ubuntu split pip out from the "main" Python package - but if so, you need to do whatever's required in Ubuntu to get a complete Python installation (which means it includes pip).

[msoultan]

So, the recommendation posted to the ticket I created in deadsnakes (referenced in this ticket) was to use sudo apt-get install python3-pip which I thought was no-longer to be used because it installed an old version so I switched to get-pip.py. That being said, I didn't know that I probably should have just run python3.9 -m pip install pip --upgrade in order to bring my python3.9 pip version up-to-date. So now I guess the right thing to do is use python3-pip instead of get-pip.py? That being said, the person in the other thread on deadsnakes said that this issue that I mentioned in this thread is a bug - not sure what to do other than report this stuff because I'm far from an expert on this!

One side-effect of this is that it seems like I don't get a pip3.9 executable - which I guess really isn't the end of the world as it seems like the general recommendation is to use python3.9 -m pip install ... moving forward.

Also, performing python3.9's pip update overwrites pip and pip3 to point to python3.9, which doesn't seem like a good idea for system-level scripts that use those with the assumption that pip -> python2 and pip3 -> system-level python version (i.e. 3.6 for Ubuntu 16). I'm not sure if that's a problem, but it does seem weird that pip/pip3 are changed.

[uranusjr]

python3-pip contains pip for the default python3 package, not python3.9 from deadsnakes, so I’m not quite sure how it’d help with your issue (because I don’t understand Debian stuff in general). I think you need to discuss this more in detail with Anthony in the deadsnake thread since he has more knowledge on Ubuntu issues than most pip maintainers (and very likely more than both Paul and I combined 😛).

[msoultan]

So after installing pip this way:

# install python3.9
sudo apt-get update
sudo apt-get install software-properties-common
sudo add-apt-repository ppa:deadsnakes/ppa --yes
sudo apt-get update
sudo apt-get install python3.9 --yes
# install pip
sudo apt-get install python3-pip --yes
python3.9 -m pip install pip --upgrade

Then I get the following:

$ python3.9 -m pip --version
pip 20.2.4 from /home/tomcat6/.local/lib/python3.9/site-packages/pip (python 3.9)

$ python3.9 -m pip install psutil
Defaulting to user installation because normal site-packages is not writeable
Collecting psutil
  Using cached psutil-5.7.3.tar.gz (465 kB)
    ERROR: Command errored out with exit status 1:
     command: /usr/bin/python3.9 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-y10psv8s/psutil/setup.py'"'"'; __file__='"'"'/tmp/pip-install-y10psv8s/psutil/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-u7pszexi
         cwd: /tmp/pip-install-y10psv8s/psutil/
    Complete output (11 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 14, in <module>
        from setuptools.dist import Distribution, Feature
      File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 24, in <module>
        from setuptools.depends import Require
      File "/usr/lib/python3/dist-packages/setuptools/depends.py", line 7, in <module>
        from .py33compat import Bytecode
      File "/usr/lib/python3/dist-packages/setuptools/py33compat.py", line 54, in <module>
        unescape = getattr(html, 'unescape', html_parser.HTMLParser().unescape)
    AttributeError: 'HTMLParser' object has no attribute 'unescape'
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

So Anthony is pointing to pip saying there's a bug here, and you're saying it's an issue there... ahhh.. I don't know what to do :)