Fix isolated builds when enum34 is installed

With the prior code, a python library installed into the same directory
as pip will be promoted to be found before other libraries of the same
name when an isolated build is performed.  This is because
pip/__main__.py adds the directory holding pip to the front of sys.path
where both the pip library and the other libraries will be found before
the other libraries.

This is especially problematic for libraries which shadow the names of
stdlib libraries.  The enum34 library, for instance, will shadow the
stdlib enum library.  When pip runs an isolated build and the enum34
library is installed, enum.py from the enum34 library will be found and
imported instead of enum.py from the stdlib.  That, in turn, breaks the
stdlib re module which then breaks pip.

This commit fixes that by changing build_env to invoke pip a different
way (by calling python on pip/__main__.py instead of calling python on
pip/).  Using __main__.py changes what python sets the value of
__package__ to which, in turn, makes it so the code in __main__.py
does not add the directory holding pip to sys.path.

Fixes pypa#8214

news/8214.bugfix

@@ -0,0 +1 @@
+Fix installing packages which have a pyproject.toml when enum34 is installed.

src/pip/__main__.py

@@ -10,8 +10,22 @@
 if sys.path[0] in ('', os.getcwd()):
     sys.path.pop(0)
 
-# If we are running from a wheel, add the wheel to sys.path
-# This allows the usage python pip-*.whl/pip install pip-*.whl
+# __package__ is empty string when invoking a pip wheel directly and when
+# invoking the pip directory.  __package__ is None when invoking __main__.py
+# directly.
+# * When pip is a wheel we want it to add the wheel to sys.path so that the
+#   rest of the pip code is available.
+# * This code allows pip as a directory to work but we don't want to support
+#   it as it can have unintended consequences if there are other python
+#   modules in the upper directory which shadow other libraries (enum34
+#   shadowing the stdlib enum, for instance, will break pip)
+# * If the invocation was of __main__.py itself, we need this code to *not* be
+#   run otherwise shadowing libraries can break it just like the directory
+#   case.  isolated builds are implemented using __main__.py so it's important
+#   that this case not break.
+#
+# Please see https://www.python.org/dev/peps/pep-0366/#id11
+# and https://github.com/pypa/pip/issues/8214
 if __package__ == '':
     # __file__ is pip-*.whl/pip/__main__.py
     # first dirname call strips of '/__main__.py', second strips off '/pip'

src/pip/_internal/build_env.py

@@ -167,8 +167,9 @@ def install_requirements(
         prefix.setup = True
         if not requirements:
             return
+        command = os.path.join(os.path.dirname(pip_location), '__main__.py')
         args = [
-            sys.executable, os.path.dirname(pip_location), 'install',
+            sys.executable, command, 'install',
             '--ignore-installed', '--no-user', '--prefix', prefix.path,
             '--no-warn-script-location',
         ]  # type: List[str]