Fixes #16669

[alexlshon]

This Pull Request Fixes #16669 . This pull request makes a one line change to mypy/pyinfo.py to force the stdout of the python process to use utf-8.

One can replicate this error in Windows using Python3.8 just by calling the mypy/pyinfo.py module using a slightly modified code of the get_search_dirs function where the python executable doesn't match the value of sys.executable. The only modification made to this code from get_search_dirs is the adding of a non-ascii-path to the env parameter to subprocess.

    python_executable: str = r"<your_path_to_python_executable>\python.exe"
    non_ascii_path = r"<any_non_ascii_path>"
    env = {**dict(os.environ), "PYTHONSAFEPATH": "1"}
    env.update({"PYTHONPATH": non_ascii_path}) 
 # all code below is the same as get_search_dirs
    try:
        sys_path, site_packages = ast.literal_eval(
            subprocess.check_output(
                [python_executable, pyinfo.__file__, "getsearchdirs"],
                env=env,
                stderr=subprocess.PIPE,
            ).decode(encoding="utf-8")
        )
    except subprocess.CalledProcessError as err:
        print(err.stderr)
        print(err.stdout)
        raise
    except OSError as err:
        reason = os.strerror(err.errno)
        raise CompileError(
             [f"mypy: Invalid python executable '{python_executable}': {reason}"]
        ) from err

Running this code with out the fix returns this error in windows

Running pyinfo.py after forcing stdout to use utf-8 returns the expected value for sys_path and site_packages

[hauntsaninja]

Awesome, thank you!

[github-actions]

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅