Avoid using requests in tests

[jayvdb]

test_specific_package_environment_markers using requests
fails when requests is pre-installed via PYTHONPATH.

Closes #3644

[techalchemy]

hm, so is the problem that the already installed version is being imported? if so, why?

[jayvdb]

It is being imported because it is in PYTHONPATH before the test runner is started.

This is needed because a rpm package contents are installed into a build root which isnt the same as the system image.

So the de-vendored requests exists in /usr/lib/python3.6/site-packages and pipenv exists at /foo/bar/build/root/usr/lib/python3.6/site-packages.

In order the run the tests, PYTHONPATH needs to be set to /usr/lib/python3.6/site-packages:/foo/bar/build/root/usr/lib/python3.6/site-packages.

I suspect there is a 'bug' in pipenv where it is not removing items from PYTHONPATH when it is running commands supposed to be inside the pipenv venv, so libraries in PYTHONPATH in the outer scope are accessible inside pipenv run. I put 'bug' in quotes as de-vendoring is of course not supported, and shouldnt be until there are solid tests which confirm that pipenv works correctly when de-vendored. At present I think the focus should be on identifying any ways in which pipenv misbehaves when de-vendored, so distros can decide whether the de-vendoring is more important than any resulting strange behaviour. I'm also seeing some misbehaviour when the de-vendored packages were installed using only distutils instead of setuptools, or if the egg-info isnt present (both are scenarios which happen in rpm managed python packages, although I think openSUSE has egg-info installed with most packages, and I assume Fedora also does, so that may be less of a concern now-a-days.).

[uranusjr]

Won’t using tablib have the same problem anyway, just with a different package? If we’re to change that, I’d want it to be changed to an actual environment-free implementation.

[jayvdb]

@uranusjr , I provide this patch because it fixes the test in my de-vendored builds. I also have a fix for test_pipenv_graph and test_pipenv_graph_reverse mentioned in #3644, but need to polish it before I submit it. The other five errors at #3644 are probably the 'real' problems, where it should fail, and I havent started on those yet.

Using tablib doesnt have the same problem because tablib is not a dependency of pipenv, so it is not pre-installed into PYTHONPATH in order to test pipenv, and thus will not exist unless it is installed into the venv. The markers in this test case prevent anything from being installed, so neither tablib nor requests should be accessible in the venv, but as I have explained requests does exist because it is in the outer scopes PYTHONPATH and somewhere pipenv isnt managing PYTHONPATH sufficiently to prevent it from existing inside the venv. The underlying problem may even be impossible to solve, if pipenv needs to existing inside the venv, as that means a de-vendored pipenv will have all deps also existing inside the venv , and a hell of mess if the Pipfile requires different versions than pipenv. However the underlying problem should have dedicated tests, and documentation about it (because de-vendorers will de-vendor anyway, unless the problems caused are very clear and verifiable), and it shouldnt trigger inexplicable failures all over the test suite. Using requests here is one such trigger, and it is unnecessary as any module could be used in this test case - it doesnt need to be requests.

[uranusjr]

I see, thanks for the explanation.

[techalchemy]

Yeah that all makes sense to me, thanks for the details

[techalchemy]

Thanks for the PR and the explanation, this makes sense