Installing from requirements file does not respect package index options

[barrykan]

Environment

• pip version: 19.2.3
• Python version: 2.7.12
• OS: Ubuntu 16.04.6 LTS

Description
In the docs, one of the possible formats is <requirement specifier> [; markers] [[--option]...]. However, when you use this format, it seems to ignore options such as --find-links and --no-index.

Expected behavior
When you use a requirement specifier in a requirements files, it should respect package index options. e.g. If I want to pass a local path to an archive to --find-links, it should try to install from that archive, not just from https://pypi.org/simple.

How to Reproduce

1. Create a requirements file that specifies a package name and a link to a package -- remote or local. e.g.

echo cdecimal --no-index --find-links http://www.bytereef.org/software/mpdecimal/releases/cdecimal-2.3.tar.gz > requirements.txt

2. Try to pip install that requirements file.

pip install -vr requirements.txt

Output

(envs) vagrant@ubuntu-xenial:/tmp/envs$ pip install -vr requirements.txt
Created temporary directory: /tmp/pip-ephem-wheel-cache-KJN18l
Created temporary directory: /tmp/pip-req-tracker-nOIQc8
Created requirements tracker '/tmp/pip-req-tracker-nOIQc8'
Created temporary directory: /tmp/pip-install-Fmm3ZY
Collecting cdecimal (from -r requirements.txt (line 1))
  1 location(s) to search for versions of cdecimal:
  * https://pypi.org/simple/cdecimal/
  Getting page https://pypi.org/simple/cdecimal/
  Found index url https://pypi.org/simple
  Looking up "https://pypi.org/simple/cdecimal/" in the cache
  Request header has "max_age" as 0, cache bypassed
  Starting new HTTPS connection (1): pypi.org:443
  https://pypi.org:443 "GET /simple/cdecimal/ HTTP/1.1" 304 0
  Analyzing links from page https://pypi.org/simple/cdecimal/
  Given no hashes to check 0 links for project 'cdecimal': discarding no candidates
  ERROR: Could not find a version that satisfies the requirement cdecimal (from -r requirements.txt (line 1)) (from versions: none)
Cleaning up...
Removed build tracker '/tmp/pip-req-tracker-nOIQc8'
ERROR: No matching distribution found for cdecimal (from -r requirements.txt (line 1))
Exception information:
Traceback (most recent call last):
  File "/tmp/envs/local/lib/python2.7/site-packages/pip/_internal/cli/base_command.py", line 188, in main
    status = self.run(options, args)
  File "/tmp/envs/local/lib/python2.7/site-packages/pip/_internal/commands/install.py", line 345, in run
    resolver.resolve(requirement_set)
  File "/tmp/envs/local/lib/python2.7/site-packages/pip/_internal/legacy_resolve.py", line 196, in resolve
    self._resolve_one(requirement_set, req)
  File "/tmp/envs/local/lib/python2.7/site-packages/pip/_internal/legacy_resolve.py", line 359, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "/tmp/envs/local/lib/python2.7/site-packages/pip/_internal/legacy_resolve.py", line 307, in _get_abstract_dist_for
    self.require_hashes
  File "/tmp/envs/local/lib/python2.7/site-packages/pip/_internal/operations/prepare.py", line 134, in prepare_linked_requirement
    req.populate_link(finder, upgrade_allowed, require_hashes)
  File "/tmp/envs/local/lib/python2.7/site-packages/pip/_internal/req/req_install.py", line 211, in populate_link
    self.link = finder.find_requirement(self, upgrade)
  File "/tmp/envs/local/lib/python2.7/site-packages/pip/_internal/index.py", line 1228, in find_requirement
    'No matching distribution found for %s' % req
DistributionNotFound: No matching distribution found for cdecimal (from -r requirements.txt (line 1))

Note that it shouldn't even try to find the package on https://pypi.org/simple because I also passed --no-index to it.

Conversely, if I pass the same arguments to pip install without using a requirements file, it succeeds as desired, so this quirk seems isolated to how pip installs from a requirements file:

(envs) vagrant@ubuntu-xenial:/tmp/envs$ pip install cdecimal -v --no-index --find-links http://www.bytereef.org/software/mpdecimal/releases/cdecimal-2.3.tar.gz
Ignoring indexes: https://pypi.org/simple
Created temporary directory: /tmp/pip-ephem-wheel-cache-DCqmGq
Created temporary directory: /tmp/pip-req-tracker-b6cOWq
Created requirements tracker '/tmp/pip-req-tracker-b6cOWq'
Created temporary directory: /tmp/pip-install-zIKx40
Looking in links: http://www.bytereef.org/software/mpdecimal/releases/cdecimal-2.3.tar.gz
Collecting cdecimal
  WARNING: The repository located at www.bytereef.org is not a trusted or secure host and is being ignored. If this repository is available via HTTPS we recommend you use HTTPS instead, otherwise you may silence this warning and allow it anyway with '--trusted-host www.bytereef.org'.
  0 location(s) to search for versions of cdecimal:
  Found link http://www.bytereef.org/software/mpdecimal/releases/cdecimal-2.3.tar.gz (from -f), version: 2.3
  Given no hashes to check 1 links for project 'cdecimal': discarding no candidates
  Using version 2.3 (newest of versions: 2.3)
  Using cached wheel link: file:///home/vagrant/.cache/pip/wheels/34/6e/38/49440cc68027fbd93618a16b414140d6a61a0b737bbdaeeef5/cdecimal-2.3-cp27-cp27mu-linux_x86_64.whl
  Added cdecimal from file:///home/vagrant/.cache/pip/wheels/34/6e/38/49440cc68027fbd93618a16b414140d6a61a0b737bbdaeeef5/cdecimal-2.3-cp27-cp27mu-linux_x86_64.whl to build tracker '/tmp/pip-req-tracker-b6cOWq'
  Removed cdecimal from file:///home/vagrant/.cache/pip/wheels/34/6e/38/49440cc68027fbd93618a16b414140d6a61a0b737bbdaeeef5/cdecimal-2.3-cp27-cp27mu-linux_x86_64.whl from build tracker '/tmp/pip-req-tracker-b6cOWq'
Installing collected packages: cdecimal

Successfully installed cdecimal-2.3
Cleaning up...
Removed build tracker '/tmp/pip-req-tracker-b6cOWq'

Alternative solution for me
The reason I want this to work is I want to include local package archives in my requirements.txt, but I don't want it to try to install every single time I run pip install -r requirements.txt. i.e. I could just use an alternative format for requirements.txt and just include the url/path alone on a line; however, if I do this, it doesn't check whether the package is already installed.

If there is an alternative way of achieving this goal, I'd be very appreciative.

Thank you!

[emmanuelantony2000]

Rather that using the format that you specified using this format (mentioned below) solves the problem.

--no-index
--find-links http://www.bytereef.org/software/mpdecimal/releases/cdecimal-2.3.tar.gz
cdecimal

Mentioning cdecimal at the last solves the problem.
If you think that there is a problem with the documentation do mention it.

[chrahunt]

Hi @barrykan, as mentioned by @emmanuelantony2000 the --no-index and --find-links options need to be on their own lines.

I've opened #7456 for a related documentation change, which should also result in having these different types of options clearly distinguished.

I'll close this now, but please let us know if you have any other issues. Thanks!