ResolutionImpossible error message

[ei8fdb]

This ticket is work in progress.

What is the user story

As a pip user when pip can't resolve package dependencies, I need it to help me find a solution, so that I can install the packages I require.

What's the problem this will solve?
When a user tries to install a combination of packages, due to conflicting dependencies, pip is unable to complete the install. With the new resolver we are trying to improve the error messages to help users achieve the goal (of installing packages).

The objectives of these error messages are to communicate to pip users:

1. what has happened
2. what specifically has caused their dependency conflict
3. provide them with some possible ways to troubleshoot the conflict and achieve their goal (of installing packages)

Describe the solution you'd like
An error message that has the following structure:

1. displays what the error is (i.e. what the user has tried to do)
2. displays what has caused the error
3. displays possible ways to solve the error

Iterations
NB: the error messages displayed here are based on #8220.

The first iteration is:

Iteration 1: Error message contains - what the error is, what caused it, and possible ways to solve it

pip install ward==0.44.1b0 py2neo==4.3.0 --unstable-feature=resolver
Collecting ward==0.44.1b0
  Downloading ward-0.44.1b0-py3-none-any.whl (28 kB)
Collecting py2neo==4.3.0
  Downloading py2neo-4.3.0.tar.gz (71 kB)
     |████████████████████████████████| 71 kB 4.8 MB/s
ERROR: Cannot install ward v0.44.1b0 and py2neo v4.3.0 because these package versions have conflicting dependencies.
The conflict is caused by:
    ward 0.44.1b0 depends on pygments <3.0.0,>=2.4.2 (lower than 3.0.0 but greater or equal to 2.4.2)
    py2neo 4.3.0 depends on pygments ~=2.3.1 (approximately equal to version 2.3.1)

There are a number of possible solutions. You can try:
    removing package versions from your requirements, and letting pip try to resolve the problem for you
    trying a version of ward that depends on pygments v2.3.1. Try pip-search ward --dep pygments~=2.3.1
    replacing ward or py2neo with a different package altogether
    patching py2neo to use pygments <3.0.0,>=2.4.2
    force installing (Be aware!)

For instructions on how to do these steps visit: https://pypa.io/SomeLink"
To debug this further you can run pip-tree to see all of your dependencies.

Iteration 2: Error message contains - what the error is, what caused it, and a link to documentation for possible ways to solve it

pip install ward==0.44.1b0 py2neo==4.3.0 --unstable-feature=resolver
Collecting ward==0.44.1b0
  Downloading ward-0.44.1b0-py3-none-any.whl (28 kB)
Collecting py2neo==4.3.0
  Downloading py2neo-4.3.0.tar.gz (71 kB)
     |████████████████████████████████| 71 kB 4.8 MB/s
ERROR: Cannot install ward v0.44.1b0 and py2neo v4.3.0 because these package versions have conflicting dependencies.
The conflict is caused by:
    ward 0.44.1b0 depends on pygments <3.0.0,>=2.4.2
    py2neo 4.3.0 depends on pygments ~=2.3.1

There are a number of possible solutions. For instructions on how to do these steps visit: https://pypa.io/SomeLink"

Reactions to 1st proposal
The main reaction from the team has been that:

• providing 3. (displays possible ways to solve the error) is too verbose.
• the work involved in creating human readable version signifier is too much work and there is uncertainty on where this functionality should be located.

Alternative Solutions
The ideal solution would be:

• all python packages can be installed in whatever combination the user desires, or
• pip is able to resolve all dependency conflicts automatically and with no input from the users

These are not possible! :)

Additional context
To give a concrete example of what this error message would look like see #8220.

How can we do this?
We will test the iteration 2 error message content with pip users.

What we need to do this design or research

• research other package managers
• access to users
• write a testing plan for the these error messages

This ticket is work in progress.

[pfmoore]

Is the following sufficient?

I can't easily make the "the conflict is caused by" lines appear in red without them also having an "ERROR" prefix. This is basically how pip's logging works, and while I could dive into it and start trying to "fix" it, my instinct is that it's not worth the effort. Would that be reasonable, @ei8fdb @nlhkabu?

If this looks OK, I can start working on seeing how many tests I broke 🙂

Edit: Yes, I noticed I missed out the "There are a number of possible solutions" line. I'll add it in the final code. Sorry.

[pfmoore]

A technical note on how this is getting implemented:

pip install ward==0.44.1b0 py2neo==4.3.0 --unstable-feature=resolver
Collecting ward==0.44.1b0
  Downloading ward-0.44.1b0-py3-none-any.whl (28 kB)
Collecting py2neo==4.3.0
  Downloading py2neo-4.3.0.tar.gz (71 kB)
     |████████████████████████████████| 71 kB 4.8 MB/s
ERROR: Cannot install ward v0.44.1b0 and py2neo v4.3.0 because these package versions have conflicting dependencies.
The conflict is caused by:
    ward 0.44.1b0 depends on pygments <3.0.0,>=2.4.2
    py2neo 4.3.0 depends on pygments ~=2.3.1

There are a number of possible solutions. For instructions on how to do these steps visit: https://pypa.io/SomeLink"

In the line "Cannot install ward v0.44.1b0 and py2neo v4.3.0", the package name and version that we're quoting is what we actually tried to install, not what the user requested. In this example (where the user has pinned exact versions) this isn't obvious, but in a case where the user doesn't pin: pip install "ward>0.40" "py2neo!=4.1.0" we'd still see the same message, saying pip cannot install exact versions. The user could legitimately ask "well, OK, why diudn't you pick a different version" - to which the answer is basically "we did, but nothing else worked either".

Unfortunately, the exception that we get in the code doesn't include enough information to answer the questions "what else did you try?" or "what was the requirement I typed on the command line that caused you to attempt ward 0.44.1b0?".

I don't propose to address this point in the version of the code I'm currently writing. If it turns out to be confusing for users, we can iterate on it if needed. I just wanted to make a note here so the point was recorded and didn't get lost in all of the other threads on this topic...

[ei8fdb]

@pfmoore Looks good as a start. A few comments:

1. Is it possible to put a row of - or = before the line ERROR: Cannot install... and after the line ERROR: NO matching...?

Why?

Creating a visual border around the error message focuses the users attention here as this is where we're suggesting they start.

2. The line ERROR: No matching distribution found for pygments, I presume means (verbosely):

ERROR: I've not been able to find a version of pygments that will satisfy ward and py2neo's requirements - is that right? If so, can we modify this line?

If so, can we modify it to:

ERROR: No version of pygments found that will satisfy these package dependencies.

Why?

"Distributions" is a widely used term, especially in open source.

We've been talking about packages and versions up til now - we've not mentioned distributions at all.

Consistent language allows the user to make connections between related things. If we can keep it consistent that'd be best.

Edit: Yes, I noticed I missed out the "There are a number of possible solutions" line. I'll add it in the final code. Sorry.

👍

[pfmoore]

Bah. I'm sure I had a half-completed response in one of my many open browsers. No idea where it went 🙁

Is it possible to put a row of - or = before the line ERROR: Cannot install... and after the line ERROR: NO matching...?

Not easily. It's tied rather messily into pip's logging infrastructure. However, what I said above about not being able to make all of the output red was wrong, I've managed to. Is having the output in red sufficient distinction?

The line ERROR: No matching distribution found for pygments

This is a different issue. This line is produced by both the old and new resolvers, and is tested for explicitly in a number of places in the test suite. Changing this is a lot trickier than changing the lines above it (which are unique to the new resolver).

We could do it, but we'd have to consider the trade-offs in having the old and new resolvers diverge in behaviour in a way that impacts test coverage. If you want to take this further, can we put it into a separate issue, focused specifically on that one message line? (I know that seems petty in the broader sense of "how does pip report the problem", but it really would make the implementation details easier to discuss).

[ei8fdb]

We've carried out usability testing of the ResolutionImpossible error message. Recommendations have been made and decisions on what is possible to implement have been made. Below outlines the 3 implementable changes.

Error message: Recommendations to be implemented

1. Include small number of concise suggestions on how to solve this in the error message

The recommendation here is to provide possible next steps for people to try to resolve the conflict. See below "Proposed error message" for how this will look.

2. Include ResolutionImpossible (or another label) in the error message

I propose we use the ResolutionImpossible label. See below "Proposed error message" for how this will look.

3. There is still a strong case for explaining operators in plain language, especially ~=

It was mentioned in the team call (10 June 2020) the ~= version specifier is a rare occurrence in pip.

Without strong evidence to point to lack of understanding of the more common specifiers, the decision was to keep the operators as is for now.

4. Improve how the dependencies are displayed

This recommendation is to flip the version specifier - e.g.

• from this
  ward 0.44.1b0 depends on pygments <3.0.0,>=2.4.2

• to this:
  ward 0.44.1b0 depends on pygments >=2.4.2,<3.0.0

It was mentioned in the team call (10 June 2020) this information comes from package metadata and modifying this was not a good idea. The decision was to keep as is.

5. The layout of the error message needs to be improved

This recommendation focused on 1) changing the layout of the error message and 2) removing the Error: no matching distribution found for PackageName line.

See below "Proposed error message" for how this will look.

Proposed error message

Based on this recommendations and what was agreed, we're proposing an updated error message text:

Conflicting dependencies error message

Inconsistent requirements error message

Copyable text is listed as iteration 5.

These messages takes into account:

• comments that ~= is a rare occurrence (addressing 3 above)
• the possible recommendations listed above

@pfmoore Here are the 3 changes we need to implement:

• 1. Include small number of concise suggestions on how to solve this in the error message
• 2. Include ResolutionImpossible (or another label) in the error message
• 5. The layout of the error message needs to be improved
• 6. In a list of constraints replace the first , between version signifier with and

Thoughts and comments please!

[pfmoore]

Current code (only available locally on my PC right now) produces

ERROR: Cannot install ward 0.44.1b0 and py2neo 4.3.0 because these package versions have conflicting dependencies.

The conflict is caused by:
    ward 0.44.1b0 depends on pygments<3.0.0 and >=2.4.2
    py2neo 4.3.0 depends on pygments~=2.3.1

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible For help visit: https://pip.pypa.io/en/stable/user_guide/#dependency-conflicts-resolution-impossible

The two lines starting ERROR are all red, all the other lines are default colour.

For comparison, the same code handles the user error caused by the user specifying inconsistent requirements, such as click==5.0 click==7.0. The message then is

>.\.venv\Scripts\pip.exe install --unstable-feature=resolver click==5.0 click==7.0
ERROR: The following requirements are inconsistent:

    The user requested click==5.0
    The user requested click==7.0

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible For help visit: https://pip.pypa.io/en/stable/user_guide/#dependency-conflicts-resolution-impossible

[pfmoore]

@ei8fdb @nlhkabu How does the above version of the message look to you?

[ei8fdb]

@pfmoore I've added the "Inconsistent requirements error message" text in the issue comment above.

[pfmoore]

Thanks @ei8fdb. My next day when I have dedicated resolver time is next Wednesday, but I'll try to take a look over the weekend. There are some aspects of what you suggested that I suspect may be a little tricky to achieve, so I may be back with some "how important is this" questions when I've taken a look.

(Basically, the problem is that as with anything on a computer, anything is possible, the crucial question is whether it's worth the effort. I'm able to look at a request and judge "hmm, that's tricky", but I'm very bad at seeing why

The user requested:
    click==5.0, and
    click==7.0

is so much better than

The conflict is caused by:
    User requested click==5.0
    User requested click==7.0

Yes, "User requested" is duplicated, but we also need to consider the hybrid case

The conflict is caused by:
    User requested click==5.0
    foo 1.0 depends on click==7.0

The point here is that those "cause" lines are generated one by one, where "User requested" or "foo 1.0 depends on" is selected depending on whether that specific "cause" is linked to a "parent requirement" (the project that depends on it) or not (when we assume it was user specified).

Anyway, enough background. Let me see how far I get implementing this and I'll respond more then.

[pfmoore]

@ei8fdb Is this sufficiently close?

ERROR: Cannot install click==5.0 and click==7.0.
The conflict is caused by the user requesting:
    click==5.0
    click==7.0

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible For help visit: https://pip.pypa.io/en/stable/user_guide/#dependency-conflicts-resolution-impossible

I'm a bit concerned that it's stating the obvious ("Cannot install click 5 and click 7 because the user requested click 5 and click 7") but to say "cannot install click because" requires me to assume that the two "causes" in the exception are for the same project - and that means I need a separate case for when those two "causes" are for different projects. And I can't think of an example of how that would happen, but I have to say something ("even if it's only "Internal error, we don't know how this happened"!)

This is about as far as I can go with the proposed messages you describe above.

Another thing I think we need to be careful of here is that the code is now doing a lot of "if this is the case, do X, otherwise do Y" branching. I can explain what the cases are, but only in terms of internal program variables (e.g., I test if the "causes" attribute of the exception has any item with a non-None "parent"). As a result, I can't be 100% sure that the conditions I'm testing won't come up for situations that we haven't thought of - and where the message we've chosen would be confusing or inappropriate.

This is one of the (few 🙂) advantages of writing messages in terms of the internal state of pip - we can at least be sure that we're accurately describing the situation, even if it's hard to do so in a way that is informative to the user.

I think we probably need more extensive tests (maybe manual tests rather than stuff in the test suite, as I don't want us to be too tied to specific message wording - we're just looking for "does this condition trigger the message for scenario X") to be sure that the errors are appropriate. But that brings us round full circle to the issue that we don't know how to write tests that cover all of the various situations. Not sure what we can do about that...

[pfmoore]

@ei8fdb @nlhkabu OK, here's the next iteration, based on discussion Bernard and I had today. I've reverted to a single message format, which "degrades gracefully" as we get to less common and/or unexpected scenarios. So for the common case the message should be as described above for the ward/py2neo case, and for other cases it should be understandable.

I did manage to solve the issue of reporting the "top level" requested requirement in the summary message at the top (thanks @uranusjr for the pointer).

The following examples are all based on a dependency tree:

• A 1.0 depends on C 1.0
• B 1.0 depends on D 1.0
• C 1.0 depends on E 1.0
• D 1.0 depends on E 2.0

>.\.venv\Scripts\pip.exe install --no-index --find-links=. --unstable-feature=resolver A B
Looking in links: .
Processing c:\work\projects\pip\a-1.0-py3-none-any.whl
Processing c:\work\projects\pip\b-1.0-py3-none-any.whl
Processing c:\work\projects\pip\c-1.0-py3-none-any.whl
Processing c:\work\projects\pip\d-1.0-py3-none-any.whl
ERROR: Cannot install A and B because these package versions have conflicting dependencies.

The conflict is caused by:
    c 1.0 depends on E==1.0
    d 1.0 depends on E==2.0

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible For help visit: https://pip.pypa.io/en/stable/user_guide/#dependency-conflicts-resolution-impossible
PS 16:28 [Git: nr_conflict_message] {00:00.574} C:\Work\Projects\pip
>.\.venv\Scripts\pip.exe install --no-index --find-links=. --unstable-feature=resolver C D
Looking in links: .
Processing c:\work\projects\pip\c-1.0-py3-none-any.whl
Processing c:\work\projects\pip\d-1.0-py3-none-any.whl
ERROR: Cannot install c 1.0 and d 1.0 because these package versions have conflicting dependencies.

The conflict is caused by:
    c 1.0 depends on E==1.0
    d 1.0 depends on E==2.0

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible For help visit: https://pip.pypa.io/en/stable/user_guide/#dependency-conflicts-resolution-impossible
PS 16:28 [Git: nr_conflict_message] {00:00.555} C:\Work\Projects\pip
>.\.venv\Scripts\pip.exe install --unstable-feature=resolver click==5.0 click==7.0
ERROR: Cannot install click==5.0 and click==7.0 because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested click==5.0
    The user requested click==7.0

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible For help visit: https://pip.pypa.io/en/stable/user_guide/#dependency-conflicts-resolution-impossible
>.\.venv\Scripts\pip.exe install --no-index --find-links=. --unstable-feature=resolver A D
Looking in links: .
Processing c:\work\projects\pip\a-1.0-py3-none-any.whl
Processing c:\work\projects\pip\d-1.0-py3-none-any.whl
Processing c:\work\projects\pip\c-1.0-py3-none-any.whl
ERROR: Cannot install d 1.0 and A because these package versions have conflicting dependencies.

The conflict is caused by:
    d 1.0 depends on E==2.0
    c 1.0 depends on E==1.0

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible For help visit: https://pip.pypa.io/en/stable/user_guide/#dependency-conflicts-resolution-impossible

One (hopefully minor!) point: In the initial message "Cannot install A and B because these package versions have conflicting dependencies", I don't report the package version in all cases. That's because, in some cases, there is no version and if I try to get one, I'll get an error while trying to report the error - which is a really bad thing to display to the user. So I take the cautious approach and don't try to report the version when there's a risk. If this is a problem, I can likely work out how to write the code a little bit more "defensively" and get a version when I can without risking an error. But I'll look at that tomorrow, I want to publish this version now and leave it for the day.