Homebrew pip issue 5599 on Intel macOS Big Sur

Question

(I'm new here, so please be gentle)

When using topgrade in homebrew (I installed topgrade after installing homebrew with brew install topgrade) on macOS big sur on a 2020 Intel 13" default upper specs MacBook Pro I ran into the following issue:

xcrun: error: invalid active developer path (/Library/Developer/CommandLineTools), missing xcrun at: /Library/Developer/CommandLineTools/usr/bin/xcrun

Looking for a solution on the web I found this article.

Following the advice I ran xcode-select --install and restarted afterwards.

After running topgrade again I became a bit suspicious when it said:

    ...
Successfully installed pip-21.1.1
    WARNING: You are using pip version 19.2.3, however version 21.1.1 is available.
    You should consider upgrading via the 'pip install --upgrade pip' command.
    ...

but I though well, seems like it updated itself. So I ran topgrade again and it said

WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue.
To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.
Requirement already satisfied: pip in ./Library/Python/3.8/lib/python/site-packages (21.1.1)

Since then I've been getting the same answer. I followed the link but found no solution other than the Python workaround, which seems to be a short term solution.

As far as I know have both pip version 19.2.3 as well as 21.1.1 and need to remove the old one right? But how do I do that? And does Requirement already satisfied: pip in ./Library/Python/3.8/lib/python/site-packages (21.1.1) mean it already fixed itself? And what happens if I ignore the problem?

If anyone could help me out it would really be appreciated. While I love tech I've got no coding experience at all in this is just over my head. That you so much!

Edit: I ran the advice given here and got the following results:

which -a python python2 python2.7 python3 python3.6

showed

/usr/bin/python
/usr/bin/python2
/usr/bin/python2.7
/usr/bin/python3

typing python and hitting tab (equivalent to compgen -c python) gave me

python            python2           python2.7-config  pythonw
python-config     python2.7         python3           pythonw2.7

and running locate python after building the database like described gave me a super long list, which wouldn't fit here.

New Question: I don't really understand the steps I'm supposed to take here before running python -m pip, but I don't really see any difference whether or not I do these steps anyway. Can I just run python -m pip without really having a clue what happens?

Second Edit

python -m pip --version gives /System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python: No module named pip

and python3 -m pip --version gives pip 21.1.1 from /Users/MyName/Library/Python/3.8/lib/python/site-packages/pip (python 3.8)

Regarding the second comment python -V gives Python 2.7.16

and python3 -V gives Python 3.8.2

Also sorry that it's been a while, but we have exams at the moment so I've been kind of busy.

Answer 1

The important thing is this:

Figure out what installation of Python you want to be using, make sure that's the one you get when you just type python at the prompt, and make sure that the pip you get when you just type pip points to that same installation.

Apple is moving away from having a default installation of scripting languages including Python, but so far they still come with one - that you don't have permission to upgrade without turning off System Integrity Protection. So the usual advice of "just keep the system install upgraded" is harder to do on Macs.

You get a completely different version when you install Xcode, and it doesn't supply a python binary at all, only one called python3.

My advice is to install python with Homebrew, so you have full control over it, but it also doesn't provide a python, at least not one linked by default. You have to make that link yourself. But I think you should - symlink all the 3-less Homebrew python binaries to /usr/local/bin; and put /usr/local/bin in your PATH before any other system-type locations.

So these steps, basically:

brew install python
ln -s /usr/local/opt/python/libexec/bin/* /usr/local/bin
/usr/local/bin/python -mpip install --upgrade pip

Then in your shell startup file (~/.bash_profile or ~/.zprofile or ~/.bashrc or ~/.zshrc) make sure that /usr/local/bin comes first in your PATH with something like this:

PATH=/usr/local/bin:${PATH#/usr/local/bin:}

Then quit your shell and start a new one and then you should be good to go.

Answer 2

Some thoughts:

As far as I know have both pip version 19.2.3 as well as 21.1.1 and need to remove the old one right?

Regarding removal, not necessarily. While it's certainly more true on Linux, Python is often tightly integrated into *nix-style systems. Messing with a default system installation of Python can (hypothetically) cause failures in key system software (the most visible often being package managers than run Python under the hood).

How do I do that?

Assuming the older version is part of your system's default Python installation, ideally you shouldn't. In any case, it's often better to use any built-in system tools or take other appropriate steps to upgrade to the most recent versions of the right software (Python/pip). The arguable exception to this is any virtual environment or Python application that requires specific versions of Python or any related packages to be installed.

It is probably worth mentioning that while pip 19.2.3 can be used with Python 2 or Python 3, pip 20.3 was the last version to have a .whl (Python prebuilt installation file) supporting Python 2. pip 21.0+ is squarely aimed at Python 3. A history of pip releases is available here.

I followed the link but found no solution other than the Python workaround, which seems to be a short term solution.

The basic advice on that page is solid and will help you avoid problems, assuming you follow it. You should read it carefully.

Note that the reference to using python -m pip as a workaround is somewhat misleading. That form is typically the best way to use pip, independent of the issue you are encountering. It won't necessarily keep you completely out of trouble but, as a rule, using pip by itself often isn't advisable.

Does Requirement already satisfied: pip in ./Library/Python/3.8/lib/python/site-packages (21.1.1) mean it has already fixed itself?

It likely means that you have a current version of pip installed in that installation of Python (./Library/Python/3.8) or whatever it refers to. To really find out what's what, you will likely want to check for multiple installations of Python (as a suggestion, see this Stack Overflow question perhaps for some ideas on how to do that), and then check each installation's version of pip.

Note that, as far as I am aware, Xcode installs its own copy of Python 3 (default python3, pip3) which is separate from the Python 2 installation MacOS has used in the past by default (often python, pip).

What happens if I ignore the problem?

Your computer will crash, turning your screen into the "falling code" from The Matrix and then your system will spontaneously explode. ;)

Seriously, though, I unfortunately don't have an answer for you. You should edit your question with the results of the testing suggested above, which might offer a bit better window into what's happening.