Upgraded Python; Do I have to reinstall all site-packages manually?

Question

I have upraded my Linux distro recently. Python 3.5 was replaced by Python 3.6.

All site packages I have installed with pip3 are still in the /usr

Answer 1

I just hit the same problem upgrading from Python 3.6 to Python 3.7, I forgot to run pip freeze before I upgraded to Python 3.7. The solution that worked is to specify the --path option as the old site-packages/ directory (which was not deleted):

pip3 freeze --path /usr/local/lib/python3.6/site-packages/ > python3.6_requirements.txt
pip3 install -r python3.6_requirements.txt

Answer 2

Python 3.5 was replaced by Python 3.6. But you still have the backup option of using python 3.5.

If you want to use python 3.6 you will have to reinstall all pip packages again for python 3.6. And it makes sense.

Say you were changing from 2.7 to 3.5. You would want to preserve both the environments separately. Hence 3.6 environment is different from 3.5.

A quick way to do this would be to pip freeze for 3.5 and then install those dependencies for 3.6.

pip freeze > reqs.txt

upgrade

pip install -r reqs.txt

Since you don't have this option anymore, first try and list all packages in your python3.5

for that you can install pip3.5 as answered by @kabanus.

sudo apt-get install python3=3.5.1*
sudo python3.5 easy_install.py pip

Also it's advised to use virtual environment per project so you can maintain separate environments for each of them.

Answer 3

This would have made things simpler for you to reinstall. Checkout the description. Using freeze you could have done something like:

$ env1/bin/pip3 freeze > requirements.txt
$ env2/bin/pip3 install -r requirements.txt

Generally the recommended method is you use a virtualenv for site packages, so you don't litter your installation areas, but TBH it never broke something for me. Another option is to check if the linux distribution has the package available for proper retrieval, as in:

sudo apt-get install python3-<somemodule>

This is what I prefer - and could have been upgraded with the distro. As for what to do now, If you really don't want to re-install everything properly you could try to cp /usr/lib/python3.5/site-packages/* /usr/lib/python3.6/site-packages. The differences between versions are not so great such that I believe most packages would work off the bat. You may have to sed to replace python3.5 with python3.6 in all files there though. Forgot delete all pyc files if you do this.

Python modules are self contained enough that if something is broken it can be handled per package, and the site packages are self contained completely, so you could always just remove everything and re-install.

A final note - you can try and install pyton3.5/pip3.5 for your linux, and then do the freeze thing. If there is no package you could install manually (whl or such) or compile a stand alone and configure the site path properly. If you want to keep things on a global site package directory or migrate to virtualenv this may be the safest option.