sys.path and sys.executable is incorrect in jupyter, but no applied fix is working

a 夏天 提交于 2021-01-28 07:52:35

问题


I've configured jupyter to be used from a remote computer and set a password to it while initial anaconda setup. Then after fixing this issue, I am trapped in another one. sys.path and sys.executable is incorrect in jupyter, but correct in python and ipython. Please see the details below. Anaconda3 is installed for all users in /opt/anaconda3 and I am using an environment zud for my programs.

Background

Once the above-mentioned problem is fixed, I tried to import igraph in jupyter but it gave ModuleNotFoundError as below:

import igraph
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-2-73ffca960e68> in <module>
----> 1 import igraph

ModuleNotFoundError: No module named 'igraph'

Then quickly I checked the following details using my environment zud, which were correct.

(zud) zuddler@silp252-dl360:~$ which python
/home/zuddler/.conda/envs/zud/bin/python
(zud) zuddler@silp252-dl360:~$ which ipython
/home/zuddler/.conda/envs/zud/bin/ipython
(zud) zuddler@silp252-dl360:~$ which jupyter
/home/zuddler/.conda/envs/zud/bin/jupyter
(zud) zuddler@silp252-dl360:~$ which conda
/opt/anaconda3/bin/conda

(zud) zuddler@silp252-dl360:~$ conda env list
# conda environments:
#
zud                   *  /home/zuddler/.conda/envs/zud
base                     /opt/anaconda3

Then I checked sys.path and sys.executable in python, ipython, and jupyter notebook/lab:

In Python:

>>> import sys
>>> sys.path
['', '/home/zuddler/.conda/envs/zud/lib/python39.zip', '/home/zuddler/.conda/envs/zud/lib/python3.9', '/home/zuddler/.conda/envs/zud/lib/python3.9/lib-dynload', '/home/zuddler/.conda/envs/zud/lib/python3.9/site-packages']
>>> sys.executable
'/home/zuddler/.conda/envs/zud/bin/python'

In ipython:

**In [1]: import sys

In [2]: sys.path
Out[2]:
['/home/zuddler/.conda/envs/zud/bin',
 '/home/zuddler/.conda/envs/zud/lib/python39.zip',
 '/home/zuddler/.conda/envs/zud/lib/python3.9',
 '/home/zuddler/.conda/envs/zud/lib/python3.9/lib-dynload',
 '',
 '/home/zuddler/.conda/envs/zud/lib/python3.9/site-packages',
 '/home/zuddler/.conda/envs/zud/lib/python3.9/site-packages/IPython/extensions',
 '/home/zuddler/.ipython']

In [3]: sys.executable
Out[3]: '/home/zuddler/.conda/envs/zud/bin/python'**

In Jupyter Notebook/Lab:

[1]: import sys
[2]: sys.path
[2]:
['/home/zuddler/python scripts',
 '/opt/anaconda3/lib/python38.zip',
 '/opt/anaconda3/lib/python3.8',
 '/opt/anaconda3/lib/python3.8/lib-dynload',
 '',
 '/opt/anaconda3/lib/python3.8/site-packages',
 '/opt/anaconda3/lib/python3.8/site-packages/IPython/extensions',
 '/home/zuddler/.ipython']
[3]: sys.executable
[3]: '/opt/anaconda3/bin/python'

sys.path and sys.executable were different in Jupyter Notebook/Lab but were correct in Python and ipython.

Also, to confirm if the current jupyter is running, I ran !which jupyter in the jupyter notebook and received /home/zuddler/.conda/envs/zud/bin/jupyter which confirmed that the correct jupyter notebook was running.

Tried Remedies

Remedy 1: I tried the fix mentioned here and here and added the following line in the /home/zuddler/.ipython/profile_default/ipython_config.py file

c.InteractiveShellApp.exec_lines = [
  'import sys; sys.path.append("/home/zuddler/.conda/envs/zud/lib/python3.9/site-packages")'
]

then restarted the jupyter notebook and tried to import igraph again and got a slightly different ModuleNotFoundError

import igraph
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-2-73ffca960e68> in <module>
----> 1 import igraph

~/.conda/envs/zud/lib/python3.9/site-packages/igraph/__init__.py in <module>
     32 # pylint: disable-msg=W0401
     33 # W0401: wildcard import
---> 34 from igraph._igraph import *
     35 from igraph.clustering import *
     36 from igraph.cut import *

ModuleNotFoundError: No module named 'igraph._igraph'

I quickly checked sys.path and sys.executable:

[1]: sys.path
[1]:
['/home/zuddler/python scripts',
 '/opt/anaconda3/lib/python38.zip',
 '/opt/anaconda3/lib/python3.8',
 '/opt/anaconda3/lib/python3.8/lib-dynload',
 '',
 '/opt/anaconda3/lib/python3.8/site-packages',
 '/opt/anaconda3/lib/python3.8/site-packages/IPython/extensions',
 '/home/zuddler/.ipython',
 '/home/zuddler/.conda/envs/zud/lib/python3.9/site-packages']
[2]: sys.executable
[2]: '/opt/anaconda3/bin/python'

It seems like a dead-end, so I rolled back this fix.

Remedy 2: I tried the fix given on this post and replaced the correct executable path in kernel.json

Kernels listed are:

(zud) zuddler@silp252-dl360:~$ jupyter kernelspec list
Available kernels:
  python3    /opt/anaconda3/share/jupyter/kernels/python3

I made the following changes in /opt/anaconda3/share/jupyter/kernels/python3/kernel.json

{
"argv": [
"/opt/anaconda3/bin/python", "/home/zuddler/.conda/envs/zud/bin/python",
"-m",
"ipykernel_launcher",
"-f",
"{connection_file}"
],
"display_name": "Python 3",
"language": "python"
}

after that, I restarted the jupyter notebook/lab and it worked for the environment zud, but I noticed that for the base environment it imported igraph which was not even installed in base. I checked the following values from them, which jupyter was correct for base

[1]: !which jupyter
[1]: /opt/anaconda3/bin/jupyter

but it was picking up sys.path and sys.executable from zud, and that means everone will be compelled to use zud environment.

[2]: sys.path
[2]:
['/home/zuddler/python scripts',
 '/home/zuddler/.conda/envs/zud/lib/python39.zip',
 '/home/zuddler/.conda/envs/zud/lib/python3.9',
 '/home/zuddler/.conda/envs/zud/lib/python3.9/lib-dynload',
 '',
 '/home/zuddler/.conda/envs/zud/lib/python3.9/site-packages',
 '/home/zuddler/.conda/envs/zud/lib/python3.9/site-packages/IPython/extensions',
 '/home/zuddler/.ipython']
[3]: sys.executable
[3]: '/home/zuddler/.conda/envs/zud/bin/python'

this was again a dead end, so I rolled back this fix.

Remedy 3: Applying the following steps from this post also did not solve the issue and gave the wrong sys.path and sys.executable

  1. Deleting IPython and Jupyter from the system
  2. rm -rf ~/.local/share/ipython
  3. Reinstall IPython and Jupyter

Remedy 4: I tried opening ~/.conda/envs/zud/bin/jupyter lab --no-browser as suggested in this post, but it too gave wrong sys.path and sys.executable.

Other Remedies: I read this, but found no solution.

PS: I noticed that this post is also mentioning a related issue as mine, but got no solution.

I'll be doing more research on this in the hope to reach a solution, I'll be grateful if you could provide a solution to it.


回答1:


I've had this issue for quite a while as well. Here's how I've solved it.

After you've created and entered the virtual environment, you probably did something like

pip install ipykernel

ipython kernel install --user --name=myenv in order to make it visible in jupyter lab/notebook.

There's this ipykernel mapping file at /home/user/.local/share/jupyter/kernels/myenv/kernel.json

It should look something like this:

{
 "argv": [
  "/usr/bin/python3",
  "-m",
  "ipykernel_launcher",
  "-f",
  "{connection_file}"
 ],
 "language": "python",
 "display_name": "myenv"
}

The problem here is that the virtual env python is mapped to system python for some reason. You want to change the python pointer to point to your virtual env python, which would be similar to /home/user/.virtualenvs/myenv/bin/python3.

TL;DR:

cat /home/user/.local/share/jupyter/kernels/myenv/kernel.json

if it has /usr/bin/python3 in it, change it to /home/user/.virtualenvs/myenv/bin/python3

You might need to restart your Jupyter Lab/Notebook for changes to take effect.

I don't fully understand why it sometimes happens and sometimes doesn't, but hopefully this fix will help you too.



来源:https://stackoverflow.com/questions/65373063/sys-path-and-sys-executable-is-incorrect-in-jupyter-but-no-applied-fix-is-worki

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!