A similar approach could work for virtualenvs or other Python environments. Once you do this, switching to the myenv kernel will automatically activate the myenv conda environment, which changes your $CONDA_PREFIX, $PATH and other system variables such that !conda install XXX and !pip install XXX will work correctly. # this is the critical part, and should be at the end of your script:Įxec python -m ipykernel in your kernel.json file, modify the argv field to look like this: 'argv': [ Perhaps: for example, this github issue shows an approach to modifying shell variables as part of kernel startup.īasically, in your kernel directory, you can add a script kernel-startup.sh that looks something like this (and make sure you change the permissions so that it's executable): #!/usr/bin/env bash So, could we massage kernel specifications such that they force the two to match? Users/jakevdp/anaconda/envs/python3.6/bin/pythonĪs I mentioned, the fundamental issue is a mismatch between Jupyter's shell environment and compute kernel. Doing this can have bad consequences, as often the operating system itself depends on particular versions of packages within that Python installation.įor day-to-day Python usage, you should isolate your packages from the system Python, using either virtual environments or Anaconda/Miniconda - I personally prefer conda for this, but I know many colleagues who prefer virtualenv. It will always lead to problems in the long term, even if it seems to solve them in the short-term.įor example, if pip install gives you a permission error, it likely means you're trying to install/update packages in a system python, such as /usr/bin/python. If you installed Python any other way (from source, using pyenv, virtualenv, etc.), then use pip to install Python packagesįinally, because it often comes up, I should mention that you should never use sudo pip install.

If conda tells you the package you want doesn't exist, then use pip (or try conda-forge, which has more packages available than the default conda channel). If you installed Python using Anaconda or Miniconda, then use conda to install Python packages. If you already have a Python installation that you're using, then the choice of which to use is easy:

conda installs any package in conda environments.

pip installs python packages in any environment.

I wrote way more than you ever want to know about these in a post last year, but the essential difference between the two is this: Other package managers exist (including platform-specific tools like yum, apt, homebrew, etc., as well as cross-platform tools like enstaller), but I'm less familiar with them and won't be remarking on them further.įor many users, the choice between pip and conda can be a confusing one.

This post will focus on two approaches to installing Python packages: pip and conda. Third, I'll talk about some ideas the community might consider to help smooth-over these issues, including some changes that the Jupyter, Pip, and Conda developers might consider to ease the cognitive load on users. Second, I'll dive into some of the background of exactly what the Jupyter notebook abstraction is doing, how it interacts with the complexities of the operating system, and how you can think about where the 'leaks' are, and thus better understand what's happening when things stop working.

In the wake of several discussions on this topic with colleagues, some online ( exhibit A, exhibit B) and some off, I decided to treat this issue in depth here.įirst, I'll provide a quick, bare-bones answer to the general question, how can I install a Python package so it works with my jupyter notebook, using pip and/or conda?. In other words, the Jupyter notebook, like all abstractions, is leaky. In the simplest contexts this issue does not arise, but when it does, debugging the problem requires knowledge of the intricacies of the operating system, the intricacies of Python package installation, and the intricacies of Jupyter itself. etc.).įundamentally the problem is usually rooted in the fact that the Jupyter kernels are disconnected from Jupyter's shell in other words, the installer points to a different Python version than is being used in the notebook. this, that, here, there, another, this one, that one, and this. This issue is a perrennial source of StackOverflow questions (e.g. I installed package X and now I can't import it in the notebook. I most often see this manifest itself with the following issue: In software, it's said that all abstractions are leaky, and this is true for the Jupyter notebook as it is for any other software.

Conda install package local