Setting up Jupyter notebook with Tensorflow, Keras and Pytorch for Deep Learning

I was trying to set up my Jupyter notebook to work on some deep learning problem (some image classification on MNIST and imagenet dataset) on my laptop (Ubuntu 16.04 LTS). Previously I have used a little bit of Keras (which runs on top of Tensorflow) on a small dataset, but I did not use that with Jupyter. For that purpose I installed Tensorflow and Keras independently and used them in a Python script. However, it was not working from my Jupyter notebook. I googled for the solution, but found nothing concrete. I tried to activate the tensorflow environment and run jupyter notebook from their but in vein. I guess the reason is, I have downloaded different packages in different times and that might make some compatibility issues. Therefore, I decided to create a BRAND NEW conda environment for my deep learning endeavor. This is how it goes:

# create an environment named tensorflow35
shanto@shanto:~$ conda create --name tensorflow35 python=3.5
# following happens
Solving environment: done


==> WARNING: A newer version of conda exists. <==
  current version: 4.4.8
  latest version: 4.4.10

Please update conda by running

    $ conda update -n base conda



## Package Plan ##

  environment location: /home/shanto/anaconda3/envs/tensorflow35

  added / updated specs: 
    - python=3.5


The following NEW packages will be INSTALLED:

    ca-certificates: 2017.08.26-h1d4fec5_0
    certifi:         2018.1.18-py35_0     
    libedit:         3.1-heed3624_0       
    libffi:          3.2.1-hd88cf55_4     
    libgcc-ng:       7.2.0-h7cc24e2_2     
    libstdcxx-ng:    7.2.0-h7a57d05_2     
    ncurses:         6.0-h9df7e31_2       
    openssl:         1.0.2n-hb7f436b_0    
    pip:             9.0.1-py35h7e7da9d_4 
    python:          3.5.4-h417fded_24    
    readline:        7.0-ha6073c6_4       
    setuptools:      38.4.0-py35_0        
    sqlite:          3.22.0-h1bed415_0    
    tk:              8.6.7-hc745277_3     
    wheel:           0.30.0-py35hd3883cf_1
    xz:              5.2.3-h55aa19d_2     
    zlib:            1.2.11-ha838bed_2    

Proceed ([y]/n)? y

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
#     $ conda activate tensorflow35
#
# To deactivate an active environment, use
#
#     $ conda deactivate

Now we have this brand new environment name tensorflow35. We need to activate that environment and install all the necessary packages. As I am doing a brand new installation, I hope this time the packages will not have any compatibility issues.

If we activate the new environment, we see something like this. So we should see this (tensorflow35) thing in all the lines, if this new environment is activated.

shanto@shanto:~$ conda activate tensorflow35
(tensorflow35) shanto@shanto:~$

Now we can install all the necessary packages (Tensorflow, Keras, PyTorch, Numpy, Scipy, Pandas). But, lets install Jupyter notebook first.

(tensorflow35) shanto@shanto:~$ conda install jupyter
Solving environment: done
... ...
... ...
(tensorflow35) shanto@shanto:~$ conda install scipy
Solving environment: done
... ...
... ...
(tensorflow35) shanto@shanto:~$ conda install pandas
Solving environment: done
... ...
... ...
(tensorflow35) shanto@shanto:~$ conda install spyder
Solving environment: done
... ...
... ...
(tensorflow35) shanto@shanto:~$ pip install pyyaml h5py
Collecting pyyaml
 ... ...
... ...
Successfully installed h5py-2.7.1 pyyaml-3.12
# install Tensorflow using pip (pip is already installed globally and works fine here)
(tensorflow35) shanto@shanto:~$ pip install tensorflow
Collecting tensorflow
  Downloading tensorflow-1.5.0-cp35-cp35m-manylinux1_x86_64.whl (44.4MB)
    100% |████████████████████████████████| 44.4MB 33kB/s 
Requirement already satisfied: six>=1.10.0 in ./anaconda3/envs/tensorflow35/lib/python3.5/site-packages (from tensorflow)
... ...
... ...
Successfully installed absl-py-0.1.10 bleach-1.5.0 html5lib-0.9999999 markdown-2.6.11 protobuf-3.5.1 tensorflow-1.5.0 tensorflow-tensorboard-1.5.1 werkzeug-0.14.1
# install Keras
(tensorflow35) shanto@shanto:~$ pip install keras
Collecting keras
  Downloading Keras-2.1.4-py2.py3-none-any.whl (322kB)
    100% |████████████████████████████████| 327kB 2.1MB/s 
... ...
... ...
Successfully installed keras-2.1.4
# install opencv
(tensorflow35) shanto@shanto:~$ pip install opencv-python
Collecting opencv-python
... ...
... ...
Successfully installed opencv-python-3.4.0.12
# install matplotloib, seaborn, argspace
(tensorflow35) shanto@shanto:~$ pip install matplotlib seaborn argparse
... ...
... ...
# install PyTorch
(tensorflow35) shanto@shanto:~$ conda install -c soumith pytorch
Solving environment: done


==> WARNING: A newer version of conda exists. <==
  current version: 4.4.8
  latest version: 4.4.10
... ...
... ...

Proceed ([y]/n)? y


Downloading and Extracting Packages
pytorch 0.3.1: ######################################################################################### | 100% 
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
(tensorflow35) shanto@shanto:~$ 

Now, we are ready to run our notebook and test if everything works fine. We need to run Jupyter notebook from the same environment.

(tensorflow35) shanto@shanto:~$ jupyter notebook
[I 02:20:03.816 NotebookApp] Serving notebooks from local directory: /home/shanto
[I 02:20:03.816 NotebookApp] 0 active kernels
[I 02:20:03.816 NotebookApp] The Jupyter Notebook is running at:
... ...
... ...
... ...

This is what we get, if we try to import a few packages in our notebook.

We can see scikit-learn is not found. The reason is clear, we did not installed it. So lets install it like this.

(tensorflow35) shanto@shanto:~$ conda install scikit-learn
Solving environment: done
... ...
... ...
... ...

Downloading and Extracting Packages
scikit-learn 0.19.1: ################################################################################### | 100% 
Preparing transaction: done
Verifying transaction: done
Executing transaction: done

And try to load it again.

This way we can install any module we need to our new environment. Now, everything is set up for my journey with deep learning.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s