SETTING UP THE ENVIRONMENT

SETTING UP THE ENVIRONMENT

The following instructions are for Linux(Ubuntu) Operating System only.

The most popular distribution among data scientists is Anaconda. Anaconda is a distribution of packages like python, Jupyter Notebook,  IDE Spyder for python etc. In anaconda, you can create virtual environments to install required packages for a specific project. This way you can avoid dependency collisions between projects.You can download anaconda from the website. After downloading the file execute the command to start the installer.

 ./file.sh 

Anaconda asks if Path should be added to path variable at end of the installation. If you have answered no to that anaconda-navigator command won't work in the terminal. We should give entire path when executing the command. To add path open .profile file using vi .bashrc from home. To check if the file exists use ls -a command which displays hidden files. Add the following line at the end of the file. If you get an overwrite error while saving the file, close it and open it as superuser.

export PATH="/home/abhi/anaconda3/bin:$PATH" to end of the file.

To create a new environment use the following command from the terminal.

 conda create --name envname python=3.6

This creates the environment with python version 3.6. You can create the environment with another package by just using the package name. To activate the environment envname use

 source activate envname

The rest of the packages can be installed in the environment using pip or conda install 

The best way to export the environment is using yml files. We can then use these yml files to install the same environment with all the packages with single command. Activate the environment and execute the following command

conda env export > environment.yml

To install using the environment from yml file use the command

conda env create -f environment.yml

Installing Cuda
To see current Nvidia driver version and gcc version

 cat /proc/driver/nvidia/version

Adding the repository to ubuntu change the version number according to requirement: The following adds the Cuda 9.0 version

wget 'https://developer.nvidia.com/compute/cuda/9.0/Prod/local_installers/cuda_9.0.176_384.81_linux-run' -O cuda-run

sudo sh cuda-run

If latest nvidea driver is already installed answer no the question


Check /usr/local for cuda and cuda 9.0 folders to know if the installation is a success or not.

Adding cuda to the path: After adding this the user should either logout or restart the system.

# add cuda tools to command path
 export PATH=/usr/local/cuda/bin:${PATH}
 export MANPATH=/usr/local/cuda/man:${MANPATH}
 # add cuda libraries to library path
 if [[ "${LD_LIBRARY_PATH}" != "" ]]
    then
       export LD_LIBRARY_PATH=/usr/local/cuda/lib64:${LD_LIBRARY_PATH}
    else
       export LD_LIBRARY_PATH=/usr/local/cuda/lib64
 fi

Current version of Tensorflow(1.5) needs cuda 9.0 and Cudnn 7.5 to work. Cuda 9.1 is not supported in Tensorflow. Check Tensorflow website for other compitable versions.