The Ultimate Guide to Installing NVIDIA Drivers, CUDA Toolkit, cuDNN and NVIDIA-Docker on Ubuntu 20.04 for Deep Learning Activities using RTX3080/RTX3090

So one fine morning, I had this discussion with some of my colleagues and they all shared the same pain — Installing the Prerequisites for Deep Learning Tasks in Ubuntu machines.

So that was the main reason why I thought to conquer this task and get a feel of where the difficulty is. But to make things easier for me, our company bought 3 brand new machines with RTX 3090 cards with the below system information:

I also tried the same steps to install my RTX 3080 machine for my personal use as well.

So I decided to try out installing the libraries and eventually came up with this document as a guide to install this setup in an Ubuntu 20.04 Linux system.

So let’s begin the installation!

Step-1

Install Ubuntu 20.04.3 LTS onto the new machine and continue with the login. I used balenaEtcher application to install Ubuntu via USB into the machines. I didn’t install any third party libraries during the Ubuntu installation.

Step-2

Run the updates, upgrades, removes and the autoremove to remove any unwanted library installation present in the machine

sudo apt-get -y update
sudo apt-get -y upgrade

Run the cleanup and auto-removes to remove any previous installation of nvidia or cuda already present in the machine

sudo apt-get —y purge remove “*cublas*” “*cufft*” “*curand*” “*cusolver*” “*cusparse*” “*npp*” “*nvjpeg*” “cuda*” “nsight*” “*nvidia*”
sudo apt-get —y purge remove “*nvidia*”
sudo apt-get -y autoremove

Step-3

Install the CUDA from the CUDA repository

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/cuda-ubuntu2004.pin
sudo mv cuda-ubuntu2004.pin /etc/apt/preferences.d/cuda-repository-pin-600
wget https://developer.download.nvidia.com/compute/cuda/11.4.3/local_installers/cuda-repo-ubuntu2004-11-4-local_11.4.3-470.82.01-1_amd64.deb
sudo dpkg -i cuda-repo-ubuntu2004–11–4-local_11.4.3–470.82.01–1_amd64.deb
sudo apt-key add /var/cuda-repo-ubuntu2004–11–4-local/7fa2af80.pub
sudo apt-get update
sudo apt-get -y install cuda

Step-4

Now open ~/.bashrc file in a terminal and add the below lines to the bottom of the file

export PATH=/usr/local/cuda-11.4/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda-11.4/lib64:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/usr/local/cuda-11.4/include:$LD_LIBRARY_PATH

Now close the terminal and open a new terminal. The above env variables would be updated in this new session.

Step-5

wget http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu2004/x86_64/nvidia-machine-learning-repo-ubuntu2004_1.0.0-1_amd64.deb
sudo apt-get -y install ./nvidia-machine-learning-repo-ubuntu2004_1.0.0–1_amd64.deb

Step-6

Reboot the machine.

Step-7

We can check the nvidia version

cat /proc/driver/nvidia/version

The output of the above command will look similar to the one shown below:

NVRM version: NVIDIA UNIX x86_64 Kernel Module  470.82.01  Wed Oct 27 21:21:55 UTC 2021
GCC version:  gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)

The we can check the NVIDIA (R) Cuda compiler driver version by executing the below command:

nvcc -V

The output of the above command will look similar to the one shown below:

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Mon_Oct_11_21:27:02_PDT_2021
Cuda compilation tools, release 11.4, V11.4.152
Build cuda_11.4.r11.4/compiler.30521435_0

Now we can also check the NVIDIA System Management Interface by executing the below command:

nvidia-smi

The output of the above command will look similar to the one shown below:

And finally we can also check the NVIDIA Persistence Daemon status by executing the below command:

systemctl status nvidia-persistenced

The output of the above command will look similar to the one shown below:

Step-8

Now we need to install NVIDIA cuDNN which is a GPU-accelerated library of primitives for deep neural networks.. For that just go here and download the version I am going to list below. You’ll have to log in, so downloading of the cuDNN binary packages has to be down manually, unless you decide to host it somewhere else specifically for your installation process (I have a local hosted version of the libraries).

I used this version cuDNN v8.2.4 (September 2nd, 2021), for CUDA 11.4 as shown below from the cuDNN archive.

After the debian packages are downloaded, you can install the librareis by executing the below commands:

sudo dpkg -i libcudnn8_8.2.4.15-1+cuda11.4_amd64.deb
sudo dpkg -i libcudnn8-dev_8.2.4.15-1+cuda11.4_amd64.deb
sudo dpkg -i libcudnn8-samples_8.2.4.15-1+cuda11.4_amd64.deb

Step-9

The final step is to install NVIDIA-Docker and for this please execute the below commands in the given order. This basically installs Docker CE into the machine and then gives the docker permissions to be run without adding sudo in front of the docker commands.

sudo apt-get -y remove docker docker-engine docker.io containerd runc
sudo apt-get -y update
sudo apt-get -y install apt-transport-https ca-certificates curl gnupg lsb-release
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
echo "deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get -y update
sudo apt-get -y install docker-ce docker-ce-cli containerd.io
sudo groupadd docker
sudo usermod -aG docker $USER
newgrp docker

Once the Docker-CE is installed, now we will install the Docker-Compose Utility in the machine by executing the below commands:

sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
docker-compose --version
docker -v

And now the final step is to install the NVIDIA-Docker utility into the machine and this can be done by executing the below commands:

distribution=$(. /etc/os-release;echo $ID$VERSION_ID) && curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -    && curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list
sudo apt-get update
sudo apt-get install -y nvidia-docker2
sudo systemctl restart docker

Step-10

Now you can test the NVIDIA-Docker installation by executing the below commands:

docker run --rm --gpus all nvidia/cuda:11.4.0-cudnn8-runtime-ubuntu20.04 nvidia-smi

The output of the above command will look similar to the one shown below:

Final Steps:

Now we can go ahead and install Anaconda or Miniconda to setup the Python Environments and then install Pytorch and Tensorflow.

Your system is now ready for all your Deep Learning Adventures.