Steps to install CUDA 11.7, cuDNN 8, and nvidia-docker2 on Ubuntu 22.04. Ubuntu is prepared with minimal installation and third-party packages installed.

CUDA 11.7

If you install third-party packages, CUDA gets installed automatically, so remove everything. (It would be better not to install third-party packages, but it’s troublesome when WiFi drivers don’t get installed)

sudo apt purge *nvidia*
sudo apt autopurge
sudo apt purge nvidia-*
sudo apt purge nvidia-compute-*
sudo apt purge libnvidia*
sudo apt purge linux-objects-nvidia*

Install CUDA from NVIDIA official.

CUDA Toolkit 12.1 Downloads

Get the latest feature updates to NVIDIA's proprietary compute stack.

developer.nvidia.com

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-ubuntu2204.pin
sudo mv cuda-ubuntu2204.pin /etc/apt/preferences.d/cuda-repository-pin-600
sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/3bf863cc.pub
sudo add-apt-repository "deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/ /"
sudo apt-get update
sudo apt-get -y install cuda
sudo apt-get -y install cuda-drivers

Restart once here.

To check if GPU is visible, add CUDA to PATH.

# CUDA
export PATH=/usr/local/cuda/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH

Try nvidia-smi and nvcc -V.

Installing cuDNN

Installing cuDNN requires NVIDIA membership registration. Download from the following: Install Local Installer Ubuntu2*.04 x86_64. For 22.04, installing the 20.04 version worked fine.

CUDA Deep Neural Network

cuDNN provides researchers and developers with high-performance GPU acceleration.

developer.nvidia.com

docs.nvidia.com

docs.nvidia.com

Install according to the installation guide above. This time I chose dpkg:

sudo dpkg -i cudnn-local-repo-${OS}-8.x.x.x_1.0-1_amd64.deb
sudo apt-key add /var/cudnn-local-repo-*/7fa2af80.pub
sudo apt-get install libcudnn8
sudo apt-get install libcudnn8-dev

Installing Docker

Install Docker according to the official site procedures. The GUI version Docker-Desktop for Linux is recommended, but Docker Engine alone is sufficient for CLI operation. Note that Desktop has restrictions on commercial use and number of users.

Ubuntu

Jumpstart your client-side server applications with Docker Engine on Ubuntu. This guide details prerequisites and multiple methods to install Docker Engine on Ubuntu.

docs.docker.com

To run docker commands without sudo, add the user to the Docker group.

sudo groupadd docker
sudo gpasswd -a $USER docker
sudo systemctl restart docker

Log out and log back in after this.

Installing NVIDIA-Docker-2

docs.nvidia.com

docs.nvidia.com

Install according to the above site.

distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \
      && curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
      && curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | \
            sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
            sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt-get update
sudo apt-get install -y nvidia-docker2
sudo systemctl restart docker

Test if GPU is visible from container.

sudo docker run --rm --gpus all nvidia/cuda:11.0.3-base-ubuntu20.04 nvidia-smi

Caution

If you run nvcc without having CUDA in PATH, you’ll be told to install cuda-toolkit, but don’t install it. Installing it will remove the latest CUDA 11.7 and fall back to 11.5 that was installed right after installation.