eddy implemented in FSL is time-consuming program. FSL recommends using eddy_cuda, GPU version of eddy. They ship eddy_cuda8.0 and edddy_cuda9.1. If you use Ubuntu 18.04, you can make use of eddy_cuda9.1 with only 4 commands.
Disclaimer: Installing nvidia-driver could cause display problem. I am not responsible for the problem…
Summary
- Type the following 4 commands and you’re set to go.
sudo update-pciids
sudo ubuntu-drivers autoinstall
(reboot)
sudo apt install nvidia-cuda-toolkit
ln -sf ${FSLDIR}/bin/eddy_cuda9.1 ${FSLDIR}/bin/eddy_cuda
How to make use of eddy_cuda of FSL 6.0.4 on Ubuntu 18.04
running topup and eddy_openmp for comparison
- I ran dwifslpreproc in mritrix3 to measure the processing time of topup and eddy for a dwi image with 30 axes.
real 17m25.986s user 85m45.748s sys 0m56.166s
It took 85min for one subject. It would be a long way to finish eddy for all subjects…
Download the latest PCI ID list
update-pciidswill download the latest PCI ID list. This will help your PC identify the graphic board on your PC.
sudo update-pciids
Check your graphic board
- The command below will show you installed graphic board.
lspci | grep -i nvidia
- Below is the result of mine. I use GeForce RTX 2080 Ti.
01:00.0 VGA compatible controller: NVIDIA Corporation TU102 [GeForce RTX 2080 Ti Rev. A] (rev a1) 01:00.1 Audio device: NVIDIA Corporation TU102 High Definition Audio Controller (rev a1) 01:00.2 USB controller: NVIDIA Corporation TU102 USB 3.1 Host Controller (rev a1) 01:00.3 Serial bus controller [0c80]: NVIDIA Corporation TU102 USB Type-C UCSI Controller (rev a1)
Check GPU drivers
ubuntu-drivers devicesshows you candidate drivers.
ubuntu-drivers devices == /sys/devices/pci0000:00/0000:00:01.1/0000:02:00.0 == modalias : pci:v000010DEd00001E07sv000010DEsd000012FAbc03sc00i00 vendor : NVIDIA Corporation driver : nvidia-driver-450 - distro non-free driver : nvidia-driver-460 - distro non-free recommended driver : nvidia-driver-450-server - distro non-free driver : nvidia-driver-418-server - distro non-free driver : xserver-xorg-video-nouveau - distro free builtin
As of Jan 22, 2021, nvidia-driver-460 is recommended. I decided to use this one.
In stall a GPU driver
ubuntu-drivers autoinstallwill install the recommended driver.
sudo ubuntu-drivers autoinstall
Reboot
- Reboot your machine to reflect the installation.
Check installation
- You can check your driver installation by running
nvidia-smi. nvidia-smi stands for “NVIDIA System Management Interface program.”
nvidia-smi Fri Jan 22 19:53:41 2021 +-----------------------------------------------------------------------------+ | NVIDIA-SMI 460.32.03 Driver Version: 460.32.03 CUDA Version: 11.2 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |===============================+======================+======================| | 0 GeForce RTX 208... Off | 00000000:01:00.0 On | N/A | | 0% 42C P8 22W / 260W | 230MiB / 11016MiB | 1% Default | | | | N/A | +-------------------------------+----------------------+----------------------+ +-----------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=============================================================================| | 0 N/A N/A 1569 G /usr/lib/xorg/Xorg 118MiB | | 0 N/A N/A 2602 G ...gAAAAAAAAA --shared-files 106MiB | +-----------------------------------------------------------------------------+
- Note that it shows “CUDA Version: 11.2” though CUDA is not installed yet.
Installation of CUDA 9.1
- FSL 6.0.4 ships eddy_cuda8.0 and eddy_cuda9.1. If you can install CUDA 8.0 or 9.1, you can use these programs. Ubuntu has
nvidia-cuda-toolkitto install CUDA-related programs. Let’s check which version will be installed byapt show nvidia-cuda-toolkit.
apt show nvidia-cuda-toolkit Package: nvidia-cuda-toolkit Version: 9.1.85-3ubuntu1 Priority: extra Section: multiverse/devel Origin: Ubuntu Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com> Original-Maintainer: Debian NVIDIA Maintainers <pkg-nvidia-devel@lists.alioth.debian.org> Bugs: https://bugs.launchpad.net/ubuntu/+filebug Installed-Size: 61.0 MB Depends: nvidia-profiler (= 9.1.85-3ubuntu1), nvidia-cuda-dev (= 9.1.85-3ubuntu1), nvidia-opencl-dev (= 9.1.85-3ubuntu1) | opencl-dev, gcc-6 | clang-4.0 | clang-3.9 | clang-3.8 | clang (<< 1:5~) | gcc-5 | gcc-4.9 | gcc-4.8, g++-6 | clang-4.0 | clang-3.9 | clang-3.8 | clang (<< 1:5~) | g++-5 | g++-4.9 | g++-4.8, libc6 (>= 2.4), libgcc1 (>= 1:3.0), libnvvm3 (>= 7.0), libstdc++6 (>= 4.1.1)
We see cuda 9.1 will be installed, so we will use this.
sudo apt install nvidia-cuda-toolkit
Check version of CUDA
nvcc -Vwill show you which version of CUDA is used.
nvcc -V nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2017 NVIDIA Corporation Built on Fri_Nov__3_21:07:56_CDT_2017 Cuda compilation tools, release 9.1, V9.1.85
We see CUDA 9.1 is used.
Prepare eddy_cuda
- eddyのUsersGuide describes how to prepare eddy_cuda. We prepare symbolic link for eddy_cuda9.1. You need to use sudo to make a symbolic link.
sudo ln -sf ${FSLDIR}/bin/eddy_cuda9.1 ${FSLDIR}/bin/eddy_cuda
Now you’re ready to go!
Run topup and eddy_cuda
- I ran dwifslpreproc for comparison with eddy_openmp
real 7m2.535s user 7m3.179s sys 0m16.221s
It’s 12 times faster!
