In this tutorial, we will see how to compile and install FFmpeg with VMAF support using libvmaf and then we shall see how to compute or calculate the VMAF score between two videos using FFmpeg with VMAF support enabled.
FFmpeg has always been at the top of everyone’s mind when it came to open source video compression software and is best described by Wikipedia as
FFmpeg is a free and open-source project consisting of a vast software suite of libraries and programs for handling video, audio, and other multimedia files and streams.
VMAF from Netflix stands for Video Multi-method Assessment Fusion, and it is a video quality metric that combines human vision modeling with machine learning. It’s become very popular as it succeeds (not fully) at automating subjective quality testing that usually requires humans to watch and score videos.
FFmpeg and Netflix’s VMAF are now part of every video processing and compression engineer’s toolbox. But, installing and using them appears to be difficult for a large number of users.
I see these questions often on Stackoverflow and similar websites –
- how to compute VMAF?
- how do I compile VMAF on Ubuntu?
- how do I calcuate VMAF using FFmpeg?
- no such filter
libvmafwhen trying to use VMAF with FFmpeg.
- VMAF example commandline
and the list simply goes on. So, I decided to write down the steps of compiling and installing VMAF, FFmpeg, and computing VMAF with FFmpeg. Along the way, I’ll show you a few gotchas and common errors in the installation and usage procedure.
Step 0: Machine Setup
For the purpose of this document, I will set up a new $5 droplet on DigitalOcean with Ubuntu 18.04 as its operating system.By the way, if you click here, you’ll receive $100 in DigitalOcean credits over 60 days! That’s 60 days and $100 to experiment and play around with DigitalOcean 🙂
Step 1: Install VMAF on Ubuntu 18.04
You can use VMAF in several different ways and you can read about it here. In order to use VMAF with FFmpeg, we need the help of
libvmaf – a C library that can be used with C/C++ code. In other words, we can use it with FFmpeg.
So, here is how to install VMAF on an Ubuntu 18.04 machine –
Step 1: Download the latest release of VMAF from here.
Step 2: Untar it
untar -xvf https://github.com/Netflix/vmaf/archive/v1.5.2.tar.gz
Step 3: Install dependencies.
sudo apt update sudo apt install python3 python3-pip python3-setuptools python3-wheel ninja-build doxygen pip3 install meson pip3 install Cython pip3 install numpy
Step 4: Make and Install VMAF
cd vmaf-1.5.2/ sudo make sudo make install
Make a note of where VMAF’s
libvmaf got installed. In my machine, it was installed to
/usr/local/lib/x86_64-linux-gnu as per the console log.
Installing src/libvmaf.so.0.0.0 to /usr/local/lib/x86_64-linux-gnu Installing src/libvmaf.a to /usr/local/lib/x86_64-linux-gnu
Step 2: Install FFmpeg with VMAF support on Ubuntu 18.04
The first part is complete and you have successfully installed VMAF onto your machine. Now, let’s compile and install FFmpeg with VMAF support. These instructions are taken from FFmpeg’s Ubuntu Compilation Guide
Step 1: Install the Dependencies (including
nasm). And, remember to create the following folders
mkdir -p ~/ffmpeg_sources ~/bin
Step 2: Install the Video and Audio Codecs that you are going to need. I installed
Step 3: Make sure FFmpeg knows where to find
Step 4: Install
libunistring-dev to prevent the following error :
ERROR: gnutls not found using pkg-config
sudo apt-get install libunistring-dev
Step 5: Download and compile FFmpeg from sources. Notice that we are now using
--enable libvmaf to enable VMAF with FFmpeg.
cd ~/ffmpeg_sources && \ wget -O ffmpeg-snapshot.tar.bz2 https://ffmpeg.org/releases/ffmpeg-snapshot.tar.bz2 && \ tar xjvf ffmpeg-snapshot.tar.bz2 && \ cd ffmpeg && \ PATH="$HOME/bin:$PATH" PKG_CONFIG_PATH="$HOME/ffmpeg_build/lib/pkgconfig" ./configure \ --prefix="$HOME/ffmpeg_build" \ --pkg-config-flags="--static" \ --extra-cflags="-I$HOME/ffmpeg_build/include" \ --extra-ldflags="-L$HOME/ffmpeg_build/lib" \ --extra-libs="-lpthread -lm" \ --bindir="$HOME/bin" \ --enable-gpl \ --enable-gnutls \ --enable-libass \ --enable-libfdk-aac \ --enable-libfreetype \ --enable-libmp3lame \ --enable-libopus \ --enable-libvorbis \ --enable-libvpx \ --enable-libx264 \ --enable-libx265 \ --enable-libvmaf \ --enable-nonfree && \ PATH="$HOME/bin:$PATH" make && \ make install && \ hash -r
That’s it! You should now have an FFmpeg build with VMAF enabled.
Step 3: Computing FFmpeg with VMAF
Now let’s see how we can calculate the VMAF score between two videos using FFmpeg and VMAF.
Step 1: Let’s create two test videos – a 1920x1080p30 video and a 1280x720p30 video using FFmpeg’s built-in libraries that generates test data. The commandlines are shown below and as you can see, it will generate two 5-second duration clips, 30 fps, of 1920x1080p and 1280x720p resolution.
bin/ffmpeg -f lavfi -i testsrc=duration=5:size=1920x1080:rate=30 test_1080p30.mpg bin/ffmpeg -f lavfi -i testsrc=duration=5:size=1280x720:rate=30 test_720p30.mpg
Step 2: Compute the VMAF score between the two videos.
From the commandline below, you can see that we are providing the two input sequences, and then we are scaling both to 1920×1080 using a
bicubic filter. This is important because VMAF works only on sequences having the same resolution (and the frame rate should match, else you’ll have serious problems). We are also asking FFmpeg to compute the PSNR score for these two sequences.
bin/ffmpeg -i test_720p30.mpg -i test_1080p30.mpg -filter_complex "[0:v]scale=1920x1080:flags=bicubic[main]; [1:v]scale=1920x1080:flags=bicubic,format=pix_fmts=yuv420p,fps=fps=30/1[ref]; [main][ref]libvmaf=psnr=true:log_path=vmaflog.json:log_fmt=json" -f null -
and it should print the VMAF score as follows on the console
[libvmaf @ 0x55803f80c7c0] VMAF score: 86.375268
- The detailed per-frame details are contained in the JSON file. It contains the per-frame VMAF and PSNR scores as requested.
- In FFmpeg’s filter, you can use either
lancsozfilters for the upsampling process. Netflix suggests that
bicubicis a better choice.
Options for VMAF using FFmpeg
The following commandline options are available in FFmpeg for computing VMAF.
model_pathSet the model path which is to be used for SVM. Default value:
log_pathSet the file path to be used to store logs.
log_fmtSet the format of the log file (xml or json).
enable_transformThis option can enable/disable the score_transform applied to the final predicted VMAF score, if you have specified score_transform option in the input parameter file passed to run_vmaf_training.py Default value:
phone_modelInvokes the phone model which will generate VMAF scores higher than in the regular model, which is more suitable for laptop, TV, etc. viewing conditions. Default value:
psnrEnables computing psnr along with vmaf. Default value:
ssimEnables computing ssim along with vmaf. Default value:
ms_ssimEnables computing ms_ssim along with vmaf. Default value:
poolSet the pool method to be used for computing vmaf. Options are
n_threadsSet number of threads to be used when computing vmaf. Default value:
0, which makes use of all available logical processors.
n_subsampleSet interval for frame subsampling used when computing vmaf. Default value:
enable_conf_intervalEnables confidence interval. Default value:
Common problems while using VMAF and FFmpeg.
P1. While installing FFmpeg from the official compilation guide on Ubuntu 18.04, you will face this error
ERROR: gnutls not found using pkg-config. To prevent this, you need to install
sudo apt-get install libunistring-dev.
P2. Another common problem when running FFmpeg with VMAF is this
error libvmaf xxx not found using pkg config. This means that while compiling FFmpeg, you didn’t tell it where the
libvmaf files are. Simply run the export command and it should fix the issue.
I hope you were able to follow this guide and could successfully install VMAF, FFmpeg and compute the VMAF score using FFmpeg. If you faced any issues, let me know via email (check the contact page).
Take care and see you next time!