VMAF in FFmpeg – Installation and Usage Guide for Ubuntu

In this tutorial, we will see how to compile and install FFmpeg with VMAF support using libvmaf and then we shall see how to compute or calculate the VMAF score between two videos using FFmpeg with VMAF support enabled.

---

FFmpeg has always been at the top of everyone’s mind when it came to open source video compression software and is best described by Wikipedia as

FFmpeg is a free and open-source project consisting of a vast software suite of libraries and programs for handling video, audio, and other multimedia files and streams.

VMAF from Netflix stands for Video Multi-method Assessment Fusion, and it is a video quality metric that combines human vision modeling with machine learning. It’s become very popular as it succeeds (not fully) at automating subjective quality testing that usually requires humans to watch and score videos.

FFmpeg and Netflix’s VMAF are now part of every video processing and compression engineer’s toolbox. But, installing and using them appears to be difficult for a large number of users.

I see these questions often on Stackoverflow and similar websites –

• how to compute VMAF?
• how do I compile VMAF on Ubuntu?
• how do I calcuate VMAF using FFmpeg?
• no such filter libvmaf when trying to use VMAF with FFmpeg.
• VMAF example commandline

and the list simply goes on. So, I decided to write down the steps of compiling and installing VMAF, FFmpeg, and computing VMAF with FFmpeg. Along the way, I’ll show you a few gotchas and common errors in the installation and usage procedure.

Step 0: Machine Setup

For the purpose of this document, I will set up a new $5 droplet on DigitalOcean with Ubuntu 18.04 as its operating system.By the way, if you click here, you’ll receive $100 in DigitalOcean credits over 60 days! That’s 60 days and $100 to experiment and play around with DigitalOcean 🙂

Step 1: Install VMAF on Ubuntu 18.04

You can use VMAF in several different ways and you can read about it here. In order to use VMAF with FFmpeg, we need the help of libvmaf – a C library that can be used with C/C++ code. In other words, we can use it with FFmpeg.

So, here is how to install VMAF on an Ubuntu 18.04 machine –

Step 1: Download the latest release of VMAF from here.

wget https://github.com/Netflix/vmaf/archive/v1.5.2.tar.gz

Step 2: Untar it

untar -xvf https://github.com/Netflix/vmaf/archive/v1.5.2.tar.gz

Step 3: Install dependencies.

sudo apt update
sudo apt install python3 python3-pip python3-setuptools python3-wheel ninja-build doxygen
pip3 install meson
pip3 install Cython
pip3 install numpy

Step 4: Make and Install VMAF

cd vmaf-1.5.2/
sudo make 
sudo make install

Make a note of where VMAF’s libvmaf got installed. In my machine, it was installed to /usr/local/lib/x86_64-linux-gnu as per the console log.

Installing src/libvmaf.so.0.0.0 to /usr/local/lib/x86_64-linux-gnu
Installing src/libvmaf.a to /usr/local/lib/x86_64-linux-gnu

Step 2: Install FFmpeg with VMAF support on Ubuntu 18.04

The first part is complete and you have successfully installed VMAF onto your machine. Now, let’s compile and install FFmpeg with VMAF support. These instructions are taken from FFmpeg’s Ubuntu Compilation Guide

Step 1: Install the Dependencies (including nasm). And, remember to create the following folders

mkdir -p ~/ffmpeg_sources ~/bin

Step 2: Install the Video and Audio Codecs that you are going to need. I installed libx264, libx265-dev, libnuma-dev, libvpx-dev, libfdk-aac-dev, libmp3lame-dev, and libopus-dev.

Step 3: Make sure FFmpeg knows where to find libvmaf

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib/x86_64-linux-gnu/

Step 4: Install libunistring-dev to prevent the following error : ERROR: gnutls not found using pkg-config

sudo apt-get install libunistring-dev

Step 5: Download and compile FFmpeg from sources. Notice that we are now using --enable libvmaf to enable VMAF with FFmpeg.

cd ~/ffmpeg_sources && \
wget -O ffmpeg-snapshot.tar.bz2 https://ffmpeg.org/releases/ffmpeg-snapshot.tar.bz2 && \
tar xjvf ffmpeg-snapshot.tar.bz2 && \
cd ffmpeg && \
PATH="$HOME/bin:$PATH" PKG_CONFIG_PATH="$HOME/ffmpeg_build/lib/pkgconfig" ./configure \
  --prefix="$HOME/ffmpeg_build" \
  --pkg-config-flags="--static" \
  --extra-cflags="-I$HOME/ffmpeg_build/include" \
  --extra-ldflags="-L$HOME/ffmpeg_build/lib" \
  --extra-libs="-lpthread -lm" \
  --bindir="$HOME/bin" \
  --enable-gpl \
  --enable-gnutls \
  --enable-libass \
  --enable-libfdk-aac \
  --enable-libfreetype \
  --enable-libmp3lame \
  --enable-libopus \
  --enable-libvorbis \
  --enable-libvpx \
  --enable-libx264 \
  --enable-libx265 \
  --enable-libvmaf \
  --enable-nonfree && \
PATH="$HOME/bin:$PATH" make && \
make install && \
hash -r

That’s it! You should now have an FFmpeg build with VMAF enabled.

Step 3: Computing FFmpeg with VMAF

Now let’s see how we can calculate the VMAF score between two videos using FFmpeg and VMAF.

Step 1: Let’s create two test videos – a 1920x1080p30 video and a 1280x720p30 video using FFmpeg’s built-in libraries that generates test data. The commandlines are shown below and as you can see, it will generate two 5-second duration clips, 30 fps, of 1920x1080p and 1280x720p resolution.

bin/ffmpeg -f lavfi -i testsrc=duration=5:size=1920x1080:rate=30 test_1080p30.mpg
bin/ffmpeg -f lavfi -i testsrc=duration=5:size=1280x720:rate=30 test_720p30.mpg

Step 2: Compute the VMAF score between the two videos.
From the commandline below, you can see that we are providing the two input sequences, and then we are scaling both to 1920×1080 using a bicubic filter. This is important because VMAF works only on sequences having the same resolution (and the frame rate should match, else you’ll have serious problems). We are also asking FFmpeg to compute the PSNR score for these two sequences.

bin/ffmpeg -i test_720p30.mpg -i test_1080p30.mpg -filter_complex "[0:v]scale=1920x1080:flags=bicubic[main]; [1:v]scale=1920x1080:flags=bicubic,format=pix_fmts=yuv420p,fps=fps=30/1[ref]; [main][ref]libvmaf=psnr=true:log_path=vmaflog.json:log_fmt=json" -f null -

and it should print the VMAF score as follows on the console

[libvmaf @ 0x55803f80c7c0] VMAF score: 86.375268

Note:

1. The detailed per-frame details are contained in the JSON file. It contains the per-frame VMAF and PSNR scores as requested.
2. In FFmpeg’s filter, you can use either bicubic or lancsoz filters for the upsampling process. Netflix suggests that bicubic is a better choice.

Options for VMAF using FFmpeg

The following commandline options are available in FFmpeg for computing VMAF.

• model_path Set the model path which is to be used for SVM. Default value: /usr/local/share/model/vmaf_v0.6.1.pkl
• log_path Set the file path to be used to store logs.
• log_fmt Set the format of the log file (xml or json).
• enable_transform This option can enable/disable the score_transform applied to the final predicted VMAF score, if you have specified score_transform option in the input parameter file passed to run_vmaf_training.py Default value: false
• phone_model Invokes the phone model which will generate VMAF scores higher than in the regular model, which is more suitable for laptop, TV, etc. viewing conditions. Default value: false
• psnr Enables computing psnr along with vmaf. Default value: false
• ssim Enables computing ssim along with vmaf. Default value: false
• ms_ssim Enables computing ms_ssim along with vmaf. Default value: false
• pool Set the pool method to be used for computing vmaf. Options are min, harmonic_mean or mean (default).
• n_threads Set number of threads to be used when computing vmaf. Default value: 0, which makes use of all available logical processors.
• n_subsample Set interval for frame subsampling used when computing vmaf. Default value: 1
• enable_conf_interval Enables confidence interval. Default value: false

Common problems while using VMAF and FFmpeg.

P1. While installing FFmpeg from the official compilation guide on Ubuntu 18.04, you will face this error ERROR: gnutls not found using pkg-config. To prevent this, you need to install sudo apt-get install libunistring-dev.

P2. Another common problem when running FFmpeg with VMAF is this error libvmaf xxx not found using pkg config. This means that while compiling FFmpeg, you didn’t tell it where the libvmaf files are. Simply run the export command and it should fix the issue.

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib/x86_64-linux-gnu/

That’s It

I hope you were able to follow this guide and could successfully install VMAF, FFmpeg and compute the VMAF score using FFmpeg. If you faced any issues, let me know via email (check the contact page).

Take care and see you next time!

Check out EasyVMAF – a tool for making VMAF calculations simple.