BD-Rate and BD-PSNR are objective measures used in video compression to compare the rate-distortion performance or compression efficiency of two different video codecs or different settings of the same video codec over a range of bitrate or quality values. This article looks at the BD-Rate & BD-PSNR metrics and teaches how to calculate and interpret BD-Rate readings.
Rate-Distortion (RD) Plots
When you evaluate a codec’s performance, you generally arrive at a rate-distortion (RD) curve that informs you about the quality (PSNR in dB) at different bitrates (typically, in kbps or Mbps). The BD-Rate metric uses the information from the RD curves and tells you how much one codec has improved over the other.
But, first, let’s learn about RD Curves/Plots that are common in the field of video compression.
We know that video compression is the process of reducing the size of a video either for storage or transmission requirements. And that the process of video compression will affect the video quality, and more often than not, reduce the video quality.
So, the goal of video compression is to preserve the video quality, or reduce the level of distortion while compressing the video! To learn more about video compression, check out the articles in OTTVerse’s Video Compression series.
When you start working with a video codec, the first thing you do is examine its performance by doing the following –
- Choose certain codec settings (tunings) and don’t change them.
- Compress several videos with different spatial and temporal features across a series of increasing bitrates. For each compression run, note down the bitrate achieved by the codec and the quality using an objective metric like PSNR (dB) or SSIM.
- Tabulate these results and then plot them with Bitrate on the x-axis and Quality on the y-axis.
Let’s try this out. We will use the same codec but choose different tunings or settings and run our experiments on the same video at four different CBR settings – 100, 2000, 3000, 4000 kbps. For the sake of this tutorial, we assume that our codec produced the exact bitrate that we set (highly unlikely in the real world), and we calculate the PSNR for each experiment using FFmpeg (here’s a step-by-step tutorial for calculating the PSNR using FFmpeg).
After all the interations, we arrive at the following table with the Bitrate-PSNR results.
Then, we plot these results to get RD-Plots or RD-Curves. An RD Plot is a convinient way of visually comparing two different codecs, or two different settings of the same codec, as seen below.
From these RD curves, it is quite obvious that Settings B performs much better than Settings A for this particular video at the chosen encoding conditions.
But, how do we qualify these differences mathematically, via a single number that tells us almost everything we need to know about the difference between the two curves in the RD Plot.
That’s where the BD-Rate analysis comes in.
BD-Rate or Bjontegaard Delta-Rate
The Bjontegaard metric calculation involves two parts – BD PSNR and BD Rate. It was introduced and explained in a paper titled “G. Bjontegaard, Calculation of average PSNR differences between RD-curves (VCEG-M33) [2] S. Pateux, J. Jung, An excel add-in for computing Bjontegaard metric and its evolution” and has since provided a nice of comparing the RD performance of two different codecs or two different settings for the same codec.
The calcuation of the BD-Rate or BD-PSNR is complex and I’ve reproduced the text from the IETF doc here. If you don’t understand it, then don’t worry!
1. Rate/distortion points are calculated for the reference and test codec.
2. At least four points must be computed. These points should be the same quantizers when comparing two versions of the same codec.
3. Additional points outside of the range should be discarded.
4. The rates are converted into log-rates.
5. A piecewise cubic hermite interpolating polynomial is fit to the points for each codec to produce functions of log-rate in terms of distortion.
Metric score ranges are computed:
1. If comparing two versions of the same codec, the overlap is the intersection of the two curves, bound by the chosen quantizer points.
2. If comparing dissimilar codecs, a third anchor codec’s metric scores at fixed quantizers are used directly as the bounds.
3. The log-rate is numerically integrated over the metric range for each curve, using at least 1000 samples and trapezoidal integration.
4. The resulting integrated log-rates are converted back into linear rate, and then the percent difference is calculated from the reference to the test codec.Luckily, the authors have provided an Excel sheet with a plugin that makes the calculations very simple. We don’t have to worry about the explanation above (phew!)
All you need to do is go to their GitHub repo (https://github.com/tbr/bjontegaard_etro) and download both the Excel sheet and the Excel add-in. You’ll get them both if you download the zip file. By the way, the instructions are unambiguous on the repo’s README.md – don’t skip it 🙂
Here’s what you need to do use the Excel sheet to compute BD-Rate and BD-PSNR.
Step 1: Unpack the zip file and enter the resulting folder.
Step 2: Open bjontegaard_etro_standalone_example using Microsoft Excel and approve the dialog box that pops up – Click “Update”. This picks up the bjontegaard_etro Excel add-in file that contains the calculations and math.
Step 3: Go to the third tab titled “Verification with VCEG-M33” and you’ll find a table with the row headings “Curve1” and “Curve2”. That’s where you need to enter your data from the tests you ran (look at the table above to understand how to fetch and enter the data).
After you enter the data, you’ll see that two values are calculated automatically
- B-DSNR (dB) — same as BD-PSNR
- B-DBR (%) — same as BD-Rate
These are the BD values that you need for your analysis.
Let’s look at what they mean.
B-DSNR (dB): This tells you about the quality gains made by Curve2 vs. Curve1 at the equivalent bitrate. In other words, if you spend the same amount of bits in Codec Setting 1 and 2, then which one has better quality and by how much?
- So, if the value is 5 dB, then it means that the second settings give 5 dB more quality at equivalent bitrates than the first settings.
- Also, the calculations make sense whether you use SSIM or VMAF as well, instead of PSNR. If you use some other objective quality metric, make sure you ignore the dB units.
B-DBR (%): This tells you the bitrate savings at the equivalent quality and is also referred to as the BD-Rate value. In other words, if the quality of both codec settings are the same, then which one uses more bits and how much?
- If the value of B-DBR is 50%, it means that Curve2 needs 50% more bits than Curve1 to achieve the same quality.
- If the value is -25%, it means that Curve2 needs 25% fewer bits than Curve1 to achieve the same video quality.
- Be careful with this interpretation and the +/- signs.
In our experiment, we get the following results –
- B-DBR (%) = -19.893 %
- B-DSNR (dB) = 1.833 dB
This means, that,
- Settings B saves 19.893 % bits over Settings A for the same quality; Or, if you use Settings B, you need to spend 19.893% fewer bits to achieve the same quality as Settings A.
- And, Settings B get 1.833 dB more video quality than Settings A for the same number of bits spent.
This confirms what we concluded by looking at the RD curves.
Conclusion
So, that’s how you compute the BD-Rate and BD-PSNR from your rate-distortion (RD) experiments to get an idea of the compression efficiency of one codec over another or one group of codec settings over another for the same codec.
Until next time, keep streaming and stay safe.
