What is Video Transcoding and Why is it Critical to Video Streaming

It is a common fact that almost 80% of the internet traffic is made up of video, thus, putting the onus on video content providers to stream more efficiently and in a greener manner. A large part of the effort to stream more efficiently boils down to compressing or transcoding video better and delivering fewer bits across the Internet for the same video quality.

In this article, we talk about video transcoding to understand what it means, what factors affect it, and finally, learn why transcoding is so critical to the video streaming industry and OTT in general.

So, lets get started and understand all there is to know about video transcoding!

By the way, if you are interested in going deeper into the science and art of video transcoding and compression, please be sure to check out OTTVerse’s Video Compression and Transcoding articles.

What is Video Transcoding

Video transcoding is the art and science of taking an encoded video file, decode it into an intermediary format, and then re-encode it into the final video format (combination of file type, codec, resolution, etc.). The output can different from the input in terms of the video codec used, video resolution, bitrate, framerate, etc.

Thus, a transcoder must be capable of

• decoding the incoming video (i.e., capable of understanding the video codec employed to create the input video) to create a raw, intermediary file.
• convert the raw file to a different resolution (image processing)
• encode the new file to the final output based on the encoder settings.

Since the process of transcoding also involves decoding and encoding, there is a lot of confusion around the phrases “Video Encoding” and “Video Transcoding”. You’d see those words being used inter-changeably leading to a lot of confusion, but, let’s clear that up now.

• Video Transcoding refers to re-encoding compressed files into a specified bitrate, resolution, and quality. And hence, video transcoding has to also decode the incoming video before encoding it (or rather re-encode it). Think of the word transcoding to be a combination of “trans” and “coding”
  • Trans – converting from one format to another
  • Coding – compressing a video
• Video Encoding typically refers to the process of compressing raw, uncompressed video to a desired size, bitrate, and quality.

While we are on the topic of definitions, let’s also talk quickly about Transrating and Transmuxing.

• Transrating is the process of changing the bitrate of the video without changing the resolution, framerate, etc. For example, you might convert 1080p @ 5mbps to 720p @ 3mbps.
• Transmuxing is the act of changing the container format (e.g. mp4 to avi or ts).

Okay, now that we have a basic idea of what Video Transcoding is, let’s move on to understanding how it works.

How does Video Transcoding Work?

The process of Video transcoding can be broken down roughly into three buckets –

1. Decode the Incoming Video and Convert it into a Raw Video Format
   • a transcoder needs to be able to decode different container formats (mp4, ts) to extract the underlying packets of video.
   • Then, the transcoder needs to decode the video packets to extract the raw video. This requires the ability to decode bitstreams using different video codecs such as H.264/AVC, HEVC, AV1, VP9, etc.
2. Perform any required image processing, filtering, and prepare the video for encoding
   • In this step, the raw video is converted to multiple different resolutions (if required).
   • Then the videos are filtered to remove noise, sharpened, color enhanced, and logos are inserted (based on the configuration)
3. Encode the video to the final formats
   • the processed raw video is then re-encoded based on the configuration (bitrate, resolution, frame-rate, codec profile, etc.)
   • The output is stored in a standard file format such as mp4 or ts to complete the transcoding process.

Now, let’s take a look at some of the most important factor that impact video transcoding.

PS: Learn the difference between bitrate and resolution here.

Important Factors that Impact Video Transcoding

There are several factors that impact video transcoding such as the output video resolutions, bitrates, framerates, choice of video codecs, presets, rate control modes, and so much more. These factors have a big impact on the transcoder’s speed, cost, quality and size of the output video. Let’s see why –

• Resolution – you need to decide what resolutions you want the output of the transcoding in – 1920×1080, 1280×720, etc.
• Bitrate – and then, you need to decide the bitrate for each resolution. i.e., 1920×1080 at 5 mbps, 1280×720 at 2.5 mbps, etc.
• Codec – finally, one of the most important factors is the choice of video codec. Are you going to use H.264/AVC, H.265/HEVC, AV1, or something else? And, which profile are you going to choose within the codec (baseline, main, etc.).

All of these parameters have a huge impact on video transcoding and should be chosen very carefully. Next, let’s move on to the types of video transcoders and see what they’re about.

PS: If you want to learn more about video codecs, then check out these articles on OTTVerse.

• Creating the Perfect Bitrate Ladder for Video Encoding
• Explanation of the Block-based Transforms in VVC (Versatile Video Coding)
• What is Per-Title Encoding?
• BD-Rate & BD-PSNR: Calculation and Interpretation
• Video Bitrate vs. Resolution For Video Streaming
• Video Engineering for OTT – A 10K Foot View
• What is Essential Video Coding (EVC) MPEG-5 Part-1?
• Closed GOP and Open GOP – Simplified Explanation
• Discrete Cosine Transform in Video Compression – Explain Like I’m 5
• VVC, EVC, LCEVC – MPEG’s New Video Codecs

Types of Video Transcoders

There are predominantly two types of video transcoders – hardware and software-based transcoders. Both versions are very popular and serve specific use cases.

• Hardware Transcoders
  • As the name suggests, the transcoding engine and algorithms are hardware-based (using VLSI or similar tech) and there are dedicated chips or boards for transcoding the incoming video.
  • The advantage of hardware transcoders
    • they are a one-time cost (CAPEX)
    • typically, they provide high throughput since they are hardware-based and optimized
    • typically, they offer lower latency than software-based transcoders
  • The disadvantage is that its quite impossible to further tune or change the algorithms that have been burned into the chip. For example, if the quality of the grass is not great, there’s not much you can do about the algorithms to improve the grass’s quality.

• Software Transcoders
  • These are basically software programs designed for compressing video files. The most popular among these is FFmpeg – a transcoding software that is used in several companies, labs, and hobby projects around the world. If you want to learn more about FFmpeg, please head to our FFmpeg tutorials page.
  • The advantage of a software transcoders are –
    • they can be run on a variety of hardware starting from laptops, mobile phones, browsers, all the way to industrial grade servers in the cloud
    • they are not CAPEX-heavy and can typically be run on an OPEX basis.
    • they can be tuned and fixed easily via software updates (without the hassle of shipping hardware)
  • One of the biggest disadvantages is that they cannot run as fast as hardware transcoders for jobs that require huge through-puts or very low latency. Sometimes, hardware transcoders are best suited for the job.

Ultimately, the choice of going with a hardware or a software transcoder depends on your use-case and requirements – in terms of cost, time to encode, and quality.

If you want to go deeper into the differences between hardware and software transcoders, then be sure to check out this article on hardware and software transcoders.

Challenges & Trade-Offs in Video Transcoding

In the previous sections, we learned about video transcoding, how it works, and the major contributing factors to the process. Now, let’s take a look at some of the trade-off decisions that one has to take while transcoding a video. There are three primary challenges in video transcoding and that is to control the

• cost of transcoding
• time needed to transcode a video
• and quality of the transcoded video output.

We shall see in this section that the three factors are inter-related and controlling them is a trade-off and cost optimization exercise.

Cost

So, the cost of transcoding is not determined by a single factor and has a large number of variables determining the overall transcoding cost. Some of the most common factors are –

• manufacturer/brand
• the codecs available in the transcoder (e.g. AV1 vs. H264) and the algorithms therein
• the input and output resolutions that the transcoder can handle
• through-put that your transcoder can handle

For example, if you have 10 live channels that you want to transcode into 5 resolutions and transmit over the internet via HLS or DASH, then you might need multiple transcoders to handle this job depending on the video codec, bitrates, resolutions, and through-put that you want. The cost will grow as your input and output requirements change and this is something you need to be aware of.

Time

The time required to transcode a video is directly proportional to the video duration, choice of video codec, output bitrate ladder configuration, and any additional graphics overlays or processing that needs to be done on the video.

A complex (but efficient) video codec typically takes more time to process owing to its advanced algorithms, and this slows down the speed of transcoding. For example, AV1 is much slower than H.264/AVC, at very similar input parameters, but you are almost guaranteed a smaller sized video at a quality comparable to the one produced by H.264/AVC.

It’s a trade-off at the end of the day!

Quality

The quality of a transcoder’s output is highly dependent on the choice of

• video codec (H.264/AVC, HEVC, AV1, VP9, etc.)
• the rate control mode (CBR vs. VBR vs. Capped VBR, or FFmpeg‘s CRF — learn about the different rate control modes here.)
• the codec parameters (bitrates, resolutions, profiles (baseline, main), frame-rates, B-frame structures, etc.)
• pre/post-processing (frame rate conversion, up/down sampling filtering, etc.)

There are presets for these parameters in most commercial transcoders, but, if you want to get the best out of your setup, you should invest the time and money to tune these parameters and optimize the encoding to your exact requirements.

Why is Video Transcoding Critical to Video Streaming’s Success?

Each year, companies are faced with the dilemma of delivering great streaming experiences for their users while keeping the cost under controls. Unfortunately, this is easier said than done. A content provider needs to ensure that

• the video is encoded very well
• the video is streamed in a buffer-free fashion
• the video can be streamed on all types of devices
• and more.

One of the main use cases for video transcoding is the creation of the multiple bitrates required for ABR (Adaptive Bitrate) video streaming. If you don’t know what is ABR video streaming, you can learn all about it here.

To successfully deliver video using ABR, it needs to be transcoded into multiple versions, each with their own bitrate and resolution combination. This is a pre-requisite of ABR video streaming and ensuring that each of the versions/renditions looks great visually is the job of a video transcoder.

To summarize, video transcoding ensures that –

Buffer-free Video Streaming

• The viewing experience is kept buffer-free by enabling ABR streaming via multiple version of the same video at different bitrates and resolutions
• ABR allows the video player to switch to the right rendition of the video in-line with the available bandwidth.

Greener Streaming and Lower Storage Needs

• By reducing the size of the files using video compression algorithms, video transcoders enable a greener streaming experience.
• Both the content provider and the user needs to use less battery power to stream the video.
• This reduces cost and power wastage, and reduces the storage requirements.

Enabling Streaming to all Devices

• Different devices can handle different codecs and codec parameters.
• Using proper transcoding, device coverage is achieved through a combination of bitrates, resolutions, video codecs, video codec profiles.

Lower CDN Costs

• Using the right bitrates for the right content type and complexity is very important in order to reduce the transcoded video’s filesize.
• Smaller videos (in terms of file size) ensures a lower CDN cost as your streaming service scales.
• The use of Content Adaptive Encoding, Per-Title Encoding, or FFMpeg’s CRF encoding are useful techniques to reduce your file sizes, and consequently, your CDN costs.

Conclusion

Video transcoding is critical to the success of video streaming and enables content providers to deliver fantastic viewing experiences for their end users. Transcoding is a highly complicated science and is constantly evolving to better serve the needs of the next generation streaming platforms that aim to stream greener, smaller videos to the users while ensuring a great viewing experience.