What’s a Video Codec? Everything you need to know

A video codec is a software or hardware component that is used to encode and decode video data. It is a combination of algorithms aimed at compression video data with a minimal loss in video quality while adhering to certain conditions like bitrate, resolution, and frame-rate constraints. A video codec, thus, is an essential part of any video system, as it determines how the video data is compressed and stored, as well as how it is transmitted and displayed.

In this article, we go deeper into understanding what a video codec is, types of video codecs, and how they are created.

Let’s get started!

What is a Video Codec?

Video codecs use various algorithms to compress video data in order to reduce the amount of data required to represent the video. This allows video data to be transmitted and stored more efficiently, while still maintaining a high level of quality. There are many different types of video codecs, each with its own set of features and capabilities.

Some common video codecs include H.264, HEVC, VP9, LCEVC, and AV1 and you can learn more about these video codecs in this article.

Finally, in order to play back a video, a device or software application must have a compatible video codec installed. If a device or software application does not have the necessary codec, it will be unable to decode and play the video.

How is a Codec Created?

A video codec is typically created by a committee (backed by hundreds of researchers and engineers) and the process is typically like this –

  • the committee tells the world that there is a need for a new codec and specify the requirements (example: 50% better compression than the previous codec, ability to compress 8K video, new color formats, etc.)
  • ask for contributions, proposals, and suggestions and evaluate their feasibility.
  • coordinate all the documentation, testing, experiments that are necessary to decide on the rules and tools for the codec
  • and, finally, publish the codec’s specifications for the world to use.

This is the same process which every video codec out there (H.264/AVC, HEVC, AV1, VP9, etc.) has gone through and there are international bodies that regulate the creation, maintenance, and licensing of a video codec.

Why is a Codec Specification Important?

As we’ve understood, a video codec specifies the constraints and structure of an encoded bitstream so that anyone in the world can write a decoder for it.

A video codec specification doesn’t tell you how to create that bitstream – it only specifies the structure of the bitstream.

Here’s a real-world example of the importance of a video codec and its specification. Let’s say you read online that “Apple has added support for HEVC in their next release”. What this means is that an engineering team at Apple

  • downloaded a copy of the HEVC spec,
  • read it (again and again and again and again – open the HEVC spec and you’ll see why)
  • wrote a program that can decode a bitstream created using the HEVC codec,
  • and are planning to release it in their next software update.
Related:  VVC, EVC, LCEVC - MPEG's New Video Codecs

And why were they able to do this?

‘Cause a bunch of engineers got together to create the HEVC video compression standard and published a document spelling out every step of the process and the bitstream – so that anyone in the world can write a decoder for it.

Do Codecs Specifications Define the Encoders?

No, but, there is a bit of word-jugglery going on here here. Let me explain.

A codec specification tells you what tools are present in the language, what the output of an encoder should look like, and how a decoder will parse it.

But, what this also means is that you can program the encoder anyway which you want to – as long as the bitstream that is produced by the encoder conforms to the codec’s specification. That is the most critical point I am trying to make here.

When you program an encoder or a decoder, how do you know that it works correctly? In other words, where is the gold standard or the reference encoder / decoder for you to compare your code against?

Test Models and Reference Encoders & Decoders

The “need” for a gold standard or a reference is why codec committees and working groups release “test models” and reference encoders and decoders.

These implementations are not optimized for speed or performance. They only contain an implementation of the encoder and decoder as per the specification and you can use them to verify the bitstream that your encoder implementation produced.

In addition, committees often release encoded bitstreams that adhere to the standard. Decoder manufacturers can use these bitstreams to test their software or hardware for codec compliance.


I hope by now you understood what a video codec is, what video coding standards are, and how they pertain to encoder and decoder development. In future articles, let’s understand how the various tools in video codecs work.

krishna rao vijayanagar
Krishna Rao Vijayanagar

I’m Dr. Krishna Rao Vijayanagar, founder of OTTVerse. I have a Ph.D. in Video Compression from the Illinois Institute of Technology, and I have worked on Video Compression (AVC, HEVC, MultiView Plus Depth), ABR streaming, and Video Analytics (QoE, Content & Audience, and Ad) for several years.

I hope to use my experience and love for video streaming to bring you information and insights into the OTT universe.

3 Trackbacks / Pingbacks

  1. EME, CDM, AES, CENC, and Keys - The Essential Building Blocks of DRM - OTTVerse
  2. I, P, and B-frames - Differences and Use Cases Made Easy - OTTVerse
  3. Video Encoding and Transcoding - Simplified Guide - OTTVerse

Leave a Reply

Your email address will not be published.