How did internet streaming and HD channels suddenly become mainstream? The answer is mpeg4 (H.264), which is a technique used to compress video. The successor to mpeg4 is almost here! We are on the verge of yet another breakthrough with HEVC (H.265) that will not only make 4K (and 8K) broadcast and streaming possible, but also take mobile video streaming to the next level. In this article we explain what HEVC is, and why it is the future.
How video compression works
We have all heard about mpeg (mp3, mpeg4 etc.) in some form or another. Mpeg is short for “Moving Picture Experts Group” and is a compression technology. As video files taken directly from a camera or a studio can take up many, many gigabytes - even terabytes (1 terabyte = 1024 gigabytes) – it is necessary to compress video in order to broadcast or stream it to your favorite TV box, Smart TV, web player or smartphone/tablet.
So, compression is needed, but how does it work? Simply put, a compression system analyses a stream of images (that a video consists off) and searches for areas where it can reduce the data. For example, imagine that you have a 2-minute movie scene; just two actors sitting on a bench talking. In this scenario a compression system would look at the background and say; “hey, nothing is updating here so it is not necessary to refresh this part of the picture constantly”. Only the parts of the video that actually changes will be updated.
All video that goes into your TV – TV channels, online streaming, Blu-ray, DVD – has been compressed in some form
That is obviously a simplification, and compression technology is far more sophisticated than that. During high-speed action scenes or sports it takes far more advanced analysis to compress video, and that is why you sometimes see picture artefacts and color issues during fast motion. For example, some of the hardest parts of a video to compress are running water and confetti. Often, variable bitrate is used as a method to dedicate more data to parts of the movie with action or fast motion and less data to less intense scenes of the movie, depending on the medium (DVD, Blu-ray, TV channel etc.).
The history of compression
Compression is not a new concept. In the mpeg age it all started with mpeg1 that only had a brief period of adoption and use in Asia when movies were sold on VCDs (video CDs). The first real breakthrough came with mpeg2 (or H.262), which marked the beginning of the digital video era. Mpeg2 was quickly adopted in a range of applications; TV channels, DVDs and more. It was an important milestone. In many countries mpeg2 is still widely used for standard-resolution TV channels today.
Just like H.264 enabled TV providers to enter the HD & web video era, H.265 will enable them to enter the 4K and mobile era
When HD (high definition) video came along broadcasters and industry members recognized the need for a new and more effective compression standard, as mpeg2 was incapable of dealing with the data required. Mpeg3 was developed as an extension of mpeg2, but was later implemented into mpeg2 as a “profile” that TV broadcasters and other video companies could take advantage of. But the industry quickly realized that an even more effective HD compression standard was needed.
Mpeg4 was developed and is still widely used today. Mpeg4 not only allowed TV channel networks to adopt HD quality, it was also the breakthrough for video streaming over the internet. Mpeg4 allowed studios to compress video to an extent that made video distribution via relatively slow internet connections feasible. During the mpeg4 regime many different “profiles” have been developed, with H.264 being the most famous. Just like mpeg3 was implemented as a high-grade profile in mpeg2 that companies could take advantage of, there are many different profiles in mpeg4 – some more effective than others.
H.264 is the most widely used and is the “official” version - also used on Blu-rays - that comes complete with licensing fees, but free alternatives such as “x264” have also been developed.
We are now entering a new era with even higher 4K and 8K resolution, so what now? What is next? Mpeg4 H.264 can theoretically support 4K, just like mpeg2 could theoretically support HD, but cannot accommodate the relatively steep data requirements to a satisfying degree, and has not been optimized for this. This is where HEVC / H.265 comes into the picture (literally).
HEVC (High Efficiency Video Coding) – or H.265 - is the answer. HEVC is the successor to mpeg4. Just like mpeg4 and H.264 enabled TV providers to enter the HD and web video era, HEVC will enable them to enter the 4K era, and perhaps just as importantly, propel us into the real mobile video era (via 3G and 4G networks).
The HEVC (H.265) standard is more or less finalized, and has been approved by ITU. We will likely see some revisions to it, but it is here and it is ready. So let us take a look at the virtues of this HEVC compression technology.
HEVC is the future of compression
HEVC was approved by ITU (The International Telecommunications Union) in January 2013 and it boasts impressive capabilities. Let us take a look at the details.
HEVC is said to be twice as effective as H.264, which means that you can basically take a 2-hour movie in 1080p resolution and halve the file size with no loss in picture quality. If the movie took up 10 GB when using H.264 it will only take up around 5 GB with H.265 with no visually loss in picture quality – or so it is claimed.
Notice how H.265 uses only half the bitrate (610 kbps) compared to H.264 (1183 kbps)
That is interesting for a number of reasons. Many TV providers only have a limited data capacity in their TV pipes, so HEVC will enable them to either carry more TV channels or to improve bitrate (picture quality) on the existing TV channels. You can guess what most of them will do. For internet TV providers such as Netflix or YouTube prospects are even greater. They do not have the same limitations as the internet pipes are huge (and expanding constantly), so they can grow they content catalogue the way they like. HEVC will allow them to improve picture quality in the high end, and allow them stream content to you in better quality even through a slow internet connection.
HEVC is said to be twice as effective as H.264, halving file size with no loss of quality
But that is not all. As mentioned in the first section, mpeg4 made HD distribution feasible, and HEVC will make 4K and maybe even 8K feasible. This is not just a pipe dream, this is for real and it will happen relatively soon. HEVC will enable players such as Netflix, Google, Amazon, and Apple to stream 4K content over the internet. It will also enable you to stream Full HD content to your tablet or smartphone. Granted, Full HD streaming to mobile devices is also possible with H.264, but H.265 will make it feasible. HEVC can theoretically help take the TV industry even further with support for 4K resolution at up to 300 frames per second or 8K resolution at up to 120 frames per second.
H.265 is far more sophisticated in its compression than H.264 – but it also requires more CPU power to decode
Right now there are three profiles in the HEVC standard, but more will be added in the future. The ‘Main’ profile supports 8-bit colors, the ‘Main 10’ profile supports 10-bit colors, and the ‘Main Still Picture’ profile is aimed at still photography compression. Future profiles will include 12-bit color support in 4:2:2 or 4:4:4 chroma as well as 3D.
By know you are probably thinking; that sound great and everything, but can my existing devices handle HEVC? No, unfortunately not. HEVC will require new devices, mostly. Your TV set top box does not support HEVC or 4K output, so you will need a new one eventually. Streaming media boxes ditto. Even if for example Netflix decides to go ahead with its 4K streaming plans you will still need a new Apple TV, Roku or WD TV box. Some Smart TVs claim to support HEVC (H.265) and a few TV boxes, too, but besides that only few devices offer support for the new standard.
So, adoption will take some time. Mpeg4 was approved way back in 1998, but mpeg2 is still widely used today. HEVC will see faster adoption in today’s connected world, but it will still take years to reach the mass market. Electronics manufacturers need to add support via a hardware chip or the GPU, but only relatively powerful devices will be able to decode HEVC. The 4K trend will push HEVC forward and vice versa, but the acceleration of mobile technology will probably be the most significant driver. Just read this:
- It could be 12 to 18 months, maybe longer, before the first devices with H.265 hardware acceleration make it to market. Once those initial devices do make it to market, however, we can probably expect a quick ramp up in the amount of content that begins to take advantage of H.265. Since the launch of the iPad, the percentage of video published in H.264 has climbed from less than 10 percent to more than 84 percent in less than three years, according to MeFeedia.- Source: TechCrunch
This was written in January 2013, but the first devices with HEVC have actually already been released. Similarly, new iPads as well as other smartphones and tablets with HEVC will be significant factors. As for a specific time frame, some people have suggested that it will take 5 years. We suspect that it will happen faster than that, but it obviously depends on whether we are talking about 10, 20 or 50% market penetration.
The competition - Google VP8 & VP9
Over the last few years, Google has pushed its own VP video compression technology (that it acquired in 2010). VP8 is the direct alternative to H.264, but it failed to see widespread adoption due to a number of reasons. For example, Apple chose to adopt H.264 and therefore refused to implement support for VP8 in its iOS devices and Safari on Mac. Similarly, Microsoft refused to implement support for VP8 in Internet Explorer.
The pattern seems to repeat itself. Do not expect Google’s VP9 to take off. We would be surprised if it did
Another reason was that VP8 was late to the party and companies had already started adopting H.264. At one point Google announced that they would drop support for H.264 in the Chrome browser, just like the others had done to VP8, but Google never took the step.
Now comes HEVC, and Google has a competitor called VP9. This time at a much earlier point in the adoption phase, so Google hopes that VP9 will be widely adopted as it compares fairly equally to HEVC. VP9 is already built into the Chrome browser, but the same power struggle will play out – only this time Apple has even more iOS devices out in consumers’ hands.
Apple has already indicated that they plan to support HEVC (and thus not VP9). Microsoft has not taken a stance on the matter yet, but already the pattern seems to repeat itself. Most TV makers and TV service providers also plan to support HEVC. Do not expect VP9 to take off. We would be surprised if it did. So, when you are shopping for new devices in the next few years HEVC is probably what you want to look for instead of VP9 as the must-have feature.