Thursday, June 18, 2015

USPTO invited talk, "structures in video coding"

This week I had the honour of presenting an invited talk to examiners at the US Patent and Trademark Office in Washington DC:

The topic of my talk was "structures in video coding". I explained how video codec structures have evolved from simple, repetitive 16x16 macroblocks in early standards such as MPEG-1....

..... to complex hierarchies of blocks in recent standards such as HEVC:

I examined the effects of increasingly complex block structures. These changes have led to dramatic improvements in compression performance but also increasing computational demands. I left the audience with a question: after 25+ years of intensive research and development, why do mainstream video codecs still rely on rectangular block structures?

- Iain Richardson

Tuesday, June 2, 2015

7 important video compression concepts that are more than 20 years old

The latest MPEG / ITU video compression standard, H.265 or HEVC, was published in 2013. HEVC is a significant technical achievement, but it's partly based on fundamental work carried out many decades ago.

An HEVC video codec includes the basic building blocks of:
  • Prediction : create an estimate or prediction of a current block of video data
  • Transform : convert a block of samples into a spatial frequency representation
  • Entropy coding : encode video information into a compressed bitstream

Here are seven important research papers and patents dating back to the 1950s that helped to shape present day video coding technology.


1. “A Method for the Construction of Minimum Redundancy Codes”, D A Huffman, Proceedings of the I.R.E., September 1952
- Variable length binary codes for data compression.

2. “Transform coding of image difference signals”, M R Schroeder, US Patent 3679821, 1972
- Coding moving images using frame differencing, i.e. simple inter-frame prediction.

3.  “Discrete Cosine Transform”, Ahmed, Natarajan and Rao, IEEE Transactions on Computers, Jan 1974
- The classic paper on the DCT, widely used in image and video compression.

4. “Generalized Kraft inequality and arithmetic coding”, J J Rissanen, IBM J. Res. Dev. 20, May 1976
- Arithmetic coding, a forerunner of H.264 and HEVC’s CABAC.

5. “Displacement measurement and its application in interframe image coding”, J R Jain and A K Jain, IEEE Trans. Communications, December 1981
- An early description of motion compensated prediction for video coding.

6. “Variable size block matching motion compensation with applications to video coding”, M H Chan, Y B Yu and A G Constantinides, IEE Proceedings Vol 137, August 1990
- Motion compensated prediction with variable size blocks.

7. “MPEG: A video compression standard for multimedia applications”, D Le Gall, Communications of the ACM, Vol 34 No 4, April 1991
- Bidirectional prediction as used in the MPEG-1 video compression standard.

Thursday, May 21, 2015

Talk: The Ultra HD Codecs - HEVC and VP9

I'm looking forward to giving this talk to broadcast professionals in Stornoway, Isle of Lewis, one of Scotland's beautiful Western Isles

The Ultra HD Codecs: HEVC and VP9
When: Friday 22nd May 2015, 11am
Where: MGAlba Studios, Seaforth Road, Stornoway

Like it or hate it, Ultra HD or 4K is making a big impact on the broadcast industry. 4K content has four times the number of pixels of full HD, making storing, transferring and streaming very demanding. This has significant implications for workflows.

The new HEVC/H.265 and VP9 video compression codecs are designed to help handle the challenges of UHD / 4K video. This talk will introduce you to these new codecs. You will learn:
- how the codecs compress and deliver 4K video
- what’s changed from older codecs such as H.264
- how the new codecs perform
- what software and hardware support is available.


If you are interested in arranging a specialist lecture on video coding or streaming technology, please get in touch.

Monday, April 27, 2015

The challenges facing a new codec

I was asked recently to comment on how easy or difficult it might be to introduce a new video compression codec. Here's a summary of my opinion - you can read the full article by John Moulding here.

At present, the market is dominated by standards-based and open source solutions, including H.264 / AVC, VP8, HEVC and VP9. What challenges might be faced by a new entrant to the market?

1. Interoperability: One of the key motivations for the development of standards has been interoperability. Streaming provider X needs to be compatible with playback client Y. Building a critical mass of support for a new codec requires widespread adoption of encoders and decoders that interoperate with each other.

2. Performance: The development of standards such as H.264 or HEVC involved rigorous and thorough testing using agreed protocols for measurement of quality, bitrate, computational requirements, etc. Performance testing needs to be repeatable, such that multiple organisations can check performance and reach the same conclusions.

3. Intellectual property: An open source or standardised solution provides a degree of transparency about who might own intellectual property that is used in a compression solution. For example, implementors of the H.264/AVC standard can take a license to several hundred patents that may be essential to the standard, via the MPEG-LA patent pool.

4. Hardware support: When you play back or stream a video on a consumer device such as a cellphone or tablet, the computationally intensive process of decoding video is assisted by dedicated hardware, enabling smoother playback and better battery life. Support for existing formats such as H.264 and VP8 is built in to chipsets, operating systems and protocols.

A new codec technology has to overcome many hurdles if it is to be widely adopted. However, I am always interested in genuinely new and disruptive approaches to video compression. Is there a challenger out there to the 25-year-old codec model that has been the basis of video compression standards from MPEG-1 to HEVC?

Thursday, December 11, 2014

Video compression - early history

Present day video codecs such as HEVC are based on fundamental concepts that go back a surprisingly long way. Key components such as motion compensated prediction, transform coding and entropy coding were developed in the decades before 1990.

I've written a short overview of the early days of video coding, when the building blocks were developed that would enable the emergence of mainstream digital video products in the 1990s. Click here to download the article, or download it from the Vcodex resources page here.

"En L'An 2000", early videophone concept, 1910

Friday, November 21, 2014

A nice example of 4K video encoded using VP9

Do you want to see 4K video in action? Phil Holland has put together a rather good demo clip, captured using a 6K RED camera and encoded using VP9 at 4K resolution (3840 x 2160 pixels).

The clip is 1 minute 55 seconds, 200MBytes in size, which works out at around 13Mbits per second. To view it for yourself, download the Zip file here, extract the .webm file from the Zip file, and drag it into a Chrome browser window. Of course, you'll need a 4K resolution monitor to get the full benefit and you may need a fast processor for smooth playback.

Monday, October 6, 2014

MPEG-LA releases its HEVC patent pool license

On 29th September 2014, licensing company MPEG LA LLC released details of a patent portfolio license covering technologies related to the new High Efficiency Video Coding (HEVC) standard.

Video coding or video compression is an essential technology for video broadcast, streaming and playback. Released in 2013, HEVC is the latest in a series of industry standards for video compression that offers higher performance than its predecessors.

The license grants certain rights to patents held by 23 organisations, which MPEG LA believes to be essential to the practice of the new standard. Under the terms of the license, licensees pay a royalty to MPEG LA for end user products over a certain threshold (currently 100,000 units per year).

MPEG LA lists a total of 55 patents in the HEVC pool (see Figure), including 22 Korean patents and 12 US patents. We can expect to see the number of patents rise over the coming months, as patent applications filed during the development of HEVC are granted.