Thursday, December 11, 2014

Video compression - early history

Present day video codecs such as HEVC are based on fundamental concepts that go back a surprisingly long way. Key components such as motion compensated prediction, transform coding and entropy coding were developed in the decades before 1990.

I've written a short overview of the early days of video coding, when the building blocks were developed that would enable the emergence of mainstream digital video products in the 1990s. Click here to download the article, or download it from the Vcodex resources page here.


"En L'An 2000", early videophone concept, 1910

Friday, November 21, 2014

A nice example of 4K video encoded using VP9

Do you want to see 4K video in action? Phil Holland has put together a rather good demo clip, captured using a 6K RED camera and encoded using VP9 at 4K resolution (3840 x 2160 pixels).

The clip is 1 minute 55 seconds, 200MBytes in size, which works out at around 13Mbits per second. To view it for yourself, download the Zip file here, extract the .webm file from the Zip file, and drag it into a Chrome browser window. Of course, you'll need a 4K resolution monitor to get the full benefit and you may need a fast processor for smooth playback.

http://www.reduser.net/forum/showthread.php?111230-Google-VP9-4K-HD-Sample



Monday, October 6, 2014

MPEG-LA releases its HEVC patent pool license

On 29th September 2014, licensing company MPEG LA LLC released details of a patent portfolio license covering technologies related to the new High Efficiency Video Coding (HEVC) standard.

Video coding or video compression is an essential technology for video broadcast, streaming and playback. Released in 2013, HEVC is the latest in a series of industry standards for video compression that offers higher performance than its predecessors.

The license grants certain rights to patents held by 23 organisations, which MPEG LA believes to be essential to the practice of the new standard. Under the terms of the license, licensees pay a royalty to MPEG LA for end user products over a certain threshold (currently 100,000 units per year).

MPEG LA lists a total of 55 patents in the HEVC pool (see Figure), including 22 Korean patents and 12 US patents. We can expect to see the number of patents rise over the coming months, as patent applications filed during the development of HEVC are granted.










Friday, September 5, 2014

How to stream better quality video: Part 3 - Ultra High Definition, 4K and next generation video codecs


4K or Ultra High Definition (UHD) video has four times the detail of 1080p full HD. How do you deliver such high resolution video to a TV or mobile device over a limited bandwidth connection? Next-generation video codecs may provide the answer. This article, part 3 of “How to stream better quality video”, gives an overview of two new video codecs: High Efficiency Video Coding (HEVC or H.265) and VP9.

The High Efficiency Video Coding standard

H.264 is widely used amongst broadcasters and in the online video streaming industry, but this may change with the introduction of HEVC. If you want to get up to speed on the technical side of HEVC, the Vcodex website has material to get you started.

Key features of HEVC:

  • Up to twice as efficient as H.264 - which means that you can send the same picture quality using much smaller files.
  • Support for resolution up to 8K video (7680 × 4320) and frame rates up to 120 frames per second.
  • Improved picture quality in terms of noise levels, color spaces and dynamic range.

This means you can shoot and produce video at the higher resolutions of UHD/4K without dramatically increasing the bandwidth required to broadcast or stream your video.

VP9

VP9 is Google’s answer to the problem of streaming 4K video. Offered as royalty free by Google, VP9 also claims to be significantly more efficient than H.264. Like HEVC, VP9 supports compression of videos at resolutions above HD as part of the WebM media format.


When can I start using 4K / Ultra High Definition?

You can try 4K on the web right now. For example, this Youtube playlist features 4K/UHD videos, which your display may or may not be able to handle. DivX's HEVC showcase includes examples of 4K videos. You'll need to download the DivX HEVC plugin to play back these clips.

Hardware support for HEVC is increasing, with a number of demonstrations at this year's IBC show. A reference hardware implementation of VP9 is available for chip developers.


New codecs - a performance comparison

We ran a comparison test to find out how HEVC and VP9 codecs performed compared with H.264 using a 720p video clip downloaded from the Xiph website. Following standard test guidelines for video quality assessment, we carried out subjective video quality tests with 10 non-expert users.
Figure 1: File size versus perceptual quality for H.264, VP9 and HEVC codecs

Figure 1 shows the variation of perceptual video quality measured as mean opinion score (MOS), with file size for different codecs. At the higher end of the quality scale (i.e. lower compression), both HEVC and VP9 achieve a similar viewing quality to H.264 with a 40-45% reduction in file size. At lower quality ratings, the gains of HEVC and VP9 are smaller.

This implies that you should see the most benefit from HEVC and VP9 codecs at high quality / low compression settings.


We hope the three-part series of the blog "How to stream better quality video" has been useful. To find out more about video compression and streaming, please visit the Vcodex website here.


About the authors:

Iain Richardson is an internationally known expert on video compression and the author of several books, including The H.264 Advanced Video Compression Standard: http://www.vcodex.com/books.html

Abharana Bhat specialises in video compression, media streaming and video quality assessment.


Copyright (c) Vcodex Limited 2014-2015

Thursday, September 4, 2014

How to stream better quality video: Part 2 - Adaptive streaming

by Iain Richardson and Abharana Bhat
You want to stream a video clip to several customers. One customer has a high bandwidth leased line connection, while another is trying to view your clip on a mobile device using a poor-quality 3G signal. How do you make sure they each get the best possible video quality? Adaptive streaming may be the answer.

Video streaming sends your video (source) to play back on a computer or mobile device (client), using an internet connection to transport the video. Streaming video is a complex process involving many different components. In this series of articles, we help you understand the basics of streaming video and the various factors that can make a big difference to the quality of your video clip.

In our first article, we looked at the basics of streaming video and discussed good and bad practice. In this article, we look at how to stream video effectively over networks with varying connection speeds.


1. Adaptive streaming: the basics


The general concept of adaptive streaming is shown in Figure 1.

a. Multiple versions:

Starting with a high quality video source such as a High Definition video clip, a video encoder produces versions at multiple qualities and bitrates, for example:

  1. An HD version at 2Mbps 
  2. A Standard Definition (SD) version at 1Mbps 
  3. A very low resolution version at 50kbps. 

Each of these versions is split into small chunks, typically each a few seconds long, and stored on a web server ready for streaming.

b. Index file:

The server maintains an index or playlist file which lists all the media chunks, including the bitrate of each stream and the starting position of each chunk.

c. Streaming to the client:

When the client (the user's software application) requests playback of the video stream, the server returns the index file to the client.

The client requests chunks at the most suitable bitrate, based on considerations such as the available connection bandwidth, playback delay and device capabilities. The server sends each requested chunk in order.

The result: video and audio playback that adapts to suit the network conditions.


2. Examples
Example 1:

The client is initially connected to the internet via a good quality WiFi connection. Streaming starts at 2Mbps and the user sees good quality, HD video with low delay.

The WiFi signal is lost and the client switches to a 3G mobile connection. Streaming switches to 1Mbps, then to 50kbps. The user sees continuous video playback. The video quality steps down to SD then to lower resolution because of the drop in network bandwidth.

Example 2:

The client is configured to start playback as quickly as possible. It automatically requests the lowest quality stream and steps up if the bandwidth is good enough. The user sees basic quality playback very soon after pressing "play". After a few seconds, the video quality improves progressively without any delays or breaks in playback.

Example 3:

Click the link below and play each video to see adaptive streaming in action. Click on the "HD" button on the player to select one of 5 different bitrates, or select "Auto" to allow the stream to adapt according to your network connection.





3. Practical considerations
Adaptive streaming format. There are a number of choices of adaptive streaming methods, including:

Each of these methods has pros and cons. When selecting an adaptive streaming technology, it is important to consider:
  • Availability and cost of encoding software tools 
  • Compatibility with your target clients. 
Stream count. How many streams should be created for each video file? More streams will give the client more flexibility to choose the best stream based on the available bitrate and display capabilities. However, each stream needs to be converted (transcoded) from the original and stored on a server. This may increase computing and storage costs.

Chunks and key frames. How many seconds per chunk? Each chunk starts with a key frame, i.e. a video frame encoded without any prediction from other frames. Key frames require significantly more bit rate to transmit than predicted ("inter") frames. Smaller chunks make it possible for the client to switch streams more frequently, leading to faster startup and smoother adaptation. However, smaller chunks require more frequent key frames, leading to poorer transmission efficiency. A good compromise may be to use smaller chunks early in the video stream, increasing the size later on.


4. Going further Part 3 in this series will compare the latest video codecs in depth.

Copyright (c) Vcodex Limited / Onecodec Limited, 2014

How to stream better quality video: Part 1 - Basics and good practices

by Iain Richardson and Abharana Bhat

You're trying to stream a video to a work colleague who is complaining that it's very, very slow to load. Or perhaps you are trying to embed a video clip in your web page and are not satisfied with the quality of the result. Is there anything that can be done to improve the situation?

Video streaming sends your video (source) to play back on a computer or mobile device (client), using an internet connection to transport the video. Streaming video is a complex process involving many different components. In this series if articles, we help you understand the basics of streaming videoand the various factors that make a big difference to the quality of your video clip.

In this first article, we look at the basics of streaming video and discuss good and bad practice.


1. Streaming video: the basics
These are the basic components involved in streaming video from source to destination.

Source - A video clip you've recorded or created.

Editing and compression - Once you have edited your media file, it is converted into a form suitable for sending over a network. This involves compressing or encoding the video and audio information.

Server - Stores the media file and sends it to the client on demand.

Network - Transports (streams) the video from the server to the client.

Client - Receives the media file, extracts (decodes) video and audio information and plays back the video clip.  

"Streaming" means that the client can start playing the video clip before it's fully downloaded, as soon as there is enough data to start decoding and playback. In this screenshot, the client has received and stored enough data to decode and play the clip, even though the whole clip hasn't been downloaded yet. The video clip plays smoothly, while the client continues to download and store ("buffer") the remaining data.

Figure 2: Streaming playback


2. Compression and codecs

2.1 The basics of compression

A codec is an encoder and a decoder. An encoder compresses audio or video so it takes up less disk space. A decoder extracts audio or video information from the compressed file. Video and audio compression is a complex technical process, but the basic aim of a codec is quite straightforward:
(a) Reduce the size of the compressed media file as much as possible, but...(b) Keep the quality of the decoded audio and video as good as possible.
Most codecs work by removing information that is not noticeable to the viewer / listener and by exploiting similarities in the audio / video data.
Example 1: Small, subtle variations in texture will tend to be "masked" by strong shadows and edges. A video codec can exploit this by keeping stronger image features and throwing away some of the fine detail.
Example 2: A video clip contains 25 or more frames per second. Instead of sending the entire frame every time, a video codec can save a lot of data by only sending the differences between frames. 
Any device that records or plays back video contains a codec, either as part of a chip (hardware) or in software. When you record video on your mobile handset, watch anything on TV, or play video on the web, you're using a codec.

2.2 What's in a compressed media file?

Video or audio files on your computer or mobile device are usually stored in a compressed form. Common media file formats have extensions such as .mp4, .mov, .avi, etc.

There are three main components of a compressed media file: compressed video, compressed audio and the structure of the file itself.

The compressed video will typically use one of a number of standard video codec formats, such as MPEG-2, MPEG-4 or H.264, VP8, etc. The compressed audio will use one of a number of audio codec formats, such as MP3 or AAC. The compressed video and audio are typically packaged into a container format, i.e. a special type of file that is designed to carry video and audio along with other information known as metadata. Examples of container formats include MP4, MOV, MKV and AVI.

Many combinations of video codec, audio codec and container format are possible. One challenge you may face is that not all combinations are supported by every computer or mobile client.

3. Important concepts

Bitrate and bandwidth - The information-carrying capacity of a network connection is commonly known as bitrate or bandwidth and is measured in giga/mega/kilobits per second. A 3G mobile connection might be capable of carrying a few tens of kilobits per second; a home broadband connection may support hundreds of kilobits up to a few megabits per second; a fibre or leased line may be capable of carrying tens of megabits or more per second.

To stream video smoothly, it is important that the bitrate or bandwidth of the network connection is greater than the bitrate of the streaming media file. Even after compression, video and audio can require a significant bitrate capacity.

Video resolution - Most cameras, even on mobile devices, are capable of recording video in High Definition (HD) resolution. However, significantly more bandwidth or bitrate is required to handle HD video compared with lower resolution video.

A single frame of "full HD" or "1080p" video (1920x1080 pixels) has more than twice the pixels of a single frame of "720p" video (1280x720 pixels), which in turn is more than twice the size of a single frame of "Standard Definition" or "SD" video (720x576 pixels or smaller).

Transcoding - If your video/audio material is not suitable for streaming, you may need to transcode it. This may involve:
Changing resolution: For example, resample from 1080p HD down to 720p or SD.
Changing codec or container: For example, change the video codec from MPEG-2 to H.264, or change the file format (container) from AVI to MP4.
Improving streaming performance: For example, reorganising the audio and video samples in the container file, or converting to an adaptive streaming format.

4. Tips for better streaming performance

  • Keep the camera as steady as possible. More movement means more data after compression, which means that you will need more network bandwidth to achieve good quality video playback.

  • Use good lighting where possible. Poor lighting can produce increased camera “noise” which leads to more data after compression. Here's an example: the first scene is well lit, the second is dark and "grainy" in appearance. The poorly-lit scene will require more network bandwidth to transmit.
  • Choose your video codec carefully. Older codec formats such as MPEG-2 are less efficient, i.e. the coded video file will be considerably larger than a newer codec such as H.264. The latest codecs, such as HEVC and VP9, may give the best compression but there is limited software and playback support for these codecs at present (see Part 3).
  • Use a lower resolution (e.g. SD rather than HD) or consider adaptive streaming (see Part 2).
  • Be aware that video content has a significant effect on the bandwidth required for streaming and/or on the video playback quality. For example, if two SD video clips are both compressed at the same bitrate (400 kilobits per second). The clip with more movement will result in the video quality looking worst even though the video resolution and the streaming bitrate are the same.

  • Does your editing software have options such as “slow” or "multi-pass" encoding? If so, consider selecting these options. Processing the file will take longer because the software will spend more time to choose the best compression options for each video frame. However, the end result may be better video quality and/or a smaller compressed file.

5. Going further

Part 2 of this series of articles will cover more advanced topics such as Adaptive Streaming. Part 3 will compare the latest video codecs in depth.

To find out more about video compression and streaming, visit Iain Richardson's website: vcodex.com


Copyright (c) Vcodex Limited / Onecodec Limited, 2014

Monday, August 25, 2014

HEVC Analysers


Back to the main Vcodex site: video compression and streaming resources. Register with Vcodex for free to read the full version of this article.

Iain Richardson and Abharana Bhat, August 2014


High Efficiency Video Coding (HEVC) [1,2] is a next generation video coding standard which has the potential to improve delivery of High Definition and Ultra High Definition video.  A HEVC analyser is software with a user-friendly interface for visualising and testing of HEVC bitstreams. It can be a useful tool for broadcasters and content delivery professionals, quality assurance teams, educators, codec architects and for anyone new to the standard. This white paper gives an overview of some popular HEVC analysers as well as their installation guides, usage and available features. HEVC bitstream analyzers featured include Elecard HEVC analyser, Parabola Explorer and Zond265.

The full whitepaper is available here. If you would like us to feature your analyzer product, please get in touch.

Elecard HEVC analyser

Developed by Elecard, this bitstream analyzer comes in trial and full versions. The analyzer currently runs on Windows machines with Microsoft .NET framework installed. To install, download the installer from [3] and follow the on-screen instructions. The User manual [4] explains the system requirements. Once installed, start by opening a HEVC file and explore the various features available by using the menu bar in the main window. Figure 1 shows the main window, which includes the frame navigation view, picture view, metadata information, bitstream view and coding unit information view.

Figure 1: Elecard HEVC analyzer - main window


Parabola Explorer

Developed by Parabola Research, the analyser is available in trial and full versions from their website [5]. The software currently runs on Windows machines only. To install, register with Parabola Research [6] and a download link to the software installer is sent via email. Run the installer and follow the on-screen instructions to install the software. The user manual is available at [7]. Figure 2 shows the main window which includes the frame navigation view, picture view, bitstream pane and coding statistics pane.


Figure 2: Parabola Explorer HEVC bitstream analyzer - main window


Zond265

Developed by Solveig Multimedia, the software is available from [8] and comes in demo and full versions. The demo version allows analysis of the first 10 frames per sequence with no time limit. Registration is required to get the download link for the demo version. Documentation and FAQs are available at [9]. Figure 3 shows the main window, which includes picture view, toolbar options, bitstream pane and the timeline window.


Figure 3: Zond265 bitstream analyzer - main window

Find out more about these analysers here.


References

[1] HEVC video coding standard: http://www.itu.int/rec/T-REC-H.265-201304-I
[2] HEVC material on the Vcodex website: www.vcodex.com/h265.html
[6] Parabola Explorer  download instructions: http://www.parabolaresearch.com/download.html