What is Ultra HDR? Everything you need to know about the new imaging format

Google’s new Pixel 8 and Pixel 8 Pro are undoubtedly great phones with much to offer. However, with their release, as much talk has been focused on the phones themselves as on their new and improved software tricks.

There are, of course, various new photography capabilities that new smartphones have, most of them powered by AI. There is also Android 14, which brings a few cosmetic improvements to the OS. However, there are also changes under the hood that might have a much more significant impact than it might seem.

One of the most significant changes that the Pixel 8 series brought to the table is undoubtedly the introduction of a new image format. Ultra HDR photos promise to get your photography into a new HDR age with little to no pain from changing the image format.

What is an image format?

Before moving on to the Ultra HDR, let’s talk about image formats in general. If you want to see a picture on your computer screen, it must be compressed. Of course, by using specific applications, such as Photoshop or Lightroom, you can view RAW, uncompressed files. Still, for the sake of viewing images, it’s a big loss of computational power, space, and bandwidth. Many compressed image files can be around 8-10 times smaller than uncompressed files while maintaining a similar level of detail.

Compressed image files come in all shapes and sizes, and each format has strengths and use cases. Probably the two most well-known ones today are JPEG and PNG. The first one is synonymous with a digital image and offers a lossy compression. At the same time, PNGs are a lossless compression format, which maintains transparency and other vital metadata while still significantly reducing file size.

When browsing the internet or viewing pictures on your PC, most will be stored as one of these two formats, but many others serve a bit more advanced functions. The easiest way to recognize what your image format is is to look at the extension of the file. There, you should see the name of your file and, after that, a dot with the extension. Most of these formats are best known by these extensions, often acronyms of their full names. So, JPEG files will end with .jpeg/.jpg, PNGs will end with .png, and so on.

What is Ultra HDR?

As we’ve mentioned, Ultra HDR is a new image format developed and introduced by Google with the release of the Google Pixel 8 and Google Pixel 8 Pro. As the name implies, the biggest change that this file type brings to the table is a superb handling of HDR content, making it look even better on devices that support it (and the new file format). So, you need not only a device that saves the images it takes in the Ultra HDR format but also a device with a screen capable of displaying HDR content.

Interestingly, Google developed Ultra HDR as an extension of the JPEG format, so Ultra HDR photos and pictures are saved with a .jpeg extension. This also ensures backward compatibility – you can view Ultra HDR photos on every device that supports standard JPEGs, only without all the HDR goodness.

For now, you can only take Ultra HDR photos on a Pixel 8 and Pixel 8 Pro with the newest version of the Camera app installed. If you are a bit adventurous, you can also root your Pixel 6 and Pixel 7, sideload the app, and give the new format a go – it works just fine once you get through the setup process.

What are the benefits of Ultra HDR?

Google claims that Ultra HDR is the next step in HDR content. There are a few benefits, and the biggest one is compatibility since it uses the same extension as JPEG files. Images shot in Ultra HDR can be easily viewed on every device that supports the .jpeg extension but only in their standard dynamic range version.

However, Ultra HDR also brings benefits and new possibilities to HDR users. On supported devices, Google promises Ultra HDR gives a more lifelike look than standard HDR thanks to a different way of handling contrast. While some users see standard HDR content as too contrasty and artificial, Google’s implementation is supposedly much more true-to-life, using gain maps instead of tone mapping. It’s vital for situations such as a strong light that peeks from behind a subject. Ultra HDR will be able to show that more organically.

How does Ultra HDR work?

Ultra HDR is an exciting format, as it essentially uses standard JPEG images with a layer of gain maps embedded into the picture’s metadata. This ensures that Ultra HDR not only produces much more natural HDR but is also not much bigger than SDR JPEG pictures.

When shooting in SDR, your photos have essentially only one brightness slider, which makes the whole picture brighter or dimmer. That’s why SDR photos are bad when shooting pictures with bright and dark spots. They often crush them, resulting in blobs of white and black with no discernable detail.

Standard HDR photos handle brightness differently. These photos use tone mapping, which is, to put it simply, compositing your images from shots taken at different brightness levels. This way, you can have a detailed view of the bright sky, as well as the shadow under a tree, all in the same shot. However, the downsides to this approach are some echoing artifacts and a bit of an artificial, oversharpened look that many people complain about when talking about modern (especially smartphone) photography.

Google Ultra HDR handles HDR content a bit differently. Instead of tone mapping, it utilizes gain maps that are embedded into the metadata of your JPEG photos. The gain maps are essentially info about the brightness level of every picture.

This approach allows you to show your photos on both HDR and SDR screens. With the SDR screen, you’ll see a standard JPEG version of the pictures. However, when displayed on a supported screen with HDR capabilities, these gain maps allow each pixel to be lit precisely as when the photo was taken, as this information comes from the RAW files before compression. This ensures that the photos are compatible with SDR devices and look great and natural when viewed on HDR screens.

What challenges lie ahead of Ultra HDR?

Of course, there are still some hurdles to get through with this technology. The biggest one, as is usually the case with newer image formats, is compatibility. Even though Ultra HDR is backward compatible, for it to be useful and famous, its implementation of HDR has to be usable in various applications. This file format is extremely young, but Google already made sure that it is readable by Android 14 devices, as well as in Google Chrome.

The other problem that the image format is facing is compression issues. It appears that when undergoing compression (for example, when uploading the photo to social media or even to a webpage through its CMS), Ultra HDR photos get compressed to regular JPEGs and lose the metadata responsible for their HDR-ness. Unfortunately, such a compression makes these details unable to showcase properly, creating quite a difficulty in showing the differences between SDR and Ultra HDr in articles such as this one.

The last thing that Google has to worry about is the spread of Ultra HDR. So far, there are only two pieces of tech that can take Ultra HDR photos – Google Pixel 8 and Pixel 8 Pro. You can sideload the camera app to Google Pixel 6 and 7 series, but even with these unofficial solutions, the hardware capable of producing Ultra HDR pics is extremely narrow for now. If Google wants this image format to become the new standard, it still has a long way to go.