JPEG is really old. Will any of these alternatives ever catch on?

JPEG is 26 years old, and yet no picture format has come close to replacing it in terms of widespread use. Are we doomed to an 8-bit format for all eternity? Let's look at the alternatives, and hope one of them catches on!

By the time it’s become clear which technical standards are likely to succeed, they’re usually pretty well known. For anyone looking for a heads-up on what’s coming, though, we’ve tried to keep abreast of new developments, perhaps most recently Apple’s favourite HEIC. It’s one of several formats which have sought to replace JPEG, which is almost certainly one of the most  widely-used means of moving pictures around ever invented. It’s a matter of opinion, but the world seems to agree that it can reduce the data required to transmit an image by almost ten times without unduly affecting the image quality. 

But with more modern techniques, it’s possible to do a lot better than that. Doing better is subject to two things. The first is that better compression invariably means more computer horsepower. That’s fine, as things have come on a bit since 1993, when the original JPEG was formally standardised. The second is that we have to get everyone to agree on a replacement. That’s fine too, because the process of developing and deploying standards is always quick, easy, and never leads to patent trolling, format wars or decades of intercine squabbling among commercial interests.

Possibly because of those problems, recent attempts to replace JPEG have been – well - perplexing. Several options come from the JPEG group itself and seem to compete with one another. Apple is involved in the Alliance for Open Media, the organisation behind the AV1 codec that’s intended to ease patent concerns over HEVC, while simultaneously choosing to use HEVC in its HEIC image files. JPEG-2000, an option that’s far better than JPEG, has been largely ignored since its launch almost twenty years ago.

So, things are already shaping up to make very little sense. Let’s dive in!

JPEG

The idea of JPEG dates back to the mid-80s, though updates came as recently as 2014. The Joint Photographic Experts Group, currently resident at the Leipzig Institute for Applied Informatics, added scaling, 12-bit and lossless encoding, and discussed RGB modes, in update 9.1. However, as we’ll see again and again, there can be some distance between what’s in the standard and what’s actually implemented in the software designed to create and display files. In theory, a JPEG file can include a wide variety of things, even possibly an alpha channel in its auxiliary data areas, but practically nothing would support the result.

In practice, then, JPEG is invariably an 8-bit, standard dynamic range, YCrCb 4:2:0 format with no transparency and sRGB/Rec. 709 or Adobe RGB colour. The progressive decoding mode, which provides a low-resolution image then sharpens it up as more data arrives, is commonly avoided simply because it requires something like three times the processing power to decode on someone’s cellphone, as opposed to a single pass to arrive at the same result. Finally, ICC profiles can be embedded which imply a gamma curve, and thus most of the JPEGs in the world are capable of encoding about eleven stops of dynamic range. There is perhaps some argument that a really compliant decoder could read an appropriate ICC profile and somehow achieve some sort of HDR, but it would be a severely off-piste excursion and liable to encounter unanticipated problems.

JPEG 2000

The most obvious pretender to JPEG’s throne is JPEG 2000, released in the eponymous year with the idea that a discrete wavelet transform (DWT,) as opposed to a discrete cosine transform (DCT,) would lead to better performance. And it did: at high bitrates, JPEG-2000 is only slightly better than the original, but it really shines at low bitrates because the artefacts of DWT are generally less objectionable than the artefacts of DCT. The mathematical terms make it sound more complicated than it is. Both transforms describe an image in terms of lots of wave shapes of different frequencies added together in different proportions. Cosine transforms use oscillating waves; a repeating shape. Wavelet transforms use wave shapes that are isolated in space; a wobble in an otherwise straight line. Because the wavelet decays to back to that straight line, blocks of image data don’t such visible edges where the wave shape is interrupted.

Theoretically, JPEG 2000 supports any bit depth; the amount of compression used will affect the precision of the result. JPEG 2000 has been used as the basis for video encoding systems such as the original Red camera codec, where this capability is used. Another nice trick of wavelet encoding is scaling: the image is effectively broken up into sequential chunks of data which represent smaller and smaller details. As such, as the image begins to arrive at the decoder, it’s possible to decode a low-resolution version of the image quickly, then add more detail as necessary, or when the rest of the data becomes available.

Despite the fact that it was last updated in 2013, making it actually older than the most recent JPEG, JPEG-2000 includes a lot more modern features than the original. There’s alpha channel transparency and 16- or 32-bit encoding by default, and programmers might be able to rely on slightly more robust handling of unusual ICC profiles. The IEEE has talked about HDR. Still, most decoders treat it as a better-performing JPEG and concentrate on 8-bit, sRGB/709 or Adobe RGB imaging.

JPEG XL

Not content with JPEG 2000, the Group asked for ideas in a request for proposals on JPEG XL in 2017. Current ideas for the format achieve 60% better compression performance than the original, though JPEG 2000 could often do that. The real aim appears to have been to provide a backward-compatibility layer to original JPEGs while offering new features. JPEG images can be losslessly converted to and from JPEG XL, making life a lot easier during a potential transition for anyone running a website with a lot of images to serve. The standard is designed to make that conversion easy on the CPU, meaning a web server could do it on the fly to suit whatever browser it was addressing.

High bit depth images up to 32 bits per component are supported, as are alpha channels, animations and various other options. RGB and YCrCb encoding is supported, and for the first time, there is fully-described consideration for specific colour gamuts and transfer functions other than the most common, including Rec. 2020, as well as both PQ and HLG brightness handling. A the final version of the JPEG XL standard is expected in October this year.

JPEG XR

Our final morsel from the Group’s oeuvre is actually based on a Microsoft idea from the mid-2000s, which was originally called Windows Media Photo. It’s not widely supported outside a rather Microsoft-centric ecosystem. XR offers transparency, RGB, various bit depths including a low-bit-depth 16-bit RGB565 mode, CMYK modes, and a specific reference to the Radiance RGBE high dynamic range encoding. Still, JPEG XR seems likely to be left behind. XL does many of the same things while offering that useful layer of backward and forward compatibility. XR gets a mention because the actual compression mathematics are interesting; it’s beyond the scope of this article to go into the photo core transform but it’s felt to offer the compression performance of JPEG-2000 with the complexity of JPEG, or to look better than JPEG at half the file size.

Next time, we look at the alteratives outside of the JPEG Group.