HEIF/HEIC has been on the news recently, both because Apple’s latest iPhones are using the format to store images, which briefly allowed them to circumvent an image size hurdle on Google Photos that Google’s own latest Pixel phones could not clear (Google has now apparently “fixed” this “bug”), and because Canon will be introducing HEIF in upcoming cameras as an output format option.
HEIF is a file format based on a collection of essential patents from 23 companies that can be licensed as a bundle. Among the better known companies (to photographers) in that pool are Samsung (>4,000 patents) and Canon (180). Depending on which source you read, Apple may also own 69 patents relevant patents, or none, so it’s unclear if they’d have to pay to use the format, especially in an rather asymmetric patent pool in which Samsung owns 60 times as many patents as Apple. In the photography market, though, Canon gets a biscuit every time a competitor sells a camera supporting HEIF.
This basically turn the clock back to 2002, when Forgent Networks wanted control over the JPEG standard based on a patent they’d acquired by purchasing another company, or to 1993, when Unisys started a long drawn out campaign to milk its LZW patent, used in a GIF implementation by CompuServe that at the time was the only existing GIF version – this became such an unhappy situation for everyone involved (including, actually, Unisys) that GIF variants circumventing the patent soon appeared, as well as PNG, designed to replace it.
One of the organisations involved with the HEIF patent pool, MPEG LA, has already gotten in hot waters for claiming that the Theora and VP8 video codecs violate the patents it represents, without ever fully substantiating this.
So why are we picking this over, say, JPEG 2000, JPEG XR, BPG, FLIF, WebP or zopflipng? To speak individually about all of the above is beyond this article, but me divulge a few quick opinions. WebP has recently gained some traction, and while it is officially stated to be available optionally as a lossless format, the common deployment I’ve seen is extremely lossy, and this has been known for some time. FLIF and zopflipng are unencumbered formats that offer compression that is certainly competitive with WebP and probably HEIC:
Unfortunately, at the current state of FLIF and zopflipng, compression is slow, and there is no clear statement on whether optimisations are possible that would change this.
With photography and videography having become commonplace activities for virtually all mobile phone users, there is significant pressure for finding image storage formats that can be encoded efficiently using onboard smartphone circuitry, and that have a small footprint in terms of needed space with a view to cloud storage.
Looking at the situation surrounding HEIF, one could easily come away with the impression that, in comparison with any single patent, the HEIF patent pool is an extreme anticompetitive instrument that has the potential to stifle innovation for years to come.
Indeed, David Balto, former FTC Policy Director for the Bureau of Competition has come to this exact warning regarding the MPEG LA, which describes itself as the “world’s leading packager of patent pools” in a research paper.
For the immediate future, it seems unlikely that significant disadvantages will arise to end users in dealing with HEI files. Open-source applications ImageMagick, GIMP and Krita are available free for anyone to convert such files into an unencumbered format such as uncompressed TIFF, and if needed this could be automated on desktop computing platforms. In practice, users may be paying indirectly, especially for devices made by companies other than those in the patent pool, which may help cement market leader positions of companies such as Apple and Canon, while, indeed, stifling innovation.
However, there is a school of thought that says that the web and the browsers needed to navigate it should be based entirely on free and open standards. It may be fortunate in that sense that browser market leader Google may be more interested in advancing its own invention, WebP, which Mozilla Firefox as well as open source graphics packages GIMP and ImageMagick also already support.
So perhaps the “bug” Google was really helping to fix is in fact one of anticompetitive practices by patent pools such as MPEG LA – Google’s own anticompetitive act may have simply balanced the force on this occasion. It remains to be hoped that companies are able to learn from the history disputes over GIF and JPEG, where there was really only negative fallout for everyone involved in trying to enforce patents on things that had de facto become public property through widespread unchecked use (cf. genericisation). The lesson must be that once something has become part of infrastructure, part of the fabric of the world wide web, any attempt to declare ownership over it will be futile and (at) best met with derision.