What's the best image optimization for web and mobile apps?

Vector graphics scale, but not always

Generate enough images to cover the responsive spectrum, but not too many where we end up harming ourselves.

One promising best image optimization approach lies in taking advantage of more scalable vector graphics formats, like SVG and PNG, that encode images using vectors rather than pixels. These provide considerable size improvements that scale well compared to other formats, like GIF and JPEG. But they are not well-supported across all browsers or good for all image types.

Simple illustrations work great for SVG, but photos don't. Also, SVG support is a bit spotty still and hasn't been worked into workflows quite yet. With better workflows, it's possible that SVGs will completely replace PNGs and JPEGs for logos and illustrations.

Robert Moseley, director of solution engineering at Cloudinary, suggested one of the biggest gaps that developers have without feedback is accounting for the image type and the browser in which the image will be displayed.

"Choosing the right format based on the image type can get more complicated as browsers vary in their support for image formats, so even basic errors, like using a PNG instead of a JPEG, happen all the time," Moseley said.

Images don't scale equally

Another best image optimization gap occurs as developers adjust the various settings for each image format, which, in turn, affects the visual quality of the image. An 80% quality JPEG might look great for a photograph but generate artifacts and hard lines for an image with embedded text. This often leads developers to try a one-size-fits-all approach, and the end result is a highly archaic system that is much heavier than it needs to be. "Analyzing each image for its unique contents, colors, type (photograph vs. illustration, etc.) is needed to develop an optimal system," Moseley said.

Another gap for best image optimization is that delivering the image at the proper dimensions -- especially for a responsive site -- rarely gets done.

"Now that we have the HTML5 reset attribute, there's no reason we should still be delivering the largest required image no matter the breakpoint anymore," Moseley said. "Each pixel requires data, so reducing the pixels of the image to what is going to be displayed is a no-brainer. However, because images are often difficult to optimize, this is rarely done."

Cloudinary has developed one novel best image optimization approach to improve the kind of feedback that data developers can use. Client Hints cleans up the HTML markup and moves the decision about optimizing image selection to the server.

"Ultimately, to improve the user interface, we need a real-time image generation pipeline that can utilize all of the above to ensure that each unique visitor is getting the best assets for their experience," Moseley said.

Consider new formats

Tools are emerging for converting existing JPEG and GIF assets into more scalable formats. But this is a two-part problem. The first part is transcoding a JPG or GIF to other, better formats. Then, the second part is delivering those assets to the users.

"Modern image formats, like WebP and JPEG XR, are not universally supported, so while there are tools out there to transcode into those formats, you still need to conditionally deliver them only to the visitors that are using browsers or devices that support them," Moseley said.

GIFs tend to be a highly inefficient format, so enterprises must be thoughtful in adopting better formats. WebP supports animation as well, and there is now universal browser support for animated PNGs. An even better option is to just use a video format, like MP4 or WebM, and loop or autoplay it. The new GIFV format is just an MP4 video. Using a video format will save around 90 percent over an image format, even WebP or APNG.

Create a bandwidth budget

If bandwidth was the only best image optimization concern, creating an image at every pixel between your smallest and largest breakpoints would be the way to go. Everyone, no matter their viewport width, would get a pixel-perfect image and not have to download a single byte of extra data. However, there are other considerations, like the CDN cache hit ratio.

"If you created an image at every pixel, your cache hit ratio is going to be approaching 0 percent, negating any benefit you would see in bandwidth reduction," Moseley said. "So, we really need to strike a balance between the two -- generate enough images to cover the responsive spectrum, but not too many where we end up harming ourselves."

One good approach lies in setting a "bandwidth budget." This entails creating different versions of images in KB-size steps. For instance, an image might be displayed as small as 300px wide in the smallest breakpoint but as big as 1400px wide in the largest. The best way to support the entire spectrum between 300px and 1400px would be to create a new size for the image every 20 KB. This ensures that no visitor will ever have to download more than 20 KB than is necessary and will still have a very high CDN cache hit ratio.