SlideShare a Scribd company logo

Image optimization q_auto - f_auto

•
•
Can Ozdoruk
Can Ozdoruk

For image optimization, reducing the quality doesn’t always lead to degradation of visual experience. In fact, precise adjustment of compression level and fine tuning of encoding settings can reduce significantly the file size without any noticeable degradation. But, there is no standard quality setting that works for all images - it depends on the compression algorithm, image format and content. And manually experimentation is not scalable. In this webinar we cover how to find the best quality compression level and optimal encoding settings, in order to produce a perceptually fine image while minimizing the file size.

1 of 41
Download to read offline
q_auto,f_auto A peek under the hood IMAGE COMPRESSION:
 GRACEFUL DEGRADATION
 FOR BETTER PERFORMANCE Jon Sneyers
 October 2016
AUTO EVERYTHING: G_AUTO,W_AUTO,DPR_AUTO,Q_AUTO,F_AUTO DELIVER THE RIGHT IMAGE TO EVERYONE ▸ Different crops depending on context (g_auto,w_auto)
 Responsive design Android tablet
 185K WebP
 1500x844 pixels Firefox desktop
 381K JPEG
 2000x1125 pixels iPhone
 54K JPEG
 500x500 pixels
AUTO EVERYTHING: G_AUTO,W_AUTO,DPR_AUTO,Q_AUTO,F_AUTO DELIVER THE RIGHT IMAGE TO EVERYONE Big Android phone
 High bandwidth
 144K WebP
 1000x1000 pixels Windows phone
 142K JPEG XR
 1000x1000 pixels iPhone
 54K JPEG
 500x500 pixels Retina iPhone
 201K JPEG
 1000x1000 pixels Small Android phone
 Low bandwidth
 43K WebP
 500x500 pixels Topic of this webinar ▸ Same image, different encoding depending on image and context (q_auto,f_auto,dpr_auto)

Q_AUTO,F_AUTO - INTRO FIXED QUALITY, FIXED FORMAT ▸ Common practice: fixed format, fixed quality, fixed encode settings ▸ E.g. WordPress default: re-encode as JPEG quality 82 ▸ E.g. Facebook, Twitter, Instagram: re-encode as low-quality JPEG ▸ This is suboptimal for three reasons: ▸ Optimal quality depends on the image: some images look fine at JPEG quality 50, others have visible compression artifacts at quality 85 ▸ Optimal quality/size trade-off depends on context
 (viewing device, network conditions, …) ▸ Best format depends on 1) image, and 2) receiver decode capabilities
Q_AUTO,F_AUTO - INTRO INTRODUCING Q_AUTO,F_AUTO ▸ Automatic quality, automatic image format ▸ q_auto:low, q_auto:eco, q_auto:good, q_auto:best ▸ Deliver the optimal file to every browser/device ▸ Most suitable format is picked among all formats the receiver accepts (Accepts header), can depend on the image ▸ Suitable quality settings are picked automatically ▸ Depending on the image content ▸ Save-Data client hint lowers the target quality
 (from q_auto:good to q_auto:eco)
Q_AUTO,F_AUTO - TARGET FORMATS TARGET IMAGE FORMATS ▸ JPEG ▸ All browsers, only opaque images (no alpha channel supported) ▸ PNG ▸ All browsers, not very useful for opaque photographs ▸ WebP ▸ Chrome/Opera/Android only, useful for all kinds of images ▸ WDP (JPEG XR) ▸ IE/Edge only, mostly useful for photographs
Q_AUTO,F_AUTO - TARGET FORMATS LOSSY IMAGE COMPRESSION BASICS ▸ Luma (Y or L) and Chroma (U,V or Cb,Cr or Co,Cg or a,b) ▸ Chroma subsampling
 4:4:4 vs 4:2:0
Q_AUTO,F_AUTO - TARGET FORMATS LOSSY IMAGE COMPRESSION BASICS ▸ Discrete Cosine Transform (DCT) LOW FREQUENCY
 NOTTOOLOSSY HIGHFREQUENCY
 VERYLOSSY
Q_AUTO,F_AUTO - TARGET FORMATS JPEG ENCODE PARAMETERS TO DECIDE ▸ Quality (amount of DCT quantization) ▸ Separate quality setting for luma and chroma ▸ Chroma subsampling: 4:4:4 vs 4:2:0 ▸ or “in between” (4:4:4 with lower quality for chroma than luma) ▸ MozJPEG vs libjpeg-turbo ▸ Progressive (smaller, previews) vs non-progressive (faster to decode) ▸ or “in between” (custom progressive scan script)
Q_AUTO,F_AUTO - TARGET FORMATS LOSSY PNG? SURE! ▸ PNG is a lossless format, but it can be used in a lossy way ▸ Two options: ▸ Quantize to PNG8 ▸ Maximum 256 RGBA colors ▸ Can use fewer than 256 colors for better compression ▸ Apply a lossy PNG24/32 transformation (diagonal blur)
Q_AUTO,F_AUTO - TARGET FORMATS WEBP: ACTUALLY TWO IMAGE FORMATS ▸ Lossy WebP (VP8 intra-frame) ▸ Always 4:2:0 chroma subsampling (can be problematic) ▸ Usually better than JPEG but certainly not always ▸ Supports alpha ▸ Lossless WebP ▸ Mostly similar to PNG but better compression ▸ Decisions: lossy or lossless? which quality?
 or fall back to JPEG or PNG?
Q_AUTO - ALGORITHM THE Q_AUTO ALGORITHM: OVERVIEW ▸ Heuristics that examine the original pixels ▸ Trial encodes ▸ Perceptual metric and heuristics to score trial encodes ▸ Heuristics to select the best format and to decide the actual encode settings based on the above information ▸ Important constraint: it has to be reasonably fast if we want to deploy it at “Cloudinary scale”. Latency and cpu cost has to be reasonable.
Uncompressed Image
 (no alpha channel) NON-PHOTOGRAPHIC? Photographic Non-photographic
 (text, line art, screenshot, …) CHROMA SUBSAMPLING?
 TWO TRIAL JPEG ENCODINGS
 (4:4:4 AND 4:2:0) Y-DSSIM SCORE Target quality
 (low, eco, good,best) TRIAL LOSSLESS PNG/WEBP LIBJPEG 4:2:0 ENCODING PNG ENCODING (PNGQUANT) LOSSLESS WEBP LOSSY WEBP ANALYSIS ADJUSTING QUALITY ENCODING LIBJPEG 4:4:4 ENCODING MOZJPEG 4:2:0 ENCODING Too large
 compared to
 trial JPEG very bad score:
 encode another trial JPEG
 at higher quality Few colors
Uncompressed Image
 with alpha channel NON-PHOTOGRAPHIC? Non-photographic
 (text, line art, screenshot, …) CHROMA SUBSAMPLING?
 TWO TRIAL JPEG ENCODINGS
 (4:4:4 AND 4:2:0) Y-DSSIM SCORE TRIAL LOSSLESS WEBP LIBJPEG 4:2:0 ENCODING PNG ENCODING (PNGQUANT) LOSSLESS WEBP LOSSY WEBP ANALYSIS ADJUSTING QUALITY ENCODING LIBJPEG 4:4:4 ENCODING MOZJPEG 4:2:0 ENCODING Photographic Too large
 compared to
 trial JPEG Possible, but not useful
 if WebP can be used JPEG does not support alpha Target quality
 (low, eco, good,best) Uncompressed Image
 with alpha channel very bad score:
 encode another trial JPEG
 at higher quality
Uncompressed Image
 with alpha channel LIBJPEG 4:2:0 ENCODING PNG ENCODING (PNGQUANT) LOSSLESS WEBP LOSSY WEBP ANALYSIS ADJUSTING QUALITY ENCODING LIBJPEG 4:4:4 ENCODING MOZJPEG 4:2:0 ENCODING Target quality
 (low, eco, good,best)
Q_AUTO - ALGORITHM PHOTOGRAPHIC OR NON-PHOTOGRAPHIC? Original: 634K PNG
 (lossless) 71K JPEG
 (lossy) 204K PNG
 (lossy)👍 👎
Q_AUTO - ALGORITHM PHOTOGRAPHIC OR NON-PHOTOGRAPHIC? Original: 221K PNG
 (lossless) 97K JPEG
 (lossy) 50K PNG
 (lossy) 👍👎
Q_AUTO - ALGORITHM CHROMA SUBSAMPLING? ▸ Compute a map of “chroma-related compression artifacts” ▸ A lot of artifacts? Use 4:4:4 with high chroma quality ▸ Don’t use lossy WebP in this case ▸ Some artifacts? Use 4:4:4 with lower chroma quality ▸ (Almost) no artifacts? Use 4:2:0 NON-PHOTOGRAPHIC? CHROMA SUBSAMPLING? Y-DSSIM SCORE TRIAL LOSSLESS PNG/WEBP LIBJPEG PNG LOSSLESS WEBP LOSSY
 WEBPMOZJPEG
Q_AUTO - ALGORITHM CHROMA SUBSAMPLING? Original: 505K PNG
 (lossless) 65K JPEG
 (q_75, 4:4:4) 57K JPEG
 (q_75, 4:2:0) 👍👎
Q_AUTO - ALGORITHM CHROMA SUBSAMPLING? Original image
 (lossless) 4:4:4 JPEG
 (lossy, q_100 and q_75) 4:2:0 JPEG
 (lossy, q_100 and q_75) 😱
Q_AUTO - ALGORITHM PERCEPTUAL METRIC: Y-DSSIM ▸ Compute multi-scale structural similarity metric (SSIM) between uncompressed and trial JPEG ▸ Adjust quality accordingly (repeat if needed) ▸ Only look at luma (Y) ▸ No need to look at chroma, chroma subsampling test already covers that NON-PHOTOGRAPHIC? CHROMA SUBSAMPLING? Y-DSSIM SCORE TRIAL LOSSLESS PNG/WEBP LIBJPEG PNG LOSSLESS WEBP LOSSY
 WEBPMOZJPEG
Q_AUTO - JPEG ENCODING ACTUAL JPEG ENCODE ▸ Use either MozJPEG or libjpeg-turbo (depending on quality parameters) ▸ Almost always use 5-scan “semi-progressive” scan script ▸ Faster encode/decode than default (~10 scans) progressive, still most of the benefits ▸ Only use non-progressive for ▸ very small images
 (no advantage) ▸ very large images
 (receiver might not have the memory) NON-PHOTOGRAPHIC? CHROMA SUBSAMPLING? Y-DSSIM SCORE TRIAL LOSSLESS PNG/WEBP LIBJPEG PNG LOSSLESS WEBP LOSSY
 WEBPMOZJPEG
Q_AUTO - JPEG ENCODING JPEG - DECODE TIME ▸ JPEG decode speed: ▸ Non-progressive: ~215 MP/s ▸ Default progressive: ~111 MP/s ▸ Semi-progressive: ~184 MP/s ▸ Total decode time has a (minor) effect on time-to-render, in particular if the image is cached locally ▸ If transfer is the bottleneck, (semi-)progressive has the advantage of progressive previews and smaller total size
Q_AUTO - PNG ENCODING ACTUAL PNG ENCODE ▸ Use pngquant to quantize colors & improve compression ▸ Use PNG8 if the error is not too bad ▸ The number of colors is picked to reach a specific MSE ▸ Otherwise use lossy PNG24/32 (rarely needed) ▸ Quantize not the pixel values, but the filtered pixel values w.r.t. the ‘average’ predictor ((left+up)/2) ▸ Slight diagonal blur,
 no quantization/dithering artifacts ▸ Then use optipng to optimize further NON-PHOTOGRAPHIC? CHROMA SUBSAMPLING? Y-DSSIM SCORE TRIAL LOSSLESS PNG/WEBP LIBJPEG PNG LOSSLESS WEBP LOSSY
 WEBPMOZJPEG
Q_AUTO - PNG ENCODING COLOR QUANTIZATION Original PNG24, 573KB 256 colors, 203KB 100 colors, 148KB 30 colors, 98KB 5 colors, 39KB 3 colors, 24KB
Q_AUTO - PNG ENCODING SOMETIMES PNG8 IS TOO LOSSY ▸ Original: 88,358 bytes ▸ PNG8: 21,739 bytes ▸ Too lossy: banding / visible dither
 
 ▸ Lossy PNG32: 57,552 bytes ▸ 53,101 bytes at q_auto:eco ▸ 64,771 bytes at q_auto:best
Q_AUTO - PNG ENCODING SOMETIMES PNG8 IS TOO LOSSY ▸ Original: 88,358 bytes ▸ PNG8: 21,739 bytes
 
 ▸ Lossy PNG32: 53,101 bytes
EVALUATION Q_AUTO
Q_AUTO - EVALUATION EVALUATION: A DELICATE TRADE-OFF ▸ File size (bandwidth/storage cost) ▸ Perceptual quality compared to original ▸ Encode time (cpu cost, latency of new images) ▸ End-user experience: ▸ time-to-render
 (includes latency, download, decode, progressive rendering) ▸ perceptual quality with no access to original 
 (e.g. blur vs blockiness, etc.)
Q_AUTO - EVALUATION VISUAL INSPECTION
Q_AUTO - EVALUATION VISUAL INSPECTION
Q_AUTO - EVALUATION VISUAL INSPECTION
Q_AUTO - EVALUATION VISUAL INSPECTION
Q_AUTO - EVALUATION VISUAL INSPECTION
Q_AUTO - EVALUATION VISUAL INSPECTION
Q_AUTO - EVALUATION PERCEPTUAL METRICS ▸ Commonly used simple metrics are not really good enough (e.g. RGB-PSNR) ▸ Perceptually uniform color space:
 RGB < YCbCr < HSL < L*a*b* ▸ Kornel Lesiński’s DSSIM: L*a*b* multi-scale SSIM ▸ Our own metric: a variant of L*a*b* multi-scale SSIM with some improvements
Q_AUTO - EVALUATION - PHOTOGRAPHIC IMAGES CLOUDINARY_DSSIM (MOST ACCURATE) Y-DSSIM (USED IN Q_AUTO) KORNEL’S DSSIM ARTIFACT DANGER ZONE
Q_AUTO - EVALUATION - NON-PHOTOGRAPHIC IMAGES
Q_AUTO,F_AUTO - CONCLUSION CONCLUSION ▸ Using q_auto,f_auto ensures that every image gets encoded at the right quality, in the best available format ▸ No visible compression artifacts ▸ No wasted bandwidth ▸ Less bandwidth needed on average 
 (though not for every individual image: the q_auto quality might be higher than the default quality)
UPCOMING CLOUDINARY WEBINARS Responsive Images: Past, Present & Future Wednesday Nov 16th , 11am PT Image Management: Buy vs. Build, Build.com Wednesday Dec 7th , 11am PT Don’t fall behind. Check upcoming webinars: http://info.cloudinary.com/webinars.html
QUESTIONS? q_auto,f_auto Uncompressed Image NON-PHOTOGRAPHIC? Photographic Non-photographic CHROMA SUBSAMPLING?
 TWO TRIAL JPEG ENCODINGS
 (4:4:4 AND 4:2:0) Y-DSSIM SCORE Target quality TRIAL LOSSLESS PNG/WEBP LIBJPEG 4:2:0 PNG ENCODING (PNGQUANT) LOSSLESS WEBP LOSSY WEBP LIBJPEG 4:4:4 MOZJPEG 4:2:0 jon@cloudinary.com

More Related Content

Image optimization q_auto - f_auto

  • 1. q_auto,f_auto A peek under the hood IMAGE COMPRESSION:
 GRACEFUL DEGRADATION
 FOR BETTER PERFORMANCE Jon Sneyers
 October 2016
  • 2. AUTO EVERYTHING: G_AUTO,W_AUTO,DPR_AUTO,Q_AUTO,F_AUTO DELIVER THE RIGHT IMAGE TO EVERYONE ▸ Different crops depending on context (g_auto,w_auto)
 Responsive design Android tablet
 185K WebP
 1500x844 pixels Firefox desktop
 381K JPEG
 2000x1125 pixels iPhone
 54K JPEG
 500x500 pixels
  • 3. AUTO EVERYTHING: G_AUTO,W_AUTO,DPR_AUTO,Q_AUTO,F_AUTO DELIVER THE RIGHT IMAGE TO EVERYONE Big Android phone
 High bandwidth
 144K WebP
 1000x1000 pixels Windows phone
 142K JPEG XR
 1000x1000 pixels iPhone
 54K JPEG
 500x500 pixels Retina iPhone
 201K JPEG
 1000x1000 pixels Small Android phone
 Low bandwidth
 43K WebP
 500x500 pixels Topic of this webinar ▸ Same image, different encoding depending on image and context (q_auto,f_auto,dpr_auto)

  • 4. Q_AUTO,F_AUTO - INTRO FIXED QUALITY, FIXED FORMAT ▸ Common practice: fixed format, fixed quality, fixed encode settings ▸ E.g. WordPress default: re-encode as JPEG quality 82 ▸ E.g. Facebook, Twitter, Instagram: re-encode as low-quality JPEG ▸ This is suboptimal for three reasons: ▸ Optimal quality depends on the image: some images look fine at JPEG quality 50, others have visible compression artifacts at quality 85 ▸ Optimal quality/size trade-off depends on context
 (viewing device, network conditions, …) ▸ Best format depends on 1) image, and 2) receiver decode capabilities
  • 5. Q_AUTO,F_AUTO - INTRO INTRODUCING Q_AUTO,F_AUTO ▸ Automatic quality, automatic image format ▸ q_auto:low, q_auto:eco, q_auto:good, q_auto:best ▸ Deliver the optimal file to every browser/device ▸ Most suitable format is picked among all formats the receiver accepts (Accepts header), can depend on the image ▸ Suitable quality settings are picked automatically ▸ Depending on the image content ▸ Save-Data client hint lowers the target quality
 (from q_auto:good to q_auto:eco)
  • 6. Q_AUTO,F_AUTO - TARGET FORMATS TARGET IMAGE FORMATS ▸ JPEG ▸ All browsers, only opaque images (no alpha channel supported) ▸ PNG ▸ All browsers, not very useful for opaque photographs ▸ WebP ▸ Chrome/Opera/Android only, useful for all kinds of images ▸ WDP (JPEG XR) ▸ IE/Edge only, mostly useful for photographs
  • 7. Q_AUTO,F_AUTO - TARGET FORMATS LOSSY IMAGE COMPRESSION BASICS ▸ Luma (Y or L) and Chroma (U,V or Cb,Cr or Co,Cg or a,b) ▸ Chroma subsampling
 4:4:4 vs 4:2:0
  • 8. Q_AUTO,F_AUTO - TARGET FORMATS LOSSY IMAGE COMPRESSION BASICS ▸ Discrete Cosine Transform (DCT) LOW FREQUENCY
 NOTTOOLOSSY HIGHFREQUENCY
 VERYLOSSY
  • 9. Q_AUTO,F_AUTO - TARGET FORMATS JPEG ENCODE PARAMETERS TO DECIDE ▸ Quality (amount of DCT quantization) ▸ Separate quality setting for luma and chroma ▸ Chroma subsampling: 4:4:4 vs 4:2:0 ▸ or “in between” (4:4:4 with lower quality for chroma than luma) ▸ MozJPEG vs libjpeg-turbo ▸ Progressive (smaller, previews) vs non-progressive (faster to decode) ▸ or “in between” (custom progressive scan script)
  • 10. Q_AUTO,F_AUTO - TARGET FORMATS LOSSY PNG? SURE! ▸ PNG is a lossless format, but it can be used in a lossy way ▸ Two options: ▸ Quantize to PNG8 ▸ Maximum 256 RGBA colors ▸ Can use fewer than 256 colors for better compression ▸ Apply a lossy PNG24/32 transformation (diagonal blur)
  • 11. Q_AUTO,F_AUTO - TARGET FORMATS WEBP: ACTUALLY TWO IMAGE FORMATS ▸ Lossy WebP (VP8 intra-frame) ▸ Always 4:2:0 chroma subsampling (can be problematic) ▸ Usually better than JPEG but certainly not always ▸ Supports alpha ▸ Lossless WebP ▸ Mostly similar to PNG but better compression ▸ Decisions: lossy or lossless? which quality?
 or fall back to JPEG or PNG?
  • 12. Q_AUTO - ALGORITHM THE Q_AUTO ALGORITHM: OVERVIEW ▸ Heuristics that examine the original pixels ▸ Trial encodes ▸ Perceptual metric and heuristics to score trial encodes ▸ Heuristics to select the best format and to decide the actual encode settings based on the above information ▸ Important constraint: it has to be reasonably fast if we want to deploy it at “Cloudinary scale”. Latency and cpu cost has to be reasonable.
  • 13. Uncompressed Image
 (no alpha channel) NON-PHOTOGRAPHIC? Photographic Non-photographic
 (text, line art, screenshot, …) CHROMA SUBSAMPLING?
 TWO TRIAL JPEG ENCODINGS
 (4:4:4 AND 4:2:0) Y-DSSIM SCORE Target quality
 (low, eco, good,best) TRIAL LOSSLESS PNG/WEBP LIBJPEG 4:2:0 ENCODING PNG ENCODING (PNGQUANT) LOSSLESS WEBP LOSSY WEBP ANALYSIS ADJUSTING QUALITY ENCODING LIBJPEG 4:4:4 ENCODING MOZJPEG 4:2:0 ENCODING Too large
 compared to
 trial JPEG very bad score:
 encode another trial JPEG
 at higher quality Few colors
  • 14. Uncompressed Image
 with alpha channel NON-PHOTOGRAPHIC? Non-photographic
 (text, line art, screenshot, …) CHROMA SUBSAMPLING?
 TWO TRIAL JPEG ENCODINGS
 (4:4:4 AND 4:2:0) Y-DSSIM SCORE TRIAL LOSSLESS WEBP LIBJPEG 4:2:0 ENCODING PNG ENCODING (PNGQUANT) LOSSLESS WEBP LOSSY WEBP ANALYSIS ADJUSTING QUALITY ENCODING LIBJPEG 4:4:4 ENCODING MOZJPEG 4:2:0 ENCODING Photographic Too large
 compared to
 trial JPEG Possible, but not useful
 if WebP can be used JPEG does not support alpha Target quality
 (low, eco, good,best) Uncompressed Image
 with alpha channel very bad score:
 encode another trial JPEG
 at higher quality
  • 15. Uncompressed Image
 with alpha channel LIBJPEG 4:2:0 ENCODING PNG ENCODING (PNGQUANT) LOSSLESS WEBP LOSSY WEBP ANALYSIS ADJUSTING QUALITY ENCODING LIBJPEG 4:4:4 ENCODING MOZJPEG 4:2:0 ENCODING Target quality
 (low, eco, good,best)
  • 16. Q_AUTO - ALGORITHM PHOTOGRAPHIC OR NON-PHOTOGRAPHIC? Original: 634K PNG
 (lossless) 71K JPEG
 (lossy) 204K PNG
 (lossy)👍 👎
  • 17. Q_AUTO - ALGORITHM PHOTOGRAPHIC OR NON-PHOTOGRAPHIC? Original: 221K PNG
 (lossless) 97K JPEG
 (lossy) 50K PNG
 (lossy) 👍👎
  • 18. Q_AUTO - ALGORITHM CHROMA SUBSAMPLING? ▸ Compute a map of “chroma-related compression artifacts” ▸ A lot of artifacts? Use 4:4:4 with high chroma quality ▸ Don’t use lossy WebP in this case ▸ Some artifacts? Use 4:4:4 with lower chroma quality ▸ (Almost) no artifacts? Use 4:2:0 NON-PHOTOGRAPHIC? CHROMA SUBSAMPLING? Y-DSSIM SCORE TRIAL LOSSLESS PNG/WEBP LIBJPEG PNG LOSSLESS WEBP LOSSY
 WEBPMOZJPEG
  • 19. Q_AUTO - ALGORITHM CHROMA SUBSAMPLING? Original: 505K PNG
 (lossless) 65K JPEG
 (q_75, 4:4:4) 57K JPEG
 (q_75, 4:2:0) 👍👎
  • 20. Q_AUTO - ALGORITHM CHROMA SUBSAMPLING? Original image
 (lossless) 4:4:4 JPEG
 (lossy, q_100 and q_75) 4:2:0 JPEG
 (lossy, q_100 and q_75) 😱
  • 21. Q_AUTO - ALGORITHM PERCEPTUAL METRIC: Y-DSSIM ▸ Compute multi-scale structural similarity metric (SSIM) between uncompressed and trial JPEG ▸ Adjust quality accordingly (repeat if needed) ▸ Only look at luma (Y) ▸ No need to look at chroma, chroma subsampling test already covers that NON-PHOTOGRAPHIC? CHROMA SUBSAMPLING? Y-DSSIM SCORE TRIAL LOSSLESS PNG/WEBP LIBJPEG PNG LOSSLESS WEBP LOSSY
 WEBPMOZJPEG
  • 22. Q_AUTO - JPEG ENCODING ACTUAL JPEG ENCODE ▸ Use either MozJPEG or libjpeg-turbo (depending on quality parameters) ▸ Almost always use 5-scan “semi-progressive” scan script ▸ Faster encode/decode than default (~10 scans) progressive, still most of the benefits ▸ Only use non-progressive for ▸ very small images
 (no advantage) ▸ very large images
 (receiver might not have the memory) NON-PHOTOGRAPHIC? CHROMA SUBSAMPLING? Y-DSSIM SCORE TRIAL LOSSLESS PNG/WEBP LIBJPEG PNG LOSSLESS WEBP LOSSY
 WEBPMOZJPEG
  • 23. Q_AUTO - JPEG ENCODING JPEG - DECODE TIME ▸ JPEG decode speed: ▸ Non-progressive: ~215 MP/s ▸ Default progressive: ~111 MP/s ▸ Semi-progressive: ~184 MP/s ▸ Total decode time has a (minor) effect on time-to-render, in particular if the image is cached locally ▸ If transfer is the bottleneck, (semi-)progressive has the advantage of progressive previews and smaller total size
  • 24. Q_AUTO - PNG ENCODING ACTUAL PNG ENCODE ▸ Use pngquant to quantize colors & improve compression ▸ Use PNG8 if the error is not too bad ▸ The number of colors is picked to reach a specific MSE ▸ Otherwise use lossy PNG24/32 (rarely needed) ▸ Quantize not the pixel values, but the filtered pixel values w.r.t. the ‘average’ predictor ((left+up)/2) ▸ Slight diagonal blur,
 no quantization/dithering artifacts ▸ Then use optipng to optimize further NON-PHOTOGRAPHIC? CHROMA SUBSAMPLING? Y-DSSIM SCORE TRIAL LOSSLESS PNG/WEBP LIBJPEG PNG LOSSLESS WEBP LOSSY
 WEBPMOZJPEG
  • 25. Q_AUTO - PNG ENCODING COLOR QUANTIZATION Original PNG24, 573KB 256 colors, 203KB 100 colors, 148KB 30 colors, 98KB 5 colors, 39KB 3 colors, 24KB
  • 26. Q_AUTO - PNG ENCODING SOMETIMES PNG8 IS TOO LOSSY ▸ Original: 88,358 bytes ▸ PNG8: 21,739 bytes ▸ Too lossy: banding / visible dither
 
 ▸ Lossy PNG32: 57,552 bytes ▸ 53,101 bytes at q_auto:eco ▸ 64,771 bytes at q_auto:best
  • 27. Q_AUTO - PNG ENCODING SOMETIMES PNG8 IS TOO LOSSY ▸ Original: 88,358 bytes ▸ PNG8: 21,739 bytes
 
 ▸ Lossy PNG32: 53,101 bytes
  • 29. Q_AUTO - EVALUATION EVALUATION: A DELICATE TRADE-OFF ▸ File size (bandwidth/storage cost) ▸ Perceptual quality compared to original ▸ Encode time (cpu cost, latency of new images) ▸ End-user experience: ▸ time-to-render
 (includes latency, download, decode, progressive rendering) ▸ perceptual quality with no access to original 
 (e.g. blur vs blockiness, etc.)
  • 36. Q_AUTO - EVALUATION PERCEPTUAL METRICS ▸ Commonly used simple metrics are not really good enough (e.g. RGB-PSNR) ▸ Perceptually uniform color space:
 RGB < YCbCr < HSL < L*a*b* ▸ Kornel Lesiński’s DSSIM: L*a*b* multi-scale SSIM ▸ Our own metric: a variant of L*a*b* multi-scale SSIM with some improvements
  • 37. Q_AUTO - EVALUATION - PHOTOGRAPHIC IMAGES CLOUDINARY_DSSIM (MOST ACCURATE) Y-DSSIM (USED IN Q_AUTO) KORNEL’S DSSIM ARTIFACT DANGER ZONE
  • 38. Q_AUTO - EVALUATION - NON-PHOTOGRAPHIC IMAGES
  • 39. Q_AUTO,F_AUTO - CONCLUSION CONCLUSION ▸ Using q_auto,f_auto ensures that every image gets encoded at the right quality, in the best available format ▸ No visible compression artifacts ▸ No wasted bandwidth ▸ Less bandwidth needed on average 
 (though not for every individual image: the q_auto quality might be higher than the default quality)
  • 40. UPCOMING CLOUDINARY WEBINARS Responsive Images: Past, Present & Future Wednesday Nov 16th , 11am PT Image Management: Buy vs. Build, Build.com Wednesday Dec 7th , 11am PT Don’t fall behind. Check upcoming webinars: http://info.cloudinary.com/webinars.html
  • 41. QUESTIONS? q_auto,f_auto Uncompressed Image NON-PHOTOGRAPHIC? Photographic Non-photographic CHROMA SUBSAMPLING?
 TWO TRIAL JPEG ENCODINGS
 (4:4:4 AND 4:2:0) Y-DSSIM SCORE Target quality TRIAL LOSSLESS PNG/WEBP LIBJPEG 4:2:0 PNG ENCODING (PNGQUANT) LOSSLESS WEBP LOSSY WEBP LIBJPEG 4:4:4 MOZJPEG 4:2:0 jon@cloudinary.com