{"id":37123,"date":"2022-09-23T16:36:12","date_gmt":"2022-09-23T20:36:12","guid":{"rendered":"https:\/\/mjtsai.com\/blog\/?p=37123"},"modified":"2022-09-23T16:36:51","modified_gmt":"2022-09-23T20:36:51","slug":"stable-diffusion-based-image-compression","status":"publish","type":"post","link":"https:\/\/mjtsai.com\/blog\/2022\/09\/23\/stable-diffusion-based-image-compression\/","title":{"rendered":"Stable Diffusion Based Image Compression"},"content":{"rendered":"

Matthias Bühlmann<\/a> (via Hacker News<\/a>):<\/p>\n

\n

These examples make it quite evident that compressing these images with Stable Diffusion results in vastly superior image quality at a smaller file sizes compared to JPG and WebP. This quality comes with some important caveats which must be considered, as I will explain in the evaluation section, but at first glance, this is a very promising option for aggressive lossy image compression.<\/p>\n

[…]<\/p>\n

The main algorithm of Stable Diffusion, which generates new images from short text descriptions, operates on this latent space representation of images. It starts with random noise in the latent space representation and then iteratively de-noises this latent space image by using the trained U-Net, which in simple terms outputs predictions of what it thinks it “sees” in that noise, similar to how we sometimes see shapes and faces when looking at clouds. When Stable Diffusion is used to generate images, this iterative de-noising step is guided by the third ML model, the text encoder, which gives the U-Net information about what it should try to see in the noise. For the experimental image codec presented here, the text encoder is not needed.<\/p>\n

[…]<\/p>\n

To use Stable Diffusion as an image compression codec, I investigated how the latent representation generated by the VAE could be efficiently compressed.<\/p>\n<\/blockquote>\n\n

Previously:<\/p>\n