Stable Diffusion With Core ML on Apple Silicon
Apple (Hacker News):
Today, we are excited to release optimizations to Core ML for Stable Diffusion in macOS 13.1 and iOS 16.2, along with code to get started with deploying to Apple Silicon devices.
[…]
Beyond image generation from text prompts, developers are also discovering other creative uses for Stable Diffusion, such as image editing, in-painting, out-painting, super-resolution, style transfer and even color palette generation.
[…]
To learn more about how we optimized a model of this size and complexity to run on the Apple Neural Engine, you can check out our previous article on Deploying Transformers on the Apple Neural Engine. The optimization principles outlined in the article generalize to Stable Diffusion despite the fact that it is 19x larger than the model studied in the previous article. Optimizing Core ML for Stable Diffusion and simplifying model conversion makes it easier for developers to incorporate this technology in their apps in a privacy-preserving and economically feasible way, while getting the best performance on Apple Silicon.
This repository comprises:
python_coreml_stable_diffusion, a Python package for converting PyTorch models to Core ML format and performing image generation with Hugging Face diffusers in PythonStableDiffusion, a Swift package that developers can add to their Xcode projects as a dependency to deploy image generation capabilities in their apps. The Swift package relies on the Core ML model files generated bypython_coreml_stable_diffusion
An M2 MacBook Air is significantly faster than an M1 Pro MacBook Pro.
Previously:
2 Comments RSS · Twitter
This is pretty fascinating, I had originally thought the M2 a minor upgrade but the neural engine has ~45% more compute, which matches up pretty well with the speed increases. It is interesting that there is so much packed into this engine, which is complementary to the CPU and GPU. My guess would be that this is laying the hardware for the upcoming xrOS.
Read M1 Ultra was as fast as the fastest of Nvidia GPUs. At this application. Does the update improve on that?
