Stable streaming in your pocket? “Draw Things” brings AI images to the iPhone

On Wednesday, a San Francisco-based developer named Liu Liu released Draw Things: AI Generation, a free app available from the App Store that allows iPhone owners to run the popular Stable Diffusion AI image generator. Type in a description and the app generates an image in minutes. This is a significant step towards bringing computer graphics to a wider audience, with the added privacy of running it on your own hardware.

Introduced in August, Stable Diffusion (SD) is an AI image generator model that creates new images from text descriptions (called “prompts”). Typically, users run SD through the DreamStudio commercial service, on a remote cloud machine with rented compute time, or locally on a PC using a custom open source implementation. When run locally, SD requires a fairly powerful GPU to generate frames quickly, but some developers have optimized the model to run on older GPUs with less VRAM (if you don’t mind the wait longer to see results).

In a similar vein, Liu Liu managed to optimize Stable Diffusion to work on the iPhone, a somewhat difficult process that the developer described in a blog post. “The main challenge is to run the app on iPhone devices with 6GB RAM,” writes Liu Liu. “6GiB seems like a lot, but iOS will start killing your app if you use more than 2.8GiB on a 6GiB device, and more than 2GiB on a 4GiB device.”

When you first run Draw Things, the app downloads several necessary files, including the Stable Diffusion 1.4 model, to your iPhone. To use it, type a prompt at the top of the screen, then press “Generate”. Between image generation, tap the number at the top center of the screen to randomize the seed, which is a number that partially guides the image generation.

On our iPhone 11 Pro, generating a 384×384 image took just over two minutes. It’s faster on an iPhone 14 Pro, according to Liu Liu, generating an image in about a minute. In any case, the SD is greedy in calculations. After successive generations, our iPhone has become particularly hot to the touch.

It should be noted that with Stable Diffusion, 384×384 images often yield relatively poor results with little detail, as the creators of SD trained the model using 512×512 images. When attempting to generating a 512×512 image on our iPhone 11 Pro, we got a warning and continued anyway, but the app crashed into a black screen.

In addition to the usual image generation tasks, Draw Things also supports inpainting, which allows you to replace part of an image with AI-generated images, and load synthetic models from additional images such as the unauthorized “Modern Disney Diffusion” model (which generates Disney-looking characters) and the anime-powered “Waifu Diffusion” model (we’ve tested, and it’s possible to generate NSFW material using the app, so be warned). The inclusion of these features means that Draw Things might not stay in the App Store for long if it becomes popular, as the content it generates may violate Apple’s terms of service.

Whatever its fate, Draw Things feels like an important proof of concept, showing that Stable Diffusion can run locally on the iPhone, albeit slowly. If the app stays, maybe with some filters or tweaks, Liu Liu describes the possibility of potential future optimizations that could speed things up: “I probably still left 50% performance on the table.”

This isn’t the first time Stable Diffusion has worked on an iPhone. In September, developer Matt Waller ran Stable Broadcast locally on his iPhone XS, but he didn’t provide an App Store app that would allow others to replicate the exploit. We are likely to see more local AI image synthesis on smartphones as the computing power of devices continues to increase.

Go to chat…