Diffusion models: why restoration-like training still generates new images

The explainer places diffusion in the same lineage as the series' earlier builds. Super-resolution and face restoration used GANs, which generate in a single pass but can suffer mode collapse, while object removal used LaMa, a dedicated filling model. Diffusion takes the opposite approach from GANs: it removes noise step by step, trading speed for stability, quality, and variety, which the author identifies as the reason it became dominant in image generation.

The key distinction the author draws is that restoration and generation run the same repeated noise removal but differ in starting point and conditions. Restoration moves toward an original image, making it a form of generation conditioned on a degraded input, while text-to-image starts from random noise and follows the prompt. That shared frame is how the same mechanism covers super-resolution, inpainting, and text2img, and the author notes ControlNet as a likely next topic.

On the practical side, Latent Diffusion compresses images into a latent space so computation drops enough to run on consumer GPUs, and Classifier-Free Guidance strengthens prompt adherence at the cost of unnatural results when pushed too far. The author also separates the engine from the product, noting that Stable Diffusion 3 and Flux use DiT with Flow Matching, DALL·E 1 was autoregressive rather than diffusion, and Midjourney and Firefly are presumed diffusion-based but undisclosed.