AI image editing gets smarter, cuts manual work out of design pipeline
Researchers have cracked a longstanding problem in AI-powered image editing: getting the tool to change only what you want without mangling the rest. The breakthrough could speed up creative workflows in design studios, e-commerce, and media companies by reducing the back-and-forth needed to fix unwanted alterations.
Originaltitel: StyleDiffusion: Prompt-Embedding Inversion for Text-Based Editing
<p>A significant research effort is focused on exploiting the outstanding capacities of pretrained diffusion models for image editing. Approaches either fine tune the model, or invert the image in the latent space of the pretrained model. However, they suffer from two problems: (i) unsatisfactory results in selected regions and unexpected changes in non-selected regions, and (ii) the need for careful text prompt editing: the prompt should include all visual objects in the input image. To address this, we propose two improvements: (i) only optimizing the input of the value linear network in the cross-attention layers is sufficiently powerful to reconstruct a real image, and (ii) attention regularization to preserve the object-like attention maps after reconstruction and editing, enabling accurate style editing without causing significant structural change. We further improve the editing technique used for the unconditional branch of classifier-free guidance as used by P2P. Extensive experimental prompt-editing results on a variety of images demonstrate qualitatively and quantitatively that our method has editing capabilities superior to those of existing and concurrent works. Our StyleDiffusion code is available at https://github.com/sen-mao/StyleDiffusion.</p>