Click here to flash read.
Diffusion models (DMs) have recently gained attention with state-of-the-art
performance in text-to-image synthesis. Abiding by the tradition in deep
learning, DMs are trained and evaluated on the images with fixed sizes.
However, users are demanding for various images with specific sizes and various
aspect ratio. This paper focuses on adapting text-to-image diffusion models to
handle such variety while maintaining visual fidelity. First we observe that,
during the synthesis, lower resolution images suffer from incomplete object
portrayal, while higher resolution images exhibit repetitively disordered
presentation. Next, we establish a statistical relationship indicating that
attention entropy changes with token quantity, suggesting that models aggregate
spatial information in proportion to image resolution. The subsequent
interpretation on our observations is that objects are incompletely depicted
due to limited spatial information for low resolutions, while repetitively
disorganized presentation arises from redundant spatial information for high
resolutions. From this perspective, we propose a scaling factor to alleviate
the change of attention entropy and mitigate the defective pattern observed.
Extensive experimental results validate the efficacy of the proposed scaling
factor, enabling models to achieve better visual effects, image quality, and
text alignment. Notably, these improvements are achieved without additional
training or fine-tuning techniques.
No creative common's license