Home > Research > Publications & Outputs > Unified Prompt Attack Against Text-to-Image Gen...

Electronic data

  • paper

    Accepted author manuscript, 1.46 MB, PDF document

Links

Text available via DOI:

View graph of relations

Unified Prompt Attack Against Text-to-Image Generation Models

Research output: Contribution to Journal/MagazineJournal articlepeer-review

Published
  • Duo Peng
  • Qiuhong Ke
  • Mark He Huang
  • Ping Hu
  • Jun Liu
Close
<mark>Journal publication date</mark>30/06/2025
<mark>Journal</mark>IEEE Transactions on Pattern Analysis and Machine Intelligence
Issue number6
Volume47
Number of pages19
Pages (from-to)4816-4834
Publication StatusPublished
Early online date25/02/25
<mark>Original language</mark>English

Abstract

Text-to-Image (T2I) models have advanced significantly, but their growing popularity raises security concerns due to their potential to generate harmful images. To address these issues, we propose UPAM, a novel framework to evaluate the robustness of T2I models from an attack perspective. Unlike prior methods that focus solely on textual defenses, UPAM unifies the attack on both textual and visual defenses. Additionally, it enables gradient-based optimization, overcoming reliance on enumeration for improved efficiency and effectiveness. To handle cases where T2I models block image outputs due to defenses, we introduce Sphere-Probing Learning (SPL) to enable optimization even without image results. Following SPL, our model bypasses defenses, inducing the generation of harmful content. To ensure semantic alignment with attacker intent, we propose Semantic-Enhancing Learning (SEL) for precise semantic control. UPAM also prioritizes the naturalness of adversarial prompts using In-context Naturalness Enhancement (INE), making them harder for human examiners to detect. Additionally, we address the issue of iterative queries–common in prior methods and easily detectable by API defenders–by introducing Transferable Attack Learning (TAL), allowing effective attacks with minimal queries. Extensive experiments validate UPAM’s superiority in effectiveness, efficiency, naturalness, and low query detection rates.