🐦 Twitter Post Details

Viewing enriched Twitter post

@_akhaliq

Dense Text-to-Image Generation with Attention Modulation github: https://t.co/HWAIot62Di web demo: https://t.co/ihQV6thM00 Existing text-to-image diffusion models struggle to synthesize realistic images given dense captions, where each text prompt provides a detailed description for a specific image region. To address this, we propose DenseDiffusion, a training-free method that adapts a pre-trained text-to-image model to handle such dense captions while offering control over the scene layout. We first analyze the relationship between generated images' layouts and the pre-trained model's intermediate attention maps. Next, we develop an attention modulation method that guides objects to appear in specific regions according to layout guidance. Without requiring additional fine-tuning or datasets, we improve image generation performance given dense captions regarding both automatic and human evaluation scores. In addition, we achieve similar-quality visual results with models specifically trained with layout conditions.

🔧 Raw API Response

{
  "user": {
    "created_at": "2014-04-27T00:20:12.000Z",
    "default_profile_image": false,
    "description": "AI research paper tweets, ML @Gradio (acq. by @HuggingFace 🤗)\n\ndm for promo",
    "fast_followers_count": 0,
    "favourites_count": 26248,
    "followers_count": 229989,
    "friends_count": 1862,
    "has_custom_timelines": true,
    "is_translator": false,
    "listed_count": 3046,
    "location": "subscribe → ",
    "media_count": 13651,
    "name": "AK",
    "normal_followers_count": 229989,
    "possibly_sensitive": false,
    "profile_banner_url": "https://pbs.twimg.com/profile_banners/2465283662/1610997549",
    "profile_image_url_https": "https://pbs.twimg.com/profile_images/1451191636810092553/kpM5Fe12_normal.jpg",
    "screen_name": "_akhaliq",
    "statuses_count": 21274,
    "translator_type": "none",
    "url": "https://t.co/TbGnXZJwEc",
    "verified": false,
    "withheld_in_countries": [],
    "id_str": "2465283662"
  },
  "id": "1696155079458406758",
  "conversation_id": "1696155079458406758",
  "full_text": "Dense Text-to-Image Generation with Attention Modulation\n\ngithub: https://t.co/HWAIot62Di\nweb demo: https://t.co/ihQV6thM00\n\nExisting text-to-image diffusion models struggle to synthesize realistic images given dense captions, where each text prompt provides a detailed description for a specific image region. To address this, we propose DenseDiffusion, a training-free method that adapts a pre-trained text-to-image model to handle such dense captions while offering control over the scene layout. We first analyze the relationship between generated images' layouts and the pre-trained model's intermediate attention maps. Next, we develop an attention modulation method that guides objects to appear in specific regions according to layout guidance. Without requiring additional fine-tuning or datasets, we improve image generation performance given dense captions regarding both automatic and human evaluation scores. In addition, we achieve similar-quality visual results with models specifically trained with layout conditions.",
  "reply_count": 2,
  "retweet_count": 85,
  "favorite_count": 325,
  "hashtags": [],
  "symbols": [],
  "user_mentions": [],
  "urls": [
    {
      "url": "https://t.co/kE6t3GU3ZH",
      "expanded_url": "https://github.com/naver-ai/densediffusion",
      "display_url": "github.com/naver-ai/dense…"
    },
    {
      "url": "https://t.co/8RtFuYiPxJ",
      "expanded_url": "https://github.com/naver-ai/DenseDiffusion/blob/main/gradio_app.py",
      "display_url": "github.com/naver-ai/Dense…"
    }
  ],
  "media": [
    {
      "media_url": "https://pbs.twimg.com/media/F4nz8llXcAErqgy.jpg",
      "type": "photo"
    }
  ],
  "url": "https://twitter.com/_akhaliq/status/1696155079458406758",
  "created_at": "2023-08-28T13:37:38.000Z",
  "#sort_index": "1696155079458406758",
  "view_count": 57207,
  "quote_count": 3,
  "is_quote_tweet": false,
  "is_retweet": false,
  "is_pinned": false,
  "is_truncated": true,
  "startUrl": "https://twitter.com/_akhaliq/status/1696155079458406758"
}