Viewing enriched Twitter post
Vision transformers need registers! Or at least, it seems they 𝘸𝘢𝘯𝘵 some… ViTs have artifacts in attention maps. It’s due to the model using these patches as “registers”. Just add new tokens (“[reg]”): - no artifacts - interpretable attention maps 🦖 - improved performances! https://t.co/mineuOjRid
{
"user": {
"created_at": "2021-03-15T09:44:31.000Z",
"default_profile_image": false,
"description": "PhD student, building big vision models @ INRIA & MetaAI",
"fast_followers_count": 0,
"favourites_count": 396,
"followers_count": 754,
"friends_count": 403,
"has_custom_timelines": false,
"is_translator": false,
"listed_count": 12,
"location": "",
"media_count": 25,
"name": "Timothee Darcet @ICCV 🥐🇫🇷",
"normal_followers_count": 754,
"possibly_sensitive": false,
"profile_image_url_https": "https://pbs.twimg.com/profile_images/1393146576055513088/g6hUgGsA_normal.png",
"screen_name": "TimDarcet",
"statuses_count": 134,
"translator_type": "none",
"verified": false,
"withheld_in_countries": [],
"id_str": "1371396662925606913"
},
"id": "1707769575981424866",
"conversation_id": "1707769575981424866",
"full_text": "Vision transformers need registers!\nOr at least, it seems they 𝘸𝘢𝘯𝘵 some…\nViTs have artifacts in attention maps. It’s due to the model using these patches as “registers”.\nJust add new tokens (“[reg]”):\n- no artifacts\n- interpretable attention maps 🦖\n- improved performances! https://t.co/mineuOjRid",
"reply_count": 22,
"retweet_count": 161,
"favorite_count": 1038,
"hashtags": [],
"symbols": [],
"user_mentions": [],
"urls": [],
"media": [
{
"media_url": "https://pbs.twimg.com/media/F7M2QmKWMAEJRKx.jpg",
"type": "photo"
}
],
"url": "https://twitter.com/TimDarcet/status/1707769575981424866",
"created_at": "2023-09-29T14:49:30.000Z",
"#sort_index": "1707769575981424866",
"view_count": 203504,
"quote_count": 26,
"is_quote_tweet": false,
"is_retweet": false,
"is_pinned": false,
"is_truncated": false,
"startUrl": "https://twitter.com/timdarcet/status/1707769575981424866"
}