🐦 Twitter Post Details

Viewing enriched Twitter post

@seb_ruder

There is a huge amount of variety in this research area spanning - when sparse attention is used (prefilling vs decoding) - which units are sparsified (blocks or vertical slashes) - what type of patterns are used (fixed or content-aware) - how the computational budget is distributed (uniform or adaptive) - how the KV cache is managed (evicted or fully maintained) - for evals, how difficult information is to locate in the text (high or low dispersion) - how much information methods need to capture (high or low scope) - what type of data is used (natural or synthetic) I learned a lot throughout this study and from my collaborators. I hope our study can help others deepen their understanding and enable them to do exciting work in this space.

Media 1
Media 2
View on Twitter

📊 Media Metadata

{
  "media": [
    {
      "url": "https://crmoxkoizveukayfjuyo.supabase.co/storage/v1/object/public/media/posts/1916857006352216355/media_0.jpg?",
      "media_url": "https://crmoxkoizveukayfjuyo.supabase.co/storage/v1/object/public/media/posts/1916857006352216355/media_0.jpg?",
      "type": "photo",
      "filename": "media_0.jpg"
    },
    {
      "url": "https://crmoxkoizveukayfjuyo.supabase.co/storage/v1/object/public/media/posts/1916857006352216355/media_1.jpg?",
      "media_url": "https://crmoxkoizveukayfjuyo.supabase.co/storage/v1/object/public/media/posts/1916857006352216355/media_1.jpg?",
      "type": "photo",
      "filename": "media_1.jpg"
    }
  ],
  "downloaded_to_supabase": true,
  "processed_at": "2025-08-14T07:00:00Z"
}

🔧 Raw API Response

{
  "type": "tweet",
  "id": "1916857006352216355",
  "url": "https://x.com/seb_ruder/status/1916857006352216355",
  "twitterUrl": "https://twitter.com/seb_ruder/status/1916857006352216355",
  "text": "There is a huge amount of variety in this research area spanning\n- when sparse attention is used (prefilling vs decoding)\n- which units are sparsified (blocks or vertical slashes)\n- what type of patterns are used (fixed or content-aware)\n- how the computational budget is distributed (uniform or adaptive)\n- how the KV cache is managed (evicted or fully maintained)\n- for evals, how difficult information is to locate in the text (high or low dispersion)\n- how much information methods need to capture (high or low scope)\n- what type of data is used (natural or synthetic)\n\nI learned a lot throughout this study and from my collaborators. I hope our study can help others deepen their understanding and enable them to do exciting work in this space.",
  "source": "Twitter for iPhone",
  "retweetCount": 0,
  "replyCount": 2,
  "likeCount": 5,
  "quoteCount": 0,
  "viewCount": 3210,
  "createdAt": "Mon Apr 28 14:08:18 +0000 2025",
  "lang": "en",
  "bookmarkCount": 4,
  "isReply": true,
  "inReplyToId": "1916854423533326566",
  "conversationId": "1916850948741239173",
  "inReplyToUserId": "2785337469",
  "inReplyToUsername": "seb_ruder",
  "author": {
    "type": "user",
    "userName": "seb_ruder",
    "url": "https://x.com/seb_ruder",
    "twitterUrl": "https://twitter.com/seb_ruder",
    "id": "2785337469",
    "name": "Sebastian Ruder @ ACL",
    "isVerified": false,
    "isBlueVerified": true,
    "verifiedType": null,
    "profilePicture": "https://pbs.twimg.com/profile_images/1100429844830146560/mgKi3vLm_normal.jpg",
    "coverPicture": "https://pbs.twimg.com/profile_banners/2785337469/1438539641",
    "description": "",
    "location": "Berlin, Deutschland",
    "followers": 91963,
    "following": 1458,
    "status": "",
    "canDm": false,
    "canMediaTag": true,
    "createdAt": "Fri Sep 26 12:47:34 +0000 2014",
    "entities": {
      "description": {
        "urls": []
      },
      "url": {}
    },
    "fastFollowersCount": 0,
    "favouritesCount": 9896,
    "hasCustomTimelines": true,
    "isTranslator": true,
    "mediaCount": 380,
    "statusesCount": 4263,
    "withheldInCountries": [],
    "affiliatesHighlightedLabel": {},
    "possiblySensitive": false,
    "pinnedTweetIds": [],
    "profile_bio": {
      "description": "Research Scientist @AIatMeta • Ex @Cohere @GoogleDeepMind",
      "entities": {
        "description": {
          "user_mentions": [
            {
              "id_str": "0",
              "indices": [
                19,
                28
              ],
              "name": "",
              "screen_name": "AIatMeta"
            },
            {
              "id_str": "0",
              "indices": [
                34,
                41
              ],
              "name": "",
              "screen_name": "Cohere"
            },
            {
              "id_str": "0",
              "indices": [
                42,
                57
              ],
              "name": "",
              "screen_name": "GoogleDeepMind"
            }
          ]
        },
        "url": {
          "urls": [
            {
              "display_url": "ruder.io",
              "expanded_url": "http://www.ruder.io",
              "indices": [
                0,
                23
              ],
              "url": "https://t.co/1GdhVCfGj9"
            }
          ]
        }
      }
    },
    "isAutomated": false,
    "automatedBy": null
  },
  "extendedEntities": {
    "media": [
      {
        "allow_download_status": {
          "allow_download": true
        },
        "display_url": "pic.twitter.com/SQMXmXnyRv",
        "expanded_url": "https://twitter.com/seb_ruder/status/1916857006352216355/photo/1",
        "ext_media_availability": {
          "status": "Available"
        },
        "features": {
          "large": {},
          "orig": {}
        },
        "id_str": "1916856573596508160",
        "indices": [
          307,
          330
        ],
        "media_key": "3_1916856573596508160",
        "media_results": {
          "id": "QXBpTWVkaWFSZXN1bHRzOgwAAQoAARqaCuXSl6AACgACGpoLSpTXsSMAAA==",
          "result": {
            "__typename": "ApiMedia",
            "id": "QXBpTWVkaWE6DAABCgABGpoK5dKXoAAKAAIamgtKlNexIwAA",
            "media_key": "3_1916856573596508160"
          }
        },
        "media_url_https": "https://pbs.twimg.com/media/GpoK5dKXoAAwgOv.png",
        "original_info": {
          "focus_rects": [
            {
              "h": 756,
              "w": 1350,
              "x": 928,
              "y": 0
            },
            {
              "h": 756,
              "w": 756,
              "x": 1225,
              "y": 0
            },
            {
              "h": 756,
              "w": 663,
              "x": 1272,
              "y": 0
            },
            {
              "h": 756,
              "w": 378,
              "x": 1414,
              "y": 0
            },
            {
              "h": 756,
              "w": 2566,
              "x": 0,
              "y": 0
            }
          ],
          "height": 756,
          "width": 2566
        },
        "sizes": {
          "large": {
            "h": 603,
            "w": 2048
          }
        },
        "type": "photo",
        "url": "https://t.co/SQMXmXnyRv"
      },
      {
        "allow_download_status": {
          "allow_download": true
        },
        "display_url": "pic.twitter.com/SQMXmXnyRv",
        "expanded_url": "https://twitter.com/seb_ruder/status/1916857006352216355/photo/1",
        "ext_media_availability": {
          "status": "Available"
        },
        "features": {
          "large": {},
          "orig": {}
        },
        "id_str": "1916856589958496256",
        "indices": [
          307,
          330
        ],
        "media_key": "3_1916856589958496256",
        "media_results": {
          "id": "QXBpTWVkaWFSZXN1bHRzOgwAAQoAARqaCumh18AACgACGpoLSpTXsSMAAA==",
          "result": {
            "__typename": "ApiMedia",
            "id": "QXBpTWVkaWE6DAABCgABGpoK6aHXwAAKAAIamgtKlNexIwAA",
            "media_key": "3_1916856589958496256"
          }
        },
        "media_url_https": "https://pbs.twimg.com/media/GpoK6aHXwAAYaDB.png",
        "original_info": {
          "focus_rects": [
            {
              "h": 1164,
              "w": 2078,
              "x": 0,
              "y": 0
            },
            {
              "h": 1384,
              "w": 1384,
              "x": 0,
              "y": 0
            },
            {
              "h": 1384,
              "w": 1214,
              "x": 0,
              "y": 0
            },
            {
              "h": 1384,
              "w": 692,
              "x": 0,
              "y": 0
            },
            {
              "h": 1384,
              "w": 2078,
              "x": 0,
              "y": 0
            }
          ],
          "height": 1384,
          "width": 2078
        },
        "sizes": {
          "large": {
            "h": 1364,
            "w": 2048
          }
        },
        "type": "photo",
        "url": "https://t.co/SQMXmXnyRv"
      }
    ]
  },
  "card": null,
  "place": {},
  "entities": {},
  "quoted_tweet": null,
  "retweeted_tweet": null,
  "article": null
}