{
  "user": {
    "created_at": "2018-11-30T04:28:00.000Z",
    "default_profile_image": false,
    "description": "Toolmaker. Software creator, optimizer and harmonizer. \n\nMakes ML systems work and fly.\n\nLLM Training and Inference/Generative AI/Scalability/Troubleshooting",
    "fast_followers_count": 0,
    "favourites_count": 2283,
    "followers_count": 8289,
    "friends_count": 278,
    "has_custom_timelines": false,
    "is_translator": false,
    "listed_count": 181,
    "location": "Nanaimo, BC, Canada",
    "media_count": 200,
    "name": "Stas Bekman",
    "normal_followers_count": 8289,
    "possibly_sensitive": false,
    "profile_image_url_https": "https://pbs.twimg.com/profile_images/1068362113205231616/kJyKU2F5_normal.jpg",
    "screen_name": "StasBekman",
    "statuses_count": 2038,
    "translator_type": "none",
    "url": "https://t.co/Hqj5wVj1xv",
    "verified": true,
    "withheld_in_countries": [],
    "id_str": "1068360975898660864"
  },
  "id": "1854233704572612814",
  "conversation_id": "1854233704572612814",
  "full_text": "If you're evaluating which accelerator to choose for your future workloads beware that AMD's peer-to-peer bandwidth is 7x slower than all-to-all bandwidth.\n\nAs long as you use full nodes or single gpus, you have nothing to worry about. But if you have to deploy TP=2, TP=4, or ZeRO-DP/FSDP over 2 or 4 gpus, be it training or inference, the network will become a bottleneck.\n\nTo validate this the all_reduce_bench.py benchmark  was run on a 8 gpu AMD MI300X node with a 4GB payload and the `busbw` measurements were:\n\n- 2 gpus:  47.671 GBps\n- 8 gpus:  312.912 GBps\n\ni.e. 2 gpus performed 6.5x slower than 8.\n\nI have created a new table specific for peer-to-peer bw:\nhttps://t.co/Nnr7ria6pu\n\nI have also recently talked to AMD engineers and was told that AMD is planning on improving this situation in future offerings. So we need to be patient and let AMD catch up. It's great that we have competition!\n\nThanks a lot to https://t.co/j5Ba1bVZGs for the peer-to-peer insight and running the all_reduce benchmark for me.",
  "reply_count": 7,
  "retweet_count": 29,
  "favorite_count": 145,
  "hashtags": [],
  "symbols": [],
  "user_mentions": [],
  "urls": [],
  "media": [
    {
      "media_url": "https://pbs.twimg.com/media/GbuNQvAbYAAi_Lz.png",
      "type": "photo"
    }
  ],
  "url": "https://twitter.com/StasBekman/status/1854233704572612814",
  "created_at": "2024-11-06T18:45:39.000Z",
  "#sort_index": "1854233704572612814",
  "view_count": 19009,
  "quote_count": 1,
  "is_quote_tweet": false,
  "is_retweet": false,
  "is_pinned": false,
  "is_truncated": true,
  "startUrl": "https://x.com/stasbekman/status/1854233704572612814"
}