{
  "user": {
    "created_at": "2014-12-24T20:14:33.000Z",
    "default_profile_image": false,
    "description": "Figuring out AI @allen_ai, \"the RLHF person\"\nDM me papers. Writes @interconnectsai, talks @retortai\nHas phd and some credentials",
    "fast_followers_count": 0,
    "favourites_count": 22442,
    "followers_count": 23161,
    "friends_count": 660,
    "has_custom_timelines": true,
    "is_translator": false,
    "listed_count": 407,
    "location": "Berkeley, CA",
    "media_count": 443,
    "name": "Nathan Lambert",
    "normal_followers_count": 23161,
    "possibly_sensitive": false,
    "profile_banner_url": "https://pbs.twimg.com/profile_banners/2939913921/1706802626",
    "profile_image_url_https": "https://pbs.twimg.com/profile_images/1732079679610425344/YqSwiBqA_normal.jpg",
    "screen_name": "natolambert",
    "statuses_count": 5376,
    "translator_type": "none",
    "url": "https://t.co/hLflF9uMnT",
    "verified": true,
    "withheld_in_countries": [],
    "id_str": "2939913921"
  },
  "id": "1759970827053994132",
  "conversation_id": "1759970827053994132",
  "full_text": "A few big papers throwing question on \"does RLAIF work\" yesterday.\n\nThe first is a paper by @archit_sharma97 is a pretty timely critique of RLAIF. It shows SFT on GPT 4 outputs > DPO + RLAIF on GPT4 ratings of GPT3.5 completions. A few things aren't surprising:\n1. The most important thing you can do right now is have better completions going into fine-tuning, rather than worrying about algorithm\n2. The second best thing is worrying about prompts, which us RLHFers don't do. ShareGPT and similar things without filtering are bad. Preference optimization is likely more sensitive than SFT, given the newer optimizers.\n\nThere's still lots more headroom in improving RLAIF imo than SFT. Looking at popular datasets like Nectar or Ultrafeedback there are so many bugs that can be filtered by heuristics.\n\nWe also got HELM-Instruct, which showed wacky results on different evaluators. All of Amazon MTurk, Scale AI, GPT4, and Claude aren't very correlated. Seeing this, I thought training a model with the evaluator you use is prolly needed, they may not transfer too.\n\nSo, while GPT4-as-a-judge is popular, it has a long way to go. Finally, in the appendix of @Muennighoff 's paper GRIT was more uncertainty over how we use GPT4 as judgments for MT Bench. \nArchit's paper: https://t.co/QeXRzseLCZ\nHELM-Instruct: https://t.co/wODR3Srx44\nGRIT paper: https://t.co/afm4c0hVt0",
  "reply_count": 7,
  "retweet_count": 40,
  "favorite_count": 169,
  "hashtags": [],
  "symbols": [],
  "user_mentions": [
    {
      "id_str": "3269288695",
      "name": "Archit Sharma",
      "screen_name": "archit_sharma97",
      "profile": "https://twitter.com/archit_sharma97"
    }
  ],
  "urls": [],
  "media": [
    {
      "media_url": "https://pbs.twimg.com/media/GGypPUjbQAEBNoE.jpg",
      "type": "photo"
    },
    {
      "media_url": "https://pbs.twimg.com/media/GGyrOYJakAADRmf.jpg",
      "type": "photo"
    },
    {
      "media_url": "https://pbs.twimg.com/media/GGyrgEQbIAAnzV_.jpg",
      "type": "photo"
    }
  ],
  "url": "https://twitter.com/natolambert/status/1759970827053994132",
  "created_at": "2024-02-20T15:58:38.000Z",
  "#sort_index": "1759970827053994132",
  "view_count": 25690,
  "quote_count": 4,
  "is_quote_tweet": false,
  "is_retweet": false,
  "is_pinned": false,
  "is_truncated": true,
  "startUrl": "https://twitter.com/natolambert/status/1759970827053994132"
}