Viewing enriched Twitter post
Eight years later, Yann LeCun’s cake 🍰 analogy was spot on: self-supervised > supervised > RL > “If intelligence is a cake, the bulk of the cake is unsupervised learning, the icing on the cake is supervised learning, and the cherry on the cake is reinforcement learning (RL).” https://t.co/ZmCvC7UOlk
{
"user": {
"created_at": "2009-04-25T01:51:11.000Z",
"default_profile_image": false,
"description": "ML, RecSys, LLMs @ Amazon; prev led ML at Alibaba, startups.\nWriting: https://t.co/DEUfIuYC47. Building: https://t.co/jJRZ8MOSnj.",
"fast_followers_count": 0,
"favourites_count": 14042,
"followers_count": 21333,
"friends_count": 507,
"has_custom_timelines": true,
"is_translator": false,
"listed_count": 580,
"location": "",
"media_count": 486,
"name": "Eugene Yan",
"normal_followers_count": 21333,
"possibly_sensitive": false,
"profile_banner_url": "https://pbs.twimg.com/profile_banners/35109534/1733079930",
"profile_image_url_https": "https://pbs.twimg.com/profile_images/1651817216629960706/lw4ZsN8u_normal.jpg",
"screen_name": "eugeneyan",
"statuses_count": 4229,
"translator_type": "none",
"url": "https://t.co/3olGmmyOZB",
"verified": true,
"withheld_in_countries": [],
"id_str": "35109534"
},
"id": "1858178714971828647",
"conversation_id": "1858178714971828647",
"full_text": "Eight years later, Yann LeCun’s cake 🍰 analogy was spot on: self-supervised > supervised > RL\n\n> “If intelligence is a cake, the bulk of the cake is unsupervised learning, the icing on the cake is supervised learning, and the cherry on the cake is reinforcement learning (RL).” https://t.co/ZmCvC7UOlk",
"reply_count": 4,
"retweet_count": 37,
"favorite_count": 338,
"hashtags": [],
"symbols": [],
"user_mentions": [],
"urls": [],
"media": [
{
"media_url": "https://pbs.twimg.com/media/GcmSdL_WcAA1gP4.jpg",
"type": "photo"
}
],
"url": "https://twitter.com/eugeneyan/status/1858178714971828647",
"created_at": "2024-11-17T16:01:43.000Z",
"#sort_index": "1858178714971828647",
"view_count": 42608,
"quote_count": 7,
"is_quote_tweet": true,
"is_retweet": false,
"is_pinned": false,
"is_truncated": false,
"quoted_tweet": {
"user": {
"created_at": "2009-04-21T06:49:15.000Z",
"default_profile_image": false,
"description": "Building @EurekaLabsAI. Previously Director of AI @ Tesla, founding team @ OpenAI, CS231n/PhD @ Stanford. I like to train large deep neural nets 🧠🤖💥",
"fast_followers_count": 0,
"favourites_count": 15184,
"followers_count": 1108887,
"friends_count": 948,
"has_custom_timelines": true,
"is_translator": false,
"listed_count": 14081,
"location": "Stanford",
"media_count": 752,
"name": "Andrej Karpathy",
"normal_followers_count": 1108887,
"possibly_sensitive": false,
"profile_banner_url": "https://pbs.twimg.com/profile_banners/33836629/1407117611",
"profile_image_url_https": "https://pbs.twimg.com/profile_images/1296667294148382721/9Pr6XrPB_normal.jpg",
"screen_name": "karpathy",
"statuses_count": 9215,
"translator_type": "none",
"url": "https://t.co/0EcFthjJXM",
"verified": true,
"withheld_in_countries": [],
"id_str": "33836629"
},
"id": "1857980896776990830",
"conversation_id": "1857916967157379248",
"full_text": "It’s hard to understand now, the Atari RL paper of 2013 and its extensions was the by far dominant meme. One single general learning algorithm discovered an optimal strategy to Breakout and so many other games. You just had to improve and scale it enough. My recollection of the memetics is that Yann LeCun was one prominent person who really didn’t care much and talked about the cake over and over again, where RL was just the final cherry on top with representation learning as the meat and supervised learning the icing, and he was conceptually exactly right about that at least with today’s stack and hindsight (pretraining = meat, SFT = icing, RLHF = cherry, ie the basic ChatGPT training pipeline). Which is fun because today he really doesn’t care much for LLMs either. (But for reasons that I tbh don’t always fully follow.)",
"reply_count": 20,
"retweet_count": 42,
"favorite_count": 644,
"hashtags": [],
"symbols": [],
"user_mentions": [
{
"id_str": "332196424",
"name": "JB Rubinovitz",
"screen_name": "rubinovitz",
"profile": "https://twitter.com/rubinovitz"
},
{
"id_str": "925453672812650496",
"name": "joshwa",
"screen_name": "BldrInvstTech",
"profile": "https://twitter.com/BldrInvstTech"
}
],
"urls": [],
"media": [],
"url": "https://twitter.com/karpathy/status/1857980896776990830",
"created_at": "2024-11-17T02:55:40.000Z",
"#sort_index": "1858178714971828700",
"view_count": 86307,
"quote_count": 8,
"is_quote_tweet": false,
"replying_to_tweet": "https://twitter.com/rubinovitz/status/1857976034299134324",
"is_retweet": false,
"is_pinned": false,
"is_truncated": true
},
"startUrl": "https://x.com/eugeneyan/status/1858178714971828647"
}