@torchcompiled
Ok this is interesting, because this paper basically says the opposite. So long as 2% of your data is real, collapse is substantially mitigated. thoughts? feelings? https://t.co/JKGeAM7OT8
Viewing enriched Twitter post
Ok this is interesting, because this paper basically says the opposite. So long as 2% of your data is real, collapse is substantially mitigated. thoughts? feelings? https://t.co/JKGeAM7OT8
{
"user": {
"created_at": "2022-04-28T22:18:59.000Z",
"default_profile_image": false,
"description": "a boy and his gpu vs the world. cofounder/directing research at @leonardoai_. (now at @canva) trying to feel the magic.",
"fast_followers_count": 0,
"favourites_count": 24720,
"followers_count": 7618,
"friends_count": 783,
"has_custom_timelines": true,
"is_translator": false,
"listed_count": 144,
"location": "tampa - sydney",
"media_count": 2012,
"name": "Ethan (NeurIPS 🔜)",
"normal_followers_count": 7618,
"possibly_sensitive": false,
"profile_banner_url": "https://pbs.twimg.com/profile_banners/1519803319186825219/1724227722",
"profile_image_url_https": "https://pbs.twimg.com/profile_images/1848834193444466688/stRMnaB6_normal.jpg",
"screen_name": "torchcompiled",
"statuses_count": 10937,
"translator_type": "none",
"url": "https://t.co/lVEusSsQZr",
"verified": true,
"withheld_in_countries": [],
"id_str": "1519803319186825219"
},
"id": "1845607429914218502",
"conversation_id": "1845607429914218502",
"full_text": "Ok this is interesting, because this paper basically says the opposite.\nSo long as 2% of your data is real, collapse is substantially mitigated.\nthoughts? feelings? https://t.co/JKGeAM7OT8",
"reply_count": 10,
"retweet_count": 16,
"favorite_count": 210,
"hashtags": [],
"symbols": [],
"user_mentions": [],
"urls": [],
"media": [
{
"media_url": "https://pbs.twimg.com/media/GZzp-bVawAAr119.jpg",
"type": "photo"
},
{
"media_url": "https://pbs.twimg.com/media/GZzp-39a0AAYIQ_.png",
"type": "photo"
}
],
"url": "https://twitter.com/torchcompiled/status/1845607429914218502",
"created_at": "2024-10-13T23:27:55.000Z",
"#sort_index": "1845607429914218502",
"view_count": 30287,
"quote_count": 1,
"is_quote_tweet": true,
"is_retweet": false,
"is_pinned": false,
"is_truncated": false,
"quoted_tweet": {
"user": {
"created_at": "2016-05-16T14:01:31.000Z",
"default_profile_image": false,
"description": "Postdoc@UC Berkeley CS; Research: ML, NLP, AI Safety",
"fast_followers_count": 0,
"favourites_count": 887,
"followers_count": 1700,
"friends_count": 347,
"has_custom_timelines": false,
"is_translator": false,
"listed_count": 12,
"location": "Goleta, CA",
"media_count": 53,
"name": "Xuandong Zhao @ NeurIPS 2024",
"normal_followers_count": 1700,
"possibly_sensitive": false,
"profile_banner_url": "https://pbs.twimg.com/profile_banners/732209366812631041/1720926207",
"profile_image_url_https": "https://pbs.twimg.com/profile_images/1742771546715025408/5Q2KWvle_normal.jpg",
"screen_name": "xuandongzhao",
"statuses_count": 383,
"translator_type": "none",
"url": "https://t.co/5rXJZ1thxE",
"verified": true,
"withheld_in_countries": [],
"id_str": "732209366812631041"
},
"id": "1845330594185806117",
"conversation_id": "1845330594185806117",
"full_text": "🚨 Fascinating insights from the paper “Strong Model Collapse”! (https://t.co/WTg2dujtkE) It concludes that even the smallest fraction of synthetic data (as little as 1% of the total dataset) can lead to model collapse. 🧠 Larger datasets don’t necessarily improve performance! https://t.co/8ILPRQ5f8b",
"reply_count": 8,
"retweet_count": 64,
"favorite_count": 361,
"hashtags": [],
"symbols": [],
"user_mentions": [],
"urls": [
{
"url": "https://t.co/WTg2dujtkE",
"expanded_url": "https://arxiv.org/abs/2410.04840",
"display_url": "arxiv.org/abs/2410.04840"
}
],
"media": [
{
"media_url": "https://pbs.twimg.com/media/GZvs4gsaAAAYgTo.png",
"type": "photo"
}
],
"url": "https://twitter.com/xuandongzhao/status/1845330594185806117",
"created_at": "2024-10-13T05:07:52.000Z",
"#sort_index": "1845607429914218500",
"view_count": 66346,
"quote_count": 3,
"is_quote_tweet": false,
"is_retweet": false,
"is_pinned": false,
"is_truncated": false
},
"startUrl": "https://x.com/ethan_smith_20/status/1845607429914218502"
}