🐦 Twitter Post Details

Viewing enriched Twitter post

@fchollet

If you haven't read the ARC Prize 2024 technical report, check it out (link in next tweet). One important bit: we'll be releasing a v2 of the benchmark early next year (human testing is currently being finalized). Why? Because AGI progress in 2025 is going to need a better compass than v1. v1 fulfilled its mission well over the past 5 years, but what we've learned from it enables us to ship something better. In 2020, an ensemble of all Kaggle submissions in that year's competition scored 49% -- and that was all crude program enumeration with relatively low compute. This signals that about half of the benchmark was not a strong signal towards AGI. Today, an ensemble of all Kaggle submissions in the 2024 competition is scoring 81%. This signals the benchmark is saturating, and that enough compute / brute force will get you over the finish line. v2 will fix these issues and will increase the "signal strength" of the benchmark.

View on Twitter

📊 Media Metadata

{
  "media": [
    {
      "id": "",
      "type": "photo",
      "url": null,
      "media_url": "https://pbs.twimg.com/media/GeThZofboAANiJS.jpg",
      "media_url_https": null,
      "display_url": null,
      "expanded_url": null
    }
  ],
  "nlp": {
    "sentiment": "positive",
    "processed_at": "2025-08-06T12:54:57.272353"
  },
  "original_structure": "had_media_only"
}

🔧 Raw API Response

{
  "user": {
    "created_at": "2009-08-25T17:09:25.000Z",
    "default_profile_image": false,
    "description": "Creator of Keras and ARC-AGI. Author of 'Deep Learning with Python'. Co-founder @arcprize.",
    "fast_followers_count": 0,
    "favourites_count": 9715,
    "followers_count": 513224,
    "friends_count": 796,
    "has_custom_timelines": true,
    "is_translator": false,
    "listed_count": 7564,
    "location": "United States",
    "media_count": 1349,
    "name": "François Chollet",
    "normal_followers_count": 513224,
    "possibly_sensitive": false,
    "profile_banner_url": "https://pbs.twimg.com/profile_banners/68746721/1463719109",
    "profile_image_url_https": "https://pbs.twimg.com/profile_images/1611009368765468673/lLWbGjjj_normal.jpg",
    "screen_name": "fchollet",
    "statuses_count": 23515,
    "translator_type": "none",
    "url": "https://t.co/6miFIZSFAQ",
    "verified": true,
    "withheld_in_countries": [],
    "id_str": "68746721"
  },
  "id": "1865865271728390515",
  "conversation_id": "1865865271728390515",
  "full_text": "If you haven't read the ARC Prize 2024 technical report, check it out (link in next tweet).\n\nOne important bit: we'll be releasing a v2 of the benchmark early next year (human testing is currently being finalized).\n\nWhy? Because AGI progress in 2025 is going to need a better compass than v1. v1 fulfilled its mission well over the past 5 years, but what we've learned from it enables us to ship something better.\n\nIn 2020, an ensemble of all Kaggle submissions in that year's competition scored 49% -- and that was all crude program enumeration with relatively low compute. This signals that about half of the benchmark was not a strong signal towards AGI.\n\nToday, an ensemble of all Kaggle submissions in the 2024 competition is scoring 81%. This signals the benchmark is saturating, and that enough compute / brute force will get you over the finish line.\n\nv2 will fix these issues and will increase the \"signal strength\" of the benchmark.",
  "reply_count": 24,
  "retweet_count": 71,
  "favorite_count": 613,
  "hashtags": [],
  "symbols": [],
  "user_mentions": [],
  "urls": [],
  "media": [
    {
      "media_url": "https://pbs.twimg.com/media/GeThZofboAANiJS.jpg",
      "type": "photo"
    }
  ],
  "url": "https://twitter.com/fchollet/status/1865865271728390515",
  "created_at": "2024-12-08T21:05:21.000Z",
  "#sort_index": "1865865271728390515",
  "view_count": 54694,
  "quote_count": 9,
  "is_quote_tweet": false,
  "is_retweet": false,
  "is_pinned": false,
  "is_truncated": true,
  "startUrl": "https://x.com/fchollet/status/1865865271728390515"
}