{
  "type": "tweet",
  "id": "1966527903568637972",
  "url": "https://x.com/ShashwatGoel7/status/1966527903568637972",
  "twitterUrl": "https://twitter.com/ShashwatGoel7/status/1966527903568637972",
  "text": "Paper fresh of the press: The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMs.\n\nAre small models the future of agentic AI? Is scaling LLM compute not worth the cost due to diminishing returns? Are autoregressive LLMs doomed, and thinking an illusion? \n\nThe bear cases for LLM scaling are all connected to a single capability: Long Horizon Execution. However, thats exactly why you should be bullish on scaling model size, and test-time compute!\n\n> First, remember the METR plot? It might be explained by @ylecun 's model of compounding errors\n> the horizon length of a model grows super-exponentially (@DaveShapi) in single step accuracy.\n> Upshot 1: Don't be fooled by slowing progress on typical short-task benchmarks\n> that is enough for exponential growth in horizon length.\n\nBut we go beyond @ylecun's model, testing LLMs empirically...\n\n> Just execution is also hard for LLMs, even when you provide them the needed plan and knowledge.\n> We should not misinterpret execution failures as an inability to \"reason\".\n> Even when a small model has 100% single-step accuracy, larger models can execute far more turns above a success rate threshold.\n\n> Noticed how your agent performs worse as the task gets longer?  Its not just long-context limitations..\n> We observe: The Self-Conditioning Effect!\n> When models see errors they made earlier in their history, they become more likely to make errors in future turns.\n> Increasing model size worsens this problem - a rare case of inverse scaling!\n\nSo what about thinking...?\n\n> Thinking is not an illusion. It is the engine for execution!\n> Where even DeepSeek v3, Kimi K2 fail to execute even 5 turns latently when asked to execute without CoT...\n> With CoT, they can do 10x more.\n\nSo what about the frontier?\n\n> GPT-5 Thinking is far ahead of all other models we tested. It can execute 1000+ step tasks in one go.\n> At second with 432 steps is Claude 4 Sonnet... and then Grok-4 at 384\n> Gemini 2.5 Pro and DeepSeek R1 lag far behind, at just 120.\n\n> Is that why GPT-5 was codenamed Horizon? 🤔\n> Open-source has a long ;) way to go!\n> Let's grow it together! We release all code and data.\n\nWe did a longggg deep dive, and present you the best takeaways with awesome plots below 👇",
  "source": "Twitter for iPhone",
  "retweetCount": 121,
  "replyCount": 27,
  "likeCount": 655,
  "quoteCount": 22,
  "viewCount": 232550,
  "createdAt": "Fri Sep 12 15:42:43 +0000 2025",
  "lang": "en",
  "bookmarkCount": 572,
  "isReply": false,
  "inReplyToId": null,
  "conversationId": "1966527903568637972",
  "displayTextRange": [
    0,
    277
  ],
  "inReplyToUserId": null,
  "inReplyToUsername": null,
  "author": {
    "type": "user",
    "userName": "ShashwatGoel7",
    "url": "https://x.com/ShashwatGoel7",
    "twitterUrl": "https://twitter.com/ShashwatGoel7",
    "id": "1277988007304261632",
    "name": "Shashwat Goel",
    "isVerified": false,
    "isBlueVerified": true,
    "verifiedType": null,
    "profilePicture": "https://pbs.twimg.com/profile_images/1799099355502419968/F2ztQFj8_normal.jpg",
    "coverPicture": "",
    "description": "",
    "location": "London, United Kingdom",
    "followers": 1986,
    "following": 986,
    "status": "",
    "canDm": true,
    "canMediaTag": true,
    "createdAt": "Tue Jun 30 15:31:10 +0000 2020",
    "entities": {
      "description": {
        "urls": []
      },
      "url": {}
    },
    "fastFollowersCount": 0,
    "favouritesCount": 5349,
    "hasCustomTimelines": true,
    "isTranslator": false,
    "mediaCount": 134,
    "statusesCount": 938,
    "withheldInCountries": [],
    "affiliatesHighlightedLabel": {},
    "possiblySensitive": false,
    "pinnedTweetIds": [
      "1966527903568637972"
    ],
    "profile_bio": {
      "description": "Scaling supervision for AI on evals that matter.\n\n👨‍🍳Forecasting, Long Horizon, Synth Data for RL\n\nRS Intern @AIatMeta\nPhDing @ELLISInst_Tue @MPI_IS",
      "entities": {
        "description": {
          "user_mentions": [
            {
              "id_str": "0",
              "indices": [
                109,
                118
              ],
              "name": "",
              "screen_name": "AIatMeta"
            },
            {
              "id_str": "0",
              "indices": [
                126,
                140
              ],
              "name": "",
              "screen_name": "ELLISInst_Tue"
            },
            {
              "id_str": "0",
              "indices": [
                141,
                148
              ],
              "name": "",
              "screen_name": "MPI_IS"
            }
          ]
        },
        "url": {
          "urls": [
            {
              "display_url": "shash42.github.io",
              "expanded_url": "http://shash42.github.io",
              "indices": [
                0,
                23
              ],
              "url": "https://t.co/aZsWKm82N0"
            }
          ]
        }
      }
    },
    "isAutomated": false,
    "automatedBy": null
  },
  "extendedEntities": {
    "media": [
      {
        "allow_download_status": {
          "allow_download": true
        },
        "display_url": "pic.twitter.com/BoyKCRNJP0",
        "expanded_url": "https://twitter.com/ShashwatGoel7/status/1966527903568637972/photo/1",
        "ext_media_availability": {
          "status": "Available"
        },
        "features": {
          "large": {},
          "orig": {}
        },
        "id_str": "1966527333084401664",
        "indices": [
          278,
          301
        ],
        "media_key": "3_1966527333084401664",
        "media_results": {
          "id": "QXBpTWVkaWFSZXN1bHRzOgwAAQoAARtKgjC5l5AACgACG0qCtY0aMBQAAA==",
          "result": {
            "__typename": "ApiMedia",
            "id": "QXBpTWVkaWE6DAABCgABG0qCMLmXkAAKAAIbSoK1jRowFAAA",
            "media_key": "3_1966527333084401664"
          }
        },
        "media_url_https": "https://pbs.twimg.com/media/G0qCMLmXkAAcjM4.jpg",
        "original_info": {
          "focus_rects": [
            {
              "h": 682,
              "w": 1218,
              "x": 0,
              "y": 0
            },
            {
              "h": 1218,
              "w": 1218,
              "x": 0,
              "y": 0
            },
            {
              "h": 1389,
              "w": 1218,
              "x": 0,
              "y": 0
            },
            {
              "h": 1442,
              "w": 721,
              "x": 324,
              "y": 0
            },
            {
              "h": 1442,
              "w": 1218,
              "x": 0,
              "y": 0
            }
          ],
          "height": 1442,
          "width": 1218
        },
        "sizes": {
          "large": {
            "h": 1442,
            "w": 1218
          }
        },
        "type": "photo",
        "url": "https://t.co/BoyKCRNJP0"
      }
    ]
  },
  "card": null,
  "place": {},
  "entities": {
    "user_mentions": [
      {
        "id_str": "48008938",
        "indices": [
          530,
          537
        ],
        "name": "Yann LeCun",
        "screen_name": "ylecun"
      },
      {
        "id_str": "1795781228399869952",
        "indices": [
          628,
          638
        ],
        "name": "David Shapiro ⏩",
        "screen_name": "DaveShapi"
      },
      {
        "id_str": "48008938",
        "indices": [
          823,
          830
        ],
        "name": "Yann LeCun",
        "screen_name": "ylecun"
      }
    ]
  },
  "quoted_tweet": null,
  "retweeted_tweet": null,
  "isLimitedReply": false,
  "article": null
}