{
  "type": "tweet",
  "id": "2067006779255619912",
  "url": "https://x.com/xdotli/status/2067006779255619912",
  "twitterUrl": "https://twitter.com/xdotli/status/2067006779255619912",
  "text": "A big pain point in using AI benchmarks is encountering errors after its first release. Today, we're releasing SkillsBench 1.1, the first benchmark for how well AI agents use skills, now audited end to end and verified error-free. Prof. @dawnsongtweets joins 1.1 as advising author.\n\nWe worked through every task with several frontier labs to eliminate the errors in the previous version. We also added new tasks, moved the ones with external dependencies into a separate set so the core suite runs clean, and expanded coverage to more models.\n\nCapability is climbing fast. The best with-skills resolution rate rose from ~36% (Claude Sonnet 4.5, Sep 2025) to 67% (GPT-5.5, May 2026), about +1.9 points per month. The frontier is hill-climbing SkillsBench fast.\n\nThe right skills still matter. Across the fleet, curated skills lift resolution rate by +16.6 points on average (33.9% → 50.5%), and by as much as +25.7 points for a single model. The top configuration is GPT-5.5 on OpenHands at 67.3%.\n\nBy popular demand (thx Nate @cursor_ai), we're now tracking skills invocation: how often an agent actually uses the skills it's given. Recent flagship configurations invoke them 90–99% of the time (Codex 99%, OpenHands + GPT-5.5 92%, Gemini CLI 90%), versus roughly 50% for older setups.\n\nAlso new in 1.1: @OpenHands joins as a fourth harness, alongside Claude Code, Codex, and Gemini CLI; a rebuilt leaderboard with refined categories, subdomain skill rankings, and Skill Lift; and native task . md on BenchFlow, with multi-scene environments and rollout branching. We also partnered with @k_dense_ai to add scientific skills to some science tasks.\n\nOne implication for deployment: skills can substitute for scale. GLM 5.1 with skills (58.4%) outperforms Opus 4.8 without (45.7%). A smaller model with the right procedural knowledge can beat a larger one running without it.\n\nHuge thanks to @nick_kango @ivanleomk @kaggle @GoogleDeepMind for hosting a launch event with us. Thanks for everyone who's come on May 27!\n\nAlso thanks to our partners @gneubig @OpenHandsDev @ivanburazin @daytonaio @jackminong @johannes_hage @PrimeIntellect @TimothyKassis @k_dense_ai for providing support in credits, compute, and skills.\n\nSkillsBench live leaderboard will also come to @ValsAI. Many people have told us they use SkillsBench as an index to measure models' agentic capability over diverse and high GDP value domains. Great work on Valkyrie as well! @ Jarett @nikilravi @langstonnashold @RayanKrishnan\n\nSkillsBench is fully open-source. Explore the leaderboard and tasks, read the docs, or contribute your own skill set or harness and join the leaderboard. 🧵",
  "source": "Twitter for iPhone",
  "retweetCount": 30,
  "replyCount": 13,
  "likeCount": 121,
  "quoteCount": 9,
  "viewCount": 16160,
  "createdAt": "Tue Jun 16 22:09:53 +0000 2026",
  "lang": "en",
  "bookmarkCount": 46,
  "isReply": false,
  "inReplyToId": null,
  "conversationId": "2067006779255619912",
  "displayTextRange": [
    0,
    274
  ],
  "inReplyToUserId": null,
  "inReplyToUsername": null,
  "author": {
    "type": "user",
    "userName": "xdotli",
    "url": "https://x.com/xdotli",
    "twitterUrl": "https://twitter.com/xdotli",
    "id": "1484803981050200067",
    "name": "Xiangyi Li",
    "isVerified": false,
    "isBlueVerified": true,
    "verifiedType": null,
    "profilePicture": "https://pbs.twimg.com/profile_images/1988139953508020224/n89FwQSW_normal.jpg",
    "coverPicture": "https://pbs.twimg.com/profile_banners/1484803981050200067/1774940833",
    "description": "",
    "location": "San Francisco",
    "followers": 5214,
    "following": 1535,
    "status": "",
    "canDm": true,
    "canMediaTag": true,
    "createdAt": "Sat Jan 22 08:23:57 +0000 2022",
    "entities": {
      "description": {
        "urls": []
      },
      "url": {}
    },
    "fastFollowersCount": 0,
    "favouritesCount": 58691,
    "hasCustomTimelines": true,
    "isTranslator": false,
    "mediaCount": 645,
    "statusesCount": 3279,
    "withheldInCountries": [],
    "affiliatesHighlightedLabel": {},
    "possiblySensitive": false,
    "pinnedTweetIds": [
      "2022409507826213048"
    ],
    "profile_bio": {
      "description": "your friendly neighborhood eval guy, creator SkillsBench ClawsBench @benchflow_ai\nchat about data, evals, rl environments, skills\nhttps://t.co/Jl1qzLItZn",
      "entities": {
        "description": {
          "urls": [
            {
              "display_url": "discord.gg/mZ9Rc8q8W3",
              "expanded_url": "https://discord.gg/mZ9Rc8q8W3",
              "indices": [
                130,
                153
              ],
              "url": "https://t.co/Jl1qzLItZn"
            }
          ],
          "user_mentions": [
            {
              "id_str": "",
              "indices": [
                68,
                81
              ],
              "name": "",
              "screen_name": "benchflow_ai"
            }
          ]
        },
        "url": {
          "urls": [
            {
              "display_url": "benchflow.ai",
              "expanded_url": "https://benchflow.ai/",
              "indices": [
                0,
                23
              ],
              "url": "https://t.co/kbe9qBhkw4"
            }
          ]
        }
      }
    },
    "isAutomated": false,
    "automatedBy": null
  },
  "extendedEntities": {
    "media": [
      {
        "display_url": "pic.twitter.com/IPSPHAePLk",
        "expanded_url": "https://twitter.com/xdotli/status/2067006779255619912/photo/1",
        "ext_master_playlist_only": [],
        "ext_media_availability": {
          "status": "Available"
        },
        "ext_playlists": [],
        "features": {
          "large": {
            "faces": []
          },
          "orig": {
            "faces": []
          }
        },
        "id_str": "2067006771215126528",
        "indices": [
          275,
          298
        ],
        "media_key": "3_2067006771215126528",
        "media_results": {
          "id": "QXBpTWVkaWFSZXN1bHRzOgwAAQoAARyve7T6mtAACgACHK97ttnbAUgAAA==",
          "result": {
            "__typename": "ApiMedia",
            "id": "QXBpTWVkaWE6DAABCgABHK97tPqa0AAKAAIcr3u22dsBSAAA",
            "media_key": "3_2067006771215126528"
          }
        },
        "media_url_https": "https://pbs.twimg.com/media/HK97tPqa0AA9C8l.jpg",
        "original_info": {
          "focus_rects": [
            {
              "h": 679,
              "w": 1212,
              "x": 0,
              "y": 0
            },
            {
              "h": 1212,
              "w": 1212,
              "x": 0,
              "y": 0
            },
            {
              "h": 1382,
              "w": 1212,
              "x": 0,
              "y": 0
            },
            {
              "h": 1862,
              "w": 931,
              "x": 281,
              "y": 0
            },
            {
              "h": 1862,
              "w": 1212,
              "x": 0,
              "y": 0
            }
          ],
          "height": 1862,
          "width": 1212
        },
        "sizes": {
          "large": {
            "h": 1862,
            "w": 1212
          }
        },
        "type": "photo",
        "url": "https://t.co/IPSPHAePLk"
      },
      {
        "display_url": "pic.twitter.com/IPSPHAePLk",
        "expanded_url": "https://twitter.com/xdotli/status/2067006779255619912/photo/1",
        "ext_master_playlist_only": [],
        "ext_media_availability": {
          "status": "Available"
        },
        "ext_playlists": [],
        "features": {
          "large": {
            "faces": []
          },
          "orig": {
            "faces": []
          }
        },
        "id_str": "2067006774880907264",
        "indices": [
          275,
          298
        ],
        "media_key": "3_2067006774880907264",
        "media_results": {
          "id": "QXBpTWVkaWFSZXN1bHRzOgwAAQoAARyve7XVGjAACgACHK97ttnbAUgAAA==",
          "result": {
            "__typename": "ApiMedia",
            "id": "QXBpTWVkaWE6DAABCgABHK97tdUaMAAKAAIcr3u22dsBSAAA",
            "media_key": "3_2067006774880907264"
          }
        },
        "media_url_https": "https://pbs.twimg.com/media/HK97tdUaMAArXid.jpg",
        "original_info": {
          "focus_rects": [
            {
              "h": 1413,
              "w": 2524,
              "x": 0,
              "y": 0
            },
            {
              "h": 2524,
              "w": 2524,
              "x": 0,
              "y": 0
            },
            {
              "h": 2524,
              "w": 2214,
              "x": 0,
              "y": 0
            },
            {
              "h": 2524,
              "w": 1262,
              "x": 0,
              "y": 0
            },
            {
              "h": 2524,
              "w": 2524,
              "x": 0,
              "y": 0
            }
          ],
          "height": 2524,
          "width": 2524
        },
        "sizes": {
          "large": {
            "h": 2048,
            "w": 2048
          }
        },
        "type": "photo",
        "url": "https://t.co/IPSPHAePLk"
      }
    ]
  },
  "card": null,
  "place": {},
  "entities": {
    "hashtags": [],
    "symbols": [],
    "urls": [],
    "user_mentions": [
      {
        "id_str": "535136727",
        "indices": [
          237,
          252
        ],
        "name": "Dawn Song",
        "screen_name": "dawnsongtweets"
      },
      {
        "id_str": "1695890961094909952",
        "indices": [
          1027,
          1037
        ],
        "name": "Cursor",
        "screen_name": "cursor_ai"
      },
      {
        "id_str": "1377882416677707776",
        "indices": [
          1305,
          1315
        ],
        "name": "Open hand solutions",
        "screen_name": "OpenHands"
      },
      {
        "id_str": "1984288934453731328",
        "indices": [
          1589,
          1600
        ],
        "name": "K-Dense",
        "screen_name": "k_dense_ai"
      },
      {
        "id_str": "1977818020690223105",
        "indices": [
          1891,
          1902
        ],
        "name": "Nick",
        "screen_name": "nick_kango"
      },
      {
        "id_str": "267160799",
        "indices": [
          1903,
          1913
        ],
        "name": "Ivan Leo",
        "screen_name": "ivanleomk"
      },
      {
        "id_str": "80422885",
        "indices": [
          1914,
          1921
        ],
        "name": "Kaggle",
        "screen_name": "kaggle"
      },
      {
        "id_str": "4783690002",
        "indices": [
          1922,
          1937
        ],
        "name": "Google DeepMind",
        "screen_name": "GoogleDeepMind"
      },
      {
        "id_str": "185910194",
        "indices": [
          2045,
          2053
        ],
        "name": "Graham Neubig",
        "screen_name": "gneubig"
      },
      {
        "id_str": "1793978852919885824",
        "indices": [
          2054,
          2067
        ],
        "name": "OpenHands",
        "screen_name": "OpenHandsDev"
      },
      {
        "id_str": "19487774",
        "indices": [
          2068,
          2080
        ],
        "name": "Ivan Burazin",
        "screen_name": "ivanburazin"
      },
      {
        "id_str": "1645813553365139462",
        "indices": [
          2081,
          2091
        ],
        "name": "Daytona",
        "screen_name": "daytonaio"
      },
      {
        "id_str": "1452187154118025218",
        "indices": [
          2092,
          2103
        ],
        "name": "Jackmin",
        "screen_name": "jackminong"
      },
      {
        "id_str": "844885422803243008",
        "indices": [
          2104,
          2118
        ],
        "name": "Johannes Hagemann",
        "screen_name": "johannes_hage"
      },
      {
        "id_str": "1274287728167915522",
        "indices": [
          2119,
          2134
        ],
        "name": "Prime Intellect",
        "screen_name": "PrimeIntellect"
      },
      {
        "id_str": "1339327739900403715",
        "indices": [
          2135,
          2149
        ],
        "name": "Timothy Kassis",
        "screen_name": "TimothyKassis"
      },
      {
        "id_str": "1984288934453731328",
        "indices": [
          2150,
          2161
        ],
        "name": "K-Dense",
        "screen_name": "k_dense_ai"
      },
      {
        "id_str": "1765633660588576768",
        "indices": [
          2265,
          2272
        ],
        "name": "Vals AI",
        "screen_name": "ValsAI"
      },
      {
        "id_str": "1437304583374741507",
        "indices": [
          2452,
          2462
        ],
        "name": "Nikil Ravi",
        "screen_name": "nikilravi"
      },
      {
        "id_str": "1778565825102057472",
        "indices": [
          2463,
          2479
        ],
        "name": "Langston Nashold",
        "screen_name": "langstonnashold"
      },
      {
        "id_str": "1117951640940597248",
        "indices": [
          2480,
          2494
        ],
        "name": "Rayan Krishnan",
        "screen_name": "RayanKrishnan"
      }
    ]
  },
  "quoted_tweet": null,
  "retweeted_tweet": null,
  "isLimitedReply": false,
  "communityInfo": null,
  "article": null
}