@HelloSurgeAI
Introducing ComplexConstraints — a new IF benchmark designed to test whether models can handle the kinds of constraints that show up in real work: 1. Conditional constraints (fire only when specific conditions are met) 2. Planning constraints (many requirements satisfied simultaneously) 3. Multistep constraints (each step feeds the next) 4. Implicit constraints (a competent colleague would just know). Models currently score between 0% and 40%. Here's an example: imagine you're a film producer drafting next week's shooting schedule. Two actresses are only available Tuesday and Thursday. The exterior scenes need daylight, but the forecast calls for rain on Wednesday. Tom's stunt double doesn't arrive until Friday, and Chris and Sydney don't get along. There are twenty-six interdependent constraints, and missing any one is a failure. What's interesting is also what the data teaches. We trained a Qwen-4B model on 1K ComplexConstraints companion examples. It reached parity with a model 60x its size, and the gains transferred to other IF benchmarks like MultiChallenge and AdvancedIF. Single-turn data even generalized to multi-turn behaviors, because tracking simultaneous requirements without dropping the lower-priority ones is the same skill multi-turn IF tests. Read more! Blog post: https://t.co/Su8UQH5EZd Leaderboard: https://t.co/WdMCiKov7Z