@emollick
My first published academic paper was on Moore's Law and right now AI development looks similar: the exponential of Moore's Law was not the result of a single technology, but rather many different technologies over many decades that were ready when one chip-making approach faltered. The regular pace of the Law served as a coordinating function so that an ever-changing group of competitors were pressured to create a self-fulfilling prophecy of continual capability growth. Similarly, AI development has already hit a number of speedbumps that had to be overcome with new techniques and research (synthetic data approaches, reasoning, new uses for RL). But unless you are an insider (or follow AI closely on X), you don't see those speedbumps: just steady exponential progress. Given the amount of money and talent in the space, I expect that even if pre-training or whatever hits a wall, we will see a quick transition of the entire industry to one or more of the many other approaches people are developing. You can see this already: work on world models, alternatives to LLMs, new training methods, etc. Even alternative ecosystems that are betting on the rise of small, fine-tuned models, etc. Some of these techniques are coming from startups, others are being developed in the AI labs themselves. People on X tend to get into the details, treating AI like a sport, cheering for or against teams and approaches. But over any reasonable span of time, it is possible that AI development looks like a smooth exponential on many metrics to everyone else.
