@teortaxesTex
People are so used to models trained for Huggingface Leaderboard they're in disbelief upon seeing a production-grade one. Maybe they shouldn't. Smol Qwens are samples of Tongyi Qianwen, not proofs of concept; to Alibaba, they're kind of like what T5-XXL is to Google. Alibaba is a world-class corporation actually trying, no BS, to capture the vast Chinese LLM assistant market. That's why this report talks so much about practical aspects and objectives they pursued, not only muh MMLU/HEval scores (and even with HEval, they go for the state-of-the-art HumanEvalPack benchmark). This paper, incomplete though it may be (it's particularly secretive about the dataset, understandably evoking extra suspicion), is a treasure trove of insight into almost-frontier proprietary LLMs. This is something like what we should've expected to see if @karpathy got his way and OpenAI published that small open-source model to teach the community a little share of their tricks. https://t.co/sQREvJ6dzz In the realm of LLaMA finetunes, @gigaml 's X1-Large and probably @XLangAI Lemur are comparable, but we know so much less about them. X1 is genuinely superior to LLama2-70B across the board, which is more than I can say for all the fancy imitative finetunes. As @iamgingertrash would probably argue, this is the difference in incentives.