@AlphaSignalAI
The highly expected Mistral 7B is out. Yes, from the french startup with a $113M seed round. The model already outperforms Llama 2 13B on every benchmark. Features - Released under Apache 2.0 licence. - Superior to LLaMA 1 34B in code, math, and reasoning - Approaches CodeLlama 7B performance on code Usability - Usable anywhere (even locally) - Deployable on any cloud (AWS/GCP/Azure) - Usable on HuggingFace Architecture - Uses Grouped-query attention (GQA) for faster inference -Uses Sliding Window Attention (SWA) to handle longer sequences at smaller cost https://t.co/vrtvl0kIpX