@realJessyLin
๐ How do we teach an LLM to ๐ฎ๐ข๐ด๐ต๐ฆ๐ณ a body of knowledge? In new work with @AIatMeta, we propose Active Reading ๐: a way for models to teach themselves new things by self-studying their training data. Results: * ๐๐% on SimpleQA w/ an 8B model by studying the wikipedia docs (+๐๐๐% vs plain finetuning) * a domain-specific expert model: ๐๐๐% vs FT on FinanceBench knowledge * an 8B wikipedia expert competitive w/ 405B on factuality (๐ฅopen-sourced!) ๐งต[1/n]