@LiorOnAI
Turns out, LLMs represent numbers on a helix and use trigonometry to do addition. A new paper reverse engineers addition in models like GPT-J-6B and finds a “Clock” algorithm. Numbers are encoded using sine and cosine terms, then added like angles. https://t.co/Ru4jkYNddl