@ylecun
A piece of neural net hardware history: Edi Säckinger with Larry Jackel. Edi is holding a VME board with two Bell Labs ANNA chips (with golden plates on the left side). The ANNA chip is a mixed analog-digital chip specifically designed to run convolutional nets efficiently. It was designed in 1989 by Edi and Bernhard Boser. ANNA had 64 neurons with 64 inputs each. All 4096 weights could perform multiply-accumulate operations simultaneously at 1MHz, i.e. 4 billion operations per second (unheard of performance at the time). The weights were stored as voltages on capacitors, refreshed from external RAM with 6 bit depth. The activations were digital with 4 bit depth. The multipliers were essentially multiplying DACs and the accumulation was just currents on a wire. Shift registers allowed for efficient convolutions with minimal external memory traffic. This could run LeNet at 10,000 characters per second. The picture was taken earlier this year, but the chip was designed in 1989, and the board was built a year later. Papers: Chip: https://t.co/NoUyPVEBMd Board: https://t.co/MO1dEUXnZT