How big artificial neural networks can we run now (either with full train-backprop cycle or just evaluating network outputs) if our total energy budget for computation is equivalent to the human brain energy budget (12.6 watts)?
Let assume one cycle per second, which seems to roughly match the firing rate of biological neurons.
126 million artificial neurons at 12.6 Watts, with IBM's True North
Back in 2014, IBM's True North chip was pushing 1 million neurons at less than 100mW.
So that's roughly 126 million artificial neurons at 12.6 Watts.
A mouse has 70 million neurons.
IBM believes they can build a human-brain scale True North mainframe at a "mere" 4kW.
Once 3D transistors come to market, I think we'll catch up to animal brain efficiency pretty fast.
If you limited yourself to 12.6 watts, you wouldn't get much done. Just lookup the power consumption for a modern GPU, look at the size networks people are training on those, and then scale down. For reference, modern GPU's appear to consume between 52-309 watts under heavy use.
Clearly energy efficiency is one area where the human brain is still far head of ANN's.
