Toward ‘Biologically Plausible’ AI

In the latest attempt at understanding the mechanism(s) by which machines learn, and AI researcher and a neuroscientist probed for similarities in the computational properties between deep neural networks and human brain.

They propose a learning algorithm that overcomes a “particularly nonbiological aspect” of deep learning: the current supervised training process requiring huge amounts of labeled data and non-local learning. They say the framework also addresses the need for a “non-local learning rule,” that is, an information requirement about the states of other specific neurons in the network in addition to the two neurons that are connected by the given synapse.

Their paper, titled “Unsupervised learning by competing hidden units,” was published at the end of March in the Proceedings of the National Academy of Sciences. It was authored by Dmitry Krotov of the MIT-IBM Watson AI Lab and John Hopfield of Princeton University’s Neuroscience Institute.

The authors described a “biologically motivated” learning algorithm operating in a “completely unsupervised fashion” using only local learning rules. That approach possesses “conceptual biological plausibility,” the researchers asserted.

Tested against standard machine learning benchmarks, they report the unsupervised learning algorithm was able to sort through raw, unlabeled data, performing on a level comparable to a neural network trained with a back-propagation algorithm on labeled data. (Back propagation of data is widely viewed as a better way to simulate how the brain processes information.)

“This is an encouraging result, since the algorithm was able to find a good solution to a task without knowing what that task is, suggesting the possibility of learning sufficiently general representations of the data that might be used for an arbitrary task,” the researchers noted.

Recent advances in deep neural networks trained with back propagation of data for tasks like image recognition have overshadowed biological approaches to machine learning. “The amount of attention given to exploring the diversity of possible biologically inspired learning rules, in the present era of large data sets and fast computers, has been rather limited,” the researchers noted.

“Our paper challenges this opinion by describing an unsupervised learning algorithm that demonstrates a very good performance” on various databases used for training image processing frameworks, they added.

Recent items:

In Search of a Common Deep Learning Stack

Deep Learning: The Confluence of Big Data, Big Models, Big Compute

Why Deep Learning May Not Be So ‘Deep’ After All