New paper: Towards biologically plausible deep learning: learning to solve the credit assigment problem

Excited to present this paper in collaboration with Prashanth Prakash and Konrad Kording at ICLR 2020 main meeting:

Backpropagation is driving today’s artificial neural networks (ANNs). However, despite extensive research, it remains unclear if the brain implements this algorithm. Among neuroscientists, reinforcement learning (RL) algorithms are often seen as a realistic alternative: neurons can randomly introduce change, and use unspecific feedback signals to observe their effect on the cost and thus approximate their gradient. However, the convergence rate of such learning scales poorly with the number of involved neurons. Here we propose a hybrid learning approach. Each neuron uses an RL-type strategy to learn how to approximate the gradients that backpropagation would provide. We provide proof that our approach converges to the true gradient for certain classes of networks. In both feedforward and convolutional networks, we empirically show that our approach learns to approximate the gradient, and can match the performance of gradient-based learning. Learning feedback weights provides a biologically plausible mechanism of achieving good performance, without the need for precise, pre-specified learning rules.

Lansdell B, Prakash P, Kording K, arXiv ICLR 2020 main meeting

Updated:

You May Also Enjoy

New preprint: Neural Spiking for Causal Inference

How do neurons learn their effect on downstream reward, so that they can update their synaptic weights? An important problem known as the credit assignment p...

Causal inference algorithms for learning in neural networks

A number of challenges in both machine learning (ML) and neuroscience are related to causation. In ML, causality relates to issues of transfer and generaliza...

What is computational neuroscience?

Recently I had the opportunity to give a talk to the undergraduate student physics group at Western University. It was a fun exercise to reflect my area of r...

New paper: How to get AI to apply causal reasoning like us – transferring from observation to action

A super important problem in reinforcement learning, and machine learning more generally, is how to improve causal reasoning. I think a key part of the solut...