The computational brain – understanding learning algorithms in neural circuitry

• Neurons have a tree-like structure: dendrites (roots), soma (cell body), axon (trunk)
• Electrically exciteable -- spikes travel along the axon to the dendrites of other neurons
• Synapses -- connections between two neurons

Some numbers:

• Human brain: 86 billion neurons
• Each neuron may synapse with 1000-10000 other neurons

• We have little idea how to treat brain disease -- treat symptoms

Hampel et al 2018

• Computation as a model for reasoning (and thought more generally?)

$\Rightarrow$ Why might this be the case? And what does it look like to answer questions in this way?

1. The historical answer: computation and the mechanization of reasoning
2. Computational neuroscience: the information processing and computational abilities of networks of neurons
3. An example: the search for the learning algorithms of the brain

Rationalism:

• Acquisition of knowledge through reason
• Inspired by Greek mathematics: Euclid's Elements
• 17th century thinkers: Descartes, Spinoza, Liebniz

Rationalism:

• Acquisition of knowledge through reason
• Inspired by Greek mathematics: Euclid's Elements
• 17th century thinkers: Descartes, Spinoza, Liebniz

Rationalism:

• Acquisition of knowledge through reason
• Inspired by Greek mathematics: Euclid's Elements
• 17th century thinkers: Descartes, Spinoza, Liebniz

Spinoza's Ethics:

Gottfried Wilhelm Leibniz (1646 - 1716):

• Philosopher, inventor, mathematician (inventor of calculus)
• Designed adding machines
• Wanted to build reasoning machines
• Calculus ratiocinator
• Universal characteristic

Gottfried Wilhelm Leibniz (1646 - 1716):

• Philosopher, inventor, mathematician (inventor of calculus)
• Designed adding machines
• Wanted to build reasoning machines:
  • Calculus ratiocinator
  • Universal characteristic

George Boole (1815 – 1864):

• Boolean algebra: logic as mathematics

"The English logician George Boole, in the 1850s, was among the first to formulate the idea--in his famous book The Laws of Thought--that thinking itself follows clear patterns, even laws, and that these laws could be mathematized. For this reason, I like to refer to this law-bound vision of the activities of the human mind as the "Boolean Dream."
– Douglas Hofstadter

Charles Babbage (1791 - 1871):

• Mathematician, inventor, philosopher, engineer
• First programmable mechanical computer (punch cards)

Alan Turing (1912 - 1954):

• Code breaker, mathematician, inventor
• Designed stored-program computer, ACE
• Wanted to build reasoning machines
• Turing machines (proposed 1936) based on models of human computers

Alan Turing (1912 - 1954):

• Code breaker, mathematician, inventor
• Designed stored-program computer, ACE
• Wanted to build reasoning machines
• Turing machines (proposed 1936) based on models of human computers

Alan Turing (1912 - 1954):

• Code breaker, mathematician, inventor
• Designed stored-program computer, ACE
• Wanted to build reasoning machines
• Turing machines (proposed 1936) based on models of human computers

• Computers have a long, intertwined history with formal systems of reasoning
• In cognitive science today – computationalism: mental states are computational states

• Computers have a long, intertwined history with formal systems of reasoning
• In cognitive science today – computationalism: mental states are computational states

• Computers have a long, intertwined history with formal systems of reasoning
• In cognitive science today – computationalism: mental states are computational states

1. The historical answer: computation and the mechanization of reasoning
2. Computational neuroscience: the information processing and computational abilities of networks of neurons
3. An example: the search for the learning algorithms of the brain

A device that performs calculations

• Input-output
• Processes information -- internal states represent external states

Interested in computation of $y = f(x)$:

• There exists mapping from physical inputs to $x$, and from physical outputs to $y$
• The physical system evolves such that $y = f(x)$

A device that performs calculations

• Input-output
• Processes information -- internal states represent external states

Interested in computation of $y = f(x)$:

• There exists mapping from physical inputs to $x$, and from physical outputs to $y$
• The physical system evolves such that $y = f(x)$

E.g. the 'x 2' computer: $y = 2x$

Modern computers: digital, serial, electronic

But not necessarily:

The brain:

• input-output: stimulus-response
• represents features of the outside world, and manipulates those representations – an information processing device

Brette 2019

$\Rightarrow$ Want an account of how it performs these information processing tasks

Marr's three levels of analysis:

• Computational: what is the system doing
• Algorithmic: what method is it using to perform the task
• Implementation: what physical substrate is used to implement the algorithm

1. The historical answer: computation and the mechanization of reasoning
2. Computational neuroscience: the information processing and computational abilities of networks of neurons
3. An example: the search for the learning algorithms of the brain

Image recognition:

• Assume we can draw images $x$ and labels $y$ from a distribution $\rho$
• Let $\hat{y} = f(x;w)$ be a function that outputs predictions of label given image $x$

Aim: make predictions close to true labels

• $L(y,\hat{y})$ our measure of closeness. E.g. mean squared error, cross-entropy loss
• $\mathcal{R} = \mathbb{E}_\rho[L(y, f(x; w))]$

Assume $f(x;w)$ is a neural network.

• $h_i = \sigma(W h_{i-1})$
• $W$ are synaptic strengths/weights

How to update weights $w$?

• Update weights using first order gradient info
• Gradient Descent: $$\Delta w = -\eta \frac{\partial \mathcal{R}}{\partial w}$$ Approximate with our data: $\mathcal{R} = \frac{1}{N}\sum_{n=1}^N L(y_n, f(x_n; w))$
• Stochastic Gradient Descent: $$\Delta w = -\eta \frac{\partial \mathcal{R}}{\partial w}$$ Approximate with randomly sampled batch of data: $\mathcal{R} = \frac{1}{M}\sum_{i=1}^M L(y_i, f(x_i; w))$
• Can be helpful to add inertia (momentum)

Use the backpropagation algorithm

• Then $\frac{\partial R}{\partial \mathbf{h}^i} = (W^{i+1})^T \mathbf{e}^{i+1}$

Use a reinforcement learning algorithm

• Assume for a given input, a neuron has noise in its output $$\tilde{h} = h + \xi, \quad \xi \sim \mathcal{N}(0, \sigma^2)$$
• Can show $$ \frac{\partial \mathcal{R}}{\partial h} \approx \mathbb{E}(\mathcal{R} \xi)$$

Reinforcement learning: biologically plausible

• A neuron needs to update its synapses proportional to $\mathcal{R}\xi$
• $\mathcal{R}$ can be communicated to all neurons in the network by a neurotransmitter
• e.g. dopamine – known to be involved in signalling reward prediction error

Backpropagation: hah, yeah right

• An implementation of BP would seem to require a complementary network that communicates error in a top-down fashion
• This network has weights that are the transpose of the weights in the original network

Although...

• Deep neural networks (trained using BP) are the only model that can solve challenging image-recognition problems
• Layers of neurons in cortex do contain both feedback and feedback connections
• Exact feedback weights may not be strictly necessary
• Pyramidal neurons in cortex are compartmentalized... for performing two separate computations?

• The brain as a computer
• What algorithms does it use to learn?
• Computation is a general concept, so the correspondance between current ML solutions to (say) image recognition and the brain's visual system is by no means automatic
• A very active area of research
• Many tools from physics can help:
  • Learning dynamics, optimization
  • Dynamics of large populations of connected, noisy neurons -- statistical mechanics