Neuron

This is the cell of a Neural Net. Holds an activation which is a continuous number from $0$ to $1$.

• A neuron is ‘lit up’ if its activation is 1
• A neuron is ‘dark’ if its activation is 0

Activation Formula

$n=f(w_1{n}_1+\cdots +w_k{n}_k+B)$ Where:

• $n$ is the current neuron’s activation
• $n_1,\dots ,n_k$ are all the previous neurons in the last Layer
• $w_1,\dots ,w_k$ are Weights
• $f(\cdot )$ is the Activation Function
• $B$ is the Bias

Example

For a given image, a neuron could represent a single pixel’s light. Then, Layers could be used to represent each row of pixels in the neural network.

Neurons as Features

The hope for a neuron is that it is able to represent some tangible feature of an output. Like a loop of an image is from one neuron. Neuron Superposition attempts to disprove this.