Gaussian Error Linear Unit

This is a Activation Function commonly seen in Multi Layer Perceptron.

$\text{GELU}\left(x\right)=xP\left(X\le x\right)=x\Phi \left(x\right)=x\cdot \frac{1}{2}\left[1+\text{erf}(x/\sqrt{2})\right],$