Difference between revisions of "Neuron (neural network)"

Latest revision as of 13:33, 22 April 2016

Definition

• an output domain, $\mathcal{O}$ typically $[-1,1]\subseteq\mathbb{R}$ or $[0,1]\subseteq\mathbb{R}$
  • Usually $\{0,1\}$ for input and output neurons
• some inputs, $I_i$, typically $I_i\in\mathbb{R}$
• some weights, 1 for each input, $w_i$, again $w_i\in\mathbb{R}$
• a way to combine each input with a weight (typically multiplication) ($I_i\cdot w_i$ - creating an "input activation", $A_i\in\mathbb{R}$
• a bias, $\theta$ (pf the same type as the result of combining an input with a weight. Typically this can be simulated by having a fixed "on" input, and treating the bias as another weight) - another input activation, $A_0$
• a way to combine the input values, typically: $\sum_{j=0}^nA_j=\sum_{j=1}^nI_jw_j+\theta$
• an activation function $\mathcal{A}(\cdot):\mathbb{R}\rightarrow\mathcal{O}\subseteq\mathbb{R}$, this maps the combined input activations to an output value.

In the example to the right, the output of the neuron would be:

• $\mathcal{A}\left(\sum_{i=1}^n(I_iw_i)+\theta\right)$

Specific models

For an exhaustive list see Category:Types of neuron in a neural network

McCulloch-Pitts neuron

• Inputs: $(I_1,\ldots,I_n)\in\mathbb{R}^n$
  • Usually each $I_i$ is confined to $[0,1]\subset\mathbb{R}$ or $[-1,1]\subset\mathbb{R}$
• A set of weights, one for each input: $(w_1,\ldots,w_n)\in\mathbb{R}^n$
• A bias: $\theta\in\mathbb{R}$
• An activation function, $\mathcal{A}:\mathbb{R}\rightarrow\mathbb{R}$
  • It is more common to see $\mathcal{A}:\mathbb{R}\rightarrow[-1,1]\subset\mathbb{R}$ or sometimes $\mathcal{A}:\mathbb{R}\rightarrow[0,1]\subset\mathbb{R}$ than the entire of $\mathbb{R}$

The output of the neuron is given by:

$$\text{Output}:=\mathcal{A}\left(\sum_{i=1}^n(I_iw_i)+\theta\right)$$

References

1. Jump up ↑ Neural Networks and Statistical Learning - Ke-Lin Du and M. N. S. Swamy

Template:Neural networks navbox

Template:Statistics Definition