Neural Networks

Internal

• Machine Learning

Overview

A neural network consists of several layers of activation units ("individual neurons"), where one layer's activation unit output is connected to the inputs of all activation units of the successive layer. The behavior of an individual activation unit is described in the "Individual Unit" section. A neural network's topology, along with conventions and notations - which are essential to get right if you want to follow the linear algebra equations - are discussed in the "Topology" section. A neural network produces predictions by forward propagating input, then activations across its layers from left to right, until the output layer computes the hypothesis function, for a specific input sample. The forward propagation process is described in the "Forward Propagation" section. Forward propagation computations are performed based on a set of parameters (or weights) that are obtained by training the network. Training the network, or "fitting the parameters", is performed by a backpropagation algorithm, which is described in the "Backpropagation" section.

Individual Unit

Input Feature

In context of an individual processing unit, the input feature refers to an individual value fed to a single input of the unit. For units in the first layer, the features are individual elements of the input matrix, while for units in the hidden layers or the output layers, the features are the activation values of the computational units from the previous layer.