Читать книгу Machine Learning for Tomographic Imaging - Professor Ge Wang - Страница 39

Artificial neural networks originated from mimicking biological neural systems. It is necessary to understand the connection between the artificial neural network and the biological neural network before one is introduced to deep learning.

The hierarchical structure of the HVS is shown in figure 1.2. In the HVS, features are extracted layer by layer. As described in chapter 1, in the ‘what’ pathway, the V1/V2 area is mainly sensitive to edges and lines, the V4 area senses object shapes, and finally the IT region completes the object recognition. In this process, the receptive field is constantly increasing in size, and the extracted features are increasingly more complex.

To a large degree, the artificial neural network attempts to duplicate the biological neural network from the perspective of information processing. As a result, an artificial neural network serves as a simple mathematical model, and different networks are defined by different interconnections among various numbers of artificial neurons. A neural network is a computational framework, including a large number of neurons as basic computing units connected to each other with varying connection strengths known as weights. This feed-forward, layered architecture is mainly intended to reflect the process of extracting visual features with biological neurons layer by layer. Not being totally the same as the HVS, the artificial neural network is composed of the following elements: neurons with weights and activation functions, and network topology or connections, which ought to be trained with training data according to some learning rules. The neuron is the node of the network; each connection transfers the output of a neuron to the next neuron as its input, weights are used to modify input data before summation, and the learning rules guide the parametric modification for the neural network to perform optimally. The components of the artificial neural network will be described in detail below.

Figure 3.1 correlates the convolutional neural network (CNN) with the HVS, in which we map the components of the CNN to the corresponding components of the HVS (Yamins et al 2014). This correspondence helps us understand the neural network. First, the input to a CNN is usually either 3D (RGB) or 1D (gray scale) pixel values after a preprocess of normalization. This roughly corresponds to part computations performed by the retina and lateral geniculate nucleus. The convolutional operations create feature maps that have a spatial layout, such as the retinotopic maps for visual information processing, and each of the artificial neurons can only process data within a receptive field of a limited extent. The convolutional filters define feature maps, which can be grouped into a number of different types.

Figure 3.1. Relating the CNN to the HVS consisting of the brain areas responsible for a sequence of visual information processing tasks.

In a good sense, artificial neural networks are engineered copies of the HVS. At the same time, an artificial neural network is not an exact replica of the human vision system or the human brain. In the biological neural networks such as the HVS or more generally the human brain, the learning process is much more complicated and is achieved by the combination of many factors, such as the surrounding environment, interest/attention/drive, mode, internal representations, and so on. While weights in an artificial network are initialized randomly, connections between biological neurons are genetically derived, and then reinforced or weakened in the learning process. Unlike the biological neural network, the artificial neural network is commonly trained by relevant data from the application domain. At present, the network topology is pre-specified based on a designer’s experience, does not change in the training process, and weights are randomly initialized and adjusted using an optimization algorithm to map input stimuli to desired output values. The single-layer neural network introduced in chapters 1 and 2 is a good example.

When we design an artificial neural network, we should select a network topology for feature extraction, a training procedure to update the network parameters, and so on. In order to output a desirable result, an artificial neural network should have certain properties. By construction, we must ensure that the artificial neural network is sufficiently expressive, and can be trained with relevant data toward a converged configuration. Convergence means that the process of training the neural network converges to a limit. When the neural network training process converges, the model has the tendency of stabilizing over time, predicting consistent, meaningful outputs from new data. As explained late in this chapter, several related problems must be addressed to have a convergent and favorable outcome.