Читать книгу Industry 4.1 - Группа авторов - Страница 66

Recently, AEN becomes an important and popular technique to efficiently reduce the dimensionality and generate the abstract of large volumes of data [11, 12]. AEN is an unsupervised backpropagation neural‐network consisting of three fully‐connected layers of encoder (input), code (middle), and decoder (output).

The encoder layer encodes and compresses the data to the code layer, and then decoder layer reconstructs the compressed internal representation of input data from the code layer into output data as closer to the original input as possible. As depicted in Figure 2.22, the architecture of the encoder, code, and decoder can be designed to constitute at least one layer each.

Figure 2.22 Architecture of the AEN.

Let x be one variable of the input set, then the mathematical relationships between layers can be defined as (2.16) and (2.17), and its output belongs to the same space with x.

(2.16)

(2.17)

where

 h compressed code of the middle layer;

  output reconstructed from c in the middle layer;

 fEN encoder layer;

 fDE decoder layer;

 fa activation function;

 WEN network weight for node in the encoder;

 WDE network weight for node in the decoder;

 bEN bias for node in the encoder layer;

 bDE bias for node in the decoder layer.

The number of input and output nodes depends on the size of raw data, while the number of nodes in the code layer is a hyperparameter that varies according to the AEN architecture and input data format as other hyperparameters do.

All weights and biases are usually initialized randomly, and then the learning procedure starts to iteratively update weights through back‐propagation algorithm, which minimizes the reconstruction errors between x and . This training technique can force each layer to learn a higher‐level representation of its previous layer.

Instead of adopting the entire AEN, the compressed code h is widely used as condensed SFs to represent the original input set. If there are cp components in the code layer, then the SF set SF_AEN can be defined as {h₁, h₂, …h_cp}. This feature extraction method is very similar to adopting the other well‐known dimensionality reduction technique: principle component analysis (PCA).