Читать книгу Multi-Processor System-on-Chip 1 - Liliana Andrade - Страница 18

In this section, we investigate in detail the requirements and processor capabilities for efficient machine learning in low-power IoT edge devices. The common theme in machine learning is that algorithms that have the ability to learn without being explicitly programmed are used (Samuel 1959). As shown in Figure 1.1, in machine learning, we distinguish between training and inference.

Figure 1.1. Training and inference in machine learning

Training starts with an untrained model, for example a multi-layered neural network with a chosen graph structure. In these neural networks, each layer transforms input data into output data while applying sets of coefficients or weights. Using a machine learning framework such as Caffe or TensorFlow, the model is trained using a large training dataset. The result of the training is a trained model, for example, a neural network with its weights tuned for classifying input data into certain categories. Such categories may, for example, be the different types of human activity in the activity tracker device mentioned above.

Inference uses the trained model for processing input data captured by sensors to infer the complex patterns it has been trained to recognize. For example, it can check whether the input data matches one of the categories that a neural network has been trained for, such as “walking” or “sitting” in the activity tracker device. Therefore, upon inference, the trained model is applied to new data. Inference is typically performed in the field. In this chapter, we focus on inference rather than on training. More specifically, we address the efficient implementation of machine learning inference on a programmable processor.

The processing requirements of machine learning inference can vary wildly for different applications. Some key factors impacting the processing requirements are:

 – input data rate: this is the rate at which data samples are captured by the sensor(s). These samples can, for example, be pixels coming from a camera or pulse-code modulation (PCM) samples coming from a microphone. The input data rate can range from tens of samples per second, for example, for human activity recognition with a small number of sensors, to hundreds of millions of samples per second, for advanced computer vision with a high-resolution camera capturing images at a high frame rate;

 – complexity of the trained model: this defines the number of operations to be performed for a set of samples (e.g. an input image) upon inference. For example, in the case of neural networks, the complexity depends on the number of layers in a graph, the sizes of the (multidimensional) input and output maps for each layer, and the number of weights to be applied in the calculation of the output maps. A lowcomplexity neural network has less than 10 layers, while a high-complexity neural network can have tens of layers (Szegedy et al. 2015).

Table 1.1 shows input data rates and model complexities for several example machine learning applications.

Table 1.1. Input data rates and model complexities for example machine learning applications

Machine learning application	Input data rate	Complexity of trained model
Human activity recognition	10s Hz (few sensors)	Low to medium
Voice control	10s kHz (e.g. 16 kHz)	Low to medium
Face detection	100s kHz (low resolution and frame rate)	Low to medium
Advanced computer vision	100s MHz (high resolution and frame rate)	High

As can be seen from Table 1.1, input data rates and model complexities can vary significantly. As a result, the compute requirements for machine learning inference can differ by several orders of magnitude. In this chapter, we focus on machine learning inference with low-to-medium compute requirements. More specifically, we target machine learning inference that can be used to build smart low-power IoT edge devices.

In the next section, we further detail the requirements for the efficient implementation of low/mid-end machine learning inference. Using the DSP-enhanced DesignWare^® ARC^® EM9D processor as an example, we discuss the features and capabilities of a programmable processor that enable efficient execution of computations often used in machine learning inference. We further present an extensive library of software functions that can be used to quickly implement inference engines for a wide array of machine learning applications. Benchmarks are presented to demonstrate the efficiency of machine learning inference implemented using this software library that executes on the ARC EM9D processor.