Читать книгу Machine learning in practice – from PyTorch model to Kubeflow in the cloud for BigData - Eugeny Shtoltc - Страница 3

Until 2015, scikit-learn was leading by a wide margin, which Caffe was catching up with, but with the release of TensorFlow, it immediately became the leader. Over time, only gaining a gap from two to three times by 2020, when there were more than 140 thousand projects on GitHub, and the closest competitor had just over 45 thousand. In 2020, Keras, scikit-learn, PyTorch (FaceBook), Caffe, MXNet, XGBoost, Fastai, Microsoft CNTK (CogNiive ToolKit), DarkNet and some other lesser known libraries are located in descending order. The most popular are the Pytorch and TenserFlow libraries. Pytorch is good for prototyping, learning and trying out new models. TenserFlow is popular in production environments and the low-level issue is addressed by Keras.

* FaceBook Pytorch is a good option for learning and prototyping due to the high level and support of various

environments, a dynamic graph, can give advantages in learning. Used by Twitter, Salesforce.

* Google TenserFlow – originally had a static solution graph, now dynamic is also supported. Used in

Gmail, Google Translate, Uber, Airbnb, Dropbox. To attract use in the Google cloud for it

Google TPU (Google Tensor Processing Unit) hardware processor is being implemented.

* Keras is a high-level tweak providing more abstraction for TensorFlow, Theano

or CNTK. A good option for learning. For example, he

allows you not to specify the dimension of layers, calculating it yourself, allowing the developer to focus on the layers

architecture. Usually used on top of TenserFlow. The code on it is maintained by Microsoft CNTK.

There are also more specialized frameworks:

* Apache MXNet (Amazon) and a high-level add-on for it Gluon. MXNet is a framework with an emphasis on

scaling, supports integration with Hadoop and Cassandra. Supported

C ++, Python, R, Julia, JavaScript, Scala, Go and Perl.

* Microsoft CNTK has integrations with Python, R, C # due to the fact that most of the code is written in C ++. That all sonova

written in C ++, this does not mean that CNTK will train the model in C ++, and TenserFlow in Python (which is slow),

since TenserFlow builds graphs and its execution is already carried out in C ++. Features CNTK

from Google TenserFlow and the fact that it was originally designed to run on Azure clusters with multiple graphical

processors, but now the situation is leveled and TenserFlow supports the cluster.

* Caffe2 is a framework for mobile environments.

* Sonnet – DeepMind add-on on top of TensorFlow for training super-deep neural networks.

* DL4J (Deep Learning for Java) is a framework with an emphasis on Java Enterprise Edition. High support for BigData in Java: Hadoop and Spark.

With the speed of availability of new pre-trained models, the situation is different and, so far, Pytorch is leading. In terms of support for environments, in particular public clouds, it is better for the farms promoted by the vendors of these clouds, so TensorFlow support is better in Google Cloud, MXNet in AWS, CNTK in Microsoft Azure, D4LJ in Android, Core ML in iOS. By languages, almost everyone has common support in Python, in particular, TensorFlow supports JavaScript, C ++, Java, Go, C # and Julia.

Many frameworks support TeserBodrd rendering. It is a complex Web interface for multi-level visualization of the state and the learning process and its debugging. To connect, you need to specify the path to the "tenserboard –logdir = $ PATH_MODEL" model and open localhost: 6006. Interface control is based on navigating through the graph of logical blocks and opening blocks of interest for subsequent repetition of the process.

For experiments, we need a programming language and a library. Often the language used is a simple language with a low entry threshold, such as Python. There may be other general-purpose languages like JavaScript or specialized languages like R. I'll take Python. In order not to install the language and libraries, we will use the free service colab.research.google.com/notebooks/intro.ipynb containing Jupiter Notebook. Notebook contains the ability not only to write code with comments in the console form, but to format it as a document. You can try Notebook features in the educational playbook https://colab.research.google.com/notebooks/welcome.ipynb, such as formatting text in the MD markup language with formulas in the TEX markup language, running scripts in Python, displaying the results of their work in text form and in the form of graphs using the standard Python library: NumPy (NamPay), matplotlib.pyplot. Colab itself provides a Tesla K80 graphics card for 12 hours at a time (per session) for free. It supports a variety of deep machine learning frameworks, including Keras, TenserFlow, and Pytorch. The price of a GPU instance in Google Cloud:

* Tesla T4: 1GPU 16GB GDDR6 0.35 $ / hour

* Tesla P4: 1GPU 8GB GDDR5 0.60 $ / hour

* Tesla V100: 1GPU 16GB HBM2 2.48 $ / hour

* Tesla P100: 1GPU 16GB HBM2 $ 1.46 / hour

Let's try. Let's follow the link colab.research.google.com and press the button "create a notepad". We will have a blank Notebook. You can enter an expression:

10 ** 3/2 + 3

and clicking on play – we get the result 503.0. You can display the graph of the parabola by clicking the "+ Code" button in the new cell in the code:

def F (x):

return x * x

import numpy as np

import matplotlib.pyplot as plt

x = np.linspace (-5, 5, 100)

y = list (map (F, x))

plt.plot (x, y)

plt.ylabel ("Y")

plt.xlabel ("X")

Or displaying an image as well:

import os

! wget https://www.python.org/static/img/python-logo.png

import PIL

img = PIL.Image.open ("python-logo.png")

img

Popular frameworks:

* Caffe, Caffe2, CNTK, Kaldi, DL4J, Keras – a set of modules for design;

* TensorFlow, Theano, MXNet – graph programming;

* Torch and PyTorch – register the main parameters, and the graph will be built automatically.

Consider the PyTorch library (NumPy + CUDA + Autograd) because of its simplicity. Let's look at operations with tensors – multidimensional arrays. Let's connect the library and declare two tensors: press + Code, enter the code into the cell and press execute:

import torch

a = torch.FloatTensor ([[1, 2, 3], [5, 6, 7], [8, 9, 10]])

b = torch.FloatTensor ([[– 1, -2, -3], [-10, -20, -30], [-100, -200, -300]])

Element-wise operations such as "+", "-", "*", "/" on two matrices of the same dimensions perform operations with their corresponding elements:

a + b

tensor ([[0., 0., 0.],

[-5., -14., -23.],

[-92., -191., -290.]])

Another option for the elementwise operation is to apply one operation to all elements one by one, for example, multiply by -1 or apply a function:

a

tensor ([[1., 2., 3.],

[5., 6., 7.],

[8., 9., 10.]])

a * -1

tensor ([[-1., -2., -3.],

[-5., -6., -7.],

[-8., -9., -10.]])

a.abs ()

tensor ([[1., 2., 3.],

[5., 6., 7.],

[8., 9., 10.]])

There are also convolution operations, such as sum, min, max, which, as input, give the sum of all elements, the smallest or largest element of the matrix:

a.sum ()

tensor (51.)

a.min ()

tensor (1.)

a.max ()

tensor (10.)

But, we will be more interested in post-column operations (the operation will be performed on each column):

a.sum (0)

tensor ([14., 17., 20.])

a.min (0)

torch.return_types.min (values = tensor ([1., 2., 3.]), indices = tensor ([0, 0, 0]))

a.max (0)

torch.return_types.max (values = tensor ([8., 9., 10.]), indices = tensor ([2, 2, 2]))

As we remember, a neural network consists of three layers, a layer of neurons, and a neuron contains connections at the input with weights in the form of prime numbers. The weight is set by an ordinary number, then the incoming connections to the neuron can be described by a sequence of numbers – a vector (one-dimensional array or list), the length of which is the number of connections. Since the network is fully connected, all the neurons of this layer are connected to the previous one, and therefore the vectors demonstrating them also have the same length, creating a list of vectors of equal length – a matrix. It is a convenient and compact layer representation optimized for use on a computer. At the output of the neuron, there is an activation function (sigmoid or, ReLU for deep and ultra-deep networks), which determines whether the neuron outputs a value or not. To do this, it is necessary to apply it to each neuron, that is, to each column: we have already seen the operation on columns.