Читать книгу Intelligent Security Management and Control in the IoT - Mohamed-Aymen Chalouf - Страница 37

Communications between objects in the Internet of Things (IoT) and particularly machine-to-machine (M2M) communications are considered as one of the most important evolutions of the Internet. Supporting these devices is, however, one of the most significant challenges that network operators need to overcome (Lin et al. 2016). In fact, the considerable number of these devices that might attempt to access a network at the same time could lead to heavy congestion, or even a total saturation, with all the consequences this might cause. In fact, as can be seen in Figure 2.1, a very limited number of devices trying simultaneously to access the network can reduce the network’s performance to zero, independently of the access opportunities available (Bouzouita et al. 2016). In these circumstances, it seems clear that effective access control mechanisms are needed to maintain a reasonable number of access attempts.

The Third Generation Partnership Project (3GPP) group has identified the overload on the Random Access Network (RAN) as a priority at a very early stage and has suggested several solutions. Among the approaches suggested, the Access Class Barring (ACB), suggested in version 8, and its extension, the Extended Access Barring (EAB), suggested in version 11, are certainly the most effective strategies (3GPP 2011). In fact, these approaches tackle the problem at its roots by preventing any attempts to access the network. However, these approaches merely provide a framework for congestion management, without providing a ready-made solution.

Figure 2.1. Impact on performance of the number of IoT devices simultaneously attempting access

The idea introduced in this chapter is quite simple, because it involves calculating a blocking factor. Nevertheless, a good implementation would require a good knowledge of the number of terminals ready to attempt access, so as to deduce from this the probability of the optimal blocking. This information is unfortunately not available in the network.

To solve this problem, two significant challenges should be taken into account: (1) designing an access control strategy for dynamic generation of the blocking factor and (2) estimating the number of devices simultaneously attempting to access the network.

In this chapter, we tackle these questions using an estimator that was suggested in earlier research (Bouzouita et al. 2019). Since this estimation is very noisy, we exploit the potential of the most advanced reinforcement learning techniques, to take account of this complex reality (the state of the network is not observable) and deduce a sub-optimal control strategy. More especially in this chapter, we use the deep reinforcement learning algorithm Twin Delayed Deep Deterministic policy gradient algorithm (TD3) (Fujimoto et al. 2018) to produce the optimal blocking factor from these past estimations (Hadjadj-Aoul and Ait-Chellouche 2020).

The remainder of this chapter is organized as follows. Section 2.2 briefly presents the NB-IoT standard considered in our proposal. Section 2.3 gives an overview of the main congestion control techniques in IoT networks, and more especially cellular IoT networks. Section 2.4 describes the model for IoT terminals to access the network. Section 2.5 describes the suggested control solution, based on the TD3 algorithm, adapted to solve the problem of calculating the blocking factor. Section 2.6 describes the environment for simulating the suggested approach and shows its effectiveness compared to the existing approach. Finally, the chapter ends with a summary recapitulating the main advantages and achievements of the system suggested in section 2.7.