Optimizing TSCH Scheduling for IIoT Networks Using Reinforcement Learning

While IEEE 802.15.4e Time-Slotted Channel Hopping (TSCH) networks should be equipped to deal with the hard wireless challenges of industrial environments, the sensor networks are often still limited by the characteristics of the used physical (PHY) layer. Therefore, the TSCH community has recently started shifting research efforts to the support of multiple PHY layers, to overcome this limitation. On the one hand, integrating such multi-PHY support implies dealing with the PHY characteristics to fit the resource allocation in the TSCH schedule, and on the other hand, defining policies on how to select the appropriate PHY for each network link. As such, first a heuristic is proposed that is a step towards a distributed PHY and parent selection mechanism for slot bonding multi-PHY TSCH sensor networks. Additionally, a proposal on how this heuristic can be implemented in the IPv6 over the TSCH mode of IEEE 802.15.4e (6TiSCH) protocol stack and its Routing Protocol for Low-power and Lossy network (RPL) layer is also presented. Slot bonding allows the creation of different-sized bonded slots with a duration adapted to the data rate of each chosen PHY. Afterwards, a TSCH slot bonding implementation is proposed in the latest version of the Contiki-NG Industrial Internet of Things (IIoT) operating system. Subsequently, via extensive simulation results, and by deploying the slot bonding implementation on a real sensor node testbed, it is shown that the computationally efficient parent and PHY selection mechanism approximates the packet delivery ratio (PDR) results of a near-optimal, but computationally complex, centralized scheduler.

The IEEE 802.15.4-2015 TSCH (Time Slotted Channel Hopping) standard provides high-reliability, deterministic delay, and energy efficiency with the channel hopping and the TDMA. The standard does not specify the scheduling method, which is an essential part of the solution. One of the representative studies in TSCH scheduling is Orchestra. It is an autonomous scheduling method with minimal overhead that does not require a negotiation process between nodes. However, with high traffic load, performance of the Orchestra is greatly degraded because of its fixed schedule. If a node has children, continuous collisions occur in the slot of the node. We propose a novel algorithm called SRCA (Slot Reallocation for Collision Avoidance), as a solution for this problem. The SRCA autonomously updates the schedule. When a child requests a slot reallocation, the parent allocates a new slot with a low probability of collision. Another algorithm called SJCA (Slot reJection for Collision Avoidance) is further proposed in this work. It improves the SRCA to operate in an environment with interference where a sending node can disturb a transmission from a hidden node. In the SJCA, a node determines whether a collision due to interference will occur in the slot newly allocated by the parent. If so, it rejects the slot and requests again. We compare the performance of the SRCA and the SJCA against the Orchestra and e-TSCH-Orch through simulations. It is confirmed that the SJCA is robust to interference. In most cases, the SJCA shows better performance of PAR (Packet Acknowledgement Ratio), PLR (Packet Loss Rate), and latency than others. It is also verified that the SJCA provides high-reliability and low latency compared to existing technologies.

IWSNs (Industrial Wireless Sensor Networks) have become the next step in the evolution of WSN (Wireless Sensor Networks) due to the nature and demands of modern industry. With this type of network, flexible and scalable architectures can be created that simultaneously support traffic sources with different characteristics. Due to the great diversity of application scenarios, there is a need to implement additional capabilities that can guarantee an adequate level of reliability and that can adapt to the dynamic behavior of the applications in use. The use of SDNs (Software Defined Networks) extends the possibilities of control over the network and enables its deployment at an industrial level. The signaling traffic exchanged between nodes and controller is heavy and must occupy the same channel as the data traffic. This difficulty can be overcome with the segmentation of the traffic into flows, and correct scheduling at the MAC (Medium Access Control) level, known as slices. This article proposes the integration in the SDN controller of a traffic manager, a routing process in charge of assigning different routes according to the different flows, as well as the introduction of the Time Slotted Channel Hopping (TSCH) Scheduler. In addition, the TSCH (Time Slotted Channel Hopping) is incorporated in the SDN-WISE framework (Software Defined Networking solution for Wireless Sensor Networks), and this protocol has been modified to send the TSCH schedule. These elements are jointly responsible for scheduling and segmenting the traffic that will be sent to the nodes through a single packet from the controller and its performance has been evaluated through simulation and a testbed. The results obtained show how flexibility, adaptability, and determinism increase thanks to the joint use of the routing process and the TSCH Scheduler, which makes it possible to create a slicing by flows, which have different quality of service requirements. This in turn helps guarantee their QoS characteristics, increase the PDR (Packet Delivery Ratio) for the flow with the highest priority, maintain the DMR (Deadline Miss Ratio), and increase the network lifetime.

The Internet Engineering Task Force 6TiSCH Working Group standardizes the protocol stack for the Industrial Internet of Things, which combines the industrial performance of IEEE802.15.4 time‐slotted channel hopping (TSCH) with the addressability and Internet integration capabilities of Internet Protocol version 6. The 6top Protocol, which is part of 6TiSCH, allows distributed management of the TSCH communication schedule. The Scheduling Function Zero (SF0) is the default algorithm for a node to decide when to add/remove cells to its neighbors. SF0 picks the cells to add randomly; hence, it is possible that 2 nearby pairs of nodes pick the same cell, resulting in a schedule collision. This article introduces cost‐aware cell relocation (CCR), which is a solution to detect and fix such schedule collisions in a distributed manner. CCR monitors the performance of individual cells and uses that information to detect collisions. When such a collision is detected on a cell, neighbor nodes communicate to relocate that cell to a different slot/channel in the TSCH schedule. We implement CCR and run it on a 5‐node network on the Internet of Things Laboratory testbed. The results show that, compared with today's SF0 and for the same traffic requirements, CCR increases the average packet delivery ratio by 2% and reduces the number of cells that need to be scheduled by 27.3%.

In this work, we design an elastic open radio access network (O-RAN) slicing for the Industrial Internet of things (IIoT). Due to the rapid spread of IoT in the industrial use-cases such as safety and mobile robot communications, the IIoT landscape has been shifted from static manufacturing processes towards dynamic manufacturing workflows (e.g., Modular Production System). But unlike IoT, IIoT poses additional challenges such as severe communication environment, network-slice resource demand variations, and on-time information update from the IIoT devices during industrial production. First, we formulate the O-RAN slicing problem for on-time industrial monitoring and control where the objective is to minimize the cost of fresh information updates (i.e., age of information (AoI)) from the IIoT devices (i.e., sensors) with the device energy consumption and O-RAN slice isolation constraints. Second, we propose the intelligent O-RAN framework based on game theory and machine learning to mitigate the problem’s complexity. We propose a two-sided distributed matching game in the O-RAN control layer that captures the IIoT channel characteristics and the IIoT service priorities to create IIoT device and small cell base station (SBS) preference lists. We then employ an actor-critic model with a deep deterministic policy gradient (DDPG) in the O-RAN service management layer to solve the resource allocation problem for optimizing the network slice configuration policy under time-varying slicing demand. Furthermore, the proposed matching game within the actor-critic model training helps to enforce the long-term policy-based guidance for resource allocation that reflects the trends of all IIoT devices and SBSs satisfactions with the assignment. Finally, the simulation results show that the proposed solution enhances the performance gain for the IIoT services by serving an average of 50% and 43.64% more IIoT devices than the baseline approaches.

A survey on network formation and scheduling algorithms for time slotted channel hopping in industrial networks

Wireless Sensor Nnetworks (WSNs) are becoming increasingly widespread owing to the introduction of the Internet of Things (IoT) and the networks' comparatively low cost and usability. WSNs are usually battery-powered, owing to their low power consumption, which however limits their computational performance. The lifetime of a WSN strongly depends on its communication-related power consumption. Time Slotted Channel Hopping (TSCH) solves the problem of power consumption by featuring both time-division and frequency- division multiplexing in the system's Media Access Control (MAC) layer. In this paper, Pruning-based Coloring Scheduling (PRCOS) is proposed, which is an algorithm for TSCH scheduling that accounts for the power consumption to extend the system's lifetime. Using mathematical analysis, we show that the proposed algorithm decreases the transfer time and energy consumption by factors of 0.43 and 0.57, respectively, compared with existing algorithms.

Autonomous and traffic-aware scheduling for TSCH networks

A Latin rectangles-based TSCH scheduling and interference mitigation design

In order to better understand state-of-the-art reinforcement learning agent, deep Q-network, a brief review of reinforcement learning and Q-learning are first described. Then recent advances of deep Q-network are presented, and double deep Q-network and dueling deep Q-network that go beyond deep Q-network are also given.

The demand for reliable and energy-efficient low-power wireless network protocols is growing up steadily with the flourish of the emerging Internet of Things (IoT) applications. The Time Slotted Channel Hopping (TSCH), part of the IEEE 802.15.4 standard, represents the latest generation of low-power and highly reliable MAC protocols. It orchestrates the medium access according to a time-frequency communication schedule. Several scheduling algorithms have been introduced for TSCH. In contrast to centralized and distributed scheduling approaches, autonomous scheduling does not occur any signaling overhead and appears as a promising approach. In this study, we review autonomous TSCH scheduling algorithms, including static and recently proposed adaptive algorithms. Then, we investigate, on a real-world testbed, the performance of representative static autonomous scheduling schemes for TSCH protocol. Experimental results show that they suffer from high number of queued packets and buffers overflow, which increase the network delay and decreases the delivery ratio. Finally, we qualitatively compare and discuss the performance of recent adaptive autonomous TSCH scheduling algorithms, showing that research on adaptive autonomous TSCH scheduling still has open issues.

ReSF: Recurrent Low-Latency Scheduling in IEEE 802.15.4e TSCH networks

Composed of constrained devices, Low-Power and Lossy Networks (LLNs) have been applied for numerous Internet of Things (IoT) applications. In addition, the Time-Slotted Channel Hopping (TSCH) media access control has been specified by IEEE aiming at Industrial IoT (IIoT). TSCH brings in the deterministic factor over wireless communication; it balances energy, bandwidth and latency, offering reliable communication. This paper conducted an experiment to reach the Contiki-NG’s TSCH Minimal Schedule breaking point. We analysed network factors to identify evidence that may lead to performance degradation. The located evidence might be employed by edge resilient counter-measures systems developers, vertical integration researchers or scheduling function designers to improve systems and increase the IIoT reliability.

Evaluating and Modeling IEEE 802.15.4 TSCH Resilience against Wi-Fi Interference in New-Generation Highly-Dependable Wireless Sensor Networks

IEEE standardized a highly efficient, low power, reliable, deterministic and time–frequency enabled medium access control protocol, IEEE 802.15.4e time slotted channel hopping (TSCH) as an amendment to the medium access control (MAC) protocol defined by the IEEE 802.15.4 standard. Once a communication schedule is built, the IEEE 802.15.4e TSCH is able to execute such a communication schedule. IEEE 802.15.4e TSCH standard lea ves out of scope defining how that communication schedule is built, updated and maintained. In order to fill this gap, we introduce such a scheduling algorithm, called as DIVA. DIVA is specifically designed for IEEE 8092.15.4e TSCH networks. It is a totally distributed scheduling algorithm for divergecast traffic where divergecast means that traffic flows in all directions in contrast to convergecast where traffic flows from leave nodes to root. In order to gain more comprehension about DIVA, we conducted extensive simulations. We also compared DIVA with Berkeley’s Aloha-based algorithm.