Network congestion control algorithm based on Actor-Critic reinforcement learning model

In this chapter, layer-2 hop-by-hop congestion control mechanisms for wireless mesh networks are studied. First, background on development of congestion control as a research area is presented. Then, congestion is defined and the causes of congestion in a mesh network are identified. The steps and challenges for development of a congestion control protocol are explored by dividing congestion control mechanism into three main functions: congestion detection, signaling, and resolution. Stability and fairness in the context of rate control are studied, and global stability in the presence of delay is identified as the main challenge faced by hop-by-bop congestion control algorithms in multihop networks. The congestion control framework in the IEEE 802.11 draft standard on mesh networking is then presented. The chapter concludes with thoughts for practitioners and areas for future research in this field.

The rapid evolution of computer networks, increase in the number of Internet users, and popularity of multimedia applications have exacerbated the congestion control problem. Congestion control is a key factor in ensuring network stability and robustness. When the underlying network and flow information are unknown, the transmission control protocol (TCP) must increase or reduce the size of the congestion window to adjust to the changes of traffic in the Internet Protocol (IP) network. However, it is possible that a software-defined approach can relieve the network congestion problem more efficiently. This approach has the characteristic of centralized control and can obtain a global topology for unified network management. In this paper, we propose a software-defined congestion control (SDCC) algorithm for an IP network. We consider the difference between TCP and the user datagram protocol (UDP) and propose a new method to judge node congestion. We initially apply the congestion control mechanism in the congested nodes and then optimize the link utilization to control network congestion.

Low link utility, RTT unfairness and unfairness of Multi-Bottleneck network are the existing problems in the present network congestion control algorithms at large. Through the analogy of network congestion control with the "El Farol Bar" problem, we establish a congestion control model based on minority game(MG), and then present a novel network congestion control algorithm based on the model. The result of simulations indicates that the proposed algorithm can make the achievements of link utility closing to 100%, zero packet lose rate, and small of queue size. Besides, the RTT unfairness and the unfairness of Multi-Bottleneck network can be solved, to achieve the max-min fairness in Multi-Bottleneck network, while efficiently weaken the "ping-pong" oscillation caused by the overall synchronization.

Traditional approach for data transmission in wireless networks with the shared medium, e.g., 802.11 standard family, incurs significant overheads to transmit messages in multiple source - multiple destination (MSMD) schemes. Network coding is new paradigm that increases effectiveness of transmissions in a wireless network with the shared medium and MSMD. We have considered the results of the emulation with COPE-like network coding and various Transmission Control Protocol (TCP) congestion control algorithms in wireless networks. In emulated scheme two nodes stream data to each other through bottleneck relay node. BBR TCP protocol showed the best results in this relaying scheme with enabled COPE-like network coding.

Deeper investigation is needed for several open issues related to congestion and admission control within the field of the high speed B‐ISDN network with ATM protocol. The system must be more effective in resource management and robust enough to cope with changes in traffic environment, which nowadays is characterised by sophisticated multimedia services. In this paper a new admission and congestion control strategy is proposed. It originates from the Fast Buffer Reservation (FBR) scheme, a virtual circuit acceptance algorithm proposed by Prof. Turner, and enhances its effectiveness, thus resulting in a complete approach for bandwidth management. The scheme is called Modified FBR (M‐FBR). The use of the FBR method, in fact, when based on the powerful Excess Probability Criterion (EPC), seemed to be impeded by the occurrence of lock out problems. By adopting a new kind of service at the buffer exit, the new mechanism offers the required flexibility and robustness, as well as ease of control, while preserving burst integrity and exhibiting a high burst‐served to burst‐sent ratio.

Researchers evaluate and contrast new network routing, admission control, congestion control and flow control algorithms through simulation. Analytically de-rived arguments justifiably lack credibility because, in the attempt to model the underlying physical system, the analyst is forced to make compromising approximations. However, unlike analytical techniques like Jackson Queueing Networks, simulations require significant computation and a simulation's state can consume a great deal of memory.This paper describes a technique that we developed to reduce the memory consumption of communication network simulators. Reduced memory makes simulations of larger and higher bandwidth-delay networks possible, but introduces an adjustable degree of approximation in the simulation. The higher the memory savings, the less accurate the computed measures. We call our technique Flowsim . The paper motivates the need to simulate computer networks rather than model them analytically, motivates why a simulator's state can grow quickly, and explains why analytical techniques have failed to model modern communication networks.

In the modern cellular data networks, with the tremendous amount of mobile devices, the constantly decreasing prices of the hardware and according the trends worldwide, provisioning of the data service and its quality will soon become an impossible task. Since current mobile cellular standards are unable to serve and meet the needs of this increase, the network will eventually become congested. As we are investigating data networks, they are based on the TCP protocol. TCP has various congestion control schemes used in both wired and wireless network, such as Cubic, Vegas, Reno, etc. Most modern cellular systems have already been migrated to 3G and 4G, so it could be very useful to identify which algorithm performs best in LTE (4G) wireless network. As most of the platforms for mobile devices, such as phones and tablets are unix/linux based and the current default algorithm in Linux is Cubic, the focus in this article will be on real-time modification and testing of the congestion control parameters of TCP CUBIC.

In the past few decades, many Transmission Control Protocol (TCP) congestion control algorithms have been investigated to meet the growing network demands and to enhance the performance of TCP in lossy or high-bandwidth-delay-product (high-BDP) networks. However, it is still challenging to implement a dynamic congestion control algorithm for wide range of diverse mobile users, network conditions and applications. This paper explores avenues for enhancement of TCP congestion control algorithm for next generation mobile networks by dynamically learning the available bandwidth and deriving the Congestion Control Factor N. N is used to Adaptive Increase/Adaptive Decrease (AIAD) the Congestion Window (CWND) dynamically instead of using the traditional approach that is Additive Increase/ Multiplicative Decrease (AIMD) paradigm. Once there is congestion, our proposed algorithm will not allow the CWND to decrease multiplicatively or steeply. After dropping to a certain level (lower than legacy), we try to take the CWND to previous state adaptively with the help of calculated bandwidth (based on learning). This in turn helps to efficiently control the CWND for better network utilization especially in case of lossy and high-BDP conditions. As soon as it reaches the original state, it remains stable for longer time as compared to legacy until packet loss or time out. We demonstrate the effectiveness of our algorithm with the help of live air experiments (performed in Samsung R&D India, Bangalore) and NS3 based simulation experiments. Through our experiments, we show that our algorithm outperforms the legacy congestion control algorithms (like CUBIC, RENO, TCPW) and the existing CLTCP algorithm in terms of goodput, intra algorithm fairness and inter algorithm fairness maintaining the scalability and friendliness.

RaceCC: A rapidly converging explicit congestion control for datacenter networks

Wireless networks are defined as that the network nodes which are connected by wireless data connections. The devices are connected via wireless link to communicate with each other. Measuring the delay performance of end-to-end nodes is the one of the most convoluted issues in wireless networks. In wireless networks, it is very difficult to maintain the substitution between throughput and delay. This paper proposes a new joint scheduling and congestion control algorithm for multihop wireless networks with dynamic route flows. The proposed algorithm achieves a provable throughput guarantee and provable end-to-end delay of every flow. The new joint scheduling and congestion control algorithm improves throughput and delay for dynamic wireless network by shifting scheduling scheme with virtual adaptation model. The proposed algorithm combines window-based flow control with a new rate based distributed scheduling algorithm to survive the quality of the system. This approach adaptively selects a set of routes according to the traffic load. Furthermore this dynamic adaptation mechanism achieves better concert in terms of throughput, end-to-end delay and packet delivery ratio.

Chapter 5 - Congestion Control in High-Speed Networks

Despite many years of improvements to it, TCP still suffers from an unsatisfactory performance. For services dominated by short flows (e.g., web search and e-commerce), TCP suffers from the flow startup problem and cannot fully utilize the available bandwidth in the modern Internet: TCP starts from a conservative and static initial window (IW, 2-4 or 10), while most of the web flows are too short to converge to the best sending rate before the session ends. For services dominated by long flows (e.g., video streaming and file downloading), the congestion control (CC) scheme manually and statically configured might not offer the best performance for the latest network conditions. To address these two challenges, we propose TCP-RL, which uses reinforcement learning (RL) techniques to dynamically configure IW and CC in order to improve the performance of TCP flow transmission. Basing on the latest network conditions observed at the server side of a web service, TCP-RL dynamically configures a suitable IW for short flows through group-based RL, and dynamically configures a suitable CC scheme for long flows through deep RL. Our extensive experiments show that for short flows, TCP-RL can reduce the average transmission time by about 23%; and for long flows, compared with the performance of 14 CC schemes, TCP-RL's performance ranks top 5 for about 85% of the 288 given static network conditions, whereas for about 90% of conditions, its performance drops by less than 12% compared with that of the best-performing CC schemes for the same network conditions.

The Internet has always done a remarkable job at surprising people. Technology oscillates, and some envisioned architectures of the future fail while others thrive. From the origination of peer-to-peer computing, it became easier task for communication and information transfer. But there exists still few hysterical issues like congestion, and security. Congestion occurs when one part of subnet becomes overloaded. The means of moderating the traffic in the networks is referred as congestion control. This paper critically analyses various issues behind congestion in various networks and various techniques to avoid and prevent from congestion and to efficiently control the congestion in the networks. This paper essentially focuses on the future networks posing various challenges and exposed issues regarding congestion control.

In wireless multimedia sensor networks (WMSNs) a sensor node may have different types of sensor which gather different kinds of data. To support quality of service (QoS) requirements for multimedia applications having a reliable and fair transport protocol is necessary. One of the main objectives of the transport layer in WMSNs is congestion control. We observe that the information provided may have different levels of importance and argue that sensor networks should be willing to spend more resources in disseminating packets carrying more important information. In WMSN event-driven applications, it is critical to report the detected events in the area, resulting in sudden bursts of traffic due to occurrence of spatially-correlated or multiple events, causing loss of data. Also, nodes have very limited power due to hardware constraints. Packet losses and retransmissions resulting from congestion, cost precious energy and shorten the lifetime of sensor nodes. Till now, in WMSNs, Congestion control techniques are based on detection of congestion and recovery, but they cannot eliminate or prevent the occurrence of congestion. Collision is a symptom of congestion in the wireless channel and can result in a time-variant channel capacity. The main intention of this protocol is to be used as a mechanism for reducing congestion in the network by free resources to set accurate rates and priority data needs. If two nodes send their packets in the shortest path to the parent node in a crowded place, a source node must prioritize the data and uses data that have lower priorities of a suitable detour nodes consisting of low or non- active consciously. Proposed algorithm is applied to the nodes near the base station (which convey more traffic) after the congestion detection mechanism detected the congestion. The results obtained from simulation test done by NS-2 simulator indicate that the proposed model is more innovative and presents better performance in compare with CCF, PCCP and DCCP protocols.

Congestion in Wireless Multimedia Sensor Networks (WMSN) may cause packet loss, buffer overflow and increases queueing delay, which has a significant negative impact on the network performance and reliability. This problem imposes more resource constraints that involve energy consumption and buffer size to meet additional requirements such as desired buffer size and limited amount of packet loss. One of the fundamental issues of congestion problem in such networks is the queue management. In fact, handling congestion without considering flexible buffer management and queueing methods, stay insignificant against dropping rate and packet loss. Consequently, a novel congestion control approach is needed to overcome these problems. In this paper, a new queue management mechanism called: Priority Queue Management based Explicit Congestion Control (PQMECC) is proposed and modeled using Stochastic Automata Networks, which are appropriate for modeling and analyzing complex systems with interacting components and complex synchronization. The congestion control technique used in PQMECC is based on priority, load repartition mechanism and explicit congestion notification which is generated only if the congestion reaches the threshold value.