UAV-Based Vehicle Detection and Tracking in Urban Environments Using Multi-Task CNN and Deep Reinforcement Learning

Congestion control is a fundamental network task that modulates the data transmission rates of traffic sources to efficiently utilize network capacity. With the advent of machine learning, congestion control based on deep reinforcement learning is the subject of extensive attention. At present, research on machine-learning-based congestion control is mainly focused on single-task scenarios -- to design a control strategy that is only used to reduce congestion. This article studies the congestion control problem based on machine learning in multi-task scenarios. Specifically, we propose a congestion control model based on multi-task deep reinforcement learning. The model takes congestion control as the main task and load balancing as the auxiliary task. Compared to the single-task method, our model can better represent the network environment by learning the shared representation of congestion features and load balancing features. Moreover, network traffic control may involve both congestion control and load balancing. Therefore, learning multiple tasks jointly while exploiting commonalities and differences across tasks can help reduce the cost of task coordination. We use software defined networking to decouple the data and control planes, making network control strategies more flexible than traditional networks. To the best of our knowledge, this is the first time multi-task learning has been applied to network traffic control. Experimental results show that the method is efficient.

Persistent target tracking in urban environments using UAV is a difficult task due to the limited field of view, visibility obstruction from obstacles and uncertain target motion. The vehicle needs to plan intelligently in 3D such that the target visibility is maximized. In this paper, we introduce Target Following DQN (TF-DQN), a deep reinforcement learning technique based on Deep Q-Networks with a curriculum training framework for the UAV to persistently track the target in the presence of obstacles and target motion uncertainty. The algorithm is evaluated through simulations. The results show that the UAV tracks the target persistently in diverse environments while avoiding obstacles on the trained environments as well as on unseen environments.

Distributed trajectory optimization for multiple solar-powered UAVs target tracking in urban environment by Adaptive Grasshopper Optimization Algorithm

In recent years, deep reinforcement learning (DRL) has made impressive achievements in many fields. However, existing DRL algorithms usually require a large amount of exploration to obtain a good action policy. In addition, in many complex situations, the reward function cannot be well designed to meet task requirements. These two problems will make it difficult for DRL to learn a good action policy within a relatively short period. The use of expert data can provide effective guidance and avoid unnecessary exploration. This study proposes a deep imitation reinforcement learning (DIRL) algorithm that uses a certain amount of expert demonstration data to speed up the training of DRL. In the proposed method, the learning agent imitates the expert's action policy by learning from demonstration data. After imitation learning, DRL is used to optimise the action policy in a self‐learning way. By experimental comparison on a video game called the Mario racing game, it is shown that the proposed DIRL algorithm with expert demonstration data can obtain much better performance than previous DRL algorithms without expert guidance.

Simultaneous monitoring of two comprehensive quality evaluation indexes of frozen-thawed beef meatballs using hyperspectral imaging and multi-task convolutional neural network

Deep reinforcement learning (DRL) is one of the emerging areas of machine learning which focuses on maximized rewards. DRL is a type of machine learning that combines reinforcement learning and deep learning. It uses a series of algorithms to enable an agent to learn how to make decisions in a complex environment. DRL is a subset of artificial intelligence that focuses on making decisions based on the environment and the rewards associated with each action.The goal of DRL is to maximize the long-term reward of an agent. In order to do this, the agent must use a combination of deep learning, reinforcement learning and other AI techniques to learn which actions will lead to the highest reward. DRL is used to solve a variety of problems, from playing video games to controlling robots. It is also used in autonomous driving and robotics, as well as for financial trading. DRL is a powerful tool for solving complex problems and has been used in a variety of research projects. DRL has the potential to revolutionize the way we interact with machines and the environment.

Convolutional neural network (CNN) based image steganalysis are increasingly popular because of their superiority in accuracy. The most straightforward way to employ CNN for image steganalysis is to learn a CNN-based classifier to distinguish whether secret messages have been embedded into an image. However, it is difficult to learn such a classifier because of the weak stego signals and the limited useful information. To address this issue, in this paper, a multi-task learning CNN is proposed. In addition to the typical use of CNN, learning a CNN-based classifier for the whole image, our multi-task CNN is learned with an auxiliary task of the pixel binary classification, estimating whether each pixel in an image has been modified due to steganography. To the best of our knowledge, we are the first to employ CNN to perform the pixel-level classification of such type. Experimental results have justified the effectiveness and efficiency of the proposed multi-task learning CNN.

Deep Reinforcement Learning (DRL) has emerged as a transformative paradigm in the field of artificial intelligence (AI), offering unprecedented capabilities in decision-making across diverse domains. This article explores the profound impact of DRL on enhancing the decision-making capabilities of AI systems, elucidating its underlying principles, applications, and implications.DRL represents a fusion of deep learning and reinforcement learning, enabling machines to learn complex behaviors and make decisions by interacting with their environment. The utilization of neural networks allows DRL algorithms to handle high-dimensional input spaces, making it well-suited for tasks that involve intricate decision-making processes.One of the key strengths of DRL lies in its ability to address problems with sparse and delayed rewards, common challenges in traditional reinforcement learning. Through a process of trial and error, DRL algorithms can learn optimal decision strategies by navigating through a vast decision space, adapting to dynamic environments, and maximizing cumulative rewards over time.The applications of DRL span various domains, including robotics, finance, healthcare, gaming, and autonomous systems. In robotics, DRL facilitates the development of intelligent agents capable of autonomously navigating complex environments, performing intricate tasks, and adapting to unforeseen circumstances. In finance, DRL is leveraged for portfolio optimization, algorithmic trading, and risk management, demonstrating its potential to revolutionize traditional financial strategies.

A new ensemble deep graph reinforcement learning network for spatio-temporal traffic volume forecasting in a freeway network

This paper discusses an autonomous driving control strategy based on Deep Reinforcement Learning (DRL), which aims to improve the decision-making ability of autonomous driving system in complex traffic environments. Deep reinforcement learning has a wide range of applications in many fields, such as robotics and medicine. Autonomous driving has emerged as a significant research focus in recent years. By combining deep learning and reinforcement learning, the model is able to autonomously learn and optimize driving behavior under dynamically changing road conditions. The DRL-based control strategy performs well in vehicle obstacle avoidance, pedestrian recognition, and traffic rule compliance in the face of complex environments such as city streets, intersections, and congested road sections, significantly improving the safety and efficiency of autonomous driving. This article will first introduce deep reinforcement learning. Then, the autonomous driving control strategy based on deep reinforcement learning is introduced. This research provides valuable insights for developing and implementing DRL-based autonomous driving systems.

Radar-based road-traffic monitoring in urban environments

Multi-UAVs tracking target in urban environment by model predictive control and Improved Grey Wolf Optimizer

Learning ability is the basic characteristic of human intelligence. The July 1, 2005 issue of Science published a list of 125 important questions in science. Among them, the question 94 “What are the limits of learning by machines?”. The annotation “Computers can already beat the world’s best chess players, and they have a wealth of information on the Web to draw on. But abstract reasoning is still beyond any machine”. In recent artificial intelligence has made great progresses. In 1997, the rise of the man-machine war, IBM Supercomputer Deep Blue defeated the chess master Garry Kasparov. On February 14, 2011, IBM’s Watson supercomputer won a practice round against Jeopardy champions Ken Jennings and Brad Rutter. In March 2016, Google DeepMind’s AlphaGo sealed a 4-1 victory over a South Korean Go grandmaster Lee Se-dol. This paper focuses on the machine learning methods of AlphaGo, including reinforcement learning, deep learning, deep reinforcement learning, analysis of the existing problems and the latest research progress. Deep reinforcement learning is the combination of deep learning and reinforcement learning, which can realize the learning algorithm from the perception to action. Simply said, this is the same as human behavior, input sensing information such as vision, and then, direct output action through the deep neural network. Deep reinforcement learning has the potential to learn a variety of skills for the robot to achieve full autonomy. Even though reinforcement learning is practiced successfully, but feature states need to manually set, for complex scene is a difficult thing, especially easy to cause the dimension disaster, and expression is not good. In 2010, Sascha Lange and Martin Riedmiller proposed deep auto-encoder neural networks in reinforcement learning to extract feature, which is used to control the visual correlation. In 2013, DeepMind proposed deep Q-network (DQN) in NIPS 2013, using convolution neural network to extract features, and then applied in reinforcement learning. They continue to improve and published an improved version of DQN on Nature in 2015, which has aroused widespread concern. In order to break through the limits of learning by machines, cognitive machine learning is proposed, which is the combination of machine learning and brain cognition, so that the machine intelligence is constantly evolving, and gradually reaches the human level of artificial intelligence. A cognitive model entitled Consciousness And Memory (CAM) is proposed by author, which consists of memory, consciousness, high-level cognitive functions, perception and motor. High-level cognitive functions of the brain include learning, language, thinking, decision making, emotion, and so on. Learning is a course to accept the stimulus through the nervous system and obtain new behavior, habits and accumulation experience. According to the current research progress of brain science and cognitive science, cognitive machine learning may be interested in learning emergence, procedural memory knowledge learning, learning evolution and so on. For intelligence, so-called evolution is refers to the learning of learning and the structure also follows the change. It is important to record the learning result by structure changing and improve the learning method.

Radar systems play a crucial role in target tracking within urban environments, where challenges such as clutter, multipath effects, and electromagnetic interference significantly impact detection accuracy. Traditional radar methods often struggle to adapt to dynamic urban conditions, leading to decreased reliability in real-time target tracking. This study aims to develop and evaluate an AI-driven adaptive radar system that enhances tracking accuracy in urban settings. The research employs a quantitative approach using simulations to model radar signal processing under various environmental conditions. The AI model, based on Convolutional Neural Networks (CNN), is trained to optimize radar performance by filtering out noise and dynamically adjusting detection parameters. The results indicate that the AI-based radar system achieves a tracking accuracy of 95.2%, significantly outperforming traditional radar systems, which only reach 80% accuracy. Additionally, the AI-enhanced radar reduces response time to 120 milliseconds, compared to 250 milliseconds in conventional systems, demonstrating improved real-time processing capabilities. The system also exhibits greater resilience to high-clutter environments, maintaining stable target detection despite signal interference. These findings highlight the potential of AI in enhancing radar functionality for applications such as surveillance, traffic monitoring, and security. Future research should focus on integrating AI-driven radar with real-world radar hardware, exploring multi-sensor fusion, and refining adaptive learning techniques to further optimize tracking performance in complex environments

A distributed approach is proposed for planning a cooperative tracking task for a team of unmanned aerial vehicles (UAVs). In the scenario of interest UAVs are required to autonomously track, using their onboard sensors, a moving target in a known urban environment. The solution methodology involves finding visibility regions, from which a UAV can maintain a line of sight to the target during the scenario; and restricted regions, in which a UAV can not fly, due to the presence of buildings or other airspace limitations. A co-evolution genetic algorithm is derived for searching, in realtime, monotonically improving solutions. In the proposed distributed search method every UAV iteratively manipulates its own population of chromosomes, each encoding its control inputs in the calculated horizon. Team performance is attained by assigning fitness to each solution in the population based on the cooperative performance when using it together with preceding iteration tracking information obtained from teammates. Important attributes of the proposed solution approach are its scalability and robustness; and consequently it can be applied to large sized problems and adapt to the loss of UAV team members. The distributed nature of the algorithm also reduces the computation and communication loads. The performance of the algorithm is studied using a high fidelity simulation test-bed incorporating a visual database of the city of Tel-Aviv, Israel.