Synchronous Deep Reinforcement Learning (SDRL) Algorithm For Small Batch Image Recognition

With the rapid development of urban rail transit in my country, the safety inspection of subway tunnel infrastructure is in urgent need of automated and intelligent theory and technology research and development. The subway tunnel lining environment is bad, there are a lot of noise and uneven illumination, and the texture features are complex. The traditional manual and image detection and recognition methods have gradually been replaced by more intelligent and effective deep learning algorithms. Therefore, this paper proposes a subway tunnel lining image recognition and detection algorithm that combines image processing and deep convolutional networks, which can quickly and accurately distinguish between cracked images and non-cracked images, and can realize intelligent target search and category labeling in complex images. In this experiment, a total of 4000 original images of subway tunnels were selected, and pixel-level shallow processing operations such as batch preprocessing, hierarchical feature analysis, and extraction of the circumscribed rectangle of the connected area were carried out. For the training sample library of crack images, pixel-level image processing experiments and classification and recognition experiments are carried out for the training sample library to obtain the optimal database processing results, and then the image recognition algorithm for complex cracks in subway tunnels is studied. Experiments show that there are more point noises and block noises in the results of the subway tunnel crack images, and the segmentation results of the crack images have relatively less noise residues. Due to the wall material, some cement materials are a little off, resulting in noise retention. This shows that the algorithm in this paper has a certain application value in the intelligent identification of actual subway tunnel diseases, and provides technical support and research ideas for the application of subsequent deep learning in subway tunnel disease detection.

Remote sensing includes all types of non-contact. surveys that are carried out from various measuring platforms. The tasks in this area are the following: inventory of agricultural land, control of the state of crops, forecasting yields. The aim of the work is to classify 6 types of crop images (wheat, rice, sugarcane, corn, cotton and jute) with greater accuracy. The paper considers an algorithm for primary processing and recognition of images of agricultural crops and algorithms for constructing a neural network for initial processing and recognition of images to solve problems of noise elimination, minimization, smoothing, normalization, segmentation and image recognition.

Nowadays, the information processing capabilities and resource storage capabilities of computers have been greatly improved, which also provides support for the neural network technology. Convolutional neural networks have good characterization capabilities in computer vision tasks, such as image recognition technology. Aiming at the problem of high similarity image recognition and classification in a specific field, this paper proposes a high similarity image recognition and classification algorithm fused with convolutional neural networks. First, we extract the image texture features, train different types, and different resolution image sets and determine the optimal texture different parameter values. Second, we decompose the image into subimages according to the texture difference, extract the energy features of each subimage, and perform classification. Then, the input image feature vector is converted into a one-dimensional vector through the alternating 5-layer convolution and 3-layer pooling of convolutional neural networks. On this basis, different sizes of convolution kernels are used to extract different convolutions of the image features, and then use convolution to achieve the feature fusion of different dimensional convolutions. Finally, through the increase in the number of training and the increase in the amount of data, the network parameters are continuously optimized to improve the classification accuracy in the training set and in the test set. The actual accuracy of the weights is verified, and the convolutional neural network model with the highest classification accuracy is obtained. In the experiment, two image data sets of gems and apples are selected as the experimental data to classify and identify gems and determine the origin of apples. The experimental results show that the average identification accuracy of the algorithm is more than 90%.

Multi-agent reinforcement learning dealing with hybrid action spaces: A case study for off-grid oriented renewable building energy system

Advancing biomedical waste classification through a hybrid ensemble of deep Learning, reinforcement Learning, and differential evolution algorithms.

Using Deep Reinforcement Learning (DRL) algorithm to deal with autonomous driving tasks usually have unsatisfied performance due to lack of robustness and means to escape local optimum. In this article, we designs a Survival-Oriented Reinforcement Learning (SORL) model that tackle these problems by setting survival rather than maximize total reward as first priority. In SORL model, we model autonomous driving task as Constrained Markov Decision Process (CMDP) and introduce Negative-Avoidance Function to learn from previous failure. The SORL model greatly speed up the training process and improve the robustness of normal Deep Reinforcement Learning algorithm.

Problem definition: The beer game is widely used in supply chain management classes to demonstrate the bullwhip effect and the importance of supply chain coordination. The game is a decentralized, multiagent, cooperative problem that can be modeled as a serial supply chain network in which agents choose order quantities while cooperatively attempting to minimize the network’s total cost, although each agent only observes local information. Academic/practical relevance: Under some conditions, a base-stock replenishment policy is optimal. However, in a decentralized supply chain in which some agents act irrationally, there is no known optimal policy for an agent wishing to act optimally. Methodology: We propose a deep reinforcement learning (RL) algorithm to play the beer game. Our algorithm makes no assumptions about costs or other settings. As with any deep RL algorithm, training is computationally intensive, but once trained, the algorithm executes in real time. We propose a transfer-learning approach so that training performed for one agent can be adapted quickly for other agents and settings. Results: When playing with teammates who follow a base-stock policy, our algorithm obtains near-optimal order quantities. More important, it performs significantly better than a base-stock policy when other agents use a more realistic model of human ordering behavior. We observe similar results using a real-world data set. Sensitivity analysis shows that a trained model is robust to changes in the cost coefficients. Finally, applying transfer learning reduces the training time by one order of magnitude. Managerial implications: This paper shows how artificial intelligence can be applied to inventory optimization. Our approach can be extended to other supply chain optimization problems, especially those in which supply chain partners act in irrational or unpredictable ways. Our RL agent has been integrated into a new online beer game, which has been played more than 17,000 times by more than 4,000 people.

Reinforcement Learning (RL) has become one of the most powerful methods in Artificial Intelligence (AI) which allows a system to learn as it is engaged in an interaction with its environment and to maximize its actions by performing a trial and error process. Although classical RL algorithms can give us a fundamental idea on RL, like the Q-Learning, its usefulness wanes in high-dimensional or continuous-state spaces. To overcome these shortcomings, Deep Reinforcement Learning (DRL) algorithms including Deep Q-Networks (DQN) build on neural networks to learn action-value functions, allowing horizontal scalability and more efficient learning. This paper reports on the experimental comparison of Q-Learning and DQN based on the environment CartPole-v1. Both of the algorithms were trained in the controlled environment and tested under various metrics that include cumulative rewards, success rate, convergence speed, safety (failure rate), and path efficiency. Findings indicate that Q-Learning moves towards moderate stability with slower convergence but DQN takes shorter learning time, is more reliable and performs better. Such results indicate the ground-breaking potential of the DRL in the progress of AI toward transcending classical RL restrictions. Future research topics encompass implementation of more challenging experimental settings, a test of other DRL algorithms (e.g., PPO, SAC), and generalization, scalability and safety in real-world environments.

<span>The utilization of games in training the reinforcement learning (RL) agent is to describe the complex and high-dimensional real-world data. By utilizing games, RL researchers will be able to evade high experimental costs in training an agent to do intelligence tasks. The objective of this research is to generate intelligent agent behaviors in multi-agent game artificial intelligence (AI) using deep reinforcement learning (DRL) algorithm. A basic RL algorithm called deep Q network is chosen to be implemented. The agent is trained by the environment's raw pixel images and the action list information. The experiments conducted by using this algorithm show the agent’s decision-making ability in choosing a favorable action. In the default setting for the algorithm, the training is set into 1 epoch and 0.0025 learning rate. The number of training iterations is set to one as the training function will be repeatedly called for every 4-timestep. However, the author also experimented with two different scenarios in training the agent and compared the results. The experimental findings demonstrate that our agents learn correctly and successfully while actively participating in the game in real time. Additionally, our agent can quickly adjust against a different enemy on a varied map because of the observed knowledge from prior training.</span>

Deep Reinforcement Learning (DRL) algorithm provides an effective and powerful way for the computer learning to make decisions according to what the computer sees, the most famous application of DRL is training the computer to play computer games, such as the computer Go and Atari Games. In this paper, we present that the DRL can used to improve the classical PID controller by training an adaptive PID controller with its parameters change according to the changing state, we name our new controller as DRPID, the deep reinforcement learning algorithm that we used to train the PID controller in this paper is deep deterministic policy gradient, which is very suitable to solve continue action control problem. We test our DRPID algorithm by training it to control an inverted pendulum in the OpenAI gym simulation environment, and we found DRPID controller works very well, outperforms the common PID controller with fixed parameters, by a great margin.

Deep convolutional neural networks have achieved great success in recent years. They have been widely used in various applications such as optical and SAR image scene classification, object detection and recognition, semantic segmentation, and change detection. However, deep neural networks rely on large-scale high-quality training data, and can only guarantee good performance when the training and test data are independently sampled from the same distribution. Deep convolutional neural networks are found to be vulnerable to subtle adversarial perturbations. This adversarial vulnerability prevents the deployment of deep neural networks in security-sensitive applications such as medical, surveillance, autonomous driving and military scenarios. This paper first presents a holistic view of security issues for deep convolutional neural network-based image recognition systems. The entire information processing chain is analyzed regarding safety and security risks. In particular, poisoning attacks and evasion attacks on deep convolutional neural networks are analyzed in detail. The root causes of adversarial vulnerabilities of deep recognition models are also discussed. Then, we give a formal definition of adversarial robustness and present a comprehensive review of adversarial attacks, adversarial defense, and adversarial robustness evaluation. Rather than listing existing research, we focus on the threat models for the adversarial attack and defense arms race. We perform a detailed analysis of several representative adversarial attacks on SAR image recognition models and provide an example of adversarial robustness evaluation. Finally, several open questions are discussed regarding recent research progress from our workgroup. This paper can be further used as a reference to develop more robust deep neural network-based image recognition models in dynamic adversarial scenarios.

In portable precision visual measurement, processing result of the measurement image has an important influence on final measurement accuracy in large field working space. However, the images taken in large field working space always have the following characteristics: containing a large amount of information, having a high redundancy and including a lot of background noises. In order to solve these problems, precision image processing and recognition algorithm is researched. Firstly, portable precision visual measurement method is introduced. Then, the design method of global control points including noncoding points and coding points is displayed. Finally, the image processing and recognition algorithm is investigated in detail, including the largest variance threshold method used to partly binary the image, the detection algorithm of global control points based on the constraint of distance and acreage and coding value recognition method through correcting the image feature. A large amount of experiments show that the image processing and recognition algorithm is accurate and effective to processing the image taken in the working space.

The online 3D bin packing problem(3D-BPP) is widely used in the logistics industry and is of great practical significance for promoting the intelligent transformation of the industry. The heuristic algorithm relies too much on manual experience to formulate more perfect packing rules. In recent years, many scholars solve 3D-BPP via deep reinforcement learning(DRL) algorithms. However, they ignore many skills used in manual packing, one of the most important skill is workers put the item aside if the item is packed improperly. Inspired by this skill, we propose a DRL algorithm with a buffer zone. Firstly, we define the wasted space and the buffer zone. And then, we integrate them into the DRL algorithm framework. Importantly, we compare the bin utilization with di erent thresholds of wasted space and di erent buffer zone sizes. Experimental results show that our algorithm outperforms existing heuristic algorithms and DRL algorithms.

The continuously changing nature of stock markets poses a non-trivial challenge to build automated trading agents that timely assist traders in their decision makings. Fixed sets of trading rules and offline pre-trained models are inefficient to adapt regarding the real-time fluctuations on the stock markets. Deep Reinforcement Learning (DRL) algorithms train autonomous agents that could potentially address such highly dynamic environments by integrating the generalization power of artificial neural networks with the online learning through experiences gained in the interactions with the environment. We investigate the capability of three DRL algorithms, namely Deep Deterministic Policy Gradient (DDPG), Twin Delayed DDPG (TD3), and Soft Actor-Critic (SAC), for tackling the automated stock trading problem. We employ two datasets, from the US and Vietnam markets, with different characteristics and market trends, on which the DRL-trained agents are compared against a random trading agent. Experimental results indicate the potentials of these DRL algorithms and also exhibit their pitfalls when being applied in datasets where the majority of the stocks are not up-trending. We propose a simple, but effective, technique to assist the agents to minimize their losses.

As vehicular technology advances, intelligent vehicles generate numerous computation‐intensive tasks, challenging the computational resources of both the vehicles and the Internet of Vehicles (IoV). Traditional IoV struggles with fixed network structures and limited scalability, unable to meet the growing computational demands and next‐generation mobile communication technologies. In congested areas, near‐end Mobile Edge Computing (MEC) resources are often overtaxed, while far‐end MEC servers are underused, resulting in poor service quality. A novel network framework utilizing sixth‐generation mobile communication (6G) and digital twin technologies, combined with task migration, promises to alleviate these inefficiencies. To address these challenges, a task migration and re‐offloading model based on task attribute classification is introduced, employing a hybrid deep reinforcement learning (DRL) algorithm—Dueling Double Q Network DDPG (QDPG). This algorithm merges the strengths of the Deep Deterministic Policy Gradient (DDPG) and the Dueling Double Deep Q‐Network (D3QN), effectively handling continuous and discrete action domains to optimize task migration and re‐offloading in IoV. The inclusion of the Mini Batch K‐Means algorithm enhances learning efficiency and optimization in the DRL algorithm. Experimental results show that QDPG significantly boosts task efficiency and computational performance, providing a robust solution for resource allocation in IoV.