Improving Cognitive Skills with a Multi-Agent DQN: A MATLAB Implementation

Introduction: Current school teaching methods including distance learning may have adverse health effects in children. Objective: To assess health effects of online education in students of comprehensive secondary schools and gymnasiums. Materials and methods: We developed a questionnaire to establish the impact of distance learning on the functional state and health of students comprising three sections: educational activities, daily routine and self-rated health. In May 2020, 3,026 fifth to ninth-year students from eight comprehensive schools and five gymnasiums of the city of Nizhny Novgorod, including 1,430 boys and 1,596 girls, filled out the questionnaire and their answers were then analyzed using IBM SPSS Statistics software, version 5.1. Results: Distance learning included online and offline classes (70.7 % in total) and self-study assignments, the latter being more common in the comprehensive schools than in the gymnasiums. The majority of the respondents from the schools (73.4 %) and gymnasiums (67.0 %) noted a greater amount of homework during e-learning compared to offline education. They also reported less time spent outdoors and much more passive leisure. Complaints of increased back and wrist pain and headaches were more frequent in the students of comprehensive secondary schools. Conclusions: Our findings show that distance learning changed the way of life of all the children surveyed. We observed an increased screen time, the prevalence of static activities, and the development of musculoskeletal and eye disorders. The comprehensive school students noted more negative consequences of online learning compared to their peers studying in the gymnasiums.

To solve problems involving sequential decision-making, Deep Reinforcement Learning (DRL) combines the advantages of Reinforcement Learning (RL) and Deep Learning (DL).These fusions are skilled at training agents for video games because they use neural networks to approximate nonlinear functions.The Deep Q Network (DQN), a key algorithm in this field, tends to overestimate Q-values despite its effectiveness.The Double Deep Q Network (Double DQN) was presented to address this.The Kera's framework in PyCharm is used in this study to examine the practical application and comparative analysis of DQN and Double DQN in the Flappy Bird game.To speed up its training, improvements were also made to the DQN model.The enhanced DQN performed better than the traditional DQN but less well than the Double DQN, according to the results.This study delves into the practical application and comparative analysis of DQN and Double DQN in the Flappy Bird game, utilizing the Kera's framework in PyCharm.Additionally, enhancements were made to the DQN model to expedite its training.Results affirmed that the enhanced DQN outperformed the conventional DQN but lagged the Double DQN.The training loss trajectory further substantiated Double DQN's superiority in mitigating overestimation issues.

Learning ability is the basic characteristic of human intelligence. The July 1, 2005 issue of Science published a list of 125 important questions in science. Among them, the question 94 “What are the limits of learning by machines?”. The annotation “Computers can already beat the world’s best chess players, and they have a wealth of information on the Web to draw on. But abstract reasoning is still beyond any machine”. In recent artificial intelligence has made great progresses. In 1997, the rise of the man-machine war, IBM Supercomputer Deep Blue defeated the chess master Garry Kasparov. On February 14, 2011, IBM’s Watson supercomputer won a practice round against Jeopardy champions Ken Jennings and Brad Rutter. In March 2016, Google DeepMind’s AlphaGo sealed a 4-1 victory over a South Korean Go grandmaster Lee Se-dol. This paper focuses on the machine learning methods of AlphaGo, including reinforcement learning, deep learning, deep reinforcement learning, analysis of the existing problems and the latest research progress. Deep reinforcement learning is the combination of deep learning and reinforcement learning, which can realize the learning algorithm from the perception to action. Simply said, this is the same as human behavior, input sensing information such as vision, and then, direct output action through the deep neural network. Deep reinforcement learning has the potential to learn a variety of skills for the robot to achieve full autonomy. Even though reinforcement learning is practiced successfully, but feature states need to manually set, for complex scene is a difficult thing, especially easy to cause the dimension disaster, and expression is not good. In 2010, Sascha Lange and Martin Riedmiller proposed deep auto-encoder neural networks in reinforcement learning to extract feature, which is used to control the visual correlation. In 2013, DeepMind proposed deep Q-network (DQN) in NIPS 2013, using convolution neural network to extract features, and then applied in reinforcement learning. They continue to improve and published an improved version of DQN on Nature in 2015, which has aroused widespread concern. In order to break through the limits of learning by machines, cognitive machine learning is proposed, which is the combination of machine learning and brain cognition, so that the machine intelligence is constantly evolving, and gradually reaches the human level of artificial intelligence. A cognitive model entitled Consciousness And Memory (CAM) is proposed by author, which consists of memory, consciousness, high-level cognitive functions, perception and motor. High-level cognitive functions of the brain include learning, language, thinking, decision making, emotion, and so on. Learning is a course to accept the stimulus through the nervous system and obtain new behavior, habits and accumulation experience. According to the current research progress of brain science and cognitive science, cognitive machine learning may be interested in learning emergence, procedural memory knowledge learning, learning evolution and so on. For intelligence, so-called evolution is refers to the learning of learning and the structure also follows the change. It is important to record the learning result by structure changing and improve the learning method.

To decrease the impact of Partially Observable MDP on deep reinforcement learning performance of intersection signal control, A deep reinforcement learning is proposed in this paper with utilizing the real-time GPS data as well as learning the control of the traffic lights in single intersection. We integrate deep reinforcement learning network (DRQN) with recurrent neural network (RNN) and apply deep network, experience pool and greedy strategy in deep reinforcement learning strategy. It solves the problem of overestimation of target Q value and insufficient long-term experience learning in the standard reinforcement learning of traffic signal control. The comparison of performance was made between the proposed method and standard Deep Q-Network (DQN) on the partial observation of traffic situations. The experimental results show that both DQN and DRQN methods can adjust their traffic signal timing control strategies according to the specific traffic conditions as well as calculating a lower average delay time of vehicle than that of fixed-time control. Besides, the simulation effect of DRQN learning method is better than that of DQN learning method in different probe vehicle proportion environment.

Reinforcement learning is a mathematical framework for agents to interact intelligently with their environment. Unlike supervised learning, where a system learns with the help of labeled data, reinforcement learning agents learn how to act by trial and error only receiving a reward signal from their environments. A field where reinforcement learning has been prominently successful is robotics [3]. However, real-world control problems are also particularly challenging because of the noise and high- dimensionality of input data (e.g., visual input). In recent years, in the field of supervised learning, deep neural networks have been successfully used to extract meaning from this kind of data. Building on these advances, deep reinforcement learning was used to solve complex problems like Atari games and Go. Mnih et al. [1] built a system with fixed hyper parameters able to learn to play 49 different Atari games only from raw pixel inputs. However, in order to apply the same methods to real-world control problems, deep reinforcement learning has to be able to deal with continuous action spaces. Discretizing continuous action spaces would scale poorly, since the number of discrete actions grows exponentially with the dimensionality of the action. Furthermore, having a parametrized policy can be advantageous because it can generalize in the action space. Therefore with this thesis we study state-of-the-art deep reinforcement learning algorithm, Deep Deterministic Policy Gradients. We provide a theoretical comparison to other popular methods, an evaluation of its performance, identify its limitations and investigate future directions of research. The remainder of the thesis is organized as follows. We start by introducing the field of interest, machine learning, focusing our attention of deep learning and reinforcement learning. We continue by describing in details the two main algorithms, core of this study, namely Deep Q-Network (DQN) and Deep Deterministic Policy Gradients (DDPG). We then provide implementatory details of DDPG and our test environment, followed by a description of benchmark test cases. Finally, we discuss the results of our evaluation, identifying limitations of the current approach and proposing future avenues of research.

With the rapid development of AI technology, personalized learning path planning is becoming an important research field. Dual system teaching models have great influence on the learning effect of students. Therefore, how to optimize the learning process of students through the personalized learning path is a great challenge. A Deep Reinforcement Learning (DRL) algorithm for personalized learning path planning is proposed. Based on deep Q-networks and reinforcement learning feedback mechanisms, the algorithm achieves adaptive optimization. The experimental results show that the algorithm based on the DRL is superior to the traditional method in learning performance, learning efficiency and satisfaction. Compared with traditional path planning algorithms based on rules and collaborative filtering, the proposed algorithm improves learning performance by 20%. Moreover, it has obvious advantages in learning motivation and efficiency. This paper provides a new way of thinking to intelligent education system, especially dual-system teaching model, and it is of great practical significance.

Waldorf Education follows a holistic approach of children’s development, where the fundamental characteristics are creative/artistic activities, integrating imagination-based teaching methods to support and enhance the development of children’s and adolescents’ physical, social, emotional, and cognitive skills. Neuroeducation provides the most relevant level of analysis for resolving today’s core problems in education. Multiple Intelligence (MI) theory investigates ways of using the theory as a framework in school for improving work quality, collaborations, opportunities for choice, and a role for the arts. To that end, we provide a systematic literature review that critiques and synthesizes representative literature on these three topics in order to reveal new perspectives towards a novel transformative educational paradigm in a digitized society. A comprehensive analysis of theoretical and empirical articles between 2000 and 2019 is provided. The search included five main academic databases (ERIC, Web of Science, ScienceDirect, SpringerLink, and Scopus) using predefined selection criteria. In total, 321 different articles were screened, from which 43 articles met the predefined inclusion criteria. The results indicate a correlation between pedagogical practices of Waldorf schools and MI theory compatible teaching practices and between Waldorf schools and neuroeducation. Further empirical research examining different facets of this relationship is still needed to establish live and effective schools as Learning Organizations.

Early action prediction in video is a challenging task where the action of a human performer is expected to be predicted using only the initial few frames. We propose a novel technique for action prediction based on Deep Reinforcement learning, employing a Deep Q-Network (DQN) and the ResNext as the basic CNN architecture. The proposed DQN can predict the actions in videos from features extracted from the first few frames of the video, and the basic CNN model is adjusted by tuning the hyperparameters of the CNN network. The ResNext model is adjusted based on the reward provided by the DQN, and the hyperparameters are updated to predict actions. The agent's stopping criteria is higher or equal to the validation accuracy value. The DQN is rewarded based on the sequential input frames and the transition of action states (i.e., prediction of action class for an incremental 10 percent of the video). The visual features extracted from the first 10 percent of the video is forwarded to the next 10 percent of the video for each action state. The proposed method is tested on the UCF101 dataset and has outperformed the state-of-the-art in action prediction.

Path planning in an unknown environment is a basic task for mobile robots to complete tasks. As a typical deep reinforcement learning, deep Q-network (DQN) algorithm has gained wide popularity in path planning tasks due to its self-learning and adaptability to complex environment. However, most of path planning algorithms based on DQN spend plenty of time for model training and the learned model policy depends only on the information observed by sensors. It will cause poor generalization capability for the new task and time waste for model retraining. Therefore, a new deep reinforcement learning method combining DQN with prior knowledge is proposed to reduce training time and enhance generalization capability. In this method, a fuzzy logic controller is designed to avoid the obstacles and help the robot avoid blind exploration for reducing the training time. A target-driven approach is used to address the lack of generalization, in which the learned policy depends on the fusion of observed information and target information. Extensive experiments show that the proposed algorithm converges faster than DQN algorithm in path planning tasks and the target can be reached without retraining when the path planning task changes.

Reinforcement Learning (RL) has become one of the most powerful methods in Artificial Intelligence (AI) which allows a system to learn as it is engaged in an interaction with its environment and to maximize its actions by performing a trial and error process. Although classical RL algorithms can give us a fundamental idea on RL, like the Q-Learning, its usefulness wanes in high-dimensional or continuous-state spaces. To overcome these shortcomings, Deep Reinforcement Learning (DRL) algorithms including Deep Q-Networks (DQN) build on neural networks to learn action-value functions, allowing horizontal scalability and more efficient learning. This paper reports on the experimental comparison of Q-Learning and DQN based on the environment CartPole-v1. Both of the algorithms were trained in the controlled environment and tested under various metrics that include cumulative rewards, success rate, convergence speed, safety (failure rate), and path efficiency. Findings indicate that Q-Learning moves towards moderate stability with slower convergence but DQN takes shorter learning time, is more reliable and performs better. Such results indicate the ground-breaking potential of the DRL in the progress of AI toward transcending classical RL restrictions. Future research topics encompass implementation of more challenging experimental settings, a test of other DRL algorithms (e.g., PPO, SAC), and generalization, scalability and safety in real-world environments.

Deep reinforcement learning has attracted attention for its application in deep Q-networks (DQNs). A DQN can attain superhuman performance, but it requires a large number of trial-and-error searches. To reduce the number of trial-and-error searches required for learning convergence in a DQN, multistep learning can be used and deep Q-networks with profit sharing (DQNwithPS) is one solution, but it has its own set of disadvantages. DQNwithPS optimizes a neural network by learning based on a DQN and profit sharing. However, multistep learning requires proper prefetching parameter tuning, and DQNwithPS has a learning performance degradation problem caused by profit sharing by not considering the expected rewards of a future episode. In this paper, we propose a learning-accelerated DQN combining multistep learning and DQNwithPS to cancel each disadvantage. The proposed method improves the prefetching parameter tuning in multistep learning by DQNwithPS and the learning performance degradation problem in DQNwithPS by multistep learning. By this method, we aim to reduce the number of trial-and-error searches compared to a DQN and DQNwithPS and to realize a manageable fast learning method.

Choosing the right parameter configurations for recurring jobs running on big data analytics platforms is difficult because there can be hundreds of possible parameter configurations to pick from. Even the selection of parameter configurations is based on different types of applications and user requirements. The difference between the best configuration and the worst configuration can have a performance impact of more than 10 times. However, parameters of big data platforms are not independent, which makes it a challenge to automatically identify the optimal configuration for a broad spectrum of applications. To alleviate these problems, we proposed MonkeyKing, a system that leverages past experience and collects new information to adjust parameter configurations of big data platforms. It can recommend key parameters, which have strong impact on performance according to job types, and then combine deep reinforcement learning (DRL) to optimize key parameters to improve job performance. We choose the current popular deep Q-network (DQN) structure and its four improved algorithms, including DQN, Double DQN, Dueling DQN, and the combined Double DQN and Dueling DQN, and finally found that the combined Double DQN and Dueling DQN has a better effect. Our experiments and evaluations on Spark show that performance can be improved by ∼25% under best conditions.

The transition to sustainable and intelligent energy systems has intensified the development of smart microgrids, which offer decentralized, resilient, and efficient power solutions. This review critically examines the integration of Artificial Intelligence (AI) and Deep Reinforcement Learning (DRL) into smart microgrid platforms, focusing on their role in optimizing sustainable energy management. Traditional energy management systems often struggle to adapt to the dynamic nature of modern energy demands, renewable energy intermittency, and grid complexity. AI-driven solutions, particularly DRL, provide adaptive, autonomous, and data-driven mechanisms for real-time decision-making and predictive control within microgrids. DRL, by learning optimal policies through interaction with the environment, is capable of handling multi-objective problems, including demand-response optimization, energy storage control, load forecasting, and distributed generation scheduling. This paper synthesizes recent advancements and applications of DRL algorithms such as Deep Q-Networks (DQN), Deep Deterministic Policy Gradient (DDPG), and Proximal Policy Optimization (PPO) in smart microgrids. It also explores hybrid models that combine DRL with other AI techniques, such as fuzzy logic and neural networks, to improve performance under uncertainty and nonlinearity. Furthermore, the review evaluates benchmark testbeds, simulation tools, and real-time platforms used to implement and validate these intelligent systems. Challenges such as high computational costs, model generalizability, real-time implementation issues, and cybersecurity vulnerabilities are discussed. The study also highlights recent research trends emphasizing decentralized control, edge computing, and federated learning to enhance scalability and privacy in DRL-based microgrid applications. Future directions suggest the need for explainable AI (XAI), robust training environments, and standardization to facilitate the wider adoption of DRL in real-world microgrid systems. The integration of DRL into smart microgrids represents a transformative shift toward resilient, efficient, and sustainable energy ecosystems. This review offers a comprehensive understanding of how AI and DRL are revolutionizing energy management, providing a foundation for future research and practical deployment in smart energy infrastructures.

Humans have to respond to everyday tasks with goal-directed actions in complex and time-continuous environments. However, modeling human behavior in such environments has been challenging. Deep Q-networks (DQNs), an application of deep learning used in reinforcement learning (RL), enable the investigation of how humans transform high-dimensional, time-continuous visual stimuli into appropriate motor responses. While recent advances in DQNs have led to significant performance improvements, it has remained unclear whether these advancements translate into improved modeling of human behavior. Here, we recorded motor responses in human participants (N = 23) while playing three distinct arcade games. We used stimulus features generated by a DQN as predictors for human data by fitting the DQN’s response probabilities to human motor responses using a linear model. We hypothesized that advancements in RL models would lead to better prediction of human motor responses. Therefore, we used features from two recently developed DQN models (Ape-X and SEED) and a third baseline DQN to compare prediction accuracy. Compared to the baseline DQN, Ape-X and SEED involved additional structures, such as dueling and double Q-learning, and a long short-term memory, which considerably improved their performances when playing arcade games. Since the experimental tasks were time-continuous, we also analyzed the effect of temporal resolution on prediction accuracy by smoothing the model and human data to varying degrees. We found that all three models predict human behavior significantly above chance level. SEED, the most complex model, outperformed the others in prediction accuracy of human behavior across all three games. These results suggest that advances in deep RL can improve our capability to model human behavior in complex, time-continuous experimental tasks at a fine-grained temporal scale, thereby opening an interesting avenue for future research that complements the conventional experimental approach, characterized by its trial structure and use of low-dimensional stimuli.

The usage of distributed Peer-to-Peer (P2P) networks has been growing steadily for a reasonable period. Various applications rely on the infrastructure of P2P networks, where nodes communicate to accomplish a task without the need for a central authority. One of the significant challenges in P2P networks is the ability of the network nodes to reach a consensus on a shared item; the challenge increases as time passes. Thus, this work proposes a new effective method for tweaking the Deep Reinforcement Learning (DRL) algorithm to train Deep Q Network (DQN) learning agents to reach a consensus among the P2P nodes. We propose various hierarchies of deep agents to address this crucial challenge in P2P networks. DRL is utilized to build and train agents; more precisely, DQN learning agents are constructed and trained. Two scenarios are proposed and evaluated. In the first scenario, one DQN agent is trained to find the consensus between the network nodes. In the second scenario, three hierarchies with different numbers of layers of agents are proposed and evaluated. In both scenarios, the P2P network used is a blockchain network. The best result was obtained using the third hierarchy of the second scenario; the overall accuracy of the model is 87.8%.