Online reinforcement learning for a continuous space system with experimental validation

Data quality is a critical aspect of data analytics since it directly influences the accuracy and effectiveness of insights and predictions generated from data. Artificial Intelligence (AI) schemes have grown in the existing era of technological advancement, which provides innovative exposure to healthcare applications. Reinforcement Learning (RL) is a subfield and an influential Machine Learning (ML) model aimed at optimizing decision-making by association with dynamic environments. In healthcare applications, RL can modify conduct strategies, enhance source application, and improve patient investigation history by using various data modalities. The worth of the data quality regulates how effective RL is in healthcare applications. In healthcare, the model predictions have a direct impact on patient's lives, and poor data quality often leads to wrong evaluations that expose patient safety and treatment quality. Biases in data quality have also presented a challenging influence on the RL model's effectiveness and accuracy. RL models have enormous potential in healthcare; however, various strategic limitations prevent their widespread acceptance and deployment. The implementation of RL in healthcare faces serious issues, mostly around data quality, bias, and tactical difficulties. This study delivers a broad survey of these challenges, emphasizing how imbalanced, imperfect, and biased data can affect the generalizability and performance of RL models. We critically assessed the sources of data bias, comprising demographic imbalances and irregularities in electronic health records (EHRs), and their impact on RL algorithms. This survey aims to present a detailed study of the complex circumstances relating to data quality, data biases, and strategic barriers in RL models deploying in healthcare applications. However, the main contribution of the proposed study is that it provides a systematic review of these challenges and delivers a roadmap for future work intended to refine the consistency, fairness, and scalability of RL in healthcare sectors.

Playing games is one of the activities used as a means of entertainment to get fun, fill spare time, or just for light exercise. For these reasons, playing games is very popular with people around the world. Seeing how enthusiastic the community is about games, developers are competing to present games that attract attention. One way is by embedding Artificial Intelligence. Artificial intelligence can be defined as a branch of science that models human thinking. One type of Machine Learning that is closest to the definition of AI is Reinforcement Learning (RL). The basic concept of RL is how to make machines/agents smart after interacting with their environment. In this study, the author tries to implement Reinforcement Learning into one of the Atari games (Breakout) using Open Ai Gym. The purpose of this research is to see how Reinforcement Learning can be implemented into Breakout video games. Based on the results obtained from the test, it is concluded that by implementing the Reinforcement Learning algorithm, agents in the breakout game can get an average reward of 18 with a rate of exploration of 0.02 through more than 30000 episodes. With these results, it can also be said that the Reinforcement Learning algorithm can make agents in breakout games able to play themselves, which can be used to create a walkthrough feature in the game. So that players can learn by seeing how the AI plays through this walkthrough.

Artificial intelligence (AI) in the fields of conversation and language has experienced immense growth as a result of the large quantities of text corpora available for training models. This paper discusses conversational AI dialogue systems concerning the components of natural language processing (NLP) and reinforcement learning that function together to produce a human-like response. The types of conversational AI, namely, task-oriented systems, question-answering agents, and social chatbots, are also expanded on to provide a systematic review. Reinforcement learning has a crucial role in resolving errors in existing conversational AI models and allows for a factor of exploration in the art of communication. The trial-and-error nature of reinforcement learning agents ensures that irrelevant patterns in a text corpus are not memorized. The implementation of reinforcement learning and NLP tools to create social chatbots with emotional intelligence brings researchers one step closer to mimicking human conversation.

With the implementation of reinforcement learning (RL) algorithms, current state-of-art autonomous vehicle technology have the potential to get closer to full automation. However, most of the applications have been limited to game domains or discrete action space which are far from the real world driving. Moreover, it is very tough to tune the parameters of reward mechanism since the driving styles vary a lot among the different users. For instance, an aggressive driver may prefer driving with high acceleration whereas some conservative drivers prefer a safer driving style. Therefore, we propose an apprenticeship learning in combination with deep reinforcement learning approach that allows the agent to learn the driving and stopping behaviors with continuous actions. We use gradient inverse reinforcement learning (GIRL) algorithm to recover the unknown reward function and employ REINFORCE as well as Deep Deterministic Policy Gradient algorithm (DDPG) to learn the optimal policy. The performance of our method is evaluated in simulation-based scenario and the results demonstrate that the agent performs human like driving and even better in some aspects after training.

Individualized manufacturing is becoming an important approach to fulfill increasingly diverse consumer expectations. While there are various solutions for the manufacturing process, such as additive manufacturing, the subsequent automated assembly remains a challenging task. As an approach to this problem, we aim to teach a collaborative robot to successfully perform pick and place tasks by implementing reinforcement learning. For the assembly of an individualized product in a constantly changing manufacturing environment, the simulated geometric and dynamic parameters will be varied. Using reinforcement learning algorithms capable of meta-learning, the tasks will first be trained in simulation, and then performed in a real-world environment where new factors are introduced that were not simulated in training to confirm the robustness of the algorithms. A concept comprised of selected machine learning algorithms, hardware components as well as further research questions to realize the outlined production scenario are the results of the presented work.

This research looks into the implementation of reinforcement learning (RL) principles into robotics with a particular interest in controlling autonomous systems. It examines various RL methods, such as Proximal Policy Optimization (PPO), and Deep Q-Networks (DQN), to improve robotic decision-making processes. Using information from Fiber Bragg Grating (FBG) sensors, we focus on real-time processing of sensor data and demonstrate greater levels of efficiency and flexibility within active settings. Quantitative data illustrates notable advancements in stability, accuracy, and spatial economic efficiency, which further the development of robotic autonomy.

Adaptive Control Strategies for Enhanced Integration of Solar Power in Smart Grids Using Reinforcement Learning

The research objective of this NSF-funded study is to explore and understand how open-ended, hands-on making work and activities can reflect student learning trajectories and learning gains in the product-based learning, undergraduate engineering classroom. The aim is to expand understanding of what making learning in the context of engineering design education might be and to illustrate educational pathways within the engineering education curriculum. Making is rooted in constructionism – learning by doing and constructing knowledge through that doing. Aspects of making work and activities that are unique to making that could appear in the engineering classroom or curriculum include: sharing, practical ingenuity, personal investment, playful invention, risk taking, community building and self-directed learning. The main research questions of this work is: How do engineering students learn and apply making? What are the attributes of making in the engineering classroom? Empirical evidence of what making in the engineering classroom looks like, and how it changes over time, and how students conceptualize making through making, designerly, and engineering ways of knowing-doing-acting will come from revisiting and additional qualitative analysis of student project data collected during a product-based learning course engineering design course. To best address the research question, this proposed study proposes multiple qualitative methods to collect and analyze data on engineering students learning making. We aim to triangulate what students think they are learning, what they are being taught, and what students are demonstrating. This work is exploratory in nature. In our approach to understanding making outside of formal engineering education, at events like Maker Faires in the Maker Community, it does seem evident that there is a lot of overlap between a making mindset and a designerly way of knowing or engineering way of knowing. In the sphere of formal engineering education however, making is regularly viewed as lesser than engineering, engineering design without the engineering science or analysis. Making is not yet valued as part of formal engineering education efforts. If making is something that can be connected to beneficial student learning and is additive to the required technical content and provides a means for students to figure out what area of problems they want to tackle in the studies and beyond, it would make for a student-centered making revolution. This study advances the knowledge of the learning pathways of making by capturing empirical evidence of such learning trajectories. This study will advance the currently limited knowledge of learning in the making community and making in the engineering classroom. Initial findings generated during this study describe the learning trajectories of engineers learning making. By examining the engineering student making learning experience through the lens of cognitive science and illustrating empirical making learning trajectories, this work may impact the quality of engineering design teaching. By sharing learning trajectories across multiple communities, we seek to change the conversation by illuminating pathways for a wider array of student makers to become the makers and engineers of the future.

Categorical Distributional Reinforcement Learning (CDRL) has demonstrated superior sample efficiency in learning complex tasks compared to conventional Reinforcement Learning (RL) approaches. However, the practical application of CDRL is encumbered by challenging projection steps, detailed parameter tuning, and domain knowledge. This paper addresses these challenges by introducing a pioneering Continuous Distributional Model-Free RL algorithm tailored for continuous action spaces. The proposed algorithm simplifies the implementation of distributional RL, adopting an actor-critic architecture wherein the critic outputs a continuous probability distribution. Additionally, we propose an ensemble of multiple critics fused through a Kalman fusion mechanism to mitigate overestimation bias. Through a series of experiments, we validate that our proposed method provides a sample-efficient solution for executing complex continuous-control tasks.

Ultrawideband (UWB) signal transmission in Si integrated circuits was performed by use of wireless interconnects. Linear dipole antennas fabricated on a Si substrate showed UWB characteristics in the frequency range of 5-20 GHz. The signal attenuation rates of Gaussian monocycle pulse through 10 Ωcm and 2.29 kΩcm substrates were -0.08 mV/chip and -0.03 mV/chip, respectively. Pseudo random binary signals were transmitted and received between Si chips and the lowest bit error rate of 128 bit binary signal with 13.5 Gbps was 2 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-6</sup> . Interference of Gaussian monocycle pulses transmitted from a dipole antenna could be suppressed by use of differential mode signals instead of common mode signals.

This study discusses the modeling and linear quadratic regulator (LQR) controller based closed loop control of a three tank hybrid (TTH) process. A pseudo random binary signal (PRBS) based excitation data obtained from a real time TTH setup is utilized in validating its first principle model (FPM). Based on top and bottom interactions, various modes prevalent are considered based on steady state physical reachability analysis of the TTH for a given input range for controller design. The FPM is linearized using nominal values of process parameters using Jacobians from each existing mode. LQR controllers are designed for each mode. A supervisory structure is designed for selecting estimation model and controller for each appropriate mode. Results from real time servo tracking and disturbance rejection experiments are discussed.

In this paper, a novel artificial neural network (ANN) based method dedicated to simultaneously estimating thermal conductivity and thermal diffusivity of CSP (concentrating solar power) plant receiver materials is presented. By monitoring the evolution of these two correlated thermophysical properties during aging cycles, CSP plants' cost efficiency could be maintained. The proposed method is based on the processing of experimental photothermal data using classification and estimation networks. All the networks are feedforward ANN trained with supervised learning algorithms. A pseudo random binary signal (PRBS) is used as excitation and the impact on performance of both the photothermal response length, which is used as model input, and the number of training examples has been evaluated. Of course, the networks' topology has been optimized, allowing the generalization ability to be controlled. Despite the lack of data, the results are promising. Mean relative errors are between 8% and 20%, and the main levers for improvement are identified. In this paper, the study deals with a large range of materials (polymers and metallic alloys).

Robust Semi-Automatic Identification of Compliantly Coupled Two-Mass Systems

A proposal is made that pseudorandom binary signals be used as key words in an associative holographic memory because this extends considerably the range of different key words. An experimental check is reported of generation of key words with the aid of optically controlled transparencies, and of the correlation characteristics of some pseudobinary signals.

As a powerful tool for solving nonlinear complex system control problems, the model-free reinforcement learning hardly guarantees system stability in the early stage of learning, especially with high complicity learning components applied. In this paper, a reinforcement learning framework imitating many cognitive mechanisms of brain such as attention, competition, and integration is proposed to realize sample-efficient self-stabilized online learning control. Inspired by the generation of consciousness in human brain, multiple actors that work either competitively for best interaction results or cooperatively for more accurate modeling and predictions were applied. A deep reinforcement learning implementation for challenging control tasks and a real-time control implementation of the proposed framework are respectively given to demonstrate the high sample efficiency and the capability of maintaining system stability in the online learning process without requiring an initial admissible control.