Human-in-the-Loop Robotic Process Automation (RPA) with Deep Reinforcement Learning (DRL)

Recent studies show that Deep Reinforcement Learning (DRL) models are vulnerable to adversarial attacks, which attack DRL models by adding small perturbations to the observations. However, some attacks assume full availability of the victim model, and some require a huge amount of computation, making them less feasible for real world applications. In this work, we make further explorations of the vulnerabilities of DRL by studying other aspects of attacks on DRL using realistic and efficient attacks. First, we adapt and propose efficient black-box attacks when we do not have access to DRL model parameters. Second, to address the high computational demands of existing attacks, we introduce efficient online sequential attacks that exploit temporal consistency across consecutive steps. Third, we explore the possibility of an attacker perturbing other aspects in the DRL setting, such as the environment dynamics. Finally, to account for imperfections in how an attacker would inject perturbations in the physical world, we devise a method for generating a robust physical perturbations to be printed. The attack is evaluated on a real-world robot under various conditions. We conduct extensive experiments both in simulation such as Atari games, robotics and autonomous driving, and on real-world robotics, to compare the effectiveness of the proposed attacks with baseline approaches. To the best of our knowledge, we are the first to apply adversarial attacks on DRL systems to physical robots.

Design automation is undergoing a new generation of changes caused by artificial intelligence technologies represented by deep learning and reinforcement learning. Notably, the advantages of deep reinforcement learning in addressing solution optimisation and decision-making tasks with cognitive automation functionality have garnered attention in design. In the context of surrogate model-driven engineering design optimisation, this paper addresses current research challenges such as reliance on domain knowledge for local development, shortcomings in the self-learning and adaptive capabilities of optimisation algorithms for global exploration, etc. Centred around the deep reinforcement learning model, Deep Q-learning, and complemented by self-organising maps and neural network technologies, we propose a methodology considering multi-fidelity simulation data for design space exploration. This approach effectively reduces sampling costs and enables the optimisation model to learn the optimal direction for high-precision predictions and achieve rapid, accurate optimisation. Finally, the effectiveness of the proposed method is comprehensively validated through four typical optimisation scenarios and a case study involving the optimisation of a wheeled robot's suspension swing arm structure. This work will be a crucial reference for applying deep reinforcement learning in simulation-driven engineering design optimisation.

Purpose The authors share a real-world case study of a multiple-year adoption process of robotic process automation (RPA) in procurement to add to the limited empirical research base on RPA in supply chain management. The Maersk case offers hard evidence of the value of RPA that can be used as a benchmark for decision-making in companies and as a basis for further research in key areas such as change management and behavioral aspects as well as return on investment on RPA and the need for RPA capability development. Design/methodology/approach Together with the co-author from Maersk, the authors cover a multiple year RPA program in procurements, to capture lessons learned and develop research questions for the future. The authors consider 39 generic action principles for RPA adoption in a specific procurement context. Findings The authors find RPA for procurement to be an enabler of strategic progress and of advancing the concept of triple values (enterprise, customer and employee) in a supply chain setting. The authors offer RPA adoption scoping guidance and show how, with growth and maturity of the program, conditions such as predictability of volume become less relevant as a scoping criterium. The authors also find that RPA augments work rather than replace staff. Maersk utilizes staff time freed up by automation to focus employees on more strategic priorities. The authors consider 39 generic action principles for RPA adoption in a procurement-specific context and develop additional action principles based upon the Maersk case. Originality/value The authors study both a unique and rich case to complement limited empirical research on RPA in procurement and supply chain management and the limited research into RPA past early stage adoptions. The authors address Hofmann et al.'s (2019) questions about the change management involved in RPA and consider generic RPA action principles from literature in this specific procurement setting. The authors extend those action principles and develop a rich set of research opportunities.

The intersection management system can increase traffic capacity, vehicle safety, and the smoothness of all vehicle movement. Platoons of connected vehicles (CVs) use communication technologies to share information with each other and with infrastructures. In this paper, we proposed a deep reinforcement learning (DRL) model that applies to vehicle platooning at an isolated signalized intersection with partial detection. Moreover, we identified hyperparameters and tested the system with different numbers of vehicles (1, 2, and 3) in the platoon. To compare the effectiveness of the proposed model, we implemented two benchmark options, actuated traffic signal control (ATSC) and max pressure (MP). The experimental results demonstrated that the DRL model has many outstanding advantages compared to other models. Through the learning process, the average waiting time of vehicles in the DRL method was improved by 20% and 28% compared with the ATSC and MP options. The results also suggested that the DRL model is effective when the CV penetration rate is over 20%.

Transformative technology such as Robotic Process Automation (RPA) and Cloud Integration is the platform to achieve streamlining of operations, enhance employee productivity, and drive innovation for your business. While RPA automates repetitive, rule based work, cloud computing is the scalable, flexible and decentralized infrastructure for storing and processing of data, and collaboration. With RPA integrated with cloud platforms, organizations gain several advantages including accessibility, low cost and real time collaboration. Choosing the right cloud provider that supports RPA workloads, ensuring data flow seamlessly between RPA bots and cloud systems, and using cloud based RPA tools that can scale and that add flexibility, are the best practices for RPA and Cloud integration. It is also important to enforce robust security protocols, data governance as well as comply with risks. Integration is successful when RPA links with cloud native apps and services with as little disruption to work as possible and as much automation effect as possible. It is therefore recommended that performance monitoring, testing and continuous improvement of automation process should be given priority. Through the implementation of these best practices, organizations can attain near full potential RPA and cloud integration for operation efficiency, better customer experience and faster digital transformation. In this paper we explore the key strategies, challenges and solutions to ensure success with the integration of RPA into cloud technologies to help businesses looking to optimize the use of automation and cloud capabilities in unison.

The term Blockchain Technology (BT) refers to information technology (IT) where data is stored in numerous blocks using multiple distributed servers. Through digital automation the data is refreshed periodically at a frequency set by data architects and network designers. The servers are sometimes on the public internet, but in the aerospace world it is more within private intranet of authorized users to maintain privacy and security. A Blockchain network registers the components within aerospace echo system in a digital ledger accompanied by relevant data. It is virtually immutable. The data residing within BT cannot be changed without the awareness of the collaborative parties. Even if a hacker manages to penetrate one block, the overall integrity of the data would remain intact since the data is distributed among thousands of blocks containing cryptographically protected digital ledgers. Deep reinforced learning (DRL) is a machine learning (ML) process based on reinforced learning (RL) and neural networks (NN). DRL training algorithms adjusts its actions dynamically motivated by a system of reward and punishment. The DRL algorithm learns by interacting with its dynamic environment. The DRL software agents receive rewards for correct actions and penalties for performing incorrectly. These agents learn without human intervention by targeting maximizing the reward and minimizing the penalty. Robotic process automation (RPA) is a business process automation technology based on software agents. It can trigger automated agent-based actions. When RPA and DRL are combined, system operates software agents that can learn in dynamic environment automatically without human interventions. They act as autonomous agents that learns continuously and operates cyber infrastructure autonomously. In this paper we show the implementation process and the benefits of integrating deep reinforced learning and robotic process automation in Blockchain digital transformation for autonomous cybersecurity.

Irrigation decision-making using Reinforcement Learning (RL) performs well in changing environment, but easily falls into sub-optimal solutions with high-dimensional data. Deep Reinforcement Learning (DRL) has fused RL with Deep Learning (DL) and excels at learning adaptive and long-term irrigation strategies directly from high-dimensional environment data. This paper systematically reviews the applications of DRL in irrigation optimization, covering both pre-trained environments based on crop growth simulators and dynamic environments driven by real-time sensors. We discussed the strengths of classic DRL algorithms, including their ability to handle dynamic and non-linear environments, and reviewed their performance in irrigation multi-objective optimization and decision-making. In addition, we identified constraints in applying DRL in irrigation decision making, which include data scarcity, poor model interpretability, and difficulties in field deployment. It shows DRL can provide a powerful framework for adaptive irrigation, but is constrained by the gap between simulation and real-world complexity. To address these limitations, we discussed approaches in future work, such as developing multi-objective DRL algorithms. These approaches will improve DRL modeling outcomes and provide a technological foundation for smart agriculture and sustainable resource management. • Review the application of Deep Reinforcement Learning (DRL) in agricultural irrigation. • Analyze the performance of DRL algorithms in irrigation decision-making. • Compare DRL models based on different environment in irrigation optimization. • Discuss the further work to improve DRL performance in irrigation optimization.

SYNOPSIS This study investigates the risk mitigation and internal control processes organizations implement in their robotic process automation (RPA) deployments in accounting. The roles of internal controls and risk mitigation are examined by exploring how RPA Program Managers have adopted internal controls and risk mitigation regarding RPA in accounting. Employing interviews with 20 RPA Program Managers, this study provides rich insights into RPA Governance for accounting processes. The interviews revealed that organizations adopt various governance models to ensure proper oversight over RPA in accounting. RPA Governance models range from being fully centralized to being entirely decentralized. RPA risk and control oversight includes unique risk assessments for the RPA accounting environment and specialized monitoring to ensure that RPA in accounting performs as expected. JEL Classifications: M41; M42.

This chapter discusses the concept of robotics, which is followed by a discussion of the robotic process automation (RPA). The benefits of the RPA are also outlined in this chapter. The chapter further explores the RPA in strategy and strategy implementation. Robotics and RPA concepts are often confused. The confusion that occurs may be caused by the fact that there are some crossovers between robotics and RPA.Robots are used to automate physical tasks, such as in manufacturing. So many types of automation have nothing to do with physical robots. Furthermore, so many branches of robotics have nothing to do with automation. Robots have consistent characteristics including a certain form of mechanical aspect. This aspect of a robot helps it complete tasks in the environment for which it is designed. They also would need electrical components that control and power the machinery, and they would typically contain some form of computer programming.RPA can be viewed as bringing together business models, strategy, process, data, humans and robots. This is done in order to achieve efficiencies within the business processes. RPA has several benefits, including cost reduction, time efficiencies, better accuracy rate, improved governance environment, better customer advocacy and retention, improved checks and balances, increased speed and productivity, easy integration into existing technologies and super-scalability.The firm applying RPA gains advantage as the performance tracking occurs on an ongoing basis and is timely and free of human intervention. Those charged with governance are able to harness this information and respond timely should the need arise to ensure that the strategic responsibilities and objectives assigned to relevant employees are on track. The firm applying RPA gains advantage of having a tool for eliminating human intervention and for integrating plans, processes and structures.

Half of the world’s population resides in cities and this percentage is going to rise in the coming years. This rapid transformation of the economy and society is making traditional city management redundant. Intensive urbanization has brought with it some challenges. Scanty resources, population growth, increased carbon footprint, reduced air quality, congestion on roads, depriving health status, and falling food quality, all are prominent parts of cities. Does this bring a question in mind is accelerated growth sustainable? The answer and solution to this is an adaption of the Smart City Concept. A smart city integrates all the available technology with the requirements of city development and builds infrastructure which can improve the quality of life of citizens. Smart cities are designed in a manner to use minimum resources with maximum efficiency in all areas of urban life. It provides smart solutions using smart technologies. Urban needs and technological advancements meet in smart cities. It manages cities more quickly and effectively by using AI, IT technologies, and robotic process automation. Intelligent transportation, intelligent energy, intelligent governance, intelligent security, intelligent healthcare are all central to the idea of smart cities. The emphasis of this essay is on smart transit as a component of smart cities. The movement of people and goods across several geographical boundaries has been significantly aided by the transportation sector. Globally, the majority of smart cities face logistical and transportation issues. Both the fast growing human population and the increase in the number of cars on the road are to blame for this. In order to design and run a sustainable transportation system, technology may be of great assistance. Robotic systems for real-time vehicle data collection, traffic management, and using mobility on demand in trip planning through a single user interface have emerged as cities struggle with traffic congestion. The transportation industry, which faces difficulties including traffic congestion, unanticipated delays, and routing issues that cause businesses to lose money, has successfully used robotic process automation. Robotic process automation provides reliable results and affordable solutions while reducing uncertainty in the decision-making process. It is very helpful in designing and running a successful transit system. Robotic Process Automation (RPA) systems for obtaining real-time data from vehicles, buses, and trucks for traffic management and using mobility on demand in trip planning through a single user interface have emerged as cities struggle with poor management and congestion. By examining different uses of robotic process automation, it is possible to evaluate precise prediction and detection models that are intended to forecast traffic volume, an appropriate route, traffic conditions, accurate information about timing, incidents, and which time and roadway for transport are suitable for running and mobility. The application of RPA in smart cities could be seen in Intelligent Transportation Systems’ Traffic Management, Public Transportation, Safety Management, Manufacturing, and Logistics. The research focuses on areas of concern for smart cities in the sphere of the transportation sector and related difficulties that may be solved via robotic process automation. The article emphasizes the challenges and solutions for making smart transportation more effective by using Intelligent Robotic Transportation Systems.

Robotic Process Automation (RPA) is rapidly transforming the landscape of investment and wealth management by enhancing efficiency, accuracy, and decision-making. This chapter explores how RPA is being integrated into portfolio management strategies. It examines the impact of RPA on client onboarding, compliance, portfolio rebalancing, and risk management, highlighting real-world use cases in asset and wealth management firms. The discussion also covers challenges related to implementation, such as regulatory compliance, data security, and integration with legacy systems. Furthermore, the chapter investigates how RPA, when combined with artificial intelligence and analytics, can support personalized investment solutions and improve customer experience. By redefining traditional processes, RPA not only reduces operational costs but also empowers financial advisors to focus more on strategic planning and client engagement. This chapter offers practical insights for institutions looking to adopt RPA-driven solutions in their portfolio strategies.

Processes that contain repetitive and well-defined steps are more suitable for robotic process automation (RPA) than others. Many financial processes such as quarterly reports and balance sheet preparation fall into this category. Also, these processes are voluminous, making the benefits of RPA even more recognizable. Financial institutions (FIs) become aware of the benefits of RPA earlier than others and adopted RPA in many of their applications. Account reconciliations, reporting, and tax planning are among the processes that have been adapted to RPA. RPA has been a popular term in the industry for the last few years, only recently has it started to become a technology that’s within the reach of most organizations. RPA market has been almost doubling every year and it is expected to grow by more than 50 through 2020. Growth in the Turkish market is also substantial: between 2018 and 2020 more than 130 top companies have employed RPA robots in their infrastructure. Banks and technology companies in Turkey adapt RPA faster than other sectors, about 50% of the companies that adapted RPA are from these sectors. This paper discusses applications and advantages of RPA by giving examples from the finance and accounting in Turkey.

Objectives: Due to the explosive growth of the internet, mobile applications, machine learning, and new information technologies, new management methods that do not require face-to-face contact are spreading rapidly. Innovative approaches to process automation are needed in the era of the coronavirus disease 2019 (COVID-19) pandemic, and firms are actively adopting and implementing solutions such as robotic process automation (RPA) to increase their operational excellence. The objective of this study is to address state-of-the-art use cases of RPA by IBM Korea to determine key factors related to successful RPA implementation. Methods: Multiple case study methods and triangulation of findings were used. This study examined several cases in which RPA was implemented and proposed an analytical framework for RPA. Results: This article described IBM’s RPA usage strategies to accelerate business process management and examined four cases of RPA implementation by client firms of IBM in South Korea. This case study found that IBM’s RPA solutions positively affected the client firms’ performance, suggesting that solutions like RPA could be essential for firms’ survival due to the benefits of process automation. Conclusions: Solutions using RPA can both reduce labor costs and improve the operational excellence of firms’ internal processes. Due to the COVID-19 pandemic, process automation is increasingly essential. Practical issues such as change management and shadow work caused by process automation are discussed.

The rising complexity of enterprise operations, coupled with the demand for real-time responsiveness, has catalyzed the integration of Robotic Process Automation (RPA) in ERP systems. SAP, being a foundational technology for many global enterprises, is now at the forefront of RPA-driven innovation. This research paper explores how intelligent automation, driven by RPA, is reshaping SAP landscapes, making them more agile, scalable, and future-ready. Drawing from the strategic implementations led by Vaibhav Gupta, Senior Director of Digital Signature and Operations at Resolve Tech Solutions, the paper provides an in-depth view of how RPA tools are being deployed to modernize traditional SAP workflows. Gupta’s approach to RPA in SAP involves identifying repetitive, rule-based processes and designing automation bots that reduce manual intervention, accelerate execution time, and ensure data integrity. By embedding RPA into core SAP modules such as FI, MM, SD, and HR, his team has enabled clients to unlock efficiency gains while minimizing human error and increasing regulatory compliance. The paper also highlights the integration of RPA tools like UiPath and SAP Intelligent RPA with both ECC and S/4HANA systems. Through detailed data analysis and real-world examples, the study showcases the transformation of legacy workflows into digitally orchestrated processes with minimal disruption. It emphasizes the importance of a well-defined automation strategy, governance model, and continuous monitoring framework to scale RPA initiatives. Additionally, it evaluates user feedback and performance metrics post-implementation. This paper serves as a blueprint for CIOs, IT leaders, and SAP architects aiming to transition toward an intelligent enterprise model. It underscores that RPA is not merely a technology trend, but a strategic enabler of next-generation business transformation within SAP ecosystems.

This paper investigates stock portfolios by application of Deep Reinforcement Learning (DRL) Models to achieve an optimal tactical asset allocation. The research problem is described as an optimization scenario that seeks to maximize the portfolio risk adjusted returns for a given portfolio asset allocation. The problem is set up with an initial capital investment which is invested in a set of assets. The initial strategic allocation is determined, which in our case is the equal weight allocation, and all of the capital is invested in the set of assets. At each point in time, the assets are reallocated according to the allocation which will increase the portfolio value. Two DRL models are implemented. The performance of the DRL models is compared with the uniform weights portfolio. The results show that, generally, two DRL models have higher cumulative returns.