RTASM: An AI-Driven Real-Time Adaptive Streaming Model for Zero-Latency Big Data Processing

While the real-time processing applications are ubiquitous, which lead to the sprout of many distributed real-time stream processing systems. In this paper, we design a high-performance streamed-oriented big data processing system, including a messaging queue on Kafka and a Spark Streaming-based processing engine, and we evaluate the effectiveness of our system through a real-time traffic statistics application. Then we focus on the relationship between the parameters of Kafka and Spark Streaming and the performance of our system. In the last, we discuss the scalability of the optimized system of Kafka and Spark Streaming.

Data processing is an effective tool for educational sector, which can improve admission selection procedures and decisions. Most research papers focus on computational and theoretical aspect of education though little effort have been put on technological aspect of applying data mining techniques on students admission process. We therefore design a simple spark streaming framework together with machine learning algorithm to guide admission processing. We implement the spark streaming model and the proposed machine learning algorithm in a selected university using its admissions' data. The focus is on the number of students that can be admitted and those that should be rejected to reduce time and cost. The case study we evaluated show the practical usefulness of Spark streaming and machine learning algorithm for data processing in a real-time to reduce time and cost. The experiment results also confirm meaningful graphical interpretation of data using spark streaming and machine learning algorithm for students' selection for admissions.

Discrete-event simulation (DES) is a powerful tool for modeling and analyzing complex systems where state changes occur at discrete points in time. This paper presents a practical and sustainable approach to implementing DES using free mathematical and programming software, making it accessible to a wider audience including educators, students, and practitioners. This study explores the use of open-source tools, such as Python and Octave, highlighting their capabilities in building and optimizing DES models without the need for expensive and unaffordable software. In the context of Industry 4.0 and smart manufacturing, the ability to simulate and optimize discrete processes with open tools contributes to the development of digital twins, the integration of cyberphysical systems, and data-driven decision-making. Through detailed case studies in industrial fields, including manufacturing, maintenance, and logistics, this study demonstrates the effectiveness of these tools in simulating real processes and promoting their sustainability. Case studies are also re-examined to emphasize their relevance to smart manufacturing, particularly in terms of predictive maintenance, process optimization, and operational flexibility. Several challenges were encountered during the research process, such as adapting DES methodologies to the limitations of general-purpose mathematical software, ensuring accurate time management and event scheduling in environments not specifically designed for simulation, and balancing model complexity with accessibility for nonexpert users. The integration of free software not only reduces costs but also promotes collaborative learning and innovation. Additionally, the paper discusses the best practices for model validation and experimentation, providing a comprehensive guide for those new to DES. By linking open-source DES tools to the objectives of Industry 4.0, we aim to reinforce the applicability of our approach to modern, connected industrial environments. By leveraging free mathematical and programming software, this approach aims to democratize the use of DES, fostering a deeper understanding and broader application of simulation techniques across diverse fields and various regions of the world.

The rapid integration of digital technology has significantly transformed the automotive sector, influencing operational efficiency, customer engagement, and financial outcomes. This study examines the role of digitalization in enhancing the financial performance of Tata Motors and other leading Indian automotive companies. With the adoption of advanced digital tools such as Artificial Intelligence (AI), Internet of Things (IoT), automation, big data analytics, and cloud-based platforms, these firms have redefined production systems, supply chain management, and after-sales services. Tata Motors, in particular, has leveraged digital platforms for predictive maintenance, smart manufacturing, and customer-centric innovations, which have resulted in cost optimization, revenue growth, and improved market competitiveness. Furthermore, the digital shift has enabled better demand forecasting, efficient resource allocation, and data-driven decision-making, leading to sustainable financial performance. Comparative analysis with other Indian automotive players reveals that firms embracing digital innovation demonstrate stronger profitability, resilience against market volatility, and enhanced shareholder value. However, challenges such as high implementation costs, skill gaps, and cybersecurity concerns remain barriers to full-scale adoption. The findings highlight that digitalization is no longer optional but a strategic necessity for Indian automotive companies to remain globally competitive. The study concludes that continued investment in digital transformation, coupled with effective workforce upskilling, will be crucial in driving long-term financial success in the industry.

As a product of the combination of computer network technology and database technology, distributed database system has the characteristics of independence and transparency, centralized node combination, replication transparency and easy expansion. However, due to its complex access structure, distributed database system naturally has a high demand for query optimization. This paper proposes a high-performance data processing method between distributed database and multi-core processors based on PSO (Particle Swarm Optimization) to solve the task scheduling problem between multi-core processors. Inertia weight is introduced, which is added to the speed of particle flight to adjust the global and local search ability of stationary particles. The research results show that this method reduces the error rate of database query, and the overall performance of database query method is better. The improved PSO algorithm improves the searching ability of particles by dynamically adjusting the inertia weight. Therefore, the improved PSO is a high-performance algorithm to solve the real-time task scheduling problem of multi-core processors.

Thanks to Industry 4.0, digital twin technology now leads the way in smart manufacturing, enabling the real-time replication of devices to facilitate maintenance, process improvements, and smooth operation. Still, many conventional Digital Twin approaches heavily rely on cloud solutions, which result in high latency, limited scaling options, and increased risks to data privacy. However, real-time, physics-enabled simulation remains very challenging for making fast and accurate choices in busy factories due to difficulties with computer speed and network capacity. We introduce a new method that combines Digital Twin, Edge AI, and FL to make co-simulation in smart factories quick, efficient, and secure. Moving the learning and simulation steps to edge devices near the factory floor enables real-time data processing and inference. Federated Learning enables these edge devices to share and learn from one another, ensuring that important production data remains within the factories. This engine, based on FMI, enables the real-time co-simulation of physical events with their virtual counterparts, allowing the system’s behavior to be tested under various operating modes. The framework achieved a latency reduction of up to 35%, a 28% decrease in cloud usage, and a 13.2% throughput gain compared to cloud-only architectures, highlighting its scalability and industrial applicability. Additional ways to enhance this area include the use of blockchain, automated learning updates, and allowing machines to transfer knowledge from plant to plant.

The active use of the latest information technologies and non-cash payment forms has led to an increase in various types of fraud in the financial sector. Moreover, virtually all spheres of public relations now fall under the risk of fraudulent schemes, starting from financial credit and insurance and ending with foreign economic activity and the Internet. In addition, some other economic factors contribute to the significant spread of fraudulent schemes in modern conditions: a wide variety of new financial instruments (types of money, securities, financial services); rapid growth in financial transactions; leveling barriers to the unhindered movement of money, goods, and services in the process of globalization, which provokes an increase in transnational financial crime. Therefore, in search of tools to preserve existing and generate potential income, especially trusting investors fall into the traps of scammers. With the development of the current economic institutionalism, the principle of rationality in human economic behavior was no longer considered absolute, therefore, representatives of the institutional theory noted the irrational nature of human behavior, including in the field of economics and finance. Modern reality and economic practice are clear evidence of the truthfulness of this thesis. After all, despite the constant warnings of the mass media and other sources regarding various fraudulent schemes, as well as (paradoxically) often their own negative experience, citizens continue to invest in various kinds of fraudulent schemes. According to experts, the main reason is that «people will always strive for «easy» money, and it is unlikely that this desire will ever disappear» (Bruton, 2015). In this paper, we study the possibilities of preventing financial fraud on an international scale. In the context of the complexity of modern business processes, one of the most urgent problems has become the problem of activating the manifestations of corporate fraud. On average, companies lose about 5% of their profits due to corporate fraud, and the annual losses from such economic crimes amount to about USD 4 trillion on a global scale. In Russia, this figure reaches 15% (and we are talking only about losses made public by companies). The lion’s share of fraudulent schemes falls on the banking sector. The implementation of fraudulent schemes in the banking sector has certain features, in particular: fraudulent actions cause damage not only to banks and their depositors, but also negatively affect the stability of the financial system as a whole; such crimes are characterized by high latency, since managers, fearing for the business reputation of their bank, only in isolated cases turn to law enforcement agencies with appropriate statements; identifying the facts of financial fraud is very difficult since fraudsters (often not without the help of bank managers) hide their actions in every possible way and take measures to launder funds obtained by criminal means.

Resource allocation is one of the principal stages of query processing in relational data grid systems. Specific characteristics of the data grid environment, such as dynamicity, heterogeneity and large scale, impose serious restrictions to the resource allocation process. Static resource allocation before the query execution may be far from optimal due to the dynamic changes of the system. One possible optimization is to adjust dynamically the allocation of resources during the query execution. Some methods of dynamic resource allocation have been proposed, however, most of them use centralized control mechanisms. In this study we argue that the decentralized approach meets better the requirements of the data grid systems. In this study we propose a decentralized method of dynamic resource allocation that is based on the mobile agent paradigm. We consider the participating nodes as autonomous and independent elements of the system, each of which can detect if it is overloaded and make the decision to react. Then we consider each relational operation as a mobile agent running on the allocated node, meaning that, it keeps track of its own status and can migrate to another node at any time. A two-level cooperation mechanism between such autonomous nodes and autonomous operations is described in detail. Performance evaluation proves the efficiency of the proposed method.

Online games with massive concurrent user populations generate torrents of operational and gameplay data every second. Real-time analysis of these metrics is crucial for ensuring a seamless player experience, rapid incident detection, player behavior insights, and data-driven live-ops decisions. However, petabyte-scale ingestion, processing, storage, visualization, and alerting at sub-second latencies present unique challenges in throughput, fault tolerance, cost, and maintainability. This article presents a comprehensive framework for architecting, implementing, and operating a real-time metrics pipeline tailored to online games supporting millions of daily active users (DAU). We cover key components—data ingestion, stream processing, scalable storage, query optimization, dashboarding, anomaly detection, security and privacy, and cost governance—illustrated by patterns and case studies. Best practices and future directions (e.g., serverless analytics, AI-driven insights, edge-native processing) are also discussed.

“Delawareans need to have access to usable, meaningful, timely, and high-quality data about the health of the communities in which they live,”1 so that data-driven evidence-based decisions result in community-focused solutions to affect positive change. This statement is the vision of the My Healthy Community (MHC) data portal,2 which serves as Delaware’s population health platform. The MHC platform shares data on a myriad of population health areas such as social vulnerability, community characteristics, the environment (e.g., public and private drinking water, air quality), chronic diseases, mental health and substance use, healthy lifestyles, maternal and child health, health care utilization, and infectious diseases. MHC continues to expand in both data and functionality to serve as a tool to track the outcomes of implementing data-driven population health interventions and utilizing technology to monitor and evaluate the effectiveness of interventions. Shortly after the coronavirus disease 2019 (COVID-19) pandemic took a foot hold in the United States, in April of 2020, only 11 months after MHC was publicly launched, the site also became Delaware’s coronavirus disease 2019 (COVID-19) dashboard, presenting statistics on COVID-19 cases, hospitalization, emergency department visits, testing, deaths, vaccination, contact tracing, and in-person contagious school cases. MHC’s origins lie in the intersection between public health informatics and epidemiology. Visionaries within Delaware’s public health agency paved the way to create MHC with the support of DPH leadership and the perseverance and dedication of a small team of public health professionals. With their own unique lenses, one of public informatics and the other of epidemiology, the team’s efforts materialized into a tool that continues to evolve and respond to Delaware’s changing needs.

In this paper, we have discussed the challenges in handling real-time medical big data collection and storage in health information system HIS. Based on challenges, we have proposed a model for real-time analysis of medical big data. We exemplify the approach through Spark Streaming and Apache Kafka using the processing of health big data Stream. Apache Kafka works very well in transporting data among different systems such as relational databases, Apache Hadoop and non-relational databases. However, Apache Kafka lacks analyzing the stream, Spark Streaming framework has the capability to perform some operations on the stream. We have identified the challenges in current real-time systems and proposed our solution to cope with the medical big data streams.

High-performance analytical data processing systems often run on servers with large amounts of main memory. A common operation in such environments is combining data from two or more sources using some algorithm. The focus of this paper is on studying hash-based and sort-based equi-join algorithms when the data sets being joined fully reside in main memory. We only consider a single node setting, which is an important building block for larger high-performance distributed data processing systems. A critical contribution of this work is in pointing out that in addition to query response time, one must also consider the memory footprint of each join algorithm, as it impacts the number of concurrent queries that can be serviced. Memory footprint becomes an important deployment consideration when running analytical data processing services on hardware that is shared by other concurrent services. We also consider the impact of particular physical properties of the input and the output of each join algorithm. This information is essential for optimizing complex query pipelines with multiple joins. Our key contribution is in characterizing the properties of hash-based and sort-based equi-join algorithms, thereby allowing system implementers and query optimizers to make a more informed choice about which join algorithm to use.

This article presents a comprehensive exploration of data quality management through automation, examining the evolution from manual processes to sophisticated, technology-driven approaches. Drawing on empirical research across multiple organizations, the article investigates how automated data quality frameworks and machine learning-based anomaly detection can address the complexities of ensuring data accuracy, completeness, consistency, and other critical dimensions in modern data ecosystems. The article introduces a structured implementation framework for organizations seeking to enhance their data quality capabilities while navigating integration challenges with existing architectures. The article reveals significant improvements in defect detection, incident reduction, and operational efficiency when automated solutions replace traditional manual methods, with most organizations achieving a return on investment within 8-14 months. Beyond technical benefits, the article highlights how quality automation catalyzes broader organizational changes in data governance, stakeholder engagement, and quality culture. As data volumes grow and business dependencies on high-quality data increase, the emerging technologies and trends identified in this research—including quality-ascode practices, federated management approaches, and self-healing systems—will likely shape the future landscape of data quality management, transforming it from a technical function to a strategic business capability.

The increasing volume, variety, and velocity of data necessitate robust data integration and transformation solutions. As enterprises shift towards data-driven decision-making, efficient data processing has become a cornerstone of business success. Talend, a widely adopted open-source and commercial data integration tool, offers comprehensive features for Extract, Transform, Load (ETL) processes, enabling organizations to manage their data efficiently. This paper explores the role of Talend in modern enterprise solutions, emphasizing its functionalities, advantages, challenges, and case studies of successful implementations. The study examines the architecture of Talend, its ability to integrate with cloud-based and on-premises environments, and its impact on data governance and security. Furthermore, it evaluates Talend’s efficiency in handling big data and real-time processing compared to traditional ETL tools. The findings indicate that Talend provides enterprises with a scalable, cost-effective, and flexible solution for data transformation and integration. By leveraging Talend, organizations can ensure high data quality, streamline business operations, and enhance analytics capabilities, thereby gaining a competitive edge in the evolving digital landscape.

Integrating SAP Enterprise Resource Planning (ERP) systems with Amazon Web Services (AWS) is transforming educational institutions to change the face of operational competency and ways of knowledge delivery. It will discuss the cloud-based infrastructure and how it benefits educational organizations in terms of scalability, cost optimization, and robust disaster recovery strategies. This article will assess how SAP on AWS implements automation processes in administrative work, data-driven decision-making, and personalized learning experiences. The article addresses critical security and compliance considerations indispensable for protecting sensitive educational data while maintaining regulatory adherence. It addresses best practices around implementation, from architecture planning down to performance-optimization strategy and change-management approach. To top it off, the article analyses the future scope of educational technology, which is very much concerned with the emerging themes of artificial intelligence, the Internet of Things, blockchain, and extended reality, leading to a redrawing of educational landscapes. The article provides insight into how educational institutions can use these technological advancements to enhance operational efficiency while maintaining security and compliance in an increasingly digital educational environment.