High Performance Distributed In-Memory Architectures for Trade Surveillance System

The proliferation of real-time artificial intelligence (AI) and machine learning (ML) systems has amplified the demand for robust, low-latency state management techniques capable of operating at scale. From streaming feature extraction to online model inference and complex event processing, stateful operations lie at the core of intelligent data-driven pipelines. However, managing this state in distributed environments presents numerous challenges, including fault tolerance, efficient recovery, incremental updates, and tight latency budgets. This paper explores RocksDB, a high-performance, embeddable key-value store based on a log-structured merge-tree (LSM) architecture, as a state backend solution for real-time applications. We delve into the internal mechanisms that make RocksDB particularly well-suited for low-latency, high-throughput workloads, such as background compaction, memory/disk tiering, and custom serialization strategies. The article details practical integration techniques with distributed stream processing engines such as Apache Flink and Kafka Streams, emphasizing checkpointing, state TTL, and asynchronous snapshotting. We also introduce a set of design patterns for real-time AI/ML applications — including online feature stores, real-time recommender systems, and stateful anomaly detection — and show how RocksDB enables efficient, fault-tolerant management of evolving application state. Through empirical evaluations, we benchmark performance trade-offs between RocksDB and alternative backends (e.g., in-memory, Redis, Cassandra), and present optimizations that significantly improve state access latency, recovery time, and disk footprint. By providing a comprehensive review of RocksDB’s role in real-time state management, this work serves as both a scholarly reference and a practical guide for engineers, researchers, and system architects building the next generation of AI/ML-driven streaming systems.

The great advance and affordability of technologies, communications and sensor technology has led to the generation of large amounts of data in the field of the Internet of Things and smart environments, as well as a great demand for smart applications and services adapted to the specific needs of each individual. This has entailed the need for systems capable of receiving, routing and processing large amounts of data to detect situations of interest with low latency, but despite the many existing works in recent years, studying highly scalable and low latency data processing systems is still necessary. In this area, the efficiency of complex event processing (CEP) technology is of particular significance and has been used in a variety of application scenarios. However, in most of these scenarios there is no performance evaluation to show how the system performs under various loads and therefore the developer is challenged to develop such CEP-based systems in new scenarios without knowing how the system will be able to handle different input data rates and address scalability and fault tolerance. This article aims to fill this gap by providing an evaluation of the various versions of one of the most reputable CEP engines—Esper CEP, as well as its integration with two renowned messaging brokers for data ingestion—RabbitMQ and Apache Kafka. For this purpose, we defined a benchmark with a series of event patterns with some of the most representative operators of the Esper CEP engine and we performed a series of tests with an increasing rate of input data to the system. We did this for three alternative software architectures: integrating open-source Esper and RabbitMQ, integrating one instance of Esper enterprise edition with Apache Kafka, and integrating two distributed instances of Esper enterprise edition with Apache Kafka. We measured the usage of CPU, RAM memory, latency and throughput time, looking for the data input rate with which the system overloads for each event pattern and we compared the results of the three proposed architectures. The results have shown a very low CPU consumption for all implementation options and input data rates; a balanced memory usage, quite similar among the three architectures, up to an input rate of 10,000 or 15,000 events per second, depending on the architecture and event pattern, and a quite efficient response time up to 10,000 or 15,000 events per second, depending on the architecture and event pattern. Based on a more exhaustive analysis of results, we have concluded that the different options offered by Esper for CEP provide very efficient solutions for real-time data processing, although each with its limitations in terms of brokers to be used for data integration, scalability, and fault tolerance; a number of suggestions have been drawn out for the developer to take as a basis for choosing which CEP engine and which messaging broker to use for the implementation depending on the of the system in question.

Critical infrastructure applications provide services upon which society depends heavily; such applications require constant, dependable operation in the face of various failures, natural disasters, and other disruptive events that might cause a loss of service. These applications are themselves dependent on distributed information systems for all aspects of their operation, so survivability of these critical information systems is an important issue. Survivability is the ability of a system to continue to provide service, though possibly alternate or degraded, in the face of various types of failure and disruption. A fundamental mechanism by which survivability can be achieved in critical information systems is fault tolerance. Much of the literature on fault-tolerant distributed systems focuses on tolerance of local faults by detecting and masking the effects of those faults. I describe a direction for fault tolerance in the face of non-local faults—faults whose effects have significant non-local impact, sometimes widespread and sometimes catastrophic—where often the effects of these faults cannot be masked using available resources. The goal is to recognize these non-local faults through detection and analysis, then to provide continued service (possibly alternate or degraded) by reconfiguring the system in response to these faults. A specification-based approach to fault tolerance, called RAPTOR, is presented that enables systematic structuring of formal specifications for error detection and recovery, utilizes a translator to synthesize portions of the implementation from the formal specifications, and provides an implementation architecture supporting fault-tolerance activities. The RAPTOR approach consists of three specifications describing the fault-tolerant system, the errors to be detected, and the actions to take to recover from those errors. The RAPTOR System includes a synthesizer, the Fault Tolerance Translator, to generate implementation of code components from the specifications to perform error detection and recovery activities. In addition, a novel implementation architecture incorporates the generated code as part of an infrastructure supporting fault tolerance at both the node and system levels. Finally, the solution approach is explored and evaluated through the use of case studies and experiments in two critical infrastructure application domains.

AMATISTA is a European IST project whose main target is the development of a fault tolerant (FT) set of tools. These include an automatic FT Insertion (FTI) tool and a FT injection and simulation (FTIS) tool. This paper presents the AMATISTA project from a user perspective. FT experience and current non-automatic techniques and methodologies used in Alcatel Espacio (AEO) are described. The advantages of having tools for automatic FTI and FT injection and simulation are summarised. The AEO requirements for both tools are also described.

Recovery in main memory database systems for engineering design applications

Fault tolerance is an important requirement for long-running parallel programs. This paper presents a different approach to fault-tolerance support in message-passing parallel programs based on their structural and behavioral characteristics, commonly known as patterns. A classification of these patterns and their applicable fault-tolerance strategies is aimed to facilitate an application developer to incorporate appropriate fault-tolerance strategies to an application. Fault-tolerance strategies for two of the patterns are discussed, and one specific strategy is elaborated and analyzed. The presented strategies have been incorporated into a fault-tolerance support framework called FT-PAS. One objective of the framework is to separate the fault tolerance related details from an application developer's main objectives (separation-of-concerns). The paper presents the additional key features of the framework, and concludes with a discussion on current and future research directions.

Supporting fault-tolerant and open distributed processing using RPC

User-directed means the process of carrying out fault tolerance is dynamic and the fault tolerance mode is chosen by users based on application requirements. In this paper, we introduce a general scheme based on standard MPI to provide the user directed support for application level algorithmic fault tolerance. The user-directed fault tolerance plays the role as a connection between applications and algorithmic fault tolerance. As a case study, our scheme has been incorporated to HPL combined with a non-blocking ABFT technique. We have tested the functional availability of our scheme for fault tolerance in real circumstance. We also evaluated that when there is no failure occurring, our support only brings 2.5 percent overhead. When failure occurs, with our scheme, the scalability of algorithmic fault tolerance maintains well.

Today’s systems are capable of storing large amounts of data in main memory. Particularly, in-memory DBMSs benefit from this development. However, the processing of data from the main memory necessarily has to run via the CPU. This creates a bottleneck, which affects the possible performance of the DBMS. Processing-In-Memory (PIM) is a paradigm to overcome this problem, which was not available in commercial systems for a long time. With the availability of UPMEM, a commercial product is finally available that provides PIM technology in hardware. In this work, we focus on the acceleration of the table scan, a fundamental database query operation. We show and investigate an approach that can be used to optimize this operation by using PIM. We evaluate the PIM scan in terms of parallelism and execution time in benchmarks with different table sizes and compare it to a traditional CPU-based table scan. The result is a PIM table scan that outperforms the CPU-based scan significantly.

The vast quantities of log-based data appearing in data centers has generated an interest in sorting almost-sorted datasets. We revisit the problem of sorting and merging data in main memory, and show that a long-forgotten technique called Patience Sort can, with some key modifications, be made competitive with today's best comparison-based sorting techniques for both random and almost sorted data. Patience sort consists of two phases: the creation of sorted runs, and the merging of these runs. Through a combination of algorithmic and architectural innovations, we dramatically improve Patience sort for both random and almost-ordered data. Of particular interest is a new technique called ping-pong merge for merging sorted runs in main memory. Together, these innovations produce an extremely fast sorting technique that we call P3 Sort (for Ping-Pong Patience+ Sort), which is competitive with or better than the popular implementations of the fastest comparison-based sort techniques of today. For example, our implementation of P3 sort is around 20% faster than GNU Quicksort on random data, and 20% to 4x faster than Timsort for almost sorted data. Finally, we investigate replacement selection sort in the context of single-pass sorting of logs with bounded disorder, and leverage P3 sort to improve replacement selection. Experiments show that our proposal, P3 replacement selection, significantly improves performance, with speedups of 3x to 20x over classical replacement selection.

The increasing main-memory capacity has allowed query execution to occur primarily in main memory. Database systems employ compression, not only to fit the data in main memory, but also to address the memory bandwidth bottleneck. Lightweight compression schemes focus on efficiency over compression rate and allow query operators to process the data in compressed form. For instance, dictionary compression keeps the distinct column values in a sorted dictionary and stores the values as index codes with the minimum number of bits. Packing the bits of each code contiguously, namely horizontal bit packing, has been optimized by using SIMD instructions for unpacking and by evaluating predicates in parallel per processor word for selection scans. Interleaving the bits of codes, namely vertical bit packing, provides faster scans, but incurs prohibitive costs for packing and unpacking. Here, we improve packing and unpacking for vertical bit packing using SIMD instructions, achieving more than an order of magnitude speedup. Also, we optimize horizontal bit packing on the latest CPUs and compare all approaches. While no single variant is better in all cases, vertical bit packing offers a good trade-off by combining the fastest scans with comparably fast packing and unpacking.

Neural network, a nonalgorithmic, nondigital, intensely parallel and distributive information processing system, is being used more and more everyday. The main interest in neural networks is rooted in the recognition that the human brain processes information in a different manner than conventional digital computers. Computers are extremely fast and precise at executing sequences of instructions that have been formulated for them. A human information processing system is composed of neurons switching at speeds about a million times slower than computer gates. Yet, humans are more efficient than computers at such computationally complex tasks as speech and other pattern-recognition problems. Artificial neural systems, or neural networks, are physical cellular systems that can acquire, store, and use experiential knowledge. The knowledge is in the form of stable states or mapping embedded in networks that can be recalled in response to the presentation of cues. In a typical neural data processing procedure, the database is divided into two separate portions called training and test sets. The training set is used to develop the desired network. In this process (depending on the paradigm that is being used), the desired output in the training set is used to help the network learn by adjusting the weights between its neurons or processing elements. Once the network has learned the information in the training set and has"converged," the test set is applied to the network for verification. It is important to note that although the user has the desired output of the test set, it has not been seen by the network. This ensures the integrity and robustness of the trained network. A handful of articles on the use of neural networks in the petroleum industry has appeared in SPE conferences, proceedings, and publications in the past 2 years. These articles can be divided into two categories: those that use neural networks to analyze formation lithology from well logs and those that use neural networks to pick a reservoir model to be used in conventional welltest interpretation studies. These tasks are usually done by log analysts and reservoir engineers, and their automation using a fault-tolerant process may prove valuable. Neural networks can help engineers and researchers by addressing some fundamental petroleum engineering problems as well as specific ones that conventional computing has been unable to solve. Petroleum engineers may benefit from neural networks on occasions when engineering data for design and interpretations are less than adequate. This is an especially common occurrence in the Appalachian basin, where some fields are quite old. Lack of adequate engineering data may also be encountered because of the high cost of coring, well testing, and so on. Neural networks have shown great potential for generating accurate analysis and results from large amounts of historical data that otherwise would seem not to be useful or relevant in the analysis. An example of such a problem was encountered by a gas company for a gas storage field in Ohio. In the absence of appropriate data, which normally would make engineering design and evaluation of the fracturing jobs virtually impossible, a carefully designed neural network was able to predict the performance of fracturing jobs with great accuracy. A linear plot of the actual fracturing job results (data never seen by the network during training) and network predictions resulted in a correlation coefficient of 0.98, where 1.00 is a perfect match. Neural networks have proved to be valuable pattern-recognition tools. They are capable of finding highly complex patterns within large amounts of data. A relevant example is well log interpretation. It is generally accepted that there is more information embedded in well logs than meets the eye. Determination, prediction, or estimation of formation permeability without actual laboratory measurement of the cores or interruption in production for well test data collection has been a fundamental problem for petroleum engineers. From geophysical well log data, it was possible to predict and/or estimate permeability of a highly heterogeneous formation in West Virginia, as shown in Fig 1. Reference Mohaghegh, S., Arefi, R., and Ameri, S.: "Design and Development of an Artificial Neural Network for Prediction of Formation Permeability,"paper SPE 28237 presented at the 1994 SPE Petroleum Computer Conference, July31-Aug. 3, Dallas.

Verifiable Dynamic Searchable Symmetric Encryption (VDSSE) enables users to securely outsource databases (document sets) to cloud servers and perform searches and updates. The verifiability property prevents users from accepting incorrect search results returned by a malicious server. However, we discover that the community currently only focuses on preventing malicious behavior from the server but ignores incorrect updates from the client, which are very likely to happen since there is no record on the client to check. Indeed most existing VDSSE schemes are not sufficient to tolerate incorrect updates from the client. For instance, deleting a nonexistent keyword-identifier pair can break their correctness and soundness. In this paper, we demonstrate the vulnerabilities of a type of existing VDSSE schemes that fail them to ensure correctness and soundness properties on incorrect updates. We propose an efficient fault-tolerant solution that can consider any DSSE scheme as a black-box and make them into a fault-tolerant VDSSE in the malicious model. Forward privacy is an important property of DSSE that prevents the server from linking an update operation to previous search queries. Our approach can also make any forward secure DSSE scheme into a fault-tolerant VDSSE without breaking the forward security guarantee. In this work, we take FAST [1] (TDSC 2020), a forward secure DSSE, as an example, implement a prototype of our solution, and evaluate its performance. Even when compared with the previous fastest forward private construction that does not support fault tolerance, the experiments show that our construction saves 9× client storage and has better search and update efficiency.

In this paper we would like to address two of the issues from the list of issues to be considered by the workshop: formal methods for specifying fault-tolerance requirements, and paradigms for supporting fault-tolerant applications.

Event-driven distributed systems have two important characteristics, which differentiate them from other system types: the existence of several software or hardware components that run simultaneously on different inter-networked nodes, and the use of events as the main vehicle to organize component intercommunication. Clearly, both attributes influence event-driven distributed architectures, which are discussed in this chapter. We start with presenting the event-driven software architecture, which describes various logical components and their roles in events generation, transmission, processing, and consumption. This is used in early phases of distributed event-driven systems’ development as a blueprint for the whole development process including concept, design, implementation, testing, and maintenance. It also grounds important architectural concepts and highlights the challenges faced by event-driven distributed system developers. The core part of the chapter presents several system architectures, which capture the physical realization of event-driven distributed systems, more specifically the ways logical components are instantiated and placed on real machines. Important characteristics such as performance, efficient use of resources, fault tolerance, security, and others are strongly determined by the adopted system architecture and the technologies behind it. The most important research results are organized along five themes: complex event processing, Event-Driven Service Oriented Architecture (ED-SOA), Grid architecture, Peer-to-Peer (P2P) architecture, and Agent architecture. For each topic, we present previous work, describe the most recent achievements, highlight their advantages and limitations, and indicate future research trends in event-driven distributed system architectures.