Multiversion Concurrency Control on Intermittent Systems

: Our recent work on the modeling human-like multi-task behaviors has led to unexpected insights into a possible mechanism contributing to individual differences in human performance. We begin by outlining our modeled conflict resolution strategy for the graceful sharing of perceptual, cognitive, and motor resources among multiple in-process tasks. We then discuss how that same mechanism might drive individual differences in concurrent task execution. The modeled strategy, in this case operating on subtask structure, may be one contributor to how individual operators differentially decompose and interleave concurrent tasks. We further suggest that the same mechanisms and structures might evolve over time as an operator learns to concurrently engage multiple related tasks. Theory building for multi-task behaviors as explored through model development has opened a new line of inquiry into potential mechanisms for individual differences in performance. Our recent work on the modeling human-like multi-task behaviors has led to unexpected insights into a possible mechanism contributing to individual differences in human performance. We begin by outlining our modeled conflict resolution strategy for the graceful sharing of perceptual, cognitive, and motor resources among multiple in-process tasks. We then discuss how that same mechanism might drive individual differences in concurrent task execution. The modeled strategy, in this case operating on subtask structure, may be one contributor to how individual operators differentially decompose and interleave concurrent tasks. We further suggest that the same mechanisms and structures might evolve over time as an operator learns to concurrently engage multiple related tasks. Theory building for multi-task behaviors as explored through model development has opened a new line of inquiry into potential mechanisms for individual differences in performance.

Multi-Version Concurrency Control (MVCC) is a database management technique that enables concurrent access to a database while maintaining consistency and isolation between transactions. By maintaining multiple versions of records, MVCC allows readers to access older versions of data while writers update the current version, ensuring that each transaction gets a consistent snapshot of the data without being blocked by others. This reduces the need for locks, allowing higher concurrency and better performance, especially in read-heavy environments. Each transaction is assigned a timestamp to determine which version of the data it can access, and older versions are eventually cleaned up through garbage collection. While MVCC provides high throughput and scalability, it introduces storage overhead due to the need to store multiple versions and can become complex as the number of versions grows. Additionally, MVCC can face issues like phantom reads and write skew, especially in transactions with complex interactions. Despite these challenges, MVCC is widely used in distributed and relational database systems, such as PostgreSQL and MySQL, to provide efficient concurrency control and ensure consistency without the need for heavy locking mechanisms. MVCC is particularly effective in environments where read and write operations are frequent, as it minimizes contention between transactions. By enabling transactions to work with a snapshot of the data, it avoids conflicts that would otherwise arise from simultaneous read and write operations. However, one downside is that the system must manage and track the various versions of each record, which can result in additional overhead for both storage and garbage collection. In some systems, when the number of versions becomes large, it may impact the system’s overall performance. It is especially beneficial in systems where many transactions read the same data concurrently while only a few perform updates. As database systems continue to evolve, improvements in MVCC implementations, such as optimized garbage collection, are helping to mitigate its limitations and enhance scalability. In conclusion, MVCC strikes a balance between concurrency and consistency, making it an ideal choice for certain high-performance, multi-user applications. This Paper addresses the storage overhead of MVCC with optimistic concurrency control.

Pooling idle computing resources from stationary and computationally powerful vehicles at a nearby parking lot is one way to meet increasing computation demands from mobile devices at the network's edge. This new computing paradigm is called vehicular cloud (VC). Despite the research in performance modeling of VC having gained increasing interest in recent years, a tractable solution for job completion time under concurrent task execution has yet to be seen. Thus, this work investigates the computation capabilities of a VC by analytically determining job completion time when the VC simultaneously executes several tasks. Under several stochastics assumptions, we derive the probability density functions of completion times of the tasks, which also include the job completion time and their first moments. Finally, we present the numerical results for the analysis and the simulation results to show the correctness of the work.

Main-memory multicore transactional systems have achieved excellent performance using single-version optimistic concurrency control (OCC), especially on uncontended workloads. Nevertheless, systems based on other concurrency control protocols, such as hybrid OCC/ locking and variations on multiversion concurrency control (MVCC), are reported to outperform the best OCC systems, especially with increasing contention. This paper shows that implementation choices unrelated to concurrency control can explain some of these performance differences. Our evaluation shows the strengths and weaknesses of OCC, MVCC, and TicToc concurrency control under varying workloads and contention levels, and the importance of several implementation choices called basis factors. Given sensible basis factor choices, OCC performance does not collapse on high-contention TPC-C. We also present two optimization techniques, deferred updates and timestamp splitting, that can dramatically improve the high-contention performance of both OCC and MVCC. These techniques are known, but we apply them in a new context and highlight their potency: when combined, they lead to performance gains of \(4.74\times \) for MVCC and \(5.01\times \) for OCC in a TPC-C workload.

Optimistic concurrency control, or OCC, can achieve excellent performance on uncontended workloads for main-memory transactional databases. Contention causes OCC's performance to degrade, however, and recent concurrency control designs, such as hybrid OCC/locking systems and variations on multiversion concurrency control (MVCC), have claimed to outperform the best OCC systems. We evaluate several concurrency control designs under varying contention and varying workloads, including TPCC, and find that implementation choices unrelated to concurrency control may explain much of OCC's previously-reported degradation. When these implementation choices are made sensibly, OCC performance does not collapse on high-contention TPC-C. We also present two optimization techniques, commit-time updates and timestamp splitting , that can dramatically improve the high-contention performance of both OCC and MVCC. Though these techniques are known, we apply them in a new context and highlight their potency: when combined, they lead to performance gains of 3.4X for MVCC and 3.6X for OCC in a TPC-C workload.

Distributed storage systems run transactions across machines to ensure serializability in single-service architecture. Traditional protocols for distributed transactions are based on two-phase commit (2PC) or multi-version concurrency control (MVCC). 2PC serializes transactions as soon as they conflict and MVCC resorts to aborts, leaving many opportunities for con-currency on the distributed storage system. In the micro-service architecture, service nodes are deployed in heterogeneous distributed systems. 2PC and MVCC are struggling to break through the performance limitations of their existence. This paper presents 2PC+, a novel concurrency control protocol for distributed transactions that outperforms 2PC allowing more concurrency in micro-service architecture. 2PC+ is based on the traditional 2PC and MVCC, combined with transaction thread synchronization blocking optimization algorithm SAOLA. Then, we verify the algorithm SAOLA using the temporal logic of action TLA+ language. Finally, we compare 2PC+ to 2PC applying a case of Ctrip MSECP. When concurrent threads of the service call reach a certain threshold, 2PC+’s RT performance is improved by 2.87 times to 3.77 times compared with 2PC, and 2PC+’s TPS performance is 323.7% to 514.4% higher than 2PC.

Concurrent execution of design and construction tasks is an important way to realize the integration of them in design-build (DB) mode, but it may bring about period risk due to multiple rework and frequent information transfer in local scope. To solve this problem, this study constructs a concurrent execution strategy model from the perspective of quantitative analysis with the decision goal of minimizing the total execution duration. The results show that: first, when the probability of information change between design and construction tasks decreases gradually, the optimal overlap degree increases and the optimal parallelism decreases. Second, learning effect can effectively shorten the minimum duration corresponding to the optimal overlap degree under any degree of parallelism. Thirdly, with the decrease of the probability of information change between design and construction tasks and the strengthening of learning effect, the shortening rate of the actual execution duration corresponding to the optimal parallelism will be gradually greater than the increasing rate of the actual execution duration. The findings of the study provide suggestions for exploring the mechanism of concurrent execution in DB mode, and also provide countermeasures for better realization of integration of design and construction in DB mode.

Real time database transactions have explicit timing constraints, such as deadlines. Conventional database systems may lack features required for supporting real time transactions. Meeting the requirements of RTDBS will require a balanced and coordinated effort between concurrency control and real time scheduling. One approach is to combine existing concurrency control protocols with real time scheduling algorithms. To meet more deadlines, concurrency control protocols can be modified to favor more urgent transactions. Another approach is to utilize the semantics based protocols in real time applications to allow more concurrent executions in favor of more urgent timing constraints. We discuss techniques that can be used to design and implement real time databases.

A formal semantics is introduced for a Process Network model, which combines streaming and reactive control processing with task parallelism properties suitable to exploit multi-cores. Applications that react to environment stimuli are implemented by communicating sporadic and periodic tasks, programmed independently from an execution platform. Two functionally equivalent semantics are defined, one for sequential execution and one real-time. The former ensures functional determinism by implying precedence constraints between jobs (task executions), hence, the program outputs are independent from the task scheduling. The latter specifies concurrent execution on a real-time platform, guaranteeing all model’s constraints; it has been implemented in an executable formal specification language. The model’s implementation runs on multi-core embedded systems, and supports integration of run-time managers for shared HW/SW resources (e.g. for controlling QoS, resource interference or power consumption). Finally, a model transformation approach has been developed, which allowed to port and statically schedule a real spacecraft on-board application on an industrial multi-core platform.

Deterministic databases are able to run transactions efficiently in a distributed setting with minimal coordination. Research has shown many benefits from having determinism, from more lightweight database replication, to no/cheaper distributed commit, to high throughput during live migration. Besides those known applications, permissioned blockchain can also be viewed as an application of the deterministic database with security added. A permissioned blockchain maintains a fully-replicated database, where each replica is synchronized by getting blocks of updates. Early permissioned blockchains run a consensus protocol (e.g., PBFT) to agree on the input and rely on the replicas to execute the transactions serially to uphold determinism. Fabric allows concurrent transaction executions but rely on running consensus to agree on the read-write-sets and broadcasting them, which incurs an excessive volume of network traffic. Deterministic concurrency control combines the best of both worlds. It enables solving consensus on the small input, and each replica is able to execute transactions independently with high concurrency. Although powerful, contemporary deterministic databases are still primitive because they support only one isolation level -- Serializable. Furthermore, many deterministic concurrency control schemes are overly pessimistic and cause many unnecessary transaction aborts. These factors motivate us to design DCC, a suite of better deterministic concurrency control schemes, not only for Serializable but also for practical isolation levels beyond Serializable.

Emergent relations and competing behavior: Effects on interruptions, accuracy and reaction time.

Mobile Edge Computing (MEC) is a promising solution to enhance the computing capability of resource-limited networks. A fundamental problem in MEC is efficiently offloading tasks from user devices to edge servers. However, there still exists a gap to deploy in real-world environments: 1) traditional centralized approaches needs complete information of edge network, ignoring the communication costs generated by synchronization, 2) previous works do not consider concurrent computation on edge servers, which may cause dynamic changes in the environment, and 3) the scheduling algorithm should deliver individualized decisions for different users independently and with high efficiency To solve this mismatch, we studied a multi-user task offloading problem where user devices make offloading decisions independently. We consider the concurrent execution of tasks and formulate a non-divisible and delay-aware task offloading problem to jointly minimize the dropped task ratio and long-term latency. We propose a decentralized task scheduling algorithm based on DRL that makes offloading decisions without knowing the information of other user devices. We employ Double-DQN, Dueling-DQN, Prioritized Replay Memory, and Recurrent Neural Network (RNN) techniques to improve the algorithm's performance. The results of simulation experiments show that our method can significantly reduce the long-term latency and dropped task ratio compared to the baseline algorithms.

The evolution and advancements in Information and Communication Technologies (ICT) have enabled large scale distributed computing with a huge chunk of applications for massive number of users. This has obviously generated large volumes of data, thus severely burdening the processing capacity of computers as well as the inflexible traditional networks. State-of-the-art methods for addressing datacenter level performance fixes are yet found wanting for sufficiently addressing this huge processing, storage, and network movement with proprietary protocols for this voluminous data. In this chapter the works have focused on addressing the backend server performance through effective reducer placement, intelligent compression policy, handling slower tasks and in-network performance boosting techniques through effective traffic engineering, traffic classification, topology discovery, energy minimization and load balancing in datacenter-oriented applications. Hadoop, the defacto standard in distributed big data storage and processing, has been designed to store data with its Hadoop Distributed File System (HDFS) and processing engine MapReduce large datasets reliably. However, the processing performance of Hadoop is critically dependent on the time taken to transfer data during the shuffle generated during MapReduce. Also, during concurrent execution of tasks, slower tasks need to be properly identified and efficiently handled to improve the completion time of jobs. To overcome these limitations, three contributions have been made; (i) Compression of generated map outputs at a suitable time when all the map tasks are yet to be completed to shift the load of network onto the CPU; (ii) Placing the reducer onto the nodes where the computation done is highest based on a couple of counters, one maintained at the rack level and another at node level, to minimize the run-time data copying; and (iii) Placing the slower map tasks onto the nodes where the computation done is highest and network is handled by prioritizing. Software defined networking (SDN) has been a boon for next generation networking owing to the separation of control plane from the data plane. It has the capability to address the network requirements in a timely manner by setting flows for every to and fro data movement and gathering large network statistics at the controller to make informed decisions about the network. A core issue in the network for the controller is traffic classification, which can substantially assist SDN controllers towards efficient routing and traffic engineering decisions. This chapter presents a traffic classification scheme utilizing three classifiers namely Feed-forward Neural Network (FFNN), Logistic Regression (LR), Naïve Bayes and employing Particle Swarm Optimization (PSO) for improved traffic classification with less overhead and without overlooking the key Quality of Service (QoS) criterion. Also lowering energy minimization and link utilization has been an important criterion for lowering the operating cost of the network and effectively utilizing the network. This issue has been addressed in the chapter by formulating a multi-objective problem while simultaneously addressing the QoS constraints by proposing a metaheuristic, since no polynomial solution exists and hence an evolutionary based metaheuristic (Clonal Selection) based energy optimization scheme, namely, Clonal Selection Based Energy Minimization (CSEM) has been devised. The obtained results show the efficacy of the proposed traffic classification scheme and CSEM based solution as compared with the state-of-the-art techniques. SDN has been a promising newer network paradigm but security issues and expensive capital procurement of SDN limit its full deployment hence moving to a hybrid SDN (h-SDN) deployment is only logical moving forward. The usage of both centralized and decentralized paradigms in h-SDN with intrinsic issues of interoperability poses challenges to key issues of topology gathering by the controller for proper allocation of network resources and traffic engineering for optimum network performance. State-of-the-art protocols for topology gathering, such as Link Layer Discovery Protocol (LLDP) and Broadcast Domain Discovery Protocol (BDDP) require a huge number of messages and such schemes only gather link information of SDN devices leaving out legacy switches’ (LS) links which results in sub-optimal performance. This chapter provides novel schemes which unearth topology discovery by requiring fewer messages and gathering link information of all the devices in both single and multi-controller environments (might be used when scalability issue is prevalent in h-SDN). Traffic engineering problems in h-SDN are addressed by proper placement of SDN nodes in h-SDN by utilizing the analyzing key criterion of traffic details and the degree of a node while lowering the link utilization in real-time topologies. The results of the proposed schemes for topology discovery and SDN node placement demonstrate the merits as compared with the state-of-the-art protocols.

A serializable concurrency control mechanism ensures consistency for OLTP systems at the expense of a reduced transaction throughput. A DBMS, therefore usually offers the possibility to allocate lower isolation levels for some transactions when it is safe to do so. However, such trading of consistency for efficiency does not come with any safety guarantees. In this article, we study the mixed robustness problem which asks whether, for a given set of transactions and a given allocation of isolation levels, every possible interleaved execution of those transactions that is allowed under the provided allocation is always serializable. That is, whether the given allocation is indeed safe. While robustness has already been studied in the literature for the homogeneous setting where all transactions are allocated the same isolation level, the heterogeneous setting that we consider in this article, despite its practical relevance, has largely been ignored. We focus on multiversion concurrency control and consider the isolation levels that are available in PostgreSQL and Oracle: read committed (RC), snapshot isolation (SI), and serializable snapshot isolation (SSI). We show that the mixed robustness problem can be decided in polynomial time. In addition, we provide a polynomial time algorithm for computing the optimal robust allocation for a given set of transactions, prioritizing lower over higher isolation levels. The present results therefore establish the groundwork to automate isolation level allocation within existing databases supporting multiversion concurrency control.

In this paper, a methodology for mapping algorithms onto heterogeneous suite of supercomputers is presented. A n approach for selecting an optimal suite of computers for solving problems with diverse computational requirements, called Heterogeneous Optimal Selection Theory (HOST), is presented. HOST is an extension t o Augmented Optimal Selection Theory in two ways: It incorporates heterogeneous parallelism embedded in the tasks, and it reflects the costs associated in using various fine grain mapping strategies at individual machine level. The proposed mapping methodology is based on the Cluster-M programming paradigm. For the mapping purpose, the input format, assumed in HOST, is modeled in terms of Hierarchical Cluster-M specification and representation. For a given problem, a hj'ierarchical Cluster-M specification is generated t o indicate the execution of concurrent tasks at different stizges of the computation. This specification is then mapped onto the Hierarchical ClusterM representation ofthe underlying heterogeneous suite of supercomputers.