Optimizing Speculative Execution in Spark Heterogeneous Environments

Many Big Data processing applications nowadays run on large-scale multi-tenant clusters. Due to hardware heterogeneity and resource contentions, straggler problem has become the norm rather than the exception in such clusters. To handle the straggler problem, speculative execution has emerged as one of the most widely used straggler mitigation techniques. Although a number of speculative execution mechanisms have been proposed, as we have observed from real-world traces, the questions of when and where to launch speculative copies have not been fully discussed and hence cause inefficiencies on the performance and energy of Big Data applications. In this paper, we propose a performance model and an energy consumption model to reveal the performance and energy variations with different speculative execution solutions. We further propose a window-based dynamic resource reservation and a heterogeneity-aware copy allocation technique to answer the when and where questions for speculative executions. Evaluations using real-world traces show that our proposed technique can improve the performance of Big Data applications by up to 30% and reduce the overall energy consumption by up to 34%.

The schedulability analysis of real-time embedded systems requires worst case execution time (WCET) analysis for the individual tasks. Bounding WCET involves not only language-level program path analysis, but also modeling the performance impact of complex micro-architectural features present in modern processors. In this paper, we statically analyze the execution time of embedded software on processors with speculative execution. The speculation of conditional branch outcomes (branch prediction) significantly improves a program's execution time. Thus, accurate modeling of control speculation is important for calculating tight WCET estimates. We present a parameterized framework to model the different branch prediction schemes. We further consider the complex interaction between speculative execution and instruction cache performance, that is, the fact that speculatively executed blocks can generate additional cache hits/misses. We extend our modeling to capture this effect of branch prediction on cache performance. Starting with the control flow graph of a program, our technique uses integer linear programming to estimate the program's WCET. The accuracy of our method is demonstrated by tight estimates obtained on realistic benchmarks.

Speculative execution, the base on which modern high-performance general-purpose CPUs are built on, has recently been shown to enable a slew of security attacks. All these attacks are centered around a common set of behaviors: During speculative execution, the architectural state of the system is kept unmodified, until the speculation can be verified. In the event that a misspeculation occurs, then anything that can affect the architectural state is reverted (squashed) and re-executed correctly. However, the same is not true for the microarchitectural state. Normally invisible to the user, changes to the microarchitectural state can be observed through various side-channels, with timing differences caused by the memory hierarchy being one of the most common and easy to exploit. The speculative side-channels can then be exploited to perform attacks that can bypass software and hardware checks in order to leak information. These attacks, out of which the most infamous are perhaps Spectre and Meltdown, have led to a frantic search for solutions.In this work, we present our own solution for reducing the microarchitectural state-changes caused by speculative execution in the memory hierarchy. It is based on the observation that if we only allow accesses that hit in the L1 data cache to proceed, then we can easily hide any microarchitectural changes until after the speculation has been verified. At the same time, we propose to prevent stalls by value predicting the loads that miss in the L1. Value prediction, though speculative, constitutes an invisible form of speculation, not seen outside the core. We evaluate our solution and show that we can prevent observable microarchitectural changes in the memory hierarchy while keeping the performance and energy costs at 11% and 7%, respectively. In comparison, the current state of the art solution, InvisiSpec, incurs a 46% performance loss and a 51% energy increase.

Speculative execution refers to the execution of parts of a computation before the execution of the conditional operations that decide whether it needs to be executed. It has been shown to be a promising technique for eliminating performance bottlenecks imposed by control flow in hardware and software implementations alike. In this paper, we present techniques to incorporate speculative execution in a fine-grained manner into scheduling of control-flow intensive behavioral descriptions. We demonstrate that failing to take into account information such as resource constraints and branch probabilities can lead to significantly sub-optimal performance. We also demonstrate that it may be necessary to speculate simultaneously along multiple paths, subject to resource constraints, in order to minimize the delay overheads incurred when prediction errors occur. Experimental results on several benchmarks show that our speculative scheduling algorithm can result in significant (upto seven-fold) improvements in performance (measured in terms of the average number of clock cycles) as compared to scheduling without speculative execution. Also, the best and worst case execution times for the speculatively performed schedules are the same as or better than the corresponding values for the schedules obtained without speculative execution.

Deriving the Worst-Case Execution Time (WCET) of a task is a challenging process, especially for processor architectures that use caches, out-of-order pipelines, and speculative execution. Despite existing contributions to WCET analysis for these complex architectures, there are open problems. The single-path code generation overcomes these problems by generating time-predictable code that has a single execution trace. However, the simplicity of this approach comes at the cost of longer execution times. This paper addresses performance improvements for single-path code. We propose a time-predictable memory hierarchy with a prefetcher that exploits the predictability of execution traces in single-path code to speed up code execution. The new memory hierarchy reduces both the cache-miss penalty time and the cache-miss rate on the instruction cache. The benefit of the approach is demonstrated through benchmarks that are executed on an FPGA implementation.

Speculative execution increases the performance of the processor by predicting and executing instructions out of order, allowing for faster computation in modern CPUs. However, it introduces critical vulnerabilities that enable transient attacks to leak sensitive data through covert hardware channels, exploiting mechanisms like branch misprediction and out-of-order execution. This project studies the time taken in different cases, both with and without LFENCE, with detailed analysis. LFENCE acts as a speculation barrier to halt risky predictions and mitigate timing leaks. Our results highlight the differences in execution times, providing insights into how speculation influences cache behavior and creates exploitable gaps. Using Python-based visualizations, we map these timing patterns to offer clear insights into vulnerabilities and defenses. By incorporating various defense mechanisms, including LFENCE and other hardware security strategies, this study demonstrates their role in reducing transient leaks, offering a practical understanding of execution timing impacts and performance trade-offs. This accessible demonstration bridges theoretical hardware security concepts with real-world observations, emphasizing effective countermeasures without requiring extensive hardware modifications.

The asynchronous iteration model, called AIAC, has been proven to be an efficient solution for heterogeneous and distributed architectures. An efficient mapping of application tasks is essential to reduce their execution time. In this paper we present a new mapping algorithm, called MAHEVE (Mapping Algorithm for HEterogeneous and Volatile Environments) which is efficient on such architectures and integrates a fault tolerance mechanism to resist computing node failures. Our experiments show gains on a typical AIAC application execution time up to 65%, executed on distributed clusters architectures containing more than 400 computing cores with the JaceP2P-V2 environment.

In the Spark distributed framework, data communication problems (network transfer overhead, network IO bottlenecks) caused by data transfer across nodes/racks are a common cause of performance degradation due to the inconsistency between the task execution location and the data location. Additionally, in heterogeneous environments, Spark's task scheduling strategy cannot fully utilize the advantages of high‐performance nodes. To address the above issues, firstly, this paper proposes a cost‐aware task selection strategy. The strategy models the cost of tasks by considering the impact of data locality and heterogeneous factors on the efficiency of job execution. For scenarios where data locality needs to be reduced for scheduling tasks, the task scheduling problem is transformed into a minimum weighted bipartite graph matching problem, and a greedy matching algorithm is used to solve for the minimum processing cost option. For scenarios that maintain the current data localization level for scheduling tasks, select the task execution with the largest change in task processing cost due to data localization changes. Secondly, the problem is that Spark's delay scheduling algorithm causes resources in the cluster to be in an unnecessary waiting state and reduces cluster resource utilization. In this paper, we propose an adaptive adjustment strategy for delay waiting time based on benefit assessment. This policy improves the resource utilization of the cluster by evaluating the benefit of delay waiting of the scheduler and dynamically adjusting the delay time based on the result of the evaluation. Finally, we implement the proposed strategy in Spark 3.0.0 and evaluate its performance using some representative benchmarks. The experimental results show that, compared with other task scheduling algorithms, the strategy proposed in this paper can effectively improve the execution efficiency of jobs, reduce the execution time of jobs by 15.8%–31.9%, and at the same time reduce the network traffic and improve the CPU utilization.

The contribution of memory latency to execution time continues to increase, and latency hiding mechanisms become ever more important for efficient processor design. While high-end processors can use elaborate techniques like multiple issue, out-of-order execution, speculative execution, value prediction etc. to tolerate high memory latencies, they are often not viable solutions for embedded processors, due to significant area, power and chip complexity overheads. This paper proposes a hardware-software cooperative approach, called priority-based execution to hide cache miss penalty for embedded processors. The compiler classifies the instructions into low-priority and high-priority instructions. The processor executes the high-priority instructions, but delays the execution of low priority instructions. They are executed on a cache miss to hide the cache miss penalty. We empirically evaluate our proposal on the Intel XScale compiler and microarchitecture. Experimental results on benchmarks from Multimedia, MediaBench, MiBench, and SPEC2000 demonstrate an average 17% performance improvements, hiding 75% cache miss penalty.

Speculative execution and instruction reuse are two important strategies that have been investigated for improving processor performance. Value prediction at the instruction level has been introduced to allow even more aggressive speculation and reuse than previous techniques. This study suggests that using compiler support to extend value reuse to a coarser granularity than a single instruction, such as a basic block, may have substantial performance benefits. We investigate the input and output values of basic blocks and find that these values can be quite regular and predictable. For the SPEC benchmark programs evaluated, 90 percent of the basic blocks have fewer than four register inputs, five live register outputs, four memory inputs, and two memory outputs. About 16 to 41 percent of all the basic blocks are simply repeating earlier calculations when the programs are compiled with the -O2 optimization level in the GCC compiler. Compiler optimizations, such as loop-unrolling and function inlining, affect the sizes of basic blocks, but have no significant or consistent impact on their value locality, nor the resulting performance. Based on these results, we evaluate the potential benefit of basic block reuse using a novel mechanism called the block history buffer. This mechanism records input and live output values of basic blocks to provide value reuse at the basic block level. Simulation results show that using a reasonably sized block history buffer to provide basic block reuse in a 4-way issue superscalar processor can improve execution time for the tested SPEC programs by 1 to 14 percent, with an overall average of 9 percent when using reasonable hardware assumptions.

In this paper, we propose stochastic scheme models with two kinds of errors for a computer system. A highly reliable computer system needs greatly to be backed up successfully in case of computer system crashes due to hardware failures and malicious software infections. We aim to determine appropriate backup times in computer processing of constant and random tasks. For this purpose, the trade-off between backup times and data losses due to errors is considered. The total execution times until the computing processing completes are obtained, and optimal backup policies to minimize them are derived analytically. Furthermore, we take up extended models of random tasks with [Formula: see text] sub-tasks and analyze them.

Evaluating the Force Matrix constitutes the most computationally intensive part of a Classical Molecular Dynamics (MD) simulation. In three-body MD simulations, the total energy of the system is determined by the energy of every unique triple in the system and the force matrix is three-dimensional. The execution time of a three-body MD algorithm is thus proportional to the cube of the number of atoms in the system. Fortunately, there exist symmetries in the Force Matrix that can be exploited to improve the running time of the algorithm. While this optimization is straight forward to implement in the case of sequential code, it has proven to be nontrivial for parallel code even in a homogeneous environment.In this paper, we present two force matrix transformations that are capable of exploiting the symmetries in a 3-body force matrix in both a homogeneous and a heterogeneous environment while balancing the load among all the participating processors. The first transformation distributes the number of interactions to be computed uniformly among all the slices of the force matrix along any of the axes. The transformed matrix can be scheduled using any well known heterogeneous slice-level scheduling technique. The second transformation distributes interactions to be computed uniformly over the entire volume of the force matrix allowing us to perform a block decomposition of the force matrix. The transformed force matrix can be scheduled by any block level scheduling algorithm. We also derive theoretical bounds for efficiency and load balance for our transformations and prove some interesting and useful properties of our transformations and evaluate their advantages and disadvantages. The performance of an MPI implementation of the transformations is studied in terms of the Step Time Variation Ratio (STVR) in a homogeneous and heterogeneous environment.

Brain tumor detection stands as a vital medical imaging operation which demands high accuracy and efficiency to achieve proper diagnostic results and therapeutic plans. This research examines the performance capability of machine learning classifiers that combine JPEG Coefficient Filter (JPEG CF) with Gabor Filter alongside three different classifiers which include Decision Stump and Decision Tree and IBK (Instance-Based Learning). Performance metrics that included accuracy, precision, recall, the ROC curve and PRC curve together with execution time determined the predictive ability of the models towards brain tumors during the assessment. The combination of JPEG CF with Decision Stump delivered the best accuracy rate of 97.16% which established it as the optimal model for tumor diagnosis. The Gabor Filter + Decision Stump approach achieved an accuracy rate of 96.71% along with the most robust classification reliability through its ROC values at 0.99% and PRC values at 0.99%. The Gabor Filter + Decision Tree produced the smallest accuracy ratings (87.43%) combined with a precision rate of 0.89 and recall rate of 0.87 while having the weakest ROC indicator at 0.83 to become the least effective detection method. The execution time for JPEG CF + IBK reached 0.25 seconds and delivered an accuracy level of 92.87% although it was the fastest model. Real-time application processing suffers from Gabor Filter + IBK (1.14 seconds) alongside Gabor Filter + Decision Stump (1.13 seconds) because these models require the longest execution times. The combination of JPEG CF models with Decision Stump or IBK results in the best performance level for brain tumor classification tasks by achieving effective accuracy and speed. The research findings show both feature selection and decision algorithms affect how well tumors are detected and the speed of the detection process.

On a distributed memory machine, hand-coded message passing leads to the most efficient execution, but it is difficult to use. Parallelizing compilers can approach the performance of hand-coded message passing by translating data-parallel programs into message passing programs, but efficient execution is limited to those programs for which precise analysis can be carried out. Shared memory is easier to program than message passing and its domain is not constrained by the limitations of parallelizing compilers, but it lags in performance. Our goal is to close that performance gap while retaining the benefits of shared memory. In other words, our goal is (1) to make shared memory as efficient as message passing, whether hand-coded or compiler-generated, (2) to retain its ease of programming, and (3) to retain the broader class of applications it supports.To this end we have designed and implemented an integrated compile-time and run-time software DSM system. The programming model remains identical to the original pure run-time DSM system. No user intervention is required to obtain the benefits of our system. The compiler computes data access patterns for the individual processors. It then performs a source-to-source transformation, inserting in the program calls to inform the run-time system of the computed data access patterns. The run-time system uses this information to aggregate communication, to aggregate data and synchronization into a single message, to eliminate consistency overhead, and to replace global synchronization with point-to-point synchronization wherever possible.We extended the Parascope programming environment to perform the required analysis, and we augmented the TreadMarks run-time DSM library to take advantage of the analysis. We used six Fortran programs to assess the performance benefits: Jacobi, 3D-FFT, Integer Sort, Shallow, Gauss, and Modified Gramm-Schmidt, each with two different data set sizes. The experiments were run on an 8-node IBM SP/2 using user-space communication. Compiler optimization in conjunction with the augmented run-time system achieves substantial execution time improvements in comparison to the base TreadMarks, ranging from 4% to 59% on 8 processors. Relative to message passing implementations of the same applications, the compile-time run-time system is 0-29% slower than message passing, while the base run-time system is 5-212% slower. For the five programs that XHPF could parallelize (all except IS), the execution times achieved by the compiler optimized shared memory programs are within 9% of XHPF.

Task scheduling is a crucial key component for the efficient execution of data-intensive applications on distributed environments, by which many machines must be coordinated to reduce execution times and bandwidth consumption. This paper presents ADAGE, a data-aware scheduler designed to efficiently execute data-intensive workflows in large-scale computers. The proposed scheduler is based on three key features: i) critical path analysis, for discovering the critical tasks of a workflow and reducing data transferring between nodes; ii) work giving, a new dynamic planning strategy for migrating tasks from overloaded to unloaded nodes; and iii) task replication, which executes task replicas on different nodes for improving both execution time and fault tolerance. Experiments performed on a distributed computing environment composed of up to 1,024 processing nodes show that ADAGE achieves better performances than existing scheduling systems, obtaining an average reduction of up to 66% in execution time.