An experimental study on execution time variation in computer experiments

Traditional hard real-time scheduling algorithms require the use of the worst-case execution times to guarantee that deadlines will be met. Unfortunately, many algorithms with parameters derived from sensing the physical world suffer large variations in execution time, leading to pessimistic overall utilization, such as visual recognition tasks. In this article, we present ZS-QRAM, a scheduling approach that enables the use of flexible execution times and application-derived utility to tasks in order to maximize total system utility. In particular, we provide a detailed description of the algorithm, the formal proofs for its temporal protection, and a detailed, evaluation. Our evaluation uses the Utility Degradation Resilience (UDR) showing that ZS-QRAM is able to obtain 4× as much UDR as ZSRM, a previous overbooking approach, and almost 2× as much UDR as Rate-Monotonic with Period Transformation (RM/TP). We then evaluate a Linux kernel module implementation of our scheduler on an Unmanned Air Vehicle (UAV) platform. We show that, by using our approach, we are able to keep the tasks that render the most utility by degrading lower-utility ones even in the presence of highly dynamic execution times.

Minor variations in execution time can lead to out-sized effects on the behavior of an application as a whole. There are many sources of such variation within modern multi-core computer systems. For an otherwise deterministic application, we would expect the execution time variation to be non-existent (effectively zero). Unfortunately, this expectation is in error. For instance, variance in the realized execution time tends to increase as the number of processes per compute core increases. Recognizing that characterizing the exact variation or the maximal variation might be a futile task, we take a different approach, focusing instead on the best case variation. We propose a modified (truncated) Levy distribution to characterize this variation. Using empirical sampling we also derive a model to parametrize this distribution that doesn’t require expensive distribution fitting, relying only on known parameters of the system. The distributional assumptions and parametrization model are evaluated on multi-core systems with the common Linux completely fair scheduler.

Providing high-quality broadcast services for soft real-time applications over wireless networks such as CDMA2000, which have high bit error rates, requires the control of errors that occur during data transmission. Reed-Solomon (RS) forward error correction (FEC) in the medium access control (MAC) layer performs this role in 3G broadcast services. We propose new analytic models for predicting the performance of RS coding and its execution time, which take into account the memory property of a fading channel, different channel conditions, and a variable level of block interleaving. We identify RS decoding as a significant cause of variability in execution time, taking the form of jitter, which depends on the channel conditions. We analyze the size of buffer required to absorb the jitter under different channel conditions. We then formulate a trade-off between the performance of RS coding and the delay that it causes in transmitting a fixed amount of data with different levels of block interleaving. Finally, we show how to balance the quality with which content is presented against an acceptable buffering delay, which is very important to soft real-time applications, by using an adequate level of block interleaving. This study offers a guide for the provision of efficient broadcast services in real time with stochastically guaranteed quality.

Multimedia applications are an increasingly important workload for general-purpose processors. This paper analyzes frame-level execution time variability for several multimedia applications on general-purpose architectures. There are two reasons for such an analysis. First, it has been conjectured that complex features of such architectures (e.g., out-of-order issue) result in unpredictable execution times, making them unsuitable for meeting real-time requirements of multimedia applications. Our analysis tests this conjecture. Second, such an analysis can be used to effectively employ recently proposed adaptive architectures.

The exponential growth of genomic and omics data has necessitated scalable, reproducible, and efficient computational frameworks. This study investigates the role of bioinformatics pipelines within marketplace ecosystems, focusing on their impact on data-driven genomic and biotech analytics. By evaluating multiple pipeline modules across pre-processing, analysis, and annotation stages, the findings reveal significant variation in execution time, memory utilization, accuracy, and reproducibility. Comparative assessment of marketplaces such as DNAnexus, Seven Bridges, and Galaxy demonstrates that technical interoperability, containerization efficiency, and regulatory compliance are critical drivers of adoption, while cost structures remain a major determinant for user segmentation. Machine learning integration within pipelines substantially enhanced predictive modeling for biomarker detection and drug-target prediction, with deep learning and random forest models achieving the strongest performance. Multivariate statistical analyses confirmed that interoperability and compliance positively influence adoption rates, whereas higher costs act as a barrier. Dimensionality reduction through PCA enabled effective cohort stratification, while cluster analysis identified distinct marketplace adoption patterns, emphasizing the heterogeneity of user needs. Overall, the study underscores the transformative potential of integrating pipelines with marketplace infrastructures, enabling scalable biotech innovation, democratizing access to advanced tools, and advancing precision medicine.

Streaming applications are promising targets for effectively utilizing multicores because of their inherent amenability to pipelined parallelism. While existing methods of orchestrating streaming programs on multicores have mostly been static, real-world applications show ample variations in execution time that may cause the achieved speedup and throughput to be sub-optimal. One of the principle challenges for moving towards dynamic orchestration has been the lack of approaches that can predict or accurately estimate upcoming dynamic variations in execution efficiently, well before they occur. In this paper, we propose an automated dynamic execution behavior prediction approach that can be used to efficiently estimate the time that will be spent in different pipeline stages for upcoming inputs without requiring program execution. This enables dynamic balancing or scheduling of execution to achieve better speedup. Our approach first uses dynamic taint analysis to automatically generates an input-based execution characterization of the streaming program, which identifies the key control points where variation in execution might occur with the associated input elements that cause these variations.We then automatically generate a light-weight emulator from the program using this characterization that can simulate the execution paths taken for new streaming inputs and provide an estimate of execution time that will be spent in processing these inputs, enabling prediction of possible dynamic variations. We present experimental evidence that our technique can accurately and efficiently estimate execution behaviors for several benchmarks. Our experiments show that dynamic orchestration using our predicted execution behavior can achieve considerably higher speedup than static orchestration.

Different from a general-purpose system, a real-time system requires stringent timing guarantees. While existing offline analysis techniques can provide timing guarantees using the worst-case execution time (WCET) of individual tasks, a variation of actual execution time makes it difficult to build covert timing channel. In this paper, we first present a novel covert timing channel, which considers actual execution time distribution of tasks and controls execution time to leak data between conspirators; we demonstrate that it is possible to leak data in real-time systems. Second, we suggest two enhancing techniques called S-R LCM (sender-receiver least common multiple) and noise area to reduce noise in communication. Through simulations, we demonstrate that our covert timing channel can serve trade-off between transmission speed and accuracy; that is, it shows average 50.2%, 54.6% and 51.3% accuracy for 100 test cases with thresholds 0, 1.4 and 2.8. Average 58.4% accuracy is accomplished with best threshold values for 100 test cases, and the maximum accuracy for a single test case is recorded 100.0%.

In this paper, processor scheduling policies that "save" processors are introduced and studied. In a multiprogrammed parallel system, a "processor saving" scheduling policy purposefully keeps some of the available processors idle in the presence of work to be done. The conditions under which processor saving policies can be more effective than their greedy counterparts, i.e., policies that never leave processors idle in the presence of work to be done, are examined. Sensitivity analysis is performed with respect to application speedup, system size, coefficient of variation of the applications' execution time, variability in the arrival process, and multiclass workloads. Analytical, simulation, and experimental results show that processor saving policies outperform their greedy counterparts under a variety of system and workload characteristics.

Recently there has been a surge of interest in developing performance debugging tools to help programmers tune their applications for better memory performance [2, 4, 10]. These tools vary both in the detail of feedback provided to the user, and in the run-time overbead of using them. MemSpy [10] is a simulation-based tool which gives programmers detailed statistics on the memory system behavior of applications. It provides information on the frequency and causes of cache misses, and presents it in terms of source-level data and code objects with which the programmer is familiar. However, using MemSpy increases a program's execution time by roughly 10 to 40 fold. This overhead is generally acceptable for applications with execution times of several minutes or less, but it can be inconvenient when tuning applications with very long execution times.This paper examines the use of trace sampling techniques to reduce the execution time overhead of tools like MemSpy. When simulating one tenth of the references, we find that MemSpy's execution time overhead is improved by a factor of 4 to 6. That is, the execution time when using MemSpy is generally within a factor of 3 to 8 times the normal exwution time. With this improved performance, we observe only small errors in the performance statistics reported by MemSpy. On moderate sized caches of 16KB to 128KB, simulating as few as one tenth of the references (in samples of 0.5M references each) allows us to estimate the program's actual cache miss rate with an absolute error no greater than 0.3% on our five benchmarks. These errors are quite tolerable within the context of performance bugging. With larger caches we can also obtain good accuracy by using longer sample lengths. We conclude that, used with care, trace sampling is a powerful technique that makes possible performance debugging tools which provide both detailed memory statistics and low execution time overheads.

Evaluation of Level of Confidence and Optimization of Roll-back Recovery with Checkpointing for Real-Time Systems

Over the last decade the relative latency of access to shared memory by multicore increased as wire resistance dominated latency and low wire density layout pushed multi-port memories farther away from their ports. Various techniques were deployed to improve average memory access latencies, such as speculative pre-fetching and branch-prediction, often leading to high variance in execution time which is unacceptable in real-time systems. Smart DMAs can be used to directly copy data into a layer-1 SRAM, but with overhead. The VLIW architecture, the de-facto signal-processing engine, suffers badly from a breakdown in lock-step execution of scalar and vector instructions. We describe the Split Latency Adaptive Pipeline (SLAP) VLIW architecture, a cache performance improvement technology that requires zero change to object code, while removing smart DMAs and their overhead. SLAP builds on the Decoupled Access and Execute concept by 1) breaking lock-step execution of functional units, 2) enabling variable vector length for variable data-level parallelism, and 3) adding a novel triangular-load mechanism. We discuss the SLAP architecture and demonstrate the performance benefits on real traces from a wireless baseband-system (where even the most compute intensive functions suffer from an Amdahl’s law limitation due to a mixture of scalar and vector processing).

Parallel computers provide an efficient and economical way to solve large-scale and/or time-constrained scientific, engineering, and industry problems. Consequently, there is a need to predict the performance order of both deterministic and non-deterministic parallel algorithms. The performance prediction of the traveling salesman problem (TSP) is a challenging problem because similar input data sets may cause significant variability in execution times. Parallel performance of data-dependent algorithms depends on the problem size, the number of processors, and other parameters. Discovering the main other parameters is the real key to obtain a good estimation of performance order. This paper presents a novel methodology to the problem of predicting the performance of a parallel algorithm for solving the TSP. The entire process explores data in search of patterns and/or relationships detecting the main parameters that affect performance. Then, it uses the measured values for this limited number of inputs to produce a multiple-linear-regression model. Finally, the regression equation allows for predicting how the algorithm will respond when given new input data sets. The preliminary experimental results are quite promising.

We present a comprehensive performance analysis of three major scientific computing platforms-Python, Julia, and R-in solving two-dimensional potential flow around dual square obstacles. We implemented a numerical solution using the Jacobi iteration method across all three platforms, with careful consideration of boundary conditions and convergence criteria. The numerical implementation successfully demonstrated convergence with an L2 norm reduction from 100 to 10-2 over 40,000 iterations, capturing fundamental flow physics including stagnation points (Cp ≅ 1), flow acceleration regions (Cp < 0), and obstacle interference effects. Through 200 independent executions and 10,000 bootstrap iterations, our comparative performance analysis revealed that Julia achieved superior computational efficiency with a median runtime of approximately 30 seconds, despite higher memory requirements (512MB), while Python offered balanced performance (45s, 108MB) and R showed higher variability in execution time (65s, 185MB). Statistical significance of these performance differences was confirmed through rigorous testing using Mann-Whitney U tests with Bonferroni correction (αadjusted = 0.0167) and bootstrap resampling methods. The results provide valuable insights into the trade-offs between computational efficiency and resource utilization across modern scientific computing platforms, while also validating the numerical accuracy of our potential flow solver implementation. This benchmarking study contributes to the growing body of knowledge regarding platform selection for computational fluid dynamics applications, particularly in educational and rapid prototyping contexts.

Execution times are crucial for effectiveness of tasks or jobs scheduling. It is very hard to accurately estimate execution times because they are influenced by many factors. Though there are some models for traditional machine scheduling problems, no attention has been paid on virtual machine scheduling in cloud computing. Based on cloud agent (VM administrator, scheduler or intelligent procedure) experiences, we develop integrated learning effects models to obtain accurate execution times. Based on the constructed learning effects model for single virtual machine scheduling, optimal schedule rules are proposed for minimizing makespan, the total completion time and the sum of (square) completion times. Problems with the total weighted completion time and the maximum lateness minimization are proved to be optimally solvable in polynomial time only for certain assumptions. Furthermore, we adapt the developed learning effects model to two special m-virtual machine MapReduce scenarios, for which optimal schedule rules are introduced correspondingly. Optimal solutions are demonstrated by examples of the problems under study using the constructed rules.

Scheduling can require analyzing not only the total computation time of a task, but also the remaining execution time, R(t)/sub /spl Delta/t/, after accumulated time /spl Delta/t. Often a software program's execution time is characterized by a single value (mean). When scheduling is based on partial execution (a common scenario in multimedia systems) a more accurate estimation of remaining time (R(t)/sub /spl Delta/t/) is desired than can be obtained from just the initial mean value, in order to have effective scheduling decisions. The remaining time approach can provide more accurate estimation, and therefore more effective scheduling, in time-sensitive situations. We developed an analytical model for computing expected remaining execution time, (R(t)/sub /spl Delta/t/)~ , of software programs from their execution time and probability distributions. To implement the equations, we further designed an algorithm that computes (R(t)/sub /spl Delta/t/)~ for operating system scheduling applications. We proved that the real time execution complexity of the algorithm is O(1) and is, therefore, independent of the size of the distribution. Our method of more accurate estimate of (R(t)/sub /spl Delta/t/)~ implies expect better scheduling performance in applications where remaining execution time is used, especially in CPU scheduling.