Realistic Workload Modeling and Its Performance Impacts in Large-Scale eScience Grids

Intuition suggests that a finite buffer limits the effect of traffic auto-correlations on the queue length. We investigate the extent to which finite buffers can, therefore, be expected to mitigate the effects of long-range dependence (LRD). With traffic sequences generated by the fractional autoregressive integrated moving average (f-ARIMA) models for LRD, and by AR models for short range dependence (SRD), we investigate the traffic performance for a range of finite buffers, both for single and multiplexed streams. For design, the aim is to 'match' a given LRD auto-correlation function with a suitable SRD function whose performance provides, for a wide range of traffic intensities and buffer sizes, a conservative bound on the performance associated with the LRD function. The results suggest that by suitably 'dominating' an LRD auto-correlation function by an SRD function, one can obtain conservative performance bounds for a realistic range of traffic intensities and buffer sizes (delays). Also, in several cases, a 'cross-over' phenomenon is observed for cell-loss probabilities as the buffer size increases, i.e., the loss probabilities are smaller for LRD than for SRD for small buffers, with the converse true for large buffers. This suggests that finite buffers, can in some cases, counteract the effects of LRD in traffic arrivals, and thus enable conservative designs to be based on Markovian traffic models.

Job arrivals can be described as point processes and it is shown that correlations and fractal behavior can be reliably revealed using the count/rate representation. Using real workload data from production Grids, we show that the second order properties such as the autocorrelation function (ACF) and the scaling behavior can be well reconstructed by a Multifractal Wavelet Model (MWM). A so-called controlled-variability integrate-and-fire (CV-InF) algorithm is applied to transform rates into interarrivals so that a full description of the arrival process can be obtained. The additive nature of rates makes it possible to model different patterns separately and aggregate them back to form a unified process. We further quantify the performance impacts of autocorrelated job arrivals in Grid scheduling using model-driven simulation. It is shown that autocorrelations in the arrival processes can cause performance degradation both at the local and the Grid level.

Network study finds simultaneous presentation of long-range dependence and short-range dependence in Network traffic., which makes the performance of the traditional model used to describe the short-range dependence for prediction lower. And in different scales of these two characteristics of network traffic have different impact for network performance. In small enough scale the short-range dependence has a greater impact on network performance, while in large enough scale, long-range dependence characteristics play a leading role. Therefore, in order to obtain better prediction this paper makes network traffic sequence more smooth and easy to be fitted by the traditional model, through weakening the dependence not affording a major role in small scale and enhancing the dependence playing a leading role in the impact of network.

The objective of this paper is to demonstrate the limitations of the variance-time (VT) test as a statistical tool for inferring long-range dependence (LRD) in network traffic. Since the early Bellcore studies, LRD has been in the center of a continuous debate within the teletraffic community. The controversy is typically focused on the utility of LRD models to predict the performance at network buffers. Our work here is not meant to advocate one modeling approach over another, but to point out (experimentally and theoretically) to the caveats in using the VT test as a tool for detecting LRD. We do that by deriving simple analytical expressions for the slope of the aggregated variance in three autocorrelated traffic models: M/G//spl infin/ process (short-range dependent (SRD) but non-Markovian), the discrete autoregressive of order one model (SRD Markovian), and the fractional ARIMA process (LRD). Our main result is that the VT test often indicates, falsely, the existence of an LRD structure (i.e., H>0.5) in synthetically generated traces from the two SRD models. The bias in the VT test, however, diminishes monotonically with the length of the trace. We provide some guidelines on selecting the minimum trace length so that the bias is negligible.

We investigate transport layer processing intended to improve the communication of multimedia data over wireless and wireline networks. Specifically, we consider data striping and thinning techniques, which are applicable to the multipath/multiflow transmission of multimedia (e.g., MPEG-4). We also introduce the shuffling procedure, which reorders the data at the network edges. We show that all three techniques break up short-term correlations in data traffic thus improving its queueing performance. We demonstrate that while both long range dependence (LRD) and short range dependence (SRD) influence the queueing performance for some timescale of the queueing system, it is the short-range statistical properties of multimedia traffic within the critical time scale that are dominant in determining the buffer efficiency of the queue. We also show that estimation of LRD for thinned and striped data may lead to a misleading notion of LRD reduction, when none is warranted. We further outline the ideas for a new transport layer protocol that explores the combination of thinning, striping and shuffling approaches to multimedia data transmission.

Time series modeling and analysis provides means of predicting the future and has been widely used in a variety of fields ranging from seismology for predicting earthquake and volcanic eruption, to finance for risk assessment, and to quantum information processing. The conventional integer order models can only capture short-range dependence; for example, Poisson processes, Markov processes, autoregressive (AR), moving average (MA), autoregressive moving average (ARMA) and autoregressive integrated moving average (ARIMA) processes. In time series analysis, one of the conventional assumptions is that the coupling between values at different time instants decreases rapidly as the time difference or distance increases. However, there are situations where strong coupling between values at different times exhibit properties of long range dependence which cannot be processed by the conventional time series analysis. Typical examples of long range dependence signals include financial time series, underwater noise, electroencephalography (EEG) signal, etc. ARFIMA, a fractional order signal processing technique, is the generalization of the conventional integer order techniques, namely, ARIMA and ARMA methods. Hence, it is capable of capturing both short-range dependence and long-range dependence in signals. Compared to conventional integer order models, the ARFIMA model gives a better fit and result when dealing with the data which possess the long range dependence property. In this paper, we investigate the application of the ARFIMA as well as AR methods to model EEG signals obtained from different brain channels. We analyze the resulting correlations for comparison the benefits of ARFIMA over AR on the EEG data exhibiting the long range dependency property. The results showed that the prediction results have a better performance compared to the conventional ARMA models.

Heavy traffic analysis for a multiplexer driven by M∨GI∨∞ input processes

Markov-modulated Poisson process (MMPP) is one of the popular non-self-similar schemes for modeling network traffic. They are used for modeling both short-range dependent (SRD) and long-range dependent (LRD) traffic. The paper verifies Algorithm LAMBDA for fitting MMPP to synthetic traffic and evaluates the accuracy of the fitted MMPP model. We provide evidence that, for self-similar traffic, the performance highly depends on Hurst parameter. In some cases, the algorithm exhibits some inaccuracy in mean queuing delays. We try to modify it by a simple change in order to make the algorithm adaptive, and consequently, make the results more realistic.

Network traffic exhibits fractal properties, such as self- similarity and impulsiveness, and posseses a complex correlation structure, i. e., both long and short-range dependence are present. Models such as Fractional Gaussian Noise can capture the long-range dependence but not the short-range dependence. Therefore, a key issue that must be adressed when using synthesized traffic for network simulations is the incorporation of short-range dependence into the generated trace. This work develops a filter-based method for the introduction of short- range dependence into long-range dependent sample paths that are synthesized via wavelets and studies a method of mapping from a given time series to a packet stream. The generator does not employ training sequences, i. e., it is not based on the use of real traffic traces as training sequences.

Accurate modeling of VoIP traffic QoS parameters in current and future networks with multifractal and Markov models

Long-range dependence has been recently asserted to be an important characteristic in modeling telecommunications traffic. Inspired by the integral relationship between the fractional Brownian motion and the standard Brownian motion, we model a process with long-range dependence, Y, as a fractional integral of Riemann-Liouville type applied to a more standard process X—one that does not have long-range dependence. When X takes the form of a sample path process with bounded stationary increments, we provide a criterion for X to satisfy a moderate deviations principle (MDP). Based on the MDP of X, we then establish the MDP for Y. Furthermore, we characterize, in terms of the MDP, the transient behavior of queues when fed with the long-range dependent input process Y. In particular, we identify the most likely path that leads to a large queue, and demonstrate that unlike the case where the input has short-range dependence, the path here is nonlinear.

Accurate evaluation of the long-range dependence in hydroclimatic time series is important for understanding its inherent characteristics. However, the reliability of its evaluation may be questioned, since different methods may yield various outcomes. In this study, we evaluate the performances of seven widely used methods for estimating long-range dependence: absolute moment estimation, difference variance estimation, residuals variance estimation, rescaled range estimation, periodogram estimation, wavelet estimation (WLE), and discrete second derivative estimation (DSDE). We examine the influences of six major factors: data length, mean value, three nonstationary components (trend, jump, and periodicity), and one stationary component (short-range dependence). Results from the Monte Carlo experiments show that WLE and DSDE have greater credibility than the other five methods. They also reveal that data length, as well as stationary and nonstationary components, have notable influences on the evaluation of long-range dependence. Following it, we use the WLE and DSDE methods to evaluate the long-range dependence of precipitation during 1961–2015 on the Tibetan Plateau. The results indicate that the precipitation variability mirrors the long-range dependence of the Indian summer monsoon but with obvious spatial difference. This result is consistent with the observations made by previous studies, further confirming the superiority of the WLE and DSDE methods. The outcomes from this study have important implications for modeling and prediction of hydroclimatic time series.

Simulation studies of grid scheduling strategies require representative workloads to produce dependable results. Real production grid workloads have shown diverse correlation structures and scaling behavior, which are different than the characteristics of the available supercomputer workloads and cannot be captured by Poisson or simple distribution-based models. We present models that are able to reproduce various correlation structures, including pseudo-periodicity and long range dependence. By conducting model-driven simulation, we quantitatively evaluate the performance impacts of workload correlations in grid scheduling. The results indicate that autocorrelations in workloads result in worse system performance, both at the local and the grid level. It is shown that realistic workload modeling is not only possible, but also necessary to enable dependable grid scheduling studies.

This paper presents a new MPEG-4 video traffic model, which efficiently exploits the frame pattern of MPEG-4 video by using IBB and PBB sequences as the basic units and modeling their dependence in the orthogonal domain after Hadamard transform. This model can accurately capture both short range dependence (SRD) and long range dependence (LRD) characteristics of MPEG-4 video data. Simulation results demonstrate that the probability density function and the autocorrelation function of model data are highly close to those of real data. And in the leaky bucket performance test, using the model data as input source obtains almost the same estimates of the required buffer size or bit loss rate as using real data. Furthermore, our new model needs very few parameters and has high computational efficiency, thus is very suitable to be used as the video source in the study such as cellular network design and evaluation, wireless resource management and so on.

The assessment of collective risk for flood risk management requires a better understanding of the space-time characteristics of flood magnitude and occurrence. In particular, classic formulation of collective risk implies hypotheses concerning the independence of intensity and number of events over fixed time windows that are unlikely to be tenable in real-world hydroclimatic processes exhibiting persistence. In this study, we investigate the links between the serial correlation properties of 473 daily stream flow time series across the major river basins in Europe, and the characteristics of over-threshold events which are used as proxies for the estimation of collective risk. The aim is to understand if some key features of the daily stream flow data can be used to infer properties of extreme events making a more efficient and effective use of the available data. Using benchmark theoretical processes such as Hurst-Kolmogorov (HK), generalized HK (gHK), autoregressive fractionally integrated moving average (ARFIMA) models, and Fourier surrogate data preserving second order linear moments, our findings confirm and expand some results previously reported in the literature, namely: (1) the interplay between short range dependence (SRD) and long range dependence (LRD) can explain the majority of the serial dependence structure of deseasonalized data, but losing information on nonlinear dynamics; (2) the standardized return intervals between over-threshold values exhibit a sub-exponential Weibull-like distribution, implying a higher frequency of return intervals longer than expected under independence, and expected return intervals depending on the previous return intervals; this results in a tendency to observe short (long) inter-arrival times after short (long) inter-arrival times; (3) as the average intensity and the number of events over one-year time windows are not independent, years with larger events are also the more active in terms of number of events; and (4) persistence influences the distribution of the collective risk producing a spike of probability at zero, which describes the probability of years with no events, and a heavier upper tail, suggesting a probability of more extreme annual losses higher than expected under independence. These results provide new insights into the clustering of stream flow extremes, paving the way for more reliable simulation procedures of flood event sets to be used in flood risk management strategies.