Transferability of data and programs between computer systems

Many scientific applications need to stage large volumes of files from one set of machines to another set of machines in a wide-area network. Efficient execution of such data transfers needs to take into account the heterogeneous nature of the environment and dynamic availability of shared resources. This paper proposes an algorithm that dynamically schedules a batch of data transfer requests with the goal of minimizing the overall transfer time. The proposed algorithm performs simultaneous transfer of chunks of files from multiple file replicas, if the replicas exist. Adaptive replica selection is employed to transfer different chunks of the same file by taking dynamically changing network bandwidths into account. We utilize GridFTP as the underlying mechanism for data transfers. The algorithm makes use of information from past GridFTP transfers to estimate network bandwidths and resource availability. The efficiency of the algorithm is evaluated on a wide-area testbed.

Effective data management across large-scale shared-use materials-science facility centers is crucial, as high-end facilities available at such centers are indispensable for cutting-edge research in modern materials science. However, establishing network connectivity for these facilities is often not practically feasible due to its uniqueness, forcing users to resort to the traditional manual method of data acquisition and transfer, such as the use of DVD disks. In this paper, we introduce an IoT device designed to facilitate direct data transfer from these non-networked and shared-use facilities to a cloud system. We propose a novel data transfer algorithm that integrates user management with data transfer processes for safely assigning each user's data to its private storage space on the cloud. The proposed technology serves as a fundamental building block in shaping a smarter laboratory information management system for the advancement of material researches and developments.

This research work presents the development of a prototype short distance, full duplex data transfer system using red pointer module as the light transmitter and PIN photodiode as the receiver. The system is low cost, stable, and has low power consumption.) This data transfer unit is connected to a computer system using the Prolific USB-to-Serial Com Port chipset which also uses the computer as its power source. The unit is used for data transfers between two nearby computer systems or peripheral devices. In this work, a data transfer program is developed. The program was written in the visual C# language. It is convenient to specify the COM port and the appropriate bit rates. Data transfers of both text and graphical data are tested and results are discussed; both with and without moving object interferences. This data transfer unit has the potential for applications in EMI-sensitive environments and moving objects interferences.

We describe a new method to accelerate neighbor searches on GRAPE, i.e. a special-purpose hardware that efficiently calculates the gravitational forces and potentials in $N$-body simulations. In addition to gravitational calculations, GRAPE simultaneously constructs lists of neighbor particles that are necessary for Smoothed Particle Hydrodynamics (SPH). However, the data transfer of the neighbor lists from GRAPE to the host computer is time-consuming, and can be a bottleneck. In fact, the data transfer can take about the same time as the calculations of the force themselves. Making use of GRAPE’s special treatment of neighbor lists, we can reduce the amount of data transfer if we search neighbors in the order that the neighbor lists, constructed in a single GRAPE run, overlap each other. We find that the Morton ordering requires very low additional calculation and programming costs, and results in successful speed-up on data transfer. We show some benchmark results in the case of GRAPE-5. Typical reduction in transferred data becomes as much as 90%. This method is suitable not only for GRAPE-5, but also for GRAPE-3 and the other versions of GRAPE.

We target the problem of managing the power states of the servers in a Cloud Data Center (CDC) to jointly minimize the electricity consumption and the maintenance costs derived from the variation of power (and consequently of temperature) on the servers’ CPU. More in detail, we consider a set of virtual machines (VMs) and their requirements in terms of CPU and memory across a set of Time Slot (TSs). We then model the consumed electricity by taking into account the VMs processing costs on the servers, the costs for transferring data between the VMs, and the costs for migrating the VMs across the servers. In addition, we employ a material-based fatigue model to compute the maintenance costs needed to repair the CPU, as a consequence of the variation over time of the server power states. After detailing the problem formulation, we design an original algorithm, called Maintenance and Electricity Costs Data Center (MECDC), to solve it. Our results, obtained over several scenarios from a real CDC, show that MECDC largely outperforms two reference algorithms, which instead either target the load balancing or the energy consumption of the servers.

Case study describing the pillars, personnel, and process of developing the TARDIS:REFLUX smartphone app

Scientific workflows are increasingly transferring large amounts of data between high performance computing (HPC) systems. Even though these HPC systems are connected via high-speed dedicated networks and use dedicated data transfer nodes (DTNs), it is still difficult to predict the data transfer throughput because of variations in data transfer protocols, host configurations, performance of file systems, and overlapping workloads. In order to provide reliable performance prediction for better resource management and job scheduling, we need models for predicting data transfer throughput under real-world conditions. In this paper, we explore different machine learning approaches for building data-driven models to improve performance and prediction of large-scale data transfer throughput. In addition to the variables already collected by the network monitoring system, we also develop heuristics to derive additional metrics for improving the prediction accuracy. We use the prediction results to identify the importance of different network parameters in predicting the throughput for large-scale data transfers. Through extensive tests, we identify key network parameters, discover interesting variations among different HPC sites, and show that we can predict throughput with high accuracy. We also analyze our models and results to provide recommendations for improving the performance of big data transfers.

As a dynamic memory virtualization technique, ballooning is widely applied in many virtualization platforms, i.e. Xen and VMware ESX Server. Since ballooning technology enables the guest OS to surrender unused memory back to the host during runtime, and it can increase utilization and flexibility of memory. Despite the rapid development and extensive use of memory virtualization technologies, it is still a challenge to analyze and verify its validity and some major properties through formal methods. In this paper, we model the ballooning under Xen architecture including Xen hyper visor, a set of virtual machines and balloon drivers using the process algebra Communication Sequential Process (CSP) in order to verify some resource control properties of ballooning. The model is implemented with Process Analysis Toolkit (PAT). As a result, the ballooning used in Xen architecture satisfies these major resource control properties in VM requirement document.

Designing a cloud infrastructure for HPC applications requires to correlate design choices for virtualization solutions, resources management strategies, and their implementation on specific hardware platforms. Such investigations can hardly be conducted without accurate simulation tools. This paper introduces VMcSim, the first micro architecture and virtualized many core simulation framework. VMcSim models a x86-based asymmetric many core micro architecture, including a virtualization layer which allows to model a hyper visor, a set of virtual machines with their virtual resources (CPU and memory), their operating system, and their applications. By simulating all the involved hardware and software layers, VMcSim outperforms other simulators. Simulation accuracy is evaluated through several benchmark (SPLASH-2). The simulator is demonstrated with the evaluation of virtual resources placement policies in multicore systems.

Possibilities and way to design and implement a ZigBee-based data transmission and monitoring wireless smart sensor network integrated with the Internet are considered and described in the paper. Implementing of a cost-effective such system requires the use and integration of different hardware elements and programming languages. Possibilities and key aspects of smart sensor nodes are considered and described in the paper. Integration of wireless smart sensor network and their sensor nodes with the Internet is described in more details. Design and implementation of an inexpensive energy saving ZigBee point-to-point wireless smart sensor network to monitor sensor data in energy efficient manner using XBee modules and different boards are proposed and described. Configuration of used XBee modules for smart sensor nodes communication is also described. Practical implementation of a battery powered smart sensor nodes and coordinator node is described in details. Data measurement with such sensor nodes is also described. Results of wireless data measurement, data transfer and monitoring with such ZigBee based wireless smart sensor network via Web technologies are shown in the paper.

: Information technology -- computers, networks and their associated software and communications -- has been evolving at an astounding rate. Viruses and other computer pathogens, which have existed only for a few years, have been evolving at an equally high rate. We must now acknowledge the potential for more aggressive virus attacks which can wreak significant strategic damage. Military and civilian administrative and logistic computer and communications systems represent the most lucrative potential targets. Tactical systems generally are closed; their operating systems have not yet been attacked, since they communicate and transfer data over non-public, tactical and strategic communications systems. However, the potential for attack by a determined person or persons exists. This threat will increase as we become more dependent on these strategic systems. The time has come to anticipate how, by whom, and under what conditions an attack will occur. Computer pathogens have matured to a state where no computer system is completely safe, unless it is stand-alone. Computer viruses avoid detection by constantly changing their identity; network worms can gain access to most networks; and Trojan Horses can do both without the user's knowledge. Once the pathogens are present and a triggering event occurs, the potential damage they may do is incalculable. If DOD intends to rely on its strategic systems in the future, it must plan for and deal with viruses and other computer pathogens now.

Compute-intensive tasks in high-end high performance computing (HPC) systems often generate large amounts of data, especially floating-point data, that need to be transmitted over the network. Although computation speeds are very high, the overall performance of these applications is affected by the data transfer overhead. Moreover, as data sets are growing in size rapidly, bandwidth limitations pose a serious bottleneck in several scientific applications. Fast floating point compression can ameliorate the bandwidth limitations. If data is compressed well, then the amount of data transfer is reduced. This reduction in data transfer time comes at the expense of the increased computation required by compression and decompression. It is important for compression and decompression rates to be greater than the network bandwidth; otherwise, it will be faster to transmit uncompressed data directly. Accelerators such as Graphics Processing Units (GPU) provide much computational power. In this paper, we show that the computational power of GPUs can be harnessed to provide sufficiently fast compression and decompression for this approach to be effective for data produced by many practical applications. In particularly, we use Holt's Exponential smoothing algorithm from time series analysis, and encode the difference between its predictions and the actual data. This yields a lossless compression scheme. We show that it can be implemented efficiently on GPUs to provide an effective compression scheme for the purpose of saving on data transfer overheads. The primary contribution of this work lies in demonstrating the potential of floating point compression in reducing the I/O bandwidth bottleneck on modern hardware for important classes of scientific applications.

In a computer network, the transport layer uses the service offered by the network layer and in turn offers its users the transport service of reliable connection management and data transfer. We provide a formal specification of the transport service in terms of an event-driven system and safety and progress properties. We construct three verified transport protocols that offer the transport service. The first transport protocol assumes a perfect network service, the second assumes loss-only network service, and the third assumes loss, reordering and duplication network service. Our transport service specifications are very realistic. Each user can be closed, listening, active opening, passive opening, open, or closing. A local incarnation number uniquely identifies every active opening and listening duration. Users can issue requests for connection, listening, closing, data send, etc. The transport layer issues indications for successful or unsuccessful connection, closing, data reception, etc. A connection is established only if one user requested the connection and the other was listening, or both requested the connection. A user receives data only from the appropriate incarnation of the distant user, and receives it insequence, without loss or duplication. Progress properties ensure that every outstanding user request is eventually responded to by an appropriate transport indication. Our protocols are constructed by stepwise refinement of the transport service. The construction method automatically generates a verification that the protocols satisfy the transport service. One distinctive feature of our protocol construction is that the events and verification of the data transfer function is directly obtained from any one of the numerous verified single-incarnation data transfer protocols already presented in the literature.

Current research activities are leveraging the Grid to create generic- and domain-specific solutions and services for data management and knowledge discovery. Knowledge and Data Management in Grids is the third volume of the CoreGRID series; it gathers contributions by researchers and scientists working on storage, data, and knowledge management in Grid and Peer-to-Peer systems. This volume presents the latest Grid solutions and research results in key areas such as distributed storage management, Grid databases, Semantic Grid and Grid-aware data mining. Written for a professional audience of researchers and practitioners in industry, it is suitable for graduate-level students in computer science.

When transmitting digital data, various types of errors may occur, including in the data address, and accordingly, there is a need to regulate the data transfer rate. In this case, simple synchronization technology is no longer enough, and it is necessary to use a data link control protocol that would provide data flow control, error detection and protection against the latter. For the effective use of high-speed communication lines, signal compression is used, which allows several transmission sources to use the high bandwidth of the channel. The paper analyzes the technology for controlling digital data transmission channels with compression in computer systems. Two forms of control are considered: frequency-division multiplexing (FDM) and time-division multiplexing (TDM). It is shown that sufficient data transfer speed and bandwidth, low price and the ability to use when operating from autonomous power sources make xDSL technology relevant, especially as a backup Internet access channel.