Data Jockey: Automatic Data Management for HPC Multi-tiered Storage Systems

Modern multi-tiered data storage systems optimize file access by managing data across a hybrid composition of caches and storage tiers while using policies whose decisions can severely impact the storage system’s performance. Recently, different Machine-Learning (ML) algorithms have been used to model access patterns from complex workloads. Yet, current approaches train their models offline in a batch-based approach, even though storage systems are processing a stream of file requests with dynamic workloads. In this manuscript, we advocate the streaming ML paradigm for modeling access patterns in multi-tiered storage systems as it introduces various advantages, including high efficiency, high accuracy, and high adaptability. Moreover, representative file access patterns, including temporal, spatial, length, and frequency patterns, are identified for individual files, directories, and file formats, and used as features. Streaming ML models are developed, trained, and tested on different file system traces for making two types of predictions: the next offset to be read in a file and the future file hotness. An extensive evaluation is performed with production traces provided by Huawei Technologies, showing that the models are practical, with low memory consumption (<1.3 MB) and low training delay (<1.8 ms per training instance), and can make accurate predictions online (0.98 F1 score and 0.07 MAE on average).

Many science facilities, such as the Department of Energy's Leadership Computing Facilities and experimental facilities including the Spallation Neutron Source, Stanford Linear Accelerator Center, and Advanced Photon Source, produce massive amounts of experimental and simulation data. These data are often shared among the facilities and with collaborating institutions. Moving large datasets over the wide-area network (WAN) is a major problem inhibiting collaboration. Next-generation, terabit-networks will help alleviate the problem, however, the parallel storage systems on the endsystem hosts at these institutions can become a bottleneck for terabit data movement. The parallel storage system (PFS) is shared by simulation systems, experimental systems, analysis and visualization clusters, in addition to wide-area data movers. These competing uses often induce temporary, but significant, I/O load imbalances on the storage system, which impact the performance of all the users. The problem is a serious concern because some resources are more expensive (e.g. super computers) or have time-critical deadlines (e.g. experimental data from a light source), but parallel file systems handle all requests fairly even if some storage servers are under heavy load. This paper investigates the problem of competing workloads accessing the parallel file system and how the performance of wide-area data movement can be improved in these environments. First, we study the I/O load imbalance problems using actual I/O performance data collected from the Spider storage system at the Oak Ridge Leadership Computing Facility. Second, we present I/O optimization solutions with layout-awareness on end-system hosts for bulk data movement. With our evaluation, we show that our I/O optimization techniques can avoid the I/O congested disk groups, improving storage I/O times on parallel storage systems for terabit data movement.

Multi-tiered storage systems today are integrating Solid State Disks (SSD) on top of traditional rotational hard disks for performance enhancement due to the significant IO improvements in SSD technology. It is widely recognized that automated data migration between SSD and HDD plays a critical role in effective integration of SSD into multi-tiered storage systems. Furthermore, effective data migration has to take into account of application specific workload characteristics, deadlines, and IO profiles. An important and interesting challenge for automated data migration in multi-tiered storage systems is how to fully release the power of data migration while guaranteeing the migration deadline is critical to maximizing the performance of SSD-enabled multi-tiered storage system. In this paper, we present an adaptive look ahead data migration model that can incorporate application specific characteristics and I/O profiles as well as workload deadlines. Our adaptive data migration model has three unique features. First, it incorporates a set of key factors that may impact on the performance of look ahead migration efficiency in our formal model develop. Second, our data migration model can adaptively determine the optimal look ahead window size, based on several parameters, to optimize the effectiveness of look ahead migration. Third, we formally and experimentally show that the adaptive data migration model can improve overall system performance and resource utilization while meeting workload deadlines. Through our trace driven experimental study, we compare the adaptive look ahead migration approach with the basic migration model and show that the adaptive migration model is effective and efficient for continuously improving and tuning of the performance and scalability of multi-tier storage systems.

Recently, the multi-tiered storage systems that are composed of hard disk drives (HDDs) and solid state disk drives (SSDs) have received significant attention. The multi-tiered storage system extends the storage hierarchy by using SSDs to cache data from the HDDs. Thus, how to cache the appropriate data on the SSDs becomes an important issue for the multi-tiered storage systems. Due to the tremendous superior access performance of SSDs, current data caching schemes tried to cache the hottest data in SSDs, expecting that all of the disk I/O requests can be served by the SSDs. However, when the disk I/O load is heavy, such a scheme would saturate the bandwidths of SSDs and cause a negative impact on the I/O performance. To address this issue, on the basis of Profit Caching, this paper proposes a new data caching scheme, called Load-Balancing Caching Scheme (LBSC), for multi-tiered storage systems. LBSC takes the instant load of different storage devices for making the caching decision. Specifically, LBSC balance the load of SSDs and HDDs so as to fully exploit both the potential bandwidths of the SSDs and HDDs. We have implemented our scheme in a DiskSim-based multi-tiered storage simulator. From the experimental results, our proposed LBSC data caching scheme can significantly enhance the I/O performance of Profit Caching.

Scientific workflows are one of the well-established pillars of modern large-scale computational science. More recently, scientists have started to leverage machine learning (ML) capabilities in their workflows, leading to a new category of scientific workflows, denoted as scientific ML workflows. ML is not only about training and inference, modern ML workflows also involve complex data processing steps before the training can start, which are not often accounted for in most performance studies. In this work, we consider scientific ML workflows, from data pre-processing to training, inference, and model evaluation. We aim to explore (i) how scientific ML workflows differ from more traditional scientific workflows and; (ii) how we can characterize ML workflows both in terms of execution time and data movements when executing on a target cloud platform. We select three representative workflows, ranging from image classification to natural language processing and image segmentation, which have been executed using the academic cloud platform, Chameleon. We build four realistic deployment scenarios for each workflow, which stress data movements during workflow executions. Then, we compare the performance observed when utilizing these different configurations and study how different settings impact overall workflows performance and efficiency when running on cloud infrastructures. Finally, we summarize our findings and discuss performance impacts when augmenting scientific workflows with ML techniques and how traditional workflow management systems can improve their support for such workflows.

Data-intensive scientific applications are posing many challenges in distributed computing systems. In the scientific field, the application data are expected to double every year over the next decade and further. With this continuing data explosion, high performance computing systems are needed to store and process data efficiently, and workflow technologies are facilitated to automate these scientific applications. Scientific workflows are typically very complex. They usually have a large number of tasks and need a long time for execution. Running scientific workflow applications usually need not only high performance computing resources but also massive storage. The emergence of cloud computing technologies offers a new way to develop scientific workflow systems. Scientists can upload their data and launch their applications on the scientific cloud workflow systems from everywhere in the world via the Internet, and they only need to pay for the resources that they use for their applications. As all the data are managed in the cloud, it is easy to share data among scientists. This kind of model is very convenient for users, but remains a big challenge to the system. This paper proposes several research topics of data management in scientific cloud workflow systems, and discusses their research methodologies and state-of-the-art solutions.

Advancements in phenotyping technology have enabled plant science researchers to gather large volumes of information from their experiments, especially those that evaluate multiple genotypes. To fully leverage these complex and often heterogeneous data sets (i.e. those that differ in format and structure), scientists must invest considerable time in data processing, and data management has emerged as a considerable barrier for downstream application. Here, we propose a pipeline to enhance data collection, processing, and management from plant science studies comprising of two newly developed open-source programs. The first, called AgTC, is a series of programming functions that generates comma-separated values file templates to collect data in a standard format using either a lab-based computer or a mobile device. The second series of functions, AgETL, executes steps for an Extract-Transform-Load (ETL) data integration process where data are extracted from heterogeneously formatted files, transformed to meet standard criteria, and loaded into a database. There, data are stored and can be accessed for data analysis-related processes, including dynamic data visualization through web-based tools. Both AgTC and AgETL are flexible for application across plant science experiments without programming knowledge on the part of the domain scientist, and their functions are executed on Jupyter Notebook, a browser-based interactive development environment. Additionally, all parameters are easily customized from central configuration files written in the human-readable YAML format. Using three experiments from research laboratories in university and non-government organization (NGO) settings as test cases, we demonstrate the utility of AgTC and AgETL to streamline critical steps from data collection to analysis in the plant sciences.

Scientific workflow systems have been introduced in response to the demand of researchers from several domains of science who need to process and analyze increasingly larger datasets. The design of these systems is largely based on the observation that data analysis applications can be composed as pipelines or networks of computations on data. In this work, we present a runtime support system that is designed to facilitate this type of computation in distributed computing environments. Our system is optimized for data-intensive workflows, in which efficient management and retrieval of data, coordination of data processing and data movement, and check-pointing of intermediate results are critical and challenging issues. Experimental evaluation of our system shows that linear speedups can be achieved for sophisticated applications, which are implemented as a network of multiple data processing components.

Modern supercomputing systems are increasingly reliant on hierarchical, multi-tiered file and storage system architectures due to cost-performance-capacity trade-offs. Within such multi-tiered systems, data management services are required to maintain healthy utilization, performance, and capacity levels. We present PoliMOR, a pragmatic and reliable policy-driven data management framework. PoliMOR is composed of modular, single-purpose agents that gather file system metadata and enforce policies on storage systems. PoliMOR facilitates automated and scalable data management with customizable agents tailored to HPC facility-specific storage systems and policies. Our evaluations demonstrate the scalability and performance of PoliMOR both by its individual agents and as a collective entity. We believe PoliMOR is widely applicable across HPC facilities with large-scale data management challenges and will garner interest from the HPC community, given its flexible and open-source nature.

Следующий запуск БАК подразумевает использование намного больших ресурсов, чем может предоставить ГРИД. Для решения данной проблемы, ALICE занимается проектом по расширению существующей вычислительной модели для того, чтобы включить в нее дополнительные ресурсы, например, суперкомпьютер Titan. В данной статье описана технология взаимодействия вычислительной среды AliEn и суперкомпьютера Titan, находящегося в Oak Ridge Leadership Computing Facility (OLCF). Эта технология использует PanDA (Production and Distributed Analysis System) WMS (Workload management system) для отправки задач в очередь пакетной обработки информации Titan и локального управления данными. Благодаря PanDA и Titan, эксперимент ALICE на Большом адронном коллайдере получает новые ресурсы для выполнения поставленных задач. Данная реализация была протестирована с применением задач ALICE. AliEn (ALIce ENvironment) – распределенная вычислительная среда, разработанная для проекта ALICE Offline. Она позволяет получить доступ к распределенным вычислительным ресурсам и ресурсам хранения всем участникам эксперимента ALICE на Большом Адронном Коллайдере (БАК). В настоящее время AliEn позволяет обрабатывать задачи примерно на 100 000 вычислительных процессорах, использующихся в более чем 80 ГРИД сайтах по всему миру. Архитектура вычислительной среды на 99% состоит из импортированных компонентов с открытым кодом, что позволяет использовать функциональные возможности без их изменения. Для связи AliEn и ГРИД – инфраструктурой используется сервис VOBOX, позволяющий запускать собственные сервисы на вычислительных сайтах, а также обеспечивающий прямое взаимодействие с очередью пакетной обработки для запуска задач. The next launch of the LHC involves using much more resources than GRID can provide. To solve this problem, ALICE is engaged in a project to expand the existing computing model in order to include additional resources in it, for example, Titan supercomputer. This article explores the interaction technology of the AliEn computing environment and the Titan supercomputer located in the Oak Ridge Leadership Computing Facility (OLCF). This technology uses the PanDA (Production and Distributed Analysis System) WMS (Workload management system) to send jobs to the Titan batch processing queue and local data management. Through PanDA and Titan, the ALICE experiment at the Large Hadron Collider receive new resources to fulfill the assigned tasks. This implementation was tested using ALICE jobs. AliEn (ALIce ENvironment) is a distributed computing environment developed for the ALICE Offline project. It allows access to distributed computing resources and storage resources for all participants of the ALICE experiment at the Large Hadron Collider (LHC). Currently, AliEn allows processing jobs on approximately 100,000 computing processors used in more than 80 GRID sites around the world. The architecture of the computing environment is 99% composed of imported components with open source, which makes possible to use the functionality without changing them. To connect AliEn and Grid infrastructure, used VOBOX service, which allows you to run your own services on computer sites, and also provides direct interaction with the batch processing queue for launching jobs.

Deep learning enables numerous applications across diverse areas. Data systems researchers are also increasingly experimenting with deep learning to enhance data systems performance. We present a tutorial on deep learning, highlighting the data systems nature of neural networks as well as research opportunities for advancements through data management techniques. We focus on three critical aspects: (1) classic design tradeoffs in neural networks which we can enrich through a systems and data management perspective, e.g., thinking critically about storage, data movement, and computation; (2) classic design problems in data systems which we can reconsider with neural networks as a viable design option, e.g., to replace or help system components that make complex decisions such as database optimizers; and (3) essential considerations for responsible application of neural networks in critical human-facing problems in society and how these also link to data management and performance considerations. While these are seemingly a diverse set of rich topics, they are strongly interconnected through data management, and their combination offers rich opportunities for future research.

The healthcare industry has been using data and technology driven innovation for a long time now with Evidence Based Medicine (EBM) being at the helm. Effectively using technology is vital towards successful data management in healthcare. A convenient approach that can collect, store, evaluate, and analyze health data will be beneficial to all the stakeholders in the healthcare delivery chain. With the advancement, clubbed with the availability and affordability of technology, it has become an effective tool for providing the right- care backed with accurate, consolidated evidence. Such approach not only helps in improving the quality of care, it also helps in early detection of diseases and effective treatment, Big Data is providing the necessary data management tools for the same. A simple Extract-Transform-Load (ETL) procedure using Hadoop can be used. As Big data analyses large amounts of data to uncover hidden patterns, correlations and other insights, its role in making the right, predictive decisions for quality care of the patients and better health planning is possible. Big data technologies such as Hadoop bring significant cost advantages when it comes to storing large amounts of data and combined with the speed of Hadoop and in-memory analytics, with the ability to analyze new sources of data, we are able to analyze information instantly – and make decisions which is very essential in the healthcare provider segment. Hadoop does not only store, but also process any and many file data: big or small, can be plain text files or binary files like images and even multiple different version of some particular data format across different time periods. It consists of three core components: a distributed file system, a parallel programming framework, and a resource/job management system. In healthcare, patient records, health plans, insurance information and other types of information is extremely difficult to manage – but, this very data is full of key insights, the need is for an effective data management and analytics that can be pragmatic. By analyzing large amounts of information – both structured and unstructured – rapidly, health care providers can provide lifesaving diagnoses. Keywords: Big Data, EBM, ETL, Hadoop

Modern biomedical research aims at drawing biological conclusions from large, highly complex biological datasets. It has become common practice to make extensive use of high-throughput technologies that produce big amounts of heterogeneous data. In addition to the ever-improving accuracy, methods are getting faster and cheaper, resulting in a steadily increasing need for scalable data management and easily accessible means of analysis.We present qPortal, a platform providing users with an intuitive way to manage and analyze quantitative biological data. The backend leverages a variety of concepts and technologies, such as relational databases, data stores, data models and means of data transfer, as well as front-end solutions to give users access to data management and easy-to-use analysis options. Users are empowered to conduct their experiments from the experimental design to the visualization of their results through the platform. Here, we illustrate the feature-rich portal by simulating a biomedical study based on publically available data. We demonstrate the software’s strength in supporting the entire project life cycle. The software supports the project design and registration, empowers users to do all-digital project management and finally provides means to perform analysis. We compare our approach to Galaxy, one of the most widely used scientific workflow and analysis platforms in computational biology. Application of both systems to a small case study shows the differences between a data-driven approach (qPortal) and a workflow-driven approach (Galaxy).qPortal, a one-stop-shop solution for biomedical projects offers up-to-date analysis pipelines, quality control workflows, and visualization tools. Through intensive user interactions, appropriate data models have been developed. These models build the foundation of our biological data management system and provide possibilities to annotate data, query metadata for statistics and future re-analysis on high-performance computing systems via coupling of workflow management systems. Integration of project and data management as well as workflow resources in one place present clear advantages over existing solutions.

Most of the domains of science are facing an explosion in the data that they have to collect and process in order to conduct research. This is true for both: scientific domains dealing with experimental data, e.g. biology, sociology, astronomy etc, but also scientific domains dealing with simulation data, e.g. seismology, physics etc. To maximize the potential outcome of scientific data analysis, respective data management applications need to fulfil the following coarse tasks: fast on-demand data processing, and effective storage and consolidation of diverse data collections. These two tasks are in general hard to realize because of: (a) the big data size, (b) the diversity of data formats, (c) their conceptual dependencies, (d) disperse data locations, and (e) intensive and systematic nature of scientific queries. We present the characteristics of big scientific data collections and their necessities in terms of data management. Based on this discussion, we discuss the structure of a framework for the processing and consolidation of heterogeneous scientific data collections. Such a framework aims to mediate between the user and a set of available data management technologies, such as relational DBMSs, key-value stores and column stores, in order to efficiently direct data management operations (insertions, updates) and especially requests (queries) to the appropriate data management application. The framework aims to distribute, dissect, and schedule data management actions, as well as integrate results, in a way that reduces response time. This entails the accommodation of methods for the selective parallelism and serialization depending on partial results and response times. Also, this entails the accommodation of methods for the gradual alteration of data formats and storage, e.g. storage of semi-structured data or raw data in files into relational databases. Furthermore, we discuss the processing of scientific query bulks or workflows with the possibility to retrieve early partial results and calibrate query parameters.

Fog computing, a novel computing paradigm, migrates cloud computing to the edge of network and consequently decrease the overhead of process and movement of the large-scale big data. Fog computing is promising to lower communication delay and offload network traffic with the scale of big data increasing in large-scale IoT application, which brings micro-data centers to the network edge, enabling the network edge to perceive, process, storage and calculate the massive data, reducing the amount of transferred data. However, numerous fog nodes at the network edge face an external attack resulting in serious security threats arising from the trusting relationship vulnerability because of the lack of the consideration of the properties of the fog computing system. This paper addresses the design for developing secure distributed data management platform for fog computing in large-scale IoT application, along with a blockchain-based data management implementation of the platform, which approaches the major challenges: how to realize the integration of data security and storage management for fog computing in large-scale IoT application and enrich rational interoperability for interconnected things. Experimental results show that the system works well to empower data provenance and transparency and defend against unauthorized access effectively in fog computing, scales well with loss of the performance of communication and computing maintaining in acceptable range, verifying the efficiency and feasibility of our design to provide privacy, integrity and fine-grained data management for fog computing in large-scale IoT application by introducing the data management platform based upon the blockchain network and smart contracts therein.