Research on Scientific Research Data Platform Based on Big Data

Fault tolerant scheduling of hard real-time tasks on multiprocessor system using a hybrid genetic algorithm

One notable advantage of Model-Driven Architecture (MDA) method is that software developers could do sufficient analysis and tests on software models in the design phase, which helps construct high confidence on the expected software behaviors and performance, especially for safety-critical real-time software. Most existing literature of reliability analysis ignores the effects from those deadline requirements of tasks which are critical properties for real-time software and thus cannot be ignored. Considering the contradictory relationship between the deadline requirements and time costs of fault tolerance in real-time tasks, in this paper, we present a novel reliability model, which takes schedulability as one of the major factors affecting the reliability, to analyze reliability of the task execution model in real-time software design phase. The tasks in this reliability model has no restrictions on their distributions and thus could be distributed on a multiprocessor or on a distributed system. Furthermore, the tasks also define arrival rates of faults and fault-tolerant mechanisms to model the occurrences of non-permanent faults and the corresponding time costs of fault handling. By analyzing the probability of tasks still being schedulable in the worst-case execution scenario with faults occurring, reliability and schedulability are combined into an unified analysis framework, and two algorithms for reliability analysis are given. To make this reliability model more pragmatic, we also present an estimation technique for estimating the fault arrival rate of each task. We show through two case studies respectively the detailed derivation process under static-priority scheduling in a multiprocessor system and in the design process of avionics software, and then analyze the factors affecting the reliability analysis by setting up simulation experiments. When no assumptions of fault occurrences made on the task model, this reliability model regresses to a generic schedulability model.

Fault-tolerant scheduling is an efficient approach to improving the reliability of multiple earth-observing satellites especially in some emergent scenarios such as obtaining photographs on battlefields or earthquake areas. Unfortunately, little work has been done to deal with the fault-tolerant scheduling on satellites. To address this issue, this paper presents a novel dynamic fault-tolerant scheduling model using primary-backup policy to tolerate one satellite's permanent failure at one time instant. On this basis, we propose a novel fault-tolerant satellite scheduling algorithm named FTSS, in which an overlapping technology is adopted to improve the resource utilization. Besides, the FTSS employs the task merging strategies to further enhance the schedulability. To demonstrate the superiority of our FTSS, we conduct extensive experiments by simulations using real-world satellite parameters from STK to compare FTSS with other baseline algorithms. The experimental results indicate that FTSS efficiently improves the scheduling quality of others and is suitable for fault-tolerant satellite scheduling.

In this paper, we consider energy-efficient and fault-tolerant scheduling of real-time tasks on heterogeneous multicore systems. Each task consists of a main copy and a backup copy which are scheduled on different cores, for fault tolerance purposes. Our framework deliberately delays the backup tasks in order to cancel them dynamically when the main task copies complete successfully (without faults). We identify and address two dimensions of the problem, i.e., partitioning tasks and determining processor voltage/frequency levels to minimize energy consumption. Our experimental results show that our proposed algorithms' performance levels are close to that of an ideal solution with optimal (but computationally prohibitive) partitioning and frequency assignment components.

Scientific research processes are accompanied by burgeoning amounts of research data that serves as findings of the research and provides a platform for the results to be verifiable. Scientific researchers are under pressure to make their data openly available to other researchers for use. Access to scientific data and its curation has become a dominant theme in scientific policy discussions. Besides the funder requirements, researchers are making their data openly available for a number of reasons that include, raising the profiles of their research outputs, to increase their citations, to validate their research outputs and for their authenticity and integrity as researchers. However, not all researchers possess the skills to curate their own data and make it openly available for use in an open environment that is FAIR (Findable, Accessible, Interoperable and Reusable). The primary objective of this paper is to raise awareness of the strategic role of data scientists in the scientific research process. The study seeks to contribute towards a growing and emergent discourse of data science in HEIs. It also seeks to contribute towards an educational model for data scientists in HEIs in South Africa. In-order to realise these objectives the paper asks the following research questions:How are the emergent roles of data scientists conceptualised in HEIs in South Africa?What is the strategic role of a data scientist in the scientific research process?.This study analysed data science programs offered by universities in South Africa to understand how the role of data scientists is conceptualised and to delineate the scope of this emergent field. A qualitative content analysis survey method is used to analyse courses offered by HEIs in South Africa. Preliminary findings from the research revealed that data scientists are drawn from fields like mathematics, statistics, information science, and computer science. The practical jobs of data scientists as conceptualised by the majority of universities employs theories drawn from the information science, computer science and statistics. The data scientists are expected to work closely with the researchers where the research is done. Most universities are offering data science programs at post-graduate level as professional courses for learners coming in from diverse backgrounds. The study recommends that the next generation of data scientists in a South African context should possess a cocktail of skills that enable them to conduct data analysis and enquiry to enable others to work with digital data. Presentation at the UCT E-Research Conference 2019: DataScience4D: Explore the practice, principles and opportunities of data science for development15 – 18 April 2019 held at the University of Cape Town

In this paper, we present an instructional HPC-based platform, ModStore, for Singapore students to learn data science through National Science Experiment (NSE) big data challenge. Before the challenge, Singapore students, ranging from secondary school, high school to junior college, polytechnics and ITE, carry specially designed sensors to collect data on their daily travel and the surrounding environment during the experiment. However, most of the students do not have any programming background to do data science for this big data challenge. To address this issue, we develop ModStore, powered by National Supercomputing Centre, Singapore, which serves as a powerful tool to enable students to learn about the Internet of Things and Big Data in a drag-and-drop manner by building the workflow, analyzing the information, creating graphs, interpreting visualizations, and comparing trends. 180 students were using ModStore to participate this challenge from October to December, 2016. It has been touted as a promising instructional platform for learning Mathematics, statistics and data science.

Data Scientist - The New ProfessionData scientist is probably the most trendy job in Information Technology (IT) nowadays. This new profession emerged with the Big Data wave. Even though there is no such thing like an exact job profile, we expect that the data scientist can handle all the Big Data challenges that are novel to us. Without being a magician she or he shall help to deliver us all the magic Big Data promises. The data scientist capitalizes on unstructured data without taking the roles of a programmer, database expert, statistician, or content manager. All these professions are around for decades. So, why invent a new one?More than anything, what data scientists do is make discoveries while swimming in data [...] they are able to bring structure to large quantities of formless data and make analysis possible. [3] They sketch, orchestrate, and control the discovery process. The leading paradigm in this process it to find information that meets a certain need or provides an answer for a certain problem. need to avoid the temptation of following a data-driven approach instead of a problem-driven one. [5] The data scientist has to develop an idea about the required information that meets our information need. We can expect that data scientists have a deep understanding of the foundations of both, the nature of information and the domain. They follow a mental model on the information demand that abstractly reflects the facts they expect to encounter in data and the way to detect and present them to us. We expect the data scientist to discover novel information that may provide us with new insights. And we want these insights to be true. It is thus part of the data scientist's responsibility to make sure that the discovered information is not only novel but also trustworthy. The data scientists cannot prove data analysis models. That exceeds their capabilities. We cannot hold them liable for information that eventually turns out to be wrong. Nevertheless their skills should include a sound sensation of plausibility that helps them to raise doubts and to prompt a closer look when the results of data analysis seem too questionable to them. However, separating questionable results from plausible ones is a task that is far from being trivial.Separating Data Science from Data FictionWhen driving a car we often encounter these nice roadside signs that sometimes make us realize that we're driving too fast. Furthermore, on some occasions there is official personnel not far from these signs measuring our speed, explaining to us our traffic infraction face-to-face and documenting it on a speeding ticket. Doesn't this sound a bit outdated? There are enough sensors in a car that measure location, speed, time, and more. Even if the car's sensors don't measure those, the driver's cellphone can do it. Combining this sensor information with the cartographic data about roads and speed limits we can easily imagine that, by the end of day, the car or the phone exactly knows every infraction we committed and can automatically trigger the issuing of speeding tickets. Aren't there just legal aspects that hamper this technically feasible scenario making its way into reality?One trait of Big Data is the availability of sensor data and their combination with already available data generating new information, like the detection of traffic infraction. We can extrapolate this scenario in many directions, including more types of infraction, driving for an irresponsibly long time without a substantial break, or detecting patterns of aggressive driving. We can extend it also towards future possibilities if we think about sensor information from smart watches indicating a possibly problematic pulse rate or from the car's air control sensor that the driver may drive under the influence of alcohol.Much like in Data Mining, the strength of Big Data originates from the combination of facts producing new information. …

Data Science is associated with new discoveries, the discovery of value from the data. It is a practice of deriving insights and developing business strategies through transformation of data in to useful information. It has been evaluated as a scientific field and research evolution in disciplines like statistics, computing science, intelligence science, and practical transformation in the domains like science, engineering, public sector, business and lifestyle. The field encompasses the larger areas of artificial intelligence, data analytics, machine learning, pattern recognition, natural language understanding, and big data manipulation. It also tackles related new scientific challenges, ranging from data capture, creation, storage, retrieval, sharing, analysis, optimization, and visualization, to integrative analysis across heterogeneous and interdependent complex resources for better decision-making, collaboration, and, ultimately, value creation. In this paper we entitled epicycles of analysis, formal modeling, from data analysis to data science, data analytics -A keystone of data science, The Big data is not a single technology but an amalgamation of old and new technologies that assistance companies gain actionable awareness. The big data is vital because it manages, store and manipulates large amount of data at the desirable speed and time. Big data addresses detached requirements, in other words the amalgamate of multiple un-associated datasets, processing of large amounts of amorphous data and harvesting of unseen information in a time-sensitive generation.
 As businesses struggle to stay up with changing market requirements, some companies are finding creative ways to use Big Data to their growing business needs and increasingly complex problems. As organizations evolve their processes and see the opportunities that Big Data can provide, they struggle to beyond traditional Business Intelligence activities, like using data to populate reports and dashboards, and move toward Data Science- driven projects that plan to answer more open-ended and sophisticated questions.
 Although some organizations are fortunate to have data scientists, most are not, because there is a growing talent gap that makes finding and hiring data scientists in a timely manner is difficult. This paper, aimed to demonstrate a close view about Data science, big data, including big data concepts like data storage, data processing, and data analysis of these technological developments, we also provide brief description about big data analytics and its characteristics , data structures, data analytics life cycle, emphasizes critical points on these issues.

The group-based preemptive scheduling provides a flexible mechanism to define the preemptive relations between tasks. However, this scheduling scheme together with a resource access synchronization protocol and requirements of fault tolerance makes the predication of a real-time system's behaviors more difficult than traditional scheduling scheme. The major contribution of our work is an algorithm to calculate the worst-case response time for tasks under the fault-tolerant group-based preemptive scheduling. We first present the algorithm assuming that no task suffers any error. Then the algorithm is extended to consider a fault model and a primary/alternative fault-tolerant mechanism. Based on the tasks' worst-case response time, the schedulability of a given task set can be determined. Moreover, we also introduce a method to calculate the minimum allowed time between two consecutive faults.

In this paper, we develop a fault tolerant job scheduling strategy in order to tolerate faults gracefully in an economy based grid environment. We propose a novel adaptive task checkpointing based fault tolerant job scheduling strategy for an economy based grid. The proposed strategy maintains a fault index of grid resources. It dynamically updates the fault index based on successful or unsuccessful completion of an assigned task. Whenever a grid resource broker has tasks to schedule on grid resources, it makes use of the fault index from the fault tolerant schedule manager in addition to using a time optimization heuristic. While scheduling a grid job on a grid resource, the resource broker uses fault index to apply different intensity of task checkpointing (inserting checkpoints in a task at different intervals). To simulate and evaluate the performance of the proposed strategy, this paper enhances the GridSim Toolkit-4.0 to exhibit fault tolerance related behavior. We also compare "checkpointing fault tolerant job scheduling strategy" with the well-known time optimization heuristic in an economy based grid environment. From the measured results, we conclude that even in the presence of faults, the proposed strategy effectively schedules grid jobs tolerating faults gracefully and executes more jobs successfully within the specified deadline and allotted budget. It also improves the overall execution time and minimizes the execution cost of grid jobs.

This paper presents a novel design of a coprocessor that performs hardware-accelerated task scheduling for embedded real-time systems consisting of mixed-criticality real-time tasks. The proposed solution is based on the Robust Earliest Deadline (RED) algorithm and previously developed hardware architectures used for scheduling of real-time tasks. Thanks to the HW implementation of the scheduler in the form of a coprocessor, the scheduler operations (i.e., instructions) are always completed in two clock cycles regardless of the actual or even maximum task amount within the system. The proposed scheduler was verified using simplified version of UVM and applying billions of randomly generated instructions as inputs to the scheduler. Chip area costs are evaluated by synthesis for Intel FPGA Cyclone V and for 28-nm TSMC ASIC. Three versions of real-time task schedulers were compared: EDF-based scheduler designed for hard real-time tasks only, GED-based scheduler and the proposed RED-based scheduler, which is suitable for tasks of various criticalities. According to the synthesis results, the RED-based scheduler consumes LUTs and occupies larger chip area than the original EDF-based scheduler with equivalent parameters used. However, the RED-based scheduler handles variations of task execution times better, achieves higher CPU utilization and can be used for the scheduling of hard real-time, soft real-time and nonreal-time tasks combined in one system, which is not possible with the former algorithms.

Efficient overloading techniques for primary-backup scheduling in real-time systems

Efficient overloading techniques for primary-backup scheduling in real-time systems

Big data is a declaration used to recognize the database whose area is afar the potential of typical database software tools to store, organize and examine. Big data has shown a new path toward the mankind. With several theoretical and technological obstacles in health huge processing, it is onerous to transfer knowledge into fortunate and valuable applications. Meeting the challenge of handling big data in healthcare information construction procedure, this paper proposes a referential architecture on the Hive and Spark platform to overcome the problems in healthcare big data process. Hive is a noteworthy project as a result of it permits exposing the simplest components of Hadoop, specifically map reduce and knowledge storage. Spark may be a memory-based computing framework that features a higher ability of computing and fault tolerance, supports batch, interactive, iterative and flow calculations. Experiment results of data upload, data query and data analysis show that the performance of the proposed framework is greatly improved, and a brief summary of the performance and the differences between two methods of Hive and Spark is also discussed.

In this study the evolution of Big Data (BD) and Data Science (DS) literatures and the relationship between the two are analyzed by bibliometric indicators that help establish the course taken by publications on these research areas before and after forming concepts. We observe a surge in BD publications along a gradual increase in DS publications. Interestingly, a new publications course emerges combining the BD and DS concepts. We evaluate the three literature streams using various bibliometric indicators including research areas and their origin, central journals, the countries producing and funding research and startup organizations, citation dynamics, dispersion and author commitment. We find that BD and DS have differing academic origin and different leading publications. Of the two terms, BD is more salient, possibly catalyzed by the strong acceptance of the pre-coordinated term by the research community, intensive citation activity, and also, we observe, by generous funding from Chinese sources. Overall, DS literature serves as a theory-base for BD publications.