TPCx-BB (Big Bench) in a Single-Node Environment

The Workshop on Big Data Benchmarking (WBDB2012), held on May 8-9, 2012 in San Jose, CA, served as an incubator for several promising approaches to define a big data benchmark standard for industry. Through an open forum for discussions on a number of issues related to big data benchmarking—including definitions of big data terms, benchmark processes and auditing — the attendees were able to extend their own view of big data benchmarking as well as communicate their own ideas, which ultimately led to the formation of small working groups to continue collaborative work in this area. In this paper, we summarize the discussions and outcomes from this first workshop, which was attended by about 60 invitees representing 45 different organizations, including industry and academia. Workshop attendees were selected based on their experience and expertise in the areas of management of big data, database systems, performance benchmarking, and big data applications. There was consensus among participants about both the need and the opportunity for defining benchmarks to capture the end-to-end aspects of big data applications. Following the model of TPC benchmarks, it was felt that big data benchmarks should not only include metrics for performance, but also price/performance, along with a sound foundation for fair comparison through audit mechanisms. Additionally, the benchmarks should consider several costs relevant to big data systems including total cost of acquisition, setup cost, and the total cost of ownership, including energy cost. The second Workshop on Big Data Benchmarking will be held in December 2012 in Pune, India, and the third meeting is being planned for July 2013 in Xi’an, China.

The average volume of data produced daily is estimated to be over 2.5 quintillion byte. Moreover, by year 2020, it is estimated that 1.79MB of data will be created every second by each person in the world. Apparently, big datasets contain tremendous amount of valuable information that can be used for improved decision making. However, big data requires incredible amount of storage and computational resources for effective processing. Machine Learning (ML) algorithms are effective tools popularly used to analyze and extract concealed insights from datasets. However, some ML algorithms were not originally designed to handle big datasets, hence their computational complexity decreases with increase in data size. Consequently, this makes big data analytics extremely slow or unrealistic. Therefore, there is an obvious need for fast and effective techniques for big data analytics. This paper introduces an intelligent hybrid ML-based technique suitable for big data analytics (called EDISA_ML). EDISA_ML is a boundary detection and instance selection algorithm, inspired by edge detection in image processing. It was evaluated on four ML algorithms and big datasets, and the results show that it achieved a storage reduction of over 50% and simultaneously improved the training speed of the evaluated ML algorithms by over 93% (in some cases), without meaningfully affecting their prediction accuracy.

As cloud computing continues to evolve, organizations are finding new ways to store the massive amounts of big data that are collected. Big data storage often require greater flexibility and scalability which can be provided by incorporating NoSQL technologies. NoSQL (Not Only SQL) is quickly becoming a popular approach to store large and unstructured data. This paper looks at the various classifications of NoSQL technologies as well as many of the notable characteristics of the technologies. The authors also describe some deficiencies of using NoSQL and give some explanation to why companies are adopting the technology. The paper concludes with suggestions for future research of NoSQL technologies and a content analysis of current articles in database management is provided in the appendix. Keywords: big data, cloud computing, document databases, MongoDB, NoSQL, Not Only SQL, unstructured database INTRODUCTION Since NoSQL database technologies entered the market in 1998, these technologies have challenged the widely-accepted relational structures and database management practices supported by traditional SQL databases. Despite this trend, the perception that NoSQL seeks to eliminate the use of relational databases is a common misconception. NoSQL is best considered with the acronym NOSQL - Not Only SQL - which more accurately represents an approach that combines non-relational databases with the use of relational ones. This approach seeks to leverage both NoSQL and SQL technologies in order to balance the demands of performance, scalability, and schema flexibility with data integrity and consistency (Purdue, 2014; Penchikala, 2013). The rise of web-scale applications creates performance, scalability, and schema flexibility demands that present significant challenges for traditional SQL / relational databases. Thus, a variety of new entrants have entered the market, each seeking to address specific challenges, including (Kent, 2012): * The need to store large volumes of data about users, objects and products * The need to provide near-real-time transaction processing * The need to provide high-speed analytics on large volumes of data (big data) * The need to support flexible data schemas for evolving business environments * The need to scale applications horizontally with predictable hardware and licensing cost structures Each NoSQL solution seeks to address one or more of the above issues, often by providing alternative approaches to long-established relational database norms. Traditional relational databases were not designed to cope with the scalability and flexibility challenges facing modern applications. Older databases were not designed to take advantage of the inexpensive storage and excess processing power available today. The defining characteristics of NoSQL technologies, as well as the relative advantages and disadvantages of NoSQL, and the subsequent influence on adoption of NoSQL technologies are each the subject of a small, yet growing body of research. This paper provides a literature review of that research and proposes potential areas where future research may focus. REVIEW OF LITERATURE NoSQL database technology is a relatively recent innovation and much of the research surrounding it is of an emerging nature. For the purpose of this literature review Galileo was primarily used for identifying the articles. Articles were selected that focused directly on NoSQL database technologies (unstructured / semi-structured databases, document databases, etc.) and articles that focused on relational databases. The contrasting articles provided the researchers with the ability to compare the features that are offered with a greater degree of variability in NoSQL databases (for example, an article on SQL scalability provided a foundation for comparing newer horizontal scaling techniques used in certain NoSQL databases). …

The present ICT era has changed the scenario of multivariate data or information usage altogether. Organizations treat data as an asset and they try to employ various methodology to come up with organization progress oriented conclusions. A wide range of database tools to manage the huge data and equally number of software’s are also developed to visualize, present and analyses the big data. XLSTAT with diversified data analyzing utilities, is one such tool that can be appended to usual Excel software. The present paper gives an application of Naive Bayes Classifier applied to a data on Global Super Store Orders-2016 (Source: secondary data obtain from data.world platform). This application will give the insight of understanding the concept of Naive Bayes Classifier. It will also show the effect of continuous data monitoring and maintenance on derived results of Naive Bayes Classification. The summary of derived output will facilitate the comparison and will also give an idea about the overall trend of the factors under study. The step based analysis of big and diverse data shows that global accuracy of Naive Bayes Classifier increases with increase in data size.

Managing, processing and understanding big healthcare data is challenging, costly and demanding. Without a robust fundamental theory for representation, analysis and inference, a roadmap for uniform handling and analyzing of such complex data remains elusive. In this article, we outline various big data challenges, opportunities, modeling methods and software techniques for blending complex healthcare data, advanced analytic tools, and distributed scientific computing. Using imaging, genetic and healthcare data we provide examples of processing heterogeneous datasets using distributed cloud services, automated and semi-automated classification techniques, and open-science protocols. Despite substantial advances, new innovative technologies need to be developed that enhance, scale and optimize the management and processing of large, complex and heterogeneous data. Stakeholder investments in data acquisition, research and development, computational infrastructure and education will be critical to realize the huge potential of big data, to reap the expected information benefits and to build lasting knowledge assets. Multi-faceted proprietary, open-source, and community developments will be essential to enable broad, reliable, sustainable and efficient data-driven discovery and analytics. Big data will affect every sector of the economy and their hallmark will be ‘team science’.

The management of big data is mainly affected by the size of the big graph data that represents the huge volumes of data. The size of this structure may increase with the size of data to be handled over the time. Facing this issue, the querying time may be affected and the introduced delay may not be tolerated by running applications. Moreover, the investigation of attacks through the collected massive data could not be ensured using traditional approaches, which do not support big data constraints. In this context, we propose in this paper, a novel temporal conceptual graph to represent the big data and to optimize the size of the derived graph. The proposed scheme built on this novel graph structure enables tracing back of attacks using big data. The efficiency of the proposed scheme for the reconstruction of attack scenarios is illustrated using a case study in addition to a conducted comparative analysis showing how smart big graph data is obtained through the optimization of the graph size.

"Big data" has become a major force of innovation across enterprises of all sizes. New platforms with increasingly more features for managing big datasets are being announced almost on a weekly basis. Yet, there is currently a lack of any means of comparability among such platforms. While the performance of traditional database systems is well understood and measured by long-established institutions such as the Transaction Processing Performance Council (TCP), there is neither a clear definition of the performance of big data systems nor a generally agreed upon metric for comparing these systems. In this article, we describe a community-based effort for defining a big data benchmark. Over the past year, a Big Data Benchmarking Community has become established in order to fill this void. The effort focuses on defining an end-to-end application-layer benchmark for measuring the performance of big data applications, with the ability to easily adapt the benchmark specification to evolving challenges in the big data space. This article describes the efforts that have been undertaken thus far toward the definition of a BigData Top100 List. While highlighting the major technical as well as organizational challenges, through this article, we also solicit community input into this process.

The importance of big data, one of the most popular and researched topics of today, has been a subject of constant debate. Big data appear in almost every aspect of our lives, such as healthcare, education, shopping, social media, and industry; however, its storage and processing is not very efficient when using traditional methods. Therefore, the aim of this study is to obtain big data using a novel social shopping application that collects data from its users. The application is designed to collect information about people, products, and friendships among people, as well as product relationships, and post creations, and also enables setting up various options to mark products, such as like, buy, have, or help. The data gathered by the system is analyzed using both relational and non-relational databases, and the performance of these databases is then compared using specific queries, to reveal the model which performs better and more efficiently. Three different database models were designed and implemented for the system; a relational database, a document database, and a graph database. As a result of the experiments, there is no single model that is superior to the others as all three databases have their advantages and disadvantages. Therefore, the database selection should be decided based on the application domain.

Nowadays a lot of massive data is stored and typically, the data itself contains a lot of non-trivial but useful information. Data mining techniques can be used to discover this information which can help the companies for decision-making. However, in real life applications, data is massive and is stored over distributed sites. One of my major research topics is to protect privacy over this kind of data. Previously, the important characteristics, issues and challenges related to management of the large amount of data has been explored. Various open source data analytics frameworks that deal with large amount of Data analytics workloads have been discussed. Comparative study between the given frameworks and suitability of the same has been proposed. Digital universe is flooded with huge amount of data generated by number of users worldwide. These data are of diverse in nature, come from various sources and in many forms. To keep with the desire to store and analyze ever larger volumes of complex data, relational databases vendors have delivered specialized analytical platforms that come in many shapes and sizes from software only to analytical services that run in third party hosted environments. In addition new technologies have emerged to address exploding volumes of complex data, including web traffic, social media content and machine generated data including sensor data, global positioning system data.Big data is defined as large amount of data which requires new technologies and architectures so that it becomes possible to extract value from it by capturing and analysis process. Due to such large size of data it becomes very difficult to perform effective analysis using the existing traditional techniques. Big data has become a prominent research field, especially when it comes to decision making and data analysis. However, Big data due to its various properties like volume, velocity, variety, variability, value and complexity put forward many challenges. Since Big data is a recent upcoming technology in the market which can bring huge benefits to the business organizations, it becomes necessary that various challenges and issues associated in bringing and adapting to this technology are brought into light. Another challenge is that data collection may not have enough accuracy which will lead to a non-consistent analysis that can critically affect the decision based on this analysis. Moreover, it is clearly apparent that organizations need to employ data-driven decision making to gain competitive advantage. Processing, integrating and interacting with more data should make it better data, providing both more panoramic and more granular views to aid strategic decision making. This is made possible via Big Data exploiting affordable and usable Computational and Storage Resources. Many offerings are based on the Map-Reduce and Hadoop paradigms and most focus solely on the analytical side. Nonetheless, in many respects it remains unclear what Big Data actually is; current offerings appear as isolated silos that are difficult to integrate and/or make it difficult to better utilize existing data and systems. Since data is growing at a huge speed making it difficult to handle such large amount of data (Exabyte). The main difficulty in handling such large amount of data is because that the volume is increasing rapidly in comparison to the computing resources. The Big data term which is being used now a days is kind of misnomer as it points out only the size of the data not putting too much of attention to its other existing properties.If data is to be used to make accurate decisions in time it becomes necessary that it should be available in accurate, complete and timely manner. This makes the data management and governance process bit complex adding the necessity to make Data open and make it available to government agencies in standardized manner with standardized APIs, metadata and formats thus leading to better decision making, business intelligence and productivity improvements.This paper presents a discussion and evaluation for the most prominent techniques used in the processes of data collection and analysis in order to identify the privacy defects in them that affects the accuracy of big data. Depending on the results of this analysis, recommendations were provided for improving data collection and analysis techniques that will help to avoid if not all then most of the problems facing the use of big data in decision making. Keywords: Big Data, Big Data Challenges, Big Data Accuracy, Big Data Collection, Big Data Analytics.

Data is becoming the world's new natural resource and big data use grows quickly. The trend of computing technology is that everything is merged into the Internet and 'big data' are integrated to comprise complete information for collective intelligence. With the increasing size of big data, refining big data themselves to reduce data size while keeping critical data (or useful information) is a new approach direction. In this paper, we provide a novel data consumption model, which separates the consumption of data from the raw data, and thus enable cloud computing for big data applications. We define a new Data-as-a-Product (DaaP) concept, a data product is a small sized summary of the original data and can directly answer users' queries. Thus, we separate the mining of big data into two classes of processing modules: the refine modules to change raw big data into small sized data products, and application-oriented mining modules to discover desired knowledge further for applications from well-defined data products. Our practices of mining big stream data, including medical sensor stream data, streams of text data and trajectory data, demonstrated the efficiency and precision of our DaaP model for answering users' queries.

Abstract. The Lo's modified rescaled adjusted range test (R/S test) (Lo, 1991), GPH test (Geweke and Porter-Hudak, 1983) and two approximate maximum likelihood estimation methods, i.e., Whittle's estimator (W-MLE) and another one implemented in S-Plus (S-MLE) based on the algorithm of Haslett and Raftery (1989) are evaluated through intensive Monte Carlo simulations for detecting the existence of long-memory. It is shown that it is difficult to find an appropriate lag q for Lo's test for different short-memory autoregressive (AR) and fractionally integrated autoregressive and moving average (ARFIMA) processes, which makes the use of Lo's test very tricky. In general, the GPH test outperforms the Lo's test, but for cases where a strong short-range dependence exists (e.g., AR(1) processes with φ=0.95 or even 0.99), the GPH test gets useless, even for time series of large data size. On the other hand, the estimates of d given by S-MLE and W-MLE seem to give a good indication of whether or not the long-memory is present. The simulation results show that data size has a significant impact on the power of all the four methods because the availability of larger samples allows one to inspect the asymptotical properties better. Generally, the power of Lo's test and GPH test increases with increasing data size, and the estimates of d with GPH method, S-MLE method and W-MLE method converge with increasing data size. If no large enough data set is available, we should be aware of the possible bias of the estimates. The four methods are applied to daily average discharge series recorded at 31 gauging stations with different drainage areas in eight river basins in Europe, Canada and USA to detect the existence of long-memory. The results show that the presence of long-memory in 29 daily series is confirmed by at least three methods, whereas the other two series are indicated to be long-memory processes with two methods. The intensity of long-memory in daily streamflow processes has only a very weak positive relationship with the scale of watershed.

The main problem has been managing databases because they are continuously expanding quickly and getting more complicated in terms of volume, variety, and velocity. Currently, Relational Database Management Systems (RDMS), also known as SQL, or traditional search engines are primarily used to manage and utilize data collection. Due to their highly semantic properties and applications, relational databases have emerged as powerful and superior during the previous two decades. The handling of a high volume of data and the variability in data type and structure has become a laborious task since Big Data entered the IT industry. Relational databases are inadequate to handle such big data due to their rigid data limitations, structure, relations, and other factors. As a result, data aggregation is rendered impossible. NoSQL databases offer a practical and understandable foundation for combining massive amounts of data, structures, and interactions. Data modeling and migration are now needed in order to define the issue. There isn't yet a tool for switching from relational to no-SQL databases. For this migration, relational (SQL) database queries must be converted to NoSQL database queries. Since relational databases (RDBMS) have found it difficult to keep up with modernization, NoSQL has emerged as the most practical database. This essay aims to highlight and assess the applications of NoSQL, the notion of NoSQL data modeling, and the NoSQL database migration procedure. NoSQL, NoSQL Roadmap, Data Modeling, ETL Approach for Data Migration, Data Loading, and Data Extraction.

After a disaster such as earthquakes, debris flows, forest fires, or landslides, etc., a lot of people have to be away from their home and gather in a shelter. In addition, the refugees suffer from the shortage of necessary resources due to impaired life infrastructure, such as damaged roads and communication networks. The degree of reducing damage depends on the amount of food, water, daily necessities and communication resources required by each shelter. How to effectively and efficiently allocate resources according to grasp the exact need of a disaster situation will be an important issue. We estimate the degree of the disaster by collecting and analyzing big data from the SNS, and building a platform for the communication resources to be efficiently and effectively allocated. In order to achieve this goal, we are challenging the following issues A) Understanding situations (user requirements) after disaster occur The SNS streams large scale semantic information about real time situation in society, especially during and after disaster. It is both domain-specific and computational challenge in processing the heterogeneous large data set to extract the exact situational content with reduced semantic uncertainty. The machine learning (ML) and natural language processing (NPL) tool kits are useful in semantic analysis, but still needs domain-specific implementation and computational improvement for the situation understanding from the SNS big data. B) Understanding distribution patterns of situations/users' requirements The disaster related situation is spatiotemporally correlated, and varies dynamically in space and time. It is also domain-specific and computational challenge in estimating the spatiotemporal distribution patterns of the disaster affect based on the spatial big data from SNS. The scan statistics such as the spatial scan have provided well tested mathematical tools and software for spatial data mining. However, new methodologies are necessary since the assumptions have to be different when it meets the spatial big data in SNS. And the computational complexity in spatial big data is also a bottleneck for real-time processing. C) Solving uncertainty of big crowd data One of the major features in big crowd data, e.g., SNS data, is uncertainty behind the data. Especially in a disaster scenario, the collecting time period cannot be long enough to smooth the data automatically. How to efficiently solve uncertainty problem in the big crowd data in a disaster scenario becomes a new and big challenge for disaster management.

An effective model for store and retrieve big health data in cloud computing

Cloud computing and big data: Technologies and applications