Research on information retrieval algorithm based on big data block and attribute matrix reasoning

Big data analytics and mining aims to discover implicit, previously unknown, and potentially useful information and knowledge from big data sets that contain huge volumes of valuable veracious data collected or generated at a high velocity from a wide variety of rich data sources. Among different big data analytic and mining tasks, this chapter focuses on frequent pattern mining. By relying on the MapReduce programming model, researchers only need to specify the “map” and “reduce” functions to discover (organizational) knowledge from (i) big data sets of precise data in a breadth-first manner or depth-first manner and/or from (ii) big data sets of uncertain data. Such a big data analytics process can be sped up by focusing the mining according to the user-specified constraints that express the user interests. The resulting (constrained or unconstrained) frequent patterns mined from big data sets provide users with new insights and a sound understanding of users' patterns. Such (organizational) knowledge is useful is many real-life information science and technology applications.

Big data analytics and mining aims to discover implicit, previously unknown, and potentially useful information and knowledge from big data sets that contain huge volumes of valuable veracious data collected or generated at a high velocity from a wide variety of rich data sources. Among different big data analytic and mining tasks, this chapter focuses on frequent pattern mining. By relying on the MapReduce programming model, researchers only need to specify the “map” and “reduce” functions to discover (organizational) knowledge from (i) big data sets of precise data in a breadth-first manner or depth-first manner and/or from (ii) big data sets of uncertain data. Such a big data analytics process can be sped up by focusing the mining according to the user-specified constraints that express the user interests. The resulting (constrained or unconstrained) frequent patterns mined from big data sets provide users with new insights and a sound understanding of users' patterns. Such (organizational) knowledge is useful is many real-life information science and technology applications.

The paper studies the processing of large information data arrays (Big Data) in construction. The issues of the applicability of the big data concept (Big Data) at various stages of the life cycle of buildings and structures are considered. Methods for data conversion for their further processing are proposed. The methods used in the analysis of "big data" allow working with unstructured data sets (Data Mining). An approach is considered, in which the analysis of arbitrary data can be reduced to text analysis, similar to the analysis of ordinary text messages. At the moment, it is important and interesting to isolate non-obvious links present in the analysed data. The advantage of using big data is that it is not necessary to advance hypotheses for testing. Hypotheses appear during data analysis. Dependence analysis is a basic approach when working with big data. The concept of an automatic big data analysis system is proposed. For data mining, text analysis algorithms should be used, and discriminant functions should be used for the main problem to be solved (data classification).

This study aimed to understand the emerging effects big data is having across different business functions as well as how much the insights gathered, from big data sets, are starting to influence business strategy. The literature highlights a lack of understanding around big data, with very little research focusing on big data’s applications outside of short term marketing strategies. This research investigated the potential barriers to big data integration within businesses, and sought to understand how these could be removed. In order to achieve an in depth understanding, an interview based approach was used. A wide range of business sources were examined to further understand the gaps identified by previous research. Primary research took the form of in-depth interviews. Respondents were carefully selected based on industry knowledge and experience, working for companies such as Ebiquity PLC, Satalia and Barclays. The interviews were used to compare and contrast the experts’ opinions and establish whether the key themes identified in the literature held true in the primary research. This approach enabled real insight to be gathered in order to concisely examine and critique the current literature as well as enabled further themes to be investigated. This investigation found how critical big data is becoming to business success, including understanding an emerging gulf between those who have adopted data big sets and those who have not. It was also understood that, on the whole, knowledge and understanding is lagging far behind benchmarks set by some businesses. The paper concludes by explaining why big data is not influencing business strategy. A focused understanding approach is outlined as the preferred solution to bridge the gulf between the ‘haves’ and ‘have not’s’.

Big data: More than big data sets

Dimension reduction or feature selection is thought to be the backbone of big data applications in order to improve performance. Many scholars have shifted their attention in recent years to data science and analysis for real-time applications using big data integration. It takes a long time for humans to interact with big data. As a result, while handling high workload in a distributed system, it is necessary to make feature selection elastic and scalable. In this study, a survey of alternative optimizing techniques for feature selection are presented, as well as an analytical result analysis of their limits. This study contributes to the development of a method for improving the efficiency of feature selection in big complicated data sets.

This paper analyzes the criteria for applying the essential facilities doctrine to intellectual property rights and the possibility of applying it in cases where Big Data is the alleged essential facility. It aims to answer the research question: “What are the specifics of the intellectual property criteria in essential facilities cases and are these criteria applicable to Big Data?” It points to the semantic openness of the “new product” and “technical progress” conditions that have been developed for assessing whether an intellectual property right constitutes an essential facility. The paper argues that the intellectual property criteria are not applicable in all access to Big Data cases because Big Data is not necessarily protected by copyright. While a set of Big Data could be protected by copyright if certain conditions are met, even in such cases the lack of intrinsic value of Big Data significantly limits the applicability of the intellectual property criteria.

Big Data Analytics methods take advantage of techniques from the fields of data mining, machine learning, or statistics with a focus on analysing large quantities of data (aka `big datasets') with modern technologies. Big data sets appear in remote sensing in the sense of large volumes, but also in the sense of an ever increasing amount of spectral bands (i.e., high-dimensional data). The remote sensing has traditionally used the above described techniques for a wide variety of application such as classification (e.g., land cover analysis using different spectral bands from satellite data), but more recently scalability challenges occur when using traditional (often serial) methods. This paper addresses observed scalability limits when using support vector machines (SVMs) for classification and discusses scalable and parallel developments used in concrete application areas of remote sensing. Different approaches that are based on massively parallel methods are discussed as well as recent developments in parallel methods.

Big data is more than a matter of size; it is an emerging paradigm of data of very large size (volume) and fast in/out (velocity), from various sources (variety), and of high value for knowledge extraction and decision making. Technological advances in data gathering have led to a rapid proliferation of big data in diverse areas such as remote sensing, medicine, the Internet, and social sectors. Such data brings opportunities and challenges to scientists and engineers. In order for us to make use of this massive amount of data, new data management and computational approaches are needed to permit scientists and engineers to analyze the data in (nearly) real time, often in a distributed or streaming manner. Various technologies are being discussed, and some have been realized, to support the handling of big data. In addition, big dataset cannot be stored in one location, and massively parallel processing databases and scalable storage systems are being designed to store the large datasets. What is more, big data generates an industry of supporting architectures; many cloud computing platforms and frameworks are developed to handle the big data operations, such as MapReduce. To deal with different properties of big data, different algorithms also need to be developed. Overall, big data is an opportunity to find insights in new and emerging types of data and content, to make models more agile, and to answer questions that were previously considered beyond our reach. The purpose of this special is to highlight some recent advancement to address such challenges in the big data era.

The average volume of data produced daily is estimated to be over 2.5 quintillion byte. Moreover, by year 2020, it is estimated that 1.79MB of data will be created every second by each person in the world. Apparently, big datasets contain tremendous amount of valuable information that can be used for improved decision making. However, big data requires incredible amount of storage and computational resources for effective processing. Machine Learning (ML) algorithms are effective tools popularly used to analyze and extract concealed insights from datasets. However, some ML algorithms were not originally designed to handle big datasets, hence their computational complexity decreases with increase in data size. Consequently, this makes big data analytics extremely slow or unrealistic. Therefore, there is an obvious need for fast and effective techniques for big data analytics. This paper introduces an intelligent hybrid ML-based technique suitable for big data analytics (called EDISA_ML). EDISA_ML is a boundary detection and instance selection algorithm, inspired by edge detection in image processing. It was evaluated on four ML algorithms and big datasets, and the results show that it achieved a storage reduction of over 50% and simultaneously improved the training speed of the evaluated ML algorithms by over 93% (in some cases), without meaningfully affecting their prediction accuracy.

<span>Recently Big Data has become one of the important new factors in the business field. This needs to have strategies to manage large volumes of structured, unstructured and semi-structured data. It’s challenging to analyze such large scale of data to extract data meaning and handling uncertain outcomes. Almost all big data sets are dirty, i.e. the set may contain inaccuracies, missing data, miscoding and other issues that influence the strength of big data analytics. One of the biggest challenges in big data analytics is to discover and repair dirty data; failure to do this can lead to inaccurate analytics and unpredictable conclusions. Data cleaning is an essential part of managing and analyzing data. In this survey paper, data quality troubles which may occur in big data processing to understand clearly why an organization requires data cleaning are examined, followed by data quality criteria (dimensions used to indicate data quality). Then, cleaning tools available in market are summarized. Also challenges faced in cleaning big data due to nature of data are discussed. Machine learning algorithms can be used to analyze data and make predictions and finally clean data automatically.</span>

Central to ethical debates in contemporary mental health research are the rhetoric of parity of esteem, challenges underpinned by the social construct of vulnerability and the tendency to homogenise the population diagnosed with mental health conditions. Such ethical dimensions are further complicated by the contemporary endeavour to work with ‘big data’ which has led to ambitious claims for discovery and knowledge. Research in mental health is challenging due to the perceived constraints of ethical principles such as the protection of autonomy, consent, risk and harms. This article discusses how ethical considerations need to be reconceptualised when using big data sets. The argument is foregrounded with an appraisal of the prevailing political discourse of parity of esteem demonstrating that ongoing disparities in services and research should also be considered when inquiry uses big data.

Big data have the potential to change the way responders make sense of crisis situations, respond, and make decisions concerning the crises. At the same time, however, the explosion to the amount of crisis-related data can create an information overload to the crisis responders, and a challenge for their efficient management and utilisation. Crisis big data streams for epidemics may lack, for instance, key demographic identifiers such as age and sex, or may underrepresent certain age groups as well as residents of developing countries. Relevant metadata information needs therefore to be obtained and validated in order to trust make predictions and decisions based on the big data set. Crisis-related big data must be meaningful to the responders in order to form the basis of sound decisions. The aim of the chapter is to review all issues pertaining to the use of metadata for big data in emergency/crisis management situations.

In this article we introduce modern statistical machine learning and bioinformatics approaches that have been used in learning statistical relationships from big data in medicine and behavioral science that typically include clinical, genomic (and proteomic) and environmental variables. Every year, data collected from biomedical and behavioral science is getting larger and more complicated. Thus, in medicine, we also need to be aware of this trend and understand the statistical tools that are available to analyze these datasets. Many statistical analyses that are aimed to analyze such big datasets have been introduced recently. However, given many different types of clinical, genomic, and environmental data, it is rather uncommon to see statistical methods that combine knowledge resulting from those different data types. To this extent, we will introduce big data in terms of clinical data, single nucleotide polymorphism and gene expression studies and their interactions with environment. In this article, we will introduce the concept of well-known regression analyses such as linear and logistic regressions that has been widely used in clinical data analyses and modern statistical models such as Bayesian networks that has been introduced to analyze more complicated data. Also we will discuss how to represent the interaction among clinical, genomic, and environmental data in using modern statistical models. We conclude this article with a promising modern statistical method called Bayesian networks that is suitable in analyzing big data sets that consists with different type of large data from clinical, genomic, and environmental data. Such statistical model form big data will provide us with more comprehensive understanding of human physiology and disease.

This paper introduces CORE-BCD-mAI, a COmposite Framework for REpresenting, Querying, and Analyzing Big Clinical Data by means of multidimensional AI Tools. The proposed framework combines Artificial Intelligence (AI) and Big Data, with the aim of accessing big clinical datasets for managing and decision-making purposes, in the healthcare application scenario. We present in detail methodologies supported by the CORE-BCD-mAI framework, along with its intrinsic anatomy. The paper also provides relevant research challenges deriving from the overall CORE-BCD-mAI proposal.