Support vector machine based on low-rank tensor train decomposition for big data applications

Fault diagnosis of multivariate signals is a significant area of engineering research. Feature information is collected from multiple dimensions. This improves the accuracy and robustness of mechanical fault diagnosis. Dynamic mode decomposition (DMD) is a method that extracts the basic structure and mode of motion, describing the behavior of dynamic systems. It decomposes a high-dimensional nonlinear dynamic system into single-frequency DMD modes. With the advent of the big data era, the interest in tensor analysis has increased. Tensor train (TT) decomposition represents the tensor structure through a series of low-rank tensor products. To extract the fault features of multivariate signals, a low-rank TT DMD (LRTTDMD) model is proposed in this article. The multivariate signals are first converted into a data tensor using phase space reconstruction. Subsequently, the data tensor is transformed into TT format using TT decomposition while ensuring a certain level of error. The tensor operation is introduced into the original DMD to process the data tensor of TT format and extract the fault feature. The construction of tensor low-rank extraction framework enhances the robustness of the algorithm to noise. Finally, the effectiveness and feasibility of LRTTDMD are validated using simulation signals and laboratory self-test bearing fault signals.

Lots of data is generated around us in today's big data age. Much of this data is time-varying or dynamic, such as the social network connections that change across time or dynamic functional brain connectivity networks constructed across multiple subjects. In these dynamic (time-varying) networks, it is important to reduce the large amount of data into a few meaningful descriptors. One way to achieve this goal is to detect change points or anomalies in the connectivity patterns across time. Recently, there has been an interest in describing the time-varying network activity as a tensor and detecting the anomalies in terms of the changes in the subspaces of the tensor along each mode [1], [2]. However, the current approaches to tensor decomposition are not robust to non-Gaussian noise, outliers, and corruption in the data. For this reason, a robust low-rank tensor recovery algorithm similar to robust principal components analysis (RPCA) has been recently proposed. In this paper, we employ higher order robust PCA (HoRPCA) for tracking dynamic networks in time and detecting anomalies using a subspace distance measure. The proposed approach assumes that most real life networks are low-rank in nature and considers a low-rank plus sparse tensor decomposition at each time point. The subspaces corresponding to each mode and each time point are described through a projection operator and the subspace distance is quantified through a Hausdorff distance measure. The proposed framework is evaluated on both simulated networks and dynamic functional connectivity brain networks.

Mining big data in brain imaging genetics is an emerging topic in brain science. It can uncover meaningful associations between genetic variations and brain structures and functions. Sparse canonical correlation analysis (SCCA) is introduced to discover bi-multivariate correlations with feature selection. However, these SCCA methods cannot be directly applied to big brain imaging genetics data due to two limitations. First, they have cubic complexity in the size of the matrix involved and are computational and memory intensive when the matrix becomes large. Second, the parameters in an SCCA method need to be fine-tuned in advance. This further dramatically increases the computational time, and gets severe in high-dimensional scenarios. In this paper, we propose two fast and efficient algorithms to speed up the structure-aware SCCA (S2CCA) implementations without modification to the original SCCA models. The fast algorithms employ a divide-and-conquer strategy and are easy to implement. The experimental results, compared with conventional algorithms, show that our algorithms reduce the time usage significantly. Specifically, the fast algorithms improve the computational efficiency by tens to hundreds of times compared to conventional algorithms. Besides, our algorithms yield similar correlation coefficients and canonical loading profiles to the conventional implementations. Our fast algorithms can be easily parallelized to further reduce the computational time. This indicates that the proposed fast scalable SCCA algorithms can be a powerful tool for big data analysis in brain imaging genetics.

The availability of digital technology in the hands of every citizenry worldwide makes an available unprecedented massive amount of data. The capability to process these gigantic amounts of data in real-time with Big Data Analytics (BDA) tools and Machine Learning (ML) algorithms carries many paybacks. However, the high number of free BDA tools, platforms, and data mining tools makes it challenging to select the appropriate one for the right task. This paper presents a comprehensive mini-literature review of ML in BDA, using a keyword search; a total of 1512 published articles was identified. The articles were screened to 140 based on the study proposed novel taxonomy. The study outcome shows that deep neural networks (15%), support vector machines (15%), artificial neural networks (14%), decision trees (12%), and ensemble learning techniques (11%) are widely applied in BDA. The related applications fields, challenges, and most importantly the openings for future research, are detailed.

The mining and application of big data in academic journals can accelerate the construction of data journals, enhance journal’s influence, and promote the sharing and dissemination of scientific data in academic journals worldwide. This paper uses bibliometric method to retrieve published articles with the theme of big data and journal in CNKI database, analyzes the academic achievements of the development of academic journals with the application of big data in the recent five years using quantitative visualization analysis, expounds the research progress of academic journals in big data field, and discusses the advantages of big data application in periodical industry. The results show that: study on the application of big data in academic journals are gradually deepening and scientific, and the relevant research still needs more financial fund from the state and social units, big data has prominent advantages such as accuracy, scientificity and value maximization in the workflows of academic journals. The mining and application of massive data is very important for promoting the development of high-quality academic journals and optimizing the supply-demand relationship of knowledge services of academic journals.

In this chapter, we introduce the methods, architectures and tools for big data analysis. The analysis of big data mainly involves analytical methods for traditional data and big data, analytical architecture for big data, and software used for mining and analysis of big data. Data analysis is the final and the most important phase in the value chain of big data, with the purpose of extracting useful values, providing suggestions or decisions. Different levels of potential values can be generated through the analysis of datasets in different fields.

The objective of this book is to introduce the basic concepts of big data computing and then to describe the total solution of big data problems using HPCC, an open-source computing platform. The book comprises 15 chapters broken into three parts. The first part, Big Data Technologies, includes introductions to big data concepts and techniques; big data analytics; and visualization and learning techniques. The second part, LexisNexis Risk Solution to Big Data, focuses on specific technologies and techniques developed at LexisNexis to solve critical problems that use big data analytics. It covers the open source High Performance Computing Cluster (HPCC Systems) platform and its architecture, as well as parallel data languages ECL and KEL, developed to effectively solve big data problems. The third part, Big Data Applications, describes various data intensive applications solved on HPCC Systems. It includes applications such as cyber security, social network analytics including fraud, Ebola spread modeling using big data analytics, unsupervised learning, and image classification. The book is intended for a wide variety of people including researchers, scientists, programmers, engineers, designers, developers, educators, and students. This book can also be beneficial for business managers, entrepreneurs, and investors.

Big data has evolved as a most challenging area in scientific study and research. It has drawn much attention during the last few years. It influences our modern society, business, government, healthcare, research and almost every discipline. In this data-driven era, where data is continuously acquired from a verity of sources for different purposes, the ability to make timely decisions based on available data is a very critical task. The massive data size, variety, velocity, accuracy and high dimensionality presents new challenges to big data. This paper attempts to present some challenges of big data. In addition, a study on the conceptual design of big data architecture presented on specific big data applications. In this paper, we presented research work on big data analytics techniques in the areas of text, sentiment, video, social media and predictive analytics. A comparative analysis on selected big data applications has been presented in detail.

PurposeInnovative firms leverage big data analytics (BDA) benefits in optimising value creation, particularly in business-to-business (B2B) contexts. Examples of this are found in new product success and product innovation performance. However, knowledge of how innovative firms and their corporate customers generate insights from big data, develop new products and gain higher-quality service from intra- and inter organisations' resources is limited. This knowledge manifests in the form of opportunities available in BDA and through the adoption of the co-creation approach to generate value in the form of new product innovation. BDA reflects an excellent means of enhancing a firm's customer agility, but how this is possible remains largely unknown.Design/methodology/approachIn this research, the authors hypothesise that new product success is a function of a firm's customer agility and product innovation performance moderated by environmental turbulences. In turn, the firm's customer agility is enhanced by the effect of big data aggregation and analytical tools. These hypotheses have been confirmed by a survey in an emerging market.FindingsThe authors use structural equation modelling to test the authors’ hypotheses. The main contribution of this research is the conceptualisation and test of an integrative framework identifying the links among a firm's customer agility, new product success and BDA capabilities.Practical implicationsThe study established that BDA tools – the effective use of data aggregation tools and the effective use of data analysis tools – shape customer agility in achieving new product success. This study contributes to one’s understanding of the relevance of BDA in B2B value creation contexts.Originality/valueThe study findings show that BDA shapes a firm's customer agility in achieving new product success.

In recent years, tensor computation has become a promising tool for solving big data analysis, machine learning, medical image, and EDA problems. To ease the memory and computation intensity of tensor processing, decomposition techniques, especially tensor-train decomposition (TTD), are widely adopted to compress the extremely high-dimensional tensor data. Despite TTD’s potential to break the curse of dimensionality, researchers have not yet leveraged its full computational potential, mainly because of two reasons: 1) executing TTD itself is time- and energy-consuming due to the singular value decomposition (SVD) operation inside each of TTD’s iteration and 2) additional software/hardware optimizations are often required to process the obtained TT-format data in certain applications such as deep learning inference. In this article, we address these challenges with two approaches. First, we propose an algorithm-hardware co-design with customized architecture, namely, TTD Engine to accelerate TTD. We use MRI image compression as a demo application to illustrate the efficacy of the proposed accelerator. Second, we present a case study demonstrating the benefit of TT-format data processing and the efficacy of using TTD Engine. In the case study, we use the TT approach to realize convolution operation, which is difficult and nontrivial for TT-format data. Experimental results show that, TTD Engine achieves, on average, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$14.9 \times $ </tex-math></inline-formula> – <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$36.9 \times $ </tex-math></inline-formula> speedup over CPU implementations and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$4.1\times $ </tex-math></inline-formula> – <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$9.9\times $ </tex-math></inline-formula> speedup compared to the GPU baseline. The energy efficiency is also improved by at least <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$14.4\times $ </tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$5.4\times $ </tex-math></inline-formula> over CPU and GPU, respectively. Moreover, our hardware-enabled TT-format data processing further leads to more efficient implementations of complicated operations and applications.

Big data applications are becoming more popular across many different fields and civil engineering is no different. The benefits of big data lie in its potential to provide valuable insights into various large datasets. Big data applications can identify patterns and trends that were previously unknown, which can help them make informed decisions and develop effective strategies. In the case of civil engineering, this could be taking large datasets that have been produced in relation to pieces of infrastructure and using them to create more efficient management strategies. One of the issues with using big data is that if the input dataset is flawed then the output resulting from any big data analysis will be compromised. Therefore, data validation is used, which is the process of ensuring that data is accurate, complete, and consistent. The consequence of not undertaking data validation may be Inaccurate or inconsistent data which can lead to incorrect insights and decisions in big data applications. This paper explores the necessity of validating data before it is used in a big data application. It outlines some of the different methods used for validating data and provides an overview of the potential issues that may arise from data validation errors. A case study is then presented showing the process of validation on data collected from four bridges and provides recommendations for implementing data validation as part of a larger big data workflow. The results of the case study show that validation of data is an important step in the big data process both for confidence in the outputs and to make big data applications more useful and more common in the civil engineering field. The discussion and presented case study in this paper highlight the necessity of validating data. It has shown some of the potential issues that may arise from not undertaking data validation.

COVID-19, caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), spread rapidly and affected most of the world since its outbreak in Wuhan, China, which presents a major challenge to the emergency response mechanism for sudden public health events and epidemic prevention and control in all countries. In the face of the severe situation of epidemic prevention and control and the arduous task of social management, the tremendous power of science and technology in prevention and control has emerged. The new generation of information technology, represented by big data and artificial intelligence (AI) technology, has been widely used in the prevention, diagnosis, treatment and management of COVID-19 as an important basic support. Although the technology has developed, there are still challenges with respect to epidemic surveillance, accurate prevention and control, effective diagnosis and treatment, and timely judgement. The prevention and control of sudden infectious diseases usually depend on the control of infection sources, interruption of transmission channels and vaccine development. Big data and AI are effective technologies to identify the source of infection and have an irreplaceable role in distinguishing close contacts and suspicious populations. Advanced computational analysis is beneficial to accelerate the speed of vaccine research and development and to improve the quality of vaccines. AI provides support in automatically processing relevant data from medical images and clinical features, tests and examination findings; predicting disease progression and prognosis; and even recommending treatment plans and strategies. This paper reviews the application of big data and AI in the COVID-19 prevention, diagnosis, treatment and management decisions in China to explain how to apply big data and AI technology to address the common problems in the COVID-19 pandemic. Although the findings regarding the application of big data and AI technologies in sudden public health events lack validation of repeatability and universality, current studies in China have shown that the application of big data and AI is feasible in response to the COVID-19 pandemic. These studies concluded that the application of big data and AI technology can contribute to prevention, diagnosis, treatment and management decision making regarding sudden public health events in the future.

Big data management and application specialty is an emerging major in our school of information and business management. Based on the Internet plus and big data era, we mainly study the application of big data analysis theory and methods in economic management and the management and management of big data. This paper aims to explore how to improve the innovation and practical ability of students in this major through practical teaching. This paper mainly elaborates from four aspects, namely, the basic principles of practical teaching, the path of students' ability improvement, the construction of the completed practical teaching system and the application of practical teaching in this major.

The chapter showcases the awareness of big data usage among librarians in Zimbabwe. The concept of big data is new, and librarians are building capacity to move with the current trends in librarianship. This chapter assists in pointing areas where big data can be applied in libraries. It also documents the challenges that are faced when using big data applications and proffer solutions that can be applied to deal with those challenges. It answers the question of whether it is practical to utilise big data in any type of library. A qualitative study was done where an online questionnaire was administered to twenty librarians in research institutions in Zimbabwe. The findings revealed that librarians are aware of the big data concept but are not utilising the tools and techniques in data mining and analysis. The authors recommend that capacity building should be done to equip librarians with the requisite skills.

With the rapid development and popularization of big data technology in the world and the world's vigorous promotion of informatization, the education and management of college students are facing new opportunities and challenges. This paper takes the innovation and application of big data technology in college students' education and management as the core, analyzes the significance and advantages of big data in the education and management of college students, and deeply studies the innovative application of big data from the aspects of education and management resource sharing, digital education and management and open education and management. In addition, It will take the innovative application of big data in college information announcement system as an example to comprehensively improve the level of college students' education and management by big data mining in the system and other ways.