Data Analysis and Related Applications 5

In smart grid, more data will be collected at finer granularity. It will promote the development of power system control and innovation technology. With the advantages of cloud computing technology are becoming increasingly prominent, it is gradually entering into power industry research. This paper focuses on cloud-based data processing and analysis techniques of user side in smart grid. To prove that the integration of cloud computing, smart grid, data analysis technique is practical, we introduced the cloud platform architecture designed for data processing and its key technologies first. Then, main sections of cloud-based data processing, including data collection, storage, analysis and visualization are discussed. Especially, based on smart grid scenes, data analysis methods in data analysis section are described in detail.

_ This article, written by JPT Technology Editor Chris Carpenter, contains highlights of paper SPE 217665, “Well Construction in the Era of Big Data: It’s Not Data Analytics, It’s Engineering With Data,” by Gregory S. Payette, SPE, ExxonMobil; Jeffrey R. Bailey, SPE, consultant; and Paul E. Pastusek, SPE, ExxonMobil, et al. The paper has not been peer reviewed. _ The authors write that data science captures value in well construction when data-analysis methods such as machine learning are underpinned by first principles derived from physics and engineering and supported by deep domain expertise. The authors provide a summary of important data pertaining to the well-construction process and discuss high-level areas where data science can add value to well construction through analysis of such data. Introduction The authors write that it is important for organizations to grow data science and physics-based modeling solutions together; well-construction data science should be what they term an “engineering with data” endeavor. A large portion of the complete paper is dedicated to discussing ways that organizations can improve their abilities to derive value from data-science efforts. Most discussion focuses on steps that data science teams can take today. However, the provided commentary on data management and governance is forward-looking. The paper concludes by examining several historical data-science case studies for well construction. Each example highlights the need to connect data and some physical or engineering process to deliver value through data science. Improving the Effect of Data Science in Well Construction Data Management and Governance. A step that can be taken to enable better holistic well-construction data management is for data collectors and creators to organize their data using data models that center around what the authors term “data entities” that are common and stable across data collectors. Common well-construction data entities include items such as Wells, Wellbores, Hole Sections, Surveys, Casing Strings, Drillstrings and Bottomhole Assemblies (BHAs), Tubing Strings, and Downhole Completion Equipment. These data entities are capitalized in this paper to denote their roles as data entities. The industry is accustomed to organizing data around entities. Two common types of databases particularly relevant to data-science efforts commonly used by operators include the following: - Well Information Management Databases (WIMD)—These databases are used to document information about the life cycle of wells. - Wellsite Information Transfer Standard Markup Language (WITSML) Databases: WITSML databases are primarily used to store time- and depth-series sensor data collected during drilling and completion activities. For the purposes of their paper, the authors do not include other data entities that can be captured through WITSML data repositories. Even though WIMD and WITSML databases share common data entities, such entities typically are not linked to one another through formal relationships at the original database levels. This creates a barrier for data-science projects that require data from both source systems.

In this thesis, we study two random models with various applications in data analysis. For our first model, we investigate subspaces spanned by biased random vectors. The underlying random model is motivated by applications in computational biology, where one aims at computing a low-rank matrix factorization involving a binary factor. In a random model with adjustable expected sparsity of the binary factor, we show for a large class of random binary factors that the corresponding factorization problem is uniquely solvable with high probability. In data analysis, such uniqueness results are of particular interest; ambiguous solutions often lack interpretability and do not give an insight into the structure of the underlying data. For proving uniqueness in this random model, small ball probability estimates are a key ingredient. Since to the best of our knowledge, there are no such estimate suitable for our application, we prove an extension of the famous Lemma of Littlewood and Offord. Hereby, we also discover a connection between the matrix factorization problem at hand and the notion of Sperner families. In the second part of this thesis, we will investigate a model for randomized ultrasonic data in nondestructive testing. Here, we aim at accelerating the data acquisition process by superposing ultrasonic measurements with random time shifts. To this end, we will first study the effects of randomized ultrasonic measurements in the context of the Synthetic Aperture Focusing Technique (SAFT), a widely used defect imaging method. By adapting SAFT to our random data model, we will significantly improve its performance for randomized data. In this way, for sparse defects and with high probability, we achieve better defect reconstructions as with SAFT applied to deterministic ultrasonic data acquired in the same amount of time.

Plant metabolomics is an important research area in plant science. Chemometrics is a useful tool for plant metabolomic data analysis and processing. Among them, high-order chemometrics represented by tensor modeling provides a new and promising technical method for the analysis of complex multi-way plant metabolomics data. This paper systematically reviews different tensor methods widely applied to the analysis of complex plant metabolomic data. The advantages and disadvantages as well as the latest methodological advances of tensor models are reviewed and summarized. At the same time, application of different tensor methods in solving plant science problems are also reviewed and discussed. The reviewed applications of tensor methods in plant metabolomics cover a wide range of important plant science topics including plant gene mutation and phenotype, plant disease and resistance, plant pharmacology and nutrition analysis, and plant products ingredient characterization and quality evaluation. It is evident from the review that tensor methods significantly promote the automated and intelligent process of plant metabolomics analysis and profoundly affect the paradigm of plant science research. To the best of our knowledge, this is the first review to systematically summarize the tensor analysis methods in plant metabolomic data analysis.

The article presents, summarizes and develops technological concepts of cyber security based on machine learning, data processing and analysis. The seven steps of cyber attacks, use of key techniques of data analysis in cyber defense, application of data processing and analysis in information security and some tools used are presented. Applying data processing and analysis as well as extracting dependencies from them in the field of cyber security has some advantages and disadvantages wich are indicated. A common scheme for data analysis is introduced. Main types of penetration of cyberattacks are involved. Cyber attack models abuse detection and anomaly detection are presented too. Тhree types of anomalies the timing, the number, and the pattern(s) are revealed and examples are pointed.

Research work is one of the most difficult types of independent work of applicants for the second level of higher education (master's students), the implementation of which requires special knowledge and skills. For master's cadets, preparing and writing a master's thesis is the first experience of research activity. In the Kherson State Maritime Academy, for master's cadets specializing in «Navigation and Management of Sea Vessels», the course «Modern Methods of Scientific Research and Data Processing» (MMSRDP) is provided, which, according to the curriculum and educational and professional program, is included in the cycle of professional training. The program of the academic discipline «Modern Methods of Scientific Research and Data Processing» is drawn up in accordance with the Code of Training, Certification and Watchkeeping for Seafarers STCW-78, IMO-MODEL COURSE 7.01, 7.03. and according to EPP. The direction of our research is to improve the teaching of the course «Modern Methods of Scientific Research and Data Processing» in the Kherson State Maritime Academy to achieve more significant standards of professional competence of applicants for the second level of higher education (master's students) in the specialization «Navigation and Management of Marine Vessels» of KhSMA. This article examines two of the most important components of the MMSRDP course. The first is the «starting» knowledge of the methodology and methods of scientific research, namely, the most significant classical paradigms that determine the essence of the concepts of methodology and methods of scientific research, concepts and issues of modern methods of empirical and theoretical research. The second, no less important component of the MMSRDP course is the study by master's students of specific methods and techniques of data processing and analysis. Our further research is aimed at a more detailed consideration of the second component of the MMSRDP course - the study by master's students of specific methods and techniques of data processing and analysis in the field of technical sciences, in particular, in the field of navigation, which are most necessary for master's students of the navigation specialty, as well as the development of a set of individual tasks for master's students aimed at the application of specific methods and techniques of data collection and analysis

Biomedical signal processing and data analysis play pivotal roles in the advanced medical expert system solutions. Signal processing tools are able to diminish the potential artifact effects and improve the anticipative signal quality. Data analysis techniques can assist in reducing redundant data dimensions and extracting dominant features associated with pathological status. Recent computational methods have greatly improved the effectiveness of signal processing and data analysis, to support the efficient point-of-care diagnosis and accurate medical decision-making. This editorial article highlights the research works published in the special issue of Computational Methods for Physiological Signal Processing and Data Analysis. The context introduces three deep learning applications in epileptic seizure detection, human exercise intensity analysis, and lung nodule CT image segmentation, respectively. The article also summarizes the research works on detection of event-related potential in the single-trial electroencephalogram (EEG) signals during the auditory tests, along with the methodology on estimating the generalized exponential distribution parameters using the simulated and real data produced under the Type I generalized progressive hybrid censoring schemes. The article concludes with perspectives and discussions on future trends in biomedical signal processing and data analysis technologies.

This study digs into the importance and usefulness of data analysis and statistical methods in making supply chain processes better and balancing supply and demand. It looks at its usefulness in many real-life situations, like figuring out the best way to use resources, making the supply chain more open, managing suppliers better, making logistics more efficient, and lowering business risks. By stressing how important it is to combine data analysis and statistical methods with supply chain process optimization, real-life business examples make the usefulness and practicality of data analysis and statistical methods stand out. At the same time, the study deeply explores the challenges and opportunities encountered in the practical application of data analysis and statistical methods. Finally, we provide an in-depth summary of the research findings and offer suggestions for future research. The goal is to provide forward-looking insights and a solid theoretical foundation for the ongoing improvement and innovation of supply chain management, as well as help businesses better adapt to the tough international market competition environment.

Due to the fierce competition in the market, customers are often faced with multiple choices when choosing products and services. So many industries, including banking, are now facing the problem of how to address customer churn. At the same time, in order to improve the quality of service for users, banks and other institutions need to conduct in-depth research on the characteristics of customers. This paper provides solutions to the above two problems by using data analysis and mining technology and machine learning technology in artificial intelligence. The study provides an in-depth exploration of cus-tomer churn likelihood by analyzing customer behavior and characteristics. This study used data analysis methods such as Chi-square test, Mann Whitney U test, linear regres-sion, and machine learning methods such as logistic regression, random forest, and XGBoost. This research uses public datasets on Kaggle. This study uses data analysis techniques to provide recommendations on how the banking industry should improve service quality, and establishes 6 models with better performance to predict customer churn. In addition, this paper also uses a variety of evaluation indicators to compare the model performance, and selects the random forest model with high predictive ability as the most suitable model. In addition, the order of importance of the factors responsible for customer churn was successfully derived.

ABSTRACTMore than 50 years ago, John Tukey called for a reformation of academic statistics. In “The Future of Data Analysis,” he pointed to the existence of an as-yet unrecognized science, whose subject of interest was learning from data, or “data analysis.” Ten to 20 years ago, John Chambers, Jeff Wu, Bill Cleveland, and Leo Breiman independently once again urged academic statistics to expand its boundaries beyond the classical domain of theoretical statistics; Chambers called for more emphasis on data preparation and presentation rather than statistical modeling; and Breiman called for emphasis on prediction rather than inference. Cleveland and Wu even suggested the catchy name “data science” for this envisioned field. A recent and growing phenomenon has been the emergence of “data science” programs at major universities, including UC Berkeley, NYU, MIT, and most prominently, the University of Michigan, which in September 2015 announced a $100M “Data Science Initiative” that aims to hire 35 new faculty. Teaching in these new programs has significant overlap in curricular subject matter with traditional statistics courses; yet many academic statisticians perceive the new programs as “cultural appropriation.” This article reviews some ingredients of the current “data science moment,” including recent commentary about data science in the popular media, and about how/whether data science is really different from statistics. The now-contemplated field of data science amounts to a superset of the fields of statistics and machine learning, which adds some technology for “scaling up” to “big data.” This chosen superset is motivated by commercial rather than intellectual developments. Choosing in this way is likely to miss out on the really important intellectual event of the next 50 years. Because all of science itself will soon become data that can be mined, the imminent revolution in data science is not about mere “scaling up,” but instead the emergence of scientific studies of data analysis science-wide. In the future, we will be able to predict how a proposal to change data analysis workflows would impact the validity of data analysis across all of science, even predicting the impacts field-by-field. Drawing on work by Tukey, Cleveland, Chambers, and Breiman, I present a vision of data science based on the activities of people who are “learning from data,” and I describe an academic field dedicated to improving that activity in an evidence-based manner. This new field is a better academic enlargement of statistics and machine learning than today’s data science initiatives, while being able to accommodate the same short-term goals. Based on a presentation at the Tukey Centennial Workshop, Princeton, NJ, September 18, 2015.

Data Science is associated with new discoveries, the discovery of value from the data. It is a practice of deriving insights and developing business strategies through transformation of data in to useful information. It has been evaluated as a scientific field and research evolution in disciplines like statistics, computing science, intelligence science, and practical transformation in the domains like science, engineering, public sector, business and lifestyle. The field encompasses the larger areas of artificial intelligence, data analytics, machine learning, pattern recognition, natural language understanding, and big data manipulation. It also tackles related new scientific challenges, ranging from data capture, creation, storage, retrieval, sharing, analysis, optimization, and visualization, to integrative analysis across heterogeneous and interdependent complex resources for better decision-making, collaboration, and, ultimately, value creation. In this paper we entitled epicycles of analysis, formal modeling, from data analysis to data science, data analytics -A keystone of data science, The Big data is not a single technology but an amalgamation of old and new technologies that assistance companies gain actionable awareness. The big data is vital because it manages, store and manipulates large amount of data at the desirable speed and time. Big data addresses detached requirements, in other words the amalgamate of multiple un-associated datasets, processing of large amounts of amorphous data and harvesting of unseen information in a time-sensitive generation.
 As businesses struggle to stay up with changing market requirements, some companies are finding creative ways to use Big Data to their growing business needs and increasingly complex problems. As organizations evolve their processes and see the opportunities that Big Data can provide, they struggle to beyond traditional Business Intelligence activities, like using data to populate reports and dashboards, and move toward Data Science- driven projects that plan to answer more open-ended and sophisticated questions.
 Although some organizations are fortunate to have data scientists, most are not, because there is a growing talent gap that makes finding and hiring data scientists in a timely manner is difficult. This paper, aimed to demonstrate a close view about Data science, big data, including big data concepts like data storage, data processing, and data analysis of these technological developments, we also provide brief description about big data analytics and its characteristics , data structures, data analytics life cycle, emphasizes critical points on these issues.

Social media has become an important reference for information during the last few decades. They have been able to be effective in various fields such as business, entertainment, science, crisis management, politics, etc. For this reason, a social media analysis has become very important for researchers and large companies. The widespread use of social media leads to a complex problem called "accumulation of data". Many data science specialists seek to analyze this data in order to identify the behavioral characteristics of users, analyze interests and needs, and improve marketing processes. Different social media platforms have the ability to use all kinds of media, including text data, video, video, audio, and location information, etc. Therefore, data analysis in social networks is very important. In this research, the concepts and applications of data analysis in social networks will be investigated.

Chapter 6 - Naturalistic Driving Studies and Data Coding and Analysis Techniques

McDonald, Erickson, and McDonald (2000) and Murtaugh (2000) have presented two interesting views of analysis and interpretation of data from before-after control-impact (BACI) studies. In my opinion, the general message they convey to the reader is that analysis of data from a BACI design, in combination with professional expertise, can add useful evidence as long as one is prudent about the resulting interpretations. Whether scientists who analyze BACI data will indeed use caution in interpretation, particularly in making statements regarding cause and effect, remains to be seen. McDonald et al.'s (2000) study is application specific, taking the reader through an analysis of bird count data from the Exxon Valdez oil spill (EVOS). The authors prudently phrase their interpretation in terms of a weight of evidence argument rather than the much more difficult one of causation. I see the primary usefulness of this paper as that of encouraging scientists to use more complex models to realistically describe count data through time. (Using these models in data analysis has been considerably enhanced by the increasing availability of various types of computer software.) Particularly with biological data, a conventional first step is to try an additive model on untransformed data or an additive model on log-transformed data with the hope that the log transform restores the assumptions of variance homogeneity and normally distributed error terms. The presence of data recorded on the same units through time necessitates that the Huynh-Feldt condition (also known as sphericity) holds in order to use a conventional repeated measures approach. In case this sphericity condition does not hold, there are references to make conservative adjustments to the usual F-tests. (I do wonder just how many people would take the trouble to test these assumptions and do the proper corrections if such corrections were not already built into the

Location‐only and use‐availability data: analysis methods converge