- Research Article
7
- 10.1002/cpe.4517
Cloud computing and big data: Technologies and applications
- May 20, 2018
- Concurrency and Computation: Practice and Experience
- Mostapha Zbakh + 3 more +3
Cloud computing and big data: Technologies and applications
The objective of this book is to introduce the basic concepts of big data computing and then to describe the total solution of big data problems using HPCC, an open-source computing platform. The book comprises 15 chapters broken into three parts. The first part, Big Data Technologies, includes introductions to big data concepts and techniques; big data analytics; and visualization and learning techniques. The second part, LexisNexis Risk Solution to Big Data, focuses on specific technologies and techniques developed at LexisNexis to solve critical problems that use big data analytics. It covers the open source High Performance Computing Cluster (HPCC Systems) platform and its architecture, as well as parallel data languages ECL and KEL, developed to effectively solve big data problems. The third part, Big Data Applications, describes various data intensive applications solved on HPCC Systems. It includes applications such as cyber security, social network analytics including fraud, Ebola spread modeling using big data analytics, unsupervised learning, and image classification. The book is intended for a wide variety of people including researchers, scientists, programmers, engineers, designers, developers, educators, and students. This book can also be beneficial for business managers, entrepreneurs, and investors.
Cloud computing and big data: Technologies and applications
Cloud computing and big data: Technologies and applications
Application of Big Data in the Promotion of Fast Selling Products
In the traditional marketing mode of fast-selling products, the sales mode of physical stores is adopted. However, in the background of the current big data era, it is a trend to optimize the promotion form by applying big data technology. Therefore, this paper puts forward the application research of big data in the promotion of fast-selling products. This paper makes an indepth study on big data technology and commodity marketing. It is believed that there is a lot of information hidden behind the information data, and the application of big data technology pays more attention to consumer behavior than before. In this paper, according to the characteristics of fast-selling products promotion activities, combined with big data technology, the effect evaluation model are established, which can better solve the shortcomings of traditional promotion activities which are difficult to evaluate. And according to the actual needs of the promotion of fast-selling products, targeted optimization is carried out. In order to further verify the data analysis ability of big data technology in the promotion of fast selling products, this paper establishes the corresponding investigation experiment. The experimental data show that big data technology can better analyze the actual effect of various promotion tools and promotion strategies, provide technical support for enterprises before and after the promotion of fast-selling products, and facilitate enterprises to adjust strategies and summarize experience. The analysis shows that big data technology has brought a variety of convenience to the promotion activities, which not only broadens the sales channels, but also provides a new basis for the decision-making of enterprises.
Read moreResearch and Thinking on the Application of Big Data in Budget Performance Management
At present, the budget performance reform has become a breakthrough in the profound reform of government governance, and is an inevitable requirement for further promoting the modernization of the national governance system and governance capacity. Big data technology can improve the quality of budget performance management from four ways: the scientific review of budget performance objectives, the accuracy of performance evaluation, the online real-time monitoring of performance operation and the visualization of performance information. Some local financial departments in China have explored the introduction of big data in the aspects of pre-performance review, database construction, performance operation and target monitoring to carry out budget performance management. Improving the legislation on "budget performance management + big data", promoting the interconnection of cross-departmental data, comprehensively applying multiple data analysis methods, and establishing a composite talent training mechanism are the directions for further improving the quality of budget performance management with big data technology.
Read moreResearch on the Performance Evaluation Path of State-owned Asset Management in Colleges and Universities Based on Big Data
With the rapid development of information technology, the application of big data has gradually deepened in various fields, bringing new opportunities and challenges to the management of state-owned assets in colleges and universities. Currently, colleges and universities generally face problems in the performance evaluation of asset management, such as insufficient attention, lack of indicators, low efficiency caused by reliance on manual operations, and distortion of basic data. These issues make it difficult to meet the requirements of modern governance. Therefore, it is necessary to rely on big data technology to build a performance evaluation platform that integrates data collection, processing, analysis, and feedback, establish fair, transparent, and traceable evaluation principles, and form a closed-loop management process. Based on this, this paper analyzes the necessity of applying "big data" management to state-owned assets in colleges and universities, identifies the problems existing in the performance evaluation of state-owned asset management in colleges and universities under the thinking and technology of big data, and expounds the path of constructing the performance evaluation system for state-owned asset management in colleges and universities from the perspective of "big data", so as to realize the intelligent management of assets in colleges and universities.
Read moreMachine Learning in Big Data
<p>Machine learning is an artificial intelligence method of discovering knowledge for making intelligent decisions. Big Data has great impacts on scientific discoveries and value creation. This paper introduces methods in machine learning, main technologies in Big Data, and some applications of machine learning in Big Data. Challenges of machine learning applications in Big Data are discussed. Some new methods and technology progress of machine learning in Big Data are also presented.</p>
Read moreResearch on the application of big data in regional industrial supply chain
Taking Artemisia industry, the pillar industry of Qichun County as an example, this paper studies the role of big data in the construction of regional industrial supply chain, and uses big data technology to analyze the application of modern information technology in the industry, such as the system of county industrial supply chain, product structure and e-commerce. The main problems of the industrial supply chain in Qichun County are: insufficient application of big data technology in the whole supply chain, low application of big data technology in the traditional manufacturing industry, low application of big data in the tourism and health industry, and low intelligence of the logistics system. It is proposed to use big data and other information technology to optimise the supply chain upgrade of the county industry and establish a complete industrial chain of smart manufacturing, smart logistics, smart tourism and artificial intelligence + big health.
Read moreResearch and Application of Big Data in E-government
The wide use of modern electronic information technology, represented by big data technology, has brought people into the era of big data. Big data is affecting our work, life, even economy and the development of the society. Nowadays, the application of big data technology in e-government speeds up the development of e-government, which is important for the overall construction of e-government. This paper summarizes the application of the big data, introduces the application of big data technology in e-government. Then this paper analyzes the current situation and problems of e-government development. Finally this paper proposes the solutions of big data platform in e-government.
Read moreChapter 28 - COVID-19 and big data technologies: Experience in China
Chapter 28 - COVID-19 and big data technologies: Experience in China
ECL-watch: A big data application performance tuning tool in the HPCC systems platform
The proliferation of Big Data processing environments such as Hadoop, Apache Spark, and HPCC Systems is driving the development of performance analysis tools in these distributed systems. The goal is to achieve high performance through the optimization of Big Data applications. However, tuning performance in a fine-grained manner is quite challenging due to the high complexity and massive size of the distributed systems. ECL-Watch is a data-flow based fine-grained comprehensive Big Data performance analysis tool utilizing the high level declarative dataflow programming language ECL in HPCC Systems. As a case study, we implement and optimize the Yinyang K-Means machine learning algorithm in ECL in HPCC Systems. The experimental results show that the performance of the native ECL version of the Yinyang K-Means algorithm increased significantly after tuning: from being about three times slower than the standard K-Means implementation in ECL, to become roughly 15% faster than standard K-Means.
Read moreThe application of big data in the development of red cultural and educational resources
The purpose of this study is to explore how big data technology can promote the development and utilization of red cultural and educational resources. Through the systematic analysis of the role of big data in the mining, sorting, analysis and dissemination of red cultural resources, a set of red cultural and educational resources evaluation model based on big data technology is proposed. The model can comprehensively consider historical data, user behavior data and social feedback data, and improve the scientific and accurate resource development. Through the construction of mathematical model, the red cultural resources are evaluated and predicted systematically. The model verification results show that big data technology can not only improve the development efficiency of red cultural resources, but also put forward personalized resource development strategies according to the cultural characteristics of different regions.
Read moreApplication of Big Data and Artificial Intelligence in COVID-19 Prevention, Diagnosis, Treatment and Management Decisions in China
COVID-19, caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), spread rapidly and affected most of the world since its outbreak in Wuhan, China, which presents a major challenge to the emergency response mechanism for sudden public health events and epidemic prevention and control in all countries. In the face of the severe situation of epidemic prevention and control and the arduous task of social management, the tremendous power of science and technology in prevention and control has emerged. The new generation of information technology, represented by big data and artificial intelligence (AI) technology, has been widely used in the prevention, diagnosis, treatment and management of COVID-19 as an important basic support. Although the technology has developed, there are still challenges with respect to epidemic surveillance, accurate prevention and control, effective diagnosis and treatment, and timely judgement. The prevention and control of sudden infectious diseases usually depend on the control of infection sources, interruption of transmission channels and vaccine development. Big data and AI are effective technologies to identify the source of infection and have an irreplaceable role in distinguishing close contacts and suspicious populations. Advanced computational analysis is beneficial to accelerate the speed of vaccine research and development and to improve the quality of vaccines. AI provides support in automatically processing relevant data from medical images and clinical features, tests and examination findings; predicting disease progression and prognosis; and even recommending treatment plans and strategies. This paper reviews the application of big data and AI in the COVID-19 prevention, diagnosis, treatment and management decisions in China to explain how to apply big data and AI technology to address the common problems in the COVID-19 pandemic. Although the findings regarding the application of big data and AI technologies in sudden public health events lack validation of repeatability and universality, current studies in China have shown that the application of big data and AI is feasible in response to the COVID-19 pandemic. These studies concluded that the application of big data and AI technology can contribute to prevention, diagnosis, treatment and management decision making regarding sudden public health events in the future.
Read moreResearch on Computer Network Information Security and Its Protection Strategy Based on Secure Big Data
The popularity of Internet technology is getting higher and higher. While computer information network brings opportunities for social progress and industry development, it also brings many problems. In the era when big data technology is widely used, computer network information security protection is particularly important. The application of big data in different fields not only helps to improve the work efficiency in various fields, but also brings challenges to the protection technology of computer network information security. The popularization of computer information networks has created new ways and means for information crimes. The theft of network information and the theft and reselling of personal network information have seriously affected the social security. In the era of big data, it is particularly urgent and important to strengthen the protection of computer network information and protect computer network information. This paper analyzes the factors that affect computer network information security in the era of big data, and further puts forward the protection strategy of computer network information security in the era of big data era.
Read moreTechnology Roadmap Development for Big Data Healthcare Applications
Big data applications indicate a wide range of opportunities to improve the overall quality and efficiency of healthcare delivery. The highest impact of big data applications is expected when data from various healthcare areas, such as clinical, administrative, financial, or outcome data, can be integrated. However, as of today, the realization of big data healthcare applications aggregating various kinds of data sources is still lacking behind. In order to foster the implementation of comprehensive big data applications, a clear understanding of short-term and long-term goals of envisioned big data scenarios is needed to forecast which emerging big data technologies are needed at what point in time. The contribution of this paper is to introduce the development of a technology roadmap for big data technologies in the healthcare domain. Beside the description of user needs and the technologies needed in order to satisfy those needs, the technology roadmap provides a basis to forecast technology developments and, thus, guidance in planning and coordinating technology developments accordingly.
Read moreBig Data Characteristics, Challenges, Architectures, Analytics and Applications: A Review
Big data has evolved as a most challenging area in scientific study and research. It has drawn much attention during the last few years. It influences our modern society, business, government, healthcare, research and almost every discipline. In this data-driven era, where data is continuously acquired from a verity of sources for different purposes, the ability to make timely decisions based on available data is a very critical task. The massive data size, variety, velocity, accuracy and high dimensionality presents new challenges to big data. This paper attempts to present some challenges of big data. In addition, a study on the conceptual design of big data architecture presented on specific big data applications. In this paper, we presented research work on big data analytics techniques in the areas of text, sentiment, video, social media and predictive analytics. A comparative analysis on selected big data applications has been presented in detail.
Read morePotentials of big data for governmental services
Big data is not a new type of data. It rather describes a set of characteristics and provides technologies and technics to better address specific challenges [1]. Often these characteristics are described as the 3V's of big data: volume, variety and velocity. In 2012 Gartner presented a report about potentials of big data in different vertical industries [2]. According Gartner, governments are dealing with increasing volumes of data that have high variety of structures. Governments additionally have high potential for improving their utilization of so-called dark data; data that is available somewhere in the system but not actively used. The continuous digitalization of governmental services and communication with citizens will further accelerate the growth of data and big data technologies will play in future an important role for efficient and customer centric services. IBM believes in new and smarter big data application for governments and established in 2011 a new research center in Dublin Ireland that focuses on improving ecosystems of cities [3]. In a particular case, IBM created a model that used 2.5 billion call records to improve public transportations in a city at the Ivory Coast in Africa [5]. The outcome of this model has been a 10 percent time savings for commuters in this area. With this project IBM won the data for development award from Orange in 2013 [4]. It shows an interesting use case on how you can use data insights to improve social environments and public services. Next to smarter public transportation IBM also works on improving water quality and optimizing energy costs in urban regions. The investment of IBM in such topics shows a commitment and strong belief of an industry giant in the future of big data applications in governmental services. In my speech I will provide an introduction and overview of big data and its underlying technologies. I then will discuss different areas of governmental services in which I see high potentials of big data application. In conclusion I then present use cases that demonstrate big data application for governmental and health care services.
Read more