The Art of Data Science and Big Data Analytics: Inspecting and Transforming Data

Nowadays a lot of massive data is stored and typically, the data itself contains a lot of non-trivial but useful information. Data mining techniques can be used to discover this information which can help the companies for decision-making. However, in real life applications, data is massive and is stored over distributed sites. One of my major research topics is to protect privacy over this kind of data. Previously, the important characteristics, issues and challenges related to management of the large amount of data has been explored. Various open source data analytics frameworks that deal with large amount of Data analytics workloads have been discussed. Comparative study between the given frameworks and suitability of the same has been proposed. Digital universe is flooded with huge amount of data generated by number of users worldwide. These data are of diverse in nature, come from various sources and in many forms. To keep with the desire to store and analyze ever larger volumes of complex data, relational databases vendors have delivered specialized analytical platforms that come in many shapes and sizes from software only to analytical services that run in third party hosted environments. In addition new technologies have emerged to address exploding volumes of complex data, including web traffic, social media content and machine generated data including sensor data, global positioning system data.Big data is defined as large amount of data which requires new technologies and architectures so that it becomes possible to extract value from it by capturing and analysis process. Due to such large size of data it becomes very difficult to perform effective analysis using the existing traditional techniques. Big data has become a prominent research field, especially when it comes to decision making and data analysis. However, Big data due to its various properties like volume, velocity, variety, variability, value and complexity put forward many challenges. Since Big data is a recent upcoming technology in the market which can bring huge benefits to the business organizations, it becomes necessary that various challenges and issues associated in bringing and adapting to this technology are brought into light. Another challenge is that data collection may not have enough accuracy which will lead to a non-consistent analysis that can critically affect the decision based on this analysis. Moreover, it is clearly apparent that organizations need to employ data-driven decision making to gain competitive advantage. Processing, integrating and interacting with more data should make it better data, providing both more panoramic and more granular views to aid strategic decision making. This is made possible via Big Data exploiting affordable and usable Computational and Storage Resources. Many offerings are based on the Map-Reduce and Hadoop paradigms and most focus solely on the analytical side. Nonetheless, in many respects it remains unclear what Big Data actually is; current offerings appear as isolated silos that are difficult to integrate and/or make it difficult to better utilize existing data and systems. Since data is growing at a huge speed making it difficult to handle such large amount of data (Exabyte). The main difficulty in handling such large amount of data is because that the volume is increasing rapidly in comparison to the computing resources. The Big data term which is being used now a days is kind of misnomer as it points out only the size of the data not putting too much of attention to its other existing properties.If data is to be used to make accurate decisions in time it becomes necessary that it should be available in accurate, complete and timely manner. This makes the data management and governance process bit complex adding the necessity to make Data open and make it available to government agencies in standardized manner with standardized APIs, metadata and formats thus leading to better decision making, business intelligence and productivity improvements.This paper presents a discussion and evaluation for the most prominent techniques used in the processes of data collection and analysis in order to identify the privacy defects in them that affects the accuracy of big data. Depending on the results of this analysis, recommendations were provided for improving data collection and analysis techniques that will help to avoid if not all then most of the problems facing the use of big data in decision making. Keywords: Big Data, Big Data Challenges, Big Data Accuracy, Big Data Collection, Big Data Analytics.

Data Scientist - The New ProfessionData scientist is probably the most trendy job in Information Technology (IT) nowadays. This new profession emerged with the Big Data wave. Even though there is no such thing like an exact job profile, we expect that the data scientist can handle all the Big Data challenges that are novel to us. Without being a magician she or he shall help to deliver us all the magic Big Data promises. The data scientist capitalizes on unstructured data without taking the roles of a programmer, database expert, statistician, or content manager. All these professions are around for decades. So, why invent a new one?More than anything, what data scientists do is make discoveries while swimming in data [...] they are able to bring structure to large quantities of formless data and make analysis possible. [3] They sketch, orchestrate, and control the discovery process. The leading paradigm in this process it to find information that meets a certain need or provides an answer for a certain problem. need to avoid the temptation of following a data-driven approach instead of a problem-driven one. [5] The data scientist has to develop an idea about the required information that meets our information need. We can expect that data scientists have a deep understanding of the foundations of both, the nature of information and the domain. They follow a mental model on the information demand that abstractly reflects the facts they expect to encounter in data and the way to detect and present them to us. We expect the data scientist to discover novel information that may provide us with new insights. And we want these insights to be true. It is thus part of the data scientist's responsibility to make sure that the discovered information is not only novel but also trustworthy. The data scientists cannot prove data analysis models. That exceeds their capabilities. We cannot hold them liable for information that eventually turns out to be wrong. Nevertheless their skills should include a sound sensation of plausibility that helps them to raise doubts and to prompt a closer look when the results of data analysis seem too questionable to them. However, separating questionable results from plausible ones is a task that is far from being trivial.Separating Data Science from Data FictionWhen driving a car we often encounter these nice roadside signs that sometimes make us realize that we're driving too fast. Furthermore, on some occasions there is official personnel not far from these signs measuring our speed, explaining to us our traffic infraction face-to-face and documenting it on a speeding ticket. Doesn't this sound a bit outdated? There are enough sensors in a car that measure location, speed, time, and more. Even if the car's sensors don't measure those, the driver's cellphone can do it. Combining this sensor information with the cartographic data about roads and speed limits we can easily imagine that, by the end of day, the car or the phone exactly knows every infraction we committed and can automatically trigger the issuing of speeding tickets. Aren't there just legal aspects that hamper this technically feasible scenario making its way into reality?One trait of Big Data is the availability of sensor data and their combination with already available data generating new information, like the detection of traffic infraction. We can extrapolate this scenario in many directions, including more types of infraction, driving for an irresponsibly long time without a substantial break, or detecting patterns of aggressive driving. We can extend it also towards future possibilities if we think about sensor information from smart watches indicating a possibly problematic pulse rate or from the car's air control sensor that the driver may drive under the influence of alcohol.Much like in Data Mining, the strength of Big Data originates from the combination of facts producing new information. …

The unprecedented proliferation of miniaturized sensors and intelligent communication, computing, and control technologies have paved the way for the development of the Industrial Internet of Things. The IIoT incorporates machine learning and massively parallel distributed systems such as clouds, clusters, and grids for big data storage, processing, and analytics. In IIoT, end devices continuously generate and transmit data streams, resulting in increased network traffic between device-cloud communication. Moreover, it increases in-network data transmissions. requiring additional efforts for big data processing, management, and analytics. To cope with these engendered issues, this article first introduces a novel concentric computing model (CCM) paradigm composed of sensing systems, outer and inner gateway processors, and central processors (outer and inner) for the deployment of big data analytics applications in IIoT. Second, we investigate, highlight, and report recent research efforts directed at the IIoT paradigm with respect to big data analytics. Third, we identify and discuss indispensable challenges that remain to be addressed for employing CCM in the IIoT paradigm. Lastly, we provide several future research directions (e.g., real-time data analytics, data integration, transmission of meaningful data, edge analytics, real-time fusion of streaming data, and security and privacy).

Big data and data analytics are currently the buzzwords in both academia and industry to become data driven. Big data has been the trending topic in the accounting industry also. Big data and data analytics will have an important impact on accounting and accountants. Big data will improve the quality of accounting information and the accounting profession will continue to provide real-time and dynamic information to assist in decision-making. The purpose of this research is to investigate the impact of big data and big data analytics in accounting. Data analytics is one of the most recent developments in the accounting context. This study is qualitative in nature and adopted a literature review methodology to gain a better understanding of the study area. This literature review seeks to provide a description and evaluation of the impact of big data analytics on accounting. This research found that big data presents great opportunities for decision making in accounting and risks analysis, which indicated that companies could improve their performance, measure performance, manage risks and allow effective real-time decision-making with data analytics. This research revealed that accountants can create more value in a world of big data analytics and encourages accountants to get started with big data to find answers to risks in business operations as well as understand financial performance. It shows that relying on big data analytics will open new possibilities for accountants. This study contributes to the research literature in the area of big data analytics on accounting. The limitations of this study are that it utilizes few recent peer reviewed articles in the general accounting practice, therefore not exhaustive in describing how big data and big data analytics impacts accounting.

Large amounts of data are generated daily, according to the wide usage of social media websites and scientific data. These data need to be stored and analyzed to help decision makers but the traditional database concepts are insufficient. Data warehouse and OLAP are useful technologies in the storage and analysis of big data. Using MapReduce will help to save processing time, using cloud computing will help in saving resources and storage. In this paper, we propose a system that integrates the OLAP and MapReduce over cloud (considering workload balance) in order to enhance the performance of query processing over big data. The proposed system is applied to large amounts of data stored in cubes located in a Peer-to-peer cloud; this process is done using an allocation approach to save resources and query processing times. The proposed system achieves enhancements as time saving in query processing and in resources usage.

The increasing power of computers, the availability of cheap hardware, and the digitization process overall have made technology an essential part of our daily routines. The massive adoption of technology has helped generate massive amounts of data, often referred to as big data. Such massive data cover all aspects of our life, from dealing with government services (e.g. getting a driving license) to recording sleep patterns with our smartphones.[1] Big data is conceptually more than “lots of data.” Big data has several characteristics in addition to huge volumes of data: Rapidly generated in real time or close to, comprehensive, relational, diverse, and fine grained. The concept of big data has been associated with rapidly advancing methods that enable novel ways of analysis that were impossible before. Machine learning, artificial intelligence, data science, and network science are just examples of such methods.[2]Having access to data, powerful methods of analysis have kindled a wave of optimism that data can solve the world’s most unyielding problems. There are, of course, several examples that have stimulated such big hopes. Many companies were able to harness the potentials of big data for their competitive advantage expanding reach and customer base. Several outstanding human innovations give compelling evidence of the power of big data and analytics. Take for example the several online translation services that can translate between several languages without knowing any and with impeccable quality.[1,2]Such advances have led to the upsurge of data-driven approaches to the degree that many claim that such data-driven methods are a replacement for scientific theory and traditional hypothesis-driven methods. In a much-discussed article, Chris Anderson argues that “we can stop looking for models. We can analyze the data without hypotheses about what it might show. We can throw the numbers into the biggest computing clusters the world has ever seen and let statistical algorithms find patterns where science cannot.[3]” While in many instances, this has proven useful, in many others, several spurious findings were obtained.[2] Thus, this argument has been largely debated as misleading and dangerous.A perfect scenario is to use big data to conduct robust analysis with maximized statistical power. Such analyses have been used to develop, for example, product recommender systems and academic dropout prediction systems. However, the data used by such systems are obtained from many individuals in most of the cases; or what is known as variable or group-based analysis. Typically, a large amount of data are collected from a sample that is hypothesized to be representative of the population. The analysis of such sample aims at devising universal laws that apply to all. For instance, the analysis of hypertensive patients would give an idea about the typical presentation of the disease, the typical prognosis, and response to treatment.[4,5] However, such generalization is a mere over-simplification and reductionist.Group-based analysis has several fundamental problems that pose considerable threat to the validity of results. The first problem is that group-based analysis assumes that the population is a homogenous group of individuals with no or little variations or heterogeneity. Such assumption is very unlikely to hold. Human physiology, pathology, and behavior are heterogeneous with obvious diversity and distinct trajectories. For example, patients with hypertension can have several different pathways of their clinical progression, response to treatment, complications, and disease outcome. Such heterogeneity has been poorly addressed in previous research.[4]The other problem with group-based analysis is that it assumes that the analyzed process is ergodic, that is, stable overtime, while human social and psychological processes have considerable time variations. Therefore, group-based analysis which lacks the dimension of time can poorly represent these processes. Such deficiencies make results of group-based analysis less likely to generalize to individuals.[4] The frustration resulting from lack of generalizability of group-based insights is well-recognized. Several threadsof research are emerging to address such challenges. Precision medicine, person-based research, and idiography are among the most promising. The main principle behind such methods is to devote more attention to the individual to identify the characteristics that define how their bodies work. Idiography, in particular, aims at collecting data from the individual over multiple times. By having several observations about an individual, researchers have enough data about a person that they can create statistical modelsbased on the person’s data. Analysis of single person data offers a laser-sharp view of a person’s internal conditions. This approach is made possible by the availability of multiple data sources as well as devices that can facilitate data collection, for example, smartphones. In summary, a paradigm shift is happening, where the focus is on collecting big data from individuals. Suchself-big data is more relevant, more representative, and offers a better opportunity for personalized insights. Such paradigm shift is moving from the big data of many others to self-big data.

Photo by Josh Riemer on Unsplash
 Introduction
 With the rapid advancements in neurotechnological machinery and improved analytical insights from machine learning in neuroscience, the availability of big brain data has increased tremendously. Neurological health research is done using digitized brain data.[1] There must be adequate data governance to secure the privacy of subjects participating in brain research and treatments. If not properly regulated, the research methods could lead to significant breaches of the subject’s autonomy and privacy. This paper will address the necessity for neuroprotection laws, which effectively govern the use of big brain data to ensure respect for patient privacy and autonomy.
 Background
 Artificial intelligence and machine learning can be integrated with neuroscience big brain data to drive research studies. This integrative technology allows patterns of electrical activity in neurons to be studied in detail.[2]Specifically, it uses a robotic system which can reason, plan, and exhibit biologically intelligent behavior. Machine learning is a method of computer programming where the code can adapt its behavior based on big brain data.[3] The big brain data is the collection of large amounts of information for the purpose of deciphering patterns through computer analysis using machine learning.[4] The information that these technologies provide is extensive enough to allow a researcher to read a patient’s mind. AI and machine learning technologies work by finding the underlying structure of brain data, which is then described by patterns known as latent factors, eventually resulting in an understanding of the brain’s temporal dynamics.[5]
 Through these technologies, researchers are able to decipher how the human brain computes its performances and thoughts. However, due to the extensive and complex nature of the data processed through AI and machine learning, researchers may gain access to personal information a patient may not wish to reveal. From a bioethical lens, tensions arise in the realm of patient autonomy. Patients are not able to control the transmission of data from their brains that is analyzed by researchers. Governing brain data through laws may enhance the extent of patient privacy in the case where brain data is being used through AI technologies.[6] A responsible approach to governing brain data would require a sophisticated legal structure.
 Analysis
 Impact on Patient Autonomy and Privacy 
 In research pertaining to big brain data, the consent forms do not fully cover the vast amounts of information that is collected. According to research, personal data has become the most sought out commodity to provide content to corporations and the web-based service industry. Unfortunately, data leaks that release private information frequently occur.[7] The storage of an individual’s data on technologies accessible on the internet during research studies makes it vulnerable to leaks, jeopardizing an individual’s privacy. These data leaks may cause the patient to be identified easily, as the degree of information provided by AI technologies are personalized and may be decoded through brain fingerprinting methods.[8]
 There has been an extensive growth in the development and use of AI. It is efficient in providing information to radiologists who diagnose various diseases including brain cancer and psychiatric disease, and AI assists in the delivery of telemedicine.[9] However, the ethical pitfall of reduced patient autonomy must be addressed by analyzing current AI technologies and creating more options for patient preference in how the data may be used. For instance, facial recognition technology[10] commonly used in health care produces more information than listed in common consent forms, threatening to undermine informed consent. Facial recognition software collects extensive data and may disclose more information than a person would prefer to provide despite being a useful tool for diagnosing medical and genetic conditions.[11] In addition, people may not be aware that their images are being used to generate more clinical data for other purposes. It is difficult to guarantee the data is anonymized. Consent requirements must include informing people about the complexity of the potential uses of the data; software developers should maximize patient privacy.[12] Furthermore, there is a “human element” in the use of AI technologies as medical providers control the use and the extent to which data is captured or accessed through the AI technologies.[13] People must understand the scope of the technology and have clear communication with the physician or health care provider about how the medical information will be used. 
 Existing Laws for Brain Data Governance 
 A strict system of defined legal responsibilities of medical providers will ensure a higher degree of patient privacy and autonomy when AI technologies and data from machine learning are used. Governing specific algorithmic data is crucial in safeguarding a patient’s privacy and developing a gold standard treatment protocol following the procurement of the information.[14] Certain AI technologies provide more data than others, and legal boundaries should be established to ensure strong performance, quality control, and scope for patient privacy and autonomy. For instance, currently AI technologies are being used in the realm of intensive neurological care. However, there is a significant level of patient uncertainty about how much control patients have over the data’s uses.[15] Calibrated legal and ethical standards will allow important brain data to be securely governed and monitored.
 Once brain signals are recorded and processed from one individual, the data may be merged with other data in Brain Computer Interface Technology (BCI).[16] To ensure a right and ability to retrieve personal data or pull it from the collection, specific regulations for varying types of data are needed.[17] The importance of consent and patient privacy must be considered through giving patients a transparent view of how brain data is governed.[18] The legal system must address discriminatory issues and risks to patients whose data is used in studies. Laws like the General Data Protection Regulation (GDPR) and the California Consumer Privacy Protection Act (CCPA) can serve as effective models to protect aggregated data. These laws govern consumer information and ensure the compliance when personal data is collected.[19] California voters recently approved expansion of the CCPA to health data. The Washington Privacy Act, which would have provided rights to access, change, and withdraw personal data, failed to pass. Other states should improve privacy as well,[20] although a federal bill would be preferable. Scientists at the Heidelberg Academy of Sciences argue for data security to be governed in a manner that balances patient privacy and autonomy with the commercial interests of researchers.[21] The balance could be achieved through privacy protections like those in the Washington Privacy Act. Although the Health Insurance Portability and Accountability Act (HIPAA) provides an overall framework to deter the likelihood of dangers to patient protection and privacy, more thorough laws are warranted to combat pervasive data transfer and analysis that technology has brought to the health care industry.[22] Breaches of patient privacy under current HIPAA regulations include releasing patient information to a reporter without their consent and sending HIV data to a patient’s employer without consent.[23] HIPAA does not cover information being shared with outside contractors who do not have an agreement with technology companies to keep patient data confidential. HIPAA regulations also do not always address blatant breaches on patient data confidentiality.[24] Patients must be provided with methods to monitor the data being analyzed to be able to view the extent of private information being generated via AI technologies. In health research, the medical purposes of better diagnosis, earlier detection of diseases, or prevention are ethical justifications for the use of the data if it was collected with permission, the person understood and approved the uses of the data, and the data was deidentified.
 A standard governance framework is required in providing the fairest system of care to patients who allow their brain data to be examined. Informed consent in the neuroscience field could reaffirm the privacy and autonomy of patients by ensuring that they understand the type of information collected. Laws also could protect data after a patient’s death. Malpractice in the scope of brain data could give people a cause of action critical in safeguarding patient’s rights. Data breach lawsuits will become common but generally do not cover deidentified data that becomes part of big data collection. A more synchronized approach to the collection and consent process will encourage an understanding of how big data is used to diagnose and treat patients. Some altruistic people may even be more likely to consent if they know the largescale data collection is helpful to treat and diagnose people. Others should have the ability to opt out of sharing neurological data, especially when there is not certainty surrounding deidentification.[25]
 Conclusion
 Artificial intelligence and machine learning technologies have the potential to aid in the diagnosis and treatment of people globally by extracting and aggregating brain data specific to individuals. However, the secure use of the data is necessary to build trust between care providers and patients, as well as in balancing the bioethical principles of beneficence and patient autonomy. We must ensure the highest quality of care to patients, while protecting their privacy, informed consent, and clinical trust. More sophis

Data science and analytics in aviation

Background: With the development of information technology and the popularization of the Internet, a large amount of data has been generated and collected. These data typically have the characteristics of massive, diverse, and high-speed growth, and traditional data storage devices and technologies are no longer able to meet the storage and analysis needs of data. And people need more and more storage space to store personal files, photos, videos, and other data. Traditional personal computer hard drives have limited capacity and cannot meet the growing data demand. The emergence of cloud storage provides more flexible and efficient solutions for big data storage and analysis, as well as larger capacity and more convenient access methods, solving the challenges of personal storage and sharing. Method: Based on comparative analysis, explore the advantages and disadvantages of cloud storage in the fields of personal data storage and sharing, as well as big data storage and analysis. Result: Cloud storage has certain advantages and limitations in the fields of personal storage and sharing, as well as big data storage and analysis. Users need to consider these factors comprehensively based on their own needs and actual situation.

The volume of data in the world is growing very fast and generated from verity of sources like social media, sensors airline industry or scientific data in different formats. Biggest challenge is how to infer meaningful insights from such a varietyful and big data along with concern of data storage and management of fast growing data. The size of the databases used in today’s enterprises has been growing at exponential rates day by day. Hence, industries requirement to quickly process and analyze the big data volumes for business decision making and customer insights has also grown exponentially. Data pouring from various sources may be can be structured or unstructured in nature. Structured data refers to a relatively well-organized information, which can be further inserted into traditional RDBMS. As Traditional RDBMS are efficient and easy queries by simple, straightforward search algorithms or SQL queries. In contrast to structured data, unstructured data can be considered as information, which does not, comes in a pre-defined data format, well organized data storage model, or cannot be stored well into relational tables. It is assumed to be fastest growing type of data, e.g. image, sensors data, web chats, social networking messaging data, video, documents, log files, and email data. There are many techniques and software available, which can process and provide efficient storage of unstructured data and help organization to perform analytics on unstructured data. Unstructured data does not well-organized and not stored in predefined manner e.g. logs, web chats. The variety and on ordered nature of data makes storage methods and structure makes execution a time and resource-consuming affair. Advancement into technology has open floodgates to push huge volume of unstructured type of data. Multimedia data is one of the example of unstructured big data, which spans all over the Internet. This needs high execution capability to extract useful information. Rapid processing of multimedia data such as video is important for e.g. criminal investigations, surveillance monitoring, news analysis, sports analytics domain, emotion extraction, etc. Hence, analysis of multimedia data in minimum timeframe is one of the latest research areas. Therefore, we have researched techniques for analyzing unstructured data to extract meaningful information hidden in the big data. In addition, we will describe about various techniques and software used to Manage, process unstructured big data in efficient manner, and increases the performance of complexity analysis.

Geospatial Big Data analytics are changing the way that businesses operate in many industries. Although a good number of research works have reported in the literature on geospatial data analytics and real-time data processing of large spatial data streams, only a few have addressed the full geospatial big data analytics project lifecycle and geospatial data science project lifecycle. Big data analysis differs from traditional data analysis primarily due to the volume, velocity and variety characteristics of the data being processed. One of a motivation of introducing new framework is to address these big data analysis challenges. Geospatial data science projects differ from most traditional data analysis projects because they could be complex and in need of advanced technologies in comparison to the traditional data analysis projects. For this reason, it is essential to have a process to govern the project and ensure that the project participants are competent enough to carry on the process. To this end, this paper presents, new geospatial big data mining and machine learning framework for geospatial data acquisition, data fusion, data storing, managing, processing, analysing, visualising and modelling and evaluation. Having a good process for data analysis and clear guidelines for comprehensive analysis is always a plus point for any data science project. It also helps to predict required time and resources early in the process to get a clear idea of the business problem to be solved.

This chapter introduces data science with its history and importance in this modern era briefly. This chapter also elaborates the discussion by relating data science to various modern fields like big data analytics, artificial intelligence, deep learning, and machine learning. This chapter also discuss the necessary of data analytics in this big data era. This chapter also briefly introduces another emerging field, Internet of Things (IoT) and explores the contribution IoT towards big data analytics and data science in research perspective. It also briefly introduces the programming and non-programming tools used in the data science field.

Recent years have witnessed a revolution in data science and ‘big data’ into which psychiatric research has also being drawn. ‘Big data’ has been defined as data sets which are so large in size, so fast to change, and so complex in structure that traditional data processing techniques are overwhelmed [1]. The mining and exploitation of such big data resources as Electronic Healthcare Records (EHRs) present an exciting challenge to the field of psychiatric epidemiology. The number of big data projects within psychiatric research is growing, and Stewart and Davis’s literature review is, therefore, timely [2]. Technological advances in data processing and storage, computer networking, mobile technology, and data manipulation have rendered huge quantities of healthcarerelated data potentially amenable to analysis. Such studies potentially offer much larger patient numbers, wider parameters of study, and longer timescales of follow-up, than is typical of randomised controlled trials (RCTs) or cohort studies. A further advantage of this data is it often arises from naturalistic clinical settings, in terms of both clinical practice and patient health and comorbidity. Indeed, while often considered the ‘gold standard’ in medical research, RCTs do have important limitations, such as overly-strict exclusion criteria. Routine clinical data sets can, therefore, be complementary to RCT data, while also making research findings more relevant to everyday clinical practice. In addition, the quantity of clinical ‘big data’ potentially allows analysis of rarer clinical conditions, or subject areas that would be unlikely to meet ethical approval for more conventional studies (for example, medication usage in pregnancy). Big data studies provide the scale and breadth of patient numbers required for stratified, predictive, and personalised medicine research. In addition, notwithstanding many challenges of working with big data, large-scale analysis of routinely collected healthcare data sets has already demonstrated effectiveness in fields such as pharmacovigilance [3] and post-marketing clinical trials. As described in detail by Stewart and Davis, the challenging aspects of working with ‘big data’ are captured by the taxonomy of ‘Vs’—volume, velocity, and variety—first described by Laney [4] and extended since [1, 2]. There are additional issues to working with healthcare big data specifically, including that clinical administrative data are generally not collected, curated, or formatted in a manner optimised for research, and the inherent sensitivity of the data in terms of personal privacy. As the possibilities for healthcare big data expand into such areas as text mining and natural language processing of clinical records, near ‘real-time’ updating of repositories, and incorporation of new data streams from mobile and wearable technology, robust systems must be in place to channel the potential data deluge. How great a role ‘cloud’ computing and storage will have in this is an intriguing question. In healthcare big data, the benefits offered by the cloud in increased power and accessibility, and reduced cost, must be weighed against concerns over data ownership, encryption, and unauthorized access. An additional question is whether unique patient identifiers (UPIs) utilised by some national healthcare systems can be extended more This comment refers to the article available at doi:10.1007/s00127016-1266-8.

Big Data OverviewDriven by the need to generate business value, the enterprise has started to adopt Big Data as a solution, migrating from the classical databases and data stores which lack the flexibility and are not optimized enough [1].The changes in the environment make big data analytics attractive to all types of organizations, while the market conditions make it practical. The combination of simplified models for development, commoditization, a wider palette of data management tools, and low-cost utility computing has effectively lowered the barrier to entry. [2]. The concept addresses large volumes of complex data, rapid growing data sets that may come from different autonomous sources.In recent approaches, Big Data is characterized by principles known as the 4VVolume, Variety, Velocity and Veracity [3]. There are opinions about accepting other principles as Big Data characteristics, such as Value.Each day more businesses realize that Big Data is relevant as the applications generate large volumes of data generated automatically, from different data sources, centralized or autonomous. As traditional databases hit limitations when the need of analyzing this data, dedicated solutions must be considered.Important BigData Solutions:* Apache HBase/Hadoop is based on Google's BigTable distributed storage system, which runs on top of Hadoop as a distributed and scalable big data store. This means that HBase can leverage the distributed processing paradigm of the Hadoop Distributed File System (HDFS) and benefit from Hadoop's MapReduce programming model. It combines the scalability of Hadoop with real-time data access as a key/value store and deep analytic capabilities of Map Reduce [4].HBase allows to query for individual records as well as derive aggregate analytic reports across a massive amount of data. It can host large tables with billions of rows, millions of columns and run across a cluster of commodity hardware. HBase is composed of three types of servers in a master slave type of architecture. Region servers are responsible to serve data for reads and writes. When accessing data, clients communicate with HBase Region Servers directly. Region assignment, DDL (create, delete tables) operations are handled by the HBase Master process.* Apache Cassandra is a distributed database used for the administration and management of large amounts of structured data across multiple servers, while providing highly available service and no single point of failure. It provides features such as continuous availability, linear scale performance, data distribution across multiple data centers and cloud availability zones. Cassandra inherits its data architecture from Google's BigTable and it borrows its distribution mechanisms from Amazon's Dynamo.The nodes in a Cassandra cluster are completely symmetrical, all having identical responsibilities. Cassandra also employs consistent hashing to partition and replicate data. it has the capability of handling large amounts of data and thousands of concurrent users or operations per second across multiple data centers.Cassandra has a hierarchy of caching mechanisms and carefully orchestrated disk I/O which ensures speed and data safety. Write operations are sent first to a persistent commit log (ensuring a durable write), then to a write-back cache called a memTable; when the memTable fills, it is flushed to a sorted string table - SSTable - on disk. A Cassandra cluster is organized as a ring, and it uses a partitioning strategy to distribute data evenly.* Redis represents an in-memory data structure store used as a database, cache and message broker. It supports data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperlogs and geospatial indexes with radius queries. Redis stores all data in RAM, allowing lightning fast reads and writes. It runs extremely efficiently in memory and handles high-velocity data, needing simple standard servers to deliver millions of operations per second with submillisecond latency. …

Recent advances in computing have lead to diverse applications in wide domains such as cyber-forensics, data science, business analytics, business intelligence, computer security, Web technology, and Big data analytics. Data analytics refer to broad term where many of the areas such as cyber-physical systems (CPS), Internet of Things (IoT), Big data, machine learning and data mining overlap among each other. However, there are subtle differences among these areas. Data analytics form as one of the key components of these wide areas in computing. For example, in IoT the data that are collected from various devices need to analyze for inferring the outcomes of it. Hence, data analytics need to be carried on it. There are various platforms available for data analytics. In this chapter, a brief overview of the term data analytics, different types of data, different types of analytics, and the analytical architecture is first discussed. It is later followed by the different phases that are involved in the lifecycle of the data analytics project and the interconnection of Big data and Hadoop ecosystem.