ANALYSIS ON COGNITIVE SOCIAL DATA FUSION

Taking a byte out of big data

Nowadays a lot of massive data is stored and typically, the data itself contains a lot of non-trivial but useful information. Data mining techniques can be used to discover this information which can help the companies for decision-making. However, in real life applications, data is massive and is stored over distributed sites. One of my major research topics is to protect privacy over this kind of data. Previously, the important characteristics, issues and challenges related to management of the large amount of data has been explored. Various open source data analytics frameworks that deal with large amount of Data analytics workloads have been discussed. Comparative study between the given frameworks and suitability of the same has been proposed. Digital universe is flooded with huge amount of data generated by number of users worldwide. These data are of diverse in nature, come from various sources and in many forms. To keep with the desire to store and analyze ever larger volumes of complex data, relational databases vendors have delivered specialized analytical platforms that come in many shapes and sizes from software only to analytical services that run in third party hosted environments. In addition new technologies have emerged to address exploding volumes of complex data, including web traffic, social media content and machine generated data including sensor data, global positioning system data.Big data is defined as large amount of data which requires new technologies and architectures so that it becomes possible to extract value from it by capturing and analysis process. Due to such large size of data it becomes very difficult to perform effective analysis using the existing traditional techniques. Big data has become a prominent research field, especially when it comes to decision making and data analysis. However, Big data due to its various properties like volume, velocity, variety, variability, value and complexity put forward many challenges. Since Big data is a recent upcoming technology in the market which can bring huge benefits to the business organizations, it becomes necessary that various challenges and issues associated in bringing and adapting to this technology are brought into light. Another challenge is that data collection may not have enough accuracy which will lead to a non-consistent analysis that can critically affect the decision based on this analysis. Moreover, it is clearly apparent that organizations need to employ data-driven decision making to gain competitive advantage. Processing, integrating and interacting with more data should make it better data, providing both more panoramic and more granular views to aid strategic decision making. This is made possible via Big Data exploiting affordable and usable Computational and Storage Resources. Many offerings are based on the Map-Reduce and Hadoop paradigms and most focus solely on the analytical side. Nonetheless, in many respects it remains unclear what Big Data actually is; current offerings appear as isolated silos that are difficult to integrate and/or make it difficult to better utilize existing data and systems. Since data is growing at a huge speed making it difficult to handle such large amount of data (Exabyte). The main difficulty in handling such large amount of data is because that the volume is increasing rapidly in comparison to the computing resources. The Big data term which is being used now a days is kind of misnomer as it points out only the size of the data not putting too much of attention to its other existing properties.If data is to be used to make accurate decisions in time it becomes necessary that it should be available in accurate, complete and timely manner. This makes the data management and governance process bit complex adding the necessity to make Data open and make it available to government agencies in standardized manner with standardized APIs, metadata and formats thus leading to better decision making, business intelligence and productivity improvements.This paper presents a discussion and evaluation for the most prominent techniques used in the processes of data collection and analysis in order to identify the privacy defects in them that affects the accuracy of big data. Depending on the results of this analysis, recommendations were provided for improving data collection and analysis techniques that will help to avoid if not all then most of the problems facing the use of big data in decision making. Keywords: Big Data, Big Data Challenges, Big Data Accuracy, Big Data Collection, Big Data Analytics.

Cloud computing and big data: Technologies and applications

This paper adopts a 5G smart sensor network to design intelligent sensing of environmental information in aquaculture waters, and in this way, we study the significance of optimizing the intelligent sensing model. Microservice architecture and big data analysis and processing will become the innovation of technology in the new era. In this regard, it is very important to study a set of aquarium monitoring and management system based on microservice architecture, so that users can understand the changes in the environment in the aquarium, manage a large amount of historical data quickly, and help users solve existing problems. The design pattern of MVVM architecture, unified management of the life cycle, and microservice architecture make the system very easy to expand and maintain; secondly, it realizes the automatic monitoring of the aquarium environment while making multilevel storage optimization for a large amount of data and realizes fast retrieval of a large amount of data; finally, the system can make intelligent regulation in the environmental control of ornamental fish and can automatically adapt to the environmental requirements of different ornamental fish. Finally, the system can make intelligent control in the environment control of ornamental fish and can automatically adapt to the needs of different ornamental fish environments. After testing its server concurrent processing capability, communication stability, and Android performance, the system runs stably. This paper analyzes the architecture module reuse rate and protocol performance; the module reuse rate analysis results show that the various types of modules in the architecture are reused many times and have strong generality, which effectively reduces the development cycle and development cost, and the architecture can quickly and efficiently integrate hardware units such as sensors, actuators, and communication modules; the protocol performance analysis shows that the designed protocol has high communication performance and meets the architecture. The protocol performance analysis shows that the designed protocol has high communication performance and meets the requirements of the architecture in terms of the transmission rate. At the same time, the edge computing module of the architecture further shifts down the edge computing tasks and lays the hardware foundation for the subsequent implementation of sensor data anomaly detection, data encryption transmission, and other functions.

Photo by Josh Riemer on Unsplash
 Introduction
 With the rapid advancements in neurotechnological machinery and improved analytical insights from machine learning in neuroscience, the availability of big brain data has increased tremendously. Neurological health research is done using digitized brain data.[1] There must be adequate data governance to secure the privacy of subjects participating in brain research and treatments. If not properly regulated, the research methods could lead to significant breaches of the subject’s autonomy and privacy. This paper will address the necessity for neuroprotection laws, which effectively govern the use of big brain data to ensure respect for patient privacy and autonomy.
 Background
 Artificial intelligence and machine learning can be integrated with neuroscience big brain data to drive research studies. This integrative technology allows patterns of electrical activity in neurons to be studied in detail.[2]Specifically, it uses a robotic system which can reason, plan, and exhibit biologically intelligent behavior. Machine learning is a method of computer programming where the code can adapt its behavior based on big brain data.[3] The big brain data is the collection of large amounts of information for the purpose of deciphering patterns through computer analysis using machine learning.[4] The information that these technologies provide is extensive enough to allow a researcher to read a patient’s mind. AI and machine learning technologies work by finding the underlying structure of brain data, which is then described by patterns known as latent factors, eventually resulting in an understanding of the brain’s temporal dynamics.[5]
 Through these technologies, researchers are able to decipher how the human brain computes its performances and thoughts. However, due to the extensive and complex nature of the data processed through AI and machine learning, researchers may gain access to personal information a patient may not wish to reveal. From a bioethical lens, tensions arise in the realm of patient autonomy. Patients are not able to control the transmission of data from their brains that is analyzed by researchers. Governing brain data through laws may enhance the extent of patient privacy in the case where brain data is being used through AI technologies.[6] A responsible approach to governing brain data would require a sophisticated legal structure.
 Analysis
 Impact on Patient Autonomy and Privacy 
 In research pertaining to big brain data, the consent forms do not fully cover the vast amounts of information that is collected. According to research, personal data has become the most sought out commodity to provide content to corporations and the web-based service industry. Unfortunately, data leaks that release private information frequently occur.[7] The storage of an individual’s data on technologies accessible on the internet during research studies makes it vulnerable to leaks, jeopardizing an individual’s privacy. These data leaks may cause the patient to be identified easily, as the degree of information provided by AI technologies are personalized and may be decoded through brain fingerprinting methods.[8]
 There has been an extensive growth in the development and use of AI. It is efficient in providing information to radiologists who diagnose various diseases including brain cancer and psychiatric disease, and AI assists in the delivery of telemedicine.[9] However, the ethical pitfall of reduced patient autonomy must be addressed by analyzing current AI technologies and creating more options for patient preference in how the data may be used. For instance, facial recognition technology[10] commonly used in health care produces more information than listed in common consent forms, threatening to undermine informed consent. Facial recognition software collects extensive data and may disclose more information than a person would prefer to provide despite being a useful tool for diagnosing medical and genetic conditions.[11] In addition, people may not be aware that their images are being used to generate more clinical data for other purposes. It is difficult to guarantee the data is anonymized. Consent requirements must include informing people about the complexity of the potential uses of the data; software developers should maximize patient privacy.[12] Furthermore, there is a “human element” in the use of AI technologies as medical providers control the use and the extent to which data is captured or accessed through the AI technologies.[13] People must understand the scope of the technology and have clear communication with the physician or health care provider about how the medical information will be used. 
 Existing Laws for Brain Data Governance 
 A strict system of defined legal responsibilities of medical providers will ensure a higher degree of patient privacy and autonomy when AI technologies and data from machine learning are used. Governing specific algorithmic data is crucial in safeguarding a patient’s privacy and developing a gold standard treatment protocol following the procurement of the information.[14] Certain AI technologies provide more data than others, and legal boundaries should be established to ensure strong performance, quality control, and scope for patient privacy and autonomy. For instance, currently AI technologies are being used in the realm of intensive neurological care. However, there is a significant level of patient uncertainty about how much control patients have over the data’s uses.[15] Calibrated legal and ethical standards will allow important brain data to be securely governed and monitored.
 Once brain signals are recorded and processed from one individual, the data may be merged with other data in Brain Computer Interface Technology (BCI).[16] To ensure a right and ability to retrieve personal data or pull it from the collection, specific regulations for varying types of data are needed.[17] The importance of consent and patient privacy must be considered through giving patients a transparent view of how brain data is governed.[18] The legal system must address discriminatory issues and risks to patients whose data is used in studies. Laws like the General Data Protection Regulation (GDPR) and the California Consumer Privacy Protection Act (CCPA) can serve as effective models to protect aggregated data. These laws govern consumer information and ensure the compliance when personal data is collected.[19] California voters recently approved expansion of the CCPA to health data. The Washington Privacy Act, which would have provided rights to access, change, and withdraw personal data, failed to pass. Other states should improve privacy as well,[20] although a federal bill would be preferable. Scientists at the Heidelberg Academy of Sciences argue for data security to be governed in a manner that balances patient privacy and autonomy with the commercial interests of researchers.[21] The balance could be achieved through privacy protections like those in the Washington Privacy Act. Although the Health Insurance Portability and Accountability Act (HIPAA) provides an overall framework to deter the likelihood of dangers to patient protection and privacy, more thorough laws are warranted to combat pervasive data transfer and analysis that technology has brought to the health care industry.[22] Breaches of patient privacy under current HIPAA regulations include releasing patient information to a reporter without their consent and sending HIV data to a patient’s employer without consent.[23] HIPAA does not cover information being shared with outside contractors who do not have an agreement with technology companies to keep patient data confidential. HIPAA regulations also do not always address blatant breaches on patient data confidentiality.[24] Patients must be provided with methods to monitor the data being analyzed to be able to view the extent of private information being generated via AI technologies. In health research, the medical purposes of better diagnosis, earlier detection of diseases, or prevention are ethical justifications for the use of the data if it was collected with permission, the person understood and approved the uses of the data, and the data was deidentified.
 A standard governance framework is required in providing the fairest system of care to patients who allow their brain data to be examined. Informed consent in the neuroscience field could reaffirm the privacy and autonomy of patients by ensuring that they understand the type of information collected. Laws also could protect data after a patient’s death. Malpractice in the scope of brain data could give people a cause of action critical in safeguarding patient’s rights. Data breach lawsuits will become common but generally do not cover deidentified data that becomes part of big data collection. A more synchronized approach to the collection and consent process will encourage an understanding of how big data is used to diagnose and treat patients. Some altruistic people may even be more likely to consent if they know the largescale data collection is helpful to treat and diagnose people. Others should have the ability to opt out of sharing neurological data, especially when there is not certainty surrounding deidentification.[25]
 Conclusion
 Artificial intelligence and machine learning technologies have the potential to aid in the diagnosis and treatment of people globally by extracting and aggregating brain data specific to individuals. However, the secure use of the data is necessary to build trust between care providers and patients, as well as in balancing the bioethical principles of beneficence and patient autonomy. We must ensure the highest quality of care to patients, while protecting their privacy, informed consent, and clinical trust. More sophis

Data Science is associated with new discoveries, the discovery of value from the data. It is a practice of deriving insights and developing business strategies through transformation of data in to useful information. It has been evaluated as a scientific field and research evolution in disciplines like statistics, computing science, intelligence science, and practical transformation in the domains like science, engineering, public sector, business and lifestyle. The field encompasses the larger areas of artificial intelligence, data analytics, machine learning, pattern recognition, natural language understanding, and big data manipulation. It also tackles related new scientific challenges, ranging from data capture, creation, storage, retrieval, sharing, analysis, optimization, and visualization, to integrative analysis across heterogeneous and interdependent complex resources for better decision-making, collaboration, and, ultimately, value creation. In this paper we entitled epicycles of analysis, formal modeling, from data analysis to data science, data analytics -A keystone of data science, The Big data is not a single technology but an amalgamation of old and new technologies that assistance companies gain actionable awareness. The big data is vital because it manages, store and manipulates large amount of data at the desirable speed and time. Big data addresses detached requirements, in other words the amalgamate of multiple un-associated datasets, processing of large amounts of amorphous data and harvesting of unseen information in a time-sensitive generation.
 As businesses struggle to stay up with changing market requirements, some companies are finding creative ways to use Big Data to their growing business needs and increasingly complex problems. As organizations evolve their processes and see the opportunities that Big Data can provide, they struggle to beyond traditional Business Intelligence activities, like using data to populate reports and dashboards, and move toward Data Science- driven projects that plan to answer more open-ended and sophisticated questions.
 Although some organizations are fortunate to have data scientists, most are not, because there is a growing talent gap that makes finding and hiring data scientists in a timely manner is difficult. This paper, aimed to demonstrate a close view about Data science, big data, including big data concepts like data storage, data processing, and data analysis of these technological developments, we also provide brief description about big data analytics and its characteristics , data structures, data analytics life cycle, emphasizes critical points on these issues.

The Delhi and NCR healthcare systems are rapidly registering electronic health records, diagnostic information available electronically. Furthermore, clinical analysis is rapidly advancing-large quantities of information are examined and new insights are part of the analysis of this technology-experienced as Big Data. It provides tools for storing, managing, studying, and assimilating large amounts of robust, structured and unstructured data generated by existing medical organizations. Recently, data analysis data have been used to help provide care and diagnose disease. In the current era, systems need connected devices, people, time, places and networks that are fully integrated on the Internet (IoT). The Internet has become new in developing health monitoring systems. Diabetes is defined as a group of metabolic disorders affecting human health worldwide. Extensive research (diagnosis, path physiology, treatment, etc.) produces a great deal of data on all aspects of diabetes. The main purpose of this chapter is to provide a detailed analysis of healthcare using large amounts of data and analysis. From the Hospitals of Delhi and NCR, sample of 30 subjects has been collected in random fashion who has been suffering from Diabetes from their Health Insurance Providers without disclosing any Personal Information (PI) or Sensitive Personal Information (SPI) by Law. The present study aimed to analyze diabetes with the latest IoT and Big Data analysis techniques and it’s correlation with stress (TTH) on human health. Authors have tried to include age, gender & insulin factor and its correlation with diabetes. Overall, In conclusion, TTH cases increasing with age in case of males and not following the pattern of diabetes variation with age while in case of female TTH pattern variation is same as diabetes i.e. increasing trend up to age of 60 then decreasing.

SummaryIn recent years, wireless sensor networks are pervasive and are generating tons of data every second. Performing outlier detection to detect faulty sensors from such a large amount of data becomes a challenging task. Most of the existing techniques for outlier detection in wireless sensor networks concentrate only on contents of the data source without considering correlation among different data attributes. Moreover, these methods are not scalable to big data. To address these 2 limitations, this paper proposes an outlier detection approach based on correlation and dynamic SMO (sequential minimal optimization) regression that is scalable to big data. Initially, correlation is used to find out strongly correlated attributes and then the point anomalous nodes are detected using dynamic SMO regression. For fast processing of big data, Hadoop MapReduce framework is used. The experimental analysis demonstrates that the proposed approach efficiently detects the point and contextual anomalies and reduces the number of false alarms. For experiments, real data of sensors used in body sensor networks are taken from Physionet database.

Big Data is utilized for large amount of data in the format of information to the media. The information technology based organizations are developed Big Data for huge volume of data processing like Google, Oracle, Yahoo and IBM. Big data is also used to gather the piece of information without losing any data in the organization. The dynamic and responsive business will utilize the content of the large amount of data. This paper analyzes the problems occurred for maintaining big data processing and maintaining it. Hadoop Distribution File System is developed to produce the solutions for the big data challenges with the concept of Acquire, organize, analyze and decide using Analytic application.

Big Data OverviewDriven by the need to generate business value, the enterprise has started to adopt Big Data as a solution, migrating from the classical databases and data stores which lack the flexibility and are not optimized enough [1].The changes in the environment make big data analytics attractive to all types of organizations, while the market conditions make it practical. The combination of simplified models for development, commoditization, a wider palette of data management tools, and low-cost utility computing has effectively lowered the barrier to entry. [2]. The concept addresses large volumes of complex data, rapid growing data sets that may come from different autonomous sources.In recent approaches, Big Data is characterized by principles known as the 4VVolume, Variety, Velocity and Veracity [3]. There are opinions about accepting other principles as Big Data characteristics, such as Value.Each day more businesses realize that Big Data is relevant as the applications generate large volumes of data generated automatically, from different data sources, centralized or autonomous. As traditional databases hit limitations when the need of analyzing this data, dedicated solutions must be considered.Important BigData Solutions:* Apache HBase/Hadoop is based on Google's BigTable distributed storage system, which runs on top of Hadoop as a distributed and scalable big data store. This means that HBase can leverage the distributed processing paradigm of the Hadoop Distributed File System (HDFS) and benefit from Hadoop's MapReduce programming model. It combines the scalability of Hadoop with real-time data access as a key/value store and deep analytic capabilities of Map Reduce [4].HBase allows to query for individual records as well as derive aggregate analytic reports across a massive amount of data. It can host large tables with billions of rows, millions of columns and run across a cluster of commodity hardware. HBase is composed of three types of servers in a master slave type of architecture. Region servers are responsible to serve data for reads and writes. When accessing data, clients communicate with HBase Region Servers directly. Region assignment, DDL (create, delete tables) operations are handled by the HBase Master process.* Apache Cassandra is a distributed database used for the administration and management of large amounts of structured data across multiple servers, while providing highly available service and no single point of failure. It provides features such as continuous availability, linear scale performance, data distribution across multiple data centers and cloud availability zones. Cassandra inherits its data architecture from Google's BigTable and it borrows its distribution mechanisms from Amazon's Dynamo.The nodes in a Cassandra cluster are completely symmetrical, all having identical responsibilities. Cassandra also employs consistent hashing to partition and replicate data. it has the capability of handling large amounts of data and thousands of concurrent users or operations per second across multiple data centers.Cassandra has a hierarchy of caching mechanisms and carefully orchestrated disk I/O which ensures speed and data safety. Write operations are sent first to a persistent commit log (ensuring a durable write), then to a write-back cache called a memTable; when the memTable fills, it is flushed to a sorted string table - SSTable - on disk. A Cassandra cluster is organized as a ring, and it uses a partitioning strategy to distribute data evenly.* Redis represents an in-memory data structure store used as a database, cache and message broker. It supports data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperlogs and geospatial indexes with radius queries. Redis stores all data in RAM, allowing lightning fast reads and writes. It runs extremely efficiently in memory and handles high-velocity data, needing simple standard servers to deliver millions of operations per second with submillisecond latency. …

The purpose of the work is to study the current problems and prospects of the solution for processing big data received or stored in the Internet (web data), as well as the possibility of practical realization of Data Mining technology for big web data on practical example. Materials and methods. The study included a review of bibliographic sources on big data analysis problems.Data Mining technology was used to analyze large web data, as well as computer modeling of a practical problem using the C # programming language and creating a DDL database structure for accumulating web data.Results. In the course of the work, the specifics of big data were described, the main characteristics of big data were highlighted, and modern approaches to processing big data were analyzed. A brief description of the horizontal-scalable architecture and the BI-solution architecture for big data processing is given. The problems of processing large web data are formulated: limiting the speed of access to data, providing access via network protocols through general-purpose networks.An example showing the approach to processing large web data was also implemented. Based on the idea of big data, the described complexities of web data processing and the methods of Data Mining, techniques were proposed for effectively solving the practical problem of processing and searching patterns in a large data array.The following classes have been developed in the C # programming language:Class of receiving web data via the Internet; Data conversion class;Intelligent data processing class;Created DDL script that creates a structure for the accumulation of web data.A single UML class diagram has been developed.The constructed system of data and classes allows to solve the main part of the problems of processing large web data and perform intelligent processing using Data Mining technology in order to solve the problem posed of identifying certain records in a large array. The combination of object-oriented approach, neural networks and BI-analysis to filter data will speed up the process of data processing and obtaining the result of the studyConclusion. According to the results of the study, it can be argued that the current state of technology for analyzing large web data allows you to efficiently process data objects, identify patterns, get hidden data and get full-fledged statistical data.The obtained results can be used both for the purpose of the initial study of big data processing technologies, and as a basis for developing an already real application for analyzing web data. The use of neural networks and the created universal classes-handlers makes the created architecture flexible and self-learning, and the class declarations and the base DDL structure will greatly simplify the development of program code.

There is a growing opportunity for self-improvement within the realm of higher education, represented by the combination of big data technologies and higher education. The Education Big Data Processing Framework (EBDPF) is the name of the platform that is the subject of this article, which focuses on the development of an advanced big data processing platform. In order to process huge datasets in an effective manner, the platform makes use of Hadoop's big data storage architecture, in addition to Hive, Flume, and Sqoop for data collection and synchronization, respectively. In this study, the implementation of standard data mining techniques is investigated through the use of MapReduce programming. More specifically, the study investigates the execution efficiency and scalability of these algorithms within the Hadoop platform. The data clustering task in data mining is chosen as a representative problem in this study so that the effectiveness of the EBDPF may be evaluated accordingly. An implementation of the clustering job that is based on MapReduce is created and tested on the Hadoop platform. Extensive experiments are conducted with a range of cluster sizes and data sizes. The findings provide more evidence that Hadoop distributed systems are more efficient and successful than traditional methods when it comes to managing data mining applications. The extended performance analysis of computing power not only provides evidence that the platform is currently effective, but it also provides a signal that it has a significant amount of potential that has not yet been utilized. At the convergence of big data technology and higher education, the EBDPF emerges as a strong and promising framework, demonstrating considerable steps in accelerating data processing within educational contexts. In conclusion, the EBDPF is a framework that demonstrates significant progress.

The world is becoming increasingly digital at the moment. Every day, a significant amount of data is generated by everyone who uses the internet nowadays. The data are critical for carrying out day-to-day operations, as well as assisting corporate management in achieving their objectives and making the best judgments possible based on the information gathered. BigData is the process of merging many hardware and software solutions to deal with extremely huge amounts of data that surpass storage capability. It’s possible that large amounts of data will be generated. Hadoop systems are used in a variety of areas, including healthcare, finance, and government. insurance, and social media, in order to provide a quick and cost-effective big data solution. The Apache Hadoop is a framework for storing and processing data, managing, and distributing large amounts of information over a large number of server nodes. Here are some solutions that work on top of the Apache Hadoop stack to guarantee data security. To get a complete picture of the problem, we decided to conduct an investigation into existing security solutions for Apache Hadoop security in sensitive data which is stored on a huge data platform employing distributed computing on a cluster of commodity devices. The goal of this paper is to provide knowledge of security and Big Data issues.

These tutorials discuss the following: Big Data Management and Mining; Nature-Inspired Cyber Security.

As we are living in the era of big data, huge volumes of a wide variety of complex data-which can be of different levels of veracity-are generated or collected at a high velocity from rich sources of data in various real-life applications. A rich source of these big data sources is the Internet of Things (IoT), which include a collection of sensors, smartphones and other mobile devices, wearable devices, as well as other "things" that are capable to operate within the existing Internet infrastructure. Embedded in these big data are valuable knowledge and useful information. Hence, the research problem of data mining from big IoT data have drawn attention of many researchers as it aims to discover implicit, previously unknown and potentially useful information and knowledge from the data. For instance, frequent pattern mining finds sets of frequently co-occurring items in the IoT domains. Associative classification discovers rules revealing relationships among items within the frequent patterns and their associations with the corresponding class labels. Induction based classification uses decision tree or random forest to learn from old big IoT for classifying or making predictions on new data. Over the past quarter of a century, many serial, distributed, parallel, and MapReduce-based (Hadook-based and Spark-based) big data mining algorithms have been proposed. These algorithms are run in local computers, distributed and parallel environments, clusters, grids, clouds and/or data centers. In this paper, we review some of these algorithms, discuss issues and research prospective in mining classification patterns from these big IoT data in fog. Our case study on a real-life application shows the feasibility on classifying real-life big IoT data over fog for urban analytics.