Data analysis and outlier detection in smart city

SummaryIn recent years, wireless sensor networks are pervasive and are generating tons of data every second. Performing outlier detection to detect faulty sensors from such a large amount of data becomes a challenging task. Most of the existing techniques for outlier detection in wireless sensor networks concentrate only on contents of the data source without considering correlation among different data attributes. Moreover, these methods are not scalable to big data. To address these 2 limitations, this paper proposes an outlier detection approach based on correlation and dynamic SMO (sequential minimal optimization) regression that is scalable to big data. Initially, correlation is used to find out strongly correlated attributes and then the point anomalous nodes are detected using dynamic SMO regression. For fast processing of big data, Hadoop MapReduce framework is used. The experimental analysis demonstrates that the proposed approach efficiently detects the point and contextual anomalies and reduces the number of false alarms. For experiments, real data of sensors used in body sensor networks are taken from Physionet database.

Internet of Things (IoT) and Big Data have played a significant role in the development and sustainability of Smart Cities. They are the backbones of Smart Cities' efforts to connect human resources, social capital, and ICT infrastructure to address public challenges, achieve sustainable development and improve people's standard of life. It helps Smart Cities improve public services, such as water management, healthcare, education, security issues, transportation, energy usage, and other community to achieve a better life. Big Data helps to analyze, store and back-ups a massive amount of data generated in smart cities in seconds day by day. For this matter, different analytical tools are employed, such as data analytics, machine learning, statistics, and data mining. Even though there are many benefits offered to smart cities; the Smart Cities encounters challenges that need to be addressed. Many of these challenges focus on security and privacy, infrastructures and operational and handling massive data. This paper describes the overview of IoT, Big Data, and Smart Cities. The Applications of IoT and Smart Cities were also discussed; the features of Smart Cities and Big Data; Different challenges of IoT, Big Data, and Smart Cities are described. Also, the proposed research work has presented the relationship between IoT and Big Data to Smart Cities; the role of IoT and Big Data in smart cities was identified; finally, the future directions for IoT and Big Data regarding Smart Cities are discussed.

Outlier detection (or anomaly detection) is a fundamental task in data mining. Outliers are data that deviate from the norm and outlier detection is often compared to “finding a needle in a haystack.” However, the outliers may generate high value if they are found, value in terms of cost savings, improved efficiency, compute time savings, fraud reduction and failure prevention. Detection can identify faults before they escalate with potentially catastrophic consequences. Big Data refers to large, dynamic collections of data. These vast and complex data appear problematic for traditional outlier detection methods to process but, Big Data provides considerable opportunity to uncover new outliers and data relationships. This chapter highlights some of the research issues for outlier detection in Big Data and covers the solutions used and research directions taken along with an analysis of some current outlier detection approaches for Big Data applications.

In this developing paradigm, heterogeneous network infrastructure is combined with ubiquitous devices for sensing, smart control systems and big data analytics to create “smart cities.” Enhancing the overall quality of life is one of their main goals and they do so through smart services they offer in a broad range of areas such as transportation, healthcare, entertainment (including virtual reality), the environment (including pollution) and energy. To be able to offer those services, the job of big data and their analysis is complicated, since it allows for the extraction of useful insights from the enormous amounts of data that the smart cities produce. The current state-of-the-art research efforts aimed at big data analytics in a smart city setting are investigated and presented in this chapter. In particular, a comprehensive literature review on the big data analytic frameworks for smart cities is provided. Following that, a short talk on IoT and big data is briefed. Owing to different data that the smart cities collect, which frequently have contradictory processing needs, appropriate investigative methods based on the data type are also recommended, depending on the data type. The many smart city applications were investigated and a perspective on the difficulties associated with smart city development was developed. In this chapter, the methods addressing the smart city issues and development are examined, and the possibility for the employment of integrated IoT and big data analytics is presented. Finally, as a case study, a briefing on a call service application for the fire department in a smart city is provided to round up the presentation.

The concept of smart cities is getting popularity day by day. Many countries have started adopting the idea of smart city to improve the quality of life by achieving the recommended level of sustainable development. The smart city is a technologically advanced area which could understand the world by analyzing the data in order to improve living conditions. The underlying technology infrastructure of smart cities are wireless sensor network (WSN), Internet of thing (IoT), RFID, and 6G are among others. Along with the technology, the role of machine learning and data analytics can’t be ignored. The smart cities generate massive amount of data from the monitoring equipments and sensors. Big data analytics is one of the important technologies which is capable of improving intelligent urban facilities. In smart cities large amount of data is continuously received from many sensors, autonomous machines or intelligent IoT devices. The accurate prediction depends on the approaches of data analytics and machine learning techniques. This chapter presents fundamental of smart cities, vertical in smart cities and data analytics approaches.

PurposeThe purpose of this paper is to propose a distributed deep learning architecture for smart cities in big data systems.Design/methodology/approachWe have proposed an architectural multilayer to describe the distributed deep learning for smart cities in big data systems. The components of our system are Smart city layer, big data layer, and deep learning layer. The Smart city layer responsible for the question of Smart city components, its Internet of things, sensors and effectors, and its integration in the system, big data layer concerns data characteristics 10, and its distribution over the system. The deep learning layer is the model of our system. It is responsible for data analysis.FindingsWe apply our proposed architecture in a Smart environment and Smart energy. 10; In a Smart environment, we study the Toluene forecasting in Madrid Smart city. For Smart energy, we study wind energy foresting in Australia. Our proposed architecture can reduce the time of execution and improve the deep learning model, such as Long Term Short Memory10;.Research limitations/implicationsThis research needs the application of other deep learning models, such as convolution neuronal network and autoencoder.Practical implicationsFindings of the research will be helpful in Smart city architecture. It can provide a clear view into a Smart city, data storage, and data analysis. The 10; Toluene forecasting in a Smart environment can help the decision-maker to ensure environmental safety. The Smart energy of our proposed model can give a clear prediction of power generation.Originality/valueThe findings of this study are expected to contribute valuable information to decision-makers for a better understanding of the key to Smart city architecture. Its relation with data storage, processing, and data analysis.

Smart cities work with large volumes of data from sensor networks and other sources. To prevent data from being compromised by attacks or errors, smart city IT administrators need to apply attack detection techniques to evaluate possible incidents as quickly as possible. Machine learning has proven to be effective in many fields and, in the context of wireless sensor networks (WSNs), it has proven adequate to detect attacks. However, a smart city poses a much more complex scenario than a WSN, and it has to be evaluated whether these techniques are equally valid and effective. In this work, we evaluate two machine learning algorithms (support vector machines (SVM) and isolation forests) to detect anomalies in a laboratory that reproduces a real smart city use case with heterogeneous devices, algorithms, protocols, and network configurations. The experience has allowed us to show that, although these techniques are of great value for smart cities, additional considerations must be taken into account to effectively detect attacks. Thus, through this empiric analysis, we point out broader challenges and difficulties of using machine learning in this context, both for the technical complexity of the systems, and for the technical difficulty of configuring and implementing them in such environments.

The concept of Wireless Sensor Networks (WSNs) has garnered significant global attention due to their wide range of applications. According to the IEEE 802.11 standard, WSNs are wireless networks consisting of sensor nodes (SNs) that interconnect via wireless communication links. These SNs are capable of sensing, processing, and wirelessly transmitting data, even in challenging environments. WSNs are primarily utilized for communication in various domains, including smart cities, healthcare, residential areas, and military applications. However, the deployment of WSNs in environments such as smart cities come with some challenges, particularly traffic congestion. Traffic congestion in smart cities, particularly during peak hours, is often caused by a high volume of vehicles traveling at the same time, resulting in delays, accidents, and inefficiencies under various weather conditions, including sunny and rainy days. Therefore, this paper proposes a novel algorithm called the Fusion-based Intelligent Congestion Management (FICM) algorithm, developed through the integration of the Navigation Reference Spatial Data (NRSD) algorithm and Fusion-based Multimodal Abnormal Detection (FMAD) algorithm. The objective of FICM is to mitigate on-road traffic congestion within smart cities effectively. The algorithm’s performance was evaluated using Network Simulator 3 (NS-3) by comparing its effectiveness with the NRSD and FMAD algorithms. Under sunny weather conditions, the NS-3 simulation results revealed that the FICM algorithm achieved an average False Alarm Rate (FAR) of 0.83%, a Mean Time to Detection (MTTD) of 76.0%, and a Detection Rate (DR) of 84.3%, outperforming both the NRSD and FMAD algorithms. Similarly, under rainy weather conditions, the FICM algorithm demonstrated an average FAR of 14.09%, an MTTD of 57.03%, and a DR of 78.04%, surpassing the performance of the NRSD and FMAD algorithms within the smart city environment.

Smart cities are the future cities that meet a set of technical and nontechnical criteria. It is obvious that smart cities require extensive implementation of information and communication technologies (ICTs). In particular, wireless sensor networks (WSNs) have big roles to play to support smart city operations. It requires large‐scale deployments of WSNs around the city for sensing numerous events. A massive amount of information will then be collected from those WSNs for analyses and decision‐making. The most challenging part is to secure information from various forms of attacks. Detection and prevention procedures in response to cyber‐physical attacks are resource intensive. This chapter provides a tutorial overview of some specialized WSN applications and their cyber‐physical vulnerabilities in the context of smart cities. It also includes a discussion on possible mitigation approaches.

Wireless sensor networks (WSNs) are susceptible to faults in sensor data. Outlier detection is crucial for ensuring the quality of data analysis in WSNs. This paper proposes a novel improved support vector data description method (ID-SVDD) to effectively detect outliers of sensor data. ID-SVDD utilizes the density distribution of data to compensate SVDD. The Parzen-window algorithm is applied to calculate the relative density for each data point in a data set. Meanwhile, we use Mahalanobis distance (MD) to improve the Gaussian function in Parzen-window density estimation. Through combining new relative density weight with SVDD, this approach can efficiently map the data points from sparse space to high-density space. In order to assess the outlier detection performance, the ID-SVDD algorithm was implemented on several datasets. The experimental results demonstrated that ID-SVDD achieved high performance, and could be applied in real water quality monitoring.

Data reliability in Wireless Sensor Networks (WSNs) has a substantial influence on their smooth functioning and resource limitations. In a WSN, the data aggregated from clustered sensor nodes are forwarded to the base station for analysis. Anomaly Detection (AD) focuses on detecting outlier data to ensure consistency during data aggregation. As WSNs have critical resource limitations concerning energy consumption and sensor node lifetime, AD is supposed to provide data integrity with minimum energy consumption, which has been an active research problem. Hence, researchers are striving for methods to improve the accuracy of data handled with a concern on the constraints of WSNs. This paper introduces a Feed-forward Autoencoder Neural Network (FANN) model to detect abnormal instances with improved accuracy and reduced energy consumption. The proposed model also acts as a False Positive Reducer intending to reduce false alarms. It has been compared with the other dominant unsupervised algorithms over robustness and other significant metrics with real-time datasets. Relatively, our proposed model yields an improved accuracy with fewer false alarms thereby supporting a sustainable WSN.

A wireless sensor network (WSN) is defined as a set of spatially distributed and interconnected sensor nodes. WSNs allow one to monitor and recognize environmental phenomena such as soil moisture, air pollution, and health data. Because of the very limited resources available in sensors, the collected data from WSNs are often characterized as unreliable or uncertain. However, applications using WSNs demand precise readings, and uncertainty in data reading can cause serious damage (e.g., health monitoring data). Therefore, an efficient local/distributed data processing algorithm is needed to ensure: (1) the extraction of precise and reliable values from noisy readings; (2) the detection of anomalies from data reported by sensors; and (3) the identification of outlier sensors in a WSN. Several works have been conducted to achieve these objectives using several techniques such as machine learning algorithms, mathematical modeling, and clustering. The purpose of this paper is to conduct a systematic literature review to report the available works on outlier and anomaly detection in WSNs. The paper highlights works conducted from January 2004 to October 2018. A total of 3520 papers are reviewed in the initial search process. Later, these papers are filtered by title, abstract, and contents, and a total of 117 papers are selected. These papers are examined to answer the defined research questions. The current paper presents an improved taxonomy of outlier detection techniques. This will help researchers and practitioners to find the most relevant and recent studies related to outlier detection in WSNs. Finally, the paper identifies existing gaps that future studies can fill.

Wireless Sensor Networks (WSNs) have many applications, such as climate monitoring systems, fire detection, smart homes, and smart cities. It is expected that WSNs will be integrated into the Internet of Things (IoT) and participate in various tasks. WSNs play an important role monitoring and reporting environment information and collecting surrounding context. In this paper we consider a WSN deployed for an application such as environment monitoring, and a mobile sink which acts as the gateway between the Internet and the WSN. Data gathering is a challenging problem in WSNs and in the IoT because the information has to be available quickly and effectively without delays and redundancies. In this paper we propose several distributed algorithms for composite event detection and reporting to a mobile sink. Once data is collected by the sink, it can be shared using the IoT infrastructure. We analyze the performance of our algorithms using WSNet simulator, which is specially designed for event-based WSNs. We measure various metrics such as average residual energy, percentage of composite events processed successfully at the sink, and the average number of hops to reach the sink.

Some years ago we used to talk about a very far future, a dream in where people can access to services and information into small devices; in which we can accesses to information, make calls, watch videos and surf maps. That far future is not so far, actually the advance of technology involves a more powerful hardware which is day to day is becoming into small devices with big resources [14]. The upcoming development of electronic allows that these devices could be integrated into smart networks [8]; these networks are called Wireless Sensor Networks (WSN; [13]). WSN do not need physical connections, they use communication systems such as ZigBEE, WiFi, Bluetooth, GSM [7]. Each individual device called moth have autonomous characteristics of communications, storage of data, automatic organization and recovery of the network communication among others that allow the network performs an autonomous work, with the possibility of Internet connection. These characteristic let emerge the so called Internet of Things (IoT), phrase that was coined in the early 1990's, where everyday objects have network connectivity, allowing them to send and receive data [1]. Since these days, technology has an exponential advance and now it is possible to find devices that permit earth observations in real time [11,15,4]. All this collected information use cloud and no-cloud storage [3]. These systems allow to store big amounts of data [6], that is collected from devices integrated in the concept of IoT [5] (especially incomes from smartphones, GPS's devices and various earth observation sensors). The analysis of this big data (some of them spatial information) will be helpful in treatment and finding related solutions in specific areas such as public administration, environment, urban services, etc. [12]. Also, the data that at the beginning was depicted like sparse information, it begins to converge into a shape. This shape can be seen like maps that could be paper maps or digital ones- The representation of the information into maps is linked to a wide range of disciplines and the concept is not new but the enhancement that it has reached until now is incredible, especially the digital information given in Internet which could be accessed by electronic devices (smart phones, tablets and computers). E-government services would improve efficiency and consequently citizen's satisfaction due to the implementation of Spatial Data Infrastructure (SDI, [9]). In order to good development of SDI, the Open Geospatial Consortium (OCG) has an important role. OGC is an international industry consortium of companies, government agencies and universities participating in a consensus process to develop publicly available interface standards [10]. One of the most popular OGC standards is the Web Map Service (WMS) that allows to show spatial information, considered fundamental for the development of many of the services in the way that a citizen can access to e-government services (e.g., cadastre, [2]) and e-democracy. In this talk, the objective is to introduce some concepts considered essential in the development of the modern society and smart cities. Concepts such as IoT, big data and mapping not only allow the collection or merely visualization of information; these promote the enhancement of access to e-services, also they could be used to face challenges that a good administration of the limited resources implies.

Building a smart city that follows sustainability goals enhances the quality of life and preserves environmental, human, and social capital. Yet, existing smart sustainable city projects have concentrated on the technological dimensions of smart cities such as using big data or smart devices to follow sustainability goals. Currently, there is no comprehensive category of smart sustainable city indicators in the literature. This paper aims to discover these indicators by considering the common features of sustainability and smart city concepts. Two rounds of the content analysis technique were employed to investigate semantic, lexical, and conceptual relationships between smart city and sustainability indicators. This paper employed the Sustainable Development Indicators suggested by OECD and the Smart City Index Master by Cohen as the two main groups of indicators. The findings offer a novel set of indicators that enables policymakers and researchers to consider the smartness and sustainability of their projects simultaneously. This includes socio-cultural, economic, environmental, and governance categories with 28 associated indicators. The outcome of this paper offers a unique combined category of smart sustainable city indicators by considering the key elements of sustainability and smart city concepts. Academics and policymakers can also employ this set of indicators as a guideline to build a smart sustainable community.