Score distillation for anomaly detection

Cyber-physical systems (CPS) have been widely deployed in critical infrastructures and are vulnerable to various attacks. Data integrity attacks manipulate sensor measurements and cause control systems to fail, which are one of the prominent threats to CPS. Anomaly detection methods are proposed to secure CPS. However, existing anomaly detection studies usually require expert knowledge (e.g., system model-based) or are lack of interpretability (e.g., deep learning-based). In this paper, we present Deepnoise, a deep learning-based anomaly detection method for CPS with interpretability. Specifically, we utilize the sensor and process noise to detect data integrity attacks. Such noise represents the intrinsic characteristics of physical devices and the production process in CPS. One key enabler is that we use a robust deep autoencoder to automatically extract the noise from measurement data. Further, an LSTM-based detector is designed to inspect the obtained noise and detect anomalies. Data integrity attacks change noise patterns and thus are identified as the root cause of anomalies by Deepnoise. Evaluated on the SWaT testbed, Deep-noise achieves higher accuracy and recall compared with state-of-the-art model-based and deep learning-based methods. On average, when detecting direct attacks, the precision is 95.47%, the recall is 96.58%, and F is 95.98%. When detecting stealthy attacks, precision, recall, and F scores are between 96% and 99.5%.

The data collected by sensors is streaming data in the Internet of Things(IoT). Although existing deep learningbased anomaly detection methods generally perform well on static data, they struggle to respond timely to streaming data after distribution changes. However, streaming data suffers from conceptual drift due to the highly dynamic nature of IoT. In network security, concept drift-oriented anomaly detection is a crucial task, because it can adjust the model to adapt to the latest data, and detect attacks in time. Existing streaming anomaly detection methods are confronted with some challenges, including of the latency of model updates, the uneven importance of new data, and the self-poisoning due to model self-updates. To tackle the above challenges, we propose a knowledge distillation based adaptive anomaly detection model towards concept-drift, ADTCD. ADTCD transfers the knowledge of the teacher model to the student model and only updates the student model to reduce the delay. We construct an algorithm of dynamically adjusting model parameters, which dynamically adjusts model weights through local inference on new samples, in order to improving the models responsiveness to new distribution data, meanwhile solving the problem of uneven importance of new data. In additions, we adopt a one-class support vector-based outlier removal method to tackle the self-poisoning problem. In comprehensive experiments on seven high-dimensional datasets, ADTCD achieves an AUC improvement of 12.46% compared to the state-of-the-art streaming anomaly detection methods. Our future direction will focus on exploring the concept drift problem using methods beyond autoencoders.

Since deep neural networks can learn data representation from training data automatically, deep learning methods are widely used in the network anomaly detection. However, challenges of deep learning-based anomaly detection methods still exist, the major of which is the training data scarcity problem. In this paper, we propose a novel network anomaly detection method (NAFT) using federated learning and transfer learning to overcome the data scarcity problem. In the first learning stage, a people or organization \(O_t\), who intends to conduct a detection model for a specific attack, can join in the federated learning with a similar training task to learn basic knowledge from other participants’ training data. In the second learning stage, \(O_t\) uses the transfer learning method to reconstruct and re-train the model to further improve the detection performance on the specific task. Experiments conducted on the UNSW-NB15 dataset show that the proposed method can achieve a better anomaly detection performance than other baseline methods when training data is scarce.

Time series anomaly detection is extensively studied in statistics, economics, and computer science. Over the years, numerous methods have been proposed for time series anomaly detection using deep learning-based methods. Many of these methods demonstrate state-of-the-art performance on benchmark datasets, giving the false impression that these systems are robust and deployable in many practical and industrial real-world scenarios. In this paper, we demonstrate that the performance of state-of-the-art anomaly detection methods is degraded substantially by adding only small adversarial perturbations to the sensor data. We use different scoring metrics such as prediction errors, anomaly, and classification scores over several public and private datasets ranging from aerospace applications, server machines, to cyber-physical systems in power plants. Under well-known adversarial attacks from Fast Gradient Sign Method (FGSM) and Projected Gradient Descent (PGD) methods, we demonstrate that state-of-the-art deep neural networks (DNNs) and graph neural networks (GNNs) methods, which claim to be robust against anomalies and have been possibly integrated in real-life systems, have their performance drop to as low as 0%. To the best of our understanding, we demonstrate, for the first time, the vulnerabilities of anomaly detection systems against adversarial attacks. The overarching goal of this research is to raise awareness towards the adversarial vulnerabilities of time series anomaly detectors.

Image classification is a fundamental problem in the field of computer vision and involves assigning a label to an image based on its content. In this paper, we survey both traditional machine learning-based image classification methods and deep learning-based image classification methods, specifically, we review two deep learning-based image classification methods: a convolutional neural network (CNN) method and a pre-trained CNN-based transfer learning method for image classification. We first briefly review traditional machine learning-based image classification methods and then deep learning-based image classification methods, discussing both the feature extraction methods and the classification methods used in the deep learning-based methods. We discuss several deep neural network architectures for image classification, including LeNet, AlexNet, VGGNet, GoogleNet, ResNet, and DenseNet, and finally, we conclude with a discussion on the applications of image classification and compare the different methods based on various factors both qualitatively and quantitatively, and present our experimental results, while also summarizing various methods of data augmentation, batch normalization, and regularization by dropout. We introduce techniques for practical training of deep networks, and discuss fine-tuning, pruning, and model quantization for efficient inference of trained neural networks, and finally, we introduce and discuss the runtime engines for the deployment of trained neural networks for efficient inference.

Volunteered Geographic Information (VGI), defined as geographic information contributed voluntarily by individuals, has grown exponentially with the aid of ubiquitous GPS-enabled technologies. VGI projects have generated a large amount of geographic data, providing a new data source for scientific research. However, many scientists are concerned about the quality of VGI data for research, given the lack of rigorous and systematic quality control procedures. This study contributes to the improvement of quality control procedures by proposing a Cross-Volunteer Referencing Anomaly Detection (CVRAD) method to filter anomalous data, using the crowdsourced Safecast radiation data as a case study. The anomaly detection method is validated using two data sets: (1) an official radiation survey data set collected by the KURAMA car-borne system, (2) a set of anomalous Safecast measurements filtered by Safecast moderators. The validation results show that the proposed CVRAD method outperformed the 1.5 IQR benchmark method in minimizing the overall measurement error and detecting abnormal imports of Safecast measurements, thus demonstrating the effectiveness of the proposed method in improving the overall accuracy of crowdsourced radiation measurements.

Evolving anomaly detection for network streaming data

The increasing prevalence of vision-threatening retinal diseases poses a growing challenge on healthcare systems worldwide, underscoring the need for effective strategies in disease diagnosis and management. Unsupervised artificial intelligence (AI) approaches, such as anomaly detection, offer the potential to identify established and novel retinal imaging biomarkers. They reduce reliance on expert-driven annotations, mitigate bias, and overcome the limitations of predefined disease categories. Moreover, when integrated into clinical workflows, they have the potential to enhance practical applicability by supporting efficient screening, longitudinal disease monitoring, and objective assessment of treatment response. This review examines the role of anomaly detection for discovering clinically relevant retinal imaging biomarkers, by categorizing biomarkers according to their underlying pathophysiology, summarizing the spectrum of anomaly detection methods applied in the field, and highlighting biomarkers that have been identified through these approaches. Current anomaly detection methods favor the identification of established biomarkers characterized by strong intensity contrasts and well-defined structural boundaries, while more subtle abnormalities remain difficult to capture. Future research should prioritize integrating large language models (LLMs), foundation methods, multimodality, and few-shot anomaly detection, and increase explainability to advance the clinical application of anomaly detection for retinal imaging biomarker discovery in ophthalmology.

Wind turbine condition monitoring is considered as a key task in wind power industry. A plethora of methodologies based on machine learning have been proposed but the absence of faulty data, at the amount and the variety needed, still set limitations. Therefore Anomaly Detection methodologies are proposed as alternatives for fault detection. Deep learning tools have been introduced in the research field of wind turbines’ monitoring for the purpose of higher detection accuracy. In this work, a deep learning-based anomaly detection method, the Deep Support Vector Data Description (Deep SVDD), is proposed for the monitoring of wind turbines. Compared to the classic SVDD anomaly detection approach, this method combines a deep network, more specifically a Convolutional Neural Network (CNN), with the SVDD detector in order to automatically extract effective features. To test and validate the effectiveness of the proposed method, the Deep SVDD method is applied on SCADA data from a real wind turbine use case, targeting to the ice detection on wind turbine blades. The experimental results show that the method can effectively detect the generation of ice on wind turbines’ blades with a successful detection rate of 91.45%.

Keep DRÆMing: Discriminative 3D anomaly detection through anomaly simulation

The definitions of anomaly detection and big data were presented. Due to the sampling and storage burden of anomaly detection in big data, compressive sensing theory was introduced and used in anomaly detection algorithm. The anomaly detection criterion based on wavelet packet transform and statistic process control theory was deduced. The anomaly detection method was used for through wall human detection. The experiments for detecting human behind Brick wall based on UWB radar signal was carried out. The results showed that the proposed anomaly detection algorithm could effectively detect the existence of human being through compressed signals.KeywordsAnomaly detectionBig dataThrough wall human detectionCompressive sensing

Anomaly detection is an important problem in computer vision; however, the scarcity of anomalous samples makes this task difficult. Thus, recent anomaly detection methods have used only “normal images” with no abnormal areas for training. In this work, a powerful anomaly detection method is proposed based on student-teacher feature pyramid matching (STPM), which consists of a student and teacher network. Generative models are another approach to anomaly detection. They reconstruct normal images from an input and compute the difference between the predicted normal and the input. Unfortunately, STPM does not have the ability to generate normal images. To improve the accuracy of STPM, this work uses a student network, as in generative models, to reconstruct normal features. This improves the accuracy; however, the anomaly maps for normal images are not clean because STPM does not use anomaly images for training, which decreases the accuracy of the image-level anomaly detection. To further improve accuracy, a discriminative network trained with pseudo-anomalies from anomaly maps is used in our method, which consists of two pairs of student-teacher networks and a discriminative network. The method displayed high accuracy on the MVTec anomaly detection dataset.

Predictive anomaly detection for marine diesel engine based on echo state network and autoencoder

Many anomaly detection approaches, especially deep learning methods, have been recently developed to identify abnormal image morphology by only employing normal images during training. Unfortunately, many prior anomaly detection methods were optimized for a specific "known" abnormality (e.g., brain tumor, bone fraction, cell types). Moreover, even though only the normal images were used in the training process, the abnormal images were often employed during the validation process (e.g., epoch selection, hyper-parameter tuning), which might leak the supposed "unknown" abnormality unintentionally. In this study, we investigated these two essential aspects regarding universal anomaly detection in medical images by (1) comparing various anomaly detection methods across four medical datasets, (2) investigating the inevitable but often neglected issues on how to unbiasedly select the optimal anomaly detection model during the validation phase using only normal images, and (3) proposing a simple decision-level ensemble method to leverage the advantage of different kinds of anomaly detection without knowing the abnormality. The results of our experiments indicate that none of the evaluated methods consistently achieved the best performance across all datasets. Our proposed method enhanced the robustness of performance in general (average AUC 0.956).

Retrieving the layout of different cluttered indoor scenes from monocular images is a challenging task, using a smaller deep neural network than the existing proposals. Previous geometric solutions are prone to failure in the presence of cluttered scenes because they depend strongly on hand-engineering features and the expectation of the possibility of the vanishing points’ calculation. With the growth of neural networks, the geometric methods were either replaced or fused within the emerging area of deep learning. The more recent solutions rely on dense neural networks with additional adjustments, either by the calculation of the vanishing points, position based, or by layout ranking. All these methods presented valid solutions to this challenge with the flaw of being computationally demanding. Here, we present a more lightweight solution, running the segmentation on a smaller neural network and introducing a discriminative classifier for the posterior layout ranking and optimization. Our proposed method is evaluated by two standard dataset benchmarks, achieving near state of the art results even with a fraction of the required parameters than the available state of the art methods.