Keep DRÆMing: Discriminative 3D anomaly detection through anomaly simulation

Volunteered Geographic Information (VGI), defined as geographic information contributed voluntarily by individuals, has grown exponentially with the aid of ubiquitous GPS-enabled technologies. VGI projects have generated a large amount of geographic data, providing a new data source for scientific research. However, many scientists are concerned about the quality of VGI data for research, given the lack of rigorous and systematic quality control procedures. This study contributes to the improvement of quality control procedures by proposing a Cross-Volunteer Referencing Anomaly Detection (CVRAD) method to filter anomalous data, using the crowdsourced Safecast radiation data as a case study. The anomaly detection method is validated using two data sets: (1) an official radiation survey data set collected by the KURAMA car-borne system, (2) a set of anomalous Safecast measurements filtered by Safecast moderators. The validation results show that the proposed CVRAD method outperformed the 1.5 IQR benchmark method in minimizing the overall measurement error and detecting abnormal imports of Safecast measurements, thus demonstrating the effectiveness of the proposed method in improving the overall accuracy of crowdsourced radiation measurements.

Fiber-optic communication systems serve as the backbone of modern data communication networks, with increasing demands on their reliability and robustness in various emerging applications. A key challenge in ensuring reliable fiber-optic transmission lies in addressing fiber anomalies, which can cause signal degradation, service disruptions, and even system failures. However, the current anomaly detection method is too complex to be implemented in deployed networks or consumes too much time during detection. This paper proposes a simple and effective fiber anomaly detection method for C+L-band fiber-optic communication systems, leveraging the spectral tilt induced by the stimulated Raman scattering (SRS) effect. The method reconstructs the spectral tilt along an anomalous fiber link by analyzing the input and output power profile, easily obtainable from optical channel monitors (OCMs), enabling anomaly localization and loss quantification through forward and backward spectral tilt comparison. The performance and robustness of the proposed method are studied and discussed in actual scenarios such as erbium-doped fiber amplifier gain fluctuations, OCM measurement errors, parameter inaccuracies, and modeling inaccuracies. In addition to being able to detect a single anomaly, the process of using the proposed method to achieve multiple anomaly detection links is also discussed according to the chronological order of anomalies. Experimental validation on a multi-span C+L-band transmission system achieves a maximum localization error of 2.06 km and a maximum loss estimation error of 0.14 dB within 2 s. This work underscores the potential of exploiting inherent physical phenomena like SRS for fast anomaly detection in deployed optical networks.

The increasing prevalence of vision-threatening retinal diseases poses a growing challenge on healthcare systems worldwide, underscoring the need for effective strategies in disease diagnosis and management. Unsupervised artificial intelligence (AI) approaches, such as anomaly detection, offer the potential to identify established and novel retinal imaging biomarkers. They reduce reliance on expert-driven annotations, mitigate bias, and overcome the limitations of predefined disease categories. Moreover, when integrated into clinical workflows, they have the potential to enhance practical applicability by supporting efficient screening, longitudinal disease monitoring, and objective assessment of treatment response. This review examines the role of anomaly detection for discovering clinically relevant retinal imaging biomarkers, by categorizing biomarkers according to their underlying pathophysiology, summarizing the spectrum of anomaly detection methods applied in the field, and highlighting biomarkers that have been identified through these approaches. Current anomaly detection methods favor the identification of established biomarkers characterized by strong intensity contrasts and well-defined structural boundaries, while more subtle abnormalities remain difficult to capture. Future research should prioritize integrating large language models (LLMs), foundation methods, multimodality, and few-shot anomaly detection, and increase explainability to advance the clinical application of anomaly detection for retinal imaging biomarker discovery in ophthalmology.

The Internet of Things (IoT) is an essential component of the digital age, particularly in ensuring reliable and efficient operations as well as the timely and precise recognition of anomalies within IoT systems. However, anomaly detection in time-series data—especially data collected by edge devices—poses several challenges, including concerns over data privacy and communication overhead. To address these issues, this research proposes a novel deep learning (DL)-based anomaly detection model capable of being trained in real-time on edge devices within an IoT environment. The framework ensures user privacy by enabling distributed edge devices to cooperatively train an anomaly recognition system. This study introduces a new, Intelligent Shark Smell Tuned Deep Isolation Forest (ISS-DIF), which effectively detects anomalies and identifies outliers in industrial IoT sensor data with high accuracy. The hybrid model isolates anomalies rather than profiling normal data, making it particularly suitable for identifying rare anomalies within large datasets. Industrial data are collected from real-world manufacturing environments using IoT edge devices. Following data collection, median filtering is applied to reduce noise, and min-max scaling is employed for data normalization. The ISS component allows for fine-tuning the hyperparameters of the Deep Isolation Forest (DIF) to optimize detection performance. The DIF model is tailored to enhance anomaly detection capabilities in sensor data from industrial IoT applications. For validation, 80% of the dataset was used for training, while the remaining 20% served as the test set. The results demonstrate that the proposed ISS-DIF framework achieved superior training accuracy of 99.30%, with precision of 97.89%, recall of 98.76%, and an F1-score of 97.21%, outperforming the testing metrics. This approach integrates real-time anomaly detection with the processing capabilities of edge devices, thereby improving IoT data analytics and providing a scalable, efficient solution that preserves privacy in irregularity recognition within IoT environments.

In recent years, anomaly detection techniques in time-series data have been widely used in manufacturing, cybersecurity, and other fields. Meanwhile, various anomaly detection models based on generative adversarial networks (GAN) are gradually used in time-series anomaly detection tasks. However, there are problems of unstable generator training, missed detection of anomalous data, and inconsistency between the discriminator's discriminant and the anomaly detection target in GAN networks. Aiming at the above problems, the paper proposes a DUAL-ADGAN (Dual Anomaly Detection Generative Adversarial Networks) model for the detection of anomalous data in time series. First, the Wasserstein distance satisfying the Lipschitz constraint is used as the loss function of the data reconstruction module, which improves the stability of the traditional GAN network training. Second, by adding a data prediction module to the DUAL-ADGAN model, the distinction between abnormal and normal samples is increased, and the rate of missing abnormal data in the model is reduced. Third, by introducing the Fence-GAN loss function, the discriminator is aligned with the anomaly detection target, which effectively reduces the anomaly data false detection rate of the DUAL-ADGAN model. Finally, anomaly scores derived from the DUAL-ADGAN model are compared with dynamic thresholds to detect anomalies. The experimental results show that the average F1 of the DUAL-ADGAN model is 0.881, which is better than the other nine baseline models. The conclusions demonstrate that the DUAL-ADGAN model proposed in the paper is more stable in training while effectively solving the problems of anomaly miss detection and discriminator inconsistency with the anomaly detection target in the anomaly detection task.

An integrated energy system (IES) is vulnerable to network attacks due to the coupling features of multi-energy systems, as well as the deep integration between a physical system and an information system. The anomaly detection of the time-series data in an IES is a key problem to defend against network attacks and ensure the cyber-physical security of IES. Aiming at false data injection attacks (FDIAs) on IES, this paper proposes an anomaly detection method for time-series data in a cyber-physical integrated energy system based on time-frequency feature prediction. The time-frequency features of the time-series data are extracted based on three time-frequency transform methods (DWT, EMD, and EWT). Then the extracted time-frequency features are input to the autoencoder (AE) to capture the hidden features and nonlinear structure of the original data in the frequency domain. The time-domain data within the detected time period are predicted by applying regression prediction on the top-layer features of AE. Considering the uncertainty of regression prediction, kernel density estimation (KDE) is used to estimate the probability density function of prediction error and the interval of the predicted data is estimated accordingly. The estimated lower boundary value of the predicted data is selected as the attack judgment threshold for anomaly detection. The results of the case study verify the advantages of the proposed method in reducing the false positive rate and improving the anomaly detection accuracy.

Evolving anomaly detection for network streaming data

Many anomaly detection approaches, especially deep learning methods, have been recently developed to identify abnormal image morphology by only employing normal images during training. Unfortunately, many prior anomaly detection methods were optimized for a specific "known" abnormality (e.g., brain tumor, bone fraction, cell types). Moreover, even though only the normal images were used in the training process, the abnormal images were often employed during the validation process (e.g., epoch selection, hyper-parameter tuning), which might leak the supposed "unknown" abnormality unintentionally. In this study, we investigated these two essential aspects regarding universal anomaly detection in medical images by (1) comparing various anomaly detection methods across four medical datasets, (2) investigating the inevitable but often neglected issues on how to unbiasedly select the optimal anomaly detection model during the validation phase using only normal images, and (3) proposing a simple decision-level ensemble method to leverage the advantage of different kinds of anomaly detection without knowing the abnormality. The results of our experiments indicate that none of the evaluated methods consistently achieved the best performance across all datasets. Our proposed method enhanced the robustness of performance in general (average AUC 0.956).

In response to the high accident rate associated with large-scale UAV deployment, the prediction-based anomaly detection method has emerged as an important research field. This method uses airborne sensor data to predict the expected operational status of UAVs to determine whether their actual status deviates from normal. However, when the anomaly detection model is exposed to diverse flight conditions and missions, continuously changing sensor data characteristics may lead to model mismatch, resulting in reduced detection accuracy. Therefore, this paper proposes a similarity augmented incremental learning-based flight data anomaly detection method for UAV dynamic conditions. The core goal of this method is to maintain the accuracy of anomaly detection in new conditions while preventing issues related to model mismatch. Firstly, an LSTM-based anomaly detection model and the feature buffer are established using historical sensor data. The feature buffer is used to update the detection model when new operating conditions arise. Secondly, a JS divergence-based similarity measurement is proposed to monitor changes in the distribution of sensor data and identify whether new operating conditions that differ significantly from historical data occur. Finally, a similarity augmented replay-based incremental learning mechanism is designed for model updates when new operating conditions are detected. During the model update process, similarity measurements are employed to augment the replay-based update strategy, thereby improving the adaptability of the model. Experiments with simulated and real airborne sensor data demonstrate that the proposed method improves anomaly detection accuracy, increasing ACC by 0.6% to 1.8% and reducing FPR by 16.5% to 95.2%.

The definitions of anomaly detection and big data were presented. Due to the sampling and storage burden of anomaly detection in big data, compressive sensing theory was introduced and used in anomaly detection algorithm. The anomaly detection criterion based on wavelet packet transform and statistic process control theory was deduced. The anomaly detection method was used for through wall human detection. The experiments for detecting human behind Brick wall based on UWB radar signal was carried out. The results showed that the proposed anomaly detection algorithm could effectively detect the existence of human being through compressed signals.KeywordsAnomaly detectionBig dataThrough wall human detectionCompressive sensing

Anomaly detection is an important problem in computer vision; however, the scarcity of anomalous samples makes this task difficult. Thus, recent anomaly detection methods have used only “normal images” with no abnormal areas for training. In this work, a powerful anomaly detection method is proposed based on student-teacher feature pyramid matching (STPM), which consists of a student and teacher network. Generative models are another approach to anomaly detection. They reconstruct normal images from an input and compute the difference between the predicted normal and the input. Unfortunately, STPM does not have the ability to generate normal images. To improve the accuracy of STPM, this work uses a student network, as in generative models, to reconstruct normal features. This improves the accuracy; however, the anomaly maps for normal images are not clean because STPM does not use anomaly images for training, which decreases the accuracy of the image-level anomaly detection. To further improve accuracy, a discriminative network trained with pseudo-anomalies from anomaly maps is used in our method, which consists of two pairs of student-teacher networks and a discriminative network. The method displayed high accuracy on the MVTec anomaly detection dataset.

Predictive anomaly detection for marine diesel engine based on echo state network and autoencoder

Safety enhancement is a major goal of the aviation industry owing to the predicted increase in air travel. There is also the need to prevent fatalities, increase reliability and reduce monetary costs suffered as a result of delays and accidents that still occur. Accidents today are complex as a result of many causal factors acting alone but more often as a combination with other contributing factors. In tackling this trend, proactive measures have been put in place to find hazardous combinations that occur during flights in order to mitigate them before accidents occur. Flight Anomaly Detection (AD) methods are aimed at highlighting abnormal occurrences of a flight, that are different from the norm. As an improvement on the current state-of-the-art method, previous works have proposed different AD techniques for detection of previously unknown flight risks such as component faults, aircraft operational inefficiencies and some abnormal crew behaviour. However, current AD methods individually have limitations that prevent them from detecting certain significant anomalies in flight data. This paper surveys current flight AD approaches, their strengths and limitations as well as brings to light the benefits of a hybrid AD method to extend previous work and find safety-critical events, particularly those related to abnormal crew activity: a class of events known to amount for a substantial number of accidents/incidents today. It also highlights another emerging AD application opportunity, its challenges and how AD is beneficial in addressing them.

Visual anomaly detection (AD) is currently a very active research area with great potential in many real-world applications, e.g. quality control in industry and manufacturing, where it can provide cost savings and overall better product quality. Recently, many new anomaly detection methods have been introduced and many of them are intended for industrial usage. These methods are usually evaluated on a narrow selection of datasets that may differ significantly from certain types of real-world applications. Due to this approach, some methods provide different performance when deployed in real use cases. In this paper, we perform evaluation of recent state of the art visual anomaly detection methods on the problem of defect detection in metal parts fabrication, an area not well covered in existing publications. We introduce a new dataset focused specifically on the problem of metal parts fabrication and use the dataset to perform the evaluation. For the evaluation, we selected methods that use two different feature extraction approaches for anomaly detection. One of the approaches is using feature extractors pretrained on the ImageNet dataset and the second approach is training the feature extractor from scratch using self-supervised learning. We show that, in contrast to one of the most widely used anomaly detection benchmark - the MVTec-AD dataset, self-supervised methods perform significantly better (in average by 17 % AUROC) on the proposed dataset.

This study focuses on unsupervised anomaly detection for structural systems using autoencoders. Since collecting abnormal data is difficult while normal data from actual bridges can be more easily obtained, we propose an unsupervised learning-based anomaly detection method that utilizes the differences between normal and abnormal structural states through autoencoders. The proposed approach employs two models: AE1, which is trained on known normal data and evaluates the reconstruction error for the data to be diagnosed, and AE2, which is trained on the data to be diagnosed and calculates the reconstruction error for known normal data. Anomalies are detected based on the reconstruction errors from each model. Additionally, a combined bridge anomaly detection method called BL-AE, which integrates AE1 and AE2, is proposed. Additionally, the improvement of AE1’s detection performance for actual bridges whose normal state varies due to seasonal changes is investigated. The results showed that learning with overlapping (duplicated) data improves anomaly detection accuracy. Furthermore, we evaluated the anomaly detection performance of BL-AE using artificially generated amplitude anomalies. As a result, Both the AUC and the actual positive rate at 10% false positive rate were improved.