Survey of Flight Anomaly Detection Methods: Challenges and Opportunities

In the actual maintenance of manufacturing enterprises, abnormal changes in after-sale parts demand data often make the inventory strategies unreasonable. Due to the intermittent and small-scale characteristics of demand sequences, it is difficult to accurately identify the anomalies in such sequences using current anomaly detection algorithms. To solve this problem, this paper proposes an unsupervised anomaly detection method for intermittent time series. First, a new abnormal fluctuation similarity matrix is built by calculating the squared coefficient of variation and the maximum information coefficient from the macroscopic granularity. The abnormal fluctuation sequence can then be adaptively screened by using agglomerative hierarchical clustering. Second, the demand change feature and interval feature of the abnormal sequence are constructed and fed into the support vector data description model to perform hypersphere training. Then, the unsupervised abnormal point location detection is realized at the micro-granularity level from the abnormal sequence. Comparative experiments are carried out on the actual demand data of after-sale parts of two large manufacturing enterprises. The results show that, compared with the current representative anomaly detection methods, the proposed approach can effectively identify the abnormal fluctuation position in the intermittent sequence of small samples, and also obtain better detection results.

Volunteered Geographic Information (VGI), defined as geographic information contributed voluntarily by individuals, has grown exponentially with the aid of ubiquitous GPS-enabled technologies. VGI projects have generated a large amount of geographic data, providing a new data source for scientific research. However, many scientists are concerned about the quality of VGI data for research, given the lack of rigorous and systematic quality control procedures. This study contributes to the improvement of quality control procedures by proposing a Cross-Volunteer Referencing Anomaly Detection (CVRAD) method to filter anomalous data, using the crowdsourced Safecast radiation data as a case study. The anomaly detection method is validated using two data sets: (1) an official radiation survey data set collected by the KURAMA car-borne system, (2) a set of anomalous Safecast measurements filtered by Safecast moderators. The validation results show that the proposed CVRAD method outperformed the 1.5 IQR benchmark method in minimizing the overall measurement error and detecting abnormal imports of Safecast measurements, thus demonstrating the effectiveness of the proposed method in improving the overall accuracy of crowdsourced radiation measurements.

Anomaly detection of flight data is vital to aviation safety. A real time anomaly detection for flight data can identify safety issues as soon as anomalies occur and improve the safety of aviation system. However, existing anomaly detection methods for flight data either perform not very well on unknown issues or are limited in their capabilities of real time detection. This paper provides a method to detect real time flight data anomalies with comparable detection performance. Firstly it transforms discrete and continuous flight data into discrete sequences. And each sequence obtained from discrete and continuous data is segmented into several subsequences by sliding a window of the specified size. More importantly, the variable length n-grams are obtained from each subsequence. Then all of unique n-grams become the feature space of the training data set. In this way each subsequence is mapped to this feature space and becomes an input feature vector of one-class Support Vector Machine (SVM). Finally, anomalies in flight data can be detected in real time by the classifier learned on these feature vectors corresponding to subsequences. Experimental results show that it performs better to use linear kernel to train one-class SVM on our feature vectors than Gaussian kernel and the size of the sliding window takes a little effect on the performance of our detection method. Moreover, this method outperforms Multiple Kernel Anomaly Detection (MKAD) in real time anomaly detection.

The increasing prevalence of vision-threatening retinal diseases poses a growing challenge on healthcare systems worldwide, underscoring the need for effective strategies in disease diagnosis and management. Unsupervised artificial intelligence (AI) approaches, such as anomaly detection, offer the potential to identify established and novel retinal imaging biomarkers. They reduce reliance on expert-driven annotations, mitigate bias, and overcome the limitations of predefined disease categories. Moreover, when integrated into clinical workflows, they have the potential to enhance practical applicability by supporting efficient screening, longitudinal disease monitoring, and objective assessment of treatment response. This review examines the role of anomaly detection for discovering clinically relevant retinal imaging biomarkers, by categorizing biomarkers according to their underlying pathophysiology, summarizing the spectrum of anomaly detection methods applied in the field, and highlighting biomarkers that have been identified through these approaches. Current anomaly detection methods favor the identification of established biomarkers characterized by strong intensity contrasts and well-defined structural boundaries, while more subtle abnormalities remain difficult to capture. Future research should prioritize integrating large language models (LLMs), foundation methods, multimodality, and few-shot anomaly detection, and increase explainability to advance the clinical application of anomaly detection for retinal imaging biomarker discovery in ophthalmology.

In recent years, anomaly detection techniques in time-series data have been widely used in manufacturing, cybersecurity, and other fields. Meanwhile, various anomaly detection models based on generative adversarial networks (GAN) are gradually used in time-series anomaly detection tasks. However, there are problems of unstable generator training, missed detection of anomalous data, and inconsistency between the discriminator's discriminant and the anomaly detection target in GAN networks. Aiming at the above problems, the paper proposes a DUAL-ADGAN (Dual Anomaly Detection Generative Adversarial Networks) model for the detection of anomalous data in time series. First, the Wasserstein distance satisfying the Lipschitz constraint is used as the loss function of the data reconstruction module, which improves the stability of the traditional GAN network training. Second, by adding a data prediction module to the DUAL-ADGAN model, the distinction between abnormal and normal samples is increased, and the rate of missing abnormal data in the model is reduced. Third, by introducing the Fence-GAN loss function, the discriminator is aligned with the anomaly detection target, which effectively reduces the anomaly data false detection rate of the DUAL-ADGAN model. Finally, anomaly scores derived from the DUAL-ADGAN model are compared with dynamic thresholds to detect anomalies. The experimental results show that the average F1 of the DUAL-ADGAN model is 0.881, which is better than the other nine baseline models. The conclusions demonstrate that the DUAL-ADGAN model proposed in the paper is more stable in training while effectively solving the problems of anomaly miss detection and discriminator inconsistency with the anomaly detection target in the anomaly detection task.

Keep DRÆMing: Discriminative 3D anomaly detection through anomaly simulation

Fiber-optic communication systems serve as the backbone of modern data communication networks, with increasing demands on their reliability and robustness in various emerging applications. A key challenge in ensuring reliable fiber-optic transmission lies in addressing fiber anomalies, which can cause signal degradation, service disruptions, and even system failures. However, the current anomaly detection method is too complex to be implemented in deployed networks or consumes too much time during detection. This paper proposes a simple and effective fiber anomaly detection method for C+L-band fiber-optic communication systems, leveraging the spectral tilt induced by the stimulated Raman scattering (SRS) effect. The method reconstructs the spectral tilt along an anomalous fiber link by analyzing the input and output power profile, easily obtainable from optical channel monitors (OCMs), enabling anomaly localization and loss quantification through forward and backward spectral tilt comparison. The performance and robustness of the proposed method are studied and discussed in actual scenarios such as erbium-doped fiber amplifier gain fluctuations, OCM measurement errors, parameter inaccuracies, and modeling inaccuracies. In addition to being able to detect a single anomaly, the process of using the proposed method to achieve multiple anomaly detection links is also discussed according to the chronological order of anomalies. Experimental validation on a multi-span C+L-band transmission system achieves a maximum localization error of 2.06 km and a maximum loss estimation error of 0.14 dB within 2 s. This work underscores the potential of exploiting inherent physical phenomena like SRS for fast anomaly detection in deployed optical networks.

In response to the high accident rate associated with large-scale UAV deployment, the prediction-based anomaly detection method has emerged as an important research field. This method uses airborne sensor data to predict the expected operational status of UAVs to determine whether their actual status deviates from normal. However, when the anomaly detection model is exposed to diverse flight conditions and missions, continuously changing sensor data characteristics may lead to model mismatch, resulting in reduced detection accuracy. Therefore, this paper proposes a similarity augmented incremental learning-based flight data anomaly detection method for UAV dynamic conditions. The core goal of this method is to maintain the accuracy of anomaly detection in new conditions while preventing issues related to model mismatch. Firstly, an LSTM-based anomaly detection model and the feature buffer are established using historical sensor data. The feature buffer is used to update the detection model when new operating conditions arise. Secondly, a JS divergence-based similarity measurement is proposed to monitor changes in the distribution of sensor data and identify whether new operating conditions that differ significantly from historical data occur. Finally, a similarity augmented replay-based incremental learning mechanism is designed for model updates when new operating conditions are detected. During the model update process, similarity measurements are employed to augment the replay-based update strategy, thereby improving the adaptability of the model. Experiments with simulated and real airborne sensor data demonstrate that the proposed method improves anomaly detection accuracy, increasing ACC by 0.6% to 1.8% and reducing FPR by 16.5% to 95.2%.

Evolving anomaly detection for network streaming data

In the manufacturing process, digital twin technology can provide real-time mapping, prediction, and optimization of the physical manufacturing process in the information world. In order to realize the complete expression and accurate identification of and changes in the real-time state of the manufacturing process, a digital twin framework of incremental learning driven by stream data is proposed. Additionally, a novel method of stream data-driven equipment operation state modeling and incremental anomaly detection is proposed based on the digital twin. Firstly, a hierarchical finite state machine (HFSM) for the manufacturing process was proposed to completely express the manufacturing process state. Secondly, the incremental learning detection method driven by stream data was used to detect the anomaly of the job process data, so as to change the job status in real time. Furthermore, the F1 value and time consumption of the proposed algorithm were compared and analyzed using a general dataset. Finally, the method was applied to the practical case development of a welding manufacturer’s digital twin system. The flexibility of the proposed model is calculated by the quantitative method. The results show that the proposed state modeling and anomaly detection method can help the system realize job state mapping and state change quickly, effectively, and flexibly.

This study focuses on unsupervised anomaly detection for structural systems using autoencoders. Since collecting abnormal data is difficult while normal data from actual bridges can be more easily obtained, we propose an unsupervised learning-based anomaly detection method that utilizes the differences between normal and abnormal structural states through autoencoders. The proposed approach employs two models: AE1, which is trained on known normal data and evaluates the reconstruction error for the data to be diagnosed, and AE2, which is trained on the data to be diagnosed and calculates the reconstruction error for known normal data. Anomalies are detected based on the reconstruction errors from each model. Additionally, a combined bridge anomaly detection method called BL-AE, which integrates AE1 and AE2, is proposed. Additionally, the improvement of AE1’s detection performance for actual bridges whose normal state varies due to seasonal changes is investigated. The results showed that learning with overlapping (duplicated) data improves anomaly detection accuracy. Furthermore, we evaluated the anomaly detection performance of BL-AE using artificially generated amplitude anomalies. As a result, Both the AUC and the actual positive rate at 10% false positive rate were improved.

Integration of Information and Communication Technology (ICT) in modern smart grids (SGs) offers many advantages including the use of renewables and an effective way to protect, control and monitor the energy transmission and distribution. To reach an optimal operation of future energy systems, availability, integrity and confidentiality of data should be guaranteed. Research on the cyber-physical security of electrical substations based on IEC 61850 is still at an early stage. In the present work, we first model the network traffic data in electrical substations, then, we present a statistical Anomaly Detection (AD) method to detect Denial of Service (DoS) attacks against the Generic Object Oriented Substation Event (GOOSE) network communication. According to interpretations on the self-similarity and the Long-Range Dependency (LRD) of the data, an Auto-Regressive Fractionally Integrated Moving Average (ARFIMA) model was shown to describe well the GOOSE communication in the substation process network. Based on this ARFIMA-model and in view of cyber-physical security, an effective model-based AD method is developed and analyzed. Two variants of the statistical AD considering statistical hypothesis testing based on the Generalized Likelihood Ratio Test (GLRT) and the cumulative sum (CUSUM) are presented to detect flooding attacks that might affect the availability of the data. Our work presents a novel AD method, with two different variants, tailored to the specific features of the GOOSE traffic in IEC 61850 substations. The statistical AD is capable of detecting anomalies at unknown change times under the realistic assumption of unknown model parameters. The performance of both variants of the AD method is validated and assessed using data collected from a simulation case study. We perform several Monte-Carlo simulations under different noise variances. The detection delay is provided for each detector and it represents the number of discrete time samples after which an anomaly is detected. In fact, our statistical AD method with both variants (CUSUM and GLRT) has around half the false positive rate and a smaller detection delay when compared with two of the closest works found in the literature. Our AD approach based on the GLRT detector has the smallest false positive rate among all considered approaches. Whereas, our AD approach based on the CUSUM test has the lowest false negative rate thus the best detection rate. Depending on the requirements as well as the costs of false alarms or missed anomalies, both variants of our statistical detection method can be used and are further analyzed using composite detection metrics.

The definitions of anomaly detection and big data were presented. Due to the sampling and storage burden of anomaly detection in big data, compressive sensing theory was introduced and used in anomaly detection algorithm. The anomaly detection criterion based on wavelet packet transform and statistic process control theory was deduced. The anomaly detection method was used for through wall human detection. The experiments for detecting human behind Brick wall based on UWB radar signal was carried out. The results showed that the proposed anomaly detection algorithm could effectively detect the existence of human being through compressed signals.KeywordsAnomaly detectionBig dataThrough wall human detectionCompressive sensing

Anomaly detection is an important problem in computer vision; however, the scarcity of anomalous samples makes this task difficult. Thus, recent anomaly detection methods have used only “normal images” with no abnormal areas for training. In this work, a powerful anomaly detection method is proposed based on student-teacher feature pyramid matching (STPM), which consists of a student and teacher network. Generative models are another approach to anomaly detection. They reconstruct normal images from an input and compute the difference between the predicted normal and the input. Unfortunately, STPM does not have the ability to generate normal images. To improve the accuracy of STPM, this work uses a student network, as in generative models, to reconstruct normal features. This improves the accuracy; however, the anomaly maps for normal images are not clean because STPM does not use anomaly images for training, which decreases the accuracy of the image-level anomaly detection. To further improve accuracy, a discriminative network trained with pseudo-anomalies from anomaly maps is used in our method, which consists of two pairs of student-teacher networks and a discriminative network. The method displayed high accuracy on the MVTec anomaly detection dataset.

Predictive anomaly detection for marine diesel engine based on echo state network and autoencoder