Research on Intelligent Anomaly Detection Algorithm of Launch Vehicle Telemetry Data

Computer detection intelligent mining algorithm for software defined networks

Traditional Intrusion Detection Systems (IDS) can be complemented by an Anomaly Detection Algorithm (ADA) to also identify unknown attacks. We argue that, as each ADA has its own strengths and weaknesses, it might be beneficial to rely on multiple ADAs to obtain deeper insights. ADAs are very resource intensive; thus, real-time detection with multiple algorithms is even more challenging in high-speed networks. To handle such high data rates, we developed a controlled load allocation scheme that adaptively allocates multiple ADAs on a multi-core system. The key idea of this concept is to utilize as many algorithms as possible without causing random packet drops, which is the typical system behavior in overload situations. We developed a proof of concept anomaly detection framework with a sample set of ADAs. Our experiments confirm that the detection performance can substantially benefit from using multiple algorithms and that the developed framework is also able to cope with high packet rates.

ABSTRACTAnomaly detection is an active research topic in hyperspectral remote sensing and has been applied in many areas including environmental monitoring, urban survey, mineral mapping, and national security. Usually, it makes a detection decision without any prior target information or background information. Several anomaly detection algorithms (e.g. Reed–Xiaoli detector, blocked adaptive computationally efficient outlier nominator and random-selection-based anomaly detector) have been developed, which rely on estimating background information only from a hyperspectral image without considering target information in making a detection decision. These methods may be efficient in general but sometimes with high false alarm rate (FAR). In order to reduce FAR, this study proposes a novel method that incorporates both background and target information, derived from the hyperspectral imagery, into anomaly detection algorithms. The target information is helpful to detect anomalies as outliers. With a scene of real airborne visible infrared imaging spectrometer data, the experimental results demonstrate that the proposed method has produced better detection results and higher time efficiency compared with those using the traditional algorithms that only consider background information.

A large amount of work has been done on the KDD 99 dataset, most of which include the use of a hybrid anomaly and misuse detection model done in parallel with each other. In order to further classify the intrusions, our approach to network intrusion detection includes the use of two different anomaly detection models followed by misuse detection applied to the combined output obtained from the previous step. The end goal of this is to verify the anomalies detected by the anomaly detection algorithm and clarify whether they are actually intrusions or random outliers from the trained normal (and thus to try and reduce the number of false positives). We aim to detect a pattern in this novel intrusion technique itself, and not the handling of such intrusions. The intrusions were detected to a very high degree of accuracy.

Anomaly detection in big data is a key problem in the big data analytics domain. In this paper, the definitions of anomaly detection and big data were presented. Due to the sampling and storage burden and the inadequacy of privacy protection of anomaly detection based on uncompressed data, compressive sensing theory was introduced and used in the anomaly detection algorithm. The anomaly detection criterion based on wavelet packet transform and statistic process control theory was deduced. The proposed anomaly detection technique was used for through-wall human detection to demonstrate the effectiveness. The experiments for detecting humans behind a brick wall and gypsum based on ultra-wideband radar signal were carried out. The results showed that the proposed anomaly detection algorithm could effectively detect the existence of a human being through compressed signals and uncompressed data.

The traditional recommendation algorithm for e-commerce has the problems of single product category and inconsistent recommendation products with user preferences, resulting in the lack of diversity and accuracy. An Intelligent Recommendation Algorithm for enhancing diversity of sample nodes in commercial system is proposed. After obtaining the user's past operation records, an attribute of the recommendation target is selected by calculating the constructed user preference matrix. Through user similarity calculation, similar users are screened and specific targets are further screened to achieve the function of recommendation algorithm. In order to verify the accuracy and diversity, a simulation comparison experiment is carried out with the traditional algorithm. Compared with the traditional algorithm, the intelligent algorithm with enhanced diversity of sample nodes in e-commerce system can improve the accuracy and diversity of the two indicators, and can be extended.

Anomaly detection in streaming data is an important task for many real-world applications, such as network security, fraud detection, and system monitoring. However, streaming data often exhibit concept drift, which means that the data distribution changes over time. This poses a significant challenge for many anomaly detection algorithms, as they need to adapt to the evolving data to maintain high detection accuracy. Existing streaming anomaly detection algorithms lack a unified evaluation framework that validly assesses their performance and robustness under different types of concept drifts and anomalies. In this paper, we conduct a systematic technical review of the state-of-the-art methods for anomaly detection in streaming data. We propose a new data generator, called SCAR (Streaming data generator with Customizable Anomalies and concept dRifts), that can synthesize streaming data based on synthetic and real-world datasets from different domains. Furthermore, we adapt four static anomaly detection models to the streaming setting using a generic reconstruction strategy as baselines, and then compare them systematically with 9 existing streaming anomaly detection algorithms on 76 synthesized datasets that have various types of anomalies and concept drifts. The challenges and future research directions for anomaly detection in streaming data are also presented.

Ship wake is an important military target, while the ocean background data in satellite images is huge, and ship wake targets are few. How to quickly and accurately find targets in large amounts of data is a valuable research topic. The ship wake target has a different detection method from the general target, as it has the characteristics of uniform background and prominent line characteristics. In this paper, we design a ship wake detection method which can quickly screen out easily detected samples by using traditional algorithms including Gaussian filtering, Canny edge detection and probabilistic Hough transform. We used typical representative YOLOv5 and Faster-R-CNN target detection frameworks to train single-stage and two-stage target detection models. In the process of integrating traditional algorithms into ship wake detection, we have built a ship wake target detection system that combines traditional algorithms with neural network methods. The experiment shows that the improved traditional wake detection algorithm can effectively improve the efficiency of ship wake detection system in target detection task.

Traditional Adaboost face detection algorithm uses Haar-like features training face classifiers, whose detection error rate is low in the face region. While under the complex background, the classifiers will make wrong detection easily to the background regions with the similar faces gray level distribution, which leads to the error detection rate of traditional Adaboost algorithm is high. As one of the most important features of a face, skin in YCgCr color space has good clustering. We can fast exclude the non-face areas through the skin color model. Therefore, combining with the advantages of the Adaboost algorithm and skin color detection algorithm, this paper proposes Adaboost face detection algorithm method that bases on YCgCr skin color model. Experiments show that, compared with traditional algorithm, the method we proposed has improved significantly in the detection accuracy and errors.

As robotics technologies evolve rapidly, path planning has emerged as a fundamental pillar underpinning autonomous navigation systems. Traditional algorithms such as A*, Dijkstra, and Bellman-Ford demonstrate competence in structured environments, yet exhibit notable limitations in dynamic and uncertain scenarios. In contrast, intelligent algorithmsincluding Genetic Algorithms (GA), Reinforcement Learning (RL), and hybrid strategiesprovide measurable advantages in adaptability, learning capacity, and robustness. This study centers on comparing traditional and intelligent control algorithms for path planning from a theoretical perspective, outlining their respective principles, performance traits, and application contexts. The findings suggest that intelligent algorithms, owing to their capacity for real-time learning, environmental adaptation, and heuristic generalization, consistently outperform traditional approaches in dynamic and unstructured operational contexts. The paper offers critical theoretical insights to guide future algorithm selection and optimization for robot navigation. Furthermore, the paper discusses real-world implementation challenges, such as computational complexity, sensor uncertainty, and algorithm convergence stability, to highlight the applicability of these techniques in complex systems.

Anomaly detection has been extensively studied in the last two decades. Features are usually selected or created at first for characterizing behaviours of networks, users or systems, and then anomaly detection algorithms are developed and applied. However, such traditional network anomaly detection systems detect a limited number of attack types and report a huge number of false alarms due to lack of proper features and inherent weakness of anomaly detection algorithms. In order to minimize the number of false alerts and maximize the detection accuracy, we propose in this chapter an enhanced CUSUM algorithm for network anomaly detection, modelling various features from different sources and reporting alerts according to some decision strategies. The experimental evaluation with the 1999 DARPA intrusion detection evaluation dataset shows an empirical study of applying the proposed enhanced CUSUM algorithm for detecting successfully some network attacks.

Network anomaly detection has become the promising aspect of intrusion detection. The existing anomaly detection models depict the detection profiles with a static way, which lack good adaptability and interoperability. Furthermore, the detection rate is low, so they are difficult to be deployed the realtime detection under the high-speed network environment. In this paper, the excellent mechanisms of self-learning and adaptability in the human immune system are referred and a dynamic anomaly detection algorithm with immune negative selection, named as DADAI, is proposed. The concepts and formal definitions of antigen, antibody, and memory cells in the network security domain are given; the dynamic clonal principle of antibody is integrated; the mechanism of immune vaccination is discussed, and the dynamic evolvement formulations of detection profiles are established (including the detection profiles' dynamic generation and extinction, dynamic learning, dynamic transformation, and dynamic self-organization), which will accomplish that the detection profiles dynamically synchronize with the real network environment. Both our theoretical analysis and experimental results show that DADAI is a good solution to network anomaly detection, which increase the veracity and timeliness on anomaly detection problem.

In this article, we propose an online and unsupervised anomaly detection algorithm for streaming data using an array of sliding windows and the probability density-based descriptors (PDDs) (based on these windows). This algorithm mainly consists of three steps: 1) we use a main sliding window over streaming data and segment this window into an array of nonoverlapping subwindows; 2) we propose the PDDs with dimension reduction, based on the kernel density estimation, to estimate the probability density of data in each subwindow; and 3) we design the distance-based anomaly detection rule to determine whether the current observation is anomalous. The experimental results and performances are presented based on the Numenta anomaly benchmark. Compared with the anomaly detection algorithm using the hierarchical temporal memory proposed by Numenta (which outperforms a wide range of other anomaly detection algorithms), our algorithm can perform better in many cases, that is, with higher detection rates and earlier detection for contextual anomalies and concept drifts.

The detection of anomalies in network traffic is a crucial issue affecting the security of Internet users. A statistical network anomaly detection algorithm is a promising way of detecting such anomalies, however, it has to be given appropriate parameters for accurate detection and identification. In general, it is very difficult to obtain appropriate parameter settings a priori, because network traffic is not stable in time or space. Thus, although many anomaly detection methods have been proposed, there has been little discussion about their parameter tunings. In this paper, we investigate an automatic and dynamic parameter tuning of a statistical network traffic anomaly detection method. In particular, we clarify whether one can consistently use the best parameter fixed for a certain instance; this choice clearly depends on the macroscopic and dynamic behavior of Internet traffic anomalies. We ascertain the appropriate learning period for setting a parameter of an anomaly detection algorithm based on a sketch and multi-scale gamma-function model by using real network traces measured in a trans-Pacific link over a period of six months. The main results of our study are as follows: (1) Without learning, the best parameter varies day by day. (2) With a longer learning period, the best parameter setting is affected by significant data during the learning period. (3) The appropriate period of the learning is about 3 days. (4) The performance degradation from introducing dynamic parameter tuning is 17% in the best case.

Anomaly detection has been widely studied and used in diverse applications. Building an effective anomaly detection system requires the researchers/developers to learn the complex structure from noisy data, identify the dynamic anomaly patterns and detect anomalies while lacking sufficient labels. Recent advancement in deep learning techniques has made it possible to largely improve anomaly detection performance compared to the classical approaches. This tutorial will help the audience gain a comprehensive understanding of deep learning-based anomaly detection techniques in various application domains. First, it introduces what is the anomaly detection problem, the approaches taken before the deep model era and the challenges it faced. Then it surveys the state-of-the-art deep learning models extensively and discusses the techniques used to overcome the limitations from traditional algorithms. Second to last, it studies deep model anomaly detection techniques in real world examples from LinkedIn production systems. The tutorial concludes with a discussion of future trends.