A Three-Stage Anomaly Detection Framework for Traffic Videos

The article examines the problem of detecting anomalies in the network traffic of information systems of organizations. Detection of anomalies in network traffic will allow to determine the hidden malicious activity of the data obtained on the basis of protocols that collect statistical data of the network traffic of the information system. This, in turn, will allow you to reduce the load and configure the attributes that will be used to monitor and analyze network traffic. The authors proposed the network traffic anomaly detection architecture, which is divided into functional levels. Protocols were analyzed to collect statistics, namely the Net Flow/IPFIX protocol, which provides comprehensive information based on packet headers. To process and analyze the received data, the authors developed a model for detecting anomalies in the traffic of the information system. The anomaly detection model uses statistical data for their further processing, as well as the possibility of storing data in a repository. All received data is filtered to detect malicious processes, transferred and stored in the repository of the attack database with the possibility of creating warnings and identifying the attack. For the specified model, the use of Machine Learning based on methods of predicting categorical fields is proposed. The work used a dataset with firewall data, which contains information on the number and size of transmitted and received packets of packets, data on the use of malicious software. Using the method, an experimental study of the data was conducted to predict the presence of malicious software in them. The method of forecasting categorical fields using Logistic Regression, SVM, Random Forest Classifier and other classification algorithms was investigated. Based on the obtained data, a confusion matrix was built, which allows to estimate the error of the algorithms.

We study the feasibility of developing a Communication Supervision Function for a 4G LTE wireless communications network system, to allow a vertical to monitor from its domain the Quality of Service (QoS) of the communication traversing the wireless domain. Communication supervision is performed by detecting traffic anomalies of a reference, healthy, transmission of packets uniformly spaced at intervals with ms resolution, and transmitted in uplink direction. Traffic at the base station is monitored with a LTE Medium Access Control (MAC) layer monitoring tool that aggregates traffic at intervals with seconds resolution. Measurements are performed in an operating LTE network. We use a deep learning method implementing a Long Short-Term Memory (LSTM) Recurrent Neural Network (RNN), to determine if the traffic pattern is the healthy one, or it is anomalous, with missing packets and jitter. We identify key metrics in the monitoring data, that are selected as features in the RNN, which enable the detection of fine time resolution traffic anomalies hidden in the aggregated and coarse measurements reported by the monitoring tool. We find that applying the proposed approach, a vertical is able to determine whether the communication over the wireless network is healthy or anomalous. Finally, we discuss on the use of the proposed monitoring approach in 4G networks, and learning possibilities for 5G standardization in terms of monitoring metrics, features, monitoring resolution, service concepts, etc.

Ego-centric Traffic Anomaly Detection (TAD) aims to identify abnormal events in videos captured by dashboardmounted cameras in vehicles. Compared to anomaly detection in roadside surveillance videos, ego-centric TAD poses greater challenges due to the dynamic backgrounds caused by vehicle motion. Previous frame-level methods are often vulnerable to interference from these dynamic backgrounds and struggle to detect small objects located at a distance or off-center. To address these challenges, we propose an object-prompt enhanced method that integrates detected traffic objects into a framelevel TAD framework. Our approach introduces an objectprompt scheme comprising an object prompt encoder, along with two cross-attention-based aggregation modules: an instancewise aggregation module for fusing information between object instances and the scene, and a relation-wise aggregation module for capturing relationships inter-objects. Additionally, we design an instance-level loss to supervise anomaly detection at the object level. Our method effectively mitigates interference from dynamic backgrounds, improves the detection of distant or offcenter anomalies, and enables precise spatial localization of anomalies. Experimental results on the DoTA and DADA-2000 datasets demonstrate that our method achieves state-of-the-art performance.

Detecting anomalies in the Traffic Systems could be very useful for the analysis of traffic rule violation, fault detection and other traffic-related issues. In this paper trajectory-based anomaly detection using spatial temporal analysis, K-means, linear regression, z score and Hierarchical temporal memory clustering algorithm are analyzed. The spatial localization of an object is considered as an event. Traffic anomaly detection rules are formulated in three levels: Point anomaly, Sequential anomaly and Co-occurrence anomaly. This paper analyses the performance of various traffic anomaly detection methodologies in terms of accuracy to reduce false alarm.

Traffic traces captured from backbone links have been widely used in traffic analysis for many years. By far the most popular use of such traces is replay where conditions and states of the original traffic trace are recreated almost identically in simulation or emulation environments. When the end target of such research is detection of traffic anomalies, it is crucial that some anomalies are found in the trace in the first place. Traces with many real-life anomalies are rare, however. This paper pioneers a new area of research where traffic traces are engineered to contain traffic anomalies as per user request. The method itself is non-intrusive by retaining the IP address space found in the original trace. Engineering of several popular anomalies are shown in the paper while the method is flexible enough to accommodate any level of traffic trace engineering in the future.KeywordsTrafficAnalysisTraceReplaySimulationEmulationAnomaly

Network Traffic Anomalies, Natural Language Processing, and Random Matrix Theory

With the rapid development of information network, network security is becoming more and more important. Intrusion detection is an important component of the network security system. The traditional signature-based matching detection method is difficult to cope with the increasingly complex network environment. On the contrary, anomaly detection which is based on network traffic pattern analysis has obvious advantages in dealing with encryption attacks, zero-day attacks and other new attacks. This paper studies the network traffic anomaly detection, and proposes a traffic anomaly detection model which combines convolution neural network and eXtreme Gradient Boosting algorithm. First, the collected traffic data is preprocessed into appropriate format that meets the input requirements of the model. Then the improved LeNet-5 convolution neural network is used for feature learning, and finally XGBoost algorithm is used to classify the learning features. Experimental results show that the proposed network traffic anomaly detection method based on CNN and XGBoost has a high accuracy, and good experimental results have been achieved in both two- classification and multi-classification.

Spatiotemporal relationships are widely used in urban traffic modelling for obtaining accurate and explainable forecasts. However, the majority of studies are focused on point forecasts and do not pay attention to a time-dependent variability of traffic flows and related volatility of model errors. At the same time, traffic volatility is an important value that can be effectively used for the construction of confidence intervals for traffic forecasts and detection of traffic anomalies. This study proposes an application of the spatiotemporal approach to modelling of traffic volatility and predicting the conditional variance of traffic forecasts. The approach is based on a spatiotemporal specification of the popular generalized autoregressive conditional heteroskedasticity (GARCH) model that is combined with the spatially regularised vector autoregressive model as a multivariate urban traffic predictor. The resulting hybrid model was applied for a large real-world data set and demonstrated its performance for traffic volatility prediction. The proposed approach was compared to existing univariate traffic volatility models and proved its utility for deeper understanding of traffic variance and anomalies.

Traffic anomaly detection using high performance measurement systems offers the possibility of improving the speed of detection and enabling detection of important, short-lived anomalies. In this paper we investigate the problem of detecting anomalies using traffic measurements with fine-grained timestamps. We develop a new detection algorithm (called S3) that utilizes a Bayes Net to efficiently consider multiple input signals and to explicitly define what is considered "anomalous''. The input signals considered by S3 are traffic volumes and correlations between ingress/egress packet and bit rates. These complementary signals enable identification of an expanded range of anomalies. Using a set of high precision traffic measurements collected at our campus border router over a 10 month period and an annotated anomaly log supplied by our network operators, we show that S3 is highly accurate, identifying 86% of the anomalies listed in the log. Compared with well known time series-based and wavelet-based detectors, this represents over a 20% improvement inaccuracy. Investigation of events identified by S3 that did not appear in the operator log indicate many are, in fact, true positives. Deployment of S3 in an operational environment supports this by showing zero false positives during initial tests.

Internet has become an essential part of the daily life for billions of users worldwide, who are using a large variety of network services and applications everyday. However, there have been serious security problems and network failures that are hard to resolve, for example, botnet attacks, polymorphic worm/virus spreading, DDoS, and flash crowds. To address many of these problems, we need to have a network-wide view of the traffic dynamics, and more importantly, be able to detect traffic anomalies in a timely manner. Spatial analysis methods have been proved to be effective in detecting network-wide traffic anomalies that are not detectable at a single monitor. To our knowledge, Principle Component Analysis (PCA) is the best-known spatial detection method for the coordinated low-profile traffic anomalies. However, existing PCA-based solutions have scalability problems in that they require linear running time and space to analyze the traffic measurements within a sliding window, which makes it often infeasible to be deployed for monitoring large-scale high-speed networks. We propose a sketch-based streaming PCA algorithm for the network-wide traffic anomaly detection in a distributed fashion. Our algorithm only requires logarithmic running time and space at both local monitors and Network Operation Centers (NOCs), and can detect both high-profile and coordinated low-profile traffic anomalies with bounded errors.

The Space-ground integrated network consists of deep space network, space network and ground network. The ground network is vulnerable to network attacks due to its complex networking structure, high randomness of device access, limited node processing capacity and computing resources. This paper focuses on traffic anomaly detection from the perspective of Space-ground integrated network security. We mine the characteristics of traffic data and use the GPT method to detect abnormal traffic. We suggest incorporating GPT-style loss as a semi-supervised auxiliary term to aid in training. This paper solves the problem that 1) Traditional RNN and LSTM methods have insufficient context detection performance and cannot be parallelized. 2) Improve the traffic classification performance with limited labeled data. The network simulation software CORE is used to simulate the Space-ground network structure. We inject DOS attacks into the nodes of the ground network and conduct binary classification experiments on abnormal traffic. Experiments demonstrate the proposed method greatly reduces the requirements of labeled data in network anomaly detection. On simulation dataset, given only 5% training labels, our approach achieves a significant 98.92% accuracy and 98.74% F1-score.

Research and Application of Network Anomaly Traffic Detection System

As the development of crowdsourcing technique, acquiring amounts of data in urban cities becomes possible and reliable, which makes it possible to mine useful and significant information from data. Traffic anomaly detection is to find the traffic patterns which are not expected and it can be used to explore traffic problems accurately and efficiently. In this paper, we propose LoTAD to explore anomalous regions with long-term poor traffic situations. Specifically, we process crowdsourced bus data into TS-segments (Temporal and Spatial segments) to model the traffic condition. Later, we explore anomalous TS-segments in each bus line by calculating their AI (Anomaly Index). Then, we combine anomalous TS-segments detected in different lines to mine anomalous regions. The information of anomalous regions provides suggestions for future traffic planning. We conduct experiments with real crowdsourced bus trajectory datasets of October in 2014 and March in 2015 in Hangzhou. We analyze the varieties of the results and explain how they are consistent with the real urban traffic planning or social events happened between the time interval of the two datasets. At last we do a contrast experiment with the most ten congested roads in Hangzhou, which verifies the effectiveness of LoTAD.

In intelligent transportation systems, effectively modeling and interpreting the complex interactions among autonomous agents, such as vehicles and pedestrians, is crucial for traffic management and safety. This paper introduces a novel generative traffic flow model that employs a generative pretrained Transformer to capture multi-agent interactions and detect anomalies in traffic patterns. We introduce a two-stage tokenization process for set-structured traffic data, efficiently encoding variable-sized agent states into fixed-length sequences suitable for generative modeling. We demonstrate that anomalies, which often indicate potential hazards or non-compliant behaviors, can be identified as deviations from learned normal interaction patterns among agents through a zero-shot detection mechanism. Our experimental results in simulated urban settings highlight the model's capability to detect various types of traffic anomalies with high accuracy. This work significantly advances agent-based traffic modeling and underscores its potential for enhancing traffic safety and efficiency in multi-agent systems.

Traffic anomaly detection in road networks is vital for patrol participants to adaptively optimize patrol routes for hybrid task scheduling. In most cases, the routine patrol routing problem comes down to offline task scheduling with a maximal coverage requirement of traffic violation hotspots. Meanwhile, emergent anomalies (e.g., accidents, crowded events) can happen with the unpredictable duration times and locations, leading to real-time requirements of online task scheduling. In practice, trading off coverage and emergency is usually inapplicable in hybrid scenarios, especially with decentralized routing decision makers in highly stochastic traffic environments. In this paper, we propose a Crowd-enhanced hybrid task scheduling framework with the balance of Coverage and Emergency, called <b>CCE-patrol</b>. CCE-patrol consists of offline violation hotspot patrolling, online emergency monitoring. For the offline subproblem, a contact-based integer linear problem modeling is proposed to adaptively find optimal patrol routes. Guided by offline routing, we identify and incorporate online emergent anomalies into traffic data completion for urban crowdsensing, with up-to-date importance estimation by multi-agent cooperative reinforcement learning. The secretary problem can be utilized to select important monitoring tasks in an online manner. Extensive experiments show the superiority of our proposals in comparison with baseline algorithms (i.e., HASH (Sun et al. 2022)) by 21.08<inline-formula><tex-math notation="LaTeX">$\%$</tex-math></inline-formula> improvement.