Ship Anomalous Behavior Detection in Port Waterways Based on Text Similarity and Kernel Density Estimation

To improve the efficiency of detecting abnormal traffic incidents on the road network and reduce the false alarm rate, a real-time traffic anomaly detection framework based on a graph spatiotemporal pattern learning (GSTPL) network is proposed. In this framework, a traffic pattern search algorithm based on a fluctuation similarity measure is designed to screen traffic flow data with the same traffic pattern, and a traffic pattern graph tuple is constructed as the input of the network model to avoid the sample imbalance problem and the effect of single-sample randomness for traffic pattern learning. Then the GSTPL network is designed to extract, unsupervised, the traffic spatiotemporal pattern features and make a reasonable prediction of future traffic parameters as the basis for anomaly evaluation. To further restrain the effect of random fluctuations in traffic flow parameters, an abnormal state evaluation method is designed to calculate the anomaly state likelihood by prediction error distribution learning. The overall detection framework realizes stable prediction of network key node traffic parameters by using spatiotemporal pattern features to construct the traffic pattern graph tuple, and gives incident evaluation results in real time by combination with the detection data. The experiment uses I90 and I405 highway traffic data in Seattle, WA, from 2015. Through comparative analysis, the proposed incident detection method based on GSTPL has a higher detection rate and lower false alarm rate, can adaptively learn dynamic changes of the traffic pattern, and has strong adaptability and stability to different traffic environments.

In today’s digital landscape, YouTube’s comment sections play an essential role in shaping public opinion and discussions. However, coordinated groups often exploit these platforms to spread misinformation and distort conversations. This study introduces a method to detect and quantify such anomalous behavior on YouTube channels by analyzing two key components: commenter behaviors and engagement patterns. Focusing on channels concerning news outlets, geopolitics, and the military, we draw from a dataset with 71 channels, 642,952 videos, 12,425,587 commenters, 123,882,200 comments, 139,985,870 subscribers, and 83,396,188,807 views. Our approach leverages key engagement indicators, such as comments, views, and subscriber counts, alongside an analysis of frequent commenter interactions to identify anomalous patterns. We apply unsupervised techniques, including kernel density estimation (KDE) and Gaussian mixture model (GMM), to assign a score reflecting anomalous commenter behavior. To provide a comprehensive measure of this behavior, we combine commenter and engagement scores both at the feature level and at the output level. At the feature level, we employ cosine similarity and principal component analysis (PCA). At the output level, we propose three scoring methods: harmonic mean (HM), weighted average with interaction term (WAIT), and agreement-weighted maximum (AWM). The resulting score, normalized between 0 and 1, indicates the level of anomalous activity on each channel. We validate our results by comparing detected anomalous channels with actual suspended channels from YouTube, confirming the model’s real-world applicability. By offering a quantifiable measure of this activity, our method helps preserve the integrity of YouTube discussions.

The onset of Web 3.0 has catalyzed the rapid advancement of social networking, transforming platforms into essential elements deeply embedded within the fabric of daily life. Researchers have proposed several methods for detecting anomalous behaviors in various scenarios. This article provides a comprehensive review of current research and the latest developments in anomalous behavior detection within social networks. We present a hierarchical three-layer categorization scheme based on the distinct characteristics of base-level detection technologies and various datasets. First, anomaly detection based on user behavioral characteristics can intuitively reflect deviations in individual behavior. However, it may overlook the overall network structure’s impact. Second, detecting anomalies within a network’s topological structure highlights structural significance, but may overlook the subtle nuances of individual behavior. Finally, the coordinated fusion method, which blends individual behavioral characteristics and the network’s topological structure, addresses the multifaceted nature of anomalies, yielding a more thorough and accurate anomaly detection strategy. This paper provides an overview and assesses the performance of three anomaly detection methods. Furthermore, we explore the challenges associated with social network anomaly detection and the potential pathways for further research.

Mobile networks and smart phones have become ubiquitous in our daily life. Large amount of customer related telecom data from various sources are generated every day, from which diversified behavior patterns can be revealed, including some anomalous behaviors that are vicious. It becomes increasingly important to achieve both efficient and effective customer behavior analysis based on the telecom big data. In this paper, the Multi-faceted Telecom Customer Behavior Analysis (MTCBA) framework for anomalous telecom customer behavior detection and clustering analysis is proposed. In this framework, we further design the hierarchical Locality Sensitive Hashing-Local Outlier Factor (hierarchical LSH-LOF) scheme for suspicious customer detection, and the Autoencoders with Factorization Machines (FM-AE) structure for dimension reduction to achieve more efficient clustering. Hierarchical LSH-LOF is an improved algorithm of LOF, in which we design a hierarchical LSH process that selects the approximate k nearest neighbors from coarse to fine by gradually narrowing down the scope. Experiments show its superiority over KD-tree w.r.t searching speed. FM-AE exploits Factorization Machines for learning second order feature interactions, which we prove to be useful by designing comparative experiments with five dimension reduction algorithms. With the proposed MTCBA framework, efficient and effective telecom customer behavior analysis including anomalous customer behavior detection and clustering analysis is performed on the real world telecom operating data provided by one of the major Internet service providers (ISPs) in China. Meanwhile, interpretable clustering results of six clusters are obtained to provide valuable information for the precision marketing of telecom operators, criminal combating, and social credit system construction.

Detecting anomalous behaviors of cloud platforms is one of critical tasks for cloud providers. Every anomalous behavior potentially causes incidents, especially some unaware and/or unknown issues, which severely harm their SLA (Service Level Agreement). Existing solutions generally monitor cloud platform at different layers and then detect anomalies based on rules or learning algorithms on monitoring metrics. However, complexity of nowadays cloud platforms, high dynamics of cloud workloads and thousands of various types of metrics make anomalous behavior detection more challenging to be applied in production, especially in large scale cloud production environments. In this paper, we present a practical cloud anomalous behavior detection system called DriftInsight. It firstly analyzes multi-denominational metrics of each component and identifies a set of representative steady components based on the convergences of their states. Then it generates a state model and a state transit model for each steady cloud component. Finally, it detects behavior anomalies of these steady components in near real-time and meanwhile evolve behavior models on the fly. The evaluation results of this approach in a commercial large-scale PaaS (Platform-as-a-Service) cloud are demonstrated its capability and efficiency.

As the second most visited website globally, YouTube serves as a central platform for video sharing, entertainment, and information dissemination. However, its expansive and highly active user base also facilitates problematic behavior, particularly among commenters. This study presents a methodology driven by social network analysis to detect and examine anomalous commenter behaviors, with a specific focus on commenter mobs that collaborate to artificially manipulate engagement metrics on videos. Additionally, the study seeks to characterize YouTube channels based on the prevalence of such behaviors, uncovering patterns of coordination among channels. The analysis utilizes a dataset comprising 47 YouTube channels, 26,901 videos, 1,377,902 commenters, and 2,496,558 comments, including 20 channels involved in disseminating misleading information about the U.S. Military and 27 additional channels, which serve as a control group to provide a baseline for normal behavior, helping to distinguish between anomalous and non-anomalous patterns more clearly. The methodology compares principal component analysis (PCA) with Graph2vec and uniform manifold approximation and projection (UMAP), in conjunction with K-means and hierarchical clustering, to identify and categorize anomalous behaviors across channels. Through comprehensive qualitative and quantitative analyses, the study identifies the themes of the videos where these anomalous behaviors occurred in comment sections. These findings provide valuable insights into the dynamics of online discourse and the mechanisms by which coordinated groups influence content and engagement on YouTube.

Business process anomalous behavior detection reveals unexpected cases from event logs to ensure the trusted operation of information systems. Anomaly behavior is mainly identified through a log-to-model alignment analysis or numerical outlier detection. However, both approaches ignore the influence of probability distributions or activity relationships in process activities. Based on this concern, this paper incorporates the behavioral relationships characterized by the process model and the joint probability distribution of nodes related to suspected anomalous behaviors. Moreover, a Petri Net-Based Bayesian Network (PN-BBN) is proposed to detect anomalous behaviors based on the probabilistic inference of behavioral contexts. First, the process model is filtered based on the process structure of the process activities to identify the key regions where the suspected anomalous behaviors are located. Then, the behavioral profile of the activity is used to prune it to position the ineluctable paths that trigger these activities. Further, the model is used as the architecture for parameter learning to construct the PN-BBN. Based on this, anomaly scores are inferred based on the joint probabilities of activities related to suspected anomalous behaviors for anomaly detection under the constraints of control flow and probability distributions. Finally, PN-BBN is implemented based on the open-source frameworks PM4PY and PMGPY and evaluated from multiple metrics with synthetic and real process data. The experimental results demonstrate that PN-BBN effectively identifies anomalous process behaviors and improves the reliability of information systems.

Internal employees have always been at the core of organizational security management challenges. Once an employee exhibits behaviors that threaten the organization, the resulting damage can be profound. Therefore, analyzing reasonably stored behavioral data can equip managers with effective threat monitoring and warning solutions. Through data-mining, a knowledge graph for internal threat data is deduced, and models for detecting anomalous behaviors and predicting resignations are developed. Initially, data-mining is employed to model the knowledge ontology of internal threats and construct the knowledge graph; subsequently, using the behavioral characteristics, the BP neural network is optimized with the Sparrow Search Algorithm (SSA), establishing a detection model for anomalous behaviors (SBP); additionally, behavioral sequences are processed through data feature vectorization. Utilizing SBP, the LSTM network is further optimized, creating a predictive model for employee behaviors (SLSTM); ultimately, SBP detects anomalous behaviors, while SLSTM predicts resignation intentions, thus enhancing detection strategies for at-risk employees. The integration of these models forms a comprehensive threat detection technology within the organization. The efficacy and practicality of detecting anomalous behaviors and predicting resignations using SBP and SLSTM are demonstrated, comparing them with other algorithms and analyzing potential causes of misjudgment. This work has enhanced the detection efficiency and update speed of abnormal employee behaviors, lowered the misjudgment rate, and significantly mitigated the impact of internal threats on the organization.

Anomalous behaviors in social networks can lead to privacy leaks and the spread of false information. In this paper, we propose an anomalous behavior detection method based on optimized graph embedding representation. Specifically, the user behavior logs are first extracted into a social network user behavior temporal knowledge graph, based on which the graph embedding representation method is used to transform the network topology and temporal information in the user behavior knowledge graph into structural embedding vectors and temporal information embedding vectors, and the hybrid attention mechanism is used to merge the two types of vectors to obtain the final entity embedding to complete the prediction and complementation of the temporal knowledge graph of user behavior. We use graph neural networks, which use the temporal information of user behaviors as a time constraint and capture both user behavioral and semantic information. It converts the two parts of information into vectors for concatenation and linear transformation to obtain a comprehensive representation vector of the whole subgraph, as well as joint deep learning models to evaluate abnormal behavior. Finally, we perform experiments on the Yelp dataset to validate that our method achieves a 9.56% improvement in the F1-score.

Robust adaptive multivariate Hotelling's T2 control chart based on kernel density estimation for intrusion detection system

The behavior of the users of a website may change so quickly that it becomes a real challenge to attempt to make predictions according to the frequent patterns coming from the analysis of an access log file. In order to reduce the obsolescence of behavioral patterns as much as possible, the ideal method would provide frequent patterns in real time, making the result immediately available. In this paper, we propose a method for finding frequent behavioral patterns in real time, whatever the number of connected users. Considering how fast frequent behavior patterns may have changed since the time the access log file was analyzed, this result thus provides completely appropriate navigation schemata for predicting user behavior. Based on a distributed heuristic, our method also tackles and provides answers to several problems within the framework of data mining: the discovery of ‘interesting zones’ (a large number of frequent patterns concentrated over a period of time, or ‘super-frequent’ patterns), discovering very long sequential patterns and interactive data mining (‘on-the-fly’ modification of the minimum support).

BackgroundCalcium imaging in insects reveals the neural response to odours, both at the receptor level on the antenna and in the antennal lobe, the first stage of olfactory information processing in the brain. Changes of intracellular calcium concentration in response to odour presentations can be observed by employing calcium-sensitive, fluorescent dyes. The response pattern across all recorded units is characteristic for the odour.MethodPreviously, extraction of odour response patterns from calcium imaging movies was performed offline, after the experiment. We developed software to extract and to visualise odour response patterns in real time. An adaptive algorithm in combination with an implementation for the graphics processing unit enables fast processing of movie streams. Relying on correlations between pixels in the temporal domain, the calcium imaging movie can be segmented into regions that correspond to the neural units.ResultsWe applied our software to calcium imaging data recorded from the antennal lobe of the honeybee Apis mellifera and from the antenna of the fruit fly Drosophila melanogaster. Evaluation on reference data showed results comparable to those obtained by previous offline methods while computation time was significantly lower. Demonstrating practical applicability, we employed the software in a real-time experiment, performing segmentation of glomeruli - the functional units of the honeybee antennal lobe - and visualisation of glomerular activity patterns.ConclusionsReal-time visualisation of odour response patterns expands the experimental repertoire targeted at understanding information processing in the honeybee antennal lobe. In interactive experiments, glomeruli can be selected for manipulation based on their present or past activity, or based on their anatomical position. Apart from supporting neurobiology, the software allows for utilising the insect antenna as a chemosensor, e.g. to detect or to classify odours.

Wireless devices operating under IEEE 802.11 x standards have the ability to transmit at various data-rates. However, there is no specification in the standard for dynamically selecting a suitable value of data-rate. Numerous solutions are available, mainly from device manufacturers and research community; focusing on a particular algorithm and its suitability for the overall system. However, none of the previous designs were flexible to change their behavior with changes in traffic nature and pattern. Not all traffic types have the same requirements and so should be the behavior of underlying algorithms for data-rate control. This paper highlights various data-rate control algorithms; classifying them on the basis of several parameters and discusses their suitability for different traffic patterns. It also proposes an algorithm, specifically designed for real time services. The cross-layer framework monitors traffic nature and patterns at the transport layer and activates one of three available rate control algorithms which suits the requirements of a particular type of traffic. According to the specific design of a selected algorithm, it then selects data-rate values at runtime. The results verify the tremendous difference among the three algorithms in terms of throughput and delay and signify their usability for specific traffic types.

When performing complex tasks, humans naturally reason at multiple temporal and spatial resolutions simultaneously. We contend that for an artificially intelligent agent to effectively model human teammates, i.e., demonstrate computational theory of mind (ToM), it should do the same. In this paper, we present an approach for integrating high and low-resolution spatial and temporal information to predict human behavior in real time and evaluate it on data collected from human subjects performing simulated urban search and rescue (USAR) missions in a Minecraft-based environment. Our model composes neural networks for high and low-resolution feature extraction with a neural network for behavior prediction, with all three networks trained simultaneously. The high-resolution extractor encodes dynamically changing goals robustly by taking as input the Manhattan distance difference between the humans’ Minecraft avatars and candidate goals in the environment for the latest few actions, computed from a high-resolution gridworld representation. In contrast, the low-resolution extractor encodes participants’ historical behavior using a historical state matrix computed from a low-resolution graph representation. Through supervised learning, our model acquires a robust prior for human behavior prediction, and can effectively deal with long-term observations. Our experimental results demonstrate that our method significantly improves prediction accuracy compared to approaches that only use high-resolution information.

A statistical approach was used to analyze situations of heavy and congested traffic in urban areas, which included both the recognition of particular locations that attract drivers in a multipoint-to-multipoint traffic structure and their classification (attractor’s value) as a function of the number of people visiting them by car in different time ranges. Attractors' distribution density values have been calculated using a Kernel Density Estimation (KDE) function, and they have been combined with nodal and crucial traffic points and traffic density in a "congestion" map. Finally, cross-comparing congestion values with the location of buildings, the road network and the Corine Land Use/Land Cover environmental classification, a “Quality of Life” map has been generated. The authors use this term because the congestion of traffic flows, with all the problems that it entails (such as long travel time, air and acoustic pollution, and so on) is a good indicator of the quality of life, especially in small towns. The findings demonstrate that this type of "off-line" analysis would permit administrators to quickly and cheaply identify areas where traffic noise and jumps have the greatest impact on citizens' quality of life and, as a result, concentrate costly ground measurements and interventions primarily there.