Intrusion detection based on system calls and homogeneous Markov chains

As an important active defense technique for computer networks, intrusion detection has received lots of attention in recent years. However, the performance of current intrusion detection systems (IDSs) is far from being satisfactory due to the increasing number of complex sequential attacks. Aiming at the above problem, in this paper, a novel kernel-based reinforcement learning method for sequential behavior modeling in host-based IDSs is proposed. Based on Markov process modeling of host-based intrusion detection using sequences of system calls, the performance optimization of IDSs is transformed to a sequential prediction problem using evaluative reward signals. By using the kernel-based learning prediction algorithm, i.e., the kernel least-squares temporal-difference (kernel LS-TD) algorithm, which implements LS-TD learning in a kernel-induced feature space, the nonlinear modeling and prediction problem for sequential behaviors in IDSs is efficiently solved. Experiments on system call data from the University of New Mexico illustrate that the proposed kernel-based RL approach can achieve better detection accuracy than previous sequential behavior modeling methods including Hidden Markov Models (HMMs) and linear TD algorithms.

The Host Based Intrusion Detection System (HIDS) is to prevent the host system from being compromised by intruders. To prevent the execution of malicious codes on the host, HIDS monitors the system audit and event logs. But the design of HIDS is very challenging due to the presence of high false alarm rate. This paper mainly focuses on reducing the problem of false alarm rate, using semantic based system call patterns. Here, we make use of the semantic approach to apply on the underlying kernel level system calls which can help understand the anomaly behavior. The semantic tool used is the data dictionary. The data dictionary containing every possible combinations of sequence of system call names of particular phrase length was constructed. The features satisfying the semantic hypothesis are extracted and then normalized. The normalized values are then given as input to the decision engine. The decision engine used is the Extreme Learning Machine - a new type of neural network. Performance was evaluated using the modern ADFA-LD dataset.

Intrusion Detection systems are used for detecting attacks on a system. The host-based intrusion detection system (HIDS) detect the ongoing attacks on a Host system. HIDS model is proposed using System Call Analysis consisting of two modules, an Anomaly Detection module and a Multi-HMM module for state prediction. Anomaly Detection module uses Long Short-term memory (LSTM) architecture, a special type of Recurrent Neural Network, for detection of anomalies in system call traces. It models the normal behaviour of the system using system call patterns which enables it to detect even ‘Zero-day’ attacks. The State prediction module is based on Multiple Hidden Markov Model (Multi-HMM), in which each HMM model a known attack. It takes a sequence of system calls as input and predicts next ‘N’ most probable system calls during the attack. After performing a number of experiments, results show that the model has high recognition rate and low false alarm rate.

In the field of host-based intrusion detection systems(HIDS), existing anomaly detection algorithms paid much attention to extracting system call features, such as N-gram, frequency-based and neural networks, whereas few literatures introduce effective methods of modeling system call sequences with semantic features. This paper proposes a new model to represent system call sequences with novel applications of embedding techniques. The method converts system calls into embedding vectors as inputs for anomaly detector, mainly includes two parts: 1) construct embedding vectors for all system calls; 2) model the sequences with system call embedding and weighting. This sequence representation model is intuitive and effective on ADFA-LD dataset. As is illustrated with our experiment, the FPR can be significantly reduced to 0.53%, while the TPR still reaches 91.7%, even by a simple 1-NN classifier.

Host based Intrusion Detection Systems (HIDSs) analyze the sequence of system calls in the trace and audit log files to identify intrusive system processes. HIDSs use the frequency analysis of n-gram terms in the system call traces to identify intrusive processes. However, they are computation and resource intensive as they need to analyze a large number of n-gram input features to differentiate between normal and intrusive system processes. This put a severe limitation on their real time application while analyzing voluminous system call traces. To address this issue, we propose a computation efficient HIDS framework that initially transforms the system call traces to n-gram vector representational model and then uses a dimensionality reduction process to reduce the size of the input feature vectors. The dimensionality reduced n-gram feature vectors are finally analyzed by various machine learning based classifier models to identify intrusive processes. Performance evaluation of the proposed HIDS framework on the benchmark Australian Defense Force Academy Linux Dataset (ADFA-LD) shows that it effectively detects intrusive system processes with high accuracy and low false positive rate, while at the same time incur a minimum computational overhead.

Despite the benefits containerization brings, threats and risks of attacks against containerized technology have grown in equal proportion to its adoption. Intrusion Detection Systems (IDS) have been employed to secure cloud and container environments. However, the inherent characteristics of these environments have presented new challenges to ensuring an adequate level of security. In this paper, a framework is proposed for implementing a Host-based Intrusion Detection System (HIDS) by analyzing system calls with machine learning on a Kubernetes container orchestration cluster. The presented framework prevents the overhead of the cluster nodes from processing focused on intrusion detection through a distributed and scalable architecture. Alerts generated in the occurrence of detected anomalies can be used as a complementary source of information for decision making and action by the Security Operations Center (SOC) team to deal with an eventual security incident. The proposed architecture was implemented in the GNS3 software, emulating a corporate network environment to demonstrate the feasibility of implementing the framework in a real environment.

Intrusion detection has emerged as an important approach to network security. A new method for anomaly intrusion detection is proposed based on linear prediction and Markov chain model. Linear prediction is employed to extract features from system calls sequences of the privileged processes which are used to make up of the character database of those processes, and then the Markov chain model is founded based on those features. The observed behavior of the system is analyzed to infer the probability that the Markov chain model of the norm profile supports the observed behavior. A low probability of support indicates an anomalous behavior that may result from intrusive activities. The experiments show this method is effective and efficient, and can be used in practice to monitor the computer system in real time.

Stealthy malware hides its presence from the users of a system by hooking the relevant libraries, drivers, system calls or manipulating the services commonly used to monitor system behaviour. Tampering the network sensors of host-based intrusion detection systems (HIDS) may impair their ability to detect malware and significantly hinders subsequent forensic investigations. Nevertheless, the mere attempt to hide the traffic indicates malicious intentions. In this paper we show how comparison of the data collected by multiple sensors at different levels of resilience may reveal these intentions. At the lowest level of resilience, information from untrusted sensors such as netstat and process lists are used. At the highest resilience level, we analyse mirrored traffic using a secured hardware device. This technique can be considered as fully trusted. The detection of a discrepancy between what is reported by these common tools and what is observed on a trusted system operating at a different level is a good way to force a dilemma on malware writers: either apply hiding techniques, with the risk that the discrepancy is detected, or keep the status of network connections untouched, with a greater ability for the administrator to recognize the presence and to understand the behaviour of malware. The proposed method was implemented on an evaluation testbed and is able to detect stealthy malware that hides its communication from the HIDS. The false positive rate is 0.01% of the total traffic analysed, and barring a few exceptions that can easily be white-listed, there are no legitimate processes which raise false alerts.

: Network-based intrusion detection systems (NIDSs) are one component of a comprehensive network security solution. The use of IPsec, which encrypts network traffic, renders network intrusion detection virtually useless unless traffic is decrypted at network gateways. One alternative to NIDSs, host-based intrusion detection systems (HIDSs), provides some of the functionality of NIDSs but with limitations. HIDSs cannot perform a network-wide analysis and can be subverted if a host is compromised. We propose an approach to intrusion detection that combines HIDS, NIDS, and a version of IPsec that encrypts the header and the body of IP packets separately. We refer to the latter generically as TwoKey IPsec. We show that all of the network events currently detectable by the Snort NIDS on un- encrypted network traffic are also detectable on encrypted network traffic using this approach. The NIDS detects network-level events that HIDSs have trouble detecting and HIDSs detect application-level events that can't be detected by the NIDS.

Application features such as port numbers are used by network-based intrusion detection systems (NIDSs) to detect attacks coming from networks. System calls and the operating system related information are used by host-based intrusion detection systems (HIDSs) to detect intrusions towards a host. However, the relationship between hardware architecture events and denial-of-service (DoS) attacks has not been well revealed. When increasingly sophisticated intrusions emerge, some attacks are able to bypass both the application and the operating system level feature monitors. Therefore, a more effective solution is required to enhance existing HIDSs. In this paper, we identify the following hardware architecture features: instruction count, cache miss, bus traffic and integrate them into a novel HIDS framework based on a modern statistical gradient boosting trees model. Through the integration of application, operating system and architecture level features, our proposed HIDS demonstrates a significant improvement of the detection rate in terms of sophisticated DoS intrusions.

The Intrusion Detection System (IDS) is a useful infiltration detection tool. With IDS, it may mechanically categorize intrusions, attacks, or breaches of security stratagems across host-level within the network and within the organization. The Krill Herd Optimized Deep Neural Network (KH-DNN) is proposed in this document as a multiclass cyberattack methodology. This model is suggested by uniting the network-based intrusion detection system (NIDS) and the host-based intrusion detection system (HIDS) to advance flexible and actual IDS to categorize and detect unexpected and unpredictable cyber attacks. In this method, two types of IDS are classified, that is, NIDS and HIDS. Classification of the system call is exploited to discriminate behavior into normal and attack categories. In this manuscript accuracy, the parameter of the deep neural network classifier is optimized with the Krill Herd optimization algorithm. Finally, the DNN model is performed in the NIDS dataset and the HIDS dataset. For the HIDS dataset, the KDD Cup 99 dataset was used, which employed two datasets: ADFA-LD and ADFA-WD, and several types of attacks were detected and classified, including Adduser, Hydra-FTP, Java-Meterpreter, Hydra-SSH, and Web-Shell. The suggested IDS-KH-DNN algorithm attains higher accuracy 42.56% and 23.4%, high precision 84.74% and 52.43%, high F-score 53.5% 64.455, high sensitivity 33.4% and 45.23%, and high specificity 31.45% and 44.23% for the NIDS dataset. The proposed system is compared to two current processes: intrusion detection using Gray Wolf optimization-based support vector machine (IDS-GWO-SVM) and intrusion detection using artificial bee colony optimization algorithm based support vector machine (IDS-ABCO-SVM). Finally, the simulation results show that the suggested approach is capable of quickly and accurately locating optimal solutions.

The increasing prevalence of cybercrime and cyber-attacks underscores the imperative need for organizations to implement robust network security measures. Nevertheless, current Intrusion Detection Systems (IDS) often rely on single-sensor or multi-sensor in the same type of IDS, including Host-Based IDS (HIDS) or Network-Based IDS (NIDS), which inherently possess limited detection capabilities. To address this limitation, this research combines NIDS and HIDS components into a collaborative-IDS system, thus expanding the scope of intrusion detection and enhancing the efficacy of the established attack mitigation system. However, the integration of NIDS and HIDS introduces formidable challenges, notably the elevated rates of False Positive and False Negative alerts. To surmount these challenges, the researcher employs machine learning techniques in the form of Snort plugins and comparison methods to heighten the precision of attack detection. The obtained results unequivocally illustrate the effectiveness of this approach. Using a Support Vector Machine for static analysis of the NSL-KDD dataset attains an outstanding 99% detection rate for Denial of Service (DoS) attacks and an impressive 98% detection rate for Probe attacks. Furthermore, in dynamic real-time attack simulations, the machine learning plugins exhibit remarkable proficiency in detecting various types of DoS attacks, concurrently offering more comprehensive identification of SYN Flooding DoS attacks compared to the Snort community rules set. These findings signify a significant advancement in intrusion detection, paving the way for more robust and accurate network security systems in an era of escalating cyber threats.

SummaryCloud's operating‐system‐level virtualization has introduced a new phase of lightweight virtualization through containers. The architecture of cloud‐native and microservices‐based application development strongly advocates for the use of containers due to their swift and convenient deployment capabilities. However, the security of applications within containers is important, as malicious or vulnerable content could jeopardize the container and the host system. This vulnerability also extends to neighboring containers and may compromise data integrity and confidentiality. The article focuses on developing an intrusion detection system tailored to containerized cloud environments by identifying system call analysis techniques and also proposes an anomaly‐based host intrusion detection system (Ab‐HIDS). This system employs the frequency of N‐grams system calls as distinctive features. To enhance performance, two ensemble learning models, namely voting‐based ensemble learning and XGBoost ensemble learning, are employed for training and testing the data. The proposed system is evaluated using the Leipzig Intrusion Detection Data Set (LID‐DS), demonstrating substantial performance compared to existing state‐of‐the‐art methods. Ab‐HIDS is validated for class imbalance using the imbalance ratio and synthetic minority over‐sampling technique methods. Our system achieved significant improvements in detection accuracy with 4% increase for the voting‐based ensemble model and 6% increase for the XGBoost ensemble model. Additionally, we observed reductions in the false positive rate by 0.9% and 0.8% for these models, respectively, compared to existing state‐of‐the‐art methods. These results illustrate the potential of our proposed approach in improving security measures within containerized environments.

In this thesis, the properties of some non-standard Markov chain models and their corresponding parameter estimation methods are investigated. Several practical applications and extensions are also discussed.
\n
\nThe estimation of model parameters plays a key role in the real-world applications of Markov chain models. Some widely used estimation methods for Markov chain models are based on the existence of stationary vectors. In this thesis, some weaker sufficient conditions for the existence of stationary vectors for highorder Markov chain models, multivariate Markov chain models and high-order multivariate Markov chain models are proposed. Furthermore, for multivariate Markov chain models, a new estimation method based on minimizing the prediction error is proposed. Numerical experiments are conducted to demonstrate the efficiency of the proposed estimation methods with an application in demand prediction.
\n
\nHidden Markov Model (HMM) is a bivariate stochastic process such that one of the process is hidden and the other is observable. The distribution of observable sequence depends on the hidden sequence. In a traditional HMM, the hidden states directly affect the observable states but not vice versa. However, in reality, observable sequence may also have effect on the hidden sequence. For this reason, the concept of Interactive Hidden Markov Model (IHMM) is introduced, whose key idea is that the transitions of the hidden states depend on the observable states too. In this thesis, efforts are devoted in building a highorder IHMM where the probability laws governing both observable and hidden states can be written as a pair of high-order stochastic difference equations. We also propose a new model by capturing the effect of observable sequence on the hidden sequence through using the threshold principle. In this case, reference probability methods are adopted in estimating the optimal model parameters, while for unknown threshold parameter, Akaike Information Criterion (AIC) is used. We explore asset allocation problems from both domestic and foreign perspective where asset price dynamics follows autoregressive HMM. The object of an investor is not only to maximize the expected utility of the terminal wealth, but also to ensure that the risk of the portfolio described by the Value-at-Risk (VaR) does not exceed a specified level.
\n
\nIn many decision processes, fuzziness is a major source of imprecision. As a perception of usual Markov chains, the definition of fuzzy Markov chains is introduced. Compared to traditional Markov chain models, fuzzy Markov chains are relatively new and many properties of them are still unknown. Due to the potential applications of fuzzy Markov chains, we provide some characterizations to ensure the ergodicity of these chains under both max-min and max-product compositions.

Simple SummaryUnderstanding and predicting vector population and related disease dynamics, is crucial for gaining insight into the abundance and dynamics of arthropod disease vectors, and for the design of effective vector control strategies. Several mathematical and standard epidemiological models have been proposed in studying vector transmitted infectious disease dynamics. However, most models are of deterministic nature and are not able to estimate other relevant metrics such as the probability of vector population emergence as well as the probability and expected time to reach certain population and/or infection state. Here we are focusing on stochastic modeling of mosquito abundance data using weather driven Markov chains (MCs) and are particularly interested in estimating transition probabilities (TPs) between different population levels. A MC model is based on the assumption that the future state of the variable is only dependent on the present state and is suitable in cases of short and noisy data characterized by a complex and random behavior. The aim is to introduce and generalize a formulation of conditional Markov chain models (CMSs) for predicting probability transition estimates of arthropod vector populations. In this context, first we present the basic principles and assumptions behind Markov chain modeling approach, with an intuitive interpretation of the integration of conditional Markov chains (CMCs) and then demonstrate the usefulness of the approach in predicting the abundance of Culex sp. We conclude that the conditional Markov chain technique is recommended as viable for modeling populations that explicit random dynamics and predict their future evolution. Although, the Markov models generated in this work provide an accurate abstraction of the vector disease progress observed within the dataset used for their generation, we envision the current approach as an entry point into the medical entomology literature and methods for predicting arthropod vector diseases dynamics.Understanding and predicting mosquito population dynamics is crucial for gaining insight into the abundance of arthropod disease vectors and for the design of effective vector control strategies. In this work, a climate-conditioned Markov chain (CMC) model was developed and applied for the first time to predict the dynamics of vectors of important medical diseases. Temporal changes in mosquito population profiles were generated to simulate the probabilities of a high population impact. The simulated transition probabilities of the mosquito populations achieved from the trained model are very near to the observed data transitions that have been used to parameterize and validate the model. Thus, the CMC model satisfactorily describes the temporal evolution of the mosquito population process. In general, our numerical results, when temperature is considered as the driver of change, indicate that it is more likely for the population system to move into a state of high population level when the former is a state of a lower population level than the opposite. Field data on frequencies of successive mosquito population levels, which were not used for the data inferred MC modeling, were assembled to obtain an empirical intensity transition matrix and the frequencies observed. Our findings match to a certain degree the empirical results in which the probabilities follow analogous patterns while no significant differences were observed between the transition matrices of the CMC model and the validation data (ChiSq = 14.58013, df = 24, p = 0.9324451). The proposed modeling approach is a valuable eco-epidemiological study. Moreover, compared to traditional Markov chains, the benefit of the current CMC model is that it takes into account the stochastic conditional properties of ecological-related climate variables. The current modeling approach could save costs and time in establishing vector eradication programs and mosquito surveillance programs.