Guest Editorial Deep Packet Inspection: Algorithms, Hardware, and Applications

Deep packet inspection (DPI) technology has conventionally been introduced for traffic monitoring, bandwidth control, bandwidth allocation, access control of specific types of traffic, etc. By installing a DPI equipment at any arbitrary point in a network, it is possible to monitor communications of many servers simply and this could simplify dramatically the conventional work load for system management and reduction of power consumption by servers. As a new potential application of DPI technology, this paper explores the possibility of applying DPI technology to server congestion control and reduction of power consumption by servers, and presents implementation examples of these applications. This paper first proposes an example of server congestion control method, in which the DPI equipment estimates the CPU usage of each server by constantly monitoring the number of live TCP connections established by each server, and moves some of virtual machines on the congested server to other servers through remote control timely. Next, this paper proposes an example of reducing power consumption by servers, in which the DPI equipment constantly monitors the traffic sent or received in each area with multiple servers. Finally, the feasibility of the proposed methods is demonstrated by an evaluation system with a real DPI equipment.

As a prerequisite technique, Deep Packet Inspection (DPI) plays a major role to contemporary network security and management. The key of DPI is a repository of protocol fingerprints. However, inferring and maintaining up-to-date fingerprints for various and new protocols is very difficult in order to adapt them to the continuous evolution of the protocols. In this paper, we propose ProDigger, a robust automatic protocol fingerprint learning framework for DPI traffic recognition. The key insight of ProDigger is that byte embedding, a distributional vector representation of one byte with the ability of capturing the contextual information of packet payload, is prelearned from the protocol traces which is conductive to obtain an efficient numerical representation of packet payloads with different message formats or semantic information. Multiple finegrained clusters are obtained by feeding the constructed packet payload representations to the clustering algorithm. Last, by employing byte-embedding-based payload alignment algorithm to each cluster, we attain the target protocol fingerprints in the form of a series of substrings. We implement our approach and evaluate it on real-world Internet traffic traces. The experimental results demonstrate that ProDigger is capable of identifying the corresponding traffic based on the learned fingerprints and show more excellent performance in terms of Precision and Recall in comparison with the state-of-the-art approach.

Social relationship discovery of IP addresses in the managed IP networks by observing traffic at network boundary

Most network traffic analysis applications are designed to discover malicious activity by only relying on high-level flow-based message properties. However, to detect security breaches that are specifically designed to target one network (e.g., Advanced Persistent Threats), deep packet inspection and anomaly detection are indispensible. In this paper, we focus on how we can support experts in discovering whether anomalies at message level imply a security risk at network level. In SNAPS (Semantic Network traffic Analysis through Projection and Selection), we provide a bottom-up pixel-oriented approach for network traffic analysis where the expert starts with low-level anomalies and iteratively gains insight in higher level events through the creation of multiple selections of interest in parallel. The tight integration between visualization and machine learning enables the expert to iteratively refine anomaly scores, making the approach suitable for both post-traffic analysis and online monitoring tasks. To illustrate the effectiveness of this approach, we present example explorations on two real-world data sets for the detection and understanding of potential Advanced Persistent Threats in progress.

There is an increasing demand for network devices to perform deep packet inspection (DPI) in order to enhance network security. In DPI, the packet payload is compared against a set of predefined patterns that can be specified using regular expressions (regexes). It is well-known that mapping regexes to deterministic finite automaton (DFA) may suffer from the state explosion problem. Through observation, we attribute DFA explosion to the necessity of remembering matching history. In this paper, we investigate how to manage matching history efficiently and propose an extended DFA approach for regex matching called fcq-FA, which can make a memory size reduction of about 1,000 times with a fully automated approach. In fcq-FA, we use pipeline queues and counters to help record the matching history. Hence, state explosion caused by Kleene closure and length restriction can be completely avoided. Furthermore, it achieves a fully automated signature compilation with polynomial running time and space. The equivalence between fcq-FA and the traditional DFA is guaranteed by a strict theoretical proof, which means fcq-FA can process all the regexes supported by the traditional DFA.

Many network security applications such as Intrusion Detection System (IDS), Firewall and Data Loss Prevention System (DLPS) are based on deep packet inspection, in this packets header as well as payload of the packets are checked with predefined attack signature to identify whether it contains malicious traffic or not. To perform this checking different pattern matching methods are used by NIDS. The most popular method to implement pattern matching is to use of Finite Automata (FA). Generally, regular expressions are used to represent most of the attack signatures defined by NIDS. They are implemented using finite automata, which takes the payload of packet as input string. However, existing approaches of Finite Automata (FA), both deterministic finite automata (DFA) and non-deterministic finite automata (NFA) for pattern matching are having their own advantages and some drawbacks. The DFA based pattern matching methods are fast enough but require more memory. However, NFA based pattern matching methods are comparatively takes less memory but the speed of matching is very slow, to overcome these drawbacks of finite automata there are many approaches have been proposed. This paper discuses comparative study of some Finite Automata (FA) based techniques for pattern matching in network intrusion detection system (NIDS).

This study seeks to explore extending the technology acceptance model (DPAM) from a 2011 quantitative study—Modeling Intention to Use Deep Packet Inspection Technology in the United Arab Emirates, to the cyber security practitioner community of the Gulf Cooperation Council (GCC) and greater Middle East North Africa (MENA) Region. Analysis of regression between independent variable model factors of computer self efficacy, attitude toward ICT, perceived usefulness of ecommerce, intention to use ecommerce, societal trust and Internet filtration toward the dependent variable intention to use deep packet inspection, to determine parsimony, using confirmatory factor analysis (CFA), multinomial regression to assess correlation of independent and dependent variables, and assessment of the cross-suitability of DPAM across the MENA/GCC states through a MANOVA assessment. A qualitative component of the instrument enables collection of data about specific hardware and software deployed for deep packet inspection and cyber security systems. To cite this document: Alfred H. Miller, Extending the deep packet inspection model to the GCC/MENA region, International Journal of Cyber Society and Education, Vol. 6, No. 2, pp. 101-122, 2013. Permanent link to this document: http://dx.doi.org/10.7903/ijcse.1066

DPI & DFI: A Malicious Behavior Detection Method Combining Deep Packet Inspection and Deep Flow Inspection

Network security applications such as intrusion detection systems (IDSs), firewalls, anti-virus/spyware systems, anti-spam systems, and security visualisation applications are all computing-intensive applications. These applications all heavily rely on deep packet inspection, which is to examine the content of each network packet’s payload. Today these security applications cannot cope with the speed of broadband Internet that has already been deployed, that is, the processor power is much slower than the bandwidth power. Recently the development of multi-core processors brings more processing power. Multi-core processors represent a major evolution in computing hardware technology. While two years ago most network processors and personal computer microprocessors had single core configuration, the majority of the current microprocessors contain dual or quad cores and the number of cores on die is expected to grow exponentially over time. The purpose of this chapter is to discuss the research on using multi-core technologies to parallelize deep packet inspection algorithms, and how such an approach will improve the performance of deep packet inspection applications. This will eventually provide a security system the capability of real-time packet inspection thus significantly improve the overall status of security on current Internet infrastructure.

With the increasing internet service complexity, providing secured quality service has become a major concern. In the earlier systems, data was believed to be safer on internet without being intercepted. Now these internet vulnerabilities cannot be ignored as these weaknesses are used by many to carry out malicious activities. In order to tackle these problems, internet service providers are trying to find better options. One such technique gaining popularity in the recent decade is Deep Packet Inspection (DPI), which can be provided using software or hardware methods. It is reported that hardware is providing better solution than software. In this review article, we have introduced one such hardware, Ternary Content Addressable Memory (TCAM), which could perform complete packet inspection (packet header and payload inspection). In the first section, we have focused on evolution of packet filtering system. Later the discussion is divided into two parts: (i) packet classification using TCAM, and (ii) payload inspection (pattern matching and regular expression) using TCAM.

Deep Packet Inspection and Your Online Privacy: Constitutional Concerns and the Shortcomings of Federal Statutory Protection

With the development of computer technology, network bandwidth and network traffic continue to increase. Considering the large data flow, it is imperative to perform inspection effectively on network packets. In order to find a solution of deep packet inspection which can appropriate to the current network environment, this paper built a deep packet inspection system based on many-core platform, and in this way, verified the feasibility to implement a deep packet inspection system under many-core platform with both high performance and low consumption. After testing and analysis of the system performance, it has been found that the deep packet inspection based on many-core platform TILE_Gx36 [1] [2] can process network traffic of which the bandwidth reaches up to 4 Gbps. To a certain extent, the performance has improved compared to most deep packet inspection system based on X86 platform at present.

Large scale applications in data centers are composed of computers connected with a network. Traditional network switches do not perform routing based on packet contents. Thus, packets cannot be transmitted to the optimal computer for the request which is written in the packet payload. On the other hand, Deeply Programmable Network (DPN) switches can completely analyze packet payloads and perform routing based on their contents. In this paper, we focus on packet routing based on Deep Packet Inspection (DPI), i.e. analyzing packet payload, using DPN switches, and discuss performance of applications on a network using application aware DPN switches. We introduce application aware network and propose a method for improving application performance with DPN switch support. We evaluate our method with the several applications and show that our method can increase application performance.

Implementation of DPI(Deep Packet Inspection) system on a general purpose multiprocessor platform is an attractive option from the implementation cost point of view, since it does not require high-cost customized hardware. Load balancing has been considered as a primary means to achieve high performance in multi processor systems. We claim, however, that in case of DPI system design simply balancing the load of each processor does not necessarily yield the highest system performance. Instead, we propose a method in which tasks are allocated to processors based on their functions. We implemented the proposed method in dual processor Linux system and compare its performance with the existing load balancing methods. Under the proposed method, one processor is dedicated to deal with interrupt handling and generic packet processing, while another processor is dedicated to DPI processing. According to experimental results, the proposed scheme outperforms the existing schemes by 60%, mainly because of the reduction of cache miss and spin lock occurrences.

Deep Packet Inspection (DPI), which is one of the most important network techniques, has been widely utilized in current networking systems. By comparing the payloads of traffic to an existing signature database, DPI can identify whether traffic or packet is harmful, or belong to which application. The literal matching engine, which plays a key role in DPI, is the primary determinant of the system performance. FDR, an engine that can match a character with multiple literals in only 3 SIMD operations, has been developed. However, FDR has a significant performance drop-off when the signature database is composed of small-scale literal rule sets, whose occupations are larger than 90% in the modern database. In this paper, we have designed Teddy, an engine that is highly optimized for small-scale literal rule sets. Comparing with FDR, Teddy significantly increases the efficiency in small-scale literal rule sets by designing a novel algorithm that can parallelly match up to 64 characters with only 16 SIMD operations. Meanwhile, to evaluate Teddy in real-world DPI systems, we have implemented Teddy in Hyperscan with AVX512 platforms. The evaluation results show that Teddy achieves a maximum of 35.29x performance increases than Aho-corasick (AC), 2.16x performance increases than FDR in most cases.