Tree-based string pattern matching on FPGAs

Pattern matching task plays an important role in network security applications especially Network Intrusion Detection System (NIDS). The limitation of matching throughput on general purpose processor gives rise to implementation of the task on FPGA. In this paper, we introduce a memory efficient FPGA-based pattern matching engine. We bases on Deterministic Finite Automata (DFA) and propose some modifications to reduce redundant logic. The proposed design, with better memory utilization, is capable of dynamic update and compatible to stateful NIDS. The analysis of memory efficiency and the hardware implementation of proposed design are also provided in this paper. We experiment our approach on contemporary NIDS pattern sets and build a prototype to test on real network environment. The results show that our design could save up to 90% hardware resources compare to traditional approach. The matching engine is compatible to gigabit network and could achieve 2.7-3.2x speed up to software-based matching engine.

Signature-based security tools such as network intrusion detection systems, anti-virus scanners, filters against network worms and other similar systems perform in real time computation-intensive task of multi-pattern string matching against tens of thousands or even millions of predefined malicious patterns. Due to rising traffic rates, increasing number and sophistication of attacks and the collapse of Moore's law for sequential processing, traditional software solutions can no longer meet the high requirements of today’s security challenges. Therefore, designers pay more attention to hardware approaches to accelerate pattern matching. The reconfigurable devices based on Field Programmable Gate Arrays (FPGA) combining the flexibility of software and the near-ASIC performance, have become increasingly popular for this purpose. The state-of-the-art solutions made in this area around the world were analyzed. There are three main approaches to fulfill the pattern matching using FPGA. The techniques (and underlying technologies) of these approaches are: content addressable memory (based on digital comparators), Bloom filter (based on hash-functions) and Aho-Corasick algorithm (based on finite automata). But none of them shows clear advantages over others. In this article, we propose a set of methods to increase the effectiveness of reconfigurable security tools by synthesizing optimal recognition modules that maximize the benefits of each approach. The Parallel Combination Method divides a set of patterns between several matching blocks that use different approaches to better fit each of them. The Sequential Cascading Method processes patterns in parts: if the first fragment does not match, the rest can be ignored. The Vertical Join Method couples together different approaches or techniques in a single unit to provide higher efficiency of the resulting device. The optimization procedure maximizes efficiency gains for each method. The methods and methodologies presented in this study will allow developers to create more efficient reconfigurable tools for information security systems.

Deep Packet Inspection (DPI) technology has been widely deployed in Network Intrusion Detection System (NIDS) to detect attacks and viruses. State-of-the-art NIDS uses Deterministic Finite Automata (DFA) to perform regular expression matching for its stable matching speed. However, traditional DFA algorithm’s throughput is limited by the input character’s width (usually one character per time). In this paper, we present an architecture named Parallel-DFA to accelerate regular expression matching by scanning multiple characters per time. Experimental results show that, our architecture can achieve as high as 1200 Gbps (1.17 Tbps) rate on current single Field-Programmable Gate Array (FPGA) chip. This makes it a very practical solution for NIDS in 100G Ethernet standard network, which is currently the fastest approved standard of Ethernet. To the best of our knowledge, this is the fastest matching performance architecture on a single FPGA chip. Besides, the throughput is nearly 3 orders of magnitude (916\(\times \)) than that of original DFA implemented on software. Our architecture is about 183.2\(\times \) efficiency than that of original DFA.

With the wide adoption of internet into our everyday lives, internet security becomes an important issue. Intrusion detection at the network level is an effective way of stopping malicious attacks at the source and preventing viruses and worms from wide spreading. The key component in a successful network intrusion detection system is a high performance pattern matching engine that can uncover the malicious activities in real time. In this paper, we propose a highly parallel, scalable hardware based network intrusion detection system, that can handle variable pattern length efficiently and effectively. Pattern matching for a packet is completed in O(N log M) time where N is the size of the packet and M is the longest pattern length. Implementation is done on a standard off-the-shelf field-programmable gate array. Comparison with the other techniques shows promising results.

Network intrusion detection and prevention systems (IDPS) use string matching to scan Internet packets for malicious content. Bloom filters offer a mechanism to search for a large number of strings efficiently and concurrently when implemented with field programmable gate array (FPGA) technology. A string matching circuit has been implemented within the FPX platform using Bloom filters. Using 155 block RAMs on a single Xilinx VirtexE 2000 FPGA, the circuit scans for 35,475 unique signatures.

Many network security applications such as Intrusion Detection System (IDS), Firewall and Data Loss Prevention System (DLPS) are based on deep packet inspection, in this packets header as well as payload of the packets are checked with predefined attack signature to identify whether it contains malicious traffic or not. To perform this checking different pattern matching methods are used by NIDS. The most popular method to implement pattern matching is to use of Finite Automata (FA). Generally, regular expressions are used to represent most of the attack signatures defined by NIDS. They are implemented using finite automata, which takes the payload of packet as input string. However, existing approaches of Finite Automata (FA), both deterministic finite automata (DFA) and non-deterministic finite automata (NFA) for pattern matching are having their own advantages and some drawbacks. The DFA based pattern matching methods are fast enough but require more memory. However, NFA based pattern matching methods are comparatively takes less memory but the speed of matching is very slow, to overcome these drawbacks of finite automata there are many approaches have been proposed. This paper discuses comparative study of some Finite Automata (FA) based techniques for pattern matching in network intrusion detection system (NIDS).

Deep learning-based feature extraction and optimizing pattern matching for intrusion detection using finite state machine

Intrusion detection systems are treated as vital elements of protective measures to computer systems and networks from abuse. The drastic increase in network speed and detection workloads necessitates the need for highly efficient Network Intrusion Detection Systems(NIDS). Since most NIDSs need to check for a large number of known attack patterns in every packet, pattern matching becomes the most significant part of signature-based NIDSs in terms of processing and memory resources. To support segmentation of network traffic and to detect fragmented attacks, we propose a method which performs both 'partial' and 'full' pattern matching using the data structure CDAWG (Compact Direct Acyclic Word Graph). In the present work, we designed and implemented an efficient string matching algorithm using CDAWG structure. Experimental results show that this algorithm is 2.5 times faster than the currently used Aho-Corasick algorithm.

FPGA Implementation of Explicit Model Predictive Control for Closed Loop Control of Depth of Anesthesia

The soft computing algorithms are being nowadays used for various multi input multi output complicated non linear control applications. This paper presented the development and implementation of back propagation of multilayer perceptron architecture developed in FPGA using VHDL. The usage of the FPGA (Field Programmable Gate Array) for neural network implementation provides flexibility in programmable systems. For the neural network based instrument prototype in real time application. The conventional specific VLSI neural chip design suffers the limitation in time and cost. With low precision artificial neural network design, FPGA have higher speed and smaller size for real time application than the VLSI design. The challenges are finding an architecture that minimizes the hardware cost, maximizing the performance, accuracy. The goal of this work is to realize the hardware implementation of neural network using FPGA. Digital system architecture is presented using Very High Speed Integrated Circuits Hardware Description Language (VHDL)and is implemented in FPGA chip. MATLAB ANN programming and tools are used for training the ANN. The trained weights are stored in different RAM, and is implemented in FPGA. The design was tested on a FPGA demo board. Keywords- Backpropagation, field programmable gate array (FPGA) hardware implementation, multilayer perceptron, pressure sensor, Xilinx FPGA.

Network Intrusion Detection Systems (NIDS) require the sensors to inspect the packet payloads at line rates. However, the software-only NIDS can not handle the large signature set with thousands of patterns of different lengths at line rates. Ternary content-addressable memories (TCAMs) have gained wide acceptance in the industry for storing and searching patterns in routers. But one important problem blocked the way to deploy TCAMs as deep package matching engines for NIDS: long patterns matching. A novel high speed long patterns matching architecture using cascade TCAMs for large signature set based NIDS is presented in this paper. Simple and efficient systems to handle tens of thousands of signatures with thousands of bytes length each can be built on such architecture. The matching system using for current SNORT signature set can work at the speeds greater than 2 Gbps.

This paper presents the simulation study of nonlinear dynamic of cardiac excitation based on Luo Rudy Phase I (LR-I) model towards numerical solutions of ordinary differential equations (ODEs) responsible for cardiac excitation on field programmable gate arrays (FPGAs). As computational modeling needs vast of simulation time, a real-time hardware implementation using FPGA could be the solution as it provides high configurability and performance. For rapid prototyping, MATLAB Simulink that offers a link with the FPGA has been used. Through Simulink HDL Coder, a tool in the MATLAB software that capable to convert the MATLAB Simulink blocks into hardware description language (HDL) code and an FPGA-in-the-loop (FIL) and co-simulation for verification, FPGA hardware implementation can be done. As a result, the LR-I excitation model is successfully simulated by using the MATLAB Simulink and the VHDL code has been successfully generated by the HDL Coder after fixed-point optimization is done. The FIL verification on actual FPGA board also has shown quantitatively comparable results to the MATLAB Simulink simulation. Therefore, the design flow has given a positive outlook in developing this FPGA stand-alone implementation.

Independent component analysis (ICA) is a technique that extracts independent source signals by searching for a linear or nonlinear transformation which minimizes the statistical dependence between components. ICA has been used in a variety of signal processing applications including dimensionality reduction in hyperspectral image (HSI) analysis. Due to the computation complexities and convergence rates, ICA is very time-consuming for high volume or dimension data set like hyperspectral images. Hardware implementation provides not only an optimal parallelism environment, but also a potential faster and real-time solution. This work synthesizes a parallel ICA (pICA) algorithm on field programmable gate array (FPGA). In the proposed implementation method, the pICA is partitioned into three temporally independent functional modules, and each of which is synthesized individually with several ICA-related reconfigurable components (RCs) that are developed for reuse and retargeting purpose. All modules are then integrated into a design and development environment for performing many subtasks such as FPGA synthesis, optimization, placement and routing. In a case study, we synthesize the pICA algorithm for hyperspectral image dimensionality reduction on the pilchard reconfigurahle computing platform embedded with Xilinx: VIRTEX V1000E. The FPGA executes at the maximum frequency of 20.161 MHz, and the pilchard board transfers data directly with CPU on the 64-bit memory bus at the maximum frequency of 133MHz. The performance comparisons between the proposed and another two ICA-related FPGA implementations show that the proposed FPGA implementation of pICA has potential in performing complicated algorithms on large volume data sets.

Memory efficiency with compact data structures for Internet Protocol (IP) lookup has recently regained much interest in the research community. In this paper, we revisit the classic trie-based approach for solving the longest prefix matching (LPM) problem used in IP lookup. In particular, we target our solutions for a class of large and sparsely-distributed routing tables, such as those potentially arising in the next-generation IPv6 routing protocol. Due to longer prefix lengths and much larger address space, straight-forward implementation of trie-based LPM can significantly increase the number of nodes and/or memory required for IP lookup. Additionally, due to the available on-chip memory and the number of I/O pins of Field Programmable Gate Arrays (FPGAs), state-of-the-art designs cannot support large IPv6 routing tables consisting of over $300$K prefixes. We propose two algorithms to compress the uni-bit-trie representation of a given routing table: (1) \emph{single-prefix distance-bounded path compression} and (2) \emph{multiple-prefix distance-bounded path compression}. These algorithms determine the optimal maximum \emph{skip distance} at each node of the trie to minimize the total memory requirement. Our algorithms demonstrates substantial reduction in the memory footprint compared with the uni-bit-trie algorithm ($1.86\times$ for IPv4 and $6.16\times$ for IPv6), and with the original path compression algorithm ($1.77\times$ for IPv4 and $1.53\times$ for IPv6). Furthermore, implementation on a state-of-the-art FPGA device shows that our algorithms achieve $466$ million lookups per second and are well suited for $100$Gbps lookup. This implementation also scales to support larger routing tables and longer prefix length when we go from IPv4 to IPv6.

Chapter 14 - Embedded Processing Cores