A truly scalable IP lookup algorithm for next generation internet

An efficient IP address lookup algorithm based on a small balanced tree using entry reduction

Address lookup is one of the main functions of the Internet routers and a very important feature in evaluating router performance. As the Internet traffic keeps growing and the number of routing table entries is continuously growing, efficient address-lookup mechanism is essential. In recent years, various fast address-lookup schemes have been proposed, but most of those schemes are not practical in terms of the memory size required for routing table and the complexity required in table update. In this paper, we have proposed a parallel IP address lookup architecture based on multiple hashing. The proposed scheme has advantages in required memory size, the number of memory accesses, and table update. We have evaluated the performance of the proposed scheme through simulation using data from MAE-WEST router. The simulation result shows that the proposed scheme requires a single memory access for the address lookup of each route when 203 kbytes of memory and a few-hundred-entry TCAM are used.

As traffic on the Internet has increased exponentially, it is vital that Internet routers handle the received packets faster. When a router receives a packet, it has to make a decision where to send it next. To forward an IP packet, the routers need to lookup the destination address of the received packet in the forwarding table, implement forwarding algorithms, then forward the packet to the appropriate next hop or outgoing port. We discuss a parallel approach that combines an IP packet forwarding technique based on a partitioned lookup table with linear search on prefix lengths, binary search on prefix lengths, and LC (level-compressed) trie. Simulation results show that the proposed partition schemes speeds up the IP address lookup process.

The IP address lookup in the communication link technologies is becoming a major bottleneck by exploring the exact matching and the parallel hashing in each prefix length. In this paper, we present an algorithm for IP address lookup and collision resolution to reduce memory space and solve collision for high speed routing based on multiple table indexing structure. Our scheme makes possible direct access in a designated IP prefix without replicating the forwarding table entries, and provides efficient collision resolution for fast route lookup with only small memory. Simulation results indicate that our algorithm provides a faster lookup than the conventional lookup schemes.

Internet is getting faster and more dynamic daily, with increasing number of users, and mobility on the network. This puts a high load on Internet routers, which are the most important component of the Internet infrastructure. In this paper, we present and analyze integration of Quagga open-source routing software and NetFPGA-10G hardware platform. These two open-source platforms allow us to evaluate speed of connection between control and data planes which is crucial for flexibility of Internet routers and their potential to follow fast changes on the Internet. In particular, we measure how fast lookup tables calculated by the control plane can be downloaded to hardware in order to allow high-speed packet forwarding using balanced parallelized frugal lookup (BPFL) algorithm, which we proposed earlier. BPFL achieves high speed in packet forwarding using parallel processing on FPGA chip. Further, BPFL uses less on-chip memory compared to other lookup algorithms, and, for this reason, it can more scalable support IPv4 and IPv6 addresses.

IP address lookup algorithms can be evaluated on a number of metrics lookup time, update time, memory usage, and to a lesser extent, the time to construct the data structure used to support lookups and updates. Many of the existing methods are geared towards optimizing a specific metric, and hence do not scale well with the ever expanding routing tables and the forthcoming IPv6 with 128 bit long IP address. In contrast, our effort is directed at simultaneously optimizing multiple metrics and provide solutions that scale well to IPv6. In this paper, we present two IP address lookup schemes Elevator - Stairs algorithm and logW - Elevators algorithm. For a routing table with N prefixes, The Elevator - Stairs algorithm uses optimal O(N) memory, and achieves better lookup and update times than other methods with similar memory requirements. The logW - Elevators algorithm gives O(log W) lookup time, where W is the length of an IP address, while improving upon update time and memory usage. Experimental results using the MAE-West router with 29,487 prefixes show that the Elevator - Stairs algorithm gives an average throughput of 15.7 Million lookups per second (Mlps) using 459 KB of memory, and the logW - Elevators algorithm gives an average throughput of 21.41 Mlps with a memory usage of 1259 KB.

Due to the rapid growth of traffic in the Internet, backbone links of several gigabits per second are commonly deployed. To handle gigabit-per-second traffic rates, the backbone routers must be able to forward millions of packets per second on each of their ports. Fast IP address lookup in the routers, which uses the packet's destination address to determine for each packet the next hop, is therefore crucial to achieve the packet forwarding rates required. IP address lookup is difficult because it requires a longest matching prefix search. In the last couple of years, various algorithms for high-performance IP address lookup have been proposed. We present a survey of state-of-the-art IP address lookup algorithms and compare their performance in terms of lookup speed, scalability, and update overhead.

As data rates in the Internet increase, the Internet Protocol (IP) address lookup is required to be resolved in shorter resolution times. IP address lookup involves finding the longest matching prefix from a database of prefixes that better matches the destination address of a packet. The fastest IP-address lookup solutions are based on ternary content addressable memories (TCAMs), which can resolve the IP lookup in one memory-access time. However, TCAMs have a high power consumption and large complexity that may limit their scalability and storage capacity. An alternative is to use random access memory (RAM) that stores a forwarding table in a trie form. Proposed trie-based solutions for IP lookup require three or more memory-access times in the worst-case scenario. This makes them unattractive despite their reduced power consumption. In this paper, we propose a flexible and fast trie-based IP-lookup algorithm where parallel searching is performed. This algorithm performs lookup in two memory- access times whith a feasible amount of memory or three memory access times with reduced memory.

Modern Internet routers have to handle a large number of packet classification rules, which requires classification schemes to be scalable both in time and space. In this paper, we present a scalable packet classification algorithm that is developed by combining two new concepts to the well‐known bit vector (BV) scheme. We propose a range search method based on a cache‐aware tree (CATree) which makes full use of processor's cache line to reduce the number of dynamic random access memory (DRAM) accesses. Theoretically, the number of DRAM accesses of CATree is about log(m+1) times lower than that of the widely used binary search algorithm, where m is the number of keys in a single cache line. Based on our computational results on a set of 1024 keys, the CATree algorithm is 36% faster than binary search algorithm and the performance is better when applied to a larger set of keys. In addition, we develop a rule re‐arrangement algorithm to reduce the bitmap space of BV. With this re‐arrangement, the rules for the same action may be assigned an identical priority. This reduces the number of priorities as well as the memory space of the bitmap. Furthermore, this also reduces the number of memory accesses and hence, increases the CPU cache utilization. With CATree and rule re‐arrangement, the cache‐aware bit vector with rule re‐arrangement algorithm achieves better performance in comparison with the regular BV scheme, both in space and time. In our experiments, the proposed algorithm reduces the bitmap memory space of a practical set of firewall rules by two orders of magnitude and is 91% faster than the regular BV.

In general, context-based adaptive variable length coding (CAVLC) decoding in H.264/AVC standard requires frequent access to the unstructured variable length coding tables (VLCTs) and significant memory accesses are consumed. Heavy memory accesses will cause high power consumption and time delays, which are serious problems for applications in portable multimedia devices. We propose a method for high-efficiency CAVLC decoding by using a program instead of all the VLCTs. The decoded codeword from VLCTs can be obtained without any table look-up and memory access. The experimental results show that the proposed algorithm achieves 100% memory access saving and 40% decoding time saving without degrading video quality. Additionally, the proposed algorithm shows a better performance compared with conventional CAVLC decoding, such as table look-up by sequential search, table look-up by binary search, Moon’s method, and Kim’s method.

IP address lookup operation determines the longest prefix matching each incoming destination address. As a fundamental operation for packet forwarding at Internet routers, search speed for routing table lookup is the most important performance metric. Previous researches have shown that the search performance of trie-based algorithms can be improved by adding on-chip Bloom filters. In these algorithms, an on-chip Bloom filter identifies the membership of a node in an off-chip trie, and the number of trie accesses is reduced, because the Bloom filter can filter out accesses to non-existing nodes in the trie. In this paper, we propose a new method of utilizing a Bloom filter for the IP address lookup problem. In the previous Bloom filter-based approach, false positiveness has to be identified by accessing the off-chip trie for every positive result, since false positives can produce wrong results. In our proposed approach, the false positiveness of a Bloom filter is not necessarily identified by making false positives not mislead the search. Hence the number of off-chip trie accesses are significantly reduced. Simulation results show that the best matching prefix can be found with a single off-chip access in average and in the worst-case with the reasonable size of a Bloom filter in our proposed method.

Computer networks have continued to make substantial advances in the past couple of decades through better technologies and methodologies employed. As the usage of the networks continues to increase exponentially, high throughput of the networks has to be maintained with various performance-efficient network algorithms. IP address lookup is one of the processes, the performance of which dearly affects the overall network performance. Hashing has been widely used for fast IP address lookup due to its simplicity, but mostly assuming on hashing from an address set with uniformly distributed key values. Performance from these known hashing techniques is far from optimal due to the high nonuniformity in actual IP address distribution. In this paper, we propose a preprocessing method for the IP address databases to extract certain regularity to allow for design of more efficient hashing algorithms based on XOR operations. Simulation results show an improvement in performance ranging from 35% to 72% on randomly generated addresses and several sample IP address databases. The paper also shows that the proposed algorithms deliver comparable performance to other well-known hashing algorithms such as the CRC and RS hashing while requiring much less hardware to implement and a much shorter time to perform.

In this paper, we proposed a parallel IP address lookup architecture, which is a novel concept based on graphics processing unit (GPU) via Compute Unified Device Architecture (CUDA). Device function in GPU only performs IP address lookup. Host function is exploited to construct and update the data structure of IP address lookup. Both host and device functions can be executed simultaneously to fully utilize computation resource. Accordingly, we propose an IPv6-capable data structure and implement the data structure with CUDA. One of experimental results shows that G92 GPU can achieve a throughput more than 1.3 billion packets per second (GPPS) on IPv4 routing tables with more than 350K prefixes, which signifies CUDA-based IP forwarding engine with the proposed approach has the capability of GPPS IP forwarding rate on a low-end CUDA device. By employing dual data structures, our implementation can support several hundred thousand updates per second. Furthermore, the proposed forwarding scheme is varied to be applied and compatible with other Internet schemes and devices.

Many address lookup methods for an IP router have been recently proposed to improve a packet forwarding capability, but their performance prediction is very limited because of lack of considering actual traffic characteristics. It is necessary to consider actual traffic to predict more realistic performances on routers, specially in case of layers 3 and 4 switches whose performances are more influenced by flow characteristics. In this paper, we propose new methods for predicting the router's performance based on the statistical analysis of the Internet traffic. We also present an example of its application to the existing table lookup algorithms, and show that simulation results based on our method can provide accurate performance prediction.

Large and sparse are the prominent characteristics of small-world graph. In many scientific domains, such as biomedical science and scientific computing, as small-word graph grows in scale, processing small-world graph poses severe challenges to address mapping in Massive Parallel Processing. Data driven computation, unstructured data organization, which are poor in spatial and temporal locality, which are high frequency of memory access, leads to large RAM footprint on address mapping management. This paper proposes a novel approach with special Implementation for massive parallel processors. In our technique, The block level address mapping table is stored in large pages in DDR3 memory. Considering the highly frequency in accessing memory, we maintain a big cache in RAM to store address mapping entries of data array recently searching. The search algorithm in searching the cache is binary search. The goal is to reduce address mapping overhead without excessively compromising system response time. This scheme is designed for our massive parallel coprocessor system. For reducing power consumption, we have an attempt to implement address mapping of each massive parallel coprocessor in Field-Programmable Gate Array(FPGA). The experiment have been conducted on a real System on chip(Soc). The result shows that when the number of processor node is 4096 and its frequency is 233MHz, The RAM cost is 2.4 MB in each processor, when there is missing, the largest response time is 160us, which is less than the mainstream software implementation in address translation. In the case of making full use of available storage resources, The hit ratio in graph problem could be achieve 100%.