A Throughput-Driven Task Creation and Mapping for Network Processors

In order to reduce the speed gap between today's processor and memory, many processors adopt multithreading technology, Intel IXP series Network Processor (NP) is such a typical kind. This paper takes IXP series Network Processor for example, first analyzes multithreading mechanism; then targeting on how to realize multi-thread synchronization, presents three design methods and a specific example which could illustrate the difference between these methods; at last does some experiments according to different design methods, from experiment results draws some conclusions which would be useful in designing applications based on multithreading NP, including producing as less constrains and context swaps as possible, making the program executing time be longer than the memory access latency at every stage and so on.

Network processors (NPs) are widely used in many types of networking equipment due to their high performance and flexibility. For most NPs, software cache is used instead of hardware cache due to the chip area, cost and power constraints. Therefore, programmers should take full responsibility for software cache management which is neither intuitive nor easy to most of them. Actually, without an effective use of it, long memory access latency will be a critical limiting factor to overall applications. Prior researches like hardware multi-threading, wide-word accesses and packet access combination for caching have already been applied to help programmers to overcome this bottleneck. However, most of them do not make enough use of the characteristics of packet processing applications and often perform intraprocedural optimizations only. As a result, the binary codes generated by those techniques often get lower performance than that comes from hand-tuned assembly programming for some applications. In this paper, we propose an algorithm including two techniques - Critical Path Based Analysis (CPBA) and Global Adaptive Localization (GAL), to optimize the software cache performance of packet processing applications. Packet processing applications usually have several hot paths and CPBA tries to insert localization instructions according to their execution frequencies. For further optimizations, GAL eliminates some redundant localization instructions by interprocedural analysis and optimizations. Our algorithm is applied on some representative applications. Experiment results show that it leads to an average speedup by a factor of 1.974.

In the network applications domain, different network environments and scenarios demand various line rates, and restrain the design of the network processor by several constraints such as power and area. Additionally, new network services and applications with different processing requirements are increasingly emerging day by day. In this regard, having a multi-objective and flexible performance model that can be used to minimize the cost and time of designing network processors is an inevitable need. In this paper, we propose an accurate and fast prediction model that exploits just a few numbers of system-level parameters to estimate the performance and power of a commercial network processor, Intel IXP2800. The proposed design methodology uses a non-linear learning algorithm - a combination of the polynomial transformation of design parameters and higher-order spline functions - which in the face of newly introduced applications needs only a small training set, i.e. a small number of simulations, to train the model. Our experimental results show the proposed models can achieve a median error rate as low as 8.7 percent for performance and 2.6 percent for power metric.

Mapping packet processing tasks on network processor micro-engines involves complex tradeoffs that relating to maximizing parallelism and pipelining. Due to an increase in the size of the code store and complexity of the application requirements, network processors are being programmed with heterogeneous threads that may execute code belonging to different tasks on a given micro-engine. Also, most network applications are streaming applications that are typically processed in a pipelined fashion. Thus, the tasks on different micro-engines are pipelined in such a way as to maximize the throughput. Tasks themselves could have different run time performance demands. Traditionally, runtime management involving processor sharing, real-time scheduling etc. is provided by the runtime environment (typically an operating system) using the hardware support for timers and interrupts that allows time slicing the resource amongst the tasks. However, due to stringent performance requirements on network processors (which process packets from very high speed network traffic), neither OS nor hardware mechanisms are typically feasible/available.In this paper, we show that it is very difficult and inefficient for the programmer to meet the constraints of runtime management by coding them statically. Due to the infeasibility of hardware or OS solution (even in the near future), the only choice left is a compiler approach.We propose a complete compiler solution to automatically insert explicit context switch (ctx) instructions provided on the processors so that the execution of programs is better manipulated at runtime to meet their constraints. We show that such an approach is feasible opening new application domains that would need heterogeneous thread programming. Such approaches would in general become important for multi-core processors.

Online set intersection operations have been widely used in network processing tasks, such as Quality of Service differentiation, firewall processing, and packet/traffic classification. The major challenge for online set intersection is to sustain line-rate processing speed; accelerating set intersection using state-of-the-art hardware devices is of great interest to the research community. In this paper, we present a novel high-performance set intersection approach on FPGA. In our approach, each element in any set is represented by a combination of Group ID (GID) and Bit Stride (BS); all the sets are intersected using linear merge techniques and bitwise AND operations. We map our online set intersection algorithm onto hardware; this is done by constructing modular Processing Element (PE) and concatenating multiple PEs into a tree-based parallel architecture. In order to improve the throughput on a state-of-the-art FPGA, we feed all the inputs to FPGA in a streaming fashion with the help of the synchronization GIDs. Post place-and-route results show that, for a typical set intersection problem in network processing, our design can intersect $\text{eight}$ sets, each of up to $32$ K elements, at a throughput of $47.4$ Thousand Intersections Per Second (KIPS) and a latency of $94.8\,\mu$ s per batch of inputs. Compared to the classic linear merge or bitwise AND techniques on state-of-the-art multi-core processors, our designs on FPGA achieves up to $66\times$ throughput improvement and $80\times$ latency reduction.

Along with sharply increasing bandwidth requirements, modern network applications and new protocols demand highly intelligent and sophisticated processing over the network. Since these workloads require processing capability beyond state-of-the-art microprocessors, parallel architectures are required in order to handle the packet data without slowing down line speed. Network processors with various parallel architectures are appearing in the market, however, a thorough investigation of the implications of static versus dynamic scheduling of this class of emerging workloads has not been done. In this paper, we characterize the performance and power dissipation of statically identified ILP architectures, and we also compare them to dynamically scheduled architectures for network processing. In dynamically scheduled architectures, the power consumption of the instruction window greatly increases with increasing issue widths. On the other hand, statically scheduled architectures show better performance, as well as tremendous advantages in power, since they do not have instruction window wakeup/select, reorder buffer, and other scheduling related hardware modules. With the large parallelism and the loop nature of network applications, our experimental analysis supports static scheduling as an appropriate strategy for network processor applications.

While we cannot question the high performance capabilities of the kernel bypass approach in the network functions world, we recognize that the Linux kernel provides a rich ecosystem with an efficient resource management and an effective resource sharing ability that cannot be ignored. In this work we argue that by mixing kernel-bypass and in kernel processing can benefit applications and network function middleboxes. We leverage a high-performance user space TCP stack and recent additions to the Linux kernel to propose a hybrid approach (kernel-user space) to accelerate SDN/NFV deployments leveraging services of the reliable transport layer (i.e., stateful middleboxes, Layer 7 network functions and applications). Our results show that this approach enables highperformance, high CPU efficiency, and enhanced integration with the kernel ecosystem. We build our solution by extending mTCP which is the basis of some state-of-the-art L4-L7 NFV frameworks. By having more efficient CPU usage, NFV applications can have more CPU cycles available to run the network functions and applications logic. We show that for a CPU intense workload, mTCP/AF_XDP can have up to 64% more throughput than the previous implementation. We also show that by receiving cooperation from the kernel, mTCP/AF_XDP enables the creation of protection mechanisms for mTCP. We create a simulated DDoS attack and show that mTCP/AF_XDP can maintain up to 287% more throughput than the unprotected system during the attack.

SummaryMulticore network processors have been playing an increasingly important role in computational processes, which emphasize on scalability and parallelism of the systems, in distributed environments especially in Internet‐based delay‐sensitive applications. It is an important but unsolved issue, however, to efficiently schedule tasks in network processors with multicore and multithread for improving the system throughput as much as possible. Profiling can gather runtime environment information and guide the compiler to optimize programs through scheduling tasks based on the runtime context. This paper proposes a profiling‐based task scheduling approach, targeting on improving the throughput of multicore network processor (Intel IXP) systems in the balanced pipeline way. In this work, we investigate a profiling‐based task scheduling framework, a task scheduling algorithm, and a set of performance models. Our task allocation scheme maps tasks onto the pipeline architecture and multiple threads of network processors in parallel, which incorporates the profiling context and global thread refinement. We evaluate our task scheduling algorithm by implementing representative network applications on the Intel IXP network processor. Experimental results demonstrate that our algorithm is able to schedule tasks in a balanced pipeline fashion and achieve the high throughput and data transmission rate. Copyright © 2012 John Wiley & Sons, Ltd.

Network processors (NPs), a type of multicore, multithread system that exploits system-on-chip techniques, usually adopt a distributed, shared memory hierarchy to scale up network processing. Network applications running on NP often display distinct characteristics on memory subsystem compared with traditional processors with multilevel cache. As security, especially cryptography, has become indispensable in today's networks, we present a benchmark suite called NPCryptBench. It is the first benchmark specially designed to evaluate cryptographic performance on NPs, particularly that of processor and memory subsystem. By running NPCryptBench on Intel IXP network processors, we provide quantitative evidence that performance bottlenecks under such a highly parallel environment reside in memory hierarchy and prolonged access latency, instead of limited computing resources described in recent literature. To alleviate this memory-wall effect, we suggest several software optimizations, which averagely boost the benchmark throughput by 145% on Intel IXP2800. Our analysis and optimizations are also suitable for other NPs with similar architectures.

Outlier detection has many important applications in sensor networks, e.g., abnormal event detection, animal behavior change, etc. It is a difficult problem since global information about data distributions must be known to identify outliers. In this paper, we use a histogram-based method for outlier detection to reduce communication cost. Rather than collecting all the data in one location for centralized processing, we propose collecting hints (in the form of a histogram) about the data distribution, and using the hints to filter out unnecessary data and identify potential outliers. We show that this method can be used for detecting outliers in terms of two different definitions. Our simulation results show that the histogram method can dramatically reduce the communication cost.

Massively multithreaded GPUs achieve high throughput by running thousands of threads in parallel. To fully utilize the hardware, contemporary workloads spawn work to the GPU in bulk by launching large tasks, where each task is a kernel that contains thousands of threads that occupy the entire GPU.GPUs face severe underutilization and their performance benefits vanish if the tasks are narrow, i.e., they contain less than 512 threads. Latency-sensitive applications in network, signal, and image processing that generate a large number of tasks with relatively small inputs are examples of such limited parallelism. Recognizing the issue, CUDA now allows 32 simultaneous tasks on GPUs; however, that still leaves significant room for underutilization.This paper presents Pagoda, a runtime system that virtualizes GPU resources, using an OS-like daemon kernel called MasterKernel. Tasks are spawned from the CPU onto Pagoda as they become available, and are scheduled by the MasterKernel at the warp granularity. This level of control enables the GPU to keep scheduling and executing tasks as long as free warps are found, dramatically reducing underutilization. Experimental results on real hardware demonstrate that Pagoda achieves a geometric mean speedup of 2.44x over PThreads running on a 20-core CPU, 1.43x over CUDA-HyperQ, and 1.33x over GeMTC, the state-of-the-art runtime GPU task scheduling system.

Video streaming poses significant technical challenges in quality of service guarantee and efficient resource management. Generally, it is recognized that end-to-end quality requirements of video streaming application can be reasonably achieved only by integrative study of advanced networking and content processing techniques. However, most existing integration techniques stop at the bit stream level, ignoring a deeper understanding of the media content. Yet, the underlying visual content of the video stream contains a vast amount of information that can be used to predict the bit-rate or quality more accurately. In the content-aware video streaming framework, video content is extracted automatically and used to control video quality under various manipulations and network resource requirements.

D-STOP is a parallel, scalable, fully connected optoelectronic computing architecture suitable for applications in interconnection networks, numerical processing, neural networks, and artificial intelligence. The architecture consists of an array of N optoelectronic processing elements arranged in a 2-D topology. Each processing element consists of N electronic processing sub-units having detectors. These detector sub-units are electronically connected by an H-tree interconnection. At each node of the H-tree are additional fan-in processing sub-units. At the center of the H-tree is a single processing sub-unit having an optical modulator. Fully connected interprocessor communication is achieved through space-invariant optical interconnections, leading to a scalable architecture. The architecture can be used to perform outer product and inner product operations. Since all mathematical operations are performed electronically, the products can be generalized. Additional processing is available during fan-in, generalizing the conventional summation of inner products. This feature is particularly useful in neural network applications, including backpropagation networks. The key to the architecture is the dual scale of processing: detector and fan-in sub-units at the lower level and composite processing elements at the higher level. We will present the architecture, its functionality, applications, and optoelectronic design considerations.

Iterative stencil loops (ISLs) are used in many applications and tiling is a well-known technique to localize their computation. When ISLs are tiled across a parallel architecture, there are usually halo regions that need to be updated and exchanged among different processing elements (PEs). In addition, synchronization is often used to signal the completion of halo exchanges. Both communication and synchronization may incur significant overhead on parallel architectures with shared memory. This is especially true in the case of graphics processors (GPUs), which do not preserve the state of the per-core L1 storage across global synchronizations. To reduce these overheads, ghost zones can be created to replicate stencil operations, reducing communication and synchronization costs at the expense of redundantly computing some values on multiple PEs. However, the selection of the optimal ghost zone size depends on the characteristics of both the architecture and the application, and it has only been studied for message-passing systems in distributed environments. To automate this process on shared memory systems, we establish a performance model using NVIDIA’s Tesla architecture as a case study and propose a framework that uses the performance model to automatically select the ghost zone size that performs best and generate appropriate code. The modeling is validated by four diverse ISL applications, for which the predicted ghost zone configurations are able to achieve a speedup no less than 95% of the optimal speedup.

Two-level pipelining in processor arrays (PAs) involves pipelining of operations across processing elements (PEs) and pipelining of operations in functional units in each PE. Although it is an attractive method for improving the throughput of PAs, existing methods for generating PAs with two-level pipelining are restricted and cannot systematically explore the entire space of feasible designs. In this paper, we extend a systematic design method, called General Parameter Method (GPM), we have developed earlier to find optimal designs of PAs with two-level pipelines. The basic idea is to add new constraints on periods of data flows to include the effect of internal functional pipelines in the PEs. As an illustration, we present pipelined PA designs for computing matrix products. For n-dimensional meshes and other symmetric problems, we provide an efficient scheme to obtain a pipelined PA from a non-pipelined PA using a reindexing transformation. This scheme is used in GPM as a pruning condition to arrive at optimal pipelined PAs efficiently. For pipelines with minimum initiation interval (MII) greater than unity, we show additional constraints that ensure correctness of the synthesized PAs.