Network-attached FPGAs for data center applications

FPGAs (Field Programmable Gate Arrays) are making their way into data centers (DC). They are used as accelerators to boost the compute power of individual server nodes and to improve the overall power efficiency. Meanwhile, DC infrastructures are being redesigned to pack ever more compute capacity into the same volume and power envelops. This redesign leads to the disaggregation of the server and its resources into a collection of standalone computing, memory, and storage modules. To embrace this evolution, we propose an architecture that decouples the FPGA from the CPU of the server by connecting the FPGA directly to the DC network. This proposal turns the FPGA into a network-attached computing resource that can be incorporated with disaggregated servers into these emerging data centers. We implemented a prototype and compared its network performance with that obtained from bare metal servers (Native), virtual machines (VM), and containers (CT). The results show that standalone network-attached FPGAs outperform them in terms of network latency and throughput by a factor of up to 35x and 73x, respectively. We also observed that the proposed architecture consumes only 14% of the total FPGA resources.

Intensive computation is entering data centers with multiple workloads of deep learning. To balance the compute efficiency, performance, and total cost of ownership (TCO), the use of a field-programmable gate array (FPGA) with reconfigurable logic provides an acceptable acceleration capacity and is compatible with diverse computation-sensitive tasks in the cloud. In this paper, we develop an FPGA acceleration platform that leverages a unified framework architecture for general-purpose convolutional neural network (CNN) inference acceleration at a data center. To overcome the computation bound, 4,096 DSPs are assembled and shaped as supertile units (SUs) for different types of convolution, which provide up to 4.2 TOP/s 16-bit fixed-point performance at 500 MHz. The interleaved-task-dispatching method is proposed to map the computation across the SUs, and the memory bound is solved by a dispatching-assembling buffering model and broadcast caches. For various non-convolution operators, a filter processing unit is designed for general-purpose filter-like/pointwise operators. In the experiment, the performances of CNN models running on server-class CPUs, a GPU, and an FPGA are compared. The results show that our design achieves the best FPGA peak performance and a throughput at the same level as that of the state-of-the-art GPU in data centers, with more than 50 times lower latency.

Deep convolutional neural network (DCNN) is an advanced technology in image recognition. Because of extreme computing resource requirements, DCNN implementation with software alone cannot achieve real-time requirement. Therefore, the need to implement DCNN accelerator hardware is increasing. In this paper, we present a field programmable gate array (FPGA)-based hardware accelerator design of DCNN targeting handwritten Hangul character recognition application. Also, we present design optimization techniques in SDAccel environments for searching the optimal FPGA design space. The techniques we used include memory access optimization and computing unit parallelism, and data conversion. We achieved about 11.19 ms recognition time per character with Xilinx FPGA accelerator. Our design optimization was performed with Xilinx HLS and SDAccel environment targeting Kintex XCKU115 FPGA from Xilinx. Our design outperforms CPU in terms of energy efficiency (the number of samples per unit energy) by 5.88 times, and GPGPU in terms of energy efficiency by 5 times. We expect the research results will be an alternative to GPGPU solution for real-time applications, especially in data centers or server farms where energy consumption is a critical problem.

Over the past few years, Convolutional Neural Networks (CNNs) have provided major breakthroughs in fields such as computer vision and natural language processing, resulting in a rise in the adoption of CNNs with increased levels of complexity. Consequently, the need for fast and power efficient processing of such networks has become critically important. Conventional hardware solutions, namely CPUs and GPUs, fail to address these requirements as CPUs are not suited for processing massively parallel multiply and accumulate (MAC) operations and GPUs are not power efficient. However, Field Programmable Gate Arrays (FPGAs) have emerged as a promising alternative due to their extensive programmability, ease of executing parallel operations and wide interfacing capabilities. In this paper, we have designed a hardware accelerator for speeding up the inference phase of LeNet–5 to enable faster classification of handwritten digits. We employ software quantization for ease of implementation on FPGAs, and partial pipelining to process the various layers of a typical CNN. Targeting the Xilinx Zynq-7000 SoC, we report a speedup improvement of at least 4.7x and power efficiency improvement of 32x compared to similar works.

The demand for high-performance computing and rapidly increasing power consumption has increased the necessity for application-specific accelerators. In the datacenter and mobile system, more applications are increasingly relying on accelerators. Field-programmable gate arrays (FPGAs) emerge as a good candidate because they have high programmability and power efficiency. As the number of applications requiring acceleration increases, there is huge demand for FPGAs that support multiple contexts. Previous FPGA designs that support multicontext have various shortcomings such as volatility, poor power efficiency, large performance, area, and reconfiguration overhead. In this article, we propose a spin-transfer torque magnetic RAM (STT-MRAM)-based nonvolatile multicontext FPGA (NVMC-FPGA) that overcomes these shortcomings. We introduce the NVMC-FPGA architecture and operation modes that take advantage of nonvolatility and support multicontext. We also develop the multicontext-aware FPGA computer aided design flow to make the most of the NVMC-FPGA. Compared to the conventional SRAM-based FPGA, when eight identical circuits are mapped, the NVMC-FPGA improves the performance by 15.3% on average and reduces the power consumption by 11.2%–80.7%, depending on the number of simultaneously activated circuits. Moreover, when eight different circuits are mapped, the NVMC-FPGA improves the performance by 58.5% on average and reduces the power consumption by 6.2%–63.3%, depending on the number of simultaneously activated circuits.

In the era of cloud computing, data center network (DCN) can consume a significant amount of power (e.g., 10 to 20 percent) in large-scale data centers. To reduce the power consumption of DCN, traffic consolidation has been recently proposed as an effective approach to reduce the number of DCN devices in use. However, existing consolidation approaches do not sufficiently consider the flow completion time (FCT) requirement. On one hand, missing the FCT deadlines can cause serious violation of service-level agreement, especially for delay-sensitive networking services, such as web search and E-commerce. On the other hand, keeping all the devices on to make FCTs much shorter than the desired requirements is unnecessary because 1) users may not be able to perceive the difference, and 2) such a greedy strategy can lead to unnecessarily high DCN power consumption and thus more electricity costs. In this paper, we propose FCTcon, a dynamic FCT control strategy for DCN power optimization. FCTcon is designed rigorously based on control theory to dynamically control the FCT of delay-sensitive traffic flows exactly to the requirements, such that the desired FCT performance is guaranteed while the maximum amount of DCN power savings can be achieved. Results from both hardware experiments and simulation evaluation demonstrate that, compared to the state-of-the-art DCN power optimization schemes, FCTcon can improve the DCN FCT performance, while achieving nearly the same or even more power savings. Consequently, FCTcon can result in more than 22.0 to 62.2 percent extra net profits for a data center with 50K servers. In addition, we further propose two extended designs of FCTcon to handle the coflow abstraction recently proposed for DCN, which successfully reduce the coflow deadline miss ratio by 12 to 15 percent.

Datacenter workloads demand high computational capabilities, flexibility, power efficiency, and low cost. It is challenging to improve all of these factors simultaneously. To advance datacenter capabilities beyond what commodity server designs can provide, we designed and built a composable, reconfigurable hardware fabric based on field programmable gate arrays (FPGA). Each server in the fabric contains one FPGA, and all FPGAs within a 48-server rack are interconnected over a low-latency, high-bandwidth network. We describe a medium-scale deployment of this fabric on a bed of 1632 servers, and measure its effectiveness in accelerating the ranking component of the Bing web search engine. We describe the requirements and architecture of the system, detail the critical engineering challenges and solutions needed to make the system robust in the presence of failures, and measure the performance, power, and resilience of the system. Under high load, the large-scale reconfigurable fabric improves the ranking throughput of each server by 95% at a desirable latency distribution or reduces tail latency by 29% at a fixed throughput. In other words, the reconfigurable fabric enables the same throughput using only half the number of servers.

Data centers (DCs), owing to the exponential growth of Internet services, have emerged as an irreplaceable and crucial infrastructure to power this ever-growing trend. A DC typically houses a large number of computing and storage nodes, interconnected by a specially designed network, namely, DC network (DCN). The DCN serves as a communication backbone and plays a pivotal role in optimizing DC operations. However, compared to the traditional network, the unique requirements in the DCN, for example, large scale, vast application diversity, high power density, and high reliability, pose significant challenges to its infrastructure and operations. We have observed from the premium publication venues (e.g., journals and system conferences) that increasing research efforts are being devoted to optimize the design and operations of the DCN. In this paper, we aim to present a systematic taxonomy and survey of recent research efforts on the DCN. Specifically, we propose to classify these research efforts into two areas: 1) DCN infrastructure and 2) DCN operations. For the former aspect, we review and compare the list of transmission technologies and network topologies used or proposed in the DCN infrastructure. For the latter aspect, we summarize the existing traffic control techniques in the DCN operations, and survey optimization methods to achieve diverse operational objectives, including high network utilization, fair bandwidth sharing, low service latency, low energy consumption, high resiliency, and etc., for efficient DC operations. We finally conclude this survey by envisioning a few open research opportunities in DCN infrastructure and operations.

The rapid advancement of programmable logic devices (PLDs) and field-programmable gate arrays (FPGAs) has enabled the development of adaptive VLSI architectures optimized for next-generation wireless communication systems. This research focuses on RFID and Wireless Sensor Network (WSN) applications, integrating AI-driven optimization techniques to enhance power efficiency, processing speed, and system adaptability. A hybrid AI-based hardware optimization technique, combining Reinforcement Learning (RL) and Genetic Algorithm (GA), is introduced to improve resource utilization and reduce latency. Additionally, an AI-Optimized Dynamic Frequency Scaling (DFS) technique is implemented to adjust FPGA operating frequencies in real-time, achieving a power reduction of 27.6% while maintaining high computational efficiency. Processor implementation for RFID-based power-saving appliances, reducing power consumption by 32.5% compared to conventional approaches. Hardware realization of an optimized anti-collision algorithm for RFID, achieving a 24.8% increase in tag identification speed. FPGA-based security architecture integrating lightweight cryptographic techniques, enhancing data security with a 96.2% encryption success rate and 25% lower hard-ware resource usage. Power-efficient WSN node implementation using an adaptive localization algorithm, reducing localization errors by 18.4% and extending node lifespan by 29.3%. Low-power CORDIC algorithm-based DPSK modem implementation with 32.1% lower latency in real-time signal processing applications. AI-Optimized DFS technique, reducing dynamic power consumption by 27.6% and improving computational throughput by 22.7%, making FPGA implementations more energy-efficient. The designs were developed using VHDL-based high-level hardware description, simulated in Xilinx ISE 14.3, and implemented on Virtex-5, Spartan-6, and Kintex-7 FPGA platforms. Performance evaluations and comparative studies with existing models confirm significant improvements in power efficiency, processing speed, and adaptability. The findings contribute to the development of high-performance, power-efficient, and scalable VLSI architectures for RFID and WSN applications, paving the way for next-generation secure and intelligent wireless communication systems.

Data center network (DCN) can consume a significant amount of power (e.g., 10% to 20%) in large-scale data centers. To reduce the power consumption of DCN, traffic consolidation has been recently proposed as an effective approach to reduce the number of DCN devices in use. However, existing consolidation approaches do not sufficiently consider the flow completion time (FCT) requirement. On one hand, missing the FCT deadlines can cause serious violation of service-level agreement, especially for delay-sensitive networking services, such as web search and E-commerce. On the other hand, keeping all the devices on to make FCTs much shorter than the desired requirements is unnecessary because 1) users may not be able to perceive the difference, and 2) such a greedy strategy can lead to unnecessarily high DCN power consumption and thus more electricity costs. In this paper, we propose FCTcon, a dynamic FCT control strategy for DCN power optimization. FCTcon is designed rigorously based on control theory to dynamically control the FCT of delay-sensitive traffic flows exactly to requirements, such that the desired FCT performance is guaranteed while the maximum amount of DCN power savings can be achieved. Results from both hardware experiments and simulation evaluations demonstrate that, compared to the state-of-the-art DCN power optimization schemes, FCTcon can improve the DCN FCT performance, while achieving nearly the same or even more power savings. Consequently, FCTcon can result in more than 22.0% to 62.2% extra net profits for a data center with 50K servers.

Wireless sensor networks for monitoring and steering industrial systems have emerged as an important new application area for wireless embedded technology in safety-related systems. Therefore several hardware platforms can be targeted. Due to their flexibility and reconfigurability along with high performance and power efficiency, modern Field Programmable Gate Arrays (FPGAs) are an excellent platform to realize such intelligent sensor networks. On one hand, FPGAs can integrate embedded processors and on-chip memory into a single die and can be easily configured to interface with a wide variety of popular communications protocols like SPI, UART and I2C, which are used for sensor communication. On the other hand, the reconfigurability of FPGAs supports the design of self-organizing computing systems, which are increasingly used in sensor applications. In this paper an FPGA-based wireless smart sensor network that integrates acceleration sensors for increasing safety aspects in cognitive systems used for industrial communication is presented.

In the workflow of SKA-SDP (Square Kilometer Array Radio Telescope-Scientific Data Processing), FFT (Fast Fourier Transform) calculation takes a significant proportion of computation overhead. Moreover, FFT computation has to be done within the tight power budget, which existing generic high performance computing architectures cannot meet. To explore power efficiency of FFT computation, this study is designed with initial evaluation of FFT implementation in variable precision on Xilinx ML605 FPGA (field programmable gate array). The FPGA-based implementation of FFT includes a Xilinx IP Core version 7.1, which supports fixed-point and floating-point computation in single precision. The input data width and phase factor width of fixed-point computation can be varied from 8 bits to 34 bits, allowing that calculation accuracy can be adjusted by setting the width. Since single precision is redundant to accuracy requirement of SKA-SDP, fixed-point calculation is designed to emulate single-precision floating point computation. Calculational power dissipation and throughput with different phase factors on FPGA was measured respectively. The final result demonstrates that the implementation on FPGA ensures sufficient precision at a much less power cost compared with floating-point FFT. In other words, this study indicates variable precision computation would be an efficient way to improve power efficiency.

Field-programmable gate arrays (FPGAs) can potentially be configured as beamforming platforms for ultrasound imaging, but a long design time and skilled expertise in hardware programming are typically required. In this article, we present a novel approach to the efficient design of FPGA beamformers for synthetic aperture (SA) imaging via the use of software-based high-level synthesis techniques. Software kernels (coded in OpenCL) were first developed to stage-wise handle SA beamforming operations, and their corresponding FPGA logic circuitry was emulated through a high-level synthesis framework. After design space analysis, the fine-tuned OpenCL kernels were compiled into register transfer level descriptions to configure an FPGA as a beamformer module. The processing performance of this beamformer was assessed through a series of offline emulation experiments that sought to derive beamformed images from SA channel-domain raw data (40-MHz sampling rate, 12 bit resolution). With 128 channels, our FPGA-based SA beamformer can achieve 41 frames per second (fps) processing throughput (3.44 × 10(8) pixels per second for frame size of 256 × 256 pixels) at 31.5 W power consumption (1.30 fps/W power efficiency). It utilized 86.9% of the FPGA fabric and operated at a 196.5 MHz clock frequency (after optimization). Based on these findings, we anticipate that FPGA and high-level synthesis can together foster rapid prototyping of real-time ultrasound processor modules at low power consumption budgets.

The number of data centers has been increased for various reasons such as cloud computing, big-data analysis, multimedia service, etc. With public interests on data center, many novel technologies for data center networks have been proposed and deployed to support data center operations more efficiently and effectively. However, the construction of data center network incurs significant costs. Moreover, various technologies interplay each other to achieve multiple objectives, and it makes difficult to validate and/or verify characteristics of data center network. In addition, it difficult to perform experiments with a number of hosts and switches. Therefore, it is necessary to observe the characteristics of target data center network before building it. A common approach to evaluate data center is to run simulations that should be similar with real-world data center environment. However, generating traffic with the characteristics of data center networks is not matured yet. People still employ a traffic generator based on the characteristics of Internet traffic. We design a traffic generator that shows more accurate characteristics of data center network traffic. Various traffic characteristics exploited explored by several studies are considered. The proposed method generates flow-level network traffic matrix based on Poisson Shot-Noise model. We implemented the traffic generator using Python programming language to create traffic matrix. To evaluate the proposed method, we compare the results with real data center network traffic. Our results show that the generated traffic owns similar characteristics with the real network traffic in terms of flow size, duration, and the mean and variance of total traffic rate.

Given the current expansion of cloud computing, the expected advent of the Internet of Things, and the requirements of future fifth‐generation network infrastructures, significantly larger pools of computational and storage resources will soon be required. This emphasizes the need for more scalable data centers that are capable of providing such an amount of resources in a cost‐effective way. A quick look into today's commercial data centers shows that they tend to rely on variations of well‐defined leaf‐spine/Clos data center network (DCN) topologies, offering low latency, ultrahigh bisectional bandwidth, and enhanced reliability against concurrent failures. However, DCNs are typically restricted by the use of the Transmission Control Protocol/Internet Protocol (TCP/IP) suite, thus suffering limited routing scalability. In this work, we study the benefits that replacing TCP/IP with the recursive internetwork architecture (RINA) can bring into commercial DCNs, focusing on forwarding and routing scalability. We quantitatively evaluate the benefits that RINA solutions can yield against those based on TCP/IP and highlight how, by deploying RINA, topological routing solutions can improve even more the efficiency of the network. To this goal, we propose a rule‐and‐exception forwarding policy tailored to the characteristics of several DCN variants, enabling fast forwarding decisions with merely neighbors' information. Upon failures, few exceptions are necessary, whose computation can also profit from the known topology. Extensive numerical results show that the proposed policy requirements depend mainly on the number of neighbors and concurrent failures in the DCN rather than its size, dramatically reducing the amount of forwarding and routing information stored at DCN nodes.