1 - Data center architectures

Today, there is a generalized standard usage of internet for all.The devices via multiple technologies that facilitates to provide few communication methods to scholars to work with. By forming multiple paths in the data center network, latest generation data centers offer maximum bandwidth with robustness. To utilize this bandwidth, it is necessary that different data flows take separate paths. In brief, a single-path transport seems inappropriate for such networks. By using Multipath TCP, we must reconsider data center networks, with a diverse approach as to the association between topology, transport protocols, routing. Multipath TCP allows certain topologies that single path TCP cannot use. In newer generation data centers, Multipath TCP is already deployable using extensively deployed technologies such as Equal-cost multipath routing. But, major benefits will come when data centers are specifically designed for multipath transports. Due to manifold of technologies like Cloud computing, social networking, and information networks there is a need to deploy the number of large data centers. While Transport Control Protocol is the leading Layer-3 transport protocol in data center networks, the operating conditions like high bandwidth, small-buffered switches, and traffic patterns causes poor performance of TCP. Data Center TCP algorithm has newly been anticipated as a TCP option for data centers which address these limitations. It is worth noting that traditional TCP protocol.

We are delighted to introduce a Special Issue on Optical Interconnects for the IET Optoelectronics magazine. This issue presents the current status as well as some of the outstanding progress that has been witnessed in the rapidly evolving area of optical interconnects. Today, data centres (DC) and high-performance computers are struggling with limited interconnect capacity and increased power consumption while trying to meet overwhelming bandwidth demands. As photonics technologies have been rapidly maturing, optical interconnects have come into the spotlight as the means to cope with the emerging bandwidth, energy, and capacity needs. As such, a wealth of photonic interconnect solutions across the DC hierarchy has unfolded, covering from electro-optic boards to transceiver development, on-chip and on-board switches and interconnects as well as novel noninvasive monitoring methods for light controlling for compensating thermal crosstalk, fabrication tolerances and unstable environmental conditions. At the same time, the increased penetration of optical interconnects in short-reach communications, such as board and chip levels, has triggered new challenges in photonics technologies calling for new innovations with respect to integration techniques and packaging methods. This Special Issue of IET includes integrated circuits, assembly techniques, systems-level architectures, and controlling methods related to short reach ‘chip-to-chip’, ‘in-the-box’, and ‘box-to-box’ communications. Specific areas covered by this special issue include: Silicon photonic transceiver technologies and flat DC topologies; High-speed electro-optic hosting boards and fast, energy-efficient electronics; DC switches and co-packaged optic (CPO) technologies; and Noninvasive controlling techniques for complex photonic integrated circuits. The four papers in this special section promote high-performance optical interconnect architectures and key technologies to tackle the rising bandwidth demands of DCs. They cover (a) co-packaged DC optics, (b) silicon technology transceiver modules and high-speed hosting boards for multisocket server board (MSB) interconnects and (c) automated controlling architecture for complex photonic systems; Items (b) and (c) draw from impressive achievements accomplished through the recently completed European H2020 Project ICT-STREAMS. This report written by C. Mickenberg tries to shed light on motivating factors behind co-packaged optics (CPO), the trade-offs involved and their potential impact on data centre network architectures. The article reviews the evolution of DC switch application-specific integrated circuit (ASICs) and DC optics, provides a future road map and finally discusses the potential impact of CPO on the network architecture. This topic is covered by two articles. The first, written by M. Morallis-Pegios, proposes using silicon photonics to advance interconnect throughput, socket connectivity and energy efficiency in MSB layouts by enabling a flat-topology wavelength division multiplexing (WDM) point-to-point optical MSB interconnect scheme. WDM silicon photonic transceivers coassembled with their electronic circuits achieving a 50-Gb/s line rate and 400-Gb/s aggregate data transmission are demonstrated. The article also proposes silicon photonic Arrayed Waveguide Grating Router (AWGRs) that can offer collision-less time-of-flight connectivity for up to 16 nodes. The second article covering that area is written by T. Lamprecht and reports on the development of electro-optic hosting boards for on-board transceiver subassemblies which is required to eventually form the Tb/s multi-socket onboard communication. The article also discusses the electrical interface as well as photonic board requirements, including the manufacturing technique and the assembly concept that was followed to deliver terabyte per second multichip optical communication. This article, written by F. Zanetto, introduces a novel dithering technique for real-time control of multiple cascaded photonic integrated devices, proposing a calibration-free strategy to support the development and real-life diffusion of increasingly complex photonic systems. The article elaborates on the implementation of a calibration-free automated control loop exploiting integral controllers to close a real-time feedback and stabilise the optimum working point of the employed photonic devices. The effectiveness of the proposed architectural approach is experimentally validated with the error-free routing and transmission of a 30-Gbit/s optical signal traversing a cascade of micro-ring resonator filters. All of the articles selected for this Special Issue present a wide variety of key topics and significant results addressing optical interconnects in high-performance data communication systems, including new progress on architectures, systems and packaging techniques as well as novel optimised controlling methods. As guest editors, we would like to thank all of the authors for their high-quality submissions that significantly contribute to the optical interconnect field. We would also like to thank the reviewers for their valuable and constructive technical comments that greatly enhance the Special Issue, because their feedback and comments ensure the high quality of the IET Optoelectronics Journal. We hope that this IET Special Issue will provide a useful reference of the state-of-the-art in optical interconnects to readers, and we anticipate such technological perspectives will stimulate new advancements and technological outcomes that will in turn enrich the growing field of optical interconnects. This work was supported by the E.C. through the H2020-ICT-STREAMS research project (688172).

The advantage of Smart Grid and ICT communication brings up the power reliability achieved 99.999%. Data Center availability is critical concern of business continuous and growth in Internet service competitions. Data Center Tier IV requirement is design to accomplish 99.995% available operations. However, this legacy design for availability is coming up with high investment of redundant equipments and connectivity. Understanding of Data Center power distribution system (DCPDS) cascading failures lead to optimal design equipments and connectivity topology. To take advantage of Smart Grid to reduce investment of Data Center, this paper proposes a cognitive approach model of Next Generation Data Center (NGDC) design under Smart Grid ecosystem. Load shedding technique is presented in this research; it is applied to different parts of the power distribution system to protect the critical application servers by shutting down certain pre-arranged electric loads or IT equipments when the Data Center demand response exceeded power supply capacity.

Nowadays, moderating energy consumption and building eco-friendly computing infrastructure is a major goal in large data centers. Moreover, data center energy usage has risen dramatically over the past decade and will continue to grow in-step with the High Performance Computing (HPC) intensive workloads which are at the heart of our modern life. The recent advances in the technology has driven the data center into a new phase of expansion featuring solutions with higher density. To this end, much has been done to increase server efficiency and IT space utilization. In this chapter, we will provide a state-of-the-art overview as regards energy-efficiency in High Performance Computing (HPC) facilities while describing the open challenges the research community has to face in the coming years to enable the building and usage of an Exascale platform by 2020.

Data center (DC) electricity use is increasing at an annual rate of over 20% and presents a concern for the Information Technology (IT) industry, governments, and the society. A large fraction of the energy use is consumed by the compressor cooling to maintain the recommended operating conditions for IT equipment. The most common way to improve the DC efficiency is achieved by optimally provisioning the cooling power to match the global heat dissipation in the DC. However, at a more granular level, the large range of heat densities of today's IT equipment makes the task of provisioning cooling power optimized to the level of individual computer room air conditioning (CRAC) units much more challenging. Distributed sensing within a DC enables the development of new strategies to improve energy efficiency, such as hot spot elimination through targeted cooling, matching power consumption at rack level with workload schedule, and minimizing power losses. The scope of Measurement and Management Technologies (MMT) is to develop a software tool and the underlying sensing technology to provide critical decision support and control for DC and telecommunication facilities (TF) operations. A key aspect of MMT technology is integration of modeling tools to understand how changes in one operational parameter affect the overall DC response. It is demonstrated that reduced ordered models for DC can generate, in less than 2 seconds computational time, a three dimensional thermal model in a 50 kft{sup 2} DC. This rapid modeling enables real time visualization of the DC conditions and enables 'what if' scenarios simulations to characterize response to 'disturbances'. One such example is thermal zone modeling that matches the cooling power to the heat generated at a local level by identifying DC zones cooled by a specific CRAC. Turning off a CRAC unit can be simulated to understand how the other CRAC utilization changes and how server temperature responds. Several new sensing technologies were added to the existing MMT platform: (1) air contamination (corrosion) sensors, (2) power monitoring, and (3) a wireless environmental sensing network. All three technologies are built on cost effective sensing solutions that increase the density of sensing points and enable high resolution mapping of DCs. The wireless sensing solution enables Air Conditioning Unit (ACU) control while the corrosion sensor enables air side economization and can quantify the risk of IT equipment failure due to air contamination. Validation data for six test sites demonstrate that leveraging MMT energy efficiency solutions combined with industry best practices results in an average of 20% reduction in cooling energy, without major infrastructure upgrades. As an illustration of the unique MMT capabilities, a data center infrastructure efficiency (DCIE) of 87% (industry best operation) was achieved. The technology is commercialized through IBM System and Technology Lab Services that offers MMT as a solution to improve DC energy efficiency. Estimation indicates that deploying MMT in existing DCs can results in an 8 billion kWh savings and projection indicates that constant adoption of MMT can results in obtainable savings of 44 billion kWh in 2035. Negotiations are under way with business partners to commercialize/license the ACU control technology and the new sensor solutions (corrosion and power sensing) to enable third party vendors and developers to leverage the energy efficiency solutions.

With growth of IT industry in India, there has been a long felt need for a forum where active researchers in CS and IT from academic institutions and corporate researchers interact, with focus on bringing rigor to enterprise computing, a focus area where corporates in the Indian IT industry help global clients . ACM Bangalore Compute 2008 is the first such attempt providing an integrated forum, with focus on enterprise computing. The aim of Compute 2008 conference, sponsored and organized by ACM Bangalore chapter, is to provide a forum for researchers in CS and IT to engage in a symbiotic relationship, with benefits for the Indian IT industry and CS Academia alike. The goal of the conference has been to look into the future of computing from the perspective of researchers, industries, and investors with an enterprise focus. The conference has the following plenary speakers: • Umesh Dayal (HP Labs): Trends and Challenges in Enterprise-Scale Business Intelligence • Peter Hofstee (IBM Research): Beyond multicore: Cell/Broadband Engine and trends in heterogeneous computing • Ram Shriram (Sherpalo Ventures): Innovation in Consumer Services • Pratap Pattnaik (IBM Research): Server Design for Tomorrow's Data Center Our thanks go the invited plenary speakers who have traveled long to address and participate in the conference. The program spans areas with enterprise focus, in broadly four tracks: • Software Services Engineering, drawing a wide range of areas right from software life cycle engineering aspects to newer areas like business process management and services oriented architecture. • High Performance Computing covering broadly emerging areas of grid computing, high performance computing, next generation data centers, to name a few. • Information management and Graphics covering the areas of multimedia, graphics, imaging, and emerging trends in data management, structured and unstructured information management. • Pervasive computing track covering wide areas of enterprise mobility, ad-hoc networking and sensor networking, to name a few. Apart from the regular paper and poster sessions, we have 13 tutorial workshops in related areas ranging from ½ day to full day, and a doctoral symposium for researchers to get a validation of their thoughts and ideas. Our thanks go to the enthusiastic tutorial organizers, doctoral symposium participants and organizers. The conference received a total of 124 paper and poster submissions. The submission and review was facilitated by EasyChair. Thanks go to EasyChair authors that made the process simple and collaborative. Each paper was reviewed by at least three reviewers (PC members or their nominees) and were selected through a PC meeting ranging over a week. The efforts of the authors and reviewers has resulted in an excellent programme consisting of 20 full papers and 9 poster papers.

Data centres are large energy consumers, which have become more energy efficient in recent years due to increased awareness of opportunities, increasing energy costs and corporate social responsibility pressures. A lifecycle assessment approach identifies two other significant areas of impact: the embodied impact of IT equipment and mechanical and electrical plant, and the electricity source used in operational and embodied processes. Many data centre operators publicise their energy efficiency, some also report on their carbon footprint. However, there is a need for simple tools in order to help operators better understand and quantify the embodied impact and inform green procurement. Focussing purely on energy efficiency may cause a burden shift, e.g. by replacing equipment with more efficient equipment but increasing the embodied impact. The total environmental impact could remain the same/increase, but with the perception of a ‘greener’ data centre. Although currently there is limited data available for data centre life cycle assessments and the process is resource intensive, research has identified which factors significantly impact a facility’s environmental impact. This knowledge should be used in the design process and throughout the data centre lifecycle to minimise data centre environmental impact. Practical application: As the data centre industry continues to grow and its sustainability receives closer scrutiny, it is important to increase awareness of where its highest environment impacts lie and analyse the factors which influence this. This can help inform policy and decision-making to support the design and operation of data centres which are truly more sustainable. It is important that this research is not just theoretical but can translate into practical actions which can be implemented in a cost-effective manner for the benefit of everyone. The current focus in industry has been energy consumption and energy efficiency; many practices which were considered innovative a few years ago have now become standard best practice. It is time to examine which other areas should be prioritised for improvement.

Bandwidth hungry applications such as real-time video processing are currently straining data center networking infrastructure. This is expected to become a more serious challenge in the near future. Optics enables massive data transmission rates: in telecom settings researchers have demonstrated rates in excess of 100 Tb/s in a single mode fiber; however, the data center presents a different set of challenges. Low cost, low power dissipation, low latency, small physical size, and the ability to integrate with mainstream silicon electronics are more important than the ability to transmit signals over long distances, the traditional telecoms focus. A clearer understanding of the requirements and technology challenges of the new applications which are changing the face of the Internet and the data center, should facilitate development of optical interconnection networks that will solve these challenges. By linking the end-to-end research areas we seek to identify critical technology gaps and enable the design of forward-looking, efficient, cost-effective, higher bandwidth data centers that can take optimal advantage of advances in multicore processors.KeywordsCloud ComputingData CenterInterconnection NetworkHigh BandwidthRing ResonatorThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Growing concern for reduced power dissipation, cost and latency demands in next generation Data Centers (DC) motivates us to revisit header optimizations. Headers contribute to about 30–40% of DC traffic and is responsible for equal proportion of consumed power. This amounts to significant overhead on per byte transfer of payload. In the past, highly inflexible switches have limited the focus of header optimizations primarily on end-host or edge network routers. Further, strict compliance to protocols by switches/routers along the path have limited the segments of headers that can be optimized. Recent evolution of Software Defined Network (SDN) brings in several opportunities to control over switches. We believe that if SDN's finer handle to the switching devices is exploited, it can help deal with header information. In this paper, we exploit the capabilities of SDN and introduce a new functionality that can effectively replace the redundant and repetitive header information with shorter unique identity. We present Scissor that trims the headers lower in the protocol stack. As a replacement for routing, we introduce the notion of Flow-ID, where all packets belonging to a flow are identified using this unique Flow-ID. We leverage the capabilities of the SDN to dynamically allow switching devices to route the packets based on the Flow-IDs. Our approach of trimming header at the switching devices leaves the hosts unmodified making it highly adoptable for DC environment. We show that our approach can decrease switch fabric power consumption by a factor of 2.5 compared to an existing L2 switch and reduce the latency by about 30% for a significant fraction (around 30–50%) of the network traffic.

The 2007 US EPA report to Congress (US EPA, 2007) on the state of energy consumption in data centers brought to light the true energy inefficiencies built into today’s data centers. Marquez et al. (2008) conducted an initial analysis on the productivity of a Pacific Northwest National Lab computer using The Green Grid’s Data Center Energy Productivity metric (The Green Grid, 2008). Their study highlights how the Top500 ranking of computers disguises the serious energy inefficiency of today’s High Performance Computing data centers. In the rapidly expanding Cloud Computing space, the race will be won by the providers that deliver the lowest cost of computing — such cost is heavily influenced by the operational costs incurred by data centers. As a means to address the urgent need to lower the cost of computing, solution providers have been intensely focusing on real-time monitoring, visualization, and control/management of data centers. The monitoring aspect involves the widespread use of networks of sensors that are used to monitor key data center environmental variables such as temperature, relative humidity, air flow rate, pressure, and energy consumption. Such data is then used to visualize and analyze data center problem areas (e.g., hotspots), which is then followed by control/management actions designed to alleviate such problem areas. The authors have been researching the operational benefits of a network of sensors tied in to a software package that uses the data to visualize, analyze, and control/manage the data center cooling system and IT Equipment for maximum operational efficiency. The research is being conducted in a corporate production data center that is networked in to the authors’ company’s global network of data centers. Results will be presented that highlight the operational benefits that are realizable through real-time monitoring and visualization.

The share of equipment and power use in smaller data centers (DCs) is comparable with that of more massive counterparts. However, they grabbed less attention in the literature despite being less energy-efficient. This study highlights the challenges of setting up a computational fluid dynamics (CFD) model of a 180-m2 small-size high-performance computing (HPC) DC and the validation procedure leading to a reasonably accurate model for the investigation of the thermal environment and potential energy efficiency improvements. Leaky floors, uneven placement of computing equipment and perforated tiles preventing separation of hot and cold air, low-temperature operation, and excessive cooling capacity and fan power were identified sources of energy inefficiency in the DC. Computational fluid dynamics model predictions were gradually improved by using experimental measurements for various boundary conditions (BCs) and detailed geometrical representation of large leakage openings. Eventually, the model led to predictions with an error of less than 1 °C at the rack inlet and less than 5 °C at the rack outlet. The ultimate objective was to use the validated CFD model to test various energy efficiency measures in the form of operational or design changes in line with the best practices. Impact of leakage between the raised floor and the room, reduced airflow rate, cold-aisle and hot-aisle separation, workload consolidation, and higher temperature operation were among the phenomena tested by using the validated CFD model. The estimated power usage effectiveness (PUE) value reduced from 1.95 to 1.40 with the proposed energy efficiency measures.

Two data centers in this study were within a co-location facility located on the sixth floor of a multi-story building in downtown Los Angeles, California. The facility had 37,758 gross square feet floor area with 2-foot raised-floors in the data services area. The two data centers were designated as the west data center (DC No.18) and the east data center (DC No.19). The study found that 56% of the overall electric power was consumed by sixth floor critical loads in both data centers, 33% of the power was consumed by HVAC systems, 3% of the power was consumed by UPS units, 3% of the power was for generator losses, and the remaining 5% was used by lighting and miscellaneous loads in the building. The power density of installed computer loads (rack load) in the two data centers was 20 W/ft{sup 2} and 56 W/ft{sup 2}, respectively. The power density was relatively lower in DC No.18 compared to other data centers previously studied. In addition, HVAC to IT power demand ratio was 0.6 in DC No.18 in this study, and was 0.4 in DC No.19. Two out of three chillers were running at a low partial load, making the operation very energy inefficient. The operation and control of the chillers and air-handling units should be optimized while providing sufficient cooling to the data centers. Although arranging hot aisle/cold aisle design to separate airflow streams would be difficult in such a co-location data center, optimizing air distribution should be pursued. General recommendations for improving overall data center energy efficiency include improving the design, operation, and control of mechanical systems serving the data centers with various critical loads in place. This includes chiller operation, chilled water system, AHUs, airflow management and control in data centers. Additional specific recommendations or considerations to improve energy efficiency are provided in this report.

Environment-conscious scheduling of HPC applications on distributed Cloud-oriented data centers

Data center (DC) and high performance computing (HPC) applications have traditionally used a combination of copper, multimode fiber and single-mode fiber interconnects with relative percentages that depend on factors such as the line rate, reach and connectivity costs. The balance between these transmission media has increasingly shifted towards optical fiber due to the reach constraints of copper at data rates of 10 Gb/s and higher. The percentage of single-mode fiber deployed in the DC has also grown slightly since 2014, coinciding with the emergence of mega DCs with extended distance needs beyond 100 m. This trend will likely continue in the next few years as DCs expand their capacity from 100G to 400G, increase the physical size of their facilities and begin to utilize silicon-photonics transceiver technology. However there is a still a need for the low-cost and high-density connectivity, and this is sustaining the deployment of multimode fiber for links ≤ 100 m. In this paper, we discuss options for single-mode and multimode fibers in DCs and HPCs and introduce a reduced diameter multimode fiber concept which provides intra-and inter-rack connectivity as well as compatibility with silicon-photonic transceivers operating at 1310 nm. We also discuss the trade-offs between single-mode fiber attributes such as bend-insensitivity, attenuation and mode field diameter and their roles in capacity and connectivity in data centers.

The architecture of several data centers have been proposed as alternatives to the conventional three-layer one.Most of them employ commodity equipment for cost reduction. Thus, robustness to failures becomes even more important, because commodity equipment is more failure-prone. Each architecture has a different network topology design with a specific level of redundancy. In this work, we aim at analyzing the benefits of different data center topologies taking the reliability and survivability requirements into account. We consider the topologies of three alternative data center architecture: Fat-tree, BCube, and DCell. Also, we compare these topologies with a conventional three-layer data center topology. Our analysis is independent of specific equipment, traffic patterns, or network protocols, for the sake of generality. We derive closed-form formulas for the Mean Time To Failure of each topology. The results allow us to indicate the best topology for each failure scenario. In particular, we conclude that BCube is more robust to link failures than the other topologies, whereas DCell has the most robust topology when considering switch failures. Additionally, we show that all considered alternative topologies outperform a three-layer topology for both types of failures. We also determine to which extent the robustness of BCube and DCell is influenced by the number of network interfaces per server.