POLYMER WAVEGUIDE MANUFACTURING AND PRINTED CIRCUIT BOARD INTEGRATION

In high performance computing (HPC) systems, optical network links are often utilized for the HPC networks of these systems, but they have a relatively high rate of failure compared to their electrical counterparts. Due to the high link failure rate, evaluating the impact of these failures on HPC workloads is of particular interest. We extended the Merlin network module of the Structural Simulation Toolkit (SST) in order to evaluate the impact of link failures on applications running on HPC systems which use dragonfly network topologies.We focus on dragonfly topologies as they are frequently found in HPC systems, including NERSC Cori and Edison systems.We demonstrate our changes to SST by providing a sample of performance results and routing statistics for a dragonfly network of 8,192 nodes and three HPC workloads with 1% of optical link failures. For the three motifs under consideration, we show that the impact of link failure is largely dependent on the underlying workloads running on the system.

Usually, scientific applications outlive the lifespan of the High Performance Computing (HPC) systems for which they are initially developed. The innovations in the HPC systems’ hardware and parallel programming standards drive the modernization of HPC applications so that they continue being performant. While such code modernization efforts may not be challenging for HPC experts and well-funded research groups, many domain-experts and students may find it challenging to adapt their applications for the latest HPC systems due to lack of expertise, time, and funds. The challenges of such domain-experts and students can be mitigated by providing them high-level tools for code modernization and migration. A brief overview of two such high-level tools is presented in this chapter. These tools support the code modernization and migration efforts by assisting users in parallelizing their applications and porting them to HPC systems with high-bandwidth memory. The tools are named as: Interactive Parallelization Tool (IPT) and Interactive Code Adaptation Tool (ICAT). Such high-level tools not only improve the productivity of their users and the performance of the applications but they also improve the utilization of HPC resources.

In this study, a low-cost (with bare chips) and high (optical, electrical, and thermal) performance optoelectronic system with a data rate of 10Gbps is designed and analyzed. This system consists of a rigid printed circuit board (PCB) made of FR4 material with an optical polymer waveguide, a vertical cavity surface emitted laser (VCSEL), a driver chip, a 16:1 serializer, a photo-diode detector, a Trans-Impedance Amplifier (TIA), a 1:16 deserializer, and heat spreaders. The bare VCSEL, driver chip, and serializer chip are stacked with wire bonds and then solder jointed on one end of the optical polymer waveguide on the PCB via Cu posts. Similarly, the bare photo-diode detector, TIA chip, and deserializer chip are stacked with wire bonds and then solder jointed on the other end of the waveguide on the PCB via Cu posts. Because the devices in the 3D stacking system are made with different materials, the stresses due to the thermal expansion mismatch among various parts of the system are determined.

Vertical cavity surface emitting lasers (VCSELs) are low cost and reliable light sources for high-speed local area and storage area network (LAN/SAN) optical fiber data communication systems and short-reach computer interconnects. The continuing rapid increase of serial transmission data rates driven by multi-core microprocessor's bandwidth upgrades cannot be sustained via conventional copper-based links as bit rates move beyond 10 Gbit/s and distances greater than 1 m. The intrinsic limitation of copper at high single-channel data rates facilitates the need to transition to optical fiberbased links at ever shorter distances. For LAN/SAN applications the 850 nm wavelength is standard. This same wavelength is also the standard for several other evolving short-reach application areas including Fibre Channel, CEI, USB, InfiniBand, and HDMI optical link systems. Herein we present our recent results on 850 nm oxide-confined VCSELs operating at data bit rates up to 40 Gbit/s. The low operational current density in the range of ~10 kA/cm2 ensures viable device reliability and long-term stability based on well-known industry certification specifications. Key VCSEL device parameters including the relaxation resonance frequency, damping, and parasitic cut-off frequency are determined for VCSELs with oxide-confined apertures of various diameters. We find that a parasitic cut-off frequency of 24-28 GHz limits the VCSEL's high speed operation at the highest optical modulation rates. We believe that with some effort the device parasitics can be further reduced such that current modulated VCSELs can be realized with larger than 30 GHz optical modulation bandwidth and reliable and practical operation beyond 40 Gbit/s.

Vertical cavity surface emitting laser (VCSEL) is an important laser source for their evident plentiful applications in optical communication. The present investigation reports a comparison of the modeling and optimization of long wavelengths 1310 nm and 1550 nm high speed short cavity VCSEL for continuous wave operation at various temperature (283-3230K) for various diameters. The continuous wave lasing is demonstrated for the device diameter from 2 to 5 μm with threshold current of 1.07-1.33 mA with threshold power consumption of 1.86-2.57 mW for 1310 nm and threshold current of 0.94-1.24 mA and threshold power consumption 1.67-2.1 mW for 1550 nm VCSEL. The results demonstrate that the threshold current, peak emitted power and power consumption increases with the increase in device diameter. The results confirm that VCSELs with 2 μm diameter is most suitable to achieve energy-efficient operation. Although rollover current increases with the diameter, but, due to the advantage of lower threshold current and power consumption, VCSEL having smaller diameter is best suited. The power conversion efficiency for proposed long wavelength VCSELs is approximately 50% which is extremely useful for low power applications. The proposed VCSELs are suitable for very short reach (<2 m) optical interconnects such as chip-to-chip and board-to-board communication in high performance computers.

As the size and complexity of high performance computing (HPC) systems grow in line with advancements in hardware and software technology, HPC systems increasingly suffer from performance variations due to shared resource contention as well as software- and hardware-related problems. Such performance variations can lead to failures and inefficiencies, which impact the cost and resilience of HPC systems. To minimize the impact of performance variations, one must quickly and accurately detect and diagnose the anomalies that cause the variations and take mitigating actions. However, it is difficult to identify anomalies based on the voluminous, high-dimensional, and noisy data collected by system monitoring infrastructures. This paper presents a novel machine learning based framework to automatically diagnose performance anomalies at runtime. Our framework leverages historical resource usage data to extract signatures of previously-observed anomalies. We first convert collected time series data into easy-to-compute statistical features. We then identify the features that are required to detect anomalies, and extract the signatures of these anomalies. At runtime, we use these signatures to diagnose anomalies with negligible overhead. We evaluate our framework using experiments on a real-world HPC supercomputer and demonstrate that our approach successfully identifies 98 percent of injected anomalies and consistently outperforms existing anomaly diagnosis techniques.

In addition to the performance, power consumption has become a major concern in High Performance Computing (HPC) systems. Typically the cooling system and the IT loads are the major contributors to the power bills. Understanding the power consumption at different granular levels in the HPC system is a first step to quantify the problems in HPC system. By proper monitoring and effective utilization of the cooling system, the power requirements of the HPC facility can be effectively met. In this paper we present a system where node level power measurement and WSN based rack level temperature measurement are used to provide localized control of cold air supply. A Smart Power Monitoring and Distribution Unit (PMDU) is designed and developed, to replace the existing Power Distribution Unit (PDU) in HPC. This can measure and report the power consumption, to support power profiling of large scale HPC system. This measured data are communicated to a base station via Ethernet. This base station collects all such measurements which can be used for power profiling of IT load of the HPC system. This helps to provide better insight into the power utilization pattern. Wireless Sensor Network (WSN) is used to collect exhaust and inlet air temperature of the server nodes and this information is used for directing the cold air effectively. A vent control system is designed and fabricated for intelligently directing the air flow to the server node inlet. It takes the node power and the temperature data as its inputs. This enables supplying/ redirecting more cold air towards under-cooled nodes without creating an extra load on the cooling system, thereby bringing in effective cooling at reduced power consumption. As an added advantage this could help in hot-spots mitigation.

Data rates are continuing to increase for box-to-box, rack-to-rack, board-to-board, and chip-to-chip interconnects for terabit switches and routers, multiprocessor computers and high-end servers. The increase in individual line rates and bandwidth drives the need to replace copper interconnects with optical interconnects. Fiber optics are advantageous for these applications because they allow for longer link lengths, increased bandwidth, smaller cables and connectors, less susceptibility to electromagnetic interference, and potentially lower power dissipation, depending on the data rate and link length. Towards this direction, this thesis aims to design and demonstrate low-cost, low-latency, high throughput, rack-to-rack optical interconnect architectures for exascale (i.e., performing 10^18 floating point operations per second) high-performance computing (HPC) systems and data centers. In particular, a novel, cost-effective, optical interconnect architecture for ultrafast optical switching, based on semiconductor optical amplifiers (SOAs), is studied. The proposed design of a fast N×N all-optical, wavelength-space crossbar switch for optical interconnects uses a minimum number of ON/OFF gates. This thesis compares and proves the superiority of the proposed architecture with respect to its originally-proposed counterpart, both theoretically and experimentally. Additionally, in order to increase the capacity and to minimize the impact of transmission effects (especially self-gain modulation (SGM), cross-gain modulation (XGM), self-phase modulation (SPM), cross-phase modulation (XPM), and polarization dependent gain (PDG)), we investigate the performance of conventional binary intensity modulation (IM), in conjunction with direct detection, as well as of advanced, more resilient, spectrally-efficient modulation formats (e.g., Differential Phase Shift Keying (DPSK), Orthogonal Frequency Division Multiplexing (OFDM), Polarization Division Multiplexed Quadrature Phase Shift Keying (PDM-QPSK), Single (SP)- and PDM- 16-ary Quadrature Amplitude Modulation (16QAM) in conjunction with coherent detection). Finally, as a seperate research activity, we study the performance of point-to-point links based on vertical-cavity surface-emitting lasers (VCSELs) and single- or multi- mode fibers, in conjuction with IM/DD, four-level Pulse Amplitude Modulation (4-PAM), and OFDM, to enable state-of-the-art, high-capacity, low-cost optical interconnects.

Cloud computing offers new computing paradigms, capacity and flexible solutions to high performance computing (HPC) applications. For example, Hardware as a Service (HaaS) allows users to provide a large number of virtual machines (VMs) for computation-intensive applications using the HaaS model. Due to the large number of VMs and electronic components in HPC system in the cloud, any fault during the execution would result in re-running the applications, which will cost time, money and energy. In this paper we presented a proactive fault tolerance (FT) approach to HPC systems in the cloud to reduce the wall-clock execution time and dollar cost in the presence of faults. We also developed a generic FT algorithm for HPC systems in the cloud. Our algorithm does not rely on a spare node prior to prediction of a failure. We also developed a cost model for executing computation-intensive applications on HPC systems in the cloud. We analysed the dollar cost of provisioning spare nodes and checkpointing FT to assess the value of our approach. Our experimental results obtained from a real cloud execution environment show that the wall-clock execution time and cost of running computation-intensive applications in cloud can be reduced by as much as 30%. The frequency of checkpointing of computation-intensive applications can be reduced up to 50% with our FT approach for HPC in the cloud compared with current FT approaches.

Scientific workflows are increasingly transferring large amounts of data between high performance computing (HPC) systems. Even though these HPC systems are connected via high-speed dedicated networks and use dedicated data transfer nodes (DTNs), it is still difficult to predict the data transfer throughput because of variations in data transfer protocols, host configurations, performance of file systems, and overlapping workloads. In order to provide reliable performance prediction for better resource management and job scheduling, we need models for predicting data transfer throughput under real-world conditions. In this paper, we explore different machine learning approaches for building data-driven models to improve performance and prediction of large-scale data transfer throughput. In addition to the variables already collected by the network monitoring system, we also develop heuristics to derive additional metrics for improving the prediction accuracy. We use the prediction results to identify the importance of different network parameters in predicting the throughput for large-scale data transfers. Through extensive tests, we identify key network parameters, discover interesting variations among different HPC sites, and show that we can predict throughput with high accuracy. We also analyze our models and results to provide recommendations for improving the performance of big data transfers.

In recent years, High Performance Computing (HPC) systems have been shifting from expensive massively parallel custom architectures to clusters of commodity personal computers to take advantage of cost and performance benefits. To avoid having to restart an application in case of sudden failure, checkpointing/restart fault tolerance mechanisms are commonly implemented. One drawback to checkpointing/restart is that it creates an overhead which increases the execution time of an application. We present a theoretical analysis of our technique. The results show that the PLR checkpointing/restart can significantly improve the overall reliability of an HPC system.

In this paper, we propose a higher efficiency optical coupling technique using 90-degree-bent Graded-Index (GI) core polymer waveguide which replaces conventional optical assembly (mirrors and lenses) in vertical cavity surface emitting laser (VCSEL) based optical engines. The 90-degree-bent GI-core polymer waveguides are fabricated using the Mosquito method. However, high coupling loss could be caused by the VCSEL beam divergence due to diffraction in the lens free coupling. We theoretically confirm that total insertion loss including bending loss as well as coupling loss could remain low by the waveguide NA control. Fabricating the 90-degree-bent GI-core polymer waveguides with the optimized NA, a waveguide insertion loss as low as about 2 dB is experimentally achieved.

Communication technologies such as wireless and optical fiber are vital components of today's environment. As demand for increased data transfer and new technologies increases, novel research must be performed to combine the benefits of both technologies. In this case, optical communication technologies serve as the backbone of wireless technology. Currently, only a few studies have been conducted on Free Space Optical (FSO) communication employing Field Programmable Gate Arrays (FPGAs) and VerticalCavity SurfaceEmitting Lasers (VCSELs). When VCSEL is paired in conjunction with FPGA, there are only very few existing designs. These published researches cover many subjects yet are exceedingly complex and fragmented. The principal purpose of this Systematic Literature Review (SLR) is to identify the research trends and emerging themes in the development of FPGA-based Optical Hybrid Fiber - FSO Communication Systems. This review incorporates articles from prestigious journals that were indexed in scientific databases such as Science Direct, Shodh Ganga, IEEE Xplore, Google Scholar, and Scopus. This SLR exhibits the reliability and compatibility of FPGA-based FSOs when used with a lowcost optical source VerticalCavity SurfaceEmitting Laser Diode VCSEL), a range of technologies, communication methods, and approaches. This FPGA-based FSO is designed for optical communication applications that require highly secure transmission schemes, low radiation noise, a low bit error rate (BER), and increased data transfer rates. Additionally, VCSEL has a current threshold that is low, maximum conversion efficiency of power, and least consumption of power. Findings of this study can be used to direct future research in the area of hybrid Fiber-FSO optical communication systems development utilizing VCSEL and FPGA.

In this paper, we present a low-loss optical assembly utilizing a 90°-bent graded-index (GI) core polymer optical waveguide on vertical cavity surface emitting laser (VCSEL) based optical transceivers. The proposed assembly can replace conventional components such as mirrors and lenses for realizing subminiature optical engines applicable to on-board integration. To minimize the total insertion loss of the waveguide when connected to a high-speed VCSEL and a GI-core multimode fiber (MMF) at each end, the characteristics of the beam emitted from VCSELs are measured and taken into consideration for the waveguide design. In order to confirm the effect of insertion loss reduction by the waveguide numerical aperture control, 90°-bent GI-core polymer waveguides are fabricated applying the Mosquito method. The fabricated waveguide exhibits a total insertion loss as low as about 2 dB at 850-nm wavelength, which includes the coupling losses at both ends, bending, and propagation losses. We also investigate a way to reduce the insertion loss when a gap exists between the waveguide and VCSEL chip. We theoretically and experimentally confirm that filling the gap with a high index resin can reduce the coupling loss by 5 dB.

On high-performance computing (HPC) systems, job allocation strategies control the placement of a job among available nodes. As the placement changes a job’s communication performance, allocation can significantly affects execution times of many HPC applications. Existing allocation strategies typically make decisions based on resource limit, network topology, communication patterns, etc. However, system network performance at runtime is seldom consulted in allocation, even though it significantly affects job execution times.In this work, we demonstrate using monitoring data to improve HPC systems’ performance by proposing a NetworkData-Driven (NeDD) job allocation framework, which monitors the network performance of an HPC system at runtime and allocates resources based on both network performance and job characteristics. NeDD characterizes system network performance by collecting the network traffic statistics on each router link, and it characterizes a job’s sensitivity to network congestion by collecting Message Passing Interface (MPI) statistics. During allocation, NeDD pairs network-sensitive (network-insensitive) jobs with nodes whose parent routers have low (high) network traffic. Through experiments on a large HPC system, we demonstrate that NeDD reduces the execution time of parallel applications by 11% on average and up to 34%.