Designing Photonic Processors for Ultra-Fast Data Processing in High-Performance Computing

Introduction. The significance of machine learning under the conditions of digital transformation of industry, and methods of implementing deep learning to provide the performance of trust management systems are considered. The necessity of using convolutional artificial neural networks for deep machine learning is determined. Various technologies and architectures for the implementation of artificial neural networks are briefly considered; a comparative analysis of their performance is carried out. The work objective is to study the need to develop new approaches to the architecture of computing machines for solving problems of deep machine learning in the trust management system implementation.Materials and Methods. In the context of digital transformation, the use of artificial intelligence reaches a new level. The technical implementation of artificial neural systems with deep machine learning is based on the use of one of three basic technologies: high performance computing (HPC) with parallel data processing, neuromorphic computing (NC), and quantum computing (QC).Results. Implementation models for deep machine learning, basic technologies and architecture of computing machines, as well as requirements for trust assurance in control systems using deep machine learning are analyzed. The problem of shortage of computation power for solving such problems is identified. None of the currently existing technologies can solve the full range of learning and impedance problems. The current level of technology does not provide information security and reliability of neural networks. The practical implementation of trust management systems with deep machine learning based on existing technologies for a significant part of the tasks does not provide a sufficient level of performance.Discussion and Conclusions. The study made it possible to identify the challenge of the computation power shortage for solving problems of deep machine learning. Through the analysis of the requirements for trust management systems, the external challenges of their implementation on the basis of existing technologies, and the need to develop new approaches to the computer architecture are determined.

We are delighted to introduce a Special Issue on Optical Interconnects for the IET Optoelectronics magazine. This issue presents the current status as well as some of the outstanding progress that has been witnessed in the rapidly evolving area of optical interconnects. Today, data centres (DC) and high-performance computers are struggling with limited interconnect capacity and increased power consumption while trying to meet overwhelming bandwidth demands. As photonics technologies have been rapidly maturing, optical interconnects have come into the spotlight as the means to cope with the emerging bandwidth, energy, and capacity needs. As such, a wealth of photonic interconnect solutions across the DC hierarchy has unfolded, covering from electro-optic boards to transceiver development, on-chip and on-board switches and interconnects as well as novel noninvasive monitoring methods for light controlling for compensating thermal crosstalk, fabrication tolerances and unstable environmental conditions. At the same time, the increased penetration of optical interconnects in short-reach communications, such as board and chip levels, has triggered new challenges in photonics technologies calling for new innovations with respect to integration techniques and packaging methods. This Special Issue of IET includes integrated circuits, assembly techniques, systems-level architectures, and controlling methods related to short reach ‘chip-to-chip’, ‘in-the-box’, and ‘box-to-box’ communications. Specific areas covered by this special issue include: Silicon photonic transceiver technologies and flat DC topologies; High-speed electro-optic hosting boards and fast, energy-efficient electronics; DC switches and co-packaged optic (CPO) technologies; and Noninvasive controlling techniques for complex photonic integrated circuits. The four papers in this special section promote high-performance optical interconnect architectures and key technologies to tackle the rising bandwidth demands of DCs. They cover (a) co-packaged DC optics, (b) silicon technology transceiver modules and high-speed hosting boards for multisocket server board (MSB) interconnects and (c) automated controlling architecture for complex photonic systems; Items (b) and (c) draw from impressive achievements accomplished through the recently completed European H2020 Project ICT-STREAMS. This report written by C. Mickenberg tries to shed light on motivating factors behind co-packaged optics (CPO), the trade-offs involved and their potential impact on data centre network architectures. The article reviews the evolution of DC switch application-specific integrated circuit (ASICs) and DC optics, provides a future road map and finally discusses the potential impact of CPO on the network architecture. This topic is covered by two articles. The first, written by M. Morallis-Pegios, proposes using silicon photonics to advance interconnect throughput, socket connectivity and energy efficiency in MSB layouts by enabling a flat-topology wavelength division multiplexing (WDM) point-to-point optical MSB interconnect scheme. WDM silicon photonic transceivers coassembled with their electronic circuits achieving a 50-Gb/s line rate and 400-Gb/s aggregate data transmission are demonstrated. The article also proposes silicon photonic Arrayed Waveguide Grating Router (AWGRs) that can offer collision-less time-of-flight connectivity for up to 16 nodes. The second article covering that area is written by T. Lamprecht and reports on the development of electro-optic hosting boards for on-board transceiver subassemblies which is required to eventually form the Tb/s multi-socket onboard communication. The article also discusses the electrical interface as well as photonic board requirements, including the manufacturing technique and the assembly concept that was followed to deliver terabyte per second multichip optical communication. This article, written by F. Zanetto, introduces a novel dithering technique for real-time control of multiple cascaded photonic integrated devices, proposing a calibration-free strategy to support the development and real-life diffusion of increasingly complex photonic systems. The article elaborates on the implementation of a calibration-free automated control loop exploiting integral controllers to close a real-time feedback and stabilise the optimum working point of the employed photonic devices. The effectiveness of the proposed architectural approach is experimentally validated with the error-free routing and transmission of a 30-Gbit/s optical signal traversing a cascade of micro-ring resonator filters. All of the articles selected for this Special Issue present a wide variety of key topics and significant results addressing optical interconnects in high-performance data communication systems, including new progress on architectures, systems and packaging techniques as well as novel optimised controlling methods. As guest editors, we would like to thank all of the authors for their high-quality submissions that significantly contribute to the optical interconnect field. We would also like to thank the reviewers for their valuable and constructive technical comments that greatly enhance the Special Issue, because their feedback and comments ensure the high quality of the IET Optoelectronics Journal. We hope that this IET Special Issue will provide a useful reference of the state-of-the-art in optical interconnects to readers, and we anticipate such technological perspectives will stimulate new advancements and technological outcomes that will in turn enrich the growing field of optical interconnects. This work was supported by the E.C. through the H2020-ICT-STREAMS research project (688172).

At first thought, one may not realize the need for a special issue of Parallel and Distributed Computing Practices focused on Cluster Computing when our community has already invested years of research and spent a wealth of resources on traditional Big Iron supercomputers, for instance, the SGI/Cray Origin and IBM SP. Academia, industry, and national laboratories are still using, and for the foreseeable future, will continue to use, supercomputers to solve both grand-challenge scale and high-throughput applications. Then why the interest and new popularity of research into clusters for high-performance parallel and distributed applications? Clusters incorporate the tremendous successes that are attributable to our parallel and distributed processing community. Over the next decade, clusters will span the entire range of high-performance computing platforms. Instead of being restricted solely to supercomputers for achieving high-performance computing, the spectrum of available platforms will spread out to Beowulf-class clusters, to superclusters, and on to supercomputers. Moving along this continuum represents increasing computational resources, which in turn allows the capability for solving larger classes of applications or instances of problems. Compute-bound applications on workstations often may take advantage of the extra horsepower provided by symmetric multiprocessors. Applications with even larger computational requirements and a high-degree of concurrency may use Beowulf clusters, that is, clusters that are home-built from mass-market, commercial off-the-shelf workstations and networks, for instance, PC's interconnected by Fast Ethernet. A supercluster may contain commodity SMP nodes interconnected with scalable, low-latency, high-bandwidth networks, and is integrated by a system vendor rather than the computational scientist. Applications with greater complexity, for example, grand challenges from the sciences, demand scalable systems with high-performance, interconnects, perhaps coupled with terascale I/O subsystems, and thus, are appropriate for these superclusters. Finally, supercomputers, the largest of the clusters, may provide the ultimate platform for the highest-fidelity models and simulations and data-intensive applications. At each step up this ladder of cluster architectures, the system cost increases approximately by an order of magnitude, but having the high-performance capability to solve the problem of interest may require an entry point at any of these levels, the Beowulf, or supercluster, or even supercomputer platform. Cluster computing with Beowulf- or superclusters, thus, may be one or two orders of magnitude less expensive than a supercomputer, and provide a cost-effective solution and capability that was not available on a workstation. But this cost comparison measures just the basic system price. What about the applications? As recently as a few years ago, the lack of a canonical high-performance architectural paradigm meant that migrating to a new computer system, even from the same vendor, typically required redesigning applications to efficiently use the high-performance system. Thus, the application must be redesigned with parallel algorithms and libraries that are optimized for each high-performance system. This process of porting, redesigning, optimizing, debugging, and analyzing, the application may in fact be prohibitively expensive, and certainly frustrating when the process is completed just in time to greet the next generation system! That was then, this is now. With the successful standardization of system-level interfaces, many of these issues no longer hinder our expedited use of these new, high-performance clusters. The message-passing interface (MPI) is a clear example of a standard that has contributed significantly to the success of cluster computing. Supercomputer vendors working together with the high-performance computing community embarked on a process to standardize the user-level interface and behavior for passing messages between concurrent tasks. Now, using ordinary sequential compilers and vendor-supplied or freely-available implementations of this message-passing interface, applications can be implemented efficiently on one supercomputer and ported with little or no effort to other parallel machines. The message-passing paradigm, thus, provides a reasonably good canonical high-performance model and methodology for implementing high-performance applications. An algorithm that exhibits efficient execution and scaling on one machine then is likely to perform well on another with little or no additional programming effort. Using this cluster computing paradigm, algorithms typically are optimized by minimizing the number of remote memory accesses (reducing the amount of messages passed), and carefully organizing the data placement to increase memory locality. Thus, high-performance algorithms and applications that have been optimized for supercomputers, using the same cluster paradigm, may be appropriate for Beowulf and superclusters. Cluster computing, realizing the importance of a standard high-performance programming environment that applies equally well to both ends of the architectural spectrum, highly leverages these and other successes from parallel and distributed processing research and development. This is in an exciting, new era for high-performance computing where the canonical architectural paradigm stretches from home-built Beowulf clusters, to superclusters, and on to supercomputers. But cluster computing faces some challenges ahead of it. Continued research is required for designing 1) scalable, high-performance algorithms, applications, and methodologies, that can tolerate the communication overheads, and adapt to hybrid, SMP, and heterogeneous, nodes; 2) high-performance computer architectures, scalable interconnection networks, and I/O subsystems; and 3) scalable systems software, compilers, and monitoring tools. Cluster computing has come of age, and it is high time for the parallel and distributed processing community to lay the groundwork for our own legacy of success stories. David A. Bader University of New Mexico

As bit rates increase, optical interconnects based high-performance computing (HPC) systems improve performance by increasing the available bandwidth (using wavelength-division multiplexing (WDM) and spacedivision multiplexing (SDM)) and decreasing power dissipation as compared to traditional electrical interconnects. While static allocation of wavelengths (channels) in optical interconnects provide every node with equal opportunity for communication, it can lead to network congestion for non-uniform traffic patterns. In this paper, we propose an opto-electronic interconnect for designing a flexible, highbandwidth, low-latency, dynamically reconfigurable architecture for scalable HPC systems. Reconfigurability is realized by monitoring traffic intensities, and implementing dynamic bandwidth re-allocation (DBR) technique that adapts to changes in communication patterns. We propose a DBR technique - Lock-Step (LS) that balances the load on each communication channel based on past utilization. Simulation results indicate that the reconfigured architecture shows 40% increased throughput and and 20% reduced network latency as compared to HPC electrical networks.

The papers in this special section examine neuromorphic photonics which combines optical physics and unconventional computing, resulting in a new class of ultrafast information processors for neuromorphic information and signal processing, machine learning, and high-performance computing. These processors can enable applications where low latency, high bandwidth, and low switching energies are paramount. Fundamentally, such computing concepts heavily depend on interconnects, a functionality where photonic processors can significantly outperform electronic systems. By combining the high bandwidth and efficiency of photonic devices with the adaptive, parallelism and complexity similar to the brain, photonic neural networks have the potential to be faster than conventional neural networks, while consuming less energy.

Cyber Equations, Inc. is reporting on the results of its DOE SBIR Phase I project for the development of a Scalable HPC Insider Threat Monitoring System. This final report to the Department of Energy for Task DE-FOA-0001618 covers the period from 02/21/2017 to 02/20/2018 and summarizes the larger research accomplishments, which can be divided in theoretical and laboratory experiments. One of the many challenges for the U.S. Department of Energy (DOE) is the support of the high-performance computing (HPC) centers while providing quality information services, strategic vision, leadership, technical expertise, and capabilities necessary to enable the DOE to deliver effective, efficient, economical, and secure HPC services. In order to realize a secure DOE laboratory enterprise, DOE facilities operations must counter a wide range of cyber insider threats, from unsophisticated adversaries up to advanced, nation state sponsored actors. The DOE laboratories also presents an enormous insider attack surface of an ever-growing number of computers, networks, and devices all running a myriad of software. In addition, the DOE laboratories are connected to a variety of external organizations which multiplies the possibility of insider threat. During this SBIR project Cyber Equations developed a strategic combination of supervised and unsupervised machine learning techniques to control scale and classify whether entities, such as hosts, actors, or documents, are showing suspicious insider behavior. We conducted experiments using real cyber data to show that changes in observable behavior (e.g., changes in outbound traffic patterns) are indicative of attacks executing (e.g., insiders exfiltrating data, bots calling home, or an attack spreading). The features and methods we use, have been used in the anomaly detection research and development community for a significant amount of time and robust implementations are openly available. K-Means clustering, Decision Trees, and Support Vector Machine based classifiers are readily available through the R language and open source H20 machine learning framework. The Cyber Equations approach is novel in its use of these classifiers combined with Hadoop based parallel processing technology. The five stages of processing used by CyberInsider improve scalability and reduce the high number of false positives normally associated with network anomaly detection. In summary, the Cyber Equations has successfully accomplished all of its Phase I tasks and demonstrated the feasibility of the Insider Threat Monitoring System.

The lightweight nature, application portability, and deployment flexibility of containers is driving their widespread adoption in cloud solutions. Data analysis and deep learning (DL)/machine learning (ML) applications have especially benefited from containerization. As such data analysis is adopted in high performance computing (HPC), the need for container support in HPC has become paramount. However, containers face crucial performance and I/O challenges in HPC. One obstacle is that while there have been HPC containers, such solutions have not been thoroughly investigated, especially from the aspect of their impact on the crucial HPC I/O throughput. To this end, this paper provides a first-of-its-kind empirical analysis of state-of-the-art representative container solutions (Docker, Podman, Singularity, and Charliecloud) in HPC environments. We also explore how containers interact with an HPC parallel file system like Lustre. We present the design of an analysis framework that is deployed on all nodes in an HPC environment, and captures CPU, memory, network, and file I/O statistics from the nodes and the storage system. We are able to garner key insights from our analysis, e.g., Charliecloud outperforms other container solutions in terms of container start-up time, while Singularity and Charliecloud are equivalent in I/O throughput. But this comes at a cost, as Charliecloud invokes the most metadata and I/O operations on the underlying Lustre file system. By identifying such trade-offs and optimization opportunities, we can enhance HPC containers performance and the ML/DL applications that increasingly rely on them.

The rapid adoption of machine learning (ML) has increased the computational and storage demands on high-performance computing (HPC) and cloud datacenters. Unfortunately, deploying ML models on HPC and cloud platforms is challenging because it is often difficult to manage, financially costly, and poses substantial environmental concerns. This dissertation attempts to answer a simple but ambitious question: can we deploy ML models to meet the Quality of Service (QoS) requirements while remaining cost-effective and environmentally sustainable? The dissertation demonstrates that achieving this balance is indeed possible, albeit through employing non-intuitive strategies and user engagements. We show that a combination of fast and slow hardware can outperform the common strategy of exclusively employing the latest and fastest hardware when co-optimizing for both performance and cost. Realizing this potential, however, requires a strategy to create an effective hardware mix, a process rigorously developed within this dissertation. This dissertation further reveals that the carbon footprint of ML models can be managed by opportunistically trading off answer quality transiently and re-configuring/partitioning the GPU hardware resources, depending on the carbon intensity of available energy sources in the datacenter's region. Finally, this dissertation discovers that ML deployments can be made more environmentally sustainable by strategically engaging users and tuning response brevity, especially in the context of large language models (LLMs). The scientific advancements made in this dissertation aim to relieve the community from the cost constraints, hardware inefficiencies, and potential negative environmental impacts of large-scale deployment of ML models on HPC and cloud platforms. We are hopeful that its open-source contributions will promote further innovation and broader engagement toward sustainable ML deployment.--Author's abstract

As deputy assistant secretary for research, development, and simulation in the US Department of Energy Office of Defense Programs, Gil Weigand provides programmatic and policy direction for the DoE's nuclear weapons research, development, and simulation work. In partnership with the national labs, he leads the effort to preserve the technology base and develop programs for the surety, reliability, military effectiveness, and credibility of the nation's nuclear-weapons stockpile.Prior to this job, Weigand was deputy assistant secretary for strategic computing and simulation and a senior information officer for the DoE Office of Defense Programs, where he led advanced initiatives in high-performance computer simulation, computation, and communications. In particular, he was responsible for directing the Accelerated Strategic Computing Initiative, which has created leading-edge computational modeling and simulation tools necessary to move from test-based to simulation-based assessment and certification. The ASCI initiative is based on the Comprehensive Test Ban Treaty, an international treaty to move toward no nuclear testing announced by President Bill Clinton on 11 August 1995.Paul Messina now manages ASCI as DoE deputy assistant secretary for advanced simulation and computing for defense programs. He is on a two-year leave from the California Institute of Technology, where he is assistant vice president for scientific computing, a faculty associate in scientific computing, and director of the Center for Advanced Computing Research. He has a joint appointment at the Jet Propulsion Laboratory as manager of high-performance computing. Messina is also on leave from the position of chief architect for the National Partnership for Advanced Computational Infrastructure, where he established and led the Scalable I/O Initiative.Editor-in-chief George Cybenko recently talked to Weigand and Messina about their roles in ASCI and the Alliance Program, and their thoughts about where the high end of computational performance is headed.

Future high-performance computing (HPC) platforms increasingly depend on heterogeneous node architectures to meet power and performance requirements. While modern HPC design largely incorporates GPUs with CPU resources, there is potential to further integrate novel forms of computing. The ability to leverage efficient, non-conventional computing technologies would be a fundamentally disruptive development in advancing HPC. Neuromorphic computing is such an emerging technology, which would interest the HPC community, due to its potential for implementing large-scale calculations with an extremely low power footprint. We will explore the example of mapping the connectome of the brain to illustrate advantages of using a heterogeneous system that incorporates neuromorphic hardware.

This works deals with the power-aware job dispatching problem in supercomputers; broadly speaking the dispatching consists of assigning finite capacity resources to a set of activities, with a special concern toward power and energy efficient solutions. We introduce novel optimization approaches to address its multiple aspects.
\nThe proposed techniques have a broad application range but are aimed at applications in the field of High Performance Computing (HPC) systems.
\n
\nDevising a power-aware HPC job dispatcher is a complex, where contrasting goals must be satisfied. Furthermore, the online nature of the problem request that solutions must be computed in real time respecting stringent limits. This aspect historically discouraged the usage of exact methods and favouring instead the adoption of heuristic techniques. The application of optimization approaches to the dispatching task is still an unexplored area of research and can drastically improve the performance of HPC systems.
\n
\nIn this work we tackle the job dispatching problem on a real HPC machine, the Eurora supercomputer hosted at the Cineca research center, Bologna. We propose a Constraint Programming (CP) model that outperforms the dispatching software currently in use. An essential element to take power-aware decisions during the job dispatching phase is the possibility to estimate jobs power consumptions before their execution. To this end, we applied Machine Learning techniques to create a prediction model that was trained and tested on the Euora supercomputer, showing a great prediction accuracy. Then we finally develop a power-aware solution, considering the same target machine, and we devise different approaches to solve the dispatching problem while curtailing the power consumption of the whole system under a given threshold. We proposed a heuristic technique and a CP/heuristic hybrid method, both able to solve practical size instances and outperform the current state-of-the-art techniques.

Quantum communication networks—the substrate of a future quantum Internet—demand analytical tools that account for entanglement, fidelity, and quantum-specific constraints absent from classical models. In this paper, we introduce the Quantum Network Science (QNS) framework that adapts core network metrics to the quantum setting through fidelity-, success-probability-, and capacity-aware weighting. We formalize centrality (including Quantum PageRank and continuous-time quantum-walk variants), community structure on entanglement graphs, and robustness/percolation with fidelity thresholds. The framework is validated via analytic motifs and controlled simulations on Erdős–Rényi, scale-free, and small-world topologies, as well as satellite-assisted versus fiber-only designs. Our results show that (i) fidelity weighting reorders structural importance and can reconnect networks that appear fragmented classically; (ii) heavy-tailed degree patterns improve tolerance to random failures but heighten vulnerability to targeted hub attacks; (iii) small-world shortcuts induced by long-range quantum links shrink path lengths; and (iv) overlapping “connected components” emerge from entanglement swapping, motivating revised connectivity baselines. We also discuss design implications—degree caps and hub hardening, link-type diversity, multipath routing, and buffering policies—and outline extensions to temporal and multilayer modeling that couple the quantum plane with its classical control layer. QNS thus offers a principled, measurement-oriented foundation for analyzing, comparing, and engineering resilient, high-capacity quantum networks.

Three-dimensional (3D) photonic integration is becoming a key enabler for future optical technologies, driven by the need for greater functionality, higher compactness, and superior interconnect density. In response to these demands, ultrafast laser processing has emerged as a powerful technique for realizing 3D photonic chips with enhanced design and fabrication flexibility—while avoiding the complexity of mask-based or cleanroom-dependent fabrication. This review begins by outlining the fundamental mechanisms of femtosecond laser–matter interactions, which enable precise energy deposition and spatially confined material modification. It then delineates three primary processing routes—direct modification, additive processing, and subtractive processing—alongside essential beam-shaping strategies. The compatibility of various material systems and structural platforms, including glasses, polymers, crystals, bulk substrates, thin-film layers, and optical fibers, is further discussed. Subsequently, the review details how these 3D laser-fabricated photonic structures underpin a wide range of applications, such as optical field manipulation, optical communications, optical interconnects, optical computing and processing, optical data storage, quantum information, nonlinear optics, optical trapping, and imaging. Finally, it concludes with a perspective on the prevailing challenges and future trends toward scalable, multifunctional, and hybrid-integrated photonic systems enabled by ultrafast laser microfabrication.

High-speed card-to-card optical interconnects are highly demanded in high-performance computing and data centers. Compared with other solutions, free-space optical interconnects have the capability of providing both reconfigurability and flexibility. In this paper we propose and experimentally demonstrate a free-space based reconfigurable optical interconnect architecture and it is capable of connecting cards located both inside the same rack as well as in different racks. Results show that 3xiO Gb/s data transmission is achieved with a worst-case receiver sensitivity better than -9.38 dBm.

Data rates are continuing to increase for box-to-box, rack-to-rack, board-to-board, and chip-to-chip interconnects for terabit switches and routers, multiprocessor computers and high-end servers. The increase in individual line rates and bandwidth drives the need to replace copper interconnects with optical interconnects. Fiber optics are advantageous for these applications because they allow for longer link lengths, increased bandwidth, smaller cables and connectors, less susceptibility to electromagnetic interference, and potentially lower power dissipation, depending on the data rate and link length. Towards this direction, this thesis aims to design and demonstrate low-cost, low-latency, high throughput, rack-to-rack optical interconnect architectures for exascale (i.e., performing 10^18 floating point operations per second) high-performance computing (HPC) systems and data centers. In particular, a novel, cost-effective, optical interconnect architecture for ultrafast optical switching, based on semiconductor optical amplifiers (SOAs), is studied. The proposed design of a fast N×N all-optical, wavelength-space crossbar switch for optical interconnects uses a minimum number of ON/OFF gates. This thesis compares and proves the superiority of the proposed architecture with respect to its originally-proposed counterpart, both theoretically and experimentally. Additionally, in order to increase the capacity and to minimize the impact of transmission effects (especially self-gain modulation (SGM), cross-gain modulation (XGM), self-phase modulation (SPM), cross-phase modulation (XPM), and polarization dependent gain (PDG)), we investigate the performance of conventional binary intensity modulation (IM), in conjunction with direct detection, as well as of advanced, more resilient, spectrally-efficient modulation formats (e.g., Differential Phase Shift Keying (DPSK), Orthogonal Frequency Division Multiplexing (OFDM), Polarization Division Multiplexed Quadrature Phase Shift Keying (PDM-QPSK), Single (SP)- and PDM- 16-ary Quadrature Amplitude Modulation (16QAM) in conjunction with coherent detection). Finally, as a seperate research activity, we study the performance of point-to-point links based on vertical-cavity surface-emitting lasers (VCSELs) and single- or multi- mode fibers, in conjuction with IM/DD, four-level Pulse Amplitude Modulation (4-PAM), and OFDM, to enable state-of-the-art, high-capacity, low-cost optical interconnects.