Commutative Data Reordering: A New Technique to Reduce Data Movement Energy on Sparse Inference Workloads

Capacitated Vehicle Routing Problem (CVRP) is a real life constrain satisfaction problem in which customers are optimally assign to individual vehicles (considering their capacity) to keep total travel distance of the vehicles as minimum as possible while serving customers. Various methods are investigated to solve CVRP in last few decades, the most popular way of solving CVRP is splitting the task into two different phases: firstly, assigning customers under different vehicles and secondly, finding optimal route of each vehicle. Sweep clustering algorithm is well studied for clustering nodes. On the other hand, route optimization is simply a traveling salesman problem (TSP) and a number of TSP optimization methods are investigated for this purpose. This study investigates a variant of Sweep algorithm for clustering nodes and different SI based methods for route generation to get optimal CVRP solution. In conventional Sweep algorithm, cluster formation starts from 0° and consequently advance toward 360° to consider all the nodes. In this study, Sweep cluster are considered from different starting angle. On the other hand, two well-known Swarm Intelligence (SI) methods (i.e., Ant Colony Optimization and Particle Swarm Optimization (PSO)) and two recent SI based algorithms (i.e., Producer-Scrounger Method and Velocity Tentative PSO) are considered for route optimization. We have compared the performance of these methods to solve CVRP. The experimental results on a large number of benchmark CVRP reveal that different starting angle has positive effect on Sweep clustering and finally, VTPSO is able to produce better solution than other methods.

Fuzzy C-Means (FCM) clustering technique is among the most effective partitional clustering algorithms available in the literature. The Capacitated Vehicle Routing Problem (CVRP) is an important industrial logistics and managerial NP-hard problem. Cluster-First Route-Second Method (CFRS) is one of the efficient techniques used to solve CVRP. In CFRS technique, customers are first divided into clusters in the first phase, then each cluster is solved independently as a Traveling Salesman Problem (TSP) in the second phase. This research is concerned with the clustering phase of CFRS, and TSP is then solved using a traditional optimization method. Three supervised FCM based techniques are proposed to solve the clustering phase at reduced cost via centroids (pre-FCM) initialization phase. The proposed pre-FCM initialization techniques are developed to be problem dependent. Hence, three initialization techniques are first developed using K-means technique, spatially equally distributed, and demand weighted center of mass. Then, a modified demand weighted fuzzy c-means objective function is employed to assign customers to clusters. To compare the performance of the proposed supervised FCM techniques, forty-two CVRP benchmark problems are solved using the traditional fuzzy C-means algorithm and the developed algorithms. Extensive comparisons are conducted between the traditional fuzzy C-means algorithm, the three proposed initialization techniques, and other fuzzy C-means techniques available in the literature. Results show that the proposed three initialization techniques are efficient in terms of solution quality and computational cost.

Combinatorial optimization problems are broadly studied in the literature. On the one hand, their challenging characteristics, such as the constraints and number of potential solutions, inspires their use to test new solution techniques. On the other hand, the practical application of these problems provides support of daily tasks of people and companies. Vehicle routing problems constitute a well-known class of combinatorial optimization problems, from which the Traveling Salesman Problem (TSP) is one of the most elementary ones. TSP corresponds to finding the shortest route that visits all cities within a path returning to the start city. Despite its simplicity, the difficulty in finding its exact solution and its direct application in practical problems in multiple areas make it one of the most studied problems in the literature. Algorithms inspired by biological phenomena are being successfully applied to solve optimization tasks, mainly combinatorial optimization problems. Those inspired by the collective behavior of insects produce good results for solving such problems. This article proposes the VRoptBees, a framework inspired by honeybee behavior to tackle vehicle routing problems. The framework provides a flexible and modular tool to easily build solutions to vehicle routing problems. Together with the framework, two examples of implementation are described, one to solve the TSP and the other to solve the Capacitated Vehicle Routing Problem (CVRP). Tests were conducted with benchmark instances from the literature, showing competitive results.

Fuzzy C-Means clustering is among the most successful clustering techniques available in the literature. The capacitated vehicle routing problem (CVRP) is one of the most studied NP-hard problems. CVRP has attracted the attention of many researchers due to its importance within the supply chain management field. This study aims to develop a fuzzy c-means clustering heuristic to efficiently solve the CVRP with large numbers of customers by using cluster-first route-second method (CFRS). CFRS is a two-phase technique, where in the first phase customers are grouped into, and in the second phase each cluster is solved independently as a traveling salesman problem (TSP). This work is concerned the clustering phase of the CFRS. The second phase of the CFRS method is solved using traditional optimization software. A modified demand weighted fuzzy c-means clustering algorithm is developed to solve the clustering phase. Twentyfive instances are solved to evaluate the efficiency of the proposed algorithm. Some of them are large instances with more than 500 customers. Promising results in terms of accuracy and processing time are obtained.

In this paper, we address a biomass feedstock logistics problem to supply biomass from production fields to satellite storage locations (SSLs) and from there to bioenergy plants (BePs) and then to a biorefinery. It entails a new problem feature of routing load-out equipment sets among the SSLs to perform loading/unloading of biomass and/or its pre-processing operations. The ownership of the loading equipment is a very capital-intensive link of the ethanol production supply chain, which when loaded onto trucks and routed along the logistics chain significantly brings down the ethanol production costs. This will make ethanol a cost-competitive alternative to fossil fuels, lead to sustainable use of fossil fuels and add to the overall relevance of the bioenergy sector. In this regard, the objective of our problem is to minimize the total cost incurred due to the ownership of equipment sets, fixed setups, and land rental cost, as well as the cost of transporting biomass from the fields to the BePs and biocrude oil from the BePs to the refinery. A mixed-integer mathematical model of the problem is presented, and a nested Benders decomposition-based solution approach is developed which involves decomposing this large problem into three stages. Stage 1 deals with the selection of fields, BePs, and SSLs, and assignment of fields to the SSLs. The remaining model consists of multiple Capacitated Vehicle Routing Problems (CVRPs) that are separable over individual BePs. For each BeP, the CVRP is further decomposed into Stage 2 and Stage 3 sub-problems where the Stage 2 problem is an allocation problem that assigns SSLs to tours associated to each BeP, and the Stage 3 problem is a variant of Traveling Salesman Problem (TSP) that determines the sequence in which equipment is routed over the predesignated set of SSLs for each tour. These sub-problems are integer programs rather than linear programs. First novelty of our proposed approach is to effectively handle the integrality of variables arising due to the consideration of the routing of load-out equipment. Second is solution methodology and in the use of proposed multi-cut version of optimality cuts that capture the solution value at an integer solution for the sub-problems. These cuts aid in faster convergence and are shown to be stronger than those proposed in the literature. The applicability of the proposed methodology is demonstrated by applying it to a real-life problem that utilizes available GIS data for the catchment area of regions around Gretna and Bedford in Virginia. We then solved a set of varying problem size instances using the state-of-the-art CPLEX® Branch-and-Bound and Benders Strategy methods. The CPLEX® algorithms struggled to solve instances even 10 times smaller than the real-life problem size instances; with MIP optimality gaps ranging from 5.85% to 82.79% in the allowed time limit of 10,000 s. On the other hand, our proposed nested Benders decomposition algorithm was able to achieve faster convergence and provided optimal solutions for all the considered problem instances with an average CPU run-time of around 3,700 s. This validates the efficacy and superiority of our solution approach. Lastly, we summarize our work and point out some interesting potential future research opportunities.

Vehicle routing problems (VRPs) are a fundamental class of combinatorial optimization problems (COPs) in computer science and operations research, with diverse real-world applications in logistics, transportation, and manufacturing. The intrinsic NP-hard nature makes VRPs exponentially expensive to be solved by exact solvers. As an alternative, heuristic solvers deliver suboptimal solutions within reasonable time but require extensive hand-crafted rules and domain expertise tailored to each specific problem. Recently, neural combinatorial optimization (NCO), which leverages machine learning to learn heuristics in a data-driven manner, has gained significant attention. These neural solvers demonstrate strong performance while reducing computational overhead and reliance on domain expertise compared to traditional solvers. However, they face significant generalization challenges. For example, their performance may degrade substantially when applied to instances with different data distributions, problem scales, or constraints than those encountered during training. This generalization issue severely limits their practical applicability. This thesis aims to systematically address these challenges and advance the development of generalizable neural solvers for VRPs. The thesis begins by enhancing the cross-distribution generalization of neural VRP solvers through an adversarial training approach. We introduce an ensemble-based Collaborative Neural Framework (CNF) that adversarially trains multiple models in a collaborative manner, promoting robustness against adversarial attacks while also boosting performance on clean instances. In this framework, an attacker generates hard instances by perturbing the data distribution, challenging the models to adapt and thereby strengthening cross-distribution generalization. Additionally, a neural router is designed to efficiently distribute training instances among the models, thereby improving load balancing and collaborative efficacy. Extensive experiments on two classic routing problems -- the travelling salesman problem (TSP) and the capacitated vehicle routing problem (CVRP) -- demonstrate the effectiveness and versatility of CNF in enhancing the cross-distribution generalization. Next, the thesis further expands neural solvers to simultaneously address both cross-size and cross-distribution generalization challenges. We propose Omni-VRP, a generic meta-learning framework that enables effective training of an initial model with the capacity for rapid adaptation to new tasks during inference, where each task corresponds to a set of instances with the same problem scale and data distribution. Additionally, we introduce a simple yet efficient approximation method to reduce the training overhead associated with second-order derivative calculations. Extensive experiments on both synthetic and benchmark instances of TSP and CVRP validate the effectiveness of our approach. Finally, the thesis pioneers the study of cross-problem generalization, pushing the boundaries of neural VRP solvers in learning generalizable representations across diverse constraints. We propose MVMoE, a unified neural solver capable of solving 16 VRP variants simultaneously in a zero-shot manner through attribute composition. To efficiently enhance model capacity, we employ a mixture-of-experts architecture with a hierarchical gating mechanism, striking a balance between empirical performance and computational complexity. Experimentally, our method significantly promotes zero-shot generalization on 10 unseen VRP variants, and showcases decent results on the few-shot setting and real-world benchmark instances. Overall, this thesis represents a pioneering exploration and substantial advancement in developing generalizable neural solvers for VRPs. It investigates innovative designs across various components, including model architectures, training algorithms, and problem settings, to enhance the generalization of neural solvers across diverse contexts such as varying data distributions, problem scales, and constraints. These contributions encourage neural solvers to learn more robust and generalizable representations, paving the way for the first generation of foundation models capable of addressing a wide range of COPs. Ultimately, the insights gained here not only propel the field of NCO forward but also enrich the broader landscape of learning-based optimization methods.

This is the second in a sequence of three Hardware Evaluation milestones that provide insight into the following questions: What are the sources of excess data movement across all levels of the memory hierarchy, going out to the network fabric? What can be done at various levels of the hardware/software hierarchy to reduce excess data movement? How does reduced data movement track application performance? The results of this study can be used to suggest where the DOE supercomputing facilities, working with their hardware vendors, can optimize aspects of the system to reduce excess data movement. Quantitative analysis will also benefit systems software and applications to optimize caching and data layout strategies. Another potential avenue is to answer cost-benefit questions, such as those involving memory capacity versus latency and bandwidth. This milestone focuses on techniques to reduce data movement, quantitatively evaluates the efficacy of the techniques in accomplishing that goal, and measures how performance tracks data movement reduction. We study a small collection of benchmarks and proxy mini-apps that run on pre-exascale GPUs and on the Accelsim GPU simulator. Our approach has two thrusts: to measure advanced data movement reduction directives and techniques on the newest available GPUs, and to evaluate our benchmark set on simulated GPUs configured with architectural refinements to reduce data movement.

Reinforcement learning (RL) has seen increasing success at solving a variety of combinatorial optimization problems. These techniques have generally been applied to deterministic optimization problems with few side constraints, such as the traveling salesperson problem (TSP) or capacitated vehicle routing problem (CVRP). With this in mind, the recent IJCAI AI for TSP competition challenged participants to apply RL to a difficult routing problem involving optimization under uncertainty and time windows. We present the winning submission to the challenge, which uses the policy optimization with multiple optima (POMO) approach combined with efficient active search and Monte Carlo roll-outs. We present experimental results showing that our proposed approach outperforms the second place approach by 1.7%. Furthermore, our computational results suggest that solving more realistic routing problems may not be as difficult as previously thought.KeywordsLearning to optimizeStochastic optimizationDeep reinforcement learningOrienteering problem

For the autonomous driving application, data movement has increased rapidly between a CMOS Image sensor (CIS) and the processor due to increase in image resolution. Advanced packaging techniques like 2.5D/3D integration have been proposed to reduce the data movement energy between memory and processor. In this work, we explore the use of such techniques to integrate a CIS and a backend accelerator on a silicon interposer. The data movement energy from CIS to the accelerator is thus reduced by 100× compared to using the conventional MIPI links. We perform thermal simulations to study the impact of the thermal coupling of CIS and accelerator and ensure a peak temperature increase of less than 5 C. We also vary the distance between the CIS and the processor to study the trade-offs between energy savings and peak temperature. Next, we assume the 3D stacked CIS and accelerator to reduce the data movement further and obtain an energy efficiency of 45.81 TOPS/W. Now we observe a heat dissipation challenge with an increase in the peak temperature of more than 85 C. Hence, we scale down the operational frequency and study the trade-off between performance degradation and reduction in peak temperature, while maintaining the accurate multi-object tracking on the BDD100k dataset for autonomous driving.

This work presents a hybrid approach called GA-NN for solving the Capacitated Vehicle Routing Problem (CVRP) using Genetic Algorithms (GA) and Nearest Neighbor heuristic (NN). The first technique was applied to determine the groups of customers to be served by the vehicles while the second is responsible to build the route of each vehicle. In addition, the heuristics of Gillett & Miller (GM) and Downhill (DH) were used, respectively, to generate the initial population of GA and to refine the solutions provided by GA. In the results section, we firstly present experiments demonstrating the performance of the NN heuristic for solving the Shortest Path and Traveling Salesman problems. The results obtained in such experiments constitute the main motivation for proposing the GA-NN. The second experimental study shows that the proposed hybrid approach achieved good solutions for instances of CVRP widely known in the literature, with low computational cost. It also allowed us to evidence that the use of GM and DH helped the hybrid GA-NN to converge on promising points in the search space, with a small number of generations.

Combined route capacity and route length models for unit demand vehicle routing problems

Recent years have seen an exploration of data volumes from a myriad of IoT devices, such as various sensors and ubiquitous cameras. The deluge of IoT data creates enormous opportunities for us to explore the physical world, especially with the help of deep learning techniques. Traditionally, the Cloud is the option for deploying deep learning based applications. However, the challenges of Cloud-centric IoT systems are increasing due to significant data movement overhead, escalating energy needs, and privacy issues. Rather than constantly moving a tremendous amount of raw data to the Cloud, it would be beneficial to leverage the emerging powerful IoT devices to perform the inference task. Nevertheless, the statically trained model could not efficiently handle the dynamic data in the real in-situ environments, which leads to low accuracy. Moreover, the big raw IoT data challenges the traditional supervised training method in the Cloud. To tackle the above challenges, we propose In-situ AI, the first Autonomous and Incremental computing framework and architecture for deep learning based IoT applications. We equip deep learning based IoT system with autonomous IoT data diagnosis (minimize data movement), and incremental and unsupervised training method (tackle the big raw IoT data generated in ever-changing in-situ environments). To provide efficient architectural support for this new computing paradigm, we first characterize the two In-situ AI tasks (i.e. inference and diagnosis tasks) on two popular IoT devices (i.e. mobile GPU and FPGA) and explore the design space and tradeoffs. Based on the characterization results, we propose two working modes for the In-situ AI tasks, including Single-running and Co-running modes. Moreover, we craft analytical models for these two modes to guide the best configuration selection. We also develop a novel two-level weight shared In-situ AI architecture to efficiently deploy In-situ tasks to IoT node. Compared with traditional IoT systems, our In-situ AI can reduce data movement by 28-71%, which further yields 1.4X-3.3X speedup on model update and contributes to 30-70% energy saving.

When executing Fortran 90 style data-parallel array operations on distributed-memory multiprocessors, intraprocessor data movement due to shift operations can account for a significant fraction of the execution time. This paper describes a strategy for minimizing data movement caused by Fortran 90 CSHIFT operations and presents a compiler technique that exploits this strategy automatically. The compiler technique is global in scope and can reduce data movement even when a definition of an array and its uses are separated by control flow. This technique supersedes those whose scope is restricted to a single statement. We focus on the application of this strategy on distributed-memory architectures, although it is more broadly applicable.

Performance of computing systems, from handhelds to supercomputers, is increasingly constrained by the energy consumed. A significant and increasing fraction of the energy is consumed in the movement of data. In a compute node, caches have been very effective in reducing data movement by exploiting the available data locality in programs. Program regions with poor data locality, then effect most of the data movement, and consequently consume an ever larger fraction of energy. In this paper we explore the energy efficiency opportunity of minimizing the data movement in precisely such program regions, by first imagining the possibility of compute near memory, and then partitioning the program’s execution between a compute core and the compute near memory (CnM). Due to the emergence of 3D stacked memory, a CnM implementation appears more realistic. Our focus is on evaluating the partitioning opportunity in applications and to do a limit study of systems enabled with CnM capabilities to understand and guide their architectural embodiment. We describe an automated method of analyzing the data access pattern of optimized workload binaries, via a binary-instrumentation tool called SnapCnM, to identify the beneficial program regions (loops) for CnM execution.We also perform a limit study to evaluate the impact of such partitioning over a range of parameters affecting CnM design choices. Our results show that compute partitioning a small (

The record-breaking success of convolutional neural networks (CNNs) comes at the cost of a large amount of model parameters. The resulting prohibitive memory storage and data movement energy have been limiting the extensive deployment of deep learning on daily-life edge devices which usually have limited storage capability and are battery-powered. To this end, we explore the employment of a recently published weight clustering technique, called deep k-Means which makes use of the redundancy within CNN parameters for reduced memory storage and data movement, and demonstrate k-Means's effectiveness in the context of an interactive real-time object detection using three representative daily-life devices (iPhone, iPad and FPGA).