Scalable Primal Heuristics Using Graph Neural Networks for Combinatorial Optimization

By examining the patterns of solutions obtained for various instances, one can gain insights into the structure and behavior of combinatorial optimization (CO) problems and develop efficient algorithms for solving them. Machine learning techniques, especially Graph Neural Networks (GNNs), have shown promise in parametrizing and automating this laborious design process. The inductive bias of GNNs allows for learning solutions to mixed-integer programming (MIP) formulations of constrained CO problems with a relational representation of decision variables and constraints. The trained GNNs can be leveraged with primal heuristics to construct high-quality feasible solutions to CO problems quickly. However, current GNN-based end-to-end learning approaches have limitations for scalable training and generalization on larger-scale instances; therefore, they have been mostly evaluated over small-scale instances. Addressing this issue, our study builds on supervised learning of optimal solutions to the downscaled instances of given large-scale CO problems. We introduce several improvements on a recent GNN model for CO to generalize on instances of a larger scale than those used in training. We also propose a two-stage primal heuristic strategy based on uncertainty-quantification to automatically configure how solution search relies on the predicted decision values. Our models can generalize on 16x upscaled instances of commonly benchmarked five CO problems. Unlike the regressive performance of existing GNN-based CO approaches as the scale of problems increases, the CO pipelines using our models offer an incremental performance improvement relative to CPLEX. The proposed uncertainty-based primal heuristics provide 6-75% better optimality gap values and 45-99% better primal gap values for the 16x upscaled instances and brings immense speedup to obtain high-quality solutions. All these gains are achieved through a computationally efficient modeling approach without sacrificing solution quality.

. The industrial and commercial worlds are increasingly competitive, requiring companies to be more productiveand more responsive to market changes (e.g. globalisation and privatisation). As a consequence, there is a strong need for solutions to large scale optimization problems, in domains such as production scheduling, transport, finance and network management. This means that more experts in constraint programming and optimization technology are required to develop adequate software. Given the computational complexity of Large Scale Combinatorial Optimization problems, a key question is how to help/guide in the tackling of LSCO problems in industry. Optimization technology is certainly reaching a level of maturity. Having emerged in the 50s within the Operational Research community, it has evolved and comprises new paradigms such as constraint programming and stochastic search techniques. There is a practical need, i.e. efficiency, scalability and tractability, to integrate techniques from the different paradigms. This adds complexity to the design of LSCO models and solutions. Various forms of guidance are available in the literature in terms of 1) case studies that map powerful algorithms to problem instances, and 2) visualization and programming tools that ease the modelling and solving of LSCOs. However, there is little guidance to address the process of building applications for new LSCO problems (independently of any language). This article gives an overview of the CHIC-2 methodology which aims at filling a gap in this direction. In particular, we describe some management issues specific to LSCOs such as risk management and team structures, and focus on the technical development guidance for scoping, designing and implementing LSCO appl...

Alzheimer's disease (AD) is a neurodegenerative disease whose molecular mechanisms are activated several years before cognitive symptoms appear. Genotype-based prediction of the phenotype is thus a key challenge for the early diagnosis of AD. Machine learning techniques that have been proposed to address this challenge do not consider known biological interactions between the genes used as input features, thus neglecting important information about the disease mechanisms at play. To mitigate this, we first extracted AD subnetworks from several protein–protein interaction (PPI) databases and labeled these with genotype information (number of missense variants) to make them patient-specific. Next, we trained Graph Neural Networks (GNNs) on the patient-specific networks for phenotype prediction. We tested different PPI databases and compared the performance of the GNN models to baseline models using classical machine learning techniques, as well as randomized networks and input datasets. The overall results showed that GNNs could not outperform a baseline predictor only using the APOE gene, suggesting that missense variants are not sufficient to explain disease risk beyond the APOE status. Nevertheless, our results show that GNNs outperformed other machine learning techniques and that protein–protein interactions lead to superior results compared to randomized networks. These findings highlight that gene interactions are a valuable source of information in predicting disease status.

The Maximum Weight Independent Set problem is a fundamental NP-hard problem in combinatorial optimization with several real-world applications. Given an undirected vertex-weighted graph, the problem is to find a subset of the vertices with the highest possible weight under the constraint that no two vertices in the set can share an edge. An important part of solving this problem in both theory and practice is data reduction rules, which several state-of-the-art algorithms rely on. However, the most complicated rules are often not used in practice since the time needed to check them exhaustively becomes infeasible. In this work, we introduce two main results. First, we introduce several new data reduction rules and evaluate their effectiveness on real-world data. Second, we use a machine learning screening algorithm to speed up the reduction phase, thereby enabling more complicated rules to be applied. Our screening algorithm consults a Graph Neural Network oracle to decide if the probability of successfully reducing the graph is sufficiently large. For this task, we provide a dataset of labeled vertices for use in supervised learning. We also present the first results for this dataset using established Graph Neural Network architectures. Our experimental evaluation shows that with our preprocessing technique, we can reduce instances to within one percent of what is possible using the full set of reduction rules exhaustively, while being four times faster for the reductions that undergo early screening.

The Unit Commitment Problem (UCP) is a critical component of power market decision-making and is typically formulated as Mixed Integer Programming (MIP). Given the complexity of solving MIPs, efficiently solving large-scale UCPs remains a significant challenge. This paper presents a hybrid Graph Neural Network (GNN)–Linear Programming (LP) framework to accelerate the solution of large-scale Unit Commitment Problems (UCPs) while maintaining the quality of solutions. By analyzing variable stability through historical branch-and-bound (B&B) trajectories, we classify MIP variables into dynamically adjustable stable and unstable groups. We adopt an MIP formulation that includes multiple types of binary variables—such as commitment, startup, and shutdown variables—and extract additional information from these auxiliary binary variables. This enriched representation provides more candidates for stable variable fixation, helping to improve variable refinement, mitigate suboptimality, and enhance computational efficiency. A bipartite GNN is trained offline to predict stable variables based on system topology and historical operational patterns. During online optimization, instance-specific root LP solutions refine these predictions, enabling adaptive variable fixation via a dual-threshold mechanism that integrates GNN confidence and LP relaxations. To mitigate suboptimality risks, we introduce temporally flexible fixation strategies—hard fixation for variables with persistent stability and soft fixation allowing limited temporal adjustments—alongside a GNN-guided branching rule to prioritize unstable variables. Numerical experiments demonstrate that jointly fixing commitment, startup, and shutdown variables yields better performance compared to fixing only commitment variables. Ablation studies further validate the importance of hard fixation and customized branching strategies, especially for large-scale systems.

This survey gives a selective review of recent development of machine learning (ML) for combinatorial optimization (CO), especially for graph matching. The synergy of these two well-developed areas (ML and CO) can potentially give transformative change to artificial intelligence, whose foundation relates to these two building blocks. For its representativeness and wide-applicability, this paper is more focused on the problem of weighted graph matching, especially from the learning perspective. For graph matching, we show that many learning techniques e.g. convolutional neural networks, graph neural networks, reinforcement learning can be effectively incorporated in the paradigm for extracting the node features, graph structure features, and even the matching engine. We further present outlook for the new settings for learning graph matching, and direction towards more integrated combinatorial optimization solvers with prediction models, and also the mutual embrace of traditional solver and machine learning components.

At present, the urban traffic system is faced with the problems of congestion and low efficiency, and the traditional methods have certain limitations when dealing with the complex urban road network. This paper aims to explore a new method of capacity Optimization of Road Networks in a Smart City environment and proposes a method based on a Graph Neural network, named “Smart City Road optimization Graph Neural Network” (SCRO-GNN). SCRO-GNN first collects and preprocesses multi-source data, including road network data, traffic flow, accident records, environmental factors, etc. The key node and edge features, including the number of lanes at the intersection, the traffic flow of the section, and the adjacency matrix of the road network are defined to characterize the structure of the road network. Then, the graph neural network model is constructed and trained to predict the traffic flow of different sections and evaluate the road capacity by using the graph structure of the road network. This paper tests the performance of SCRO-GNN on real road networks in multiple cities. The results show that, compared with the traditional traffic flow prediction model, SCRO-GNN can significantly improve the prediction accuracy, especially when dealing with the highly complex urban road network structure. Based on these predictions, the proposed optimization strategy performed well in reducing traffic congestion and improving the efficiency of road use. The research in this paper not only demonstrates the potential of graph neural networks in smart city road network optimization, but also provides a new direction for future traffic system research. The successful implementation of the SCRO-GNN approach is expected to provide more efficient and intelligent solutions for urban traffic management and planning.

Recent work has shown that graph neural networks (GNNs) are vulnerable to adversarial attacks on graph data. Common attack approaches are typically informed, i.e. they have access to information about node attributes such as labels and feature vectors. In this work, we study adversarial attacks that are uninformed, where an attacker only has access to the graph structure, but no information about node attributes. Here the attacker aims to exploit structural knowledge and assumptions, which GNN models make about graph data. In particular, literature has shown that structural node centrality and similarity have a strong influence on learning with GNNs. Therefore, we study the impact of centrality and similarity on adversarial attacks on GNNs. We demonstrate that attackers can exploit this information to decrease the performance of GNNs by focusing on injecting links between nodes of low similarity and, surprisingly, low centrality. We show that structure-based uninformed attacks can approach the performance of informed attacks, while being computationally more efficient. With our paper, we present a new attack strategy on GNNs that we refer to as Structack. Structack can successfully manipulate the performance of GNNs with very limited information while operating under tight computational constraints. Our work contributes towards building more robust machine learning approaches on graphs.

In recent years, heterogeneous network/graph representation learning/embedding (HNE) has drawn tremendous attentions from research communities in multiple disciplines. HNE has shown its outstanding performances in various networked data analysis and mining tasks. In fact, most of real-world information networks in multiple fields can be modelled as the heterogeneous information networks (HIN). Thus, the HNE-based techniques can sufficiently capture rich-structured and semantic latent features from the given information network in order to facilitate for different task-driven learning tasks. This is considered as fundamental success of HNE-based approach in comparing with previous traditional homogeneous network/graph based embedding techniques. However, there are recent studies have also demonstrated that the heterogeneous network/graph modelling and embedding through graph neural network (GNN) is not usually reliable. This challenge is original come from the fact that most of real-world heterogeneous networks are considered as incomplete and normally contain a large number of feature noises. Therefore, multiple attempts have proposed recently to overcome this limitation. Within this approach, the meta-path-based heterogeneous graph-structured latent features and GNN-based parameters are jointly learnt and optimized during the embedding process. However, this integrated GNN and heterogeneous graph structure (HGS) learning approach still suffered a challenge of effectively parameterizing and fusing different graph-structured latent features from both GNN- and HGS-based sides into better task-driven friendly and noise-reduced embedding spaces. Therefore, in this paper we proposed a novel attention-supplemented heterogeneous graph structure embedding approach, called as: AGSE. Our proposed AGSE model supports to not only achieve the combined rich heterogeneous structural and GNN-based aggregated node representations but also transform achieved node embeddings into noise-reduced and task-driven friendly embedding space. Extensive experiments in benchmark heterogeneous networked datasets for node classification task showed the effectiveness of our proposed AGSE model in comparing with state-of-the-art network embedding baselines.

Intensity-modulated radiotherapy (IMRT) is one of the most popular techniques for cancer treatment. However, existing IMRT planning methods can only generate one solution at a time and, consequently, medical physicists should perform the planning process many times to obtain diverse solutions to meet the requirement of a clinical case. Meanwhile, multiobjective evolutionary algorithms (MOEAs) have not been fully exploited in IMRT planning since they are ineffective in optimizing the large number of discrete variables of IMRT. To bridge the gap, this article formulates IMRT planning into a large-scale combinatorial many-objective optimization problem and proposes a coevolutionary algorithm to solve it. In contrast to the existing MOEAs handling high-dimensional search spaces via variable grouping or dimensionality reduction, the proposed algorithm evolves one population with fine encoding for local exploitation and evolves another population with rough encoding for global exploration. Moreover, the convergence speed is further accelerated by two customized local search strategies. The experimental results verify that the proposed algorithm outperforms state-of-the-art MOEAs and IMRT planning methods on a variety of clinical cases.

We propose SAT-GATv2, a graph neural network (GNN)-based model designed to solve the Boolean satisfiability problem (SAT) through graph-based deep learning techniques. SAT-GATv2 transforms SAT formulas into graph structures, leveraging message-passing neural networks (MPNNs) to propagate local information and dynamic attention mechanisms (GATv2) to accurately capture inter-node dependencies and enhance node feature representations. Unlike traditional heuristic-driven SAT solvers, SAT-GATv2 adopts a data-driven approach, learning structural patterns directly from graph representations and providing a complementary framework to existing methods. Experimental results demonstrate that SAT-GATv2 achieves an accuracy improvement of 1.75–5.51% over NeuroSAT on challenging random 3-SAT(n) instances, highlighting its effectiveness in handling difficult problem distributions, and outperforms other GNN-based models on SR(n) datasets, showcasing its scalability and adaptability. Ablation studies validate the critical roles of MPNNs and GATv2 in improving prediction accuracy and scalability. While SAT-GATv2 does not yet surpass CDCL-based solvers in overall performance, it addresses their limitations in scalability and adaptability to complex instances, offering an efficient graph-based alternative for tackling larger and more complex SAT problems. This study establishes a foundation for integrating deep learning with combinatorial optimization, emphasizing its potential for applications in artificial intelligence and operations research.

Combinatorial optimization has found its way into a variety of domains, including artificial intelligence and cybernetics. Deep Reinforcement Learning (DRL) has recently demonstrated its promise for developing heuristics for NP-hard routing problems. The current generalization performance of models needs to be improved, especially for large-scale routing problems. In this paper, we propose a hybrid approach for the Capacitated Vehicle Routing Problem (CVRP) based on DRL and adaptive large neighborhood search. The information representation of the neural network for CVRP is also improved by the combination of multi-head attention mechanism, pointer network and graph neural networks. The experimental results demonstrate that the optimization of our model on CVRP outperforms existing DRL techniques and some traditional algorithms. In addition, our method improves the training efficiency of the model and the performance of generalization to large-scale CVRP.

Driven by Moore’s law, the chip design complexity is steadily increasing. Electronic Design Automation (EDA) has been able to cope with the challenging very large-scale integration process, assuring scalability, reliability, and proper time-to-market. However, EDA approaches are time and resource-demanding, and they often do not guarantee optimal solutions. To alleviate these, Machine Learning (ML) has been incorporated into many stages of the design flow, such as in placement and routing. Many solutions employ Euclidean data and ML techniques without considering that many EDA objects are represented naturally as graphs. The trending Graph Neural Networks are an opportunity to solve EDA problems directly using graph structures for circuits, intermediate RTLs, and netlists. In this paper, we present a comprehensive review of the existing works linking the EDA flow for chip design and Graph Neural Networks.

Humanistic link oriented data (LOD) play a crucial role as they appear in technological and financial sectors including smart cities, social networks, and long supply chains. The primary difficulties in mining LOD are their volume, the rate of missing or corrupted values, the non-linear relationships, and its higher order nature. Therefore, deep learning approaches of comparable complexity should be used if meaningful knowledge is to be extracted from the LOD ocean. Tensor stack networks (TSNs) consist of layers with information flowing serially both between layers and within the same layer. TSNs natively handle multidimensional data such as images and graphs, preventing information loss from vectorization by preserving topological knowledge. Two TSNs have been implemented, one assessing the structural resilience of a graph based on connectivity patterns and one computing the correlation coefficient for graphs compressed with DCT2. In both cases TSNs achieved superior performance to baselines. Self organizing maps (SOMs) are unsupervised neural network architectures learning how to modify connections between two spaces, the data and the coordinate space. SOMs construct a low dimensional projection from the former to the latter while preserving up to an extent the primary topological properties. A tensor distance metric allows considerably more flexibility compared to vector-based ones as corroborated by clustering neuroimages from an open fMRI dataset. Graph neural networks (GNNs) have emerged as a versatile architectural paradigm. Through message passing and aggregation a global state eventually emerges from local ground truth stored at the vertices or at the edges. Research focuses on the conditions for the emergence of the steady state and the convergence rate. A GNN for discovering affective Twitter communities is proposed. Experiments on benchmark graphs suggest GNNs are more efficient than the baseline. Comparing the abovementioned architectures it follows that GNNs are the most flexible as they learn in an unsupervised and distributed way and they operate on irregular domains. However, they require a tensor distance metric to handle multidimensional data. Another conclusion is that GNNs offer higher explainability as they adjust to the underlying domain by naturally expressing its relationships no matter how complex they are without imposing any external structure on it.

A plethora of fair graph neural networks (GNNs) have been proposed to promote algorithmic fairness for high-stake real-life contexts. Meanwhile, explainability is generally proposed to help machine learning practitioners debug models by providing human-understandable explanations. However, seldom work on explainability is made to generate explanations for fairness diagnosis in GNNs. From the explainability perspective, this paper explores the problem of what subgraph patterns cause the biased behavior of GNNs, and what actions could practitioners take to rectify the bias? By answering the two questions, this paper aims to produce compact, diagnostic, and actionable explanations that are responsible for discriminatory behavior. Specifically, we formulate the problem of generating diagnostic and actionable explanations as a multi-objective combinatorial optimization problem. To solve the problem, a dedicated multi-objective evolutionary algorithm is presented to ensure GNNs' explainability and fairness in one go. In particular, an influenced nodes-based gradient approximation is developed to boost the computation efficiency of the evolutionary algorithm. We provide a theoretical analysis to illustrate the effectiveness of the proposed framework. Extensive experiments have been conducted to demonstrate the superiority of the proposed method in terms of classification performance, fairness, and interpretability.