Federated Distributionally Robust Optimization With Non-Convex Objectives: Algorithm and Analysis

Randomized decision making refers to the process of making decisions randomly according to the outcome of an independent randomization device, such as a dice roll or a coin flip. The concept is unconventional, and somehow counterintuitive, in the domain of mathematical programming, in which deterministic decisions are usually sought even when the problem parameters are uncertain. However, it has recently been shown that using a randomized, rather than a deterministic, strategy in nonconvex distributionally robust optimization (DRO) problems can lead to improvements in their objective values. It is still unknown, though, what is the magnitude of improvement that can be attained through randomization or how to numerically find the optimal randomized strategy. In this paper, we study the value of randomization in mixed-integer DRO problems and show that it is bounded by the improvement achievable through its continuous relaxation. Furthermore, we identify conditions under which the bound is tight. We then develop algorithmic procedures, based on column generation, for solving both single- and two-stage linear DRO problems with randomization that can be used with both moment-based and Wasserstein ambiguity sets. Finally, we apply the proposed algorithm to solve three classical discrete DRO problems: the assignment problem, the uncapacitated facility location problem, and the capacitated facility location problem and report numerical results that show the quality of our bounds, the computational efficiency of the proposed solution method, and the magnitude of performance improvement achieved by randomized decisions. Summary of Contribution: In this paper, we present both theoretical results and algorithmic tools to identify optimal randomized strategies for discrete distributionally robust optimization (DRO) problems and evaluate the performance improvements that can be achieved when using them rather than classical deterministic strategies. On the theory side, we provide improvement bounds based on continuous relaxation and identify the conditions under which these bound are tight. On the algorithmic side, we propose a finitely convergent, two-layer, column-generation algorithm that iterates between identifying feasible solutions and finding extreme realizations of the uncertain parameter. The proposed algorithm was implemented to solve distributionally robust stochastic versions of three classical optimization problems and extensive numerical results are reported. The paper extends a previous, purely theoretical work of the first author on the idea of randomized strategies in nonconvex DRO problems by providing useful bounds and algorithms to solve this kind of problems.

We consider the problem of designing piecewise affine policies for two-stage adjustable robust linear optimization problems under right-hand side uncertainty. It is well known that a piecewise affine policy is optimal although the number of pieces can be exponentially large. A significant challenge in designing a practical piecewise affine policy is constructing good pieces of the uncertainty set. Here we address this challenge by introducing a new framework in which the uncertainty set is “approximated” by a “dominating” simplex. The corresponding policy is then based on a mapping from the uncertainty set to the simplex. Although our piecewise affine policy has exponentially many pieces, it can be computed efficiently by solving a compact linear program given the dominating simplex. Furthermore, we can find the dominating simplex in a closed form if the uncertainty set satisfies some symmetries and can be computed using a MIP in general. We would like to remark that our policy is an approximate piecewise-affine policy and is not necessarily a generalization of the class of affine policies. Nevertheless, the performance of our policy is significantly better than the affine policy for many important uncertainty sets, such as ellipsoids and norm-balls, both theoretically and numerically. For instance, for hypersphere uncertainty set, our piecewise affine policy can be computed by an LP and gives a $$O(m^{1/4})$$ -approximation whereas the affine policy requires us to solve a second order cone program and has a worst-case performance bound of $$O(\sqrt{m})$$ .

Decomposition methods have been well studied for solving two-stage and multi-stage stochastic programming problems, see Rockafellar and Wets (Math. Oper. Res. 16:119–147, 1991), Ruszczynski and Shapiro (Stochastic Programming, Handbook in OR & MS, North-Holland Publishing Company, Amsterdam, 2003) and Ruszczynski (Math. Program. 79:333–353, 1997). In this paper, we propose an algorithmic framework based on the fundamental ideas of the methods for solving two-stage minimax distributionally robust optimization (DRO) problems where the underlying random variables take a finite number of distinct values. This is achieved by introducing nonanticipativity constraints for the first stage decision variables, rearranging the minimax problem through Lagrange decomposition and applying the well-known primal-dual hybrid gradient (PDHG) method to the new minimax problem. The algorithmic framework does not depend on specific structure of the ambiguity set. To extend the algorithm to the case that the underlying random variables are continuously distributed, we propose a discretization scheme and quantify the error arising from the discretization in terms of the optimal value and the optimal solutions when the ambiguity set is constructed through generalized prior moment conditions, the Kantorovich ball and $$\phi$$ -divergence centred at an empirical probability distribution. Some preliminary numerical tests show the proposed decomposition algorithm featured with parallel computing performs well.

Most moment-based distributionally robust optimization (DRO) problems can be reformulated as semidefinite programming (SDP) problems, which can be solved in polynomial time. However, solving high-dimensional SDPs is often time-consuming. Existing approximation methods typically reduce the dimensionality of random parameters before solving the approximated SDPs. This sequential approach relies solely on statistical information to reduce the high-dimensional uncertainty space, which may not yield the best approximation performance. Jiang et al. (2025) introduce an optimized dimensionality reduction (ODR) approach that integrates the dimensionality reduction of random parameters with subsequent optimization problems. This integration enables two outer approximations and one inner approximation of the original problem, all represented as low-dimensional SDPs that can be solved efficiently, providing two lower bounds and one upper bound, respectively. More importantly, these approximations can theoretically achieve the optimal value of the original high-dimensional SDPs, resulting in a zero gap.

In this article, a novel distributionally robust optimization (DRO) based multistage model predictive control (MPC) framework is proposed to hedge against the uncertainty in control problems. Without a priori knowledge of the exact uncertainty distribution, an ambiguity set, constructed based on principal component analysis, incorporates the first-order moment information instead. A linear performance measure is chosen so that the worst-case expected problem can be exactly dualized by adopting the affine decision rule. By considering input and state constraints robustly with respect to a support set, which specifies the domain of the uncertainty, the DRO-based MPC model is developed as a robust optimization problem. The proposed framework is illustrated on a two-mass-spring system and results show the improved control performance compared to traditional control strategies.

In modern taxi networks, large amounts of taxi occupancy status and location data are collected from networked in-vehicle sensors in real- time. They provide knowledge of system models on passenger demand and mobility patterns for efficient taxi dispatch and coordination strategies. Such approaches face a new challenge: how to deal with uncertainties of predicted customer demand while fulfilling the system's performance requirements, including minimizing taxis' total idle mileage and maintaining service fairness across the whole city. To address this problem, we develop a data- driven robust taxi dispatch framework to consider spatial-temporally correlated demand uncertainties. The robust optimization problem has an objective function that is concave in the uncertain demand and convex in the decision variables, and constraints that are convex in the decision variables according to the practical requirements. We design a data-driven algorithm for constructing uncertainty sets of random demand vectors that provide a probabilistic guarantee for the performance of robust taxi dispatch solutions. It guarantees that the probability that the actual dispatch cost under the true demand vector being smaller than the optimal cost of the robust dispatch solutions with the uncertainty sets is greater than a desired probability value. We prove equivalent computationally tractable forms of the robust dispatch problem with the constructed closed convex polytope and second order cone types of uncertainty sets, using the minimax theorem and strong duality. Evaluations on four years of taxi trip data for New York City show that the probabilistic guarantee is satisfied. By selecting a probabilistic guarantee level that does not sacrifice the average cost much, the average demand- supply ratio error is reduced by 31.7%, and the average total idle driving distance is reduced by 10.13% compared with non-robust dispatch solutions.

The various robust linear programming models investigated so far in the literature essentially appear to be based either on what is referred to as rowwise uncertainty models or on columnwise uncertainty models (these basically assume that the rows, or the columns, of the constraint matrix are subject to changes within a well-specified uncertainty set). In this chapter, we discuss a special case of columnwise uncertainty, namely the subclass of robust linear programming (LP) models with uncertainty limited to the right hand side (RHS) only (this subclass does not appear to have been significantly investigated so far). In this context we introduce the concept of a ‘two-stage robust LP model’ as opposed to the standard case (which might be referred to as a ‘single-stage robust LP model’) and we address the question of whether LP duality can be used to convert a LP problem with RHS uncertainty into a robust LP problem with uncertainty on the objective function. We show how to derive both statements of (a) the dual to the robust model and (b) the robust version of the dual. The resulting expressions of the objective function to be optimized in both cases, appear to be clearly distinct. Moreover, from a complexity point of view, one appears to be efficiently solvable (it reduces to a convex optimization problem), whereas the other, as a nonconvex optimization problem, is expected to be computationally difficult in the general case. As an application of the two-stage robust LP model introduced here, we next investigate the PERT (program evaluation and review technique) scheduling problem, considering two possible natural ways of specifying the uncertainty set for the task durations: the case where the uncertainty set is a scaled ball with respect to the L ∞ norm; the case where the uncertainty set is a scaled Hamming ball of bounded radius (which, though leading to a quite different model, bears some resemblance to the well-known Bertsimas–Sim approach to robustness). We show that in both cases the resulting robust optimization problem can be efficiently solved in polynomial time.

Chapter 4 - Data-driven robust stochastic optimization for power systems operations

Surrogate duality for robust optimization

Quadratically adjustable robust linear optimization with inexact data via generalized S-lemma: Exact second-order cone program reformulations

Advanced AI-Generated Content (AIGC) technologies have injected new impetus into teleoperation, enhancing its security and efficiency. Edge AIGC networks have been introduced to meet the stringent low-latency requirements of teleoperation. However, the inherent uncertainty of AIGC service quality and the need to incentivize AIGC service providers (ASPs) make the design of a robust incentive mechanism essential. This design is particularly challenging due to uncertainty and information asymmetry, as teleoperators have limited knowledge of the remaining resource capacities of ASPs. To this end, we propose a distributionally robust optimization (DRO)-based contract theory to design robust reward schemes for AIGC task offloading. Notably, our work extends the contract theory by integrating DRO, addressing the fundamental challenge of contract design under uncertainty. In this paper, we employ contract theory to model information asymmetry while utilizing DRO to capture the uncertainty in AIGC service quality. Given the inherent complexity of the original DRO-based contract theory problem, we reformulate it into an equivalent, tractable bi-level optimization problem. To efficiently solve this problem, we develop a Block Coordinate Descent (BCD)-based algorithm to derive robust reward schemes. Simulation results on our unitybased teleoperation platform demonstrate that the proposed method improves teleoperator utility by 2.7% to 10.74% under varying degrees of AIGC service quality shifts and increases ASP utility by 60.02% compared to the SOTA method, i.e., Deep Reinforcement Learning (DRL)-based contract theory. The code and data are publicly available at https://github.com/Zijun0819/DROContract-Theory

A Little Pessimism Goes a Long Way Data-driven optimization is concerned with finding a decision, using data and perhaps a model, that performs well when it is applied on a new unseen data point. Data-driven optimization is challenging because data are limited or the model is wrong or the environment in which the decision is being applied is different from the one in which the training data were collected. Distributionally robust optimization (DRO), a worst case optimization method for finding decisions that are insensitive to model error, can sometimes but not always deliver a decision that has a larger out-of-sample expected reward than the sample average approximation (SAA). “A Data Driven Approach to Beating SAA out of Sample” by Jun-ya Gotoh, Michael Kim, and Andrew Lim shows that if worst case (DRO) solutions fail at this task, then the solution of a best case distributionally optimistic optimization problem will do the job. As good as this sounds, there is a catch: whereas an optimistic decision might beat SAA, the improvement is very modest and comes at the cost of being much more sensitive to model misspecification than both the SAA and the DRO decisions. Moreover, it is easy to make a mistake: it can be difficult to determine with a modestly sized data set whether the best or worst case solution will have the higher expected reward than SAA. In summary, data driven optimization is a trade-off between maximizing the expected reward and controlling the sensitivity of this expectation to model misspecification. When both are considered, a little bit of pessimism goes a long way.

Clean energy resources, like wind, have a stochastic nature, which involves uncertainties in the power system. Introducing energy storage systems (ESS) to the network can compensate for the uncertainty in wind plant output and allow the plant to participate in ancillary service markets. Advance in compressed air energy storage system (CAES) technologies and their fast response make them suitable for ancillary services. This paper investigates the participation of a combined energy system composed of wind plants and compressed air energy storage system (CAES) in the energy market from a private owner’s viewpoint, including trading in energy markets and bidding for frequency regulation and reserve capacity in ancillary service markets. Since this problem contains various uncertainties associated with market prices, wind generation levels, and regulation signals, distributionally robust optimization (DRO) is used to model the uncertainties and enhance the simultaneous participation of a combined wind-CAES system in day-ahead energy and ancillary service markets. This method combines the advantages of stochastic and robust optimization. In contrast to robust optimization (RO), the method consolidates specific statistical data to reduce conservative results. Simulation results demonstrate the proposed model’s effectiveness in handling uncertainties and provide a framework for investors in this area. In addition, case study analyses are applied to assess the model’s performance and validate the coordination of a wind plant and compressed air energy storage system in participating in a deregulated electricity market. Finally, DRO and RO are compared in modeling the uncertainties of the optimization problem. The optimal outputs demonstrate the effectiveness of DRO in terms of achieving higher realized profits with less conservative results.

This paper proposes a novel methodology for robust analog/mixed-signal IC design by introducing a notion of budget of uncertainty. This method employs a new conic uncertainty model to capture process variability and describes variability-affected circuit design as a set-based robust optimization problem. For a pre-specified yield requirement, the proposed method conducts uncertainty budgeting by associating performance yield with the size of uncertainty set for process variations. Hence the uncertainty budgeting problem can be further translated into a tractable robust optimization problem. Compared with the existing robust design flow based on ellipsoid model, this method is able to produce more reliable design solutions by allowing varying size of conic uncertainty set at different design points. In addition, the proposed method addresses the limitation that the size of ellipsoid model is calculated solely relying on the distribution of process parameters, while neglecting the dependence of circuit performance upon these design parameters. The proposed robust design framework has been verified on various analog/mixed-signal circuits to demonstrate its efficiency against ellipsoid model. An up to 24% reduction of design cost has been achieved by using the uncertainty budgeting based method.

Distributionally robust co-optimization of energy and reserve dispatch for integrated electricity-gas-heating systems