Abstract In this paper, we propose algorithms that exploit negative curvature for solving noisy nonlinear nonconvex unconstrained optimization problems. We consider both deterministic inexact and stochastic settings, and develop two-step algorithms that combine directions of negative curvature and descent directions to update the iterates. Under reasonable assumptions, we prove second-order convergence results and derive complexity guarantees for both settings. To tackle large-scale problems, we develop a practical variant that utilizes the conjugate gradient method with negative curvature detection and early stopping to compute a step, a simple adaptive step size scheme, and a strategy for selecting the sample sizes of the gradient and Hessian approximations as the optimization progresses. Numerical results on two machine learning problems showcase the efficacy and efficiency of the practical method.

Abstract In this paper, we examine two-stage stochastic quadratic programming problems, where the objective functions of the first and second stages are quadratic, and the constraints are linear. The uncertainty is associated with the second-stage right-hand side and variable bounds. In large-scale settings, when the number of scenarios necessary to represent the underlying stochastic process is exuberantly large, standard decomposition-based methods that require the exact solutions are computationally prohibitive. To address this issue, we develop two inexact proximal bundle algorithms that rely on the efficient reuse of solution information. The first algorithm utilizes a collection of previously encountered second-stage dual solutions to construct inexact minorants for the expectation-valued objective function. On the other hand, a partition-based inexact proximal bundle algorithm utilizes the optimal active sets obtained in earlier iterations, along with a primal-dual active set method, to construct the inexact minorant. For both these variants, we establish their asymptotic convergence to optimal solutions. Using a carefully developed computer implementation, we demonstrate the practical behavior of these algorithms through numerical experiments conducted on power systems planning and operations problems. The results indicate that the partition-based algorithm consistently identifies solutions of comparable quality to those obtained from exact algorithms, while significantly reducing computational time.