Composite Optimization With Coupling Constraints via Penalized Proximal Gradient Method in Asynchronous Networks

In this paper, we propose an accelerated proximal gradient method with line search for solving the large-scale nonconvex penalty problems. Compared with the classic proximal gradient method, the new method does not need to evaluate the Lipschitz constant, parameters are initialized by some aggressive values and updated adaptively at each iteration. Under some common assumptions, the global subsequential convergence is established. Moreover, some numerical results on large-scale SCAD and MCP nonconvex penalty problems are reported to indicate the efficiency and superiority of the proposed method.

A joint convex penalty for inverse covariance matrix estimation

This paper proposes a novel nonmonotone proximal gradient method for solving nondifferentiable composite multiobjective optimization problems. The method integrates a nonmonotone line search technique into the proximal gradient framework, offering enhanced flexibility and efficiency for handling nonsmooth or nonconvex objective functions. We analyze the convergence properties of the proposed algorithm, demonstrate that all accumulation points of the generated sequence satisfy Pareto stationarity when the problem is nonconvex. Furthermore, under convexity and mild conditions, we establish that the sequence converges to a weakly Pareto optimal solution of the problem. Numerical experiments on robust multiobjective optimization problems and Lasso multiobjective optimization problems are provided to illustrate the computational efficiency and the practical utility of the proposed method.

In January 2017, The Government of Yunnan Province issued the implementation plan of electricity market reform for retail side. The electricity retail market has officially entered the stage of power market in Yunnan Province. The construction of power retail market is conducive to improving the activity of power market, cultivating the market awareness of power users, promoting power users to improve demand side management ability, and ensuring the safe, economic and efficient operation of power system. According to the official website of Kunming Power Exchange Center Company Limited, at present, there are 146 registered power sales companies and 33062 registered power users in Yunnan Province. The power market transactions are active, the transacted electricity quantity is increasing year by year, and the power retail market is booming. The retail package of electricity is the basis of mutual benefit and win-win between the electricity selling company and the retail users. The flexible, reasonable and effective electric retail package reflects the healthy and efficient operation of the electric retail market to a certain extent. Therefore, the research and development of electric power retail package is a very meaningful topic, which is worthy of in-depth study and practice.Firstly, based on the operation of Yunnan electric power wholesale market and electric power retail market, combined with the business practice of Yunnan electric power retail settlement payment, this paper points out the constraint boundary of the design of electric power retail package in electricity market of Yunnan province, such as the coupling constraints between electric power wholesale transaction and retail transaction, the coupling constraints between electric power wholesale settlement and retail settlement, etc. Then, combined with the characteristics of electricity load, electricity price bearing capacity, demand response and other characteristics, this paper makes a user portrait of Yunnan users, classifies and summarizes the power transaction demand of retail users, and designs the corresponding electric power retail package, such as free service, conditional service fee, price difference income, energy management and other electric power packages, which are full through the appropriate electric power retail package. It can meet the demand of electricity retail users and form a win-win situation. Secondly, this paper introduces the operation mode of all kinds of retail packages in Electricity Market of Yunnan province in detail. On the one hand, it is the business process of transaction and settlement, including the wholesale, retail transaction and settlement business process of power selling company, the transaction and settlement business process of power users, the transaction and settlement business organization flow of power exchange center et al. On the other hand, it is the cash flow and invoice flow of settlement and payment, including the cash flow and invoice flow of retail electricity payment of power users. Finally, through the practical analysis of the operation of power retail market in Yunnan province, it points out the advantages and disadvantages of various packages. Further combining with the construction and development plan of electricity market in Yunnan province, especially the operation of electric power spot market, some changes for the design of power retail package are proposed.

The low-rank plus sparse (L+S) decomposition model enables the reconstruction of under-sampled dynamic parallel magnetic resonance imaging (MRI) data. Solving for the low-rank and the sparse components involves non-smooth composite convex optimization, and algorithms for this problem can be categorized into proximal gradient methods and variable splitting methods. This paper investigates new efficient algorithms for both schemes. While current proximal gradient techniques for the L+S model involve the classical iterative soft thresholding algorithm (ISTA), this paper considers two accelerated alternatives, one based on the fast iterative shrinkage-thresholding algorithm (FISTA), and the other with the recent proximal optimized gradient method (POGM). In the augmented Lagrangian (AL) framework, we propose an efficient variable splitting scheme based on the form of the data acquisition operator, leading to simpler computation than the conjugate gradient (CG) approach required by existing AL methods. Numerical results suggest faster convergence of the efficient implementations for both frameworks, with POGM providing the fastest convergence overall and the practical benefit of being free of algorithm tuning parameters.

This paper deals with composite optimization problems having the objective function formed as the sum of two terms; one has a Lipschitz continuous gradient along random subspaces and may be nonconvex, and the second term is simple and differentiable but possibly nonconvex and nonseparable. Under these settings, we design a stochastic coordinate proximal gradient method that takes into account the nonseparable composite form of the objective function. This algorithm achieves scalability by constructing at each iteration a local approximation model of the whole nonseparable objective function along a random subspace with user-determined dimension. We outline efficient techniques for selecting the random subspace, yielding an implementation that has low cost per iteration, also achieving fast convergence rates. We present a probabilistic worst case complexity analysis for our stochastic coordinate proximal gradient method in convex and nonconvex settings; in particular, we prove high-probability bounds on the number of iterations before a given optimality is achieved. Extensive numerical results also confirm the efficiency of our algorithm. Funding: This work was supported by Norway Grants 2014-2021 [Grant ELO-Hyp 24/2020]; Unitatea Executiva pentru Finantarea Invatamantului Superior, a Cercetarii, Dezvoltarii si Inovarii [Grants PN-III-P4-PCE-2021-0720, L2O-MOC, nr 70/2022]; and the ITN-ETN project TraDE-OPT funded by the European Union’s Horizon 2020 Research and Innovation Programme under the Marie Skłodowska-Curie grant agreement [Grant 861137].

Carbon fiber-reinforced polymer (CFRP) cables have been widely applied in modern engineering due to their excellent mechanical properties and durability, and their service safety has received increasing attention. However, there is currently a lack of efficient and accurate methods for detecting defects in CFRP cables. This article proposes a CFRP cable defect detection method based on electromagnetic tomography (EMT) technology to provide technical support for ensuring the safe service of cables. Due to the ill-posedness and instability of the inverse problem of EMT, there are reconstruction errors in the image reconstruction results. To address the EMT image reconstruction problem, an L1 regularization image reconstruction algorithm based on the proximal gradient method is proposed, which improves the defect reconstruction ability and reconstruction accuracy of the algorithm. First, the objective function is gradually optimized by combining gradient descent and proximal mapping operations to achieve variable-step-size gradient descent to handle the solution problems caused by the nonsmooth terms of the regularization function. On this basis, the image color mapping is adjusted to suppress the artifacts and noise in the reconstructed image. Finally, the proposed method is verified by carbon fiber composite material cables with different defect distributions. The numerical and experimental results show that the proposed method has stronger defect detection ability and can obtain high-quality defect reconstruction results. Compared with traditional methods, the image evaluation indicators have increased by 29.6% and 51.7%, respectively.

In this work, we generalized and unified two recent completely different works of Jascha [10] and Lee [2] respectively into one by proposing the proximal stochastic Newton-type gradient (PROXTONE) method for optimizing the sums of two convex functions: one is the average of a huge number of smooth convex functions, and the other is a nonsmooth convex function. Our PROXTONE incorporates second order information to obtain stronger convergence results, that it achieves a linear convergence rate not only in the value of the objective function, but also for the solution. The proofs are simple and intuitive, and the results and technique can be served as a initiate for the research on the proximal stochastic methods that employ second order information. The methods and principles proposed in this paper can be used to do logistic regression, training of deep neural network and so on. Our numerical experiments shows that the PROXTONE achieves better computation performance than existing methods.

We consider the network traffic monitoring issue, including traffic matrix estimation and anomaly detection problems. New model called structure regularized traffic monitoring (SRTM) is presented by incorporating the spatial and temporal properties of the traffic matrices, with the sparsity of the traffic volume anomalies. The proposed SRTM model adapts more naturally to the engineering nature of the real traffic data and imposes structural regularizations. This stems from the fact that the real traffic data is usually approximately instead of exactly low-rank and exhibits strong spatial and temporal dependencies. In addition, we take advantages of the sparsity of the traffic volume anomalies and impose sparse constraints on the variables to detect the traffic anomalies. Efficient algorithm that based on the proximal gradient and gradient descent methods are proposed to solve the SRTM optimization model. Experiments in synthetic networks are carried out to verify the feasibility and effectiveness of the proposed model.

In this paper, we propose to use spectral proximal method to solve sparse optimization problems. Sparse optimization refers to an optimization problem involving the ι0 -norm in objective or constraints. The previous research showed that the spectral gradient method is outperformed the other standard unconstrained optimization methods. This is due to spectral gradient method replaced the full rank matrix by a diagonal matrix and the memory decreased from Ο(n2) to Ο(n). Since ι0-norm term is nonconvex and non-smooth, it cannot be solved by standard optimization algorithm. We will solve the ι0 -norm problem with an underdetermined system as its constraint will be considered. Using Lagrange method, this problem is transformed into an unconstrained optimization problem. A new method called spectral proximal method is proposed, which is a combination of proximal method and spectral gradient method. The spectral proximal method is then applied to the ι0-norm unconstrained optimization problem. The programming code will be written in Python to compare the efficiency of the proposed method with some existing methods. The benchmarks of the comparison are based on number of iterations, number of functions call and the computational time. Theoretically, the proposed method requires less storage and less computational time.

In this article, a novel framework based on trace norm minimization for audio classification is proposed. In this framework, both the feature extraction and classification are obtained by solving corresponding convex optimization problem with trace norm regularization. For feature extraction, robust principle component analysis (robust PCA) via minimization a combination of the nuclear norm and the ℓ 1 -norm is used to extract low-rank matrix features which are robust to white noise and gross corruption for audio signal. These low-rank matrix features are fed to a linear classifier where the weight and bias are learned by solving similar trace norm constrained problems. For this linear classifier, most methods find the parameters, that is the weight matrix and bias in batch-mode, which makes it inefficient for large scale problems. In this article, we propose a parallel online framework using accelerated proximal gradient method. This framework has advantages in processing speed and memory cost. In addition, as a result of the regularization formulation of matrix classification, the Lipschitz constant was given explicitly, and hence the step size estimation of the general proximal gradient method was omitted, and this part of computing burden is saved in our approach. Extensive experiments on real data sets for laugh/non-laugh and applause/non-applause classification indicate that this novel framework is effective and noise robust.

Primal-dual proximal bundle and conditional gradient methods for convex problems

In this paper, we explore a specific optimization problem that involves the combination of a differentiable nonconvex function and a nondifferentiable function. The differentiable component lacks a global Lipschitz continuous gradient, posing challenges for optimization. To address this issue and accelerate the convergence, we propose a Bregman proximal stochastic gradient method with extrapolation (BPSGE), which only requires smooth adaptivity of the differentiable part. Under variance reduction framework, we not only analyze the subsequential and global convergence of the proposed algorithm under certain conditions, but also analyze the sublinear convergence rate of the subsequence, and the complexity of the algorithm, revealing that the BPSGE algorithm requires at most O(epsilon\^\,(-2)) iterations in expectation to attain an epsilon-stationary point. To validate the effectiveness of our proposed algorithm, we conduct numerical experiments on three real-world applications: graph regularized nonnegative matrix factorization (NMF), matrix factorization with weakly-convex regularization, and NMF with nonconvex sparsity constraints. These experiments demonstrate that BPSGE is faster than the baselines without extrapolation. The code is available at: https://github.com/nothing2wang/BPSGE-Algorithm.

Energy-efficient power allocation in cell-free massive MIMO with zero-forcing: First order methods

Proximal gradient methods are popular in sparse optimization as they are straightforward to implement. Nevertheless, they achieve biased solutions, requiring many iterations to converge. This work addresses these issues through a suitable feedback control of the algorithm’s hyperparameter. Specifically, by designing an integral control that does not substantially impact the computational complexity, we can reach an unbiased solution in a reasonable number of iterations. In the paper, we develop and analyze the convergence of the proposed approach for strongly-convex problems. Moreover, numerical simulations validate and extend the theoretical results to the non-strongly convex framework.