Big data sketching with model mismatch

This paper studies a new Bayesian optimization algorithm for fusing hyperspectral and multispectral images. The hyperspectral image is supposed to be obtained by blurring and subsampling a high spatial and high spectral target image. The multispectral image is modeled as a spectral mixing version of the target image. By introducing appropriate priors for parameters and hyperparameters, the fusion problem is formulated within a Bayesian estimation framework, which is very convenient to model the noise and the target image. The high spatial resolution hyperspectral image is then inferred from its posterior distribution. To compute the Bayesian maximum a posteriori estimator associated with this posterior, an alternating direction method of multipliers within block coordinate descent algorithm is proposed. Simulation results demonstrate the efficiency of the proposed fusion method when compared with several state-of-the-art fusion techniques.

We consider the problem of learning a multivariate function from a set of scattered observations using a sparse low-rank separated representation (SSR) model. The model structure considered here is promising for high-dimensional learning problems; however, existing training algorithms based on alternating least-squares (ALS) are known to have convergence difficulties, particularly when the rank of the model is greater than 1. In the present work, we supplement the model structure with sparsity constraints to ensure the well posedness of the approximation problem. We propose two fast training algorithms to estimate the model parameters: (i) a cyclic coordinate descent algorithm and (ii) a block coordinate descent (BCD) algorithm. While the first algorithm is not provably convergent owing to the non-convexity of the optimization problem, the BCD algorithm guarantees convergence to a Nash equilibrium point. The computational cost of the proposed algorithms is shown to scale linearly with respect to all of the parameters in contrast to methods based on ALS. Numerical studies on synthetic and real-world regression datasets indicate that the proposed SSR model structure holds significant potential for machine learning problems.

As the number of pre-trained machine learning (ML) models is growing exponentially, data reduction tools are not catching up. Existing data reduction techniques are not specifically designed for pre-trained model (PTM) dataset files. This is largely due to a lack of understanding of the patterns and characteristics of these datasets, especially those relevant to data reduction and compressibility. This paper presents the first, exhaustive analysis to date of PTM datasets on storage compressibility. Our analysis spans different types of data reduction and compression techniques, from hash-based data deduplication, data similarity detection, to dictionary-coding compression. Our analysis explores these techniques at three data granularity levels, from model layers, model chunks, to model parameters. We draw new observations that indicate that modern data reduction tools are not effective when handling PTM datasets. There is a pressing need for new compression methods that take into account PTMs' data characteristics for effective storage reduction. Motivated by our findings, we design Elf, a simple yet effective, error-bounded, lossy floating-point compression method. Elf transforms floating-point parameters in such a way that the common exponent field of the transformed parameters can be completely eliminated to save storage space. We develop Elves, a compression framework that integrates Elf along with several other data reduction methods. Elves uses the most effective method to compress PTMs that exhibit different patterns. Evaluation shows that Elves achieves an overall compression ratio of 1.52×, which is 1.31×, 1.32× and 1.29× higher than a general-purpose compressor (zstd), an error-bounded lossy compressor (SZ3), and the uniform model quantization, respectively, with negligible model accuracy loss.

Learning the structure of Bayesian networks (BNs) from high dimensional discrete data is common nowadays but a challenging task, due to the large parameter space, the acyclicity constraint placed on the graphical structures and the difficulty in searching for a sparse structure. In this article, we propose a sparse structure learning algorithm (SSLA) to solve this problem. The algorithm uses the negative log-likelihood function of multi-logit regression as a loss function, adding the adaptive group lasso as a penalty term for sparsity, with a new penalty term to ensure that the learned graph is a directed acyclic graph. A block coordinate descent algorithm (BCD) combining with the alternating direction multiplier method (ADMM) algorithm is developed to solve the proposed model. The learned graph is proved theoretically to be a Bayesian network. In order to evaluate the proposed SSLA and compare with its competitors, we conducted intensive simulation studies and applied them to the benchmark Bayesian networks. The results indicate that the SSLA is superior to the hill climbing (HC) algorithm, the CD algorithm and the BFO-B algorithm respectively, and is competitive with K2 algorithm when the order of the nodes is given.

This paper presents a novel importance-aware quantization, subcarrier mapping, and power allocation (IA-QSMPA) framework for semantic communication in multiple-input multiple-output orthogonal frequency division multiplexing (MIMO-OFDM) systems, empowered by a pretrained Vision Transformer (ViT). The proposed framework exploits attention-based importance extracted from a pretrained ViT to jointly optimize quantization levels, subcarrier mapping, and power allocation. Specifically, IA-QSMPA maps semantically important features to high-quality subchannels and allocates resources in accordance with their contribution to task performance and communication latency. To efficiently solve the resulting nonconvex optimization problem, a block coordinate descent algorithm is employed. The framework is further extended to operate under finite blocklength transmission, where communication errors may occur. In this setting, a segment-wise linear approximation of the channel dispersion penalty is introduced to enable efficient joint optimization under practical constraints. Simulation results on multi-view image classification and single-object detection tasks demonstrate that IA-QSMPA significantly outperforms conventional methods in both ideal and finite blocklength transmission scenarios, achieving superior task performance and communication efficiency.

An improved parallel block coordinate descent (iPBCD) algorithm for solving the user equilibrium traffic assignment problem is proposed. The iPBCD algorithm is developed based on the parallel block coordinate descent algorithm (PBCD). The hybrid flow update policy is investigated to enhance the robustness and performance of the PBCD algorithm. Two update order rules, namely a cyclic rule and a greedy rule are compared for block indices. Then, the block size is optimized using a sensitivity analysis test. Finally, five index-grouping rules are tested for comparative purposes. Numerical experiments indicate that index-grouping rules have a significant influence on convergence: the information-based drop-out rule performs better in terms of convergence and efficiency.

Reconfigurable Intelligent Surfaces (RISs) and Non-Orthogonal Multiple Access (NOMA) can enhance Visible Light Communication (VLC) systems by mitigating signal blockage and improving spectrum utilization. While boosting communication efficiency is crucial, maintaining high illumination quality is equally important. This paper investigates a novel approach to simultaneously improving the sum rate (SR) and illumination uniformity (IU) in a RIS-assisted NOMA-based VLC system. Communication and illumination optimization problems are formulated as a non-convex mixed-integer non-linear programming problem, considering RIS placement, LED-user association, and power allocation. To the best of our knowledge, this is the first work to jointly optimize SR and IU with explicit consideration of RIS placement. We propose a joint optimization approach that leverages a differential evolution algorithm to optimize RIS placement. During each iteration, the obtained solutions are further refined using a block coordinate descent algorithm, which iteratively solves the decomposed sub-problems of LED-user association and power allocation. Simulation results show that the approach outperforms existing methods in both SR and IU. Moreover, RIS placement optimization is shown to be crucial, as neglecting it significantly degrades performance. Finally, the impacts of noise power and total LED power are analyzed, offering practical insights for parameter selection in VLC systems.

In this paper, we consider a cooperative diffusion-based molecular communication (MC) network consisting of single source, single decode-and-forward relay, and single destination. The source and the relay nodes use different types of molecules for information transmission. Considering both the relay and the direct links at the destination node, we have two different received signals that can be used by the destination node to retrieve the data transmitted by the source node. Therefore, diversity arises at the destination node. For detection, we adopt the energy detector applying the diversity combining technique. Since the optimum relay position in relay-aided networks plays a crucial role to enhance their performance, we study the relay location optimization problem. We first derive a closed-form expression for the bit error probability of the proposed detection scheme in which the received energy from the source and the relay nodes are linearly combined at the destination node. Then, in order to find the optimum relay position as well as the optimal destination decision threshold for a given number of molecules released by the source and the relay nodes, we formulate a joint optimization problem whose objective is to minimize the bit error probability of the network. To solve this optimization problem, we propose an iterative algorithm based on the block coordinate descent algorithm and study its convergence behavior. Numerical results show that the error performance can be improved by optimizing the position of the relay node. Our analyses help us in designing a reliable cooperative diffusion-based MC network.

Clustering is an unsupervised learning approach that explores data and seeks groups of similar objects. Many classical clustering models such as k-means and DBSCAN are based on heuristics algorithms and suffer from local optimal solutions and numerical instability. Recently convex clustering has received increasing attentions, which leverages the sparsity inducing norms and enjoys many attractive theoretical properties. However, convex clustering is based on Euclidean distance and is thus not robust against outlier features. Since the outlier features are very common especially when dimensionality is high, the vulnerability has greatly limited the applicability of convex clustering to analyze many real-world datasets. In this paper, we address the challenge by proposing a novel robust convex clustering method that simultaneously performs convex clustering and identifies outlier features. Specifically, the proposed method learns to decompose the data matrix into a clustering structure component and a group sparse component that captures feature outliers. We develop a block coordinate descent algorithm which iteratively performs convex clustering after outliers features are identified and eliminated. We also propose an efficient algorithm for solving the convex clustering by exploiting the structures on its dual problem. Moreover, to further illustrate the statistical stability, we present the theoretical performance bound of the proposed clustering method. Empirical studies on synthetic data and real-world data demonstrate that the proposed robust convex clustering can detect feature outliers as well as improve cluster quality.

The cloud radio access network (Cloud-RAN) is an emerging network architecture for the next generation mobile wireless networks. However, connecting a large number of small base stations (SBSs) through wired backhaul links is inconvenient and cost-prohibitive. Therefore, wireless backhaul has become a promising solution for Cloud-RAN. In this paper, we propose to apply the multicast beamforming technique in wireless backhaul, which is efficient to share each user's message to a cluster of SBSs for collaboratively serving the user through joint beamforming. The joint optimization of multicast beamforming in backhaul links and user-centric clustering in access links is formulated as a weighted sum rate maximization problem. To tackle this intractable problem, several approximation and transformation techniques are introduced based on the semidefinite relaxation method and successive convex approximation approach. A block coordinate descent algorithm is developed to solve the approximate optimization problem iteratively by exploiting its special structure. Simulation results show that the proposed algorithm is guaranteed to converge to the stationary point, and the performance outperforms other wireless backhaul schemes under various network scenes.

The smoothness of subdiagonals of the Cholesky factor of large covariance matrices is closely related to the degree of nonstationarity of autoregressive models for time series data. Heuristically, one expects for nearly stationary covariance matrix entries in each subdiagonal of the Cholesky factor of its inverse to be approximately the same in the sense that the sum of the absolute values of successive differences is small. Statistically, such smoothness is achieved by regularizing each subdiagonal using fused-type lasso penalties. We rely on the standard Cholesky factor as the new parameter within a regularized normal likelihood setup which guarantees: (a) joint convexity of the likelihood function, (b) strict convexity of the likelihood function restricted to each subdiagonal even when n < p, and (c) positive-definiteness of the estimated covariance matrix. A block coordinate descent algorithm, where each block is a subdiagonal, is proposed, and its convergence is established under mild conditions. Lack of decoupling of the penalized likelihood function into a sum of functions involving individual subdiagonals gives rise to some computational challenges and advantages relative to two recent algorithms for sparse estimation of the Cholesky factor, which decouple row-wise. Simulation results and real data analysis show the scope and good performance of the proposed methodology. Software for our method is freely available in R language. Supplementary materials for this article are available online.

Block coordinate descent energy minimization for dynamic cohesive fracture

An Algorithm for the Multivariate Group Lasso with Covariance Estimation

GumTree: Data reduction

The joint optimization of rate-splitting multiple access (RSMA) and reconfigurable intelligent surface (RIS) offers a mutually beneficial solution. In this paper, we study a RIS-assisted downlink user grouping-based hierarchical RSMA communication network, where the base station transmits the common streams and private streams simultaneously to serve multiple users. The sum rate (SR) optimization problem is constructed to jointly optimize user grouping, beamforming, and rate splitting. To tackle the NP-hard problem, a user grouping algorithm is first provided to form the hierarchical RSMA model. Next, an equivalent weighted minimum mean-square error (WMMSE) problem is constructed and a WMMSE-based block coordinate descent (BCD) algorithm is designed to update the remaining variables in an iterated way. Specifically, the optimization of transmit beamforming and the rate splitting matrix is reduced to solving a typical convex quadratically constrained quadratic program problem. The solution for passive beamforming is recovered from the intermediate result obtained by the manifold optimization method. Finally, the process of the BCD algorithm is presented. The numerical results indicate that the proposed scheme evidently surpasses one-layer RSMA scheme, NOMA scheme, SDMA scheme, and no RIS scheme, especially in overloaded scenarios. The proposed scheme further increases the SR of common steams compared to the hierarchical RSMA scheme and significantly reduces computational complexity compared to the generalized RSMA scheme.