Unsupervised tail modeling via noisy cross‐entropy minimization

Regression models with intractable normalizing constants are valuable tools for analyzing complex data structures, yet parameter inference for such models remains highly challenging—particularly when observations are discrete. In statistical inference, discrete state spaces introduce significant computational difficulties, as the normalizing constant often requires summation over extremely large or even infinite sets, which is typically infeasible in practice. These challenges are further compounded when observations are independent but not identically distributed. This paper addresses these issues by developing a novel generalized Bayesian inference approach tailored for regression models with intractable likelihoods. The key idea is to employ a specific form of generalized Fisher divergence to update beliefs about the model parameters, thereby circumventing the need to compute the normalizing constant. The resulting generalized posterior distribution can be sampled using standard computational tools, such as Markov Chain Monte Carlo (MCMC), effectively avoiding the intractability of the normalizing constant.

Simulating from distributions with intractable normalizing constants has been a long-standing problem in machine learning. In this letter, we propose a new algorithm, the Monte Carlo Metropolis-Hastings (MCMH) algorithm, for tackling this problem. The MCMH algorithm is a Monte Carlo version of the Metropolis-Hastings algorithm. It replaces the unknown normalizing constant ratio by a Monte Carlo estimate in simulations, while still converges, as shown in the letter, to the desired target distribution under mild conditions. The MCMH algorithm is illustrated with spatial autologistic models and exponential random graph models. Unlike other auxiliary variable Markov chain Monte Carlo (MCMC) algorithms, such as the Møller and exchange algorithms, the MCMH algorithm avoids the requirement for perfect sampling, and thus can be applied to many statistical models for which perfect sampling is not available or very expensive. The MCMH algorithm can also be applied to Bayesian inference for random effect models and missing data problems that involve simulations from a distribution with intractable integrals.

The problem of simulating from distributions with intractable normalizing constants has received much attention in recent literature. In this article, we propose an asymptotic algorithm, the so-called double Metropolis–Hastings (MH) sampler, for tackling this problem. Unlike other auxiliary variable algorithms, the double MH sampler removes the need for exact sampling, the auxiliary variables being generated using MH kernels, and thus can be applied to a wide range of problems for which exact sampling is not available. For the problems for which exact sampling is available, it can typically produce the same accurate results as the exchange algorithm, but using much less CPU time. The new method is illustrated by various spatial models.

Shor's algorithms for the integer factorization and the discrete logarithm problems can be regarded as a negative effect of the quantum mechanism on publickey cryptography. From the computational point of view, his algorithms illustrate that quantum computation could be more powerful. It is natural to consider that the power of quantum computation could be exploited to withstand even quantum adversaries. Over the last decade, quantum cryptography has been discussed and developed even from the computational complexity-theoretic point of view. In this paper, we will survey what has been studied in quantum computational cryptography.

Nowadays, many biological data are acquired via images. In this article, we study the pathological images scanned from 205 patients with lung cancer with the goal to find out the relationship between the survival time and the spatial distribution of different types of cells, including lymphocyte, stroma, and tumor cells. Toward this goal, we model the spatial distribution of different types of cells using a modified Potts model for which the parameters represent interactions between different types of cells and estimate the parameters of the Potts model using the double Metropolis-Hastings algorithm. The double Metropolis-Hastings algorithm allows us to simulate samples approximately from a distribution with an intractable normalizing constant. Our numerical results indicate that the spatial interaction between the lymphocyte and tumor cells is significantly associated with the patient’s survival time, and it can be used together with the cell count information to predict the survival of the patients.

The density estimation is one of the core problems in statistics. Despite this, existing techniques like maximum likelihood estimation are computationally inefficient in case of complex parametric families due to the intractability of the normalizing constant. For this reason, an interest in score matching has increased, being independent on the normalizing constant. However, such an estimator is consistent only for distributions with the full space support. One of the approaches to make it consistent is to add noise to the input data called Denoising Score Matching. In this work we build computationally efficient algorithm for density estimate using kernel exponential family as a model distribution. The usage of the kernel exponential family is motivated by the richness of this class of densities. To avoid calculating an intractable normalizing constant we use Denoising Score Matching objective. The computational complexity issue is approached by applying Random Fourier Features-based approximation of the kernel function. We derive an exact analytical expression for this case which allows dropping additional regularization terms based on the higher-order derivatives as they are already implicitly included. Moreover, the obtained expression explicitly depends on the noise variance, so that the validation loss can be straightforwardly used to tune the noise level. Along with benchmark experiments, the method was tested on various synthetic distributions to study the behavior of the method in different cases. The empirical study shows comparable quality to the competing approaches, while the proposed method being computationally faster. The latter one enables scaling up to complex high-dimensional data.

SUMMARYWe present some advances, both from a theoretical and a computational point of view, on a quadratic vector equation (QVE) arising in Markovian Binary Trees. Concerning the theoretical advances, some irreducibility assumptions are relaxed, and the minimality of the solution of the QVE is expressed in terms of properties of the Jacobian of a suitable function. From the computational point of view, we elaborate on the Perron vector‐based iteration proposed in [Meini and Poloni, A Perron iteration for the solution of a quadratic vector equation arising in Markovian binary trees]. In particular, we provide a condition that ensures that the Perron iteration converges to the sought solution of the QVE. Moreover we introduce a variant of the algorithm that consists of the application of the Newton method instead of a fixed‐point iteration. This method has the same convergence behavior as the Perron iteration, namely it tends to converge faster for close‐to‐critical problems. Moreover, unlike the Perron iteration, the method has a quadratic convergence. Finally, we show that it is possible to alter the bilinear form defining the QVE in several ways without changing the solution. This modification has an impact on convergence speed of the algorithms. Copyright © 2011 John Wiley & Sons, Ltd.

This paper aims to show how to calculate different efficiency measures using a technology estimator defined through the adaptation of the Gradient Tree Boosting algorithm. This adaptation shares some features with the standard nonparametric Free Disposal Hull (FDH) approach, but it overcomes data overfitting problems. Nevertheless, from a computational point of view, the new approach presents thousands of decision variables, making it difficult to solve. To tackle this problem, we also propose and check a heuristic approximation to the exact measures. To demonstrate the applicability of the proposed method, the exact and the heuristic approaches are compared through two empirical applications. The main contributions of this paper are as follows: we build a new bridge between machine learning techniques and technical efficiency measurement. In this framework, we show how to determine the output-oriented and input-oriented radial models, the Russell measure of output efficiency and the Russell measure of input efficiency, as well as the directional distance function and the Enhanced Russell Graph measure. We also prove that the new technique is better, in terms of bias and squared mean error, than the standard FDH technique. Furthermore, we show that the new approach may be seen as a possible remedy for solving the curse of dimensionality problem.

This paper addresses a single-machine scheduling problem with nonresumable jobs and flexible maintenance activities (SJM) and its online version SJMO. It is showed that the competitive ratio of the classical list scheduling (LS) algorithm is 2 for SJMO, and it is the best possible polynomial time approximation algorithm for SJMO from both the competitive ratio point of view unless P = NP and the computational time complexity point of view. Based on the relationship between SJM and SJMO, it is showed that the LS algorithm is the best possible polynomial time approximation algorithm for SJM from both the worst-case ratio point of view unless P = NP and the computational time complexity point of view, which is a significant improvement of some of the existing results on SJM and its special case in the literature. Note that the worst-case ratio of the classical longest processing time first (LPT) algorithm is also 2 for SJM. In order to show which algorithm is better for SJM on average, numerical experiments are conducted. Computational results show that the LPT algorithm performs better than the LS algorithm in all combinations from an average-case error point of view.

A todos tengo que agradecer las sugerencias y/o preguntas sobre SSm que me han ayudado a encontrar errores, a mejorarlo y a aprender un poco ms. Quiero resaltar a algunos por su relacin ms

The effect of non-uniformity of axial loading on the buckling behaviour of shells with random imperfections

Suppose we are given n independent observations (X 1, Y 1), …, (Xn, Yn ) from a conditionally specified distribution with density f(x, y). The problem of estimating the unknown parameters of f(x, y) is complicated by the presence of an intractable normalizing constant that, in this conditional approach, is chosen so that the density integrates to 1. An approach to estimation is used here that is known to result in asymptotically efficient estimators of the unknown parameters when f(x, y) is from an exponential family. It is an application of a method that has appeared in the literature and is due to J. K. Lindsey. It very conveniently avoids the dependence on the normalizing constants in the joint distribution. The usual maximum-likelihood estimates of the parameters can be obtained using software readily available for Poisson regression. The usefulness of this estimation method is illustrated for a model resulting from specifying that the conditionals are two-parameter, shape and scale, gamma. I...

With the growth of interest in network data across fields, the Exponential Random Graph Model (ERGM) has emerged as the leading approach to the statistical analysis of network data. ERGM parameter estimation requires the approximation of an intractable normalizing constant. Simulation methods represent the state-of-the-art approach to approximating the normalizing constant, leading to estimation by Monte Carlo maximum likelihood (MCMLE). MCMLE is accurate when a large sample of networks is used to approximate the normalizing constant. However, MCMLE is computationally expensive, and may be prohibitively so if the size of the network is on the order of 1,000 nodes (i.e., one million potential ties) or greater. When the network is large, one option is maximum pseudolikelihood estimation (MPLE). The standard MPLE is simple and fast, but generally underestimates standard errors. We show that a resampling method---the parametric bootstrap---results in accurate coverage probabilities for confidence intervals. We find that bootstrapped MPLE can be run in 1/5th the time of MCMLE. We study the relative performance of MCMLE and MPLE with simulation studies, and illustrate the two different approaches by applying them to a network of bills introduced in the United State Senate.

Undirected graphical models such as Markov random fields or Boltzmann machines prove useful in many signal processing and machine learning tasks. However, parameter estimation in these models is difficult due to the intractable normalising constant in their probability density functions. One powerful technique for parameter estimation in such models is score matching. This technique makes use of an objective function which is independent of the normalising constant and constitutes locally consistent estimators for the parameters of such models. However, score matching is only applicable to fully-observed models. In this paper, we extend the applicability of score matching to models with latent variables. Our estimators are unbiased, based on Monte Carlo integration. Unbiased gradient estimators open the way to optimisation through stochastic approximation. We demonstrate the performance of our methodology on two synthetic problems.

A statistical model is said to be un-normalised when its likelihood function involves an intractable normalising constant. Two popular methods for parameter inference for these models are MC-MLE (Monte Carlo maximum likelihood estimation), and NCE (noise contrastive estimation); both methods rely on simulating artificial data-points to approximate the normalising constant. While the asymptotics of MC-MLE have been established under general hypotheses (Geyer, 1994), this is not so for NCE. We establish consistency and asymptotic normality of NCE estimators under mild assumptions. We compare NCE and MC-MLE under several asymptotic regimes. In particular, we show that, when $m\rightarrow \infty $ while $n$ is fixed ($m$ and $n$ being respectively the number of artificial data-points, and actual data-points), the two estimators are asymptotically equivalent. Conversely, we prove that, when the artificial data-points are IID, and when $n\rightarrow \infty $ while $m/n$ converges to a positive constant, the asymptotic variance of a NCE estimator is always smaller than the asymptotic variance of the corresponding MC-MLE estimator. We illustrate the variance reduction brought by NCE through a numerical study.