Free-knot Splines Approximation of s -monotone Functions

We introduce a new hierarchy over monotone set functions, that we refer to as MPH (Maximum over Positive Hypergraphs). Levels of the hierarchy correspond to the degree of complementarity in a given function. The highest level of the hierarchy, MPH-m (where m is the total number of items) captures all monotone functions. The lowest level, MPH-1, captures all monotone submodular functions, and more generally, the class of functions known as XOS. Every monotone function that has a positive hypergraph representation of rank k (in the sense defined by Abraham, Babaioff, Dughmi and Roughgarden [EC 2012]) is in MPH-k. Every monotone function that has supermodular degree k (in the sense defined by Feige and Izsak [ITCS 2013]) is in MPH-(k+1). In both cases, the converse direction does not hold, even in an approximate sense. We present additional results that demonstrate the expressiveness power of MPH-k.One can obtain good approximation ratios for some natural optimization problems, provided that functions are required to lie in low levels of the MPH hierarchy. We present two such applications. One shows that the maximum welfare problem can be approximated within a ratio of k+1 if all players hold valuation functions in MPH-k. The other is an upper bound of 2k on the price of anarchy of simultaneous first price auctions.

This article explores the algorithms for solving individual classes of discrete extremal problems to find the exact optimum. When obtaining algorithms, the procedures for decoding and finding the maximum upper zero of discrete monotonic functions are used. Formulations of the tasks of decoding and searching for the maximum upper zero of discrete monotonic functions are formulated. To solve the problems of decoding and searching for the maximum upper zero of individual classes of 3-digit monotonic functions, estimates are given for the complexity of deciphering monotonic functions in a class.

Partial functions of the k-valued logic monotone with respect to an arbitrary partially ordered set with the least and largest elements and distinct from a lattice are considered. It is shown that the set of closed classes of partial monotone functions containing a precomplete in Pk class of everywhere determined monotone function is infinite.

We solve two continuous extremal problems on the classes of monotone functions: in the first problem we find extremal values for a line integral of a coordinate-wise monotone function of two variables from a rearrangement-invariant class of functions; in the second one we find extremal values for the expectation of a random process with monotone trajectories at a random time. In both cases we reduce the continuous problems to their discrete counterparts. The obtained discrete problems are on the one hand interesting on their own, and on the other hand give a natural explanation of the structure of the extremal functions for the continuous problems.

The article study compares neural network based models and difference equation based models in 2D pattern recognition of handwritten digits. Pattern classifier models based on two neural network models were implemented using multi-layer feed forward neural networks: single layer perceptron and a multi-layer perceptron with one hidden layer. These two neural network models have nonlinear sigmoidal threshold units. A classifier model based on difference equations with linear weighted sum of all inputs was also implemented. Beside linear inputs, nonlinearities were introduced in all of the models by including products of inputs as new inputs. All models were trained/adjusted and tested using a handwritten digit database composed of a training set of 2360 digits and a testing set of 1320 digits. Each handwritten digit is composed of 15/spl times/23 binary pixels and each pixel is considered as a single input to the classifier. Classifier performance was measured as the correct classification rate on the testing database. Results show that the difference equation model with only linear inputs yielded the worse results (72.4%). Nevertheless, as products among inputs were included as new inputs, the classification performance increased (84.1%), thus exceeding the results of the single-layer perceptron with linear inputs (72.7%). The single-layer perceptron with nonlinear inputs gave an improvement of the classification result to 86.3%. The products among inputs can be interpreted as convolutional filtering over the input image to detect lines in different directions producing an overall improvement in the classification performance of all models.

We had previously shown that regularization principles lead to approximation schemes that are equivalent to networks with one layer of hidden units, called regularization networks. In particular, standard smoothness functionals lead to a subclass of regularization networks, the well known radial basis functions approximation schemes. This paper shows that regularization networks encompass a much broader range of approximation schemes, including many of the popular general additive models and some of the neural networks. In particular, we introduce new classes of smoothness functionals that lead to different classes of basis functions. Additive splines as well as some tensor product splines can be obtained from appropriate classes of smoothness functionals. Furthermore, the same generalization that extends radial basis functions (RBF) to hyper basis functions (HBF) also leads from additive models to ridge approximation models, containing as special cases Breiman's hinge functions, some forms of projection pursuit regression, and several types of neural networks. We propose to use the term generalized regularization networks for this broad class of approximation schemes that follow from an extension of regularization. In the probabilistic interpretation of regularization, the different classes of basis functions correspond to different classes of prior probabilities on the approximating function spaces, and therefore to different types of smoothness assumptions. In summary, different multilayer networks with one hidden layer, which we collectively call generalized regularization networks, correspond to different classes of priors and associated smoothness functionals in a classical regularization principle. Three broad classes are (1) radial basis functions that can be generalized to hyper basis functions, (2) some tensor product splines, and (3) additive splines that can be generalized to schemes of the type of ridge approximation, hinge functions, and several perceptron-like neural networks with one hidden layer.

Time Efficient Optimal Tunning through Various Learning Rules in Unify Computing (TEOTLRUC)

Space-Division Multiple Access-OFDM based wireless communication has the potential to significantly increase the spectral efficiency, system performance and number of users. Research in the development of efficient multiuser signal detection algorithms for such systems has generated much interest in recent years. This research proposes MUD schemes utilizing three neural network (NN) models like Feed Forward NN (FNN) without any hidden layer, FNN with a single hidden layer and Recurrent Neural Network (RNN) as possible alternatives to existing Genetic Algorithm (GA) based Minimum Bit Error Rate (MBER) MUD. Further, these techniques offer low complexity. Extensive simulation based performance study is carried out to prove the efficiency of the proposed techniques. Better performance of FNN with hidden layer and RNN compared to FNN without hidden layer is clearly observed in rank deficient MIMO scenario, where number of users exceeds the number of receiving antennas. The Bit Error Rate (BER) and complexity analysis of the proposed neural MUD schemes are close to optimal ML and show improvement over the previous implemented technique.

In this study, it compares two different types of neural networks. First is a single layer neural network and other is multiple hidden layer neural network. For just comparisons it is ensured that both uses the same activation and output functions and have the same number of nodes and parameters. The networks are trained by the gradient descent algorithm to approximate linear and quadratic functions and examine their convergence properties. Here it is predicted as to how much a customer is willing to spend at a store using linear regression and different layers: single and multiple layers of neural network. Then comparison of linear regression with a single layer of neural network and single layer with multiple layers of network has done. Keywords : Comparisons, gradient descent algorithm, hidden layers, linear regression, neural networks Cite this Article Amin Deep Prakashbhai. Comparative Analysis of Neural Network and Linear Regression Applied to Black Friday Data. Current Trends in Signal Processing . 2019; 9(3): 1–4p.

We consider a convexification method for a class of nonsmooth monotone functions. Specifically, we prove that a semismooth monotone function can be converted into a convex function via certain convexification transformations. The results derived in this paper lay a theoretical base to extend the reach of convexification methods in monotone optimization to nonsmooth situations.

Coudert made a breakthrough in the two-level logic minimization problem with Ordered Binary Decision Diagrams (OBDDs for short) recently [3]. This paper discusses the relationship between the two OBDDs of a monotone function and its prime implicant set to clarify the complexity of this practically efficient method. We show that there exists a monotone function which has an O(n) size sum-of-products but cannot be represented by a polynomial size OBDD. In other words, we cannot obtain the OBDD of the prime implicant set of a monotone function in an output-size sensitive manner once we have constructed the OBDD of that function as in [3], in the worst case. A positive result is also given for a meaningful class of matroid functions.

The theory and implementation of extreme learning machine (ELM) prove that it is a simple, efficient, and accurate machine learning method. Compared with other single hidden layer feedforward neural network algorithms, ELM is characterized by simpler parameter selection rules, faster convergence speed, and less human intervention. The multiple hidden layer regularized extreme learning machine (MRELM) inherits these advantages of ELM and has higher prediction accuracy. In the MRELM model, the number of hidden layers is randomly initiated and fixed, and there is no iterative tuning process. However, the optimal number of hidden layers is the key factor to determine the generalization ability of MRELM. Given this situation, it is obviously unreasonable to determine this number by trial and random initialization. In this paper, an incremental MRELM training algorithm (FC-IMRELM) based on forced positive-definite Cholesky factorization is put forward to solve the network structure design problem of MRELM. First, an MRELM-based prediction model with one hidden layer is constructed, and then a new hidden layer is added to the prediction model in each training step until the generalization performance of the prediction model reaches its peak value. Thus, the optimal network structure of the prediction model is determined. In the training procedure, forced positive-definite Cholesky factorization is used to calculate the output weights of MRELM, which avoids the calculation of the inverse matrix and Moore-Penrose generalized inverse of matrix involved in the training process of hidden layer parameters. Therefore, FC-IMRELM prediction model can effectively reduce the computational cost brought by the process of increasing the number of hidden layers. Experiments on classification and regression problems indicate that the algorithm can be effectively used to determine the optimal network structure of MRELM, and the prediction model training by the algorithm has excellent performance in prediction accuracy and computational cost.

This paper examines properties and interrelations of several concepts of generalized concavity. It shows that the class of functions that are both “generalized concave” and “generalized convex” is closely related to the class of monotone functions of a single variable. After excluding a certain small class of exceptions, the paper shows that, for arbitrary (perhaps not differentiable) functions, concave implies pseudoconcave, pseudoconcave implies strictly quasiconcave, and strictly quasiconcave implies quasiconcave. Several results characterizing the extreme values of generalized concave functions are given.

Interpolatory estimates in monotone piecewise polynomial approximation

We study on-line gradient-descent learning in multilayer networks analytically and numerically. The training is based on randomly drawn inputs and their corresponding outputs as defined by a target rule. In the thermodynamic limit we derive deterministic differential equations for the order parameters of the problem which allow an exact calculation of the evolution of the generalization error. First we consider a single-layer perceptron with sigmoidal activation function learning a target rule defined by a network of the same architecture. For this model the generalization error decays exponentially with the number of training examples if the learning rate is sufficiently small. However, if the learning rate is increased above a critical value, perfect learning is no longer possible. For architectures with hidden layers and fixed hidden-to-output weights, such as the parity and the committee machine, we find additional effects related to the existence of symmetries in these problems.