A class of coverage growth functions and its practical application

Preparing a data set for analysis is generally the most time consuming task in a data mining project, requiring many complex SQL queries, joining tables and aggregating columns. Existing SQL aggregations have limitations to prepare data sets because they return one column per aggregated group. In general, a significant manual effort is required to build data sets, where a horizontal layout is required. We propose simple, yet powerful, methods to generate SQL code to return aggregated columns in a horizontal tabular layout, returning a set of numbers instead of one number per row. This new class of functions is called horizontal aggregations. Horizontal aggregations build data sets with a horizontal denormalized layout (e.g. point-dimension, observation-variable, instance-feature), which is the standard layout required by most data mining algorithms. We propose three fundamental methods to evaluate horizontal aggregations: CASE: Exploiting the programming CASE construct; SPJ: Based on standard relational algebra operators (SPJ queries); PIVOT: Using the PIVOT operator, which is offered by some DBMSs. Experiments with large tables compare the proposed query evaluation methods. Our CASE method has similar speed to the PIVOT operator and it is much faster than the SPJ method. In general, the CASE and PIVOT methods exhibit linear scalability, where as the SPJ method does not. In a relational database, especially with normalized tables, a significant effort is required to prepare a summary data set that can be used as input for a data mining or statistical algorithm. Most algorithms require as input a data set with a horizontal layout, with several Records and one variable or dimension per column. That is the case with models like clustering, classification, regression and PCA; consult. Each research discipline uses different terminology to describe the data set. In data mining the common terms are point-dimension. Statistics literature generally uses observation-variable. Machine learning research uses instance-feature. This article introduces a new class of aggregate functions that can be used to build data sets in a horizontal layout (denormalized with aggregations), automating SQL query writing and extending SQL capabilities. We show evaluating horizontal aggregations is a challenging and interesting problem and we introduced alternative methods and optimizations for their efficient evaluation. II. MOTIVATION As mentioned above, building a suitable data set for data mining purposes is a time- consuming task. This task generally requires writing long SQL statements or customizing SQL Code if it is automatically generated by some tool. There are two main ingredients in such SQL code: joins and aggregations; we focus on the second one. The most widely- known aggregation is the sum of a column over groups of rows. Some other aggregations return the average, maximum, minimum or row count over groups of rows. There exist many aggregations functions and operators in SQL. Unfortunately, all these aggregations have limitations to build data sets for data mining purposes. The main reason is that, in general, data sets that are stored in a relational database (or a data warehouse) come

Machine learning can provide powerful tools for the design and analysis of multi-agent systems, as well as a novel lens on important questions in the area. In this talk I will focus on the use of machine learning for understanding submodular functions, an important class of discrete functions that model laws of diminishing returns and enjoy many important applications in multi-agent settings. For example, submodular functions are commonly used to model valuation functions for bidders in auctions, the influence of various subsets of agents in social networks, and the benefits of performing different actions in a variety of situations. Traditionally it is assumed that these functions are known to the decision maker; however, for large scale systems, in the age of big data, it is often the case they must be learned from observations. In this talk, I will discuss a recent line of work on studying the learnability of submodular functions, and highlight its applications to the analysis of multi-agent systems. I will discuss both general algorithms for learning such functions, as well as even better guarantees that can be achieved for important classes appearing in multi-agent scenarios that exhibit additional structure. These classes include: probabilistic coverage functions that can be used to model the influence function in classic models of information diffusion in networks; functions with bounded complexity used in modeling bidder valuation functions in auctions, including XOS and gross-substitutes; and classes of functions appearing in cooperative game theory for expressing the values of various types of coalitions. I will additionally discuss a large scale application of our algorithms for learning the influence functions in social networks, that significantly outperforms existing approaches empirically in both synthetic and real world data.

The theory of evolutionary computation (EC) has experienced rapid and productive growth in recent years. New proof techniques and novel theoretical frameworks have allowed advances in our understanding of the processes and structures inherent in evolutionary optimization. As a result, the frontiers of our knowledge have been expanded further than ever before. Some recent trends in this field, which are covered in this issue, include developments in the understanding of the behavior of evolutionary algorithms (EAs) in dynamic environments rather than just static settings, a theoretical appreciation of the advantages arising from the parallelization of evolutionary algorithms through a greater comprehension of the underlying dynamics, and an understanding of algorithm behavior on broad function classes, including -hard problems.The primary goal of this special issue is to provide extended and polished versions of diverse examples of the best theoretical work presented at conferences in 2014, and to serve as a forum for researchers to advance the theoretical understanding of evolutionary computation methods. The papers included in this special issue span a plurality of topics and offer the reader a cross section of recent outstanding work in EC theory.In dynamic optimization the objective function changes over time, and optimization algorithms face the additional challenge of tracking these changes to be successful. The article “Analysis of Randomised Search Heuristics for Dynamic Optimisation,” by Thomas Jansen and Christine Zarges, presents a novel analytical framework for the analysis of randomized search heuristics on dynamic problems inspired by the fixed-budget computations perspective. The authors introduce a new interesting class of bi-stable dynamic functions where the optimum oscillates between two complementary strings, and apply the framework to analyze and compare the performance of evolutionary algorithms and artificial immune systems on the novel class of functions.Over three decades ago, László Lovász observed that in discrete optimization, submodularity is the counterpart to convexity. However, in contrast to the focus on convex functions in continuous evolutionary optimization, so far submodular functions have received comparatively little attention from EC theoreticians studying discrete functions. The article “Maximizing Submodular Functions under Matroid Constraints by Evolutionary Algorithms,” by Tobias Friedrich and Frank Neumann, addresses this gap by analyzing the performance of evolutionary algorithms on different classes of submodular functions. The maximization of submodular functions is -hard in general, and the authors present several approximation results for monotone submodular and nonmonotone symmetric submodular functions under different kinds of matroid constraints.The idea behind parallel evolutionary algorithms is to evolve multiple subpopulations in parallel and allow interprocess communication at given time intervals. During these migration phases, fractions of each subpopulation can be shared among the subpopulations. There is very little understanding of how the migration frequency affects algorithmic performance, so setting the migration interval parameter appropriately may be difficult. In the article “Design and Analysis of Schemes for Adapting Migration Intervals in Parallel Evolutionary Algorithms,” Andrea Mambrini and Dirk Sudholt propose two schemes to automatically adapt the migration interval of parallel EAs during execution and provide a rigorous analytical framework that yields upper bounds on the expected runtime and expected communication effort of the parallel EAs with different migration topologies for various function classes.In the field of genetic programming, a long-standing open problem is how to address the issue of bloat, the emergence during evolution of solution elements that do not contribute significantly or at all to program fitness or semantics but increase program complexity. The article “On the Performance of Different Genetic Programming Approaches for the SORTING Problem,” by Markus Wagner, Frank Neumann, and Tommaso Urli, tackles the issue of bloat control in the context of sorting. As a basis for their study, they consider program trees and use some measure of sortedness of an in-order traversal to evaluate their fitness. The authors investigate single- and multiobjective variants of genetic programming algorithms with and without bloat control mechanisms, give rigorous upper bounds on their running times, and complement the study with experiments.The topic of constraint handling has recently gained traction in the continuous domain. The article “Markov Chain Analysis of Cumulative Step-Size Adaptation on a Linear Constrained Problem,” by Alexandre Chotard, Anne Auger, and Nikolaus Hansen, presents a rigorous analysis of a (1,)-Evolution Strategy using resampling on a linear function with a linear constraint. The authors prove the previously assumed property that a Markov chain, describing the behavior of the algorithm, exhibits stability in cases with constant step-size and with cumulative step-size adaptation with cumulation parameter equal to 1. This property characterizes the divergence of the algorithm with constant step-size and the geometric divergence or convergence with step-size adaptation, implying fast convergence of Monte Carlo simulations of the divergence rate.In their seminal 2006 paper, Droste, Jansen, and Wegener introduced the concept of black box complexity in order to establish a complexity theory for general-purpose randomized search heuristics. Generally speaking, the black box complexity of a problem is a lower bound on the number of function evaluations needed by any black box algorithm to solve it. Recently, black box models have been refined and developed extensively, allowing the hardness of objective function classes to be more precisely understood. The article “Unbiased Black Box Complexities of Jump Functions,” by Benjamin Doerr, Carola Doerr, and Timo Kötzing, analyzes the unbiased black box complexity of a Jump function class where in each function a local optimum is k bits in distance from the global optimum. The authors provide polynomial upper bounds on the black box complexity of Jump for different sizes of the gap. In particular, they show that an unbiased polynomial-time black box algorithm exists even when almost all of the search space is a plateau of constant fitness.The guest editors would like to thank the authors for their contributions, the referees for their careful reviewing and constructive comments, and the editor-in-chief, Hans-Georg Beyer, for his support in preparing this special issue.

This special issue of the VLSI DESIGN: an International Journal of Custom-Chip Design, Simulation and Testing is devoted to spectral techniques, new types of decision trees and diagrams, their applications, efficient ways of their calculation and mutual relations between spectral techniques and classical decision diagrams. Manipulations and calculations with discrete functions are fundamental tasks in many areas of Computer Science and Engineering. Many problems in digital system design, simulation and testing can be expressed as a sequence of operations on discrete functions. The performance of Computer-Aided Design systems used in solving various problems in this area strongly depends on the efficiency of representations of discrete functions. Decision Diagrams (DDs) have proved very convenient data structure for discrete function representations, permitting manipulations and calculations with large functions efficiently in terms of time and space. In many applications, as for example those involving large matrices, conventional algorithms are significantly improved by using DDs. In logic design, such applications relate to basic problems of the design, verification, simulation and testing logical networks. Besides Boolean functions, some other types of discrete representations, such as multi-valued functions, cube sets, EXOR-based spectral and probabilistic formulas have to be dealt with. Hence the numbers of different decision trees and diagrams have been introduced for these various applications and representations. The spectral or polynomial representation of Boolean function is obtained by expressing it in terms of an orthogonal basis. The most popular basis is formed by discrete Walsh functions. Note that the spectral and the polynomial representation of a Boolean function are equivalent. The discrete orthogonal functions are referred to as the basis functions when the spectral or abstract harmonic terminology is used and as monomials when the polynomial terminology is used. Such polynomial representations have extensive applications in coding theory and cryptography as well as design of nonlinear filters. Spectral techniques based on Walsh, Haar, Arithmetic and Reed–Muller transforms have been used in digital logic design for more than 30 years. These techniques have been used for Boolean function classification, disjoint decomposition, identification of symmetries, parallel and serial linear decomposition, spectral translation synthesis (extraction of linear preand post-filters), multiplexer synthesis, threshold logic synthesis, state assignment and testing, and evaluation of logic complexity. The renewed interest in applications of spectral methods in design of VLSI digital circuits is caused by their excellent design for testability properties and in their efficiency in Boolean mapping problems. Furthermore, the practical applicability of these techniques was greatly improved with the development of efficient methods that allow calculating and operating on spectra of Boolean functions directly from reduced representations of such functions in the form of arrays of cubes or decision diagrams. Following from these recent developments, techniques have been presented for efficient spectral translation to decompose a circuit into a cascade of two sub circuits: linear block composed of EXOR gates fed by the primary input of the overall circuit. Such linear decomposition drastically simplifies the synthesis task. The choice of suitable linear transformation is based on a complexity measure assigned to each Boolean function that is heuristically related to the complexity of the final circuit implementation. To avoid the experimental costs of computing the Walsh transform, Binary Decision Diagrams (BDDs) based techniques are used. Recently the idea of linear transformations is applied to decision diagrams. By combining powerful spectral linear techniques with variable reordering techniques, it is possible to synthesize large Boolean functions with standard Computer-Aided Design tools that fail otherwise. The areas addressed by this special issue span spectral transforms, their applications and efficient calculation methods through decision diagrams. New spectral decision diagrams are also emerging that support larger data structures allowing more complex discrete representations to be implemented that was previously impossible.

We demonstrate that the Plancherel transform for Type-I groups provides one with a natural, unified perspective for the generalized continuous wavelet transform, on the one hand, and for a class of Wigner functions, on the other. We first prove that a Plancherel inversion formula, well known for Bruhat functions on the group, holds for a much larger class of functions. This result allows us to view the wavelet transform as essentially the inverse Plancherel transform. The wavelet transform of a signal is an L2-function on an appropriately chosen group while the Wigner function is defined on a coadjoint orbit of the group and serves as an alternative characterization of the signal, which is often used in practical applications. The Plancherel transform maps L2-functions on a group unitarily to fields of Hilbert-Schmidt operators, indexed by unitary irreducible representations of the group. The wavelet transform can essentially be looked upon as a restricted inverse Plancherel transform, while Wigner functions are modified Fourier transforms of inverse Plancherel transforms, usually restricted to a subset of the unitary dual of the group. Some known results on both Wigner functions and wavelet transforms, appearing in the literature from very different perspectives, are naturally unified within our approach. Explicit computations on a number of groups illustrate the theory.

This article explores the algorithms for solving individual classes of discrete extremal problems to find the exact optimum. When obtaining algorithms, the procedures for decoding and finding the maximum upper zero of discrete monotonic functions are used. Formulations of the tasks of decoding and searching for the maximum upper zero of discrete monotonic functions are formulated. To solve the problems of decoding and searching for the maximum upper zero of individual classes of 3-digit monotonic functions, estimates are given for the complexity of deciphering monotonic functions in a class.

The dune-functions dune module introduces a new programmer interface for discrete and non-discrete functions. Unlike the previous interfaces considered in the existing dune modules, it is based on overloading operator(), and returning values by-value. This makes user code much more readable, and allows the incorporation of newer C++ features such as lambda expressions. Run-time polymorphism is implemented not by inheritance, but by type erasure, generalizing the ideas of the std::function class from the C++11 standard library. We describe the new interface, show its possibilities, and measure the performance impact of type erasure and return-by-value.

It is almost always easier to find an accurate-but-complex model than an accurate-yet-simple model. Finding optimal, sparse, accurate models of various forms (linear models with integer coefficients, decision sets, rule lists, decision trees) is generally NP-hard. We often do not know whether the search for a simpler model will be worthwhile, and thus we do not go to the trouble of searching for one. In this work, we ask an important practical question: can accurate-yet-simple models be proven to exist, or shown likely to exist, before explicitly searching for them? We hypothesize that there is an important reason that simple-yet-accurate models often do exist. This hypothesis is that the size of the Rashomon set is often large, where the Rashomon set is the set of almost-equally-accurate models from a function class. If the Rashomon set is large, it contains numerous accurate models, and perhaps at least one of them is the simple model we desire. In this work, we formally present the Rashomon ratio as a new gauge of simplicity for a learning problem, depending on a function class and a data set. The Rashomon ratio is the ratio of the volume of the set of accurate models to the volume of the hypothesis space, and it is different from standard complexity measures from statistical learning theory. Insight from studying the Rashomon ratio provides an easy way to check whether a simpler model might exist for a problem before finding it, namely whether several different machine learning methods achieve similar performance on the data. In that sense, the Rashomon ratio is a powerful tool for understanding why and when an accurate-yet-simple model might exist. If, as we hypothesize in this work, many real-world data sets admit large Rashomon sets, the implications are vast: it means that simple or interpretable models may often be used for high-stakes decisions without losing accuracy.

The linearized Bregman iterations (LBreI) and its variants have received considerable attention in signal/image processing and compressed sensing. Recently, LBreI has been extended to a larger class of nonconvex functions, along with several theoretical issues left for further investigation. In particular, the Lipschitz gradient continuity assumption precludes its use in many practical applications. In this study, we propose a generalized algorithmic framework to unify LBreI-type methods. Our main discovery is that the Lipschitz gradient continuity assumption can be replaced by a Lipschitz-like convexity condition in both convex and nonconvex cases. As a by-product, a class of bilevel optimization problems can be solved in the proposed framework, which extends the main result made by Cai et al. [Math. Comp. 78 (2009), pp. 2127–2136]. At last, provably convergent iterative schemes on modified linear/quadratic inverse problems illustrate our finding.

k-nearest neighbors (k-NN) voting rules are an effective tool in countless many machine learning techniques. In spite of its simplicity, k-NN classification is very attractive to practitioners, as it has shown very good performances in practical applications. However, it suffers from various drawbacks, like sensitivity to “noisy” prototypes and poor generalization properties when dealing with sparse, high-dimensional data. In this paper, we tackle the k-NN classification problem at its core by providing a novel k-NN boosting approach. We propose a Universal Nearest Neighbors (UNN) algorithm, which induces a leveraged k-NN rule by globally minimizing a surrogate risk upper bounding the empirical misclassification rate over training data. Interestingly, this surrogate risk can be arbitrary chosen from a class of Bregman loss functions, including the familiar exponential, logistic and squared losses. Furthermore, we show that UNN allows to efficiently filter a dataset of instances by keeping only a small fraction of data. Experimental results on the synthetic Ripley's dataset show that such a filtering strategy is able to reject “noisy” examples, and yields a classification error close to the optimal Bayes error. Experiments on standard UCI datasets show significant improvements over the current state of the art.

The aim of this paper is to provide a large class of scaling functions for which the convergence analysis for the Galerkin method developed in [9] is applicable, whereas in that paper the only scaling functions considered for practical applications are B-splines and a few of the orthonormal Daubechies scaling functions. The functions considered here, were recently introduced in [12] where it was proved that they satisfy many properties making them interesting for the applications. In particular, here we show that the use of these functions has some advantages with respect to other basis functions.

<i>Circulation: Heart Failure</i> Editors’ Picks

: This paper introduces design methods for numeric function generators (NFGs) using decision diagrams. NFGs are hardware accelerators to compute values of numeric functions such as trigonometric, logarithmic, square root, and reciprocal functions. Most existing design methods for NFGs are intended only for a specific class of numeric functions. However, by using decision diagrams for discrete functions (i.e., word-level decision diagrams), we can systematically design fast and compact NFGs for a larger class of functions. This paper shows three design methods for NFGs using 1) multi-terminal binary decision diagrams (MTBDDs), 2) binary moment diagrams (BMDs), and 3) edge-valued binary decision diagrams (EVBDDs).

Discrete piecewise linear functions

This paper gives convergence properties and applications of the discrete analogs of reproducing kernels for various families of harmonic functions. From these results information is obtained on the solution of interpolation problems, the convergence of the discrete Neumann's function, and the solution to problems involving the discrete biharmonic operator.