Polynomial threshold functions, AC functions and spectrum norms

Polynomial threshold functions and Boolean threshold circuits

Hardness results for maximum agreement problems have close connections to hardness results for proper learning in computational learning theory. In this paper we prove two hardness results for the problem of finding a low degree polynomial threshold function (PTF) which has the maximum possible agreement with a given set of labeled examples in Rn x {−1, 1}. We prove that for any constants d ≥ 1, e > 0,• Assuming the Unique Games Conjecture, no polynomial-time algorithm can find a degree-d PTF that is consistent with a (1/2 + e) fraction of a given set of labeled examples in Rn x {−1, 1}, even if there exists a degree-d PTF that is consistent with a 1 − e fraction of the examples.• It is NP-hard to find a degree-2 PTF that is consistent with a (1/2 + e) fraction of a given set of labeled examples in Rn x {−1, 1}, even if there exists a half-space (degree-1 PTF) that is consistent with a 1 − e fraction of the examples.These results immediately imply the following hardness of learning results: (i) Assuming the Unique Games Conjecture, there is no better-than-trivial proper learning algorithm that agnostically learns degree-d PTFs under arbitrary distributions; (ii) There is no better-than-trivial learning algorithm that outputs degree-2 PTFs and agnostically learns halfspaces (i.e. degree-1 PTFs) under arbitrary distributions.

Restricted Boltzmann machines (RBMs) are used to build deep-belief networks that are widely thought to be one of the first effective deep learning neural networks. This paper studies the ability of RBMs to represent distributions over {0,1}n via softplus/hardplus RBM networks. It is shown that any distribution whose density depends on the number of 1's in their input can be approximated with arbitrarily high accuracy by an RBM of size 2n+1 , which improves the result of a previous study by reducing the size from n2 to 2n+1 . A theorem for representing partially symmetric Boolean functions by softplus RBM networks is established. Accordingly, the representational power of RBMs for distributions whose mass represents the Boolean functions is investigated in comparison with that of threshold circuits and polynomial threshold functions. It is shown that a distribution over {0,1}n whose mass represents a Boolean function can be computed with a given margin δ by an RBM of size and parameters bounded by polynomials in n , if and only if it can be computed by a depth-2 threshold circuit with size and parameters bounded by polynomials in n .

Unlike inflexible structure of soft and hard threshold function, a unified linear matrix form with flexible structure for threshold function is proposed. Based on the unified linear flexible structure threshold function, both supervised and unsupervised subband adaptive denoising frameworks are established. To determine flexible coefficients, a direct mean-square error (MSE) minimization is conducted in supervised denoising while Stein’s unbiased risk estimate as a MSE estimate is minimized in unsupervised denoising. The SURE rule requires no hypotheses or a priori knowledge about clean signals. Furthermore, we discuss conditions to obtain optimal coefficients for both supervised and unsupervised subband adaptive denoising frameworks. Applying an Odd-Term Reserving Polynomial (OTRP) function as concrete threshold function, simulations for polynomial order, denoising performance, and noise effect are conducted. Proper polynomial order and noise effect are analyzed. Both proposed methods are compared with soft and hard based denoising technologies—VisuShrink, SureShrink, MiniMaxShrink, and BayesShrink—in denoising performance simulation. Results show that the proposed approaches perform better in both MSE and signal-to-noise ratio (SNR) sense.

The degree-d Chow parameters of a Boolean function are its degree at most d Fourier coefficients. It is well-known that degree-d Chow parameters uniquely characterize degree-d polynomial threshold functions (PTFs) within the space of all bounded functions. In this paper, we prove a robust version of this theorem: For f any Boolean degree-d PTF and g any bounded function, if the degree-d Chow parameters of f are close to the degree-d Chow parameters of g in l2-norm, then f is close to g in l1-distance. Notably, our bound relating the two distances is independent of the dimension. That is, we show that Boolean degree-d PTFs are robustly identifiable from their degree-d Chow parameters. No non-trivial bound was previously known for d >1. Our robust identifiability result gives the following algorithmic applications: First, we show that Boolean degree-d PTFs can be efficiently approximately reconstructed from approximations to their degree-d Chow parameters. This immediately implies that degree-d PTFs are efficiently learnable in the uniform distribution d-RFA model. As a byproduct of our approach, we also obtain the first low integer-weight approximations of degree-d PTFs, for d>1. As our second application, our robust identifiability result gives the first efficient algorithm, with dimension-independent error guarantees, for malicious learning of Boolean degree-d PTFs under the uniform distribution. The proof of our robust identifiability result involves several new technical ingredients, including the following structural result for degree-d multivariate polynomials with very poor anti-concentration: If p is a degree-d polynomial where p(x) is very close to 0 on a large number of points in { ± 1 }n, then there exists a degree-d hypersurface that exactly passes though almost all of these points. We leverage this structural result to show that if the degree-d Chow distance between f and g is small, then we can find many degree-d polynomials that vanish on their disagreement region, and in particular enough that forces the l1-distance between f and g to also be small. To implement this proof strategy, we require additional technical ideas. In particular, in the d=2 case we show that for any large vector space of degree-2 polynomials with a large number of common zeroes, there exists a linear function that vanishes on almost all of these zeroes. The degree-d degree generalization of this statement is significantly more complex, and can be viewed as an effective version of Hilbert’s Basis Theorem for our setting.

This paper investigates the optimum OBDD representation problem based on two classes of Boolean functions. The first class is defined by OBDDs, in which the number of non-terminal nodes is equal to the number of input variables. We refer to such OBDDs and their corresponding Boolean functions as thin OBDDs and thin Boolean functions. The second class is the thin factored Boolean functions, which is defined by factored forms with each variable appearing exactly once. Many interesting properties of these two classes of functions are presented. Based on which, a revised dynamic shortest cube first OBDD variable ordering algorithm is developed. This algorithm is shown to be optimum for thin factored Boolean functions.

The number and probability of canalizing functions

The sensitivity of a Boolean function f is the maximum, over all inputs x, of the number of sensitive coordinates of x (namely the number of Hamming neighbors of x with different f-value). The well-known sensitivity conjecture of Nisan (see also Nisan and Szegedy) states that every sensitivity-s Boolean function can be computed by a polynomial over the reals of degree s^{O(1)}. The best known upper bounds on degree, however, are exponential rather than polynomial in s. Our main result is an approximate version of the conjecture: every Boolean function with sensitivity s can be eps-approximated (in l_2) by a polynomial whose degree is s * polylog(1/eps). This is the first improvement on the folklore bound of s/eps. We prove this via a new "switching lemma for low-sensitivity functions" which establishes that a random restriction of a low-sensitivity function is very likely to have low decision tree depth. This is analogous to the well-known switching lemma for AC^0 circuits. Our proof analyzes the combinatorial structure of the graph G_f of sensitive edges of a Boolean function f. Understanding the structure of this graph is of independent interest as a means of understanding Boolean functions. We propose several new complexity measures for Boolean functions based on this graph, including tree sensitivity and component dimension, which may be viewed as relaxations of worst-case sensitivity, and we introduce some new techniques, such as proper walks and shifting, to analyze these measures. We use these notions to show that the graph of a function of full degree must be sufficiently complex, and that random restrictions of low-sensitivity functions are unlikely to lead to such complex graphs. We postulate a robust analogue of the sensitivity conjecture: if most inputs to a Boolean function f have low sensitivity, then most of the Fourier mass of f is concentrated on small subsets. We prove a lower bound on tree sensitivity in terms of decision tree depth, and show that a polynomial strengthening of this lower bound implies the robust conjecture. We feel that studying the graph G_f is interesting in its own right, and we hope that some of the notions and techniques we introduce in this work will be of use in its further study.

This special issue of the VLSI DESIGN: an International Journal of Custom-Chip Design, Simulation and Testing is devoted to spectral techniques, new types of decision trees and diagrams, their applications, efficient ways of their calculation and mutual relations between spectral techniques and classical decision diagrams. Manipulations and calculations with discrete functions are fundamental tasks in many areas of Computer Science and Engineering. Many problems in digital system design, simulation and testing can be expressed as a sequence of operations on discrete functions. The performance of Computer-Aided Design systems used in solving various problems in this area strongly depends on the efficiency of representations of discrete functions. Decision Diagrams (DDs) have proved very convenient data structure for discrete function representations, permitting manipulations and calculations with large functions efficiently in terms of time and space. In many applications, as for example those involving large matrices, conventional algorithms are significantly improved by using DDs. In logic design, such applications relate to basic problems of the design, verification, simulation and testing logical networks. Besides Boolean functions, some other types of discrete representations, such as multi-valued functions, cube sets, EXOR-based spectral and probabilistic formulas have to be dealt with. Hence the numbers of different decision trees and diagrams have been introduced for these various applications and representations. The spectral or polynomial representation of Boolean function is obtained by expressing it in terms of an orthogonal basis. The most popular basis is formed by discrete Walsh functions. Note that the spectral and the polynomial representation of a Boolean function are equivalent. The discrete orthogonal functions are referred to as the basis functions when the spectral or abstract harmonic terminology is used and as monomials when the polynomial terminology is used. Such polynomial representations have extensive applications in coding theory and cryptography as well as design of nonlinear filters. Spectral techniques based on Walsh, Haar, Arithmetic and Reed–Muller transforms have been used in digital logic design for more than 30 years. These techniques have been used for Boolean function classification, disjoint decomposition, identification of symmetries, parallel and serial linear decomposition, spectral translation synthesis (extraction of linear preand post-filters), multiplexer synthesis, threshold logic synthesis, state assignment and testing, and evaluation of logic complexity. The renewed interest in applications of spectral methods in design of VLSI digital circuits is caused by their excellent design for testability properties and in their efficiency in Boolean mapping problems. Furthermore, the practical applicability of these techniques was greatly improved with the development of efficient methods that allow calculating and operating on spectra of Boolean functions directly from reduced representations of such functions in the form of arrays of cubes or decision diagrams. Following from these recent developments, techniques have been presented for efficient spectral translation to decompose a circuit into a cascade of two sub circuits: linear block composed of EXOR gates fed by the primary input of the overall circuit. Such linear decomposition drastically simplifies the synthesis task. The choice of suitable linear transformation is based on a complexity measure assigned to each Boolean function that is heuristically related to the complexity of the final circuit implementation. To avoid the experimental costs of computing the Walsh transform, Binary Decision Diagrams (BDDs) based techniques are used. Recently the idea of linear transformations is applied to decision diagrams. By combining powerful spectral linear techniques with variable reordering techniques, it is possible to synthesize large Boolean functions with standard Computer-Aided Design tools that fail otherwise. The areas addressed by this special issue span spectral transforms, their applications and efficient calculation methods through decision diagrams. New spectral decision diagrams are also emerging that support larger data structures allowing more complex discrete representations to be implemented that was previously impossible.

International audience

Boolean functions play a central role in security applications because they constitute one of the basic primitives for modern cryptographic services. In the last decades, research on Boolean functions has been boosted due to the importance of security in many diverse public systems relying on such technology. A main focus is to find Boolean functions with specific properties. An open problem in this context is to find a balanced Boolean function with an 8-bit input and nonlinearity 118. Theoretically, such a function has been shown to exist, but it has not been found yet. In this work we focus on specific classes of Boolean functions, and analyze the landscape of results obtained by integrating algebraic and evolutionary computation (EC) based approaches. Results indicate that combinations of these approaches give better results although not reaching 118 nonlinearity.

Two Boolean functions are affine equivalent if one can be obtained from the other by applying an affine transformation to the input variables. For a long time, there have been efforts to investigate the affine equivalence of Boolean functions. Due to the complexity of the general problem, only affine equivalence under certain groups of permutations is usually considered. Boolean functions which are invariant under the action of cyclic rotation of the input variables are known as rotation symmetric (RS) Boolean functions. Due to their speed of computation and the prospect of being good cryptographic Boolean functions, this class of Boolean functions has received a lot of attention from cryptographic researchers. In this paper, we study affine equivalence for the simplest rotation symmetric Boolean functions, called MRS functions, which are generated by the cyclic permutations of a single monomial. Using Pólya’s enumeration theorem, we compute the number of equivalence classes, under certain large groups of permutations, for these MRS functions in any number

It is a difficult task to compute the r-th order nonlinearity of a given function with algebraic degree strictly greater than r > 1. Though lower bounds on the second order nonlinearity are known only for a few particular functions, the majority of which are cubic. We investigate lower bounds on the second order nonlinearity of cubic Boolean functions [Formula: see text], where [Formula: see text], dl = 2il + 2jl + 1, m, il and jl are positive integers, n > il > jl. Furthermore, for a class of Boolean functions [Formula: see text] we deduce a tighter lower bound on the second order nonlinearity of the functions, where [Formula: see text], dl = 2ilγ + 2jlγ + 1, il > jl and γ ≠ 1 is a positive integer such that gcd(n,γ) = 1. Lower bounds on the second order nonlinearity of cubic monomial Boolean functions, represented by fμ(x) = Tr(μx2i+2j+1), [Formula: see text], i and j are positive integers such that i > j, were obtained by Gode and Gangopadhvay in 2009. In this paper, we first extend the results of Gode and Gangopadhvay from monomial Boolean functions to Boolean functions with more trace terms. We further generalize and improve the results to a wider range of n. Our bounds are better than those of Gode and Gangopadhvay for monomial functions fμ(x). Especially, our lower bounds on the second order nonlinearity of some Boolean functions F(x) are better than the existing ones.

We show that there is a better-than-brute-force algorithm that, when given a small constant-depth Boolean circuit C made up of gates that compute constant-degree Polynomial Threshold functions or PTFs (i.e., Boolean functions that compute signs of constant-degree polynomials), counts the number of satisfying assignments to C in significantly better than brute-force time. Formally, for any constants d, k, there is an \(\varepsilon > 0\) such that the zero-error randomized algorithm counts the number of satisfying assignments to a given depth-d circuit C made up of k-PTF gates such that C has at most \(n^{1+\varepsilon }\) many wires. The algorithm runs in time \(2^{n-n^{\Omega (\varepsilon )}}.\) Before our result, no algorithm for beating brute-force search was known for counting the number of satisfying assignments even for a single degree-k PTF (which is a depth-1 circuit with linearly many wires).We give two different algorithms for the case of a single PTF. The first uses a learning algorithm for learning degree-1 PTFs (or Linear Threshold Functions) using comparison queries due to Kane, Lovett and Moran (STOC 2018), and the second uses a proof of Hofmeister (COCOON 1996) for converting a degree-1 PTF to a depth-two threshold circuit with small weights. We show that both these ideas fit nicely into a memoization approach that yields the #SAT algorithms.

We study the complexity of approximating Boolean functions with DNFs and other depth-2 circuits, exploring two main directions: universal bounds on the approximability of all Boolean functions, and the approximability of the parity function. In the first direction, our main positive results are the first non-trivial universal upper bounds on appropriability by DNFs: : Every Boolean function can be ε-approximated by a DNF of size O <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ε</sub> (2 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</sup> / log n). : Every Boolean function can be ε-approximated by a DNF of width c <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ε</sub> n, where c <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ε</sub> <; 1. Our techniques extend broadly to give strong universal upper bounds on approximability by various depth-2 circuits that generalize DNFs, including the intersection of halfspaces, low-degree PTFs, and unate functions. We show that the parameters of our constructions come close to matching the information-theoretic inapproximability of a random function. In the second direction our main positive result is the construction of an explicit DNF that approximates the parity function: : PAR <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</sub> can be ε-approximated by a DNF of size 2 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">(1-2ε)n</sup> and width (1 - 2ε)n. Using Fourier analytic tools we show that our construction is essentially optimal not just within the class of DNFs, but also within the far more expressive classes of the intersection of halfspaces and intersection of unate functions.