Interactive Proofs for Verifying Machine Learning.

This paper proves that the computational power of quantum interactive proof systems, with a double-exponentially small gap in acceptance probability between the completeness and soundness cases, is precisely characterized by EXP, the class of problems solvable in exponential time by deterministic Turing machines. This fact, and our proof of it, has implications concerning quantum and classical interactive proof systems in the setting of unbounded error that include the following: * Quantum interactive proof systems are strictly more powerful than their classical counterparts in the unbounded-error setting unless PSPACE=EXP, as even unbounded error classical interactive proof systems can be simulated in PSPACE. * The recent proof of Jain, Ji, Upadhyay, and Watrous (STOC 2010) establishing QIP=PSPACE relies heavily on the fact that the quantum interactive proof systems defining the class QIP have bounded error. Our result implies that some nontrivial assumption on the error bounds for quantum interactive proofs is unavoidable to establish this result (unless PSPACE=EXP). * To prove our result, we give a quantum interactive proof system for EXP with perfect completeness and soundness error 1-2^{-2^poly}, for which the soundness error bound is provably tight. This establishes another respect in which quantum and classical interactive proof systems differ, because such a bound cannot hold for any classical interactive proof system: distinct acceptance probabilities for classical interactive proof systems must be separated by a gap that is at least (single-)exponentially small. We also study the computational power of a few other related unbounded-error complexity classes.

Separating models of learning with faulty teachers

Hardness of approximate two-level logic minimization and PAC learning with membership queries

A new research frontier in AI and data mining seeks to develop methods to automatically discover relevant variables among many irrelevant ones. In this paper, we present four algorithms that output such crucial variables in PAC model with membership queries. The first algorithm executes the task under any unknown distribution by measuring the distance between virtual and real targets. The second algorithm exhausts virtual version space under an arbitrary distribution. The third algorithm exhausts universal set under the uniform distribution. The fourth algorithm measures influence of variables under the uniform distribution. Knowing the number r of relevant variables, the first algorithm runs in almost linear time for r. The second and the third ones use less membership queries than the first one, but runs in time exponential for r. The fourth one enumerates highly influential variables in quadratic time for r.

Conjunctions of Unate DNF Formulas: Learning and Structure

A new smoothness quantification in kernel density estimation

On the complexity of interactive proofs with bounded communication

The total influence of a function is a central notion in analysis of Boolean functions, and characterizing functions that have small total influence is one of the most fundamental questions associated with it. The KKL theorem and the Friedgut junta theorem give a strong characterization of such functions whenever the bound on the total influence is \(o(\log n)\). However, both results become useless when the total influence of the function is \(\omega (\log n)\). The only case in which this logarithmic barrier has been broken for an interesting class of functions was proved by Bourgain and Kalai, who focused on functions that are symmetric under large enough subgroups of \(S_n\). In this paper, we build and improve on the techniques of the Bourgain–Kalai paper and establish new concentration results on the Fourier spectrum of Boolean functions with small total influence. Our results include: 1. A quantitative improvement of the Bourgain–Kalai result regarding the total influence of functions that are transitively symmetric. 2. A slightly weaker version of the Fourier-Entropy Conjecture of Friedgut and Kalai. Our result establishes new bounds on the Fourier entropy of a Boolean function f, as well as stronger bounds on the Fourier entropy of low-degree parts of f. In particular, it implies that the Fourier spectrum of a constant variance, Boolean function f is concentrated on \(2^{O(I[f]\log I[f])}\) characters, improving an earlier result of Friedgut. Removing the \(\log I[f]\) factor would essentially resolve the Fourier-Entropy Conjecture, as well as settle a conjecture of Mansour regarding the Fourier spectrum of polynomial size DNF formulas. Our concentration result for the Fourier spectrum of functions with small total influence also has new implications in learning theory. More specifically, we conclude that the class of functions whose total influence is at most K is agnostically learnable in time \(2^{O(K\log K)}\) using membership queries. Thus, the class of functions with total influence \(O(\log n/\log \log n)\) is agnostically learnable in \(\mathsf{poly}(n)\) time.

New results on the power of space-bounded probabilistic game automata are presented. Space-bounded analogues of Arthur-Merlin games and interactive proof systems, which are denoted by BC and BP respectively, for Bounded random game automata with Complete and Partial information, are considered. The main results are that BC-SPACE(s(n))⊆BP-SPACE(log s(n)) for s(n)=Ω(n) U c DTIME (2 cs(n) )=BC-SPACE(s(n)) for s(n)=Ω(log n), where s(n) is space constructible. A consequence of these results is that any language recognizable in deterministic exponential time has an interactive proof that uses only logarithmic space. The power of games with simultaneous time and space bounds is also studied, and it is shown that any language in BC-TIME(t(n)) has an interactive proof that uses time polynomial in t(n) but space only logarithmic in t(n)

Constant-space quantum interactive proofs against multiple provers

Suppose that a polynomial-time mixed-state circuit, described as a sequence of local unitary interactions followed by a partial trace, generates a state shared between two parties. One might then wonder, does this circuit produce a state that is separable or entangled? Here, we give evidence that it is computationally hard to decide the answer to this question, even if one has access to the power of computation. We begin by exhibiting a two-message interactive proof system that can decide the answer to a promise version of the question. We then prove that the promise problem is hard for the class of promise problems with quantum statistical zero knowledge (QSZK) proof systems by demonstrating a polynomial-time Karp reduction from the QSZK-complete promise problem quantum state distinguishability to our separability problem. By exploiting Knill's efficient encoding of a matrix description of a state into a description of a circuit to generate the state, we can show that our promise problem is NP-hard with respect to Cook reductions. Thus, the separability problem (as phrased above) constitutes the first nontrivial promise problem decidable by a two-message interactive proof system while being hard for both NP and QSZK. We also consider a variant of the problem, in which a given polynomial-time mixed-state circuit accepts a state as input, and the question is to decide if there is an input to this circuit which makes its output separable across some bipartite cut. We prove that this problem is a complete promise problem for the class QIP of problems decidable by interactive proof systems. Finally, we show that a two-message interactive proof system can also decide a multipartite generalization of the separability problem.

We extend the line of research initiated by Fortnow and Klivans [6] that studies the relationship between efficient learning algorithms and circuit lower bounds. In [6], it was shown that if a Boolean circuit class \(\mathcal{C}\) has an efficient deterministic exact learning algorithm, (i.e. an algorithm that uses membership and equivalence queries) then \(\mathsf{EXP}^{\mathsf{NP}} \not \subseteq \mathsf{P/poly}[\mathcal{C}]\). Recently, in [14] EXP NP was replaced by DTIME(n ω(1)). Yet for the models of randomized exact learning or Valiant’s PAC learning, the best result so far is a lower bound against BPEXP (the exponential-time analogue of BPP). In this paper, we derive stronger lower bounds as well as some other consequences from randomized exact learning and PAC learning algorithms, answering an open question posed in [6] and [14]. In particular, we show that

Automated induction is a powerful method for the validation of critical systems. However, the inductive proof process faces major challenges: it is undecidable and diverges even with small examples. Previous methods have proposed ad-hoc heuristics to speculate on additional lemmas that hopefully stop the divergence. Although these methods have succeeded in proving interesting theorems, they have significant limitations: in particular, they often fail to find appropriate lemmas, and the lemmas they provide may not be valid. We present a new method that allows us to perform inductive proofs in conditional theories. This method automatically detects divergence in proof traces and derives primal grammars as well as new lemmas that schematize the divergent sequence. This new construction allows us to break the divergence and complete the proof. Our method is presented as a set of inference rules whose soundness and refutational completeness have been formally proved. Unlike previous methods, our method is fully automated and has no risk of over-generalization. Moreover, our technique for capturing and schematizing divergence represents the most general decidable schematization, with respect to description power, among all known schematizations. Our method has been implemented in C++ and successfully proved over fifty complex examples that fail with well-known theorem provers (e.g., ACL2, Isabelle, PVS, SPIKE) and related methods for handling divergence in proofs by induction. Our method represents a significant contribution to the field of automated reasoning as it can be integrated with existing automated and interactive inductive proof systems to enhance their performance. Moreover, it has the potential to substantially reduce the time needed for the verification of critical systems.

We consider the problem of learning monotone Boolean functions over {0, 1}/sup n/ under the uniform distribution. Specifically, given a polynomial number of uniform random samples for an unknown monotone Boolean function f, and given polynomial completing time, we would like to approximate f as well as possible. We describe a simple algorithm that we prove achieves error at most 1/2-/spl Omega/(1//spl radic/n), improving on the previous best bound of 1/2-/spl Omega/((log/sup 2/ n)/n). We also prove that no algorithm, given a polynomial number of samples, can guarantee error 1/2-/spl omega/((log n)//spl radic/n), improving on the previous best hardness bound of O(1//spl radic/n). These lower bounds hold even if the learning algorithm is allowed membership queries. Thus this paper settles to an O(log n) factor the question of the best achievable error for learning the class of monotone Boolean functions with respect to the uniform distribution.

At present, problems related to data mining are aris-ing in various poorly formalizable areas of research(genetics, archaeology, medicine, etc.). Such problemsare characterized by insufficient knowledge about theobjects under study, which hampers the development ofmathematical models for them; by a large number ofmultitype (quantitative and qualitative) factors and arelatively small amount of data; and by the requirementthat analysis results be represented in a form clear toexperts in applied areas.A promising approach to solving such problems isbased on the class of logical decision functions. Theyare frequently represented in the form of a decisiontree. A detailed description of this class and referencescan be found in [1].The construction of decision functions with a mini-mal risk of erroneous predictions is an important task inmethods based on logical decision functions and inother data analysis techniques. It is well known that thecomplexity of the class of decision functions (thecapacity characteristic of the class [2]) is a importantfactor affecting the quality of decisions. To achieve thebest quality, we need a certain tradeoff between thecomplexity of the class and the accuracy of decisionsobtained on the training sample. Thus, the problemarises of choosing an optimal complexity of the class.In numerous applications, empirical data (trainingsample) are supplemented with expert knowledge,which is not tightly related to a distribution model(expert knowledge can be specified as upper bounds forthe risk; they can express constraints on the class of dis-tributions or the class of decision functions, specifypreference rules used in decision making, etc.). Whenan optimal complexity of the class is chosen, availableempirical data and expert knowledge have to be jointlytaken into account.This problem can be solved within the framework ofthe Bayesian learning theory. The idea behind thisapproach is to use a priori knowledge about the prob-lem in question so that each possible distribution (strat-egy, the state of nature) is assigned a certain weight.This weight reflects the expert’s intuitive confidence inwhether the unknown true distribution coincides withthat under consideration.In this paper, we propose a Bayesian recognitionmodel based on a finite set of events. The model is usedto develop and analyze methods for constructing logi-cal decision functions. While choosing an optimal com-plexity of the class, the model takes into account empir-ical data and expert knowledge.In pattern recognition, the task is to predict the pat-tern’s index for an arbitrary object of the populationdescribed by a set of variables. The prediction is madeby analyzing a training sample consisting of the valuesof these variables, together with the index of the corre-sponding pattern for each object. The variables can beof different types; i.e., some of them can be quantita-tive, while the others can be qualitative. As a rule, theproblem is solved in a class of decision functions inwhich an optimal function is sought according to agiven criterion. The class of logical decision functionsis defined on a set of partitions of the feature space intoa finite number of subdomains described by conjunc-tions of simple predicates. The number of subdomainsdetermines the degree of complexity of a logical func-tion.A Bayesian pattern recognition model on a finite setof events is defined by formulating certain statementsthat avoid the local metric properties of the featurespace (the transition from points of the space to events,where an event is understood as that of the original vari-ables taking values from a certain subdomain of thepartition). Moreover, the pattern recognition problem isconsidered on the values of a discrete unordered vari-able, the concept of a learning method (a mapping from