Finding Relevant Variables in PAC Model with Membership Queries

The Kushilevitz-Mansour (KM)algorithm is an algorithm that finds all the “heavy” Fourier coefficients of a boolean function. It is the main tool for learning decision trees and DNF expressions in the PAC model with respect to the uniform distribution. The algorithm requires an access to the membership query (MQ)oracle.We weaken this requirement by producing an analogue of the KM algorithm that uses extended statistical queries (SQ)(SQs in which the expectation is taken with respect to a distribution given by a learning algorithm). We restrict a set of distributions that a learning algorithm may use for its SQs to be a set of specific constant bounded product distributions. Our analogue finds all the “heavy” Fourier coefficients of degree lower than c log n (we call it BS). We use BS to learn decision trees and by adapting Freund’s boosting technique we give algorithm that learns DNF in this model. Learning in this model implies learning with persistent classification noise and in some cases can be extended to learning with product attribute noise.We develop a characterization for learnability with these extended SQs and apply it to get several negative results about the model.

Augmented reality (AR) is commonly experienced through a mobile device such as a smartphone or tablet. Accuracy in locating virtual targets in real environments can be important for tasks such as navigation and training. In this paper, we tested users’ perception of distances (10-40m) to virtual objects presented in both indoor and outdoor environments through an AR platform for a mobile device. In addition, we tested users’ perception of distances to real objects through the mobile device as a baseline comparison. Although the 40m distance was underestimated similarly for both real and virtual targets, the virtual target was judged to be farther than the real target at the closer distances. There was no effect of environmental context on either viewing condition or distance estimation. We discuss the implications of our findings for mobile augmented reality applications that require precise distance perception.

We consider the following question: using a source of labeled data and interaction with an untrusted prover, what is the complexity of verifying that a given hypothesis is "approximately correct"? We study interactive proof systems for PAC verification, where a verifier that interacts with a prover is required to accept good hypotheses, and reject bad hypotheses. Both the verifier and the prover are efficient and have access to labeled data samples from an unknown distribution. We are interested in cases where the verifier can use significantly less data than is required for (agnostic) PAC learning, or use a substantially cheaper data source (e.g., using only random samples for verification, even though learning requires membership queries). We believe that today, when data and data-driven algorithms are quickly gaining prominence, the question of verifying purported outcomes of data analyses is very well-motivated. We show three main results. First, we prove that for a specific hypothesis class, verification is significantly cheaper than learning in terms of sample complexity, even if the verifier engages with the prover only in a single-round (NP-like) protocol. Moreover, for this class we prove that single-round verification is also significantly cheaper than testing closeness to the class. Second, for the broad class of Fourier-sparse boolean functions, we show a multi-round (IP-like) verification protocol, where the prover uses membership queries, and the verifier is able to assess the result while only using random samples. Third, we show that verification is not always more efficient. Namely, we show a class of functions where verification requires as many samples as learning does, up to a logarithmic factor.

We consider the problem of learning monotone Boolean functions over {0, 1}/sup n/ under the uniform distribution. Specifically, given a polynomial number of uniform random samples for an unknown monotone Boolean function f, and given polynomial completing time, we would like to approximate f as well as possible. We describe a simple algorithm that we prove achieves error at most 1/2-/spl Omega/(1//spl radic/n), improving on the previous best bound of 1/2-/spl Omega/((log/sup 2/ n)/n). We also prove that no algorithm, given a polynomial number of samples, can guarantee error 1/2-/spl omega/((log n)//spl radic/n), improving on the previous best hardness bound of O(1//spl radic/n). These lower bounds hold even if the learning algorithm is allowed membership queries. Thus this paper settles to an O(log n) factor the question of the best achievable error for learning the class of monotone Boolean functions with respect to the uniform distribution.

In [Proceedings of the Second Symposium on Switching Circuit Theory and Logical Design (FOCS), 1961, pp. 34–38], Chow proved that every Boolean threshold function is uniquely determined by its degree-0 and degree-1 Fourier coefficients. These numbers became known as the Chow parameters. Providing an algorithmic version of Chow's theorem—i.e., efficiently constructing a representation of a threshold function given its Chow parameters—has remained open ever since. This problem has received significant study in the fields of circuit complexity, game theory and the design of voting systems, and learning theory. In this paper we effectively solve the problem, giving a randomized polynomial-time approximation scheme with the following behavior: Given the Chow parameters of a Boolean threshold function f over n bits and any constant $\epsilon>0$, the algorithm runs in time $O(n^2\log^2n)$ and with high probability outputs a representation of a threshold function $f'$ which is $\epsilon$-close to f. Along the way we prove several new results of independent interest about Boolean threshold functions. In addition to various structural results, these include $\tilde{O}(n^2)$-time learning algorithms for threshold functions under the uniform distribution in the following models: (i) the restricted focus of attention model, answering an open question of Birkendorf et al.; (ii) an agnostic-type model. This contrasts with recent results of Guruswami and Raghavendra who show NP-hardness for the problem under general distributions; (iii) the PAC model, with constant $\epsilon$. Our $\tilde{O}(n^2)$-time algorithm substantially improves on the previous best known running time and nearly matches the $\Omega(n^2)$ bits of training data that any successful learning algorithm must use.

Information in statistical physics

Hardness of approximate two-level logic minimization and PAC learning with membership queries

Based on fuzzy-set theory, a method is considered for estimating and forecasting the performance parameters of a system under the condition of unknown probability distribution and small sample. Some concepts including point estimation, interval estimation, optimal level, empirical probability distribution and confidence level are recommended. The performance prediction in the system for application is also investigated. The validity of this method is examined by computer simulation experiments under the condition of small sample and typical distributions such as normal distribution, uniform distribution, Rayleigh distribution, and unknown distribution. The confidence level is proved to be higher than 95% via engineering experiment on noise of rolling bearings.

We consider the problem of distribution-free testing of the class of monotone monomials and the class of monomials over n variables.0 While there are very efficient algorithms for testing a variety of functions classes when the underlying distribution is uniform, designing distribution-free algorithms (which must work under any arbitrary and unknown distribution), tends to be a more challenging task. When the underlying distribution is uniform, Parnas et al. (SIAM Journal on Discrete Math, 2002) give an algorithm for testing (monotone) monomials whose query complexity does not depends on n, and whose dependence on the distance parameter is (inverse) linear. In contrast, Glasner and Servedio (in Proceedings of RANDOM, 2007) prove that every distribution-free testing algorithm for monotone monomials as well as for general monomials must have query complexity Ω(n1/5) (for a constant distance parameter e). In this paper we present distribution-free testing algorithms for these classes where the query complexity of the algorithms is O(n1/2/e). We note that as opposed to previous results for distribution-free testing, our algorithms do not build on the algorithms that work under the uniform distribution. Rather, we define and exploit certain structural properties of monomials (and functions that differ from them in a nonnegligible manner), which were not used in previous work on property testing.

Some properties of progressive censored order statistics from arbitrary and uniform distributions with applications to inference and simulation

The analysis of errors in two-joint reaching movements has provided clues about sensorimotor processing algorithms. The present study extends this focus to situations where the head, trunk, and legs join with the arm to help reach targets placed slightly beyond arm's length. Subjects reached accurately to touch "real targets" or reached to the remembered locations of "virtual targets" (i.e., targets removed at the start of the reach). Subjects made large errors in the virtual-target condition and these errors were analyzed with the aim of revealing the implications for whole-body coordination. Subjects were found to rotate the head less in the virtual-target condition (when compared with accurate movements to real targets). This resulted in a more limited range of head postures, and the final head angles at the end of the movements were geometrically related to the incorrect hand locations, perhaps accounting for some portion of the errors. This suggests that head-eye-hand coordination plays an important role in the organization of these movements and leads to the hypothesis that a representation of current gaze direction may serve as a reference signal for arm motor control.

In this paper we consider an approach to passive learning. In contrast to the classical PAC model we do not assume that the examples are independently drawn according to an underlying distribution, but that they are generated by a time-driven process. We define deterministic and probabilistic learning models of this sort and investigate the relationships between them and with other models. The fact that successive examples are related can often be used to gain additional information similar to the information gained by membership queries. We show that this can be used to design on-line prediction algorithms. In particular, we present efficient algorithms for exactly identifying Boolean threshold functions, 2-term RSE, and 2-term-DNF, when the examples are generated by a random walk on {0,1}n.

Echolocating bats have to assign the received echoes to the correct call that generated them. Failing to do so will result in the perception of virtual targets that are positioned where there is no actual target. The assignment of echoes to the emitted calls can be ambiguous especially if the pulse intervals between calls are short and kept constant. Here, we present first evidence that greater mouse-eared bats deal with ambiguity by changing the pulse interval more often, in particular by reducing the number of calls in the terminal group before landing. This strategy separates virtual targets from real ones according to their change in position. Real targets will always remain in a constant position, and virtual targets will jitter back and forth according to the change in the time interval.

This work gives a polynomial time algorithm for learning decision trees with respect to the uniform distribution. (This algorithm uses membership queries.) The decision tree model that is considered is an extension of the traditional boolean decision tree model that allows linear operations in each node (i.e., summation of a subset of the input variables over $GF(2)$).This paper shows how to learn in polynomial time any function that can be approximated (in norm $L_2 $) by a polynomially sparse function (i.e., a function with only polynomially many nonzero Fourier coefficients). The authors demonstrate that any function f whose $L_1 $-norm (i.e., the sum of absolute value of the Fourier coefficients) is polynomial can be approximated by a polynomially sparse function, and prove that boolean decision trees with linear operations are a subset of this class of functions. Moreover, it is shown that the functions with polynomial $L_1 $-norm can be learned deterministically.The algorithm can also exactly identi...

The prevalence of female stress urinary incontinence is high, and young adults are also affected, including athletes, especially those involved in "high-impact" sports. To date there have been almost no studies testing pelvic floor muscle (PFM) activity during dynamic functional whole body movements. The aim of this study was the description and reliability test of PFM activity and time variables during running. A prospective cross-sectional study including ten healthy female subjects was designed with the focus on the intra-session test-retest reliability of PFM activity and time variables during running derived from electromyography (EMG) and accelerometry. Thirteen variables were identified based on ten steps of each subject: Six EMG variables showed good reliability (ICC 0.906-0.942) and seven time variables did not show good reliability (ICC 0.113-0.731). Time variables (e.g. time difference between heel strike and maximal acceleration of vaginal accelerator) showed low reliability. However, relevant PFM EMG variables during running (e.g., pre-activation, minimal and maximal activity) could be identified and showed good reliability. Further adaptations regarding measurement methods should be tested to gain better control of the kinetics and kinematics of the EMG probe and accelerometers. To our knowledge this is the first study to test the reliability of PFM activity and time variables during dynamic functional whole body movements. More knowledge of PFM activity and time variables may help to provide a deeper insight into physical strain with high force impacts and important functional reflexive contraction patterns of PFM to maintain or to restore continence.