On the Complexity of Rocchio’s Similarity-Based Relevance Feedback Algorithm

It is shown in [4] that Rocchio's similarity-based relevance feedback algorithm makes Ωn mistakes in searching for a collection of documents represented by a monotone disjunction of at most k relevant features or terms over the n-dimensional binary vector space {0, 1}n. In practice, Rocchio's algorithm often uses a fixed query updating factor and a fixed classification threshold. When this is the case, we strengthen the work in [4] in this paper and prove that Rocchio's algorithm makes Ωkn---k mistakes in searching for the same collection of documents over the binary vector space {0, 1}n. A quadratic lower bound is obtained when k is proportional to n. An Okn---k2 upper bound is also obtained.

The theorem on a normal limit distribution of the normalized number of false solutions of a beforehand consistent system of nonlinear random equations over the field GF(2) is proved. The results with the additional condition on the number of nonzero components both false solutions and true solution of the solutions are obtained.

Sparse optimization is about finding minimizers of functions characterized by a number of nonzero components as small as possible, such paradigm being of great practical relevance in Machine Learning, particularly in classification approaches based on support vector machines. By exploiting some properties of the k-norm of a vector, namely, of the sum of its k largest absolute-value components, we formulate a sparse optimization problem as a mixed-integer nonlinear program, whose continuous relaxation is equivalent to the unconstrained minimization of a difference-of-convex function. The approach is applied to Feature Selection in the support vector machine framework, and tested on a set of benchmark instances. Numerical comparisons against both the standard ell _1-based support vector machine and a simple version of the Slope method are presented, that demonstrate the effectiveness of our approach in achieving high sparsity level of the solutions without impairing test-correctness.

In this paper a quantitative measure for design-by-analogy is developed. This measure is based on the functional similarity of products. By using this product similarity measure, designers are able to formalize and quantify design-by-analogy techniques during concept and layout design. The similarity measure and its application is clarified and validated through a case study. The case study is the original design of a pickup winder.

We show that if Arthur-Merlin protocols can be derandomized, then there is a Boolean function computable in deterministic exponential-time with access to an $\bm{\NP}$ oracle, that cannot be computed by Boolean circuits of {\em exponential} size. More formally, if $\bm{\prAM \subseteq \P^{\NP}}$ then there is a Boolean function in $\bm{\E^{\NP}}$ that requires circuits of size $\bm{2^{\Omega(n)}}$. $\bm{\prAM}$ is the class of promise problems that have Arthur-Merlin protocols, $\bm{\P^{\NP}}$ is the class of functions that can be computed in deterministic polynomial-time with an $\bm{\NP}$ oracle and $\bm{\E^{\NP}}$ is its exponential analogue. The lower bound in the conclusion of our theorem suffices to construct very strong pseudorandom generators. We also show that the same conclusion holds if the problem of approximate counting the number of accepting paths of a nondeterministic Turing machine up to multiplicative factors can be done in nondeterministic polynomial-time. In other words, showing nondeterministic fully polynomial-time approximation schemes for $\bm{\sharp \P}$-complete problems require proving exponential-size circuit lower bounds. A few works have already shown that if we can find efficient deterministic solutions to some specific tasks (or classes) that are known to be solvable efficiently by randomized algorithms (or proofs), then we obtain lower bounds against certain circuit models. These lower bounds were only with respect to polynomial-size circuits {\em even if full derandomization is assumed}. Thus they only implied fairly weak pseudorandom generators (if at all). A key ingredient in our proof is a connection between computational learning theory and exponential-size lower bounds. We show that the existence of deterministic learning algorithms with certain properties implies exponential-size lower bounds, where the complexity of the hard function is related to the complexity of the learning algorithm.

The necessity of non-binary constraint satisfaction algorithms is increasing because many real problems are inherently nonbinary. Considering overconstrained problems (and Partial Forward Checking as the solving algorithm), we analyze several lower bounds proposed in the binary case, extending them for the non-binary case. We show that techniques initially developed in the context of reversible DAC can be applied in the general case, to deal with constraints of any arity. We discuss some of the issues raised for non-binary lower bounds, and we study their computational complexity. We provide experimental results of the use of the new lower bounds on overconstrained random problems, including constraints with different weights.KeywordsConstraint Satisfaction ProblemSoft ConstraintLimited VersionFuture VariableCurrent Partial AssignmentThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

We extend the line of research initiated by Fortnow and Klivans [6] that studies the relationship between efficient learning algorithms and circuit lower bounds. In [6], it was shown that if a Boolean circuit class \(\mathcal{C}\) has an efficient deterministic exact learning algorithm, (i.e. an algorithm that uses membership and equivalence queries) then \(\mathsf{EXP}^{\mathsf{NP}} \not \subseteq \mathsf{P/poly}[\mathcal{C}]\). Recently, in [14] EXP NP was replaced by DTIME(n ω(1)). Yet for the models of randomized exact learning or Valiant’s PAC learning, the best result so far is a lower bound against BPEXP (the exponential-time analogue of BPP). In this paper, we derive stronger lower bounds as well as some other consequences from randomized exact learning and PAC learning algorithms, answering an open question posed in [6] and [14]. In particular, we show that

Given a simply connected manifold M , we completely determine which rational monomial Pontryagin numbers are attained by fiber homotopy trivial M -bundles over the k -sphere, provided that k is small compared to the dimension and the connectivity of M . Furthermore, we study the vector space of rational cobordism classes represented by such bundles. We give upper and lower bounds on its dimension, and we construct manifolds for which the lower bound is attained. Our proofs are based on the classical approach to studying diffeomorphism groups via block bundles and surgery theory, and we make use of ideas developed by Krannich–Kupers–Randal-Williams. As an application, we show the existence of elements of infinite order in the homotopy groups of the spaces of positive Ricci and positive sectional curvature, provided that M is $\operatorname {Spin}$ , has a nontrivial rational Pontryagin class and admits such a metric. This is done by constructing M -bundles over spheres with nonvanishing ${\hat {\mathcal {A}}}$ -genus. Furthermore, we give a vanishing theorem for generalized Morita–Miller–Mumford classes for fiber homotopy trivial bundles over spheres. In the appendix coauthored by Jens Reinhold, we investigate which classes of the rational oriented cobordism ring contain an element that fibers over a sphere of a given dimension.

A function $f:\mathbb{F}_2^n \to \{-1,1\}$ is called linear-isomorphic to g if f=g∘A for some non-singular matrix A. In the g-isomorphism problem, we want a randomized algorithm that distinguishes whether an input function f is linear-isomorphic to g or far from being so. We show that the query complexity to test g-isomorphism is essentially determined by the spectral norm of g. That is, if g is close to having spectral norm s, then we can test g-isomorphism with poly(s) queries, and if g is far from having spectral norm s, then we cannot test g-isomorphism with o(logs) queries. The upper bound is almost tight since there is indeed a function g close to having spectral norm s whereas testing g-isomorphism requires Ω(s) queries. As far as we know, our result is the first characterization of this type for functions. Our upper bound is essentially the Kushilevitz-Mansour learning algorithm, modified for use in the implicit setting. Exploiting our upper bound, we show that any property is testable if it can be well-approximated by functions with small spectral norm. We also extend our algorithm to the setting where A is allowed to be singular.

A new lower bound for the maximal length of a multivector is obtained. It is much closer to the best known upper bound than previously reported lower bound estimates. The maximal length appears to be unexpectedly large for n-vectors, with $$n>2$$ , since the few exactly known values seem to grow only linearly with vector space dimension, whereas the new lower bound grows at power $$n-1$$ like the best known upper bound. This result has implications in quantum chemistry for the compression of information contained in an electronic wave function.

In this paper, we introduce a nonzero component graph of a finite-dimensional near-vector space constructed from a finite field. We investigate the relationship between the existence of a basis of a near-vector space over a field with that of the vector space over the same field, with the same dimension. We introduce the notion of a coordinate vector and prove an equivalent formulation for the definition of the nonzero component graph. We show that despite the vast differences in the structure of a near-vector space compared with a vector space, especially in the nonregular case, the graph fails the recognition problem in that it cannot distinguish between the near-vector space and its associated vector space. Thus, we show that the nonzero component graph of a near-vector space is identical to the one for a vector space over the same field, of the same dimension. Leveraging this result, we completely characterise and describe graph properties for the nonzero component graph on both near-vector spaces and vector spaces by partitioning the graph into components using the regularity graph of any nonregular near-vector space (with the same dimension) with its nonzero component graph. In closing we provide a reconstruction algorithm for constructing a near-vector space for a given nonzero component graph, satisfying certain conditions.

. . . .

We present a simple noise-robust margin-based active learn-ing algorithm to find homogeneous (passing the origin) linearseparators and analyze its error convergence when labels arecorrupted by noise. We show that when the imposed noisesatisfies the Tsybakov low noise condition (Mammen, Tsy-bakov, and others 1999; Tsybakov 2004) the algorithm is ableto adapt to unknown level of noise and achieves optimal sta-tistical rate up to polylogarithmic factors. We also derive lower bounds for margin based active learningalgorithms under Tsybakov noise conditions (TNC) for themembership query synthesis scenario (Angluin 1988). Ourresult implies lower bounds for the stream based selectivesampling scenario (Cohn 1990) under TNC for some fairlysimple data distributions. Quite surprisingly, we show that thesample complexity cannot be improved even if the underly-ing data distribution is as simple as the uniform distributionon the unit ball. Our proof involves the construction of a well-separated hypothesis set on the d-dimensional unit ball alongwith carefully designed label distributions for the Tsybakovnoise condition. Our analysis might provide insights for otherforms of lower bounds as well.

In 2009, Röglin and Teng showed that the smoothed number of Pareto optimal solutions of linear multi-criteria optimization problems is polynomially bounded in the number n of variables and the maximum density φ of the semi-random input model for any fixed number of objective functions. Their bound is, however, not very practical because the exponents grow exponentially in the number d + 1 of objective functions. In a recent breakthrough, Moitra and O’Donnell improved this bound significantly to \(O \big ( n^{2d} \phi^{d(d+1)/2}\big )\).An “intriguing problem”, which Moitra and O’Donnell formulate in their paper, is how much further this bound can be improved. The previous lower bounds do not exclude the possibility of a polynomial upper bound whose degree does not depend on d. In this paper we resolve this question by constructing a class of instances with Ω( (n φ)(d − logd) ·(1 − Θ1/φ) ) Pareto optimal solutions in expectation. For the bi-criteria case we present a higher lower bound of Ω( n 2 φ 1 − Θ1/φ ), which almost matches the known upper bound of O( n 2 φ).KeywordsMaximum DensityKnapsack ProblemPareto Optimal SolutionSplit StepSmooth NumberThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Recent work on structure-preserving signatures studies optimality of these schemes in terms of the number of group elements needed in the verification key and the signature, and the number of pairing-product equations in the verification algorithm. While the size of keys and signatures is crucial for many applications, another important aspect to consider for performance is the time it takes to verify a given signature. By far, the most expensive operation during verification is the computation of pairings. However, the concrete number of pairings that one needs to compute is not captured by the number of pairing-product equations considered in earlier work.To fill this gap, we consider the question of what is the minimal number of pairings that one needs to compute in the verification of structure-preserving signatures. First, we prove lower bounds for schemes in the Type II setting that are secure under chosen message attacks in the generic group model, and we show that three pairings are necessary and that at most one of these pairings can be precomputed. We also extend our lower bound proof to schemes secure under random message attacks and show that in this case two pairings are still necessary.Second, we build an automated tool to search for schemes matching our lower bounds. The tool can generate automatically and exhaustively all valid structure-preserving signatures within a user-specified search space, and analyze their (bounded) security in the generic group model. Interestingly, using this tool, we find a new randomizable structure-preserving signature scheme in the Type II setting that is optimal with respect to the lower bound on the number of pairings, and also minimal with respect to the number of group operations that have to be computed during verification.KeywordsGroup ElementSignature SchemeSignature ElementBlind SignatureSigning AlgorithmThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.