A Formal Framework for Speedup Learning from Problems and Solutions

DYNAMIC: A new role for training problems in EBL

CHAPTER 8 - Toward Scaling Up Machine Learning: A Case Study with Derivational Analogy in PRODIGY

The possibility of applying explanation-based learning (EBL), a technique from machine learning, to intelligent process planning is explored. There are currently two major approaches to process planning: the variant approach and the generative approach. Each has advantages and deficiencies. The authors' hypothesis is that EBL could successfully unite these apparently disparate approaches. EBL can be used to transform a traditional weak method planner into a strong method skeletal planner by acquiring strong method concepts which are generalized weak-method explanations of observed episodes. It would seem to be a natural vehicle to unite variant and generative process planning. A learning process planner, called EXBLIPP is implemented to test the authors' hypothesis. It is found that the system possesses many of the intended advantages. It is demonstrated that the EBL capability enables the process planning system to learn new schemata which yield many of the advantages of variant process planning.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Although much research in machine learning has been carried out on acquiring knowledge for problem-solving in many problem domains, little effort has been focused on learning search-control knowledge for solving optimization problems. This paper reports on the development of SHAPES, a system that learns heuristic search guidance for solving optimization problems in intractable, under-constrained domains based on the Explanation-Based Learning (EBL) framework. The system embodies two new and novel approaches to machine learning. First, it makes use of explanations of varying levels of approximation as a mean for verifying heuristic-based decisions, allowing heuristic estimates to be revised and corrected during problem-solving. The provision of such a revision mechanism is particularly important when working in intractable and under-constrained domains, where heuristics tend to be highly over-generalized, and hence at times will give rise to incorrect results. Second, it employs a new linear and quadratic programming-based weight-assignment algorithm formulated to direct search toward optimal solutions under best-first search. The algorithm offers a direct method for assigning rule strengths and, in so doing, avoids the need to address the credit-assignment problem faced by other iterative weight-adjustment methods.

Reinforcement learning (RL) focuses on an essential aspect of intelligent behavior – how an agent can learn to make good decisions given experience and rewards in a stochastic<br>world. Yet popular RL algorithms that have enabled exciting successes in domains with good simulators (Go, Atari, etc) still often fail to learn in other domains because they rely on<br>simple heuristics for exploration. This provides additional empirical justification for essential questions around RL, specifically around algorithms that learn in a provably efficient manner through strategic exploration in any considered domain. This thesis provides new algorithms<br>and theory that enable good performance with respect to existing theoretical frameworks for evaluating RL algorithms (specifically, probably approximately correct) and introduces new stronger evaluation criteria, that may be particularly of interest as RL is applied to more real world problems.<br>For the first line of work on probably approximately correct (PAC) RL algorithms, we introduce a series of algorithms for episodic tabular domains with substantially better PAC<br>sample complexity bounds that culminate in a new algorithm with close to minimax optimal PAC and regret bounds. Look up tables are required by most sample efficient and computationally tractable algorithms, but cannot represent many practical domains. We therefore also present a new RL algorithm that can learn a good policy in environments with high dimensional observations and hidden deterministic states; unlike predecessors, this algorithm provably<br>explores not only in a statistically but also computationally efficient manner assuming access to function classes with efficient optimization oracles. To make progress it is critical to have the right measures of success. While empirical<br>demonstrations are quite clear, we find that for theoretical properties, two of the most commonly used learning frameworks, PAC guarantees and regret guarantees, each allow undesirable algorithm behavior (e.g. ignoring new observations that could improve the policy). We present<br>a new stronger learning framework called Uniform-PAC that unifies the existing frameworks and prevents undesirable algorithm properties. One caveat of all existing learning frameworks is that for any particular episode, we do not<br>know how well the algorithm will perform. To address this, we introduce the IPOC framework that requires algorithms to provide a certificate before each episode bounding how suboptimal the current policy can be. Such certifications may be of substantial interest in high stakes scenarios when an organization may wish to track or even pause an online RL system should the potential expected performance bound drop below a required expected outcome.

Average Case Analysis of Conjunctive Learning Algorithms

: Machine Learning approaches to knowledge compilation seek to improve the performance of problem-solvers by storing solutions to previously solved problems in an efficient generalized form. The problem-solver retrieves these learned solutions in appropriate later situations to obtain results more efficiently. However, by relying on its learned knowledge to provide a solution, the problem-solver may miss an alternative solution of higher quality-one that could have been generated using the original (non-learned) problem-solving knowledge. This phenomenon is referred to as the masking effect of learning. In this paper, we examine a sequence of possible solutions for the masking effect. Each solution refines and builds on the previous one. The final solution is built on the cascaded filters. When learned knowledge is retrieved, these filters alert the system about the inappropriateness of this knowledge so that the conditions under which this solution will perform better than the others, and present experimental data supportive of the analysis. This investigation is based on simulated robot domain called Ground world. Artificial intelligence, Machine learning, Knowledge compilation, Explanation-based learning, Solution quality, Utility of learning, SOAR.

In the last few decades a computational approach to machine learning has emerged based on paradigms from recursion theory and the theory of computation. Such ideas include learning in the limit, learning by enumeration, and probably approximately correct (pac) learning. These models usually are not suitable in practical situations. In contrast, statistics based inference methods have enjoyed a long and distinguished career. Currently, Bayesian reasoning in various forms, minimum message length (MML) and minimum description length (MDL), are widely applied approaches. They are the tools to use with particular machine learning praxis such as simulated annealing, genetic algorithms, genetic programming, artificial neural networks, and the like. These statistical inference methods select the hypothesis which minimizes the sum of the length of the description of the hypothesis (also called ‘model’) and the length of the description of the data relative to the hypothesis. It appears to us that the future of computational machine learning will include combinations of the approaches above coupled with guaranties with respect to used time and memory resources. Computational learning theory will move closer to practice and the application of the principles such as MDL require further justification. Here, we survey some of the actors in this dichotomy between theory and praxis, we justify MDL via the Bayesian approach, and give a comparison between pac learning and MDL learning of decision trees.

A recent innovation in computational learning theory is the statistical query (SQ) model. The advantage of specifying learning algorithms in this model is that SQ algorithms can be simulated in the probably approximately correct (PAC) model, both in the absenceandin the presence of noise. However, simulations of SQ algorithms in the PAC model have non-optimal time and sample complexities. In this paper, we introduce a new method for specifying statistical query algorithms based on a type ofrelative errorand provide simulations in the noise-free and noise-tolerant PAC models which yield more efficient algorithms. Requests for estimates of statistics in this new model take the following form: “Return an estimate of the statistic within a 1±?factor, or return ?, promising that the statistic is less than?.” In addition to showing that this is a very natural language for specifying learning algorithms, we also show that this new specification is polynomially equivalent to standard SQ, and thus, known learnability and hardness results for statistical query learning are preserved. We then give highly efficient PAC simulations of relative error SQ algorithms. We show that the learning algorithms obtained by simulating efficient relative error SQ algorithms both in the absence of noise and in the presence of malicious noise have roughly optimal sample complexity. We also show that the simulation of efficient relative error SQ algorithms in the presence of classification noise yields learning algorithms at least as efficient as those obtained through standard methods, and in some cases improved, roughly optimal results are achieved. The sample complexities for all of these simulations are based on thed?metric, which is a type of relative error metric useful for quantities which are small or even zero. We show that uniform convergence with respect to thed?metric yields “uniform convergence” with respect to (?, ?) accuracy. Finally, while we show that manyspecificlearning algorithms can be written as highly efficient relative error SQ algorithms, we also show, in fact, thatallSQ algorithms can be written efficiently by proving general upper bounds on the complexity of (?, ?) queries as a function of the accuracy parameter?. As a consequence of this result, we give general upper bounds on the complexity of learning algorithms achieved through the use of relative error SQ algorithms and the simulations described above.

When data is distributed over a network, statistical learning needs to be carried out in a fully distributed fashion. When all nodes in the network are faultless and cooperate with each other, it is understood that the objective is probably approximately correct (PAC) learnable. However, when there are malicious nodes trying to sabotage learning by injecting false information in the network, PAC learnability of the objective remains an open question. In this paper, we discuss the distributed statistical learning problem when the risk function is strictly convex. We show that the model is PAC learnable in the presence of malicious nodes by proposing and analyzing a distributed learning algorithm. Experiments in non-convex settings are also performed to further discuss the PAC learnability of non-convex statistical learning problems from distributed data.

Machine Learning (ML) is increasingly being used in intelligent systems that can perform Artificial Intelligence (AI) functions. Analytical model development and solving problems related with it may be automated by machine learning, which explains the ability of computers to learn from problem-specific learning algorithm. Depending on artificial neural networks, "deep learning" is a kind of machine learning. The performance of deep learning techniques is superior to that of superficial machine learning techniques and conventional methods of data analysis in many situations. Deep Machine Learning (DML) algorithms and frameworks that have been implemented to and supported by wireless communication systems have been thoroughly analyzed in this paper. User associations, power latency and allocation; bandwidth assignment and user selections, and; cloud computing technology on the edge have both been suggested as potential DML implementations.

We consider learning on multilayer neural nets with piecewise polynomial activation functions and a fixed number k of numerical inputs. We exhibit arbitrarily large network architectures for which efficient and provably successful learning algorithms exist in the rather realistic refinement of Valiant's model for probably approximately correct learning ("PAC learning") where no a priori assumptions are required about the "target function" (agnostic learning), arbitrary noise is permitted in the training sample, and the target outputs as well as the network outputs may be arbitrary reals. The number of computation steps of the learning algorithm LEARN that we construct is bounded by a polynomial in the bit-length n of the fixed number of input variables, in the bound s for the allowed bit-length of weights, in 1/ε, where ε is some arbitrary given bound for the true error of the neural net after training, and in 1/δ where δ is some arbitrary given bound for the probability that the learning algorithm fails for a randomly drawn training sample. However, the computation time of LEARN is exponential in the number of weights of the considered network architecture, and therefore only of interest for neural nets of small size. This article provides details to the previously published extended abstract (Maass 1994).

Query-driven machine learning models have emerged as a promising estimation technique for query selectivities. Yet, surprisingly little is known about the efficacy of these techniques from a theoretical perspective, as there exist substantial gaps between practical solutions and state-of-the-art (SOTA) theory based on the Probably Approximately Correct (PAC) learning framework. In this paper, we aim to bridge the gaps between theory and practice. First, we demonstrate that selectivity predictors induced by signed measures are learnable, which relaxes the reliance on probability measures in SOTA theory. More importantly, beyond the PAC learning framework (which only allows us to characterize how the model behaves when both training and test workloads are drawn from the same distribution), we establish, under mild assumptions, that selectivity predictors from this class exhibit favorable out-of-distribution (OOD) generalization error bounds. These theoretical advances provide us with a better understanding of both the in-distribution and OOD generalization capabilities of query-driven selectivity learning, and facilitate the design of two general strategies to improve OOD generalization for existing query-driven selectivity models. We empirically verify that our techniques help query-driven selectivity models generalize significantly better to OOD queries both in terms of prediction accuracy and query latency performance, while maintaining their superior in-distribution generalization performance.

Explanation-based learning is using the same technique as partial evaluation, namely unfolding. However, it brings a new insight: an example can be used to guide the transformation process. In this paper, we further explore this insight and show how examples can be used to guide other kinds of program transformation, guiding not only the unfolding, but also the introduction of new predicates and the folding. On the other hand, we illustrate the more fundamental restructuring which is possible with program transformation and the relevance of completeness results to eliminate computationally inefficient knowledge.KeywordsLogic ProgramPartial EvaluationAbstract InterpretationHorn ClauseProgram TransformationThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

COMBINING EXPLANATION-BASED LEARNING AND ARTIFICIAL NEURAL NETWORKS