Learning efficient logic programs

Inductive Logic Programming: Issues, results and the challenge of Learning Language in Logic

Inductive Logic Programming (ILP) systems aim to find a set of logical rules, called a hypothesis, that explain a set of examples. In cases where many such hypotheses exist, ILP systems often bias towards shorter solutions, leading to highly general rules being learned. In some application domains like security and access control policies, this bias may not be desirable, as when data is sparse more specific rules that guarantee tighter security should be preferred. This paper presents a new general notion of a scoring function over hypotheses that allows a user to express domain-specific optimisation criteria. This is incorporated into a new ILP system, called FastLAS, that takes as input a learning task and a customised scoring function, and computes an optimal solution with respect to the given scoring function. We evaluate the accuracy of FastLAS over real-world datasets for access control policies and show that varying the scoring function allows a user to target domain-specific performance metrics. We also compare FastLAS to state-of-the-art ILP systems, using the standard ILP bias for shorter solutions, and demonstrate that FastLAS is significantly faster and more scalable.

This paper is concerned with problems that arise when submitting large quantities of data to analysis by an Inductive Logic Programming (ILP) system. Complexity arguments usually make it prohibitive to analyse such datasets in their entirety. We examine two schemes that allow an ILP system to construct theories by sampling from this large pool of data. The first, “subsampling”, is a single-sample design in which the utility of a potential rule is evaluated on a randomly selected sub-sample of the data. The second, “logical windowing”, is multiple-sample design that tests and sequentially includes errors made by a partially correct theory. Both schemes are derived from techniques developed to enable propositional learning methods (like decision trees) to cope with large datasets. The ILP system CProgol, equipped with each of these methods, is used to construct theories for two datasets—one artificial (a chess endgame) and the other naturally occurring (a language tagging problem). In each case, we ask the following questions of CProgol equipped with sampling: (1) Is its theory comparable in predictive accuracy to that obtained if all the data were used (that is, no sampling was employed)?s and (2) Is its theory constructed in less time than the one obtained with all the data? For the problems considered, the answers to these questions is “yes”. This suggests that an ILP program equipped with an appropriate sampling method could begin to address problems satisfactorily that have hitherto been inaccessible simply due to data extent.

Face detection systems have recently achieved high detection rates and real-time performance. However, these methods usually rely on a huge training database (around 5,000 positive examples for good performance). While such huge databases may be feasible for building a system that detects a single object, it is obviously problematic for scenarios where multiple objects (or multiple views of a single object) need to be detected. Indeed, even for multi-viewface detection the performance of existing systems is far from satisfactory. In this work we focus on the problem of learning to detect objects from a small training database. We show that performance depends crucially on the features that are used to represent the objects. Specifically, we show that using local edge orientation histograms (EOH) as features can significantly improve performance compared to the standard linear features used in existing systems. For frontal faces, local orientation histograms enable state of the art performance using only a few hundred training examples. For profile view faces, local orientation histograms enable learning a system that seems to outperform the state of the art in real-time systems even with a small number of training examples.

Data transformation involves the manual construction of large numbers of special-purpose programs. Although typically small, such programs can be complex, involving problem decomposition, recursion, and recognition of context. Building such programs is common in commercial and academic data analytic projects and can be labour intensive and expensive, making it a suitable candidate for machine learning. In this paper, we use the meta-interpretive learning framework (MIL) to learn recursive data transformation programs from small numbers of examples. MIL is well suited to this task because it supports problem decomposition through predicate invention, learning recursive programs, learning from few examples, and learning from only positive examples. We apply Metagol, a MIL implementation, to both semi-structured and unstructured data. We conduct experiments on three real-world datasets: medical patient records, XML mondial records, and natural language taken from ecological papers. The experimental results suggest that high levels of predictive accuracy can be achieved in these tasks from small numbers of training examples, especially when learning with recursion.

This chapter tackles the problem of few-shot image generation by investigating the combination of meta-learning and Generative Adversarial Networks (GANs). As the need for effective and high-quality image synthesis increases conventional techniques frequently find it difficult to produce a wide range of realistic images with a small number of training examples. We present a framework that combines the advantages of GANs—which are excellent at generating high-fidelity images—with meta-learning strategies which help models efficiently learn from a small number of examples. Through the combination of these two methods, we put forth new tactics that improve GANs flexibility and few-shot performance. In order to enhance performance on tasks requiring quick learning from little data we go over the fundamental ideas of both meta-learning and GAN architectures. We validate the effectiveness of our method with extensive experiments and case studies showing gains in image diversity quality and generation speed over traditional methods.

This paper presents an application of some Inductive logic programming (ILP) techniques for checking user’s answer correctness in a Computer-Aided Language Learning (CALL) system STyLE (Scientific Terminology Learning Environment). STyLE supports adaptive learning of English terminology with a target user group of non-native English speakers. In STyLE are implemented many original features that make this system intelligent and adaptive, but we will focus only on one of them: supporting learner-system communication in Natural Language (NL). The proposed ILP system Rich is used for generation of least generalization(LG) and greatest specialization(GS) of the set of possible correct answers of a given question to the user from the system. The user’s answer is correct if it is between LG and GS of the correct answers’ set.KeywordsInductive Logic ProgrammingNatural Language ProcessingFree-text input

We present a heuristic based algorithm to induce nonmonotonic logic programs that will explain the behavior of XGBoost trained classifiers. We use the technique based on the LIME approach to locally select the most important features contributing to the classification decision. Then, in order to explain the model’s global behavior, we propose the LIME-FOLD algorithm —a heuristic-based inductive logic programming (ILP) algorithm capable of learning nonmonotonic logic programs—that we apply to a transformed dataset produced by LIME. Our proposed approach is agnostic to the choice of the ILP algorithm. Our experiments with UCI standard benchmarks suggest a significant improvement in terms of classification evaluation metrics. Meanwhile, the number of induced rules dramatically decreases compared to ALEPH, a state-of-the-art ILP system.

Generative policies have been proposed as a mechanism to learn the constraints and preferences of a system—especially complex systems such as the ones found in coalitions—in a given context so that the system can adapt to unexpected changes seamlessly, thus achieving the system goals with minimal human intervention. Generative policies can help a coalition system to be more effective when working in a distributed, continuously transforming environment with a diverse set of members, resources, and tasks. Learning mechanisms based on logic programming, e.g., Inductive Logic Programming (ILP), have several properties that make them suitable and attractive for the creation and adaptation of generative policies, such as the ability to learn a general model from a small number of examples, and being able to incorporate existing background knowledge. ILP has recently been extended with the introduction of systems for Inductive Learning of Answer Set Programs (ILASP) which are capable of supporting automated acquisition of complex knowledge such as constraints, preferences and rule-based models. Motivated by the capabilities of ILASP, we present AGENP, an Answer Set Grammar-based Generative Policy Framework for Autonomous Managed Systems (AMS) that aims to support the creation and evolution of generative policies by leveraging ILASP. We describe the framework components, i.e., inputs, data structures, mechanisms to support the refinement and instantiation of policies, identification of policy violations, monitoring of policies, and policy adaptation according to changes in the AMS and its context. Additionally, we present the main work-flow for the global and local refinement of policies and their adaptation based on Answer Set Programming (ASP) for policy representation and reasoning using ILASP. We then discuss an application of the AGENP framework and present preliminary results.

Genetic Algorithms (GAs) are known for their capacity to explore large search spaces and due to this ability, they were to some extent applied to Inductive Logic Programming (ILP) problem. Although Estimation of Distribution Algorithms (EDAs) perform better in most problems when compared to standard GAs, this kind of algorithm have not been applied to ILP. This work presents an ILP system based on EDA. Preliminary results show that the proposed system is superior when compared to a "standard" GA and it is very competitive when compared to the state of the art ILP system Aleph.

An important problem in pattern recognition is that of pattern classification. The objective of classification is to determine a discriminant function which is consistent with the given training examples and performs reasonably well on an unlabeled test set of examples. The degree of performance of the classifier on the test examples, known as its generalization performance, is an important issue in the design of the classifier. It has been established that a good generalization performance can be achieved by providing the learner with a sufficiently large number of discriminative training examples. However, in many domains, it is infeasible or expensive to obtain a sufficiently large training set. Various mechanisms have been proposed in literature to combat this problem. Active Learning techniques (Angluin, 1998; Seung, Opper, & Sompolinsky, 1992) reduce the number of training examples required by carefully choosing discriminative training examples. Bootstrapping (Efron, 1979; Hamamoto, Uchimura & Tomita, 1997) and other pattern synthesis techniques generate a synthetic training set from the given training set. We present some of these techniques and propose some general mechanisms for pattern synthesis. Request access from your librarian to read this chapter's full text.

Model transformation by example is a novel trend in model-driven software engineering. The rationale behind this is to utilize existing knowledge represented by source and target models of previously developed systems; such as requirements analysis and software design models, respectively. Such knowledge can be utilized to derive transformation rules to be applied in future system developments. To achieve this goal, machine learning techniques can assist in discovering and formalizing desired transformation rules. Inductive Logic Programming (ILP) represents a highly applicable machine learning technique in this context. Given a set of examples and background knowledge encoded as a set of first-order logic descriptions, an ILP system attempts to derive rules describing different transformation steps in a purely declarative way. The induced rules follow the same logical description as the given examples and background knowledge. The objective of this work is to introduce initial setup of an ILP system that can be utilized to derive analysis-design transformation rules from a set of examples that represent pairs of analysis-design models.

This paper discusses a simulation theory with learning agents which is serving as a formal specification to guide the development of GALATEA, a multi-agent simulation platform. We have extended an existing simulation language: GLIDER, with abstractions to model systems where autonomous entities (agents) perceive and act upon their environments. We are now applying it to the study of multi-agent systems. In particular, an implementation on Biocomplexity [1] is briefly discussed in the paper. We also show how an Inductive Logic Programming system can be used to learn rules in a representation very close to the one used to guide the simulation in the biocomplex system. This establishes the feasibility of embedding (resource-bounded) learners as agents that take part in simulating a complex system, as defined by the theory.

Inductive Logic Programming (ILP) is an established sub-field of Machine Learning. Nevertheless, it is recognized that efficiency and scalability is a major obstacle to an increased usage of ILP systems in complex applications with large hypotheses spaces. In this work, we focus on improving the efficiency and scalability of ILP systems by exploring tabling mechanisms available in the underlying Logic Programming systems. Tabling is an implementation technique that improves the declarativeness and performance of Prolog systems by reusing answers to subgoals. To validate our approach, we ran the April ILP system in the YapTab Prolog tabling system using two well-known datasets. The results obtained show quite impressive gains without changing the accuracy and quality of the theories generated.

Some inductive logic programming (ILP) systems use determinate literals to efficiently induce logic programs. A determinate literal literal is a literal that does not distinguish positive examples from negative examples, but produces information in variables introduced by the literal. The concept of determinate literals, however, is not reflected by the concept of input/output mode of predicate attributes properly, and so a system using determinate literals may induce inconsistent logic programs with predicate mode or inexecutable programs. The paper extends the concept of determinate literals and proposes input and output determinate literals. These literals function as pre-processor and post-processor against other literals. The paper also describes an implementation of the method and experimentations.