Inductive Logic Programming through Estimation of Distribution Algorithm

This paper is concerned with problems that arise when submitting large quantities of data to analysis by an Inductive Logic Programming (ILP) system. Complexity arguments usually make it prohibitive to analyse such datasets in their entirety. We examine two schemes that allow an ILP system to construct theories by sampling from this large pool of data. The first, “subsampling”, is a single-sample design in which the utility of a potential rule is evaluated on a randomly selected sub-sample of the data. The second, “logical windowing”, is multiple-sample design that tests and sequentially includes errors made by a partially correct theory. Both schemes are derived from techniques developed to enable propositional learning methods (like decision trees) to cope with large datasets. The ILP system CProgol, equipped with each of these methods, is used to construct theories for two datasets—one artificial (a chess endgame) and the other naturally occurring (a language tagging problem). In each case, we ask the following questions of CProgol equipped with sampling: (1) Is its theory comparable in predictive accuracy to that obtained if all the data were used (that is, no sampling was employed)?s and (2) Is its theory constructed in less time than the one obtained with all the data? For the problems considered, the answers to these questions is “yes”. This suggests that an ILP program equipped with an appropriate sampling method could begin to address problems satisfactorily that have hitherto been inaccessible simply due to data extent.

Inductive Logic Programming (ILP) systems aim to find a set of logical rules, called a hypothesis, that explain a set of examples. In cases where many such hypotheses exist, ILP systems often bias towards shorter solutions, leading to highly general rules being learned. In some application domains like security and access control policies, this bias may not be desirable, as when data is sparse more specific rules that guarantee tighter security should be preferred. This paper presents a new general notion of a scoring function over hypotheses that allows a user to express domain-specific optimisation criteria. This is incorporated into a new ILP system, called FastLAS, that takes as input a learning task and a customised scoring function, and computes an optimal solution with respect to the given scoring function. We evaluate the accuracy of FastLAS over real-world datasets for access control policies and show that varying the scoring function allows a user to target domain-specific performance metrics. We also compare FastLAS to state-of-the-art ILP systems, using the standard ILP bias for shorter solutions, and demonstrate that FastLAS is significantly faster and more scalable.

Probabilistic Inductive Logic Programming (PILP) systems extend ILP by allowing the world to be represented using probabilistic facts and rules, and by learning probabilistic theories that can be used to make predictions. However, such systems can be inefficient both due to the large search space inherited from the ILP algorithm and to the probabilistic evaluation needed whenever a new candidate theory is generated. To address the latter issue, this work introduces probability estimators aimed at improving the efficiency of PILP systems. An estimator can avoid the computational cost of probabilistic theory evaluation by providing an estimate of the value of the combination of two subtheories. Experiments are performed on three real-world datasets of different areas (biology, medical and web-based) and show that, by reducing the number of theories to be evaluated, the estimators can significantly shorten the execution time without losing probabilistic accuracy.

Model transformation by example is a novel trend in model-driven software engineering. The rationale behind this is to utilize existing knowledge represented by source and target models of previously developed systems; such as requirements analysis and software design models, respectively. Such knowledge can be utilized to derive transformation rules to be applied in future system developments. To achieve this goal, machine learning techniques can assist in discovering and formalizing desired transformation rules. Inductive Logic Programming (ILP) represents a highly applicable machine learning technique in this context. Given a set of examples and background knowledge encoded as a set of first-order logic descriptions, an ILP system attempts to derive rules describing different transformation steps in a purely declarative way. The induced rules follow the same logical description as the given examples and background knowledge. The objective of this work is to introduce initial setup of an ILP system that can be utilized to derive analysis-design transformation rules from a set of examples that represent pairs of analysis-design models.

We present a heuristic based algorithm to induce nonmonotonic logic programs that will explain the behavior of XGBoost trained classifiers. We use the technique based on the LIME approach to locally select the most important features contributing to the classification decision. Then, in order to explain the model’s global behavior, we propose the LIME-FOLD algorithm —a heuristic-based inductive logic programming (ILP) algorithm capable of learning nonmonotonic logic programs—that we apply to a transformed dataset produced by LIME. Our proposed approach is agnostic to the choice of the ILP algorithm. Our experiments with UCI standard benchmarks suggest a significant improvement in terms of classification evaluation metrics. Meanwhile, the number of induced rules dramatically decreases compared to ALEPH, a state-of-the-art ILP system.

Inductive Logic Programming: Issues, results and the challenge of Learning Language in Logic

In this paper, we present a learning simulator that behaves like a student in a CAI (computer aided instruction) system. We call this simulator a student agent. The agent consists of an interface, an inference engine and an inductive logic programming (ILP) system. By using a student agent in a CAI system, a human student can learn as if the student is in a group learning situation. Other possible usage of the learning simulator includes checking the behavior of CAI systems, and for a teacher to check contents for study by observing the response of the simulator. The learning simulator learns interactively. We implement the student agent, using a Prolog interpreter and ILP system FOIL. We show learning results obtained through computer simulations.

The human heart’s electrical activity produces currents that radiate through the surrounding tissue to the skin. When electrodes are attached to the skin, they sense these electrical currents and transmit them to an electrocardiograph monitor. The currents are then transformed into waveforms that represent the heart’s depolarisation - repolarisation cycle. An electro cardiogram (ECG) shows the precise sequence of electrical events occurring in the cardiac cells throughout the process. It can be used to monitor phases of myocardial contraction and to identify rhythm and conduction disturbances. In this work an automated ECG analysis and prediction of the heart conditions are carried out using a continuous image segmentation phase using Inductive Logic Programming (ILP) system. ECG taken from the patients is fed to the system as input and it uses the background knowledge to predict the possible underlying heart disorders. A continuous image segmentation module runs through out the process which identifies the leads and graph distortions by eliminating the outliers. The extracted lead graphs are compared with the in store ILP system database and possible predictions are made. The system performance was automatically compared (by Brier score method) with the MIT-BIH ECG database (PhysioNet) and a prediction accuracy of 97.8% was obtained in lead-lead prediction.

Despite the considerable success of Inductive Logic Programming (ILP), deployed ILP systems still have efficiency problems when applied to complex problems. Several techniques have been proposed to address the efficiency issue. Such proposals include query transformations, query packs, lazy evaluation and parallel execution of ILP systems, to mention just a few. We propose a novel technique that avoids the procedure of deducing each example to evaluate each constructed clause. The technique takes advantage of the two stage procedure of Mode Directed Inverse Entailment (MDIE) systems. In the first stage of a MDIE system, where the bottom clause is constructed, we store not only the bottom clause but also valuable additional information. The information stored is sufficient to evaluate the clauses constructed in the second stage without the need for a theorem prover. We used a data structure called Trie to efficiently store all bottom clauses produced using all examples (positive and negative) as seeds. The technique was implemented and evaluated using two well known data sets from the ILP literature. The results are promising both in terms of execution time and accuracy.KeywordsMode Directed Inverse EntailmentEfficiencyData Structures

The predicates used for Inductive Logic Programming (ILP) systems are usually elusive and need to be hand-crafted in advance, which limits the generalization of the system when learning new rules without sufficient background knowledge. Predicate Invention (PI) for ILP is the problem of discovering new concepts that describe hidden relationships in the domain. PI can mitigate the generalization problem for ILP by inferring new concepts, giving the system a better vocabulary to compose logic rules. Although there are several PI approaches for symbolic ILP systems, PI for Neuro-Symbolic-ILP (NeSy-ILP) systems that can handle 3D visual inputs to learn logical rules using differentiable reasoning is still unaddressed. To this end, we propose a neuro-symbolic approach, NeSy- π , to invent predicates from visual scenes for NeSy-ILP systems based on clustering and extension of relational concepts, where π denotes the abbrivation of P redicate I nvention. NeSy- π processes visual scenes as input using deep neural networks for the visual perception and invents new concepts that support the task of classifying complex visual scenes. The invented concepts can be used by any NeSy-ILP system instead of hand-crafted background knowledge. Our experiments show that the NeSy- π is capable of inventing high-level concepts and solving complex visual logic patterns efficiently and accurately in the absence of explicit background knowledge. Moreover, the invented concepts are explainable and interpretable, while also providing competitive results with state-of-the-art NeSy-ILP systems. (github: https://github.com/ml-research/NeSy-PI )

Inductive Logic Programming (ILP) is an established sub-field of Machine Learning. Nevertheless, it is recognized that efficiency and scalability is a major obstacle to an increased usage of ILP systems in complex applications with large hypotheses spaces. In this work, we focus on improving the efficiency and scalability of ILP systems by exploring tabling mechanisms available in the underlying Logic Programming systems. Tabling is an implementation technique that improves the declarativeness and performance of Prolog systems by reusing answers to subgoals. To validate our approach, we ran the April ILP system in the YapTab Prolog tabling system using two well-known datasets. The results obtained show quite impressive gains without changing the accuracy and quality of the theories generated.

A comparative study of hybrid estimation distribution algorithms in solving the facility layout problem

This paper discusses a simulation theory with learning agents which is serving as a formal specification to guide the development of GALATEA, a multi-agent simulation platform. We have extended an existing simulation language: GLIDER, with abstractions to model systems where autonomous entities (agents) perceive and act upon their environments. We are now applying it to the study of multi-agent systems. In particular, an implementation on Biocomplexity [1] is briefly discussed in the paper. We also show how an Inductive Logic Programming system can be used to learn rules in a representation very close to the one used to guide the simulation in the biocomplex system. This establishes the feasibility of embedding (resource-bounded) learners as agents that take part in simulating a complex system, as defined by the theory.

Estimation of distribution algorithm (EDA) is a new class of evolutionary algorithms with a wide range of real-world applications. However, it has been well known that the performance of EDA is not satisfactory enough if its population size is small. But to simply increase its population size may result in slow convergence. To the best knowledge of the authors', very few work has been done on improving the performance of EDA under small population size. This paper illustrates why EDA does not work well under small population size and proposes a novel approach termed as Over-Selection to boost EDA under small population size. Experimental results on several benchmark problems demonstrate that Over-Selection based EDA is often able to achieve a better solution without significantly increasing its time consumption when compared with the original version of EDA.

When machine learning programs from data, we ideally want to learn efficient rather than inefficient programs. However, existing inductive logic programming (ILP) techniques cannot distinguish between the efficiencies of programs, such as permutation sort (n!) and merge sort O(n;log;n). To address this limitation, we introduce Metaopt, an ILP system which iteratively learns lower cost logic programs, each time further restricting the hypothesis space. We prove that given sufficiently large numbers of examples, Metaopt converges on minimal cost programs, and our experiments show that in practice only small numbers of examples are needed. To learn minimal time-complexity programs, including non-deterministic programs, we introduce a cost function called tree cost which measures the size of the SLD-tree searched when a program is given a goal. Our experiments on programming puzzles, robot strategies, and real-world string transformation problems show that Metaopt learns minimal cost programs. To our knowledge, Metaopt is the first machine learning approach that, given sufficient numbers of training examples, is guaranteed to learn minimal cost logic programs, including minimal time-complexity programs.