Neuro-symbolic Predicate Invention: Learning relational concepts from visual scenes

Inductive Logic Programming (ILP) systems aim to find a set of logical rules, called a hypothesis, that explain a set of examples. In cases where many such hypotheses exist, ILP systems often bias towards shorter solutions, leading to highly general rules being learned. In some application domains like security and access control policies, this bias may not be desirable, as when data is sparse more specific rules that guarantee tighter security should be preferred. This paper presents a new general notion of a scoring function over hypotheses that allows a user to express domain-specific optimisation criteria. This is incorporated into a new ILP system, called FastLAS, that takes as input a learning task and a customised scoring function, and computes an optimal solution with respect to the given scoring function. We evaluate the accuracy of FastLAS over real-world datasets for access control policies and show that varying the scoring function allows a user to target domain-specific performance metrics. We also compare FastLAS to state-of-the-art ILP systems, using the standard ILP bias for shorter solutions, and demonstrate that FastLAS is significantly faster and more scalable.

This paper is concerned with problems that arise when submitting large quantities of data to analysis by an Inductive Logic Programming (ILP) system. Complexity arguments usually make it prohibitive to analyse such datasets in their entirety. We examine two schemes that allow an ILP system to construct theories by sampling from this large pool of data. The first, “subsampling”, is a single-sample design in which the utility of a potential rule is evaluated on a randomly selected sub-sample of the data. The second, “logical windowing”, is multiple-sample design that tests and sequentially includes errors made by a partially correct theory. Both schemes are derived from techniques developed to enable propositional learning methods (like decision trees) to cope with large datasets. The ILP system CProgol, equipped with each of these methods, is used to construct theories for two datasets—one artificial (a chess endgame) and the other naturally occurring (a language tagging problem). In each case, we ask the following questions of CProgol equipped with sampling: (1) Is its theory comparable in predictive accuracy to that obtained if all the data were used (that is, no sampling was employed)?s and (2) Is its theory constructed in less time than the one obtained with all the data? For the problems considered, the answers to these questions is “yes”. This suggests that an ILP program equipped with an appropriate sampling method could begin to address problems satisfactorily that have hitherto been inaccessible simply due to data extent.

Model transformation by example is a novel trend in model-driven software engineering. The rationale behind this is to utilize existing knowledge represented by source and target models of previously developed systems; such as requirements analysis and software design models, respectively. Such knowledge can be utilized to derive transformation rules to be applied in future system developments. To achieve this goal, machine learning techniques can assist in discovering and formalizing desired transformation rules. Inductive Logic Programming (ILP) represents a highly applicable machine learning technique in this context. Given a set of examples and background knowledge encoded as a set of first-order logic descriptions, an ILP system attempts to derive rules describing different transformation steps in a purely declarative way. The induced rules follow the same logical description as the given examples and background knowledge. The objective of this work is to introduce initial setup of an ILP system that can be utilized to derive analysis-design transformation rules from a set of examples that represent pairs of analysis-design models.

Computing similarity in high-dimensional vector spaces is a long-standing problem that has recently seen significant progress with the invention of the word2vec algorithm. Usually, it has been found that using an embedded representation results in much better performance for the task being addressed. It is not known whether embeddings can similarly improve performance with data of the kind considered by Inductive Logic Programming (ILP), in which data apparently dissimilar on the surface, can be similar to each other given domain (background) knowledge. In this paper, using several ILP classification benchmarks, we investigate if embedded representations are similarly helpful for problems where there is sufficient amounts of background knowledge. We use tasks for which we have domain expertise about the relevance of background knowledge available and consider two subsets of background predicates (“sufficient” and “insufficient”). For each subset, we obtain a baseline representation consisting of Boolean-valued relational features. Next, a vector embedding specifically designed for classification is obtained. Finally, we examine the predictive performance of widely-used classification methods with and without the embedded representation. With sufficient background knowledge we find no statistical evidence for an improved performance with an embedded representation. With insufficient background knowledge, our results provide empirical evidence that for the specific case of using deep networks, an embedded representation could be useful.

Inductive Logic Programming: Issues, results and the challenge of Learning Language in Logic

The human heart’s electrical activity produces currents that radiate through the surrounding tissue to the skin. When electrodes are attached to the skin, they sense these electrical currents and transmit them to an electrocardiograph monitor. The currents are then transformed into waveforms that represent the heart’s depolarisation - repolarisation cycle. An electro cardiogram (ECG) shows the precise sequence of electrical events occurring in the cardiac cells throughout the process. It can be used to monitor phases of myocardial contraction and to identify rhythm and conduction disturbances. In this work an automated ECG analysis and prediction of the heart conditions are carried out using a continuous image segmentation phase using Inductive Logic Programming (ILP) system. ECG taken from the patients is fed to the system as input and it uses the background knowledge to predict the possible underlying heart disorders. A continuous image segmentation module runs through out the process which identifies the leads and graph distortions by eliminating the outliers. The extracted lead graphs are compared with the in store ILP system database and possible predictions are made. The system performance was automatically compared (by Brier score method) with the MIT-BIH ECG database (PhysioNet) and a prediction accuracy of 97.8% was obtained in lead-lead prediction.

Probabilistic Inductive Logic Programming (PILP) systems extend ILP by allowing the world to be represented using probabilistic facts and rules, and by learning probabilistic theories that can be used to make predictions. However, such systems can be inefficient both due to the large search space inherited from the ILP algorithm and to the probabilistic evaluation needed whenever a new candidate theory is generated. To address the latter issue, this work introduces probability estimators aimed at improving the efficiency of PILP systems. An estimator can avoid the computational cost of probabilistic theory evaluation by providing an estimate of the value of the combination of two subtheories. Experiments are performed on three real-world datasets of different areas (biology, medical and web-based) and show that, by reducing the number of theories to be evaluated, the estimators can significantly shorten the execution time without losing probabilistic accuracy.

The example semantics of Inductive Logic Programming (ILP) systems is said to be in a rough setting when the consistency and completeness criteria cannot both be fulfilled together, because the evidence, background knowledge and declarative bias are such that any induced hypothesis cannot distinguish between some of the positive and negative examples. The gRS-ILP model (generic Rough Set Inductive Logic Programming model) provides a theoretical foundation in this rough setting for an ILP system to induce hypotheses that are used to say that an example is definitely positive, or definitely negative. An illustrative example using Progol is presented. Results are presented of GOLEM experiments using the data set for drug design for Alzheimer's disease and other experiments using Progol on mutagenesis data and transmembrane domain data.

In this paper, we present a learning simulator that behaves like a student in a CAI (computer aided instruction) system. We call this simulator a student agent. The agent consists of an interface, an inference engine and an inductive logic programming (ILP) system. By using a student agent in a CAI system, a human student can learn as if the student is in a group learning situation. Other possible usage of the learning simulator includes checking the behavior of CAI systems, and for a teacher to check contents for study by observing the response of the simulator. The learning simulator learns interactively. We implement the student agent, using a Prolog interpreter and ILP system FOIL. We show learning results obtained through computer simulations.

Genetic Algorithms (GAs) are known for their capacity to explore large search spaces and due to this ability, they were to some extent applied to Inductive Logic Programming (ILP) problem. Although Estimation of Distribution Algorithms (EDAs) perform better in most problems when compared to standard GAs, this kind of algorithm have not been applied to ILP. This work presents an ILP system based on EDA. Preliminary results show that the proposed system is superior when compared to a "standard" GA and it is very competitive when compared to the state of the art ILP system Aleph.

The identification of the correct sense of a word is necessary for many tasks in automatic natural language processing like machine translation, information retrieval, speech and text processing. Automatic Word Sense Disambiguation (WSD) is difficult and accuracies with state-of-the art methods are substantially lower than in other areas of text understanding like part-of-speech tagging. One shortcoming of these methods is that they do not utilize substantial sources of background knowledge, such as semantic taxonomies and dictionaries, which are now available in electronic form (the methods largely use shallow syntactic features). Empirical results from the use of Inductive Logic Programming (ILP) have repeatedly shown the ability of ILP systems to use diverse sources of background knowledge. In this paper we investigate the use of ILP for WSD in two different ways: (a) as a stand-alone constructor of models for WSD; and (b) to build interesting features, which can then be used by standard model-builders such as SVM. In our experiments we examine a monolingual WSD task using the 32 English verbs contained in the SENSEVAL-3 benchmark data; and a bilingual WSD task using 7 highly ambiguous verbs in machine translation from English to Portuguese. Background knowledge available is from eight sources that provide a wide range of syntactic and semantic information. For both WSD tasks, experimental results show that ILP-constructed models and models built using ILP-generated features have higher accuracies than those obtained using a state-of-the art feature-based technique equipped with shallow syntactic features. This suggests that the use of ILP with diverse sources of background knowledge can provide one way for making substantial progress in the field of automatic WSD.

Automating the learning of context specific robust rules from an evolving scene has been a research challenge in the area of cognitive vision systems. A research has been conducted to develop a system that learns context specific rules from an evolving scene by abstracting a model of human visual learning. Our system treats a set of symbolic data generated from an evolving real world scene and background knowledge as input to the system and inductive logic programming techniques are used to learn rules of the scene. The observed visual scene is represented in terms of qualitative spatial and temporal relations and these learnt relations are considered as input examples for inductive learning. A prototype has been developed for learning from various scenes of setting dinner tables. Currently the system is being tested to incorporate learning from different real world scenes thus improving the generalization power as well as combine more spatial and temporal representation and reasoning mechanisms to further enhance human like learning. This work can be adopted in automating a robot learning of object manipulation based on a visual scene

This paper presents a case study of a machine-aided knowledge discovery process within the general area of drug design. Within drug design, the particular problem of pharmacophore discovery is isolated, and the Inductive Logic Programming (ILP) system progol is applied to the problem of identifying potential pharmacophores for ACE inhibition. The case study reported in this paper supports four general lessons for machine learning and knowledge discovery, as well as more specific lessons for pharmacophore discovery, for Inductive Logic Programming, and for ACE inhibition. The general lessons for machine learning and knowledge discovery are as follows. 1. An initial rediscovery step is a useful tool when approaching a new application domain. 2. General machine learning heuristics may fail to match the details of an application domain, but it may be possible to successfully apply a heuristic-based algorithm in spite of the mismatch. 3. A complete search for all plausible hypotheses can provide useful information to a user, although experimentation may be required to choose between competing hypotheses. 4. A declarative knowledge representation facilitates the development and debugging of background knowledge in collaboration with a domain expert, as well as the communication of final results.

We present a heuristic based algorithm to induce nonmonotonic logic programs that will explain the behavior of XGBoost trained classifiers. We use the technique based on the LIME approach to locally select the most important features contributing to the classification decision. Then, in order to explain the model’s global behavior, we propose the LIME-FOLD algorithm —a heuristic-based inductive logic programming (ILP) algorithm capable of learning nonmonotonic logic programs—that we apply to a transformed dataset produced by LIME. Our proposed approach is agnostic to the choice of the ILP algorithm. Our experiments with UCI standard benchmarks suggest a significant improvement in terms of classification evaluation metrics. Meanwhile, the number of induced rules dramatically decreases compared to ALEPH, a state-of-the-art ILP system.

Despite the considerable success of Inductive Logic Programming (ILP), deployed ILP systems still have efficiency problems when applied to complex problems. Several techniques have been proposed to address the efficiency issue. Such proposals include query transformations, query packs, lazy evaluation and parallel execution of ILP systems, to mention just a few. We propose a novel technique that avoids the procedure of deducing each example to evaluate each constructed clause. The technique takes advantage of the two stage procedure of Mode Directed Inverse Entailment (MDIE) systems. In the first stage of a MDIE system, where the bottom clause is constructed, we store not only the bottom clause but also valuable additional information. The information stored is sufficient to evaluate the clauses constructed in the second stage without the need for a theorem prover. We used a data structure called Trie to efficiently store all bottom clauses produced using all examples (positive and negative) as seeds. The technique was implemented and evaluated using two well known data sets from the ILP literature. The results are promising both in terms of execution time and accuracy.KeywordsMode Directed Inverse EntailmentEfficiencyData Structures