Combining Connectionist and Symbolic Learning to Refine Certainty Factor Rule Bases

Domain-specific ontologies have shown their powerful usefulness in many application areas, such as semantic web, information sharing, and natural language processing. However, manually building of domain ontologies still remains a tedious and cumbersome task. Hyponymy is a core component of domain-specific ontologies. In this paper, we propose three symbolic learning methods, which are integrated together to extract hyponymies from un-annotated domain-specific Chinese free texts. The three symbolic learning methods include seed-driven learning, pattern-mediated learning, and term composition based learning. Experimental results show that the algorithm is adequate to extracting the hyponymies from unstructured domain-specific Chinese corpus.

Despite the recent success of end-to-end deep neural networks, there are growing concerns about their lack of logical reasoning abilities, especially on cognitive tasks with perception and reasoning processes. A solution is the neural symbolic learning (NeSyL) method that can effectively utilize pre-defined logic rules to constrain the neural architecture making it perform better on cognitive tasks. However, it is challenging to apply NeSyL to these cognitive tasks because of the lack of supervision, the non-differentiable manner of the symbolic system, and the difficulty to probabilistically constrain the neural network. In this paper, we propose WS-NeSyL, a weakly supervised neural symbolic learning model for cognitive tasks with logical reasoning. First, WS-NeSyL employs a novel back search algorithm to sample the possible reasoning process through logic rules. This sampled process can supervise the neural network as the pseudo label. Based on this algorithm, we can backpropagate gradients to the neural network of WS-NeSyL in a weakly supervised manner. Second, we introduce a probabilistic logic regularization into WS-NeSyL to help the neural network learn probabilistic logic. To evaluate WS-NeSyL, we have conducted experiments on three cognitive datasets, including temporal reasoning, handwritten formula recognition, and relational reasoning datasets. Experimental results show that WS-NeSyL not only outperforms the end-to-end neural model but also beats the state-of-the-art neural symbolic learning models.

Information extraction and human collaboration techniques are widely applied in the construction of web-scale knowledge bases. However, these knowledge bases are often incomplete or uncertain. In this paper, we present ProbKB, a probabilistic knowledge base designed to infer missing facts in a scalable, probabilistic, and principled manner using a relational DBMS. The novel contributions we make to achieve scalability and high quality are: 1) We present a formal definition and a novel relational model for probabilistic knowledge bases. This model allows an efficient SQL-based inference algorithm for knowledge expansion that applies inference rules in batches; 2) We implement ProbKB on massive parallel processing databases to achieve further scalability; and 3) We combine several quality control methods that identify erroneous rules, facts, and ambiguous entities to improve the precision of inferred facts. Our experiments show that ProbKB system outperforms the state-of-the-art inference engine in terms of both performance and quality.

The technique of a new extension of fuzzy rough theory using partition of interval set-valued is proposed for granular computing during knowledge discovery in this paper. The natural intervals of attribute values in decision system to be transformed into multiple sub-interval of [0,1]are given by normalization. And some characteristics of interval set-valued of decision systems in fuzzy rough set theory are discussed. The correctness and effectiveness of the approach are shown in experiments. The approach presented in this paper can also be used as a data preprocessing step for other symbolic knowledge discovery or machine learning methods other than rough set theory.

There is an increasing need to develop appropriate techniques for merging probabilistic knowledge bases (PKB) in knowledge-based systems. To deal with merging problems, several approaches have been put forward. However, in the proposed models, the representation of the merged probabilistic knowledge base is not similar to the representation of original knowledge bases. The drawback of the solutions is that probabilistic constraints on the set of input knowledge bases must have the same structure and there is no algorithm for implementing the merging process. In this paper, we proposed two algorithms for merging probabilistic knowledge bases represented by various structures. To this aim, the method of constraint deduction is investigated, a set of mean merging operators is proposed and several desirable logical properties are presented and discussed. These are the basis for building algorithms. The complexity of algorithms as well as related propositions are also analysised and discussed.

The field of 'DNA linguistics' has emerged from pioneering work in computational linguistics and molecular biology. Most formal grammars in this field are expressed using Definite Clause Grammars but these have computational limitations which must be overcome. The present study provides a new DNA parsing system, comprising a logic grammar formalism called Basic Gene Grammars and a bidirectional chart parser DNA-ChartParser. The use of Basic Gene Grammars is demonstrated in representing many formulations of the knowledge of Escherichia coli promoters, including knowledge acquired from human experts, consensus sequences, statistics (weight matrices), symbolic learning, and neural network learning. The DNA-ChartParser provides bidirectional parsing facilities for BGGs in handling overlapping categories, gap categories, approximate pattern matching, and constraints. Basic Gene Grammars and the DNA-ChartParser allowed different sources of knowledge for recognizing E.coli promoters to be combined to achieve better accuracy as assessed by parsing these DNA sequences in real-world data sets.

We present a skill learning model CLARION. Different from existing models of high-level skill learning that use a top-down approach (that is, turning declarative knowledge into procedural knowledge), we adopt a bottom-up approach toward low-level skill learning, where procedural knowledge develops first and declarative knowledge develops later. Clarion is formed by integrating connectionist, reinforcement, and symbolic learning methods to perform on-line learning. We compare the model with human data in a minefield navigation task. A match between the model and human data is found in several respects.

9 - Knowledge Acquisition Via Bottom-up Learning

Acquisition of Qualia Elements from Corpora - Evaluation of a Symbolic Learning Method

The fourth edition of this bestselling textbook explains the principles of artificial intelligence (AI) and its practical applications. Using clear and concise language, it provides a solid grounding across the full spectrum of AI techniques, so that its readers can implement systems in their own domain of interest. The coverage includes knowledge-based intelligence, computational intelligence (including machine learning), and practical systems that use a combination of techniques. All the key techniques of AI are explained—including rule-based systems, Bayesian updating, certainty theory, fuzzy logic (types 1 and 2), agents, objects, frames, symbolic learning, case-based reasoning, genetic algorithms and other optimization techniques, shallow and deep neural networks, hybrids, and the Lisp, Prolog, and Python programming languages. The book also describes a wide range of practical applications in interpretation and diagnosis, design and selection, planning, and control. Fully updated and revised, Intelligent Systems for Engineers and Scientists: A Practical Guide to Artificial Intelligence, Fourth Edition features: A new chapter on deep neural networks, reflecting the growth of machine learning as a key technique for AI A new section on the use of Python, which has become the de facto standard programming language for many aspects of AI The rule-based and uncertainty-based examples in the book are compatible with the Flex toolkit by Logic Programming Associates (LPA) and its Flint extension for handling uncertainty and fuzzy logic. Readers of the book can download this commercial software for use free of charge. This resource and many others are available at the author’s website: adrianhopgood.com. Whether you are building your own intelligent systems, or you simply want to know more about them, this practical AI textbook provides you with detailed and up-to-date guidance.

Computational Intelligence is a relatively new branch of science which, to some extent, can be regarded as a successor of “traditional” Artificial Intelligence. Unlike AI, which relies on symbolic learning and heuristic approaches to problem solving, CI mainly involves systems that are inspired from nature, such as (artificial) neural networks, evolutionary computation, fuzzy systems, chaos theory, probabilistic methods, swarm intelligence, ant systems, and artificial immune systems. In a wider perspective CI includes also part of machine learning, in particular the reinforcement learning methods. Applying either one or a combination of the above-mentioned disciplines allows implementation of the elements of learning and adaptation in the proposed solutions which make such systems somehow intelligent. In the game playing domain the most popular CI disciplines are neural networks, evolutionary and neuro-evolutionary methods, and reinforcement learning. These domains are briefly introduced in the remainder of this chapter along with sample game-related applications. The focus of the presentation is on the aspects of learning and autonomous development. Relevant literature is provided for possible further reading.

Resolving the inconsistency that appears in knowledge bases is an extremely important stage in the process of merging knowledge bases. With probabilistic context, many studies have solved the inconsistency of a probabilistic knowledge base (PKB) with different approaches. However, these previous focus on addressing the problem of restoring consistency in a PKB by employing the principles of maximum entropy or building the consistency restorer with complex creeping functions. We discover that it is really difficult to find the functions that are utilized in the creeping consistency restorers. Therefore, the process of finding these functions may lead to a poor performance. Based on that, we proposed the inconsistency solvers for a PKB, where we consider revising the functions involved in handling the inconsistency in a simpler direction. Two the inconsistency solvers are the equitable deformation inconsistency solver and the amerced deformation inconsistency solver. Corresponding to each solver, an algorithm to solve inconsistencies in a PKB is proposed. Moreover, the cost of each algorithm also are considered and proved.

The probabilistic Description Logic Open image in new window extends the classical Description Logic Open image in new window with probabilistic conditionals of the form (D|C)[p] stating that “D follows from C with probability p.” Conditionals are interpreted based on the aggregating semantics where probabilities are understood as degrees of belief. For reasoning with probabilistic conditional knowledge bases in Open image in new window , the principle of maximum entropy provides a valuable methodology following the idea of completing missing information in a most cautious way. In this paper, we give a guidance on calculating approximations of the maximum entropy distribution for Open image in new window -knowledge bases. For this, we discuss the benefits of solving the dual maximum entropy optimization problem instead of the primal problem. In particular, we show how representations of approximations of the maximum entropy distribution can be derived from the dual problem in time polynomial in the size of the underlying domain. The domain is the crucial quantity in practical applications. For a compact representation of the objective function of the dual maximum entropy optimization problem, we apply the principle of typed model counting.

In defense of the maximum entropy inference process

Several different ways of using symbolic methods to improve reinforcement learning are identified and discussed in some detail. Each demonstrates to some extent the advantages of combining reinforcement learning and symbolic methods. These methods point to the potentials and the challenges of this line of research.