Quality assessment of knowledge-based software: some certification considerations

Research in verification and validation (V&V) of knowledge based systems (KBS) has emerged as a distinct field in the last decade and is intended to address issues associated with KBS quality aspects and to credit such applications to the same degree as conventional applications. Part of the knowledge and techniques developed are now ready to be transferred to industry for the implementation of software certification. Certification is probably the most advanced use of the V&V methods, techniques and knowledge that have been developed so far. The paper approaches the challenging issues of certifying the knowledge base (KB) component of a KBS. KB certification uses a number of different certification methods such as static analysis, testing, inspection, and modeling to assess the quality of the KB component. Although these methods are currently used in KBS development, it is not yet clear how they would be applied by an independent certification organization to arrive at a pass/fail decision. The article discusses the components of KB certification and identifies application areas directly connected with industrial needs.

Chapter 49 - User Aspects Of Knowledge-Based Systems

This paper presents a framework called logical knowledge object (LKO), which is taken as a basis of the dependable development of knowledge based systems(KBSs). LKO combines logic programming and object-oriented programming paradigms, where objects are viewed as abstractions with states, constraints, behaviors and inheritance. The operational semantics defined in the style of natural semantics is simple and clear. A hybrid knowledge representation amalgamating rule, frame, semantic network and blackboard is available for both most structured and flat knowledge. The management of knowledge bases has been formally specified. Accordingly, LKO is well suited for the formal representation of knowledge and requirements of KBSs. Based on the framework, verification techniques are also explored to enhance the analysis of requirement specifications and the validation of KBSs. In addition, LKO provides a methodology for the development of KBSs, applying the concepts of rapid prototyping and top-down design to deal with changing and incomplete requirements, and to provide multiple abstract models of the domain, where formal methods might be used at each abstract level.

Ensuring reliability and enhancing quality of Knowledge Based Systems (KBS) are critical factors for their successful deployment in real-world applications. This is a broad task involving both methodological and formal approaches for designing rigorous Validation, Verification and Testing (VVT) methods and tools. Some of these can be adapted from conventional software engineering, while others rely on specific aspects of KBS.

This article discusses the selection of the domain for a knowledge-based expert system for a corporate application. The selection of the domain is a critical task in an expert system development. At the start of a project looking into the development of an expert system, the knowledge engineering project team must investigate one or several possible expert system domains. They must decide whether the selected application(s) are best suited to solution by present expert system technology, or if there might be a better way (or, possibly, no way) to attack the problems. If there are several possibilities, the team must also rank the potential applications and select the best available. To evaluate the potential of possible application domains, it has proved very useful to have a set of desired attributes for good expert domain. This article presents such a set of attributes. The attribute set was developed as part of a major expert system development project at GTE Laboratories. It was used recurrently (and modified and expanded continually) throughout an extensive application domain evaluation and selection process.

Aircraft electrical power system (EPS), in charge of supplying power to aircraft electric systems, needs a reconfiguration capability in order to overcome cases of EPS component failures. During the design phase, EPS reconfiguration elicitation is a complex task due to the high number of failure combinations and to the aeronautical operational constraints, which are to be taken into account. Therefore a knowledge-based system is developed in order to help designers for such a task. It is based on the implementation of two different types of rules: mandatory and know-how rules. EPS connectivity is managed by graph-theory algorithms. The knowledge-based system is capable of producing several reconfiguration solutions. The knowledge-based system functioning is illustrated in detail on a simplified EPS. Results on an actual and modern EPS are also presented.

In recent years, the KDD process has been advocated to be an iterative and interactive process. It is seldom the case that a user is able to answer immediately all his questions on date with a single query. On the contrary, the work-flow of the typical user consists of several steps in which he/she iteratively refines the extracted knowledge by inspecting previous results and posing new queries. Given this view of the KDD process, in order to reduce the computational effort, it becomes crucial to have KDD systems that are able to exploit past results. This is especially true in environments in which the system knowledge base is the result of many discoveries on data made separately by the collaborative effort of different users. In this paper, we consider the problem of mining frequent association rules from database relations. We first model a general, constraint-based, mining language for this task. Then, we propose an algorithm that answers such queries reusing past results. In particular, this solution is effective for a new class of constraints, called context dependent, which are more difficult than the traditionally studied item dependent constraints. Nevertheless, we show that some typical queries of important application domains, such as market stock trading, analysis of web log, and gene microarrays in bioinformatics, have context-dependent constraints. We show with a set of experiments in these application domains that the proposed solution with an incremental approach is both effective and viable.

Research in artificial intelligence (AI) has been concerned with the development of systems that would normally be associated with human intelligence. The domains of application were initially devoted to natural-language processing (NLP), robotics and expert systems. Natural-language processing was concerned with how to communicate with a system through the computer, using text or voice input. Robotics was associated with human behavior in movement in an environment; that entails visual and tactile action. A relatively more recent application can be found in the development of programs that simulate the decision-making behavior of human experts using knowledge of a specific domain of expertise. The term “expert systems” (Harmon & King 1985) was defined by Edward Feigenbaum as “an intelligent computer program that uses knowledge and inference procedures to solve problems that are difficult enough to require significant human expertise for their solution.” Since the seventies, several systems have been developed with this objective. They are called “knowledge-based systems”. However, although some of them contain respectable knowledge bases, their behavior can not be compared to the decision making of a human expert. Sometimes, systems with a sufficient amount of expertise have been called knowledge-based systems with qualification for reasons of modesty.

Handling exceptions in a knowledge-based system is an important issue in many application domains, such as medical domain. Recently, there is an increasing interest in nonmonotonic extension of description logics to handle exceptions in ontologies. In this paper, we propose three preferential semantics for plausible subsumption to deal with exceptions in description logic-based knowledge bases. Our preferential semantics are defined in the framework of possibility theory, which is an uncertainty theory devoted to handling incomplete information. We consider the properties of these semantics and their relationships. We also discuss the relationship between two of our preferential semantics and two existing preferential semantics. We extend a description logic-based knowledge base by adding preferential subsumptions. Entailment of plausible subsumptions relative to an extended knowledge base is defined. Properties of the preferential subsumption relations relative to an extended description logic-based knowledge base are discussed. Finally, we show that our semantics for plausible subsumption can be reduced to standard semantics of an expressive description logic. Thus, the problem of plausible subsumption checking under our semantics can be reduced to the problem of subsumption checking under the classical semantics.

Knowledge management (KM) is a combination of several techniques and processes for creation, sharing and dissemination of knowledge within an enterprise. According to published reports, KM is a key ingredient in any successful customer support service business. It impacts positively the productivity of support personnel, improves the quality of their decision making, enables the extraction of knowledge from various repositories, and represents the core of automatic problem solving. The majority of the known KM systems deliver services in an open-loop fashion, where the impact of the system deployment on users or on the application domain is not known or controlled. In this paper, we report a knowledge-based system (KBS) that was deployed within Hewlett Packard Corporation to deliver customer support services. This system differs from others in that the monitoring loop is closed. This enables the support system to adapt based on observed usage patterns and evolving needs.

Knowledge Representation and Reasoning is at the heart of the great challenge of Artificial Intelligence, especially knowledge-based systems, expert systems and intelligent problem solvers. The models for knowledge such as logics, semantic networks, conceptual graphs, frames and classes are useful tools to design the above systems. However, they are not suitable to represent knowledge in the domains of reality applications. Designing of the knowledge bases and the inference engines of those systems in practice requires representations in the form of ontologies. For applications such as the intelligent problem solver in plane geometry, the knowledge contains a complicated system of concepts, relations, operators, functions, and rules. This situation motivates an ontology-based solution with such components. In this article, an ontology called Ontology of Computational Object Knowledge Base (COKB), will be presented in details. We also present a model for representing problems together reasoning algorithms for solving them, and design methods to construct applications. The above methodology has been used in designing many applications such as systems for solving analytic geometry problems, solving problems in plane geometry, and the expert system for diabetic micro vascular complication diagnosis.

Using results from a controlled experiment and simulations based on cognitive models, we show that visual presentation style can have a significant impact on performance in a complex problem-solving task. We compared subject performances in two isomorphic, but visually different, tasks based on a card game of SET. Although subjects used the same strategy in both tasks, the difference in presentation style resulted in radically different reaction times and significant deviations in scanpath patterns in the two tasks. Results from our study indicate that low-level subconscious visual processes, such as differential acuity in peripheral vision and low-level iconic memory, can have indirect, but significant effects on decision making during a problem-solving task. We have developed two ACT-R models that employ the same basic strategy but deal with different presentations styles. Our ACT-R models confirm that changes in low-level visual processes triggered by changes in presentation style can propagate to higher-level cognitive processes. Such a domino effect can significantly affect reaction times and eye movements, without affecting the overall strategy of problem solving.

Simple elements of mathematical proof were analyzed by considering the making of a proof as a complex problem-solving task. Quantitative measures of performance on certain problem-solving tasks were obtained in order to determine the relation between problem-solving ability and chronological age (6-18). A consideration of all variables suggested that, even though problem-solving ability does increase with age, certain aspects of problem solving that approximate mathematical proof can be dealt with successfully in the upper elementary grades. An analysis of problem-solving strategies is recommended as a further step in the investigation of problem solving. IQ, reading ability, and mathematical aptitude are suggested as possible covariates or blocking variables in future research.

This article describes empirical research results regarding the "history effect" in software quality evaluation processes. Most software quality models and evaluation processes models assume that software quality may be deterministically evaluated, especially when it is evaluated by experts. Consequently, software developers focus on the technical characteristics of the software product. A similar assumption is common in most engineering disciplines. However, in regard to other kinds of goods, direct violations of the assumption about objective evaluation were shown to be affected by the consequences of cognitive processes limitations. Ongoing discussion in the area of behavioral economics raises the question: are the experts prone to observation biases? If they are, then software quality models overlook an important aspect of software quality evaluation. This article proposes an experiment that aims to trace the influence of users' knowledge on software quality assessment. Measuring the influence of single variables for the software quality perception process is a complex task. There is no valid quality model for the precise measurement of product quality, and consequently software engineering does not have tools to freely manipulate the quality level for a product. This article proposes a simplified method to manipulate the observed quality level, thereby making it possible to conduct research. The proposed experiment has been conducted among professional software evaluators. The results show the significant negative influence (large effect size) of negative experience of users on final opinion about software quality regardless of its actual level.

Thermal manufacturing processes are typically those in which a material is subjected to a temporal thermal cycle like casting, extrusion and heat treatment of metals and plastics. The complexity of the design process for all these systems stems from the need to simulate complicated heat transfer, fluid flow and phase change phenomena and couple the results with the design rules and knowledge available on the manufacturing processes to obtain satisfactory designs. In this regard, the ability of expert systems to use heuristic reasoning has proved to be a powerful tool in the computer-aided-design of thermal manufacturing systems. In this paper, the salient features of a knowledge-based system developed for the design of ingot casting process has been outlined. A Prolog based decision making front-end is interfaced with a Fortran based computational engine for rapid design. The results from the heat transfer analysis obtained from the computational module, are coupled to the evaluation module, which checks for satisfaction of the design criteria and violation of the design constraints. The decision making module uses a set of design rules to manipulate the variables until the design specifications are satisfied. Modularity and flexibility are maintained using an object-oriented format. Several interesting design acceleration features like learning from simple mathematical models and design extraction from previous designs are illustrated. The main features of this knowledge-based tool and the savings in time resulting from using these special features are discussed in detail.