Exploiting Users’ Feedbacks - Towards a Task-based Evaluation of Application Ontologies Throughout Their Lifecycle

Many ontologies have been developed recently, and evaluation is important for researchers or users when searching ontologies needed in their work. Ontology evaluation also is important phase of ontology development, refinement or evolution process. Automatically-developed ontologies as a result of ontology learning also need from evaluation. So, ontology evaluation is important both for selecting ontologies, meeting some requirements, and as a part of ontology life cycle. In this paper we propose deep analysis of ontology evaluation and discuss how multilingualism affects ontology evaluation. We classify ontology evaluation approaches, methods and metrics according to several dimensions. Our analysis and conclusions will be helpful for development of good ontologies and for finding or selection of needed ontologies.

Mobile applications are expected to receive context input Mobile applications are expected to receive context in put such as location, speech, and network from different context providers. Since context can be considered as knowledge, a formal method is needed to capture this knowledge. There isless work on ontology model that could be reused to model a new context ontology for Android mobile application. Therefore, this study proposed an ontology specifically for Android mobile application, COCCC, to formalize context knowledge present within it. METHONTOLOGY method was used to create COCCC ontology as it offers intermediate representation in the form of concepts. The concepts from the context ontology were extracted from various resources, sorted and categorized based on types and functions for standardization purposes. Survey was given to five domain experts for evaluation of COCCC ontology in terms of its usability. Data from these experts were analyzed and the results have confirmed that the proposed context ontology is usable to Android mobile application developers.

The Gene Ontology (GO) provides a framework to systematically classify and annotate gene function. The annotations associated with GO play a critical role in modern biology and cover many organisms. For the human genome, over 10,000 GO terms are used to annotate gene function in an expansive database of over 200,000 annotations. Due to the importance of the GO annotations in modern biology, significant effort has been put into assessing the quality of the annotations. Providing measures of annotation completeness, accuracy, and precision is critical if researchers are to use the annotations in real-world applications with confidence. Here, we describe a task-based approach that examines the completeness and utility of GO annotations through the lens of gene enrichment analysis. Our approach can be used to model the progression of the GO annotations over time, either for a particular area of interest or for the body of annotations as a whole. Using this framework, we conducted a large-scale analysis of gene expression datasets from the NCBI Gene Expression Omnibus (GEO). In particular, we identified terms of interest for each dataset through semantic annotation of biomedical data, then tracked the behavior of these terms as a function of time. The preliminary results provide significant information about the progress and character of GO annotations over time. This framework is flexible enough to examine all or part of the GO annotations, across multiple species, and with various enrichment methods. We also discuss how this framework can be used to evaluate different annotation methods. For example, by comparing the performance of annotations generated with a particular method to the performance of canonical annotations, it is possible to determine their relative quality.

Ontology evaluation is required before using the ontology within applications. Similar with software practice, the purpose of ontology evaluation is to identify the achievement of requirement criteria. Users who require coverage criteria often seeking ontology that contain the terms related to their focused domain knowledge. Users encounter the difficulty to select a suitable ontology from variety of ontology evaluation approaches. Conceptualization of information related to ontology evaluation helps to identify the important component within ontology that helps towards coverage criteria achievement. This work proposes an algorithm to extract ontology documents gained from public ontology repositories like Falcons into its vocabulary parts focused on classes and literals. The algorithm then processes the extracted ontology components with similarity algorithm and later displays the result on the coverage match of ontology with provided terms and the terms that are synonym expanded using WordNet.

In this article, we propose a metric for the automatic evaluation of restricted domain ontologies. The metric is defined in terms of the evaluation of different lexico-syntactic, statistical and semantic approaches. A syntactic approach employed is the use of lexical syntactic patterns, other approaches as grouping by formal concept analysis, similarity, latent semantic analysis and dependence graphs are used as well. These approaches focus on reference corpora to find evidence of the validity of concepts and semantic relationships stored in the target ontology. The proposed evaluation approach is able to provide a score obtained through the metric, which is based on the accuracy measure used for each ontology evaluated. The score is associated in some way with the ontology quality. This score is given with a certain degree of reliability, and it is obtained by comparing the results given against the evaluation of human experts and a baseline.

The spectacular radiation of insects has produced a stunning diversity of phenotypes. During the past 250 years, research on insect systematics has generated hundreds of terms for naming and comparing them. In its current form, this terminological diversity is presented in natural language and lacks formalization, which prohibits computer-assisted comparison using semantic web technologies. Here we propose a Model for Describing Cuticular Anatomical Structures (MoDCAS) which incorporates structural properties and positional relationships for standardized, consistent, and reproducible descriptions of arthropod phenotypes. We applied the MoDCAS framework in creating the ontology for the Anatomy of the Insect Skeleto-Muscular system (AISM). The AISM is the first general insect ontology that aims to cover all taxa by providing generalized, fully logical, and queryable, definitions for each term. It was built using the Ontology Development Kit (ODK), which maximizes interoperability with Uberon (Uberon multispecies anatomy ontology) and other basic ontologies, enhancing the integration of insect anatomy into the broader biological sciences. A template system for adding new terms, extending, and linking the AISM to additional anatomical, phenotypic, genetic, and chemical ontologies is also introduced. The AISM is proposed as the backbone for taxon-specific insect ontologies and has potential applications spanning systematic biology and biodiversity informatics, allowing users to: 1) use controlled vocabularies and create semiautomated computer-parsable insect morphological descriptions; 2) integrate insect morphology into broader fields of research, including ontology-informed phylogenetic methods, logical homology hypothesis testing, evo-devo studies, and genotype to phenotype mapping; and 3) automate the extraction of morphological data from the literature, enabling the generation of large-scale phenomic data, by facilitating the production and testing of informatic tools able to extract, link, annotate, and process morphological data. This descriptive model and its ontological applications will allow for clear and semantically interoperable integration of arthropod phenotypes in biodiversity studies.

Since the boom of biomedical big data studies, various big data processing technologies have been developed rapidly. As an important form of knowledge representation, ontology has become an important means for the utilization and integration of biomedical big data. The emergence of new technologies for ontology development has resulted in the generation of many biomedical ontologies by many ontology development communities. The Open Biological and Biomedical Ontology Foundry, an academic organization for bio-ontology developers, has provided a set of principles to guide community-based open ontology construction. The Open Biological and Biomedical Ontology Foundry have also built many widely used ontologies, such as Gene Ontology, Human Phenotype Ontology, and Chemical Entities of Biological Interest. Other various ontology repositories have also been created and used to support ontology reuse. Many efficient tools for ontology applications, such as data annotation and terms mapping, have also been developed. High quality ontologies are also being used to develop new methods and tools for biomedical data analysis. The applications of Gene Ontology and Human Phenotype Ontology for data analysis and integration in recent years are reviewed here. To promote the development and applications of biomedical ontologies in China, a research community, OntoChina, was founded recently. OntoChina aims to support the development of reference ontologies, especially bilingual and Chinese translated ontologies. OntoChina also encourages ontology developers to follow the Open Biological and Biomedical Ontology Foundry principles.

Increasingly, in data-intensive areas of the life sciences, experimental results are being described in algorithmically useful ways with the help of ontologies. Such ontologies are authored and maintained by scientists to support the retrieval, integration and analysis of their data. The proposition to be defended here is that ontologies of this type – the Gene Ontology (GO) being the most conspicuous example – are a part of science. Initial evidence for the truth of this proposition (which some will find self-evident) is the increasing recognition of the importance of empirically-based methods of evaluation to the ontology development work being undertaken in support of scientific research. Ontologies created by scientists must, of course, be associated with implementations satisfying the requirements of software engineering. But the ontologies are not themselves engineering artifacts, and to conceive them as such brings grievous consequences. Rather, ontologies such as the GO are in different respects comparable to scientific theories, to scientific databases, and to scientific journal publications. Such a view implies a new conception of what is involved in the authoring, maintenance and application of ontologies in scientific contexts, and therewith also a new approach to the evaluation of ontologies and to the training of ontologists.

This study examines various approaches for ontology construction and evaluation in medicinal plant ontology, focusing on the challenges associated with this domain. Thirty review articles on three databases revealed five alternative approaches for ontology generation in herbal medicine: well-structured methodology, customized methodology, semiautomatic approach, socio-technical approach, domain expert techniques, and top-level approach. However, 68% of the research did not include ontology evaluation. Ontology quality was often assessed using competency questions, expert opinion, and automated error detection. The study provides ontology developers with a better understanding of established methodologies for developing and evaluating standard ontologies in this area and identifies gaps in existing research for future studies.

Geographical and taxonomical referencing of specimens and documented species observations from within and across natural history collections is vital for ongoing species research. However, much of the historical data such as field books, diaries and specimens, are challenging to work with. They are computationally inaccessable, refer to historical place names and taxonomies, and are written in a variety of languages. In order to address these challenges and elucidate historical species observation data, we developed a workflow to (i) crowd-source semantic annotations from handwritten species observations, (ii) transform them into RDF (Resource Description Framework) and (iii) store and link them in a knowledge base. Instead of full-transcription we directly annotate digital field books scans with key concepts that are based on Darwin Core standards. Our workflow stresses the importance of verbatim annotation. The interpretation of the historical content, such a resolving a historical taxon to a current one, can be done by individual researchers after the content is published as linked open data. Through the storage of annotion provenance, who created the annotation and when, we allow multiple interpretations of the content to exist in parallel, stimulating scientific discourse. The semantic annotation process is supported by a web application, the Semantic Field Book (SFB)-Annotator, driven by an application ontology. The ontology formally describes the content and meta-data required to semantically annotate species observations. It is based on the Darwin Core standard (DwC), Uberon and the Geonames ontology. The provenance of annotations is stored using the Web Annotation Data Model. Adhering to the principles of FAIR (Findable, Accessible, Interoperable & Reusable) and Linked Open Data, the content of the specimen collections can be interpreted homogeneously and aggregated across datasets. This work is part of the Making Sense project: makingsenseproject.org. The project aims to disclose the content of a natural history collection: a 17,000 page account of the exploration of the Indonesian Archipelago between 1820 and 1850 (Natuurkundige Commissie voor Nederlands-Indie) With a knowledge base, researchers are given easy access to the primary sources of natural history collections. For their research, they can aggregate species observations, construct rich queries to browse through the data and add their own interpretations regarding the meaning of the historical content.

Selecting the most appropriate candidate domain ontology is necessary to ensure that the ontology covers the domain of interest at a reasonable level of detail. Existing approaches have the following drawbacks: (1) Focused on the ontology coverage of concepts in the domain of interest, and ignored the semantic richness associated with each concept. (2) The ontology coverage metrics tend to select either large ontologies with broad scope or ontologies with a small number of concepts which may not capture a particular domain of interest. (3) The approaches are not robust in the coverage and semantic richness metric results when different term extraction and recognition algorithms are used. The limitations mentioned above will result in selecting ontologies which are not related to the domain of interest. Therefore, this paper presents a novel Terminological ONtology Evaluator (TONE). TONE uses a textual corpus to evaluate the ontology coverage and semantic richness. TONE was compared with existing ontology evaluation approaches and it proved that it was able to select the domain ontology which was intentionally developed to cover a specific domain of interest. In addition, TONE proved to be more robust in the coverage and semantic richness metric results compared to existing approaches.

Cloud computing is an integrated infrastructure for resource sharing and computing in distributed environment. In this paper, we propose an ontology management approach in cloud computing to provide a unique semantic ground to consumer. Ontology application can be performed based on MapReduce computation and provided to user browser or user application.

Both ontology content and ontology building tools evaluations play an important role before using ontologies in Semantic Web applications. In this paper we try to assess ontology evaluation functionalities of the following ontology platforms: OilEd, OntoEdit, Protege-2000, and WebODE. The goal of this paper is to analyze whether such ontology platforms prevent the ontologist from making knowledge representation mistakes in concept taxonomies during RDF(S) and DAML+OIL ontology import, during ontology building and during ontology export to RDF(S) and DAML+OIL. Our study reveals that most of these ontology platforms only detect a few mistakes in concept taxonomies when importing RDF(S) and DAML+OIL ontologies. It also reveals that most of these ontology platforms only detect some mistakes in concept taxonomies during building ontologies. Our study also reveals that these platforms do not detect any taxonomic mistake when exporting ontologies to such languages.

The recent years have witnessed a rapid development in virtual reality technology and computer aided design, which accompanied by a vast amount of 3D content springing up from various domains, has given rise to an emerging need for methods by which they can be efficiently annotated before being searched and retrieved. In this paper a new approach to semantic annotation on 3D content is proposed. In this approach, much of the manual work on annotation is replaced with a semi-automatic process based on prior gathering of domain expertise. The geometric properties of objects and their spatial relationships are first automatically extracted and associated with a general ontology, the result of which is then matched against a set of user-defined rules to create annotations represented in an application ontology. The approach is Web3D oriented so that it can take advantage of features of X3D format. It will be shown using a prototype system how this approach can succeed in accelerating the annotation process.

Cross domain-based ontology construction via Jaccard Semantic Similarity with hybrid optimization model