Semantic Cross-lingual Information Retrieval

Cross language information retrieval (CLIR) is a retrieval process in which the user fires queries in one language to retrieve information from another (different) language. The diversity of information and language barriers are the serious issues for communication and cultural exchange across the world. To solve such barriers, Cross language information retrieval system, are nowadays in strong demand. CLIR is a subset of Information Retrieval (IR) system. Information Retrieval deals with finding useful information from a large collection of unstructured, structured and semi-structured data to a user query where the query is a set of keywords. Information Retrieval can be classified into different classes such as Monolingual information retrieval, Bi-Lingual Information Retrieval, Multilingual information retrieval and Cross language information retrieval. This paper focuses on the various IR variants and techniques used in CLIR system. Further, based on available literature, a number of challenges and issues in CLIR have been identified and discussed. It gives an overview of the advantages, limitations, tools available in CLIR research. It also describes new application areas of CLIR such as medical, multimedia, question answering system etc. The need for exploring and building more specialized information system that enable speakers of an Odia language to discover valuable information beyond linguistic and cultural barriers. This study is aimed at building an experimental CLIR system between one of the under-resourced language (i.e. Odia) and one of the most commonly used online language (i.e. English) in future.

As increasing numbers of non‐English resources have become available on the Web, the interesting and important issue of how Web users can retrieve documents in different languages has arisen. Cross‐language information retrieval (CLIR), the study of retrieving information in one language by queries expressed in another language, is a promising approach to the problem. Cross‐language information retrieval has attracted much attention in recent years. Most research systems have achieved satisfactory performance on standard Text REtrieval Conference (TREC) collections such as news articles, but CLIR techniques have not been widely studied and evaluated for applications such as Web portals. In this article, the authors present their research in developing and evaluating a multilingual English–Chinese Web portal that incorporates various CLIR techniques for use in the business domain. A dictionary‐based approach was adopted and combines phrasal translation, co‐occurrence analysis, and pre‐ and posttranslation query expansion. The portal was evaluated by domain experts, using a set of queries in both English and Chinese. The experimental results showed that co‐occurrence‐based phrasal translation achieved a 74.6% improvement in precision over simple word‐by‐word translation. When used together, pre‐ and posttranslation query expansion improved the performance slightly, achieving a 78.0% improvement over the baseline word‐by‐word translation approach. In general, applying CLIR techniques in Web applications shows promise.

This article presents the performance evaluation of information retrieval system for English and Hindi cross-Indian languages. As India is a multilingual country, different varieties of data material in varieties of languages are increasing day by day on Internet database. And hence, multilingual information retrieval is big issue in country like India. Usually, people used their native dialect to represent their needs and communication, and hence, language diversity produces a great barricade. In this paper, a new research design of cross-lingual information retrieval system is proposed. Here, a new research design has been implemented for information retrieval based on English and Hindi cross-languages by applying an approach of query translation. English language is considered for the query language, and content is stored in the database in Hindi Language. Here, query translation from English language to Hindi language is done by using dictionary-based approach. Hindi WordNet is used for the query translation. Query expansion and query preprocessing methods are implemented to improve the performance of the new research design and accordingly get the good precision.KeywordsCLIRQuery translationInformation retrievalEnglish and Hindi languages

These research studies compare two methods for ontology creation and their corresponding tools. The main objective of this work is the building of a graphical editor for ontology construction, based on software usability criteria. This comparison shows which methods used in previous tools will remains in the new editor, and what new tools can be developed for enhancing the ontology management made using the old one. The use of graphics and diagrams helps in the discovery of new knowledge, from information received from various sources, being the Internet or databases. This new type of editor improves the visualization of relations between the terms of ontologies and thereby improves the efficiency of its construction for commercial application systems. In addition, this form of information representation, based on these diagrams, will allow information systems based on ontologies to identify - automatically -concepts, hidden elements, the flow time between the events of cause and consequences and the meaning of relations between the captured information. The Ontojuris project has the purpose given above, and it is described on this paper. The project consists of a multilingual information system for legal information retrieval associated with an ontology editor, called Knowledge Engineering Suite. The system accepts queries in one language, and allows information retrieval based on similarity of documents, written in one of several languages. The multilingual ontology editor works with the concept of Universal Words (UW's). The UWs are universal representations of things and objects, and it is developed inside the UNL Project (being UNL an acronym for Universal Networking Language) and allows the query expansion in a multilingual way. Query expansion can improve searching process, required by user, by including additional terms which have similar meaning to the original query. In this study, we proposed a new expansion method which is based on domain ontology and UWs, to achieve better performance for a multilingual information retrieval. So, the Ontojuris project aims to facilitate access to information on legislation in the area of Intellectual Property Law, Consumer Rights and Electronic Law in the consortium formed by researches in Brazil, Chile, Spain and Argentina.

Paraphrase identification is a crucial task in natural language understanding, especially in cross-language information retrieval. Nowadays, Multi-Task Deep Neural Network (MT-DNN) has become a state-of-the-art method that brings outstanding results in paraphrase identification [1]. In this paper, our proposed method based on MT-DNN [2] to detect similarities between English and Vietnamese sentences, is proposed. We changed the shared layers of the original MT-DNN from original the BERT [3] to other pre-trained multi-language models such as M-BERT [3] or XLM-R [4] so that our model could work on cross-language (in our case, English and Vietnamese) information retrieval. We also added some tasks as improvements to gain better results. As a result, we gained 2.3% and 2.5% increase in evaluated accuracy and F1. The proposed method was also implemented on other language pairs such as English – German and English – French. With those implementations, we got a 1.0%/0.7% improvement for English – German and a 0.7%/0.5% increase for English – French.

The study of cross-lingual Information Retrieval Systems (IRSs) and a deep analysis of system performances should provide guidelines, hints, and directions to drive the design and development of the next generation MultiLingual Information Access (MLIA) systems. In addition, effective tools for interpreting and comparing the experimental results should be made easily available to the research community. To this end, we propose a twofold methodology for the evaluation of Cross Language Information Retrieval (CLIR) systems: statistical analyses to provide MLIA researchers with quantitative and more sophisticated analysis techniques; and graphical tools to allow for a more qualitative comparison and an easier presentation of the results. We provide concrete examples about how the proposed methodology can be applied by studying the monolingual and bilingual tasks of the Cross-Language Evaluation Forum (CLEF) 2005 and 2006 campaigns.KeywordsInformation RetrievalMean Average PrecisionGraphical ToolBilingual ComparisonBilingual PerformanceThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Given a user question, the goal of a Question Answering (QA) system is to retrieve answers rather than full documents or even best-matching passages, as most Information Retrieval systems currently do. In this paper, we present BRUJA, a QA system for the management of multilingual collections. BRUJ rkstions (English, Spanish and French). The BRUJA architecture is not formed with three monolingual QA systems but instead uses English as Interlingua to make usual QA tasks such as question classifications and answer extractions. In addition, BRUJA uses Cross Language Information Retrieval (CLIR) techniques to retrieve relevant documents from a multilingual collection. On the one hand, we have more documents to find answers from but on the other hand, we are introducing noise into the system because of translations to the Interlingua (English) and the CLIR module. The question is whether the difficulty of managing three languages is worth it or whether a monolingual QA system delivers better results. We report on in-depth experimentation and demonstrate that our multilingual QA system gets better results than its monolingual counterpart whenever it uses good translation resources and, especially, CLIR techniques that are state-of-the-art.

In Cross-Language Information Retrieval (CLIR) process, the translation effects have a direct impact o n the accuracy of follow-up retrieval results. In diction ary-based approach, we are dealing with the words t hat have more than one meaning which can decrease the retrieval performance if the query translation retur n an incorrect translations. These issues need to be ove rcome using efficient technique. In this study we p roposed a Cross-Language Information Retrieval (CLIR) method based on domain ontology using Quran concepts for disambiguating translation of the query and to improve the dictionary-based query translation. For experimentation, we use Quran ontology written in E nglish and Malay languages as a bilingual parallelcorpora and Quran concepts as a resource for cross- language query translation along with dictionary-ba sed translation. For evaluation, we measure the perform ance of three IR systems. IR 1 is natural language query IR, IR 2 is natural language query CLIR based on dictionary (as a Baseline) and IR 3 is the retrieval of this research proposed method using Mean Average Precision (MAP) and average precision at 11 points of recall. The experimental result shows that our prop osed method brings significant improvement in retri eval accuracy for English document collections, but defi cient for Malay document collections. The proposed CLIR method can obtain query expansion effect and improve retrieval performance in certain language.

Proper nouns are usually primary keys in a query. Their correct translation might be necessary to maintain a good retrieval performance in a cross language information retrieval (CLIR) system. However, dictionaries only include the most commonly used proper nouns, like major countries and capitals. As they are spelling variants of each other in most languages, using an approximate string matching technique against the target database index is the common approach taken to find the target language correspondents of the original query key. N-gram technique proved to be the most effective among other approximate string matching techniques. As we are dealing with an English-Arabic CLIR system which involves two languages of different alphabets, we decided to combine transliteration with the n-gram technique to generate the different spelling variants of out of vocabulary (OOV) words. We call this technique: Transliteration Ngram (TNG). One issue that arises with the Arabic language is that words that are spelled similarly can have different meanings depending on the context of the sentence. This is particularly true for proper names, which usually have a meaning if used as a verb or adjective. To further enhance our transliteration approach, we chose to use part of speech (POS) disambiguation to reduce the number of unrelated words from the set transliterations obtained using TNG.

Cross language information retrieval (CLIR) deals with the use of queries in one language to access documents in another. A popular CLIR approach is to translate the query into the language of the documents being retrieved. The simplest yet most effective method for query translation is bilingual dictionary approach. However, direct translation using bilingual dictionary prune two problems: proper names and compound words translations. In this study, a series of experiments were conducted to test and evaluate the effectiveness of the Malay-English CLIR using proper names and compound words translation. We believe by using concept-based indexing and translations, makes proper names and compound words translation possible. The best retrieval performance was obtained from the combination of query translation approach-select all translations listed in the dictionary, the alternative weighting scheme and proper names identification and translation.

Named Entity Recognition is a subtask in Information Extraction. In Named Entity Recognition (NER) we try to identify each of the words provided into some categories predefined by us. These categories or classes can be organization, name, time, place etc. Named Entity Recognition is a part of Natural Language Processing which is aims at making human and computer interactions more meaningful and efficient. NLP is nowadays being effectively being used in the area of automatic text summarization, cross language information retrieval, speech recognition and query named entity recognition. This paper focuses on Named entity recognition, how it is performed and a how apply NER to web search queries so that correct and more user personalized results are displayed by the web search engines..

The Web has become a large repository of documents (or pages) written in many different languages. In this context, traditional information retrieval (IR) techniques cannot be used whenever the user query and the documents being retrieved are in different languages. To address this problem, new cross‐language information retrieval (CLIR) techniques have been proposed. In this work, we describe a method for cross‐language retrieval of medical information. This method combines query terms and related medical concepts obtained automatically through a categorization procedure. The medical concepts are used to create a linguistic abstraction that allows retrieval of information in a language‐independent way, minimizing linguistic problems such as polysemy. To evaluate our method, we carried out experiments using the OHSUMED test collection, whose documents are written in English, with queries expressed in Portuguese, Spanish, and French. The results indicate that our cross‐language retrieval method is as effective as a standard vector space model algorithm operating on queries and documents in the same language. Further, our results are better than previous results in the literature.

We present a method for automatic query expansion for cross-lingual information retrieval in the medical domain. The method employs machine translation of source-language queries into a document language and linear regression to predict the retrieval performance for each translated query when expanded with a candidate term. Candidate terms (in the document language) come from multiple sources: query translation hypotheses obtained from the machine translation system, Wikipedia articles and PubMed abstracts. Query expansion is applied only when the model predicts a score for a candidate term that exceeds a tuned threshold which allows to expand queries with strongly related terms only. Our experiments are conducted using the CLEF eHealth 2013–2015 test collection and show significant improvements in both cross-lingual and monolingual settings.

The present paper will seek to present an approach to bilingual lexicon extraction from non-aligned comparable corpora, phrasal translation as well as evaluations on Cross-Language Information Retrieval. A two-stages translation model is proposed for the acquisition of bilingual terminology from comparable corpora, disambiguation and selection of best translation alternatives according to their linguistics-based knowledge. Different rescoring techniques are proposed and evaluated in order to select best phrasal translation alternatives. Results demonstrate that the proposed translation model yields better translations and retrieval effectiveness could be achieved across Japanese-English language pair.

CLEF was initiated with intention of providing a catalyst to research in Cross-Language Information Retrieval (CLIR) and Multilingual Information Retrieval (MIR). Focusing principally on European languages, it initially provided CLIR benchmark tasks to the research community within an annual cycle of task design, conduct and reporting. While the early focus was on textual data, the emergence of technologies to enable collection, archiving and content processing of multimedia content led to several initiatives which sought to address search for spoken and visual content. Similar to the interest in multilingual search for text, interest arose in working multilingually with multimedia content. To support research in these areas CLEF introduced a number of tasks in multilingual search for multimedia content. While investigation of image retrieval has formed the focus of the ImageCLEF task over many years, this chapter reviews tasks examining speech and video retrieval carried out within CLEF during its first 10 years, and overviews related work reported at other information retrieval benchmarks.