About Sound and Vision: CLEF Beyond Text Retrieval Tasks

Cross language information retrieval (CLIR) is a retrieval process in which the user fires queries in one language to retrieve information from another (different) language. The diversity of information and language barriers are the serious issues for communication and cultural exchange across the world. To solve such barriers, Cross language information retrieval system, are nowadays in strong demand. CLIR is a subset of Information Retrieval (IR) system. Information Retrieval deals with finding useful information from a large collection of unstructured, structured and semi-structured data to a user query where the query is a set of keywords. Information Retrieval can be classified into different classes such as Monolingual information retrieval, Bi-Lingual Information Retrieval, Multilingual information retrieval and Cross language information retrieval. This paper focuses on the various IR variants and techniques used in CLIR system. Further, based on available literature, a number of challenges and issues in CLIR have been identified and discussed. It gives an overview of the advantages, limitations, tools available in CLIR research. It also describes new application areas of CLIR such as medical, multimedia, question answering system etc. The need for exploring and building more specialized information system that enable speakers of an Odia language to discover valuable information beyond linguistic and cultural barriers. This study is aimed at building an experimental CLIR system between one of the under-resourced language (i.e. Odia) and one of the most commonly used online language (i.e. English) in future.

Paraphrase identification is a crucial task in natural language understanding, especially in cross-language information retrieval. Nowadays, Multi-Task Deep Neural Network (MT-DNN) has become a state-of-the-art method that brings outstanding results in paraphrase identification [1]. In this paper, our proposed method based on MT-DNN [2] to detect similarities between English and Vietnamese sentences, is proposed. We changed the shared layers of the original MT-DNN from original the BERT [3] to other pre-trained multi-language models such as M-BERT [3] or XLM-R [4] so that our model could work on cross-language (in our case, English and Vietnamese) information retrieval. We also added some tasks as improvements to gain better results. As a result, we gained 2.3% and 2.5% increase in evaluated accuracy and F1. The proposed method was also implemented on other language pairs such as English – German and English – French. With those implementations, we got a 1.0%/0.7% improvement for English – German and a 0.7%/0.5% increase for English – French.

Humans have used electronic information retrieval (IR) systems for more than 50 years as they evolved from experimental systems to full-scale Web search engines and digital libraries. The fields of library and information science (LIS), cognitive science, human factors and computer science have historically been the leading disciplines in conducting research that seeks to model human interaction with IR systems for all kinds of information related behaviors. As technology problems have been mastered, the theoretical and applied framework for studying human interaction with IR systems has evolved from systems-centered to more user-centered, or cognitive-centered approaches. However, cognitive information retrieval (CIR) research that focuses on user interaction with IR systems is still largely under-funded and is often not included at computing and systems design oriented conferences. But CIR-focused research continues, and there are signs that some IR systems designers in academia and the Web search business are realizing that user behavior research can provide valuable insights into systems design and evaluation. The goal of our book is to provide an overview of new CIR research directions. This book does not provide a history of the research field of CIR. Instead, the book confronts new ways of looking at the human information condition with regard to our increasing need to interact with IR systems. The need has grown due to a number of factors, including the increased importance of information to more people in this information age. Also, IR was once considered document-oriented, but has now evolved to include multimedia, text, and other information objects. As a result, IR systems and their complexity have proliferated as users and user purposes for using them have also proliferated. Human interaction with IR systems can often be frustrating as people often lack an understanding of IR system functionality. New more holistic directions in CIR are emerging that conceptualize human-IR system interaction at the human-computer interaction level (taking into account interface issues), the information behavior level (taking into account the role of IR system interaction in the total range of people’s information behaviors), and the

Given a user question, the goal of a Question Answering (QA) system is to retrieve answers rather than full documents or even best-matching passages, as most Information Retrieval systems currently do. In this paper, we present BRUJA, a QA system for the management of multilingual collections. BRUJ rkstions (English, Spanish and French). The BRUJA architecture is not formed with three monolingual QA systems but instead uses English as Interlingua to make usual QA tasks such as question classifications and answer extractions. In addition, BRUJA uses Cross Language Information Retrieval (CLIR) techniques to retrieve relevant documents from a multilingual collection. On the one hand, we have more documents to find answers from but on the other hand, we are introducing noise into the system because of translations to the Interlingua (English) and the CLIR module. The question is whether the difficulty of managing three languages is worth it or whether a monolingual QA system delivers better results. We report on in-depth experimentation and demonstrate that our multilingual QA system gets better results than its monolingual counterpart whenever it uses good translation resources and, especially, CLIR techniques that are state-of-the-art.

Purpose Prior Information Retrieval (IR) research synthesizes progress from individual studies, yet academia-industry collaboration dynamics remain unexplored. This study investigates: (1) productivity patterns and venues, (2) citations-downloads relationships, (3) topic evolution, and (4) collaboration trends. Design/methodology/approach We perform an analysis of 53,471 ACM IR papers (2000–2018) using bibliometrics and DistilBERT topic modeling. Findings We find that industry-involved papers preferred WWW/CIKM venues; collaborations dominated RecSys/CSCW. We see that academia-industry collaborations achieved the highest download-to-citation conversion rates. Academia focused on algorithms; industry on applications; collaborations bridged both with rising human-centered themes. Research implications This is a pioneering large-scale bibliometrics revealing collaboration’s impact on IR knowledge evolution and provides a methodological framework for cross-sector analysis. Practical implications The paper identifies optimal venues (RecSys/CSCW) for partnerships and guides joint initiatives (shared datasets, grants) to bridge academia-industry divides and enhance research translation. Originality/value This is the first large-scale bibliometric analysis of IR academia-industry collaboration. The paper finds many novel insights, including the fact that collaboration boosts citation efficiency, enables complementary specialization, and drives topic convergence.

BackgroundTest collections for information retrieval are scarce. Domain specific test collections even more so, and medical test collections in the Swedish language non-existent prior to the making of the MedEval test collection. Most research in information retrieval has been performed in the English language, thus most test collections contain English documents. However, English is morphologically poor compared to many other European languages and a number of interesting and important aspects have not been investigated. Building a medical test collection in Swedish opens new research opportunities.MethodsThis article describes the making of and potential uses of MedEval, a Swedish medical test collection with assessments, not only for topical relevance, but also for target reader group: Doctors or Patients. A user of the test collection may choose if she wishes to search in the Doctors or the Patients scenario where the topical relevance assessments have been adjusted with consideration to user group, or to search in a scenario which regards only topical relevance.In addition to having three user groups, MedEval, in its present form, has two indexes, one where the terms are lemmatized and one where the terms are lemmatized and the compounds split and the constituents indexed together with the whole compound.ResultsDifferences discovered between the documents written for medical professionals and documents written for laypersons are presented. These differences may be utilized in further studies of retrieval of documents aimed at certain groups of readers. Differences between the groups of documents are, for example, that professional documents have a higher ratio of compounds, have a greater average word length and contain more multi-word expressions.An experiment is described where the user scenarios have been utilized, searching with expert terms and lay terms, separately and in combination in the different scenarios. The tendency discovered is that the medical expert gets best results using expert terms and the lay person best results using lay terms, but also quite good results using expert terms or lay and expert terms in combination.ConclusionsThe many features of MedEval gives a variety of research possibilities, such as comparing the effectiveness of search terms when it comes to retrieving documents aimed at the different user groups or to study the effect of compound decomposition in retrieval of documents. As Swedish, the language of MedEval, is a morphologically more complex language than English, it is possible to study additional aspects of the effect of natural language processing in information retrieval, for example utilizing different inflectional word forms in the retrieval of expert vs lay documents. MedEval is the first Swedish test collection of the medical domain.AvailabilityThe Department of Swedish at the University of Gothenburg is in the process of making the MedEval test collection available to academic researchers.

The Web has become a large repository of documents (or pages) written in many different languages. In this context, traditional information retrieval (IR) techniques cannot be used whenever the user query and the documents being retrieved are in different languages. To address this problem, new cross‐language information retrieval (CLIR) techniques have been proposed. In this work, we describe a method for cross‐language retrieval of medical information. This method combines query terms and related medical concepts obtained automatically through a categorization procedure. The medical concepts are used to create a linguistic abstraction that allows retrieval of information in a language‐independent way, minimizing linguistic problems such as polysemy. To evaluate our method, we carried out experiments using the OHSUMED test collection, whose documents are written in English, with queries expressed in Portuguese, Spanish, and French. The results indicate that our cross‐language retrieval method is as effective as a standard vector space model algorithm operating on queries and documents in the same language. Further, our results are better than previous results in the literature.

PurposeThis paper aims to describe a study of the queries generated from a user experiment for cross‐language information retrieval (CLIR) from a historic image archive.Design/methodology/approachA controlled lab‐based user study was carried out using a prototype Italian‐English image retrieval system. Participants were asked to carry out searches for 16 images provided to them, a known‐item search task. Italian speaking users generated 618 queries for a set of known‐item search tasks. User's interactions with the system were recorded and queries were analysed manually quantitatively and qualitatively. The queries generated by user's interaction with the system were analysed and the results used to suggest recommendations for the future development of cross‐language retrieval systems for digital image libraries.FindingsResults highlight the diversity in requests for similar visual content and the weaknesses of machine translation for query translation. Through the manual translation of queries the authors show the benefits of using high‐quality translation resources. The results show the individual characteristics of users while performing known‐item searches and the overlap obtained between query terms and structured image captions, highlighting the use of user's search terms for objects within the foreground of an image.Research limitations/implicationsThis research looks in depth into one case of interaction and one image repository. Despite this limitation, the discussed results are likely to be valid across other languages and image repositories.Practical implicationsTo develop effective systems requires studying user's search behaviours, particularly in digital image libraries.Originality/valueThe growing quantity of digital visual material in digital libraries offers the potential to apply techniques from CLIR to provide cross‐language information access services. The value of this paper is in the provision of empirical evidence to support recommendations for effective cross‐language image retrieval system design.

The study of cross-lingual Information Retrieval Systems (IRSs) and a deep analysis of system performances should provide guidelines, hints, and directions to drive the design and development of the next generation MultiLingual Information Access (MLIA) systems. In addition, effective tools for interpreting and comparing the experimental results should be made easily available to the research community. To this end, we propose a twofold methodology for the evaluation of Cross Language Information Retrieval (CLIR) systems: statistical analyses to provide MLIA researchers with quantitative and more sophisticated analysis techniques; and graphical tools to allow for a more qualitative comparison and an easier presentation of the results. We provide concrete examples about how the proposed methodology can be applied by studying the monolingual and bilingual tasks of the Cross-Language Evaluation Forum (CLEF) 2005 and 2006 campaigns.KeywordsInformation RetrievalMean Average PrecisionGraphical ToolBilingual ComparisonBilingual PerformanceThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Proper nouns are usually primary keys in a query. Their correct translation might be necessary to maintain a good retrieval performance in a cross language information retrieval (CLIR) system. However, dictionaries only include the most commonly used proper nouns, like major countries and capitals. As they are spelling variants of each other in most languages, using an approximate string matching technique against the target database index is the common approach taken to find the target language correspondents of the original query key. N-gram technique proved to be the most effective among other approximate string matching techniques. As we are dealing with an English-Arabic CLIR system which involves two languages of different alphabets, we decided to combine transliteration with the n-gram technique to generate the different spelling variants of out of vocabulary (OOV) words. We call this technique: Transliteration Ngram (TNG). One issue that arises with the Arabic language is that words that are spelled similarly can have different meanings depending on the context of the sentence. This is particularly true for proper names, which usually have a meaning if used as a verb or adjective. To further enhance our transliteration approach, we chose to use part of speech (POS) disambiguation to reduce the number of unrelated words from the set transliterations obtained using TNG.

Researchers in multilingual information retrieval and natural language processing are making progress on algorithms for guessing what a text is about and for translating queries between languages; some day we may have reliable programs for recognizing musical melodies or the content of images. In contrast, researchers in metadata focus more on seeking consensus on the meanings of categories for describing resources; on finding ways to allow simple schemas to interoperate with complex ones; and on designing frameworks for managing the messy equivalencies between metadata models in different fields and languages. Do the two perspectives form a continuum? What problems do they solve best? For resource discovery, what is the best balance between human and machine?.

Named Entity Recognition is a subtask in Information Extraction. In Named Entity Recognition (NER) we try to identify each of the words provided into some categories predefined by us. These categories or classes can be organization, name, time, place etc. Named Entity Recognition is a part of Natural Language Processing which is aims at making human and computer interactions more meaningful and efficient. NLP is nowadays being effectively being used in the area of automatic text summarization, cross language information retrieval, speech recognition and query named entity recognition. This paper focuses on Named entity recognition, how it is performed and a how apply NER to web search queries so that correct and more user personalized results are displayed by the web search engines..

The development of digital and online information repositories is creating many opportunities and also new challenges in information retrieval. The availability of online documents in many different languages makes it possible for users around the world to directly access previously unimagined sources of information. However in conventional information retrieval systems the user must enter a search query in the language of the documents in order to retrieve it. This requires that users can express their queries in those languages in which the information is available and can understand the documents returned by the retrieval process. This restriction clearly limits the amount and type of information that an individual user really has access to. Cross Lingual Information Access is concerned with technologies that let users express their query in their native language, and irrespective of the language in which the information is available, present their information in the user-preferred language or set of languages, in a manner that satisfies the user's information needs. The additional processing may take the form of machine translation of snippets, summarization and subsequent translation of summaries and/or information extraction. In recent times, research in Cross Lingual Information Access has been vigorously pursued through several international fora, such as, the Cross-Language Evaluation Forum (CLEF), NTCIR Asian Language Retrieval, Question-answering Workshop and such other fora. A workshop geared towards cross language information retrieval in Indian languages (FIRE) was organized in December 2008. In addition to CLIR, significant results have been obtained in multilingual summarization workshops and cross-language named entity extraction challenges by the ACL (Association for Computational Linguistics) and the Geographic Information retrieval (GeoCLEF) track of CLEF. The previous two issues of this workshop were held in January 2007, during IJCAI 2007 in Hyderabad, India (http://search.iiit.ac.in/CLIA2007/) and subsequently during IJCNLP 2008 in Hyderabad, India (http://search.iiit.ac.in/CLIA2008/). Both the previous workshops attracted an encouraging number of submissions, and a large number of registered participants. This third international workshop on Cross Lingual Information Access aims to bring together various trends in multi-source, cross and multilingual information retrieval and access, and provide a venue for researchers and practitioners from academia, government, and industry to interact and share a broad spectrum of ideas, views and applications. The present workshop includes an invited keynote talk, presentations of technical papers selected after peer review followed by a panel discussion.

As increasing numbers of non‐English resources have become available on the Web, the interesting and important issue of how Web users can retrieve documents in different languages has arisen. Cross‐language information retrieval (CLIR), the study of retrieving information in one language by queries expressed in another language, is a promising approach to the problem. Cross‐language information retrieval has attracted much attention in recent years. Most research systems have achieved satisfactory performance on standard Text REtrieval Conference (TREC) collections such as news articles, but CLIR techniques have not been widely studied and evaluated for applications such as Web portals. In this article, the authors present their research in developing and evaluating a multilingual English–Chinese Web portal that incorporates various CLIR techniques for use in the business domain. A dictionary‐based approach was adopted and combines phrasal translation, co‐occurrence analysis, and pre‐ and posttranslation query expansion. The portal was evaluated by domain experts, using a set of queries in both English and Chinese. The experimental results showed that co‐occurrence‐based phrasal translation achieved a 74.6% improvement in precision over simple word‐by‐word translation. When used together, pre‐ and posttranslation query expansion improved the performance slightly, achieving a 78.0% improvement over the baseline word‐by‐word translation approach. In general, applying CLIR techniques in Web applications shows promise.

In this paper we investigate some of the most important variants of information retrieval. They are Cross-lingual Information Retrieval (CLIR) and Multilingual Information Retrieval (MLIR). CLIR deals with submitting queries in one language and retrieving documents in some other language other than query language. MLIR deals with submitting questions in one or more languages and retrieving documents in one or more diverse languages. With increasing globalized economy, the knack to find information in other languages is becoming a necessity. In short the paper tries to encapsulate the overall review of the research works in Indian and Foreign languages.