Modified Firefly Algorithm and Fuzzy C-Mean Clustering Based Semantic Information Retrieval

In recent years, the ever-increasing quantities of entities in large knowledge bases on the Web, such as DBpedia, Freebase and YAGO, pose new challenges but at the same time open up new opportunities for intelligent information access. These knowledge bases (KBs) have become valuable resources in many research areas, such as natural language processing (NLP) and information retrieval (IR). Recently, almost every major commercial Web search engine has incorporated entities into their search process, including Google’s Knowledge Graph, Yahoo!’s Web of Objects and Microsoft’s Satori Graph/Bing Snapshots. The goal is to bridge the semantic gap between natural language text and formalized knowledge. Within the context of globalization, multilingual and cross-lingual access to information has emerged as an issue of major interest. Nowadays, more and more people from different countries are connecting to the Internet, in particular the Web, and many users can understand more than one language. While the diversity of languages on the Web has been growing, for most people there is still very little content in their native language. As a consequence of the ability to understand more than one language, users are also interested in Web content in other languages than their mother tongue. There is an impending need for technologies that can help in overcoming the language barrier for multilingual and cross-lingual information access. In this thesis, we face the overall research question of how to allow for semantic-aware and cross-lingual processing of Web documents and user queries by leveraging knowledge bases. With the goal of addressing this complex problem, we provide the following solutions: (1) semantic annotation for addressing the semantic gap between Web documents and knowledge; (2) semantic search for coping with the semantic gap between keyword queries and knowledge; (3) the exploitation of cross-lingual semantics for overcoming the language barrier between natural language expressions (i.e., keyword queries and Web documents) and knowledge for enabling cross-lingual semantic annotation and search. We evaluated these solutions and the results showed advances beyond the state-of-the-art. In addition, we implemented a framework of cross-lingual semantic annotation and search, which has been widely used for cross-lingual processing of media content in the context of our research projects.

After a disaster such as earthquakes, debris flows, forest fires, or landslides, etc., a lot of people have to be away from their home and gather in a shelter. In addition, the refugees suffer from the shortage of necessary resources due to impaired life infrastructure, such as damaged roads and communication networks. The degree of reducing damage depends on the amount of food, water, daily necessities and communication resources required by each shelter. How to effectively and efficiently allocate resources according to grasp the exact need of a disaster situation will be an important issue. We estimate the degree of the disaster by collecting and analyzing big data from the SNS, and building a platform for the communication resources to be efficiently and effectively allocated. In order to achieve this goal, we are challenging the following issues A) Understanding situations (user requirements) after disaster occur The SNS streams large scale semantic information about real time situation in society, especially during and after disaster. It is both domain-specific and computational challenge in processing the heterogeneous large data set to extract the exact situational content with reduced semantic uncertainty. The machine learning (ML) and natural language processing (NPL) tool kits are useful in semantic analysis, but still needs domain-specific implementation and computational improvement for the situation understanding from the SNS big data. B) Understanding distribution patterns of situations/users' requirements The disaster related situation is spatiotemporally correlated, and varies dynamically in space and time. It is also domain-specific and computational challenge in estimating the spatiotemporal distribution patterns of the disaster affect based on the spatial big data from SNS. The scan statistics such as the spatial scan have provided well tested mathematical tools and software for spatial data mining. However, new methodologies are necessary since the assumptions have to be different when it meets the spatial big data in SNS. And the computational complexity in spatial big data is also a bottleneck for real-time processing. C) Solving uncertainty of big crowd data One of the major features in big crowd data, e.g., SNS data, is uncertainty behind the data. Especially in a disaster scenario, the collecting time period cannot be long enough to smooth the data automatically. How to efficiently solve uncertainty problem in the big crowd data in a disaster scenario becomes a new and big challenge for disaster management.

Term mismatch is a common limitation of traditional information retrieval (IR) models where relevance scores are estimated based on exact matching of documents and queries. Typically, good IR model should consider distinct but semantically similar words in the matching process. In this paper, we propose a method to incorporate word embedding (WE) semantic similarities into existing probabilistic IR models for Arabic in order to deal with term mismatch. Experiments are performed on the standard Arabic TREC collection using three neural word embedding models. The results show that extending the existing IR models improves significantly baseline bag-of-words models. Although the proposed extensions significantly outperform their baseline bag-of-words, the difference between the evaluated neural word embedding models is not statistically significant. Moreover, the overall comparison results show that our extensions significantly improve the Arabic WordNet based semantic indexing approach and three recent WE-based IR language models.

There is an increasing need for both governments and businesses to discover latent anomalous activities in unstructured publicly-available data, produced by professional agencies and the general public. Over the past two decades, consumers have begun to use smart devices to both take in and generate a large volume of open-source text-based data, providing the opportunity for latent anomaly analysis. However, real-time data acquisition, and the processing and interpretation of various types of unstructured data, remains a great challenge. Recent efforts have focused on artificial intelligence / machine learning (AI/ML) solutions to accelerate the labor-intensive linear collection, exploitation, and dissemination analysis cycle and enhance it with a data-driven rapid integration and correlation process of open-source data. This paper describes an Activity Based Intelligence framework for anomaly detection of open-source big data using AI/ML to perform semantic analysis. The proposed Anomaly Detection using Semantic Analysis Knowledge (ADUSAK) framework includes four layers: input layer, knowledge layer, reasoning layer, and graphical user interface (GUI)/output layer. The corresponding main technologies include: Information Extraction, Knowledge Graph (KG) construction, Semantic Reasoning, and Pattern Discovery. Finally, ADUSAK was verified by performing Emerging Events Detection, Fake News Detection, and Suspicious Network Analysis. The generalized ADUSAK framework can be easily extended to a wide range of applications by adjusting the data collection, modeling construction, and event alerting.

<p>Digital water and energy maps allow fast information retrieval, big data analysis and resources demand prediction for real time responses in 5-G networks. A regulatory systems framework is needed to enable and promote integrated actions grounded on map-based feedback information, to facilitate resources movements and knowledge transfer for water and energy security. At the same time, the proposed regulatory system needs to safeguard national security and personal privacy when general public and the private sectors have access to big databases.</p><p>The Guangdong-Hong Kong-Macao Greater Bay Area (GBA) in China is an initiative on regional economic development involving nine mainland cities and two Special Administrative Regions (SARs). As central policies cannot be efficiently executed in the whole regions, institutional fragmentation could be a prominent barrier to achieve regional water and energy optimum rather than individual city maxima for the water and energy nexus.</p><p>In this study, we propose a systems regulatory framework that integrates natural, urban and social systems across multiple scales in which the relevant laws, policies, decisions and actions are supported by digital maps. On a planning scale, our new regulatory system based on spatial map information promotes optimum uses of natural capitals and ecosystem services (ES). For linking different urban spatial processes on different scales, satellite images and Local Climate Zone (LCZ) maps are used to describe natural environment and urban characteristics from 200km to 10km resolutions for supporting land-use planning laws and estimating regional development carrying capacity to mitigate water and energy insecurity.</p><p>On an operational scale, smart meters and remote sensor systems provide real time water and energy information from a fast developing 5-G network for the proposed digital maps. Forecasted energy and water demands from the digital maps can be used for regional or local environment regulation reinforcement. Proposed spatial maps also improve transboundary collaboration by providing visualisation of legal targets and emission limits. Through digital maps, key agencies and sectors will have a capacity to share transboundary knowledge, information and responsibility, to foster smooth system flows in terms of culture, economy, policy and technology, by active participations and decentralized actions.</p><p>On an evaluation scale, open map information increases the transparency of legal targets and pollution limits. By rapid information retrieval and big data analysis from digital maps, regulators can assess the performance of water and energy security practices.</p><p>In summary, the proposed framework based on LCZ maps for the GBA can be applied to other rapidly developing regions with emerging 5-G networks. The integrated regulatory framework also guides water and energy security practices and transfer central policies to local actions by rapid information retrieval, big data analysis and prediction of demand for real time responses based on digital water and energy maps.</p><p></p><p></p><p></p><p></p><p></p>

Online social networking is one of the most prominent features and main reasons of the big data era. In this paper, we first present the immense opportunities brought by big data in: advancing the technical level of the Chinese IT industry, leading development of the new Internet economy, and accelerating interdisciplinary research between sociology and information science. Next we analyze key issues in online social networks and point out new challenges of big social data research on semantic understanding and analysis, multi-modal association and fusion, group behavior analysis and mining, multidimensional analysis and visualization, and system development and integration. We then focus on introducing key international and domestic achievements in big data and online social networks. We conclude with a look at future trends in big social data analytics and mining.

Not many XML information retrieval (IR) systems exist that allow easy addition of and switching between different IR models. Especially for the scientific environment where building a system takes a lot of time and keeps researchers away from the real work, i.e., investigating what is the most effective IR model, a platform that would provide this functionality would be ideal. For this reason we developed such an XML IR system. It is centered around a logical algebra, named score region algebra (SRA), that enables transparent specification of IR models for XML databases (see [1] for more details). The transparency is achieved by a possibility to instantiate various retrieval models, using abstract score functions within algebra operators, while logical query plan and operator definitions remain unchanged. Our algebra operators model three important aspects of XML IR: element relevance score computation, element score propagation, and element score combination. To implement a new IR model, one only needs to provide definitions for these abstract function classes. To illustrate the usefulness of our algebra our demo system supports several, well known IR scoring models (e.g., Language Models, Okapi, and tf.idf), combined with different score propagation and combination functions. The user can select which model to use at run time. Following good practice in database systems design, our prototype system has a typical three-layered architecture. (1) The conceptual layer takes a NEXI [3] query expression as input, e.g.,

With the rapid development of the Internet, huge volumes of text data are also increasing. These rapidly growing data generate challenges for users such as access, organizing, and analyzing the required information expeditiously. Document clustering techniques mostly rely on the statistical analysis of a term. It can be hard to identify in situations when multiple terms have the same frequency value, but one term is more important in terms of meaning than the other. Also, the process to discover more relevant information regarding a user query on the Web is uncontrollable. The proposed system tries to implement a concept-based document clustering model that clusters the Web documents based on the semantics or theme of the text data. The semantic analysis is done with the help of the semantic role labeler (SRL) to find the terms that contribute more to the meaning of the sentence. This system is called a concept-based analysis mechanism (CBAM). This system can provide accurate and robust document similarity calculation as well as concept identification that lead to improved results in Web document clustering.

The Web has become a large repository of documents (or pages) written in many different languages. In this context, traditional information retrieval (IR) techniques cannot be used whenever the user query and the documents being retrieved are in different languages. To address this problem, new cross‐language information retrieval (CLIR) techniques have been proposed. In this work, we describe a method for cross‐language retrieval of medical information. This method combines query terms and related medical concepts obtained automatically through a categorization procedure. The medical concepts are used to create a linguistic abstraction that allows retrieval of information in a language‐independent way, minimizing linguistic problems such as polysemy. To evaluate our method, we carried out experiments using the OHSUMED test collection, whose documents are written in English, with queries expressed in Portuguese, Spanish, and French. The results indicate that our cross‐language retrieval method is as effective as a standard vector space model algorithm operating on queries and documents in the same language. Further, our results are better than previous results in the literature.

To satisfy the user information needs, the use of his/her profile is adopted in the information retrieval process. Our goal is to build and exploit the user profile in search process. To achieve this goal, we adopt a Bayesian network as a strategy for personalized search since it provides a clear formalism to model qualitatively and quantitatively information types, namely, a user query, documents, index terms, and user's interests. In this paper, we present a Bayesian network model that captures the relationships between this information to evaluate the relevance as matching relationships between documents, user's interest and query concepts. An experimental evaluation was designed to appraise the exploitation impact of the user profile defined by his/her interests on the search results relevance.

Measuring the semantic similarity between phrases and sentences is an important task in natural language processing (NLP) and information retrieval (IR). We compare the quality of the distributional semantic NLP models against phrase-based semantic IR. The evaluation is based on the correlation between human judgements and model scores on a distributional phrase similarity task. We experiment with four NLP and two IR model variants. On the NLP side, models vary over normalization schemes and composition operators. On the IR side, models vary with respect to estimation of the probability of a term being in a document, namely P(t|d) where only term co-occurrence information is used and P(t|d, sim) which incorporates term distributional similarity. A mixture of the two methods is presented and evaluated. For both methods, word meanings are derived from large corpora of data: the BNC and ukWaC. One of the main findings is that grammatical distributional models give better scores than the IR models. This suggests that an IR model enriched with distributional linguistic information performs better in the long standing problem in IR of document retrieval where there is no direct symbolic relationship between query and document concepts.

Large volumes of monitoring and logging data result from the operation of a large scale astrophysical observatory. In the last few years several "Big Data" technologies have been developed to deal with such volumes of data especially in the Internet of Things (IoT) framework. We present the logging, monitoring and alarm system architecture for the ASTRI Mini-Array aimed at supporting the analysis of scientific data and improving the operational activities of the telescope facility . A prototype was designed and built considering the latest software tools and concepts coming from Big Data and IoT and a particular relevance has been given in satisfying quality requirements such as performance, scalability and availability.

Big data are the data which need to be shaped for their volume, size, and shape in order to extract meaningful information for an explicit purpose. Data are ever playing a significant role in organization and industry for their daily activities to functional smoothly. The volume of healthcare data is increasing with the rising of technology and passage of time. This rise in volume of data can be a challenging task toward analysing the big data for industry and Internet of things (IoT). Numerous approaches, techniques, and tools exist in the literature for supporting, to handle, and manage processing of data. A review of the literature is needed in order to collect existing evidence from the literature to show which method or tool works for which particular situation. Therefore, the current study presents a review of the existing techniques of big data insights and scientific programming in the industry of healthcare. The report presents the summary of the literature. The study collects evidences from the existing literature and organizes it through the process of literature review with some derivations. This review will benefit practitioners to identify the right techniques for their specific purpose of research.

The most important feature of the last twenty years in the world economy is the digitalization of the social and economic field. This new trend is a process that can not be analyzed by unconventional methods, approaches, and techniques. This process is a dynamic mechanism that involves rapidly spreading effects. Therefore, digitalization has revealed an economic and social situation in which institutions are constantly transformed, innovations are applied very quickly, and are in demand. The most important problem in the studies to be carried out on this subject is the discussions about the measurement of digitalization and whether its numerical indicators are representative of the process or not. The extent of digitalization in the economy is Fintech applications in industry 4.0 money markets and financial markets in real terms. In today’s business world, the size of the relationship between production and the market changes in the digital economy. Achieving the accumulation of knowledge in the economy at lower costs with the effect of digitalization has led to the production of an important digital information. This accumulation of knowledge led to changes in economic behavior and preferences in business models. The economic area where the effect of this change is seen most rapidly is the financial area. Digitalization in the financial area is emerging as a new source of risk. In this respect, the increase in the volume of financial data with digitalization made the necessity of new analysis techniques necessary. Data sets resulting from the increase in the volume of data are defined as big data. In general, these big data have high frequency and real time or instant data feature in the financial system. The analysis of these data is a basic tool for measuring financial risks with systemic financial risks and the risk level of the markets. Digital economy is defined as a new economic structure as a result of changing the structure of the internet and communication systems. In this new structure, economic relations are created within the framework of the relationships established between the platforms. Establishing relationships between people, firms, and institutions through platforms reveal a lot of digitizable data. The continuous accumulation of this data online makes it necessary to carry out continuous analyzes according to each piece of information that is constantly received. The analysis of the information as well as the information turns into a product of economic value. The most important tool for this new transformation is artificial intelligence. Artificial intelligence and deep learning methods with machine learning, which are its tools, also cause changes in the financial and monetary relations of the new economy. The first major impact of this change was on the banking system. The changes in the banking system and the digital currencies and the developments that emerged with Facebook’s announcement on the issue of the Libra currency cause changes in the primary functions of the central banks and in the monetary transfer mechanism. The main reason for the change in the primary function of the Central Bank and the change in the monetary transmission mechanism is the differentiation in the property of the money. The differentiation in the feature and function of the central bank has to redefine its functions along with the monetary definitions of the central banks. Within the framework of this trend, the aim of this study is to analyze the change in the structure of central banks, the characteristics of money, and the functions of monetary policies, with the artificial intelligence and digitalization process.

Nowadays a lot of massive data is stored and typically, the data itself contains a lot of non-trivial but useful information. Data mining techniques can be used to discover this information which can help the companies for decision-making. However, in real life applications, data is massive and is stored over distributed sites. One of my major research topics is to protect privacy over this kind of data. Previously, the important characteristics, issues and challenges related to management of the large amount of data has been explored. Various open source data analytics frameworks that deal with large amount of Data analytics workloads have been discussed. Comparative study between the given frameworks and suitability of the same has been proposed. Digital universe is flooded with huge amount of data generated by number of users worldwide. These data are of diverse in nature, come from various sources and in many forms. To keep with the desire to store and analyze ever larger volumes of complex data, relational databases vendors have delivered specialized analytical platforms that come in many shapes and sizes from software only to analytical services that run in third party hosted environments. In addition new technologies have emerged to address exploding volumes of complex data, including web traffic, social media content and machine generated data including sensor data, global positioning system data.Big data is defined as large amount of data which requires new technologies and architectures so that it becomes possible to extract value from it by capturing and analysis process. Due to such large size of data it becomes very difficult to perform effective analysis using the existing traditional techniques. Big data has become a prominent research field, especially when it comes to decision making and data analysis. However, Big data due to its various properties like volume, velocity, variety, variability, value and complexity put forward many challenges. Since Big data is a recent upcoming technology in the market which can bring huge benefits to the business organizations, it becomes necessary that various challenges and issues associated in bringing and adapting to this technology are brought into light. Another challenge is that data collection may not have enough accuracy which will lead to a non-consistent analysis that can critically affect the decision based on this analysis. Moreover, it is clearly apparent that organizations need to employ data-driven decision making to gain competitive advantage. Processing, integrating and interacting with more data should make it better data, providing both more panoramic and more granular views to aid strategic decision making. This is made possible via Big Data exploiting affordable and usable Computational and Storage Resources. Many offerings are based on the Map-Reduce and Hadoop paradigms and most focus solely on the analytical side. Nonetheless, in many respects it remains unclear what Big Data actually is; current offerings appear as isolated silos that are difficult to integrate and/or make it difficult to better utilize existing data and systems. Since data is growing at a huge speed making it difficult to handle such large amount of data (Exabyte). The main difficulty in handling such large amount of data is because that the volume is increasing rapidly in comparison to the computing resources. The Big data term which is being used now a days is kind of misnomer as it points out only the size of the data not putting too much of attention to its other existing properties.If data is to be used to make accurate decisions in time it becomes necessary that it should be available in accurate, complete and timely manner. This makes the data management and governance process bit complex adding the necessity to make Data open and make it available to government agencies in standardized manner with standardized APIs, metadata and formats thus leading to better decision making, business intelligence and productivity improvements.This paper presents a discussion and evaluation for the most prominent techniques used in the processes of data collection and analysis in order to identify the privacy defects in them that affects the accuracy of big data. Depending on the results of this analysis, recommendations were provided for improving data collection and analysis techniques that will help to avoid if not all then most of the problems facing the use of big data in decision making. Keywords: Big Data, Big Data Challenges, Big Data Accuracy, Big Data Collection, Big Data Analytics.