Text Retrieval analysis based on Deep Learning

In the text clustering, the traditional clustering algorithm is easy to fall into the local optimal for exploring the clustering center, which will affect the results of text clustering. This paper proposes a text clustering method based on the improved heuristic three-parent genetic algorithm. In this paper, the heuristic three-parent genetic algorithm is improved. Furthermore, by enhancing the crossover strategy and introducing the unequal crossover, the genetic algorithm’s convergence speed and global search ability are improved. In text clustering, three-parent with the improved heuristic genetic algorithm to find the optimal clustering center. Through the change of fitness function, the clustering center is more suitable for text clustering results. Using the clustering center guide text embedded to lower-dimensional coding of high-dimensional text, finally use the Deep Embedded Clustering (DEC) to improve the effect of clustering. The clustering results are evaluated in terms of Accuracy and Mutual Information. The experimental results show that the algorithm performs well in selecting clustering centers and clustering results.

Deep learning (also known as deep structured learning, hierarchical learning, or deep machine learning) is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using a deep graph with multiple processing layers, composed of multiple linear and nonlinear transformations. Deep learning has been characterized as a class of machine learning algorithms with the following characteristics [257]: • They use a cascade of many layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. The algorithms may be supervised or unsupervised and applications include pattern analysis (unsupervised) and classification (supervised). • They are based on the (unsupervised) learning of multiple levels of features or representations of the data. Higher-level features are derived from lower-level features to form a hierarchical representation. • They are part of the broader machine learning field of learning representations of data. • They learn multiple levels of representations that correspond to different levels of abstraction; the levels form a hierarchy of concepts. These definitions have in common: multiple layers of nonlinear processing units and the supervised or unsupervised learning of feature representations in each layer, with the layers forming a hierarchy from low-level to high-level features. Various deep learning architectures such as deep neural networks, convolutional deep neural networks, deep belief networks (DBN), and recurrent neural networks have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition, and bioinformatics where they have been shown to produce state-of-the-art results on various tasks. In this chapter we start in Section 7.1 with an introduction, giving a brief history of this field, the relevant literature, and its applications. Then we study some basic concepts of deep learning such as convolutional neural networks, recurrent neural networks, backpropagation algorithm, restricted Boltzmann machines, and deep learning networks in Section 7.2. Then we illustrate three examples for Apache Spark implementation for mobile big data (MBD), user moving pattern extraction, and combination with nonparametric Bayesian learning, respectively, in Sections 7.3 through 7.5. Finally, we have summary in Section 7.6.

Introduction. The significance of machine learning under the conditions of digital transformation of industry, and methods of implementing deep learning to provide the performance of trust management systems are considered. The necessity of using convolutional artificial neural networks for deep machine learning is determined. Various technologies and architectures for the implementation of artificial neural networks are briefly considered; a comparative analysis of their performance is carried out. The work objective is to study the need to develop new approaches to the architecture of computing machines for solving problems of deep machine learning in the trust management system implementation.Materials and Methods. In the context of digital transformation, the use of artificial intelligence reaches a new level. The technical implementation of artificial neural systems with deep machine learning is based on the use of one of three basic technologies: high performance computing (HPC) with parallel data processing, neuromorphic computing (NC), and quantum computing (QC).Results. Implementation models for deep machine learning, basic technologies and architecture of computing machines, as well as requirements for trust assurance in control systems using deep machine learning are analyzed. The problem of shortage of computation power for solving such problems is identified. None of the currently existing technologies can solve the full range of learning and impedance problems. The current level of technology does not provide information security and reliability of neural networks. The practical implementation of trust management systems with deep machine learning based on existing technologies for a significant part of the tasks does not provide a sufficient level of performance.Discussion and Conclusions. The study made it possible to identify the challenge of the computation power shortage for solving problems of deep machine learning. Through the analysis of the requirements for trust management systems, the external challenges of their implementation on the basis of existing technologies, and the need to develop new approaches to the computer architecture are determined.

Chapter 2 - Computational intelligence in healthcare and biosignal processing

In recent years, various data acquisition methods have produced a large number of data acquisition methods from multiple perspectives. Mufti view data clustering is of great significance and challenge in large-scale data analysis. In particular, the importance of clustering analysis in data mining is discussed. Based on the practical application and theoretical research of data mining, the research status of data mining technology at home and abroad is analyzed. The fuzzy membership method is used to describe the probability that the sample belongs to a certain class. Firstly, the constraint parameter analysis model is established. The evaluation model suitable for modern teaching mode such as micro course and flipped classroom is constructed, and fuzzy clustering technology is adopted. The number of clusters depends on the distance between the cluster node and the whole cluster.

Text clustering is used to extract specific information from textual data and even categorizes text based on topic and sentiment. Due to inherent overlapping in textual documents, overlapping clustering algorithms have become a suitable approach for text analysing. However, state-of-the-art algorithms are not fast enough to analyse a large volume of textual data within tolerable time limits. In this research, we propose our text clustering algorithm, FOCT, which is a fast overlapping extension of SOM, one of the best algorithms for clustering textual data. We apply some heuristics to extract special characteristics presented in textual data and establish a very fast overlapping clustering algorithm. We use fast methods to represent the vectors of documents, compute the similarity of documents and neurons and update the weights of neurons. In our algorithm, each document can belong to one or more neurons and this is in line with what many documents have in their essence. We analyse the efficiency of the proposed algorithm over k-means, OKM, SOM and OSOM clustering approaches and experimentally demonstrate that it runs 12 to 690 times faster, and the overlap size of FOCT clusters is closer to the overlap size of the original data. The quality of clusters is also measured by four different internal and external evaluation criteria where FOCT clusters represent up to 64% better quality.

As an unsupervised machine learning technology, document clustering has been widely used in many fields, such as Information Retrieval (IR) and Text Categorization (TC). But, because of the bag of words used in document clustering as document index, the feature space of corpus must be high dimension space. This problem makes a negative effect to the efficiency and precision of text clustering. Based on category resolve power, a new feature selection function is constructed. Through integration between this function and document clustering algorithm, a high-powered text clustering algorithm is presented. Experiments on a universal corpus show that it has a good performance.

Deep learning belongs to the field of artificial intelligence, where machines perform tasks that typically require some kind of human intelligence. Deep learning tries to achieve this by drawing inspiration from the learning of a human brain. Similar to the basic structure of a brain, which consists of (billions of) neurons and connections between them, a deep learning algorithm consists of an artificial neural network, which resembles the biological brain structure. Mimicking the learning process of humans with their senses, deep learning networks are fed with (sensory) data, like texts, images, videos or sounds. These networks outperform the state-of-the-art methods in different tasks and, because of this, the whole field saw an exponential growth during the last years. This growth resulted in way over 10,000 publications per year in the last years. For example, the search engine PubMed alone, which covers only a sub-set of all publications in the medical field, provides already over 11,000 results in Q3 2020 for the search term ‘deep learning’, and around 90% of these results are from the last three years. Consequently, a complete overview over the field of deep learning is already impossible to obtain and, in the near future, it will potentially become difficult to obtain an overview over a subfield. However, there are several review articles about deep learning, which are focused on specific scientific fields or applications, for example deep learning advances in computer vision or in specific tasks like object detection. With these surveys as a foundation, the aim of this contribution is to provide a first high-level, categorized meta-survey of selected reviews on deep learning across different scientific disciplines and outline the research impact that they already have during a short period of time. The categories (computer vision, language processing, medical informatics and additional works) have been chosen according to the underlying data sources (image, language, medical, mixed). In addition, we review the common architectures, methods, pros, cons, evaluations, challenges and future directions for every sub-category.

Text clustering is one of the fundamental tasks in natural language processing which involves grouping similar documents together based on their content thus enabling efficient organization and analysis of textual data. In this paper we provide a comprehensive survey of text clustering techniques, its applications, challenges, and future directions. We begin by discussing the fundamentals of text clustering, including key concepts such as similarity measures, text feature representations, and clustering algorithms. We also explore popular text clustering algorithms such as K-means, hierarchical clustering, density- based clustering, spectral clustering, affinity propagation, and Latent Dirichlet Allocation (LDA) popularly used for topic modelling. For every algorithm we discuss its methodology, strengths, limitations, and parameter tuning considerations. We also dive deep into the real-world applications of text clustering across diverse domains, including document organization, information retrieval, text summarization, sentiment analysis, and recommendation systems and highlight their effectiveness with case studies and examples. We also identify several challenges and open research questions in text clustering, such as scalability, handling high-dimensional data, incorporating domain specific knowledge in clustering, evaluation metrics, and integration with other NLP tasks such NER, classification, etc. Finally, we propose potential future directions for research to address these challenges in order to advance the field of text clustering. In conclusion, text clustering continues to be an interesting area of research with immense potential for applications in various domains which helps drive innovation in natural language processing.

Nowadays, a number of short messages or short text contents created on the Internet are rapidly increasing. Tasks to manipulate, analyze, and extract knowledge from them lead mining techniques such as text clustering to become more important. However, applying traditional text clustering algorithms which consider only common words or phrases to group short texts is inefficient due to the problem of sparsity. In this paper, we propose a new clustering technique, called word semantic graph clustering, based on the use of text concepts. We apply the word embedding model from Word2Vec to capture the semantic meaning of words and later construct semantic subgraphs in which those words represented as vertices are connected by some high semantic similarities. Finally, short text documents will be assigned to the same cluster if they contain at least one word belonging to the same semantic subgraph. Experimental results conducted on two real datasets show that the proposed approach outperforms the state-of-the-art text clustering algorithms. In addition, it can also produce more appropriate label for each cluster than the comparative algorithms do.

Improving Alzheimer’s disease classification using novel rewards in deep reinforcement learning

Image segmentation is the process of delineating the regions occupied by objects of interest in a given image. This operation is a fundamentally required first step in numerous applications of medical imagery. In the medical imaging field, this activity has a rich literature that spans over 45 years. In spite of numerous advances, including deep learning (DL) networks (DLNs) in recent years, the problem has defied a robust, fail-safe, and satisfactory solution, especially for objects that are manifest with low contrast, are spatially sparse, have variable shape among individuals, or are affected by imaging artifacts, pathology, or post-treatment change in the body. Although image processing techniques, notably DLNs, are uncanny in their ability to harness low-level intensity pattern information on objects, they fall short in the high-level task of identifying and localizing an entire object as a gestalt. This dilemma has been a fundamental unmet challenge in medical image segmentation. In this paper, we demonstrate that by synergistically marrying the unmatched strengths of high-level human knowledge (i.e., natural intelligence (NI)) with the capabilities of DL networks (i.e., artificial intelligence (AI)) in garnering intricate details, these challenges can be significantly overcome. Focusing on the object recognition task, we formulate an anatomy-guided DL object recognition approach named Automatic Anatomy Recognition-Deep Learning (AAR-DL) which combines an advanced anatomy-modeling strategy, model-based non-DL object recognition, and DL object detection networks to achieve expert human-like performance.

The advent of the novel coronavirus disease (COVID-19) in late December 2019 led to the dramatic loss of human life worldwide and presented an unprecedented challenge to public health, education, social life, world economics, and the world of work. Equal access to safe and effective vaccines is very vital to ending the coronavirus pandemic. This research paper aims to perform text clustering on COVID-19 vaccine tweets. It investigates the optimal number of clusters prevalent in the COVID-19 vaccine corpus using deep learning techniques and machine learning algorithms. The study also investigates how using word embeddings can improve the accuracy of the proposed models by evaluating unsupervised learning methods. Machine learning clustering algorithms such as k-means and HDBSCAN, deep learning-based clustering techniques, and UMAP a dimensionality reduction algorithm were employed to perform text clustering. The results of this research showed the optimal clusters obtained by using deep learning clustering techniques and machine-learning algorithms for text clustering. HDBSCAN clustering algorithm showed better clustering results based on features learned while k-means performed better clustering based on various evaluation metrics.

Knowledge-driven building extraction method exhibits a restricted adaptability scope and is vulnerable to external factors that affect its extraction accuracy. On the other hand, data-driven building extraction method lacks interpretability, heavily relies on extensive training data, and may result in extraction outcomes with building boundary blur issues. The integration of pre-existing knowledge with data-driven learning is essential for the intelligent identification and extraction of buildings from high-resolution aerial images. To overcome the limitations of current deep learning building extraction networks in effectively leveraging prior knowledge of aerial images, a geometric significance-aware deep mutual learning network (GSDMLNet) is proposed. Firstly, the GeoSay algorithm is utilized to derive building geometric significance feature maps as prior knowledge and integrate them into the deep learning network to enhance the targeted extraction of building features. Secondly, a bi-directional guidance attention module (BGAM) is developed to facilitate deep mutual learning between the building feature map and the building geometric significance feature map within the dual-branch network. Furthermore, the deployment of an enhanced flow alignment module (FAM++) is utilized to produce high-resolution, robust semantic feature maps with strong interpretability. Ultimately, a multi-objective loss function is crafted to refine the network’s performance. Experimental results demonstrate that the GSDMLNet excels in building extraction tasks within densely populated and diverse urban areas, reducing misidentification of shadow-obscured regions and color-similar terrains lacking building structural features. This approach effectively ensures the precise acquisition of urban building information in aerial images.

Deep learning networks were employed to predict the maximum differential deflection of stiffened and waffle rafts due to reactive soil movements, Δmax. Four deep learning networks were used to predict Δmax, these are (1) stiffened rafts on shrinking soil, (2) stiffened rafts on swelling soil, (3) waffle rafts on shrinking soil, and (4) waffle rafts on swelling soil. The deep learning models were used to create design lines, which showed that both soil and structural features strongly influence the stiffened rafts. In contrast, waffle rafts showed a strong dependence on soil features in shrinking soils and beam depth in swelling soils. This demonstrates that the finite element-based deep learning networks captured the effect of the embedment of the beams. The results of the deep learning models led to non-linear design curves, which are disparate from the suggested standard Australian design. These results suggest that increasing the value of beam depth can have a positive or negative impact on the global residential slab depending on the type of substructure and whether the founding reactive soil is shrinking or swelling. Global sensitivity analyses of the deep learning models showed that for stiffened rafts on shrinking soil, the slab length, slab width and active depth zone of reactive soil had the most significant influence on Δmax, whilst for stiffened rafts on swelling soil, the primary drivers are ground movement, beam depth, and slab width. The prediction of Δmax for waffle rafts on shrinking soil was driven by the surface characteristic and mound movements, and the active depth zone, whilst waffle rafts on swelling soil was driven by the beam depth. Overall, the finite element-based deep learning showed the capacity to estimate Δmax in both shrinking and swelling design scenarios for different types of residential footing systems to further understand the characteristic behaviour of shallow residential slab foundations on reactive soils leading to improved designs.