Detection and Localization of Text Information in Video Frames

Image retrieval is one of the most interesting and fastest growing research areas in all fields. It is an effective and efficient tool for managing large image databases. In most Content-Based Image Retrieval (CBIR) systems, an image is represented by a set of low-level visual features; hence a direct correlation with high-level semantic information will be absent. Therefore, a gap exists between high-level information and low-level features, which is the main reason that hinders the improvement of the image retrieval accuracy. In this work, main focus is on the semantic based image retrieval system using Gray Level Co-occurrence Matrix (GLCM) for texture feature extraction. Based on the texture features, semantic interpretations are given to the extracted textures. The images are retrieved according to user satisfaction and thereby reduce the semantic gap between low level features and high level features.

In a Content Based Image Retrieval (CBIR) system, images are retrieved based on the content of the images such as color, shapes and objects. However, often end users experience a semantic gap between the notion of similarity used by the ranking model versus the users’ perception about image similarity. Explainable AI (XAI) is an emerging research field that attempts to provide transparency of “black box” models, to make AI systems trustworthy and gain user trust. This work aims at building an I mage R etrieval system with T extual EX planations such as “The (global) results are similar to the query by X% due to shape, Y% due to color”. Local explanations are generated using various methods such as comparing images with respect to overlap between low level features such as color, shape and regions (MPEG-7 features). Additionally, high level features such as background-foreground segmentation, deep learned features and major key-points, objects identified (SIFT features) are used to enrich the explanations. We evaluate the quality of rankings on benchmark data-sets such as PASCAL VOC. The XAI facet of user satisfaction and usefulness of the system is evaluated in a lab based user study. Our results show that the semantic gap is better bridged using high level features. While low level features might be better suited for re-ranking of the retrieved images.

Content-based video retrieval system aims at assisting a user to retrieve targeted video sequence in a large database. Most of the search engines use textual annotations to retrieve videos. These types of engines offer a low-level abstraction while the user seeks high-level semantics. Bridging this type of semantic gap in video retrieval remains an important challenge. In this paper, colour, texture and shapes are considered to be low-level features and motion is a high-level feature. Colour histograms convert the RGB colour space into YcbCr and extract hue and saturation values from frames. After colour extraction, filter mask is applied and gradient value is computed. Gradient and threshold values are compared to draw the edge map. Edges are smoothed for sharpening to remove the unnecessary connected components. These diverse shapes are then extracted and stored in shape feature vectors. Finally, an SVM classifier is used for classification of low-level features. For high-level features, depth images are extracted for motion feature identification and classification is done via echo state neural networks (ESN). ESN are a supervised learning technique and follow the principle of recurrent neural networks. ESN are well known for time series classification and also proved their effective performance in gesture detection. By combining the existing algorithms, a high-performance multimedia event detection system is constructed. The effectiveness and efficiency of proposed event detection mechanism is validated using MSR 3D action pair dataset. Experimental results show that the detection accuracy of proposed combination is better than those of other algorithms

As the prevalence of online services and platforms continues to grow, so too does the threat posed by automated attacks seeking to circumvent security measures such as CAPTCHA (Completely Automated Public Turing test to tell Computers and Humans Apart) mechanisms. This research paper delves into the various attack vectors and vulnerabilities inherent in CAPTCHA systems, aiming to provide a comprehensive understanding of the methodologies employed by attackers to bypass these safeguards. Through an investigation into machine learning-based algorithms, optical character recognition (OCR) techniques, and adversarial attacks, this study sheds light on the evolving landscape of CAPTCHA circumvention strategies. By analyzing the strengths and limitations of each attack vector, as well as the potential countermeasures available, this research contributes to the ongoing efforts to enhance the resilience of CAPTCHA systems against automated threats. Ultimately, the findings of this study serve to inform the development of more robust and adaptive CAPTCHA designs, thereby bolstering cyber security in the digital realm. This research paper focuses on investigating attack vectors and vulnerabilities in CAPTCHA mechanisms, including machine learning-based algorithms, OCR techniques, and adversarial attacks. It emphasizes the importance of understanding these methodologies to inform the development of more resilient CAPTCHA designs that can secure online platforms from various attacks.

As the use of digital media increases, effective retrieval and management techniques become more important. In order to extract useful information from this huge amount of data, many content based image retrieval (CBIR) systems have been developed. A typical CBIR system captures image features such as color, texture, or shape of objects in the query image and tries to retrieve images from the database with similar features. The main advantage of CBIR systems with relevance feedback is that these systems take into account the gap between the high-level concepts and low-level features and subjectivity of human perception of visual content. In this paper, a CBIR system based on color histogram has been presented and a new approach for image storage and retrieval called association-based image retrieval (ABIR) has been studied. The results of both systems have been shown.

With the development of theories and technologies in medical imaging, most of the tumors can be detected in the early stage. However, the nature of ovarian cysts lacks accurate judgement, leading to that many patients with benign nodules still need Fine Needle Aspiration (FNA) biopsies or surgeries, increasing the physical pain and mental pressure of patients as well as unnecessary medical health care costs. Therefore, we present an image diagnosis system for classifying the ovarian cysts in color ultrasound images, which novelly applies the image features fused by both high-level features from deep learning network and low-level features from texture descriptor. Firstly, the ultrasound images are enhanced to improve the quality of training data set and the rotation invariant uniform local binary pattern (ULBP) features are extracted from each of the images as the low-level texture features. Then the high-level deep features extracted by the fine-tuned GoogLeNet neural network and the low-level ULBP features are normalized and cascaded as one fusion feature that can represent both the semantic context and the texture patterns distributed in the image. Finally, the fusion features are input to the Cost-sensitive Random Forest classifier to classify the images into "malignant" and "benign". The high-level features extracted by the deep neural network from the medical ultrasound image can reflect the visual features of the lesion region, while the low-level texture features can describe the edges, direction and distribution of intensities. Experimental results indicate that the combination of the two types of features can describe the differences between the lesion regions and other regions, and the differences between lesions regions of malignant and benign ovarian cysts.

Automatic liver tumor segmentation from 3D Computed Tomography (CT) is a necessary prerequisite in the interventions of hepatic abnormalities and surgery planning. However, accurate liver tumor segmentation remains challenging due to the large variability of tumor sizes and inhomogeneous texture. Recent advances based on Fully Convolutional Network (FCN) in liver tumor segmentation draw on success of learning discriminative multi-level features. In this paper, we propose a Deep Attentive Refinement Network (DARN) for improved liver tumor segmentation from CT volumes by fully exploiting both low and high level features embedded in different layers of FCN. Different from existing works, we exploit attention mechanism to leverage the relation of different levels of features encoded in different layers of FCN. Specifically, we introduce a Semantic Attention Refinement (SemRef) module to selectively emphasize global semantic information in low level features with the guidance of high level ones, and a Spatial Attention Refinement (SpaRef) module to adaptively enhance spatial details in high level features with the guidance of low level ones. We evaluate our network on the public MICCAI 2017 Liver Tumor Segmentation Challenge dataset (LiTS dataset) and it achieves state-of-the-art performance. The proposed refinement modules are an effective strategy to exploit multi-level features and has great potential to generalize to other medical image segmentation tasks.

Content-Based Image Retrieval (CBIR) has become a critical technology for efficiently searching and retrieving images from large datasets based on their visual content. Traditional CBIR systems, which rely on low-level features like color, texture, and shape, often struggle with semantic gaps and scalability issues. With the rapid advancements in deep learning and cloud computing, there is a growing need to enhance CBIR performance for real-world applications. In order to tackle the issues, an enhanced CBIR process leveraging advanced neural networks, particularly Convolutional Neural Networks (CNNs), Siamese Networks, and attention mechanisms is proposed. It integrates multimodal data, including text and audio, to improve retrieval accuracy. Additionally, cloud-based infrastructure is employed to support large-scale image processing, enabling faster retrieval times and real-time performance. Edge computing techniques are also incorporated to reduce latency in applications requiring immediate responses. The proposed model demonstrates significant improvements in retrieval accuracy and efficiency compared to traditional CBIR methods. Deep learning models, particularly CNNs with transfer learning and attention mechanisms, effectively capture high-level semantic features. The integration of cloud infrastructure enhances scalability and real-time processing capabilities, while multimodal retrieval improves search relevance. The use of explainable AI techniques adds transparency to the decision-making process, increasing user trust. Hence, the advanced neural networks, coupled with cloud and edge computing, can significantly optimize CBIR systems, making them more robust, scalable, and applicable to a wide range of industries such as healthcare, security, and e-commerce.

In order to improve the performance of content-based image retrieval (CBIR) systems, the 'semantic gap' between the low-level visual features and the high-level semantic features attracts more and more research interest. We propose an approach to describe and to extract the fuzzy color semantics. According to human color perception model, we utilize the linguistic variable to describe the image color semantics, so it becomes possible to depict the image in linguistic expression such as mostly red. Furthermore, we apply the feedforward neural network to model the vagueness of human color perception and to extract the fuzzy semantic feature vector. Our experiments show that the color semantic features have good accordance with the human perception, and also have good retrieval performance. In some extent, our approach shows the potential to reduce the semantic gap in CBIR.

In this paper, we propose a text extraction method from camera-captured document style images and propose a text segmentation method based on a color clustering method. The proposed extraction method detects text regions from the images using two low-level image features and verifies the regions through a high-level text stroke feature. The two level features are combined hierarchically. The low-level features are intensity variation and color variance. And, we use text strokes as a high-level feature using multi-resolution wavelet transforms on local image areas. The stroke feature vector is an input to a SVM (support vector machine) for verification, when needed. The proposed text segmentation method uses color clustering to the extracted text regions. We improved K-means clustering method and it selects K and initial seed values automatically. We tested the proposed methods with various document style images captured by three different cameras. We confirmed that the extraction rates are good enough to be used in real-life applications.

With the emergence and explosion of huge image databases there is an increasing necessity for effective methods to assess visual information on the level of objects and scene types. A wide variety of Content – Based Image Retrieval (CBIR) systems already exists. As a key issue in CBIR, similarity measure quantifies the resemblance in contents between a pair of images. Depending on the type of features, the formulation of the similarity measure varies greatly. The primary goal of our study is to reduce the computation time and user interaction. The secondary goal is to reduce the semantic gap between high level concepts and low level features. A third goal is to evaluate system performance with regard to speed and accuracy. In the proposed study transform color after statistical transform, such as the Adaptive Color Karhunen Loeve Transform (ACKLT) is used as a color descriptor. The results are showing the advantage of the new algorithm for ACKLT in comparison with the YCrCb color model. Based on the experimental results, we concluded that correct selection of descriptors invariant to light intensity and light color changes affects object and scene category recognition.Keywordscontent-based image retrievalAdaptive Color Karhunen Loeve Transform

How to properly involve text characteristics like multi-scale, arbitrary direction, length aspect ratio, into detection network design has become a hot topic in computer vision. Feature Pyramid Network (FPN) is a typical method to achieve robust text detection, where its low-level and high-level feature map retains spatial structure and global semantic information, respectively. However, its strict hierarchical structure fails to fuse low-level and high-level information to improve the distinguish ability of feature map. To address this problem, we propose a novel feature fusion pyramid network for end-to-end scene text detection by fusing multi-modal information. By dividing pyramid structure into high-level and low-level layers, channel and spatial attention modules are adopted to enhance high-level and low-level feature representation by encoding channel and spatial-wise context information, respectively. In order to reduce information loss by layer transmission, a special residual network is designed to achieve short-cut between high-level and low-level features, so as to realize multi-modal feature fusion. Experiments show the precision and recall of the proposed method on ICDAR2015, ICDAR2017-MLT, and MSRA-TD500 datasets reach 88.7%/82.1%, 77.0%/60.3%, and 85.3%/74.8%, respectively.

Retrieving images depend of specific features In Content Based Image Retrieval (CBIR). A common approach is to divide retrieval process into two stages; the first one is based on high-level features followed by the second that is based on low-level features. We focus primarily on medical images, and follow the above approach but make the following two basic contributions: a) introduce the gray cluster co-occurrence matrix as texture feature extraction and use it as high-level features, and b) introduce edge strength levels as shape feature extraction and use it as low-level features. Our proposed system suggests the precision rate was 94.90% and recall rate was 89.72%. The distance variance achieved lowest rate (0.0022) in images retrieval compared to each of partial systems individually and related works. Our method has better performance in retrieving the results than other related works and each of partial system individually.

Self-similarity is often an indicator of highly aesthetic paintings; a key aspect is what feature to use for evaluating self-similarity. Previous works use low-level features such as the gradient orientation to show that artworks and natural scenes share a similar degree of self-similarity. In this study, we take advantage of the AlexNet model to evaluate the changes of self-similarity at different convolutional layers in a CNN model. Compared to previous measures, our approach takes into account low, mid, and high level features. Different behaviors with regards to self-similarity at different layers is observed in paintings. The results confirm previous findings that artworks and natural scenes share similar degrees of self-similarity. For paintings and photographs with similar subject matters, while different degrees of self-similarity are observed at the first layer, other layers show closer values. Finally, the proposed measure of self-similarity is able to better differentiate between images which belong to a similar category but different datasets of images.

Hyperspectral images have high spectral resolution, but this is often at the expense of spatial resolution. Although deep learning-based super-resolution (SR) algorithms have shown comparative performance for spatial resolution enhancement, most of them cannot effectively extract features of different size objects because of single scale convolution. In deep architectures, low level features also tend to disappear during transmission. In this paper, an efficient network (MRBMFF) for enhancing the spatial resolution of hyperspectral image is proposed. Based on the multiscale residual block (MRB), features at different scales can be effectively extracted and fused. Meanwhile, the multilevel feature fusion (MFF) is introduced to concatenate the low and high level features. Effective SR images could be recovered after inputting their low-resolution counterparts to the proposed network. Experimental results show that the proposed network achieves superior reconstruction performance compared with the state-of-the-art approaches.