Feature Fusion Pyramid Network for End-to-End Scene Text Detection

Recently, scene text detection has become an active research field, which is an essential component of scene text reading. Especially, segmentation-based methods are commonly used, since the segmentation results can describe text of arbitrary shape. However, curve texts have a diversity of shapes, scales and orientations, which are difficult to locate, so the detector requires to adjust the local receptive fields size adaptively, which can aggregate multi-scale spatial information to accurately locate the curve text instance. Moreover, the low-level features are critical for localizing large text instances. When using Feature Pyramid Network (FPN) for multi-scale feature fusion, it will prevent the flow of accurate localization signals due to the long path from low-level to top-level. In order to solve these two problems, this paper proposes an Adaptive Convolution and Path Enhancement Pyramid Network (ACPEPNet), which can more accurately locate the text instances with arbitrary shapes. Firstly, an Adaptive Convolution Unit is introduced to improve the ability of backbone to aggregate multi-scale spatial information at the same stage. Specially, this unit is a lightweight component and without the cost of computations, based on this component we present a backbone network for text features extraction. Secondly, the original FPN structure is redesigned to build a short path from the low-level to top-level, in this way, we modify the path from one-way flow to two-way flow and add original features to the final stage of information fusion. Experiments on CTW1500, Total-Text, ICDAR 2015 and MSRA-TD500 validate the robustness of the proposed method. When there is no bells and whistles, this method achieves an F-measure of 80.8% without external training data on CTW1500.

A pooling-based feature pyramid network for salient object detection

With the development of theories and technologies in medical imaging, most of the tumors can be detected in the early stage. However, the nature of ovarian cysts lacks accurate judgement, leading to that many patients with benign nodules still need Fine Needle Aspiration (FNA) biopsies or surgeries, increasing the physical pain and mental pressure of patients as well as unnecessary medical health care costs. Therefore, we present an image diagnosis system for classifying the ovarian cysts in color ultrasound images, which novelly applies the image features fused by both high-level features from deep learning network and low-level features from texture descriptor. Firstly, the ultrasound images are enhanced to improve the quality of training data set and the rotation invariant uniform local binary pattern (ULBP) features are extracted from each of the images as the low-level texture features. Then the high-level deep features extracted by the fine-tuned GoogLeNet neural network and the low-level ULBP features are normalized and cascaded as one fusion feature that can represent both the semantic context and the texture patterns distributed in the image. Finally, the fusion features are input to the Cost-sensitive Random Forest classifier to classify the images into "malignant" and "benign". The high-level features extracted by the deep neural network from the medical ultrasound image can reflect the visual features of the lesion region, while the low-level texture features can describe the edges, direction and distribution of intensities. Experimental results indicate that the combination of the two types of features can describe the differences between the lesion regions and other regions, and the differences between lesions regions of malignant and benign ovarian cysts.

Image retrieval is one of the most interesting and fastest growing research areas in all fields. It is an effective and efficient tool for managing large image databases. In most Content-Based Image Retrieval (CBIR) systems, an image is represented by a set of low-level visual features; hence a direct correlation with high-level semantic information will be absent. Therefore, a gap exists between high-level information and low-level features, which is the main reason that hinders the improvement of the image retrieval accuracy. In this work, main focus is on the semantic based image retrieval system using Gray Level Co-occurrence Matrix (GLCM) for texture feature extraction. Based on the texture features, semantic interpretations are given to the extracted textures. The images are retrieved according to user satisfaction and thereby reduce the semantic gap between low level features and high level features.

Selective feature connection mechanism: Concatenating multi-layer CNN features with a feature selector

Scene text detection and recognition are vital challenging tasks in computer vision, which are to detect and recognize sequences of texts in natural scenes. Recently, researchers have investigated a lot of state-of-the-art methods to improve the accuracy and efficiency of text detection and recognition. However, there has been little research on text detection and recognition in natural scenes in Vietnam. In this paper, a deep learning-based empirical investigation of Vietnamese scene text detection and recognition is presented. Firstly, four detection models including differentiable binarization network (DBN), pyramid mask text detector (PMTD), pixel aggregation network (PAN), and Fourier contour embedding network (FCEN), are employed to detect text regions from the images. Then, four text recognition models including convolutional recurrent neural network (CRNN), self-attention text recognition network (SATRN), no-recurrence sequence-to-sequence text recognizer (NRTR), and RobustScanner (RS) are also investigated to recognize the texts. Moreover, data augmentation methods are also applied to enrich data for improving the accuracy and enhancing the performance of scene text detection and recognition. To validate the effectiveness of scene text detection and recognition models, the VinText dataset is employed for evaluation. Empirical results show that PMTD and SATRN achieve the highest scores among the others for text detection and recognition, respectively. For knowledge-sharing, our implementation is publicly available at https://github.com/ThorPham/VN_scene_text_detection_recognition.

Video text detection plays an important role in semantic-based video analysis. In the field of information retrieval, text detection and extraction from video has become an emerging area to solve the fundamental problem of content-based image retrieval (CBIR) to fill in the semantic gap between low level and high level features. Text detection and extraction enables the understanding of video contents with the help of text recognition using optical character recognition techniques, to give a partial solution to bridge the semantic gap between the low level and high level features. In this paper we have proposed a method which detect & localize horizontal text with uniform background.

The influence of architectural design on well-being is increasingly recognised, underscoring the need to understand how humans evaluate their surroundings. This study examines the impact of low-level visual features and high-level architectural elements in images of house facades generated by a generative adversarial network (GAN). We introduce a novel method to create controlled image datasets using the style mixing component of StyleGAN2-ADA, enabling the combination of specific architectural features with styles representing low-level image properties, such as brightness or colour. Our dataset consists of 900 images encompassing five high-level features (house size, garage door visibility, number of windows, entrance door visibility, and roof shape) and 16 low-level visual features. These GAN-generated images were rated in an online experiment by 303 participants on six dimensions: facelikeness, hominess, relaxation, invitingness, safety, and price. Linear mixed-effects models identified significant predictors, including the number of floors as a high-level feature and green pixel percentage, saturation (SD), brightness (SD), and contrast as low-level features. This indicated that houses with two floors and images with an overall higher amount of green pixel, as well as diversity of saturation and brightness and a lower contrast were rated as, e.g., more inviting and safer. The analysis showed that high- and low-level features explained up to 54% of the variance in house price ratings, with high-level features accounting for the larger share and highlighting their relevance for house facade evaluations. This work advances the use of GANs and offers a method to modify visual elements that shape architectural evaluations.

Effective scene representation is a fundamental part of high-resolution scene classification systems. In this letter, we present a holistic scene representation method, i.e., the two-level feature representation (TLFR) model. The TLFR is composed of low-level and high-level features. Low-level features are obtained by computing the residual error between a local descriptor and its corresponding visual word, and the high-level features are obtained using a proposed selection-constrained sparse coding method. In addition, low-level features in a cluster are integrated by summation pooling, whereas high-level features are fused by maximization pooling. The holistic scene representation is finally generated by incorporating these two levels of features into the bag-of-visual-words framework. Experimental results show that the TLFR model is robust to translation and rotation variations and demonstrates promising performance with the Land Use and Land Cover Database data set and a newly released Singapore data set.

Scene text detection has attracted great interest from the computer vision and pattern recognition communities since text information plays an important role in image indexing and scene understanding. Deep neural networks have become popular for the task of scene text detection, especially for their ability to learn strong text features. However, existing deep learning based state-of-the-art scene text detection methods detect texts only from a single feature map which is unable to capture semantic information at all scales. In this paper, we propose a novel deep learning based model that leverages the pyramid structure of feature maps for accurate scene text detection. We also design a deep convolutional neural network model for non-maximum suppression. In addition, we develop a novel loss function and training method for end-to-end training. The experimental results validate that our end-to-end system is simple, fast, and achieves high accuracy on standard datasets, namely, ICDAR 2015 and MSRA-TD500. We also create a dataset for scene text detection.

Automatic liver tumor segmentation from 3D Computed Tomography (CT) is a necessary prerequisite in the interventions of hepatic abnormalities and surgery planning. However, accurate liver tumor segmentation remains challenging due to the large variability of tumor sizes and inhomogeneous texture. Recent advances based on Fully Convolutional Network (FCN) in liver tumor segmentation draw on success of learning discriminative multi-level features. In this paper, we propose a Deep Attentive Refinement Network (DARN) for improved liver tumor segmentation from CT volumes by fully exploiting both low and high level features embedded in different layers of FCN. Different from existing works, we exploit attention mechanism to leverage the relation of different levels of features encoded in different layers of FCN. Specifically, we introduce a Semantic Attention Refinement (SemRef) module to selectively emphasize global semantic information in low level features with the guidance of high level ones, and a Spatial Attention Refinement (SpaRef) module to adaptively enhance spatial details in high level features with the guidance of low level ones. We evaluate our network on the public MICCAI 2017 Liver Tumor Segmentation Challenge dataset (LiTS dataset) and it achieves state-of-the-art performance. The proposed refinement modules are an effective strategy to exploit multi-level features and has great potential to generalize to other medical image segmentation tasks.

Traffic sign detection algorithm based on improved YOLOv4-Tiny

Human scene categorization is characterized by its remarkable speed. While many visual and conceptual features have been linked to this ability, significant correlations exist between feature spaces, impeding our ability to determine their relative contributions to scene categorization. Here, we used a whitening transformation to decorrelate a variety of visual and conceptual features and assess the time course of their unique contributions to scene categorization. Participants (both sexes) viewed 2250 full-color scene images drawn from 30 different scene categories while having their brain activity measured through 256-channel EEG. We examined the variance explained at each electrode and time point of visual event-related potential (vERP) data from nine different whitened encoding models. These ranged from low-level features obtained from filter outputs to high-level conceptual features requiring human annotation. The amount of category information in the vERPs was assessed through multivariate decoding methods. Behavioral similarity measures were obtained in separate crowdsourced experiments. We found that all nine models together contributed 78% of the variance of human scene similarity assessments and were within the noise ceiling of the vERP data. Low-level models explained earlier vERP variability (88 ms after image onset), whereas high-level models explained later variance (169 ms). Critically, only high-level models shared vERP variability with behavior. Together, these results suggest that scene categorization is primarily a high-level process, but reliant on previously extracted low-level features.SIGNIFICANCE STATEMENT In a single fixation, we glean enough information to describe a general scene category. Many types of features are associated with scene categories, ranging from low-level properties, such as colors and contours, to high-level properties, such as objects and attributes. Because these properties are correlated, it is difficult to understand each property's unique contributions to scene categorization. This work uses a whitening transformation to remove the correlations between features and examines the extent to which each feature contributes to visual event-related potentials over time. We found that low-level visual features contributed first but were not correlated with categorization behavior. High-level features followed 80 ms later, providing key insights into how the brain makes sense of a complex visual world.

In this paper, we propose a text extraction method from camera-captured document style images and propose a text segmentation method based on a color clustering method. The proposed extraction method detects text regions from the images using two low-level image features and verifies the regions through a high-level text stroke feature. The two level features are combined hierarchically. The low-level features are intensity variation and color variance. And, we use text strokes as a high-level feature using multi-resolution wavelet transforms on local image areas. The stroke feature vector is an input to a SVM (support vector machine) for verification, when needed. The proposed text segmentation method uses color clustering to the extracted text regions. We improved K-means clustering method and it selects K and initial seed values automatically. We tested the proposed methods with various document style images captured by three different cameras. We confirmed that the extraction rates are good enough to be used in real-life applications.

Hyperspectral images have high spectral resolution, but this is often at the expense of spatial resolution. Although deep learning-based super-resolution (SR) algorithms have shown comparative performance for spatial resolution enhancement, most of them cannot effectively extract features of different size objects because of single scale convolution. In deep architectures, low level features also tend to disappear during transmission. In this paper, an efficient network (MRBMFF) for enhancing the spatial resolution of hyperspectral image is proposed. Based on the multiscale residual block (MRB), features at different scales can be effectively extracted and fused. Meanwhile, the multilevel feature fusion (MFF) is introduced to concatenate the low and high level features. Effective SR images could be recovered after inputting their low-resolution counterparts to the proposed network. Experimental results show that the proposed network achieves superior reconstruction performance compared with the state-of-the-art approaches.