A Comprehensive Analysis of Weakly-Supervised Semantic Segmentation in Different Image Domains

Text recognition in natural scene images is a challenging problem in computer vision. Different than the optical character recognition (OCR), text recognition in natural scene images is more complex due to variations in text size, colors, fonts, orientations, complex backgrounds, occlusion, illuminations and uneven lighting conditions. In this paper, we propose a segmentation-free method based on a deep convolutional recurrent neural network to solve the problem of cursive text recognition, particularly focusing on Urdu text in natural scenes. Compared to the non-cursive scripts, Urdu text recognition is more complex due to variations in the writing styles, several shapes of the same character, connected text, ligature overlapping, stretched, diagonal and condensed text. The proposed model gets a whole word image as an input without pre-segmenting into individual characters, and then transforms into the sequence of the relevant features. Our model is based on three components: a deep convolutional neural network (CNN) with shortcut connections to extract and encode the features, a recurrent neural network (RNN) to decode the convolutional features, and a connectionist temporal classification (CTC) to map the predicted sequences into the target labels. To increase the text recognition accuracy further, we explore deeper CNN architectures like VGG-16, VGG-19, ResNet-18 and ResNet-34 to extract more appropriate Urdu text features, and compare the recognition results. To conduct the experiments, a new large-scale benchmark dataset of cropped Urdu word images in natural scenes is developed. The experimental results show that the proposed deep CRNN network with shortcut connections outperform than other network architectures. The dataset is publicly available and can be downloaded from <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://data.mendeley.com/datasets/k5fz57zd9z/1</uri> .

Pixel-level annotation of lung cavities (LCs) in computed tomography (CT) images is challenging due to their morphological diversity and complexity. Weakly supervised semantic segmentation (WSSS) methods, which utilize sparse annotations (e.g., image-level labels), offer a promising solution. However, existing WSSS approaches often generate coarse pseudo-labels and lack sufficient spatial supervision, resulting in under- or over-segmentation of irregular lesions. To address these limitations, we introduce several key innovations. First, we propose a novel Graph-based Affinity Network (GA-Net) that, unlike conventional methods relying on low-level pixel features, models long-range contextual relationships and structural dependencies using a superpixel graph and learned edge inference kernel, enabling structure-aware pseudo-label refinement for complex lesion morphology. Second, we introduce region-wise affinity propagation, which refines segmentation by propagating activations within semantically coherent 3D regions, offering more precise control over under-/over-segmentation compared to global affinity methods. Additionally, we incorporate Exponential Moving Average (EMA) ensembling for training stability and a scribble-based segmentation module that utilizes pseudo-label contours to provide direct boundary supervision. Extensive experiments on three benchmark datasets demonstrate that our method outperforms existing state-of-the-art medical WSSS techniques, achieving precise and reliable segmentation of complex LCs in CT scans.

The bag of visual words (BOW) model is an efficient image representation technique for image categorization and annotation tasks. Building good visual vocabularies, from automatically extracted image feature vectors, produces discriminative visual words, which can improve the accuracy of image categorization tasks. Most approaches that use the BOW model in categorizing images ignore useful information that can be obtained from image classes to build visual vocabularies. Moreover, most BOW models use intensity features extracted from local regions and disregard colour information, which is an important characteristic of any natural scene image. In this paper, we show that integrating visual vocabularies generated from each image category improves the BOW image representation and improves accuracy in natural scene image classification. We use a keypoint density-based weighting method to combine the BOW representation with image colour information on a spatial pyramid layout. In addition, we show that visual vocabularies generated from training images of one scene image dataset can plausibly represent another scene image dataset on the same domain. This helps in reducing time and effort needed to build new visual vocabularies. The proposed approach is evaluated over three well-known scene classification datasets with 6, 8 and 15 scene categories, respectively, using 10-fold cross-validation. The experimental results, using support vector machines with histogram intersection kernel, show that the proposed approach outperforms baseline methods such as Gist features, rgbSIFT features and different configurations of the BOW model.

Weakly supervised semantic segmentation (WSSS) faces challenges in generating reliable pseudo masks from image-level labels due to limited spatial supervision. While prototype-based contrastive learning (PCL) improves class activation maps (CAMs), existing methods struggle with background-foreground ambiguity and incomplete object activation. We propose EPCL, which addresses these issues through two key innovations: (1) a class-background(CB)contrast to better discriminate hard background semantics, and (2) an inter-class (IC) contrast sampling strategy that enhances object region activation. EPCL is architecture-agnostic and integrates into various WSSS baselines. Experiments on two segmentation benchmarks show EPCL outperforms existing methods across different WSSS settings.

Now days reading words from an unconstrained and noisy image is not easy. Text localization and recognition in an image is a research area which takes efforts to develop a computer system with an ability to automatically read the text from images. The Optical Character Recognition (OCR) tool gives good results obtained to read the text from an image. The objective of this study is to propose a new method for text localization and recognition in natural scene images with complex background. In this paper, a hybrid methodology is suggested which extracts text from natural scene image with chaotic backgrounds. The proposed approach involves four stages. First, superimposed text regions in an image are extracted based on character descriptors features like Area, Bounding box, Perimeter, Euler number, Horizontal crossings. In the second step, superimposed text regions are tested for text content or nontext using character descriptors and SVM classifier. In the third step, detection of multiple lines in localized text regions is done and line segmentation is performed using horizontal profiles. In the final step, using vertical profiles each character of the segmented line is extracted. The workout has been done using images drawn from ICDAR 2013 and SVT 2010 datasets. The results demonstrate the effectiveness of the proposed method, which can be used as an efficient method for text localization and recognition in natural scene images.

Weakly supervised semantic segmentation (WSSS) has gained significant popularity as it relies only on weak labels such as image level annotations rather than the pixel level annotations required by supervised semantic segmentation (SSS) methods. Despite drastically reduced annotation costs, typical feature representations learned from WSSS are only representative of some salient parts of objects and less reliable compared to SSS due to the weak guidance during training. In this paper, we propose a novel Multi-Strategy Contrastive Learning (MuSCLe) framework to obtain enhanced feature representations and improve WSSS performance by exploiting similarity and dissimilarity of contrastive sample pairs at image, region, pixel and object boundary levels. Extensive experiments demonstrate the effectiveness of our method and show that MuSCLe outperforms current state-of-the-art methods on the widely used PASCAL VOC 2012 dataset.

Existing weakly supervised semantic segmentation (WSSS) methods based on image‐level labels always rely on class activation maps (CAMs), which measure the relationships between features and classifiers. However, CAMs only focus on the most discriminative regions of images, resulting in their poor coverage performance. We attribute this to the deficiency in the recognition ability of a single classifier and the negative impacts caused by magnitudes during the CAMs normalisation process. To address the aforementioned issues, we propose to construct selective multiple classifiers (SMC). During the training process, we extract multiple prototypes for each class and store them in the corresponding memory bank. These prototypes are divided into foreground and background prototypes, with the former used to identify foreground objects and the latter aimed at preventing the false activation of background pixels. As for the inference stage, multiple prototypes are adaptively selected from the memory bank for each image as SMC. Subsequently, CAMs are generated by measuring the angle between SMC and features. We enhance the recognition ability of classifiers by adaptively constructing multiple classifiers for each image, while only relying on angle measurement to generate CAMs can alleviate the suppression phenomenon caused by magnitudes. Furthermore, SMC can be integrated into other WSSS approaches to help generate better CAMs. Extensive experiments conducted on standard WSSS benchmarks such as PASCAL VOC 2012 and MS COCO 2014 demonstrate the superiority of our proposed method.

Weakly supervised semantic segmentation (WSSS) models relying on class activation maps (CAMs) have achieved desirable performance comparing to the non-CAMs-based counterparts. However, to guarantee WSSS task feasible, we need to generate pseudo labels by expanding the seeds from CAMs which is complex and time-consuming, thus hindering the design of efficient end-to-end (single-stage) WSSS approaches. To tackle the above dilemma, we resort to the off-the-shelf and readily accessible saliency maps for directly obtaining pseudo labels given the image-level class labels. Nevertheless, the salient regions may contain noisy labels and cannot seamlessly fit the target objects, and saliency maps can only be approximated as pseudo labels for simple images containing single-class objects. As such, the achieved segmentation model with these simple images cannot generalize well to the complex images containing multi-class objects. To this end, we propose an end-to-end multi-granularity denoising and bidirectional alignment (MDBA) model, to alleviate the noisy label and multi-class generalization issues. Specifically, we propose the online noise filtering and progressive noise detection modules to tackle image-level and pixel-level noise, respectively. Moreover, a bidirectional alignment mechanism is proposed to reduce the data distribution gap at both input and output space with simple-to-complex image synthesis and complex-to-simple adversarial learning. MDBA can reach the mIoU of 69.5% and 70.2% on validation and test sets for the PASCAL VOC 2012 dataset. The source codes and models have been made available at https://github.com/NUST-Machine-Intelligence-Laboratory/MDBA.

Textual matter present in a natural scene image provides indispensable information about it. The semantics and information present in the natural scene images can be perceived by extracting the text regions in them. Detection and localization of text from natural scene images is a challenging task for analysis of images due to various font size, font type, and illumination. In this paper, we propose a hybrid approach for text detection and localization based on text confidence score using three attributes namely stroke width dissimilarity, color dissimilarity and occupy rate convex area to discern text and non-text constituents. The aim of this paper is to achieve fast detection and localization of text regions in low resolution and blurred images. To accomplish this, the possible candidate regions are extracted using edge smoothing by fast guided filter followed by MSER. The text confidence score on these constituents is calculated using the Bayesian framework with the help of above mentioned three attributes. Experimental results on benchmark ICDAR 2013 testing dataset shows the efficacy of our method in the form of precision, recall, and f-measure.

In this paper, we proposed feature extraction method in natural scene images. And also, we develop character recognition system using feature extraction method in natural scene images. A graph matching method use structural feature of character. It is recognition method that it considers relation of position and structural relation. It is robust method to change of font or rotation. In order to proof it, we tested two cases that training font and test font are same case or difference case.

Buffer-text: Detecting arbitrary shaped text in natural scene image

Text detection in natural scene images is a challenging problem in computer vision. To robust detect various texts in complex scenes, a hierarchical recursive text detection method is proposed in this paper. Usually, texts in natural scenes are not alone and arranged into lines for easy reading. To find all possible text lines in an image, candidate text lines are obtained using text edge box and conventional neural network at first. Then, to accurately find out the true text lines in the image, these candidate text lines are analyzed in a hierarchical recursive architecture. For each of them, connected components segmentation and hierarchical random field based analysis are recursively employed until the detected text line no more changes. Now the detected text lines are output as the text detection result. Experiments on ICDAR 2003 dataset, ICDAR 2013 dataset and Street View Dataset show that the hierarchical recursive architecture can improve text detection performance and the proposed method achieves the state-of-art in scene text detection.

Text detection in natural scene images is a challenging problem due to many variations and uncontrollable factors comparing to text detection on scanned document. Unlike the existing Thai text detection methods which focus on using connected component analysis combining with other rule-based techniques to localize text, our proposed method is based on a well-known automatic feature extractor neural networks called Convolutional Neural Networks (CNN). The CNN is first trained with both English and Thai text datasets. A multi-scaled text confidence maps are constructed in order to cope with the text size variations. Some post-processing and Thai text analysis are also employed to acquire text locations in the image. Base on our experimental results, the proposed method can detect English and Thai text from natural scene images with a promising accuracy comparing to the state-of-the-art method.

Extracting text character from natural scene images is a complex task due to differences in text style, font, size, orientation, alignment and background variation. Texts in images conveys important information, which helps to make images more understandable. A new hybrid method for text detection from natural scene images, proposes an enhanced model based on fractional poisson to increases the quality of the images obtained by Laplacian operation. Here image enhancement operation refers the actual values of pixels to obtain better contrast between targets and their backgrounds. After the enhancement, images looks brighter than input and Laplacian images. The Enhanced image does not contain any noise as well as in Laplacian image. It occurs text detection and recognition methods to achieve better accuracies. The Maximally Stable Extremal Region algorithm has been used to detect almost all characters or text in the image, even if in low quality. It is a connected component of some gray level sets of the image. A new method adopts for word segmentation that uses the bounding box regions to enclose the letters or characters of the words, then the resulting letter spaces are continuously filled to combine the character bounding boxes and to get the word bounding boxes.

Crosstalk remains an important determinant of stereoscopic 3D (S3D) image quality. Defined as the leakage of one eye's image into the image of the other eye it affects all commercially available stereoscopic viewing systems. Previously we have shown that crosstalk affects perceived depth magnitude in S3D displays. We found that perceived depth between two lines separated in depth decreased as crosstalk increased. The experiments described here extend our previous work to complex images of natural scenes. We controlled crosstalk levels by simulating them in images presented on a zero-crosstalk mirror stereoscope display. The observers were asked to estimate the amount of stereoscopic depth between pairs of objects in stereo-photographs of cluttered rooms. Data show that as crosstalk increased perceived depth decreased; an effect found at all disparities. Similarly to our previous experiments a significant decrease in perceived depth was observed with as little as 2-4% crosstalk. Taken together these results demonstrate that our previous findings generalize to natural scenes and show that crosstalk reduces perceived depth magnitude even in natural scenes with pictorial depth cues.