Image Segmentation-based Unsupervised Multiple Objects Discovery

Motivated by the recent success of supervised and weakly supervised common object discovery, in this paper, we move forward one step further to tackle common object discovery in a fully unsupervised way. Generally, object co-localization aims at simultaneously localizing objects of the same class across a group of images. Traditional object localization/detection usually trains specific object detectors which require bounding box annotations of object instances, or at least image-level labels to indicate the presence/absence of objects in an image. Given a collection of images without any annotations, our proposed fully unsupervised method is to simultaneously discover images that contain common objects and also localize common objects in corresponding images. Without requiring to know the total number of common objects, we formulate this unsupervised object discovery as a sub-graph mining problem from a weighted graph of object proposals, where nodes correspond to object proposals, and edges represent the similarities between neighbouring proposals. The positive images and common objects are jointly discovered by finding sub-graphs of strongly connected nodes, with each sub-graph capturing one object pattern. The optimization problem can be efficiently solved by our proposed maximal-flow-based algorithm. Instead of assuming that each image contains only one common object, our proposed solution can better address wild images where each image may contain multiple common objects or even no common object. Moreover, our proposed method can be easily tailored to the task of image retrieval in which the nodes correspond to the similarity between query and reference images. Extensive experiments on PASCAL VOC 2007 and Object Discovery data sets demonstrate that even without any supervision, our approach can discover/localize common objects of various classes in the presence of scale, view point, appearance variation, and partial occlusions. We also conduct broad experiments on image retrieval benchmarks, Holidays and Oxford5k data sets, to show that our proposed method, which considers both the similarity between query and reference images and also similarities among reference images, can help to improve the retrieval results significantly.

Unsupervised object discovery (UOD) refers to the task of discriminating the whole region of objects from the background within a scene without relying on labeled datasets, which benefits the task of bounding-box-level localization and pixel-level segmentation. This task is promising due to its ability to discover objects in a generic manner. We roughly categorize existing techniques into two main directions, namely the generative solutions based on image resynthesis, and the clustering methods based on self-supervised models. We have observed that the former heavily relies on the quality of image reconstruction, while the latter shows limitations in effectively modeling semantic correlations. To directly target at object discovery, we focus on the latter approach and propose a novel solution by incorporating weakly-supervised contrastive learning (WCL) to enhance semantic information exploration. We design a semantic-guided self-supervised learning model to extract high-level semantic features from images, which is achieved by fine-tuning the feature encoder of a self-supervised model, namely DINO, via WCL. Subsequently, we introduce Principal Component Analysis (PCA) to localize object regions. The principal projection direction, corresponding to the maximal eigenvalue, serves as an indicator of the object region(s). Extensive experiments on benchmark unsupervised object discovery datasets demonstrate the effectiveness of our proposed solution. The source code and experimental results are publicly available via our project page at https://github.com/npucvr/WSCUOD.git.

Unsupervised object discovery and localization aims to detect or segment objects in an image without any supervision. Recent efforts have demonstrated a notable potential to identify salient foreground objects by utilizing self-supervised transformer features. However, their scopes only build upon patch-level features within an image, neglecting region/image-level and cross-image relationships at a broader scale. Moreover, these methods cannot differentiate various semantics from multiple instances. To address these problems, we introduce Hierarchical mErging framework via contrAstive grouPing (HEAP). Specifically, a novel lightweight head with cross-attention mechanism is designed to adaptively group intra-image patches into semantically coherent regions based on correlation among self-supervised features. Further, to ensure the distinguishability among various regions, we introduce a region-level contrastive clustering loss to pull closer similar regions across images. Also, an image-level contrastive loss is present to push foreground and background representations apart, with which foreground objects and background are accordingly discovered. HEAP facilitates efficient hierarchical image decomposition, which contributes to more accurate object discovery while also enabling differentiation among objects of various classes. Extensive experimental results on semantic segmentation retrieval, unsupervised object discovery, and saliency detection tasks demonstrate that HEAP achieves state-of-the-art performance.

Unsupervised object discovery via self-organisation

We propose a novel approach to investigate and implement unsupervised image content understanding and segmentation of color industrial images like medical imaging, forensic imaging, security and surveillance imaging, biotechnical imaging, biometrics, mineral and mining imaging, material science imaging, and many more. In this particular work, our focus will be on medical images only. The aim is to develop a computer aided diagnosis (CAD) system based on a newly developed Multidimensional Spatially Variant Finite Mixture Model (MSVFMM) using Markov Random Fields (MRF) Model. Unsupervised means automatic discovery of classes or clusters in images rather than generating the class or cluster descriptions from training image sets. The aim of this work is to produce precise segmentation of color medical images on the basis of subtle color and texture variation. Finer segmentation of images has tremendous potential in medical imaging where subtle information related to color and texture is required to analyze the image accurately. In this particular work, we have used CIE-Luv and Daubechies wavelet transforms as color and texture descriptors respectively. Using the combined effect of a CIE-Luv color model and Daubechies transforms, we can segment color medical images precisely in a meaningful manner. The evaluation of the results is done through comparison of the segmentation quality with another similar alternative approach and it is found that the proposed approach is capable of producing more faithful segmentation.

Textured surface defects detection can be a complicated task. Maintenance and monitoring of big infrastructures, such as roads, is expensive, time-consuming, and requires many human resources. In many areas, humans are being replaced by computer vision systems that perform faster and more precise. Moreover, some inspection tasks can reach incredibly high levels of complexity. Recently, deep learning approaches showed impressive results in object detection and image segmentation. It can provide a state-of-the-art solution for most computer vision tasks, including pattern inspection and defect detection. In this work, we present a pixel-wise road pavement defects detection method by using U-Net convolutional neural network. We have experimentally evaluated the impact of a different number of layers, filter sizes and the number of features in segmentation performance and processing time. The best-suggested configuration for road pavement cracks segmentation task has received up to 98.92% of accuracy in 0.049 s per image.

Object discovery has advanced significantly with the emergence of deep learning models; however, existing algorithms often fail to deliver highly accurate and feature-focused detection, particularly in challenging visual environments. This study addresses the limitation by proposing a novel framework that integrates the high-level detection capabilities of YOLOv8 with the precision-focused characteristics of SURF-based feature extraction (referred to here as "Surzet"). The proposed method leverages YOLOv8 to perform comprehensive object detection while utilizing Surzet to enhance the identification of fine-grained features and local properties, ensuring robustness to scale and rotation. Experimental evaluation on complex image datasets revealed that this hybrid model significantly outperforms YOLOv8 alone, showing higher detection accuracy and a noticeable reduction in false positives. The initial results demonstrate that integrating YOLOv8 with Surzet creates a more reliable and precise object discovery framework. This approach holds great promise for high-level detection and detailed feature recognition applications.

PO-1691 A deeply supervised convolutional neural network ensemble for multilabel segmentation of pelvic OARs

A useful capability for a mobile robot is the ability to recognize the objects in its environment that move and change (as distinct from background objects, which are largely stationary). This ability can improve the accuracy and reliability of localization and mapping, enhance the ability of the robot to interact with its environment, and facilitate applications such as inventory management and theft detection. Rather than viewing this task as a difficult application of object recognition methods from computer vision, this work is in line with a recent trend in the community towards unsupervised object discovery and tracking that exploits the fundamentally temporal nature of the data acquired by a robot. Unlike earlier approaches, which relied heavily upon computationally intensive techniques from mapping and computer vision, our approach combines visual features and RGB-D data in a simple and effective way to segment objects from robot sensory data. We then use a Dirichlet process to cluster and recognize objects. The performance of our approach is demonstrated in several test domains.

This paper studies the problem of object discovery - separating objects from the background without manual labels. Existing approaches utilize appearance cues, such as color, texture, and location, to group pixels into object-like regions. However, by relying on appearance alone, these methods fail to separate objects from the background in cluttered scenes. This is a fundamental limitation since the definition of an object is inherently ambiguous and context-dependent. To resolve this ambiguity, we choose to focus on dynamic objects - entities that can move independently in the world. We then scale the recent auto-encoder based frameworks for unsuper-vised object discovery from toy synthetic images to complex real-world scenes. To this end, we simplify their architecture, and augment the resulting model with a weak learning signal from general motion segmentation algorithms. Our experiments demonstrate that, despite only capturing a small subset of the objects that move, this signal is enough to generalize to segment both moving and static instances of dynamic objects. We show that our model scales to a newly collected, photo- realistic synthetic dataset with street driving scenarios. Additionally, we leverage ground truth segmentation and flow annotations in this dataset for thorough ablation and evaluation. Finally, our experiments on the real-world KITTI benchmark demonstrate that the proposed approach outperforms both heuristic- and learning-based methods by capitalizing on motion cues.

Diabetic retinopathy, a retinal disorder resulting from diabetes mellitus, is a prominent cause of visual degradation and loss among the global population. Therefore, the identification and classification of diabetic retinopathy are of utmost importance in the clinical diagnosis and therapy. Currently, these duties are extensively carried out by manual examination utilizing the human visual system. Nevertheless, manual examination is sometimes arduous, time-consuming, and prone to errors. Deep learning-based methods have recently demonstrated encouraging results in several areas, such as image categorization and natural language mining. The majority of deep learning techniques developed for medical image analysis rely on convolutional modules to extract the inherent structure of images within a certain local receptive field. Furthermore, transformer-based models have been utilized to tackle medical image processing problems by capitalizing on global connections among distant pixels in the images. Considering these analyses, this work presents a comprehensive deep learning model that combines convolutional neural network and vision mamba models. This model is designed to accurately identify and classify diabetic retinopathy lesions displayed in fundus images. Furthermore, the vision mamba component incorporates the bidirectional state space method and positional embedding to enable the location sensitivity of visual data samples and meet the conditions for global relationship context. An evaluation of the suggested method was carried out by comparison experiments between state-of-the-art algorithms and the proposed methodology. Empirical findings demonstrate that the suggested methodology surpasses the most advanced algorithms on the datasets that are accessible openly. Hence, the suggested approach may be regarded as a helpful tool for therapeutic processes.

IntroductionOptical Coherence Tomography Angiography (OCTA) is a new non-invasive imaging modality that gains increasing popularity for the observation of the microvasculatures in the retina and the conjunctiva, assisting clinical diagnosis and treatment planning. However, poor imaging quality, such as stripe artifacts and low contrast, is common in the acquired OCTA and in particular Anterior Segment OCTA (AS-OCTA) due to eye microtremor and poor illumination conditions. These issues lead to incomplete vasculature maps that in turn makes it hard to make accurate interpretation and subsequent diagnosis.MethodsIn this work, we propose a two-stage framework that comprises a de-striping stage and a re-enhancing stage, with aims to remove stripe noise and to enhance blood vessel structure from the background. We introduce a new de-striping objective function in a Stripe Removal Net (SR-Net) to suppress the stripe noise in the original image. The vasculatures in acquired AS-OCTA images usually exhibit poor contrast, so we use a Perceptual Structure Generative Adversarial Network (PS-GAN) to enhance the de-striped AS-OCTA image in the re-enhancing stage, which combined cyclic perceptual loss with structure loss to achieve further image quality improvement.Results and discussionTo evaluate the effectiveness of the proposed method, we apply the proposed framework to two synthetic OCTA datasets and a real AS-OCTA dataset. Our results show that the proposed framework yields a promising enhancement performance, which enables both conventional and deep learning-based vessel segmentation methods to produce improved results after enhancement of both retina and AS-OCTA modalities.

Abstract: Satellite image analysis plays a crucial role in fields such as environmental monitoring, urban planning, agriculture, and disaster management. This project presents a machine learning-based methodology for the automated segmentation and classification of objects in satellite images. The proposed system employs a two-step algorithm: (1) K-Means clustering for unsupervised segmentation of satellite images into meaningful clusters, and (2) Support Vector Machine (SVM) for supervised classification of the segmented regions into predefined categories such as land, water, and vegetation. A Flask-based web application is developed to provide a userfriendly interface for uploading satellite images and visualizing classified results. The backend is supported by a structured SQL database to manage image metadata and classification outputs efficiently. The system is designed to handle large datasets, ensuring robustness and scalability. Experimental results on benchmark datasets demonstrate high accuracy in object detection and classification, highlighting the system's potential to support real-world decision-making in environmental and infrastructural domains.

We introduce a segmentation-based detection and top-down figure-ground delineation algorithm. Unlike common methods which use appearance for detection, our method relies primarily on the shape of objects as is reflected by their bottom-up segmentation. Our algorithm receives as input an image, along with its bottom-up hierarchical segmentation. The shape of each segment is then described both by its significant boundary sections and by regional, dense orientation information derived from the segment's shape using the Poisson equation. Our method then examines multiple, overlapping segmentation hypotheses, using their shape and color, in an attempt to find a "coherent whole," i.e., a collection of segments that consistently vote for an object at a single location in the image. Once an object is detected, we propose a novel pixel-level top-down figure-ground segmentation by "competitive coverage" process to accurately delineate the boundaries of the object. In this process, given a particular detection hypothesis, we let the voting segments compete for interpreting (covering) each of the semantic parts of an object. Incorporating competition in the process allows us to resolve ambiguities that arise when two different regions are matched to the same object part and to discard nearby false regions that participated in the voting process. We provide quantitative and qualitative experimental results on challenging datasets. These experiments demonstrate that our method can accurately detect and segment objects with complex shapes, obtaining results comparable to those of existing state of the art methods. Moreover, our method allows us to simultaneously detect multiple instances of class objects in images and to cope with challenging types of occlusions such as occlusions by a bar of varying size or by another object of the same class, that are difficult to handle with other existing class-specific top-down segmentation methods.

At present, the region growing algorithm has been used as a segmentation technique of digital images. Most region growing algorithms are using fixed or determinate criterions to distinguish disease spots from leaf image with gray level differences between leaf and disease spot. But in practice, the objects in the disease leaf image have fuzziness and uncertainty, and edges of the objects are unclear. What's more, the color of leaf and disease spots is uneven, and the gray level is overlapping, so it is difficult to use fixed threshold or determinate criteria to determine the uncertain objects in leaf disease spot images accurately. In order to improve the crop leaf spot disease image segmentation accuracy, an adaptive segmentation algorithm by integrating local threshold and seeded region growing (LTSRG) is proposed. The algorithm was implemented on VC6.0. The segmentation algorithm uses the pixels of which the R-channel gray level is more than the G-channel gray level as initial seed points (pixels), and then local threshold C i is calculated for each connected seed region by Otsu. New seed pixels are included and the threshold C is re-calculated until no new seed pixel can be included. The results of LTSRG are compared with the results of threshold-based Otsu and clustering -based EM. The experiments show: The adapted segmentation method is satisfactory and highly efficient to separate disease spots from normal part of corn leaves. LTSRG algorithm is easy to realize, and can improve the precision of crop disease spot segmentation. Its image segmentation results have good region consistency and high efficiency. It is an adapting algorithm for image segmentation.