Bayesian supervised segmentation of objects in natural images using low-level information

Salient object detection is evaluated using binary ground truth (GT) with the labels being salient object class and background. In this study, the authors corroborate based on three subjective experiments on a novel image dataset that objects in natural images are inherently perceived to have varying levels of importance. The authors' dataset, named SalMoN (saliency in multi-object natural images), has 588 images containing multiple objects. The subjective experiments performed record spontaneous attention and perception through eye fixation duration, point clicking and rectangle drawing. As object saliency in a multi-object image is inherently multi-level, they propose that salient object detection must be evaluated for the capability to detect all multi-level salient objects apart from the salient object class detection capability. For this purpose, they generate multi-level maps as GT corresponding to all the dataset images using the results of the subjective experiments, with the labels being multi-level salient objects and background. They then propose the use of mean absolute error, Kendall's rank correlation and average area under precision–recall curve to evaluate existing salient object detection methods on their multi-level saliency GT dataset. Approaches that represent saliency detection on images as local-global hierarchical processing of a graph perform well in their dataset.

In this paper we propose a deep learning approach for segmenting sub-cortical structures of the human brain in Magnetic Resonance (MR) image data. We draw inspiration from a state-of-the-art Fully-Convolutional Neural Network (F-CNN) architecture for semantic segmentation of objects in natural images, and adapt it to our task. Unlike previous CNN-based methods that operate on image patches, our model is applied on a full blown 2D image, without any alignment or registration steps at testing time. We further improve segmentation results by interpreting the CNN output as potentials of a Markov Random Field (MRF), whose topology corresponds to a volumetric grid. Alpha-expansion is used to perform approximate inference imposing spatial volumetric homogeneity to the CNN priors. We compare the performance of the proposed pipeline with a similar system using Random Forest-based priors, as well as state-of-art segmentation algorithms, and show promising results on two different brain MRI datasets.

The objects in natural images are often texturally inhomogeneous and prone to be falsely segmented into different parts by conventional methods. To overcome the difficulties caused by texture inhomogeneity, a new active contour model is proposed to extract inhomogeneous insulators from aerial images. First, a semilocal operator is employed to extract the texture features of insulators under the Beltrami framework. The layer of semilocal texture feature is single, and thus, it can avoid the high dimensionality of feature space. Then, a new convex energy functional is defined by taking the Xie's nonconvex model into a global minimization active contour framework during the process of segmentation. The proposed energy functional consists of not only the semilocal texture features of insulators but also their spatial relationship, which improves its ability to deal with textural inhomogeneity. Moreover, it can also avoid the existence of local minima in the minimization of the Xie's nonconvex model, thereby being independent of initial contour. In the process of contour evolution and numerical minimization, a fast dual formulation is employed to overcome the drawbacks of the usual level set and gradient descent method and to make the evolution of the contour more efficient. The experimental results on aerial insulator images confirm the ability of the proposed algorithm to effectively segment inhomogeneous textures with an overall average rmse of 1.87 pixels, a precision of 85.59%, and a recall of 86.47%. In addition, the proposed algorithm is extended to animal images, and satisfactory segmentation results can be obtained as well.

A neural network approach to computation of optical flow: [formula omitted]. Signal and Image Processing Institute, Department of EE-Systems, University of Southern California, CA 90089-0272 USA

The visual system needs to identify perceptually relevant borders to segment complex natural scenes. The primary visual cortex (V1) is thought to extract local borders, and higher visual areas are thought to identify the perceptually relevant borders between objects and the background. To test this conjecture, we used natural images that had been annotated by human observers who marked the perceptually relevant borders. We assessed the effect of perceptual relevance on V1 responses using human neuroimaging, macaque electrophysiology, and computational modeling. We report that perceptually relevant borders elicit stronger responses in the early visual cortex than irrelevant ones, even if simple features, such as contrast and the energy of oriented filters, are matched. Moreover, V1 neurons discriminate perceptually relevant borders surprisingly fast, during the early feedforward-driven activity at a latency of ~50 ms, indicating that they are tuned to the features that characterize them. We also revealed a delayed, contextual effect that enhances the V1 responses that are elicited by perceptually relevant borders at a longer latency. Our results reveal multiple mechanisms that allow V1 neurons to infer the layout of objects in natural images.

Vision-based industrial inspection (VII) aims to locate defects quickly and accurately. Supervised learning under a close-set setting and industrial anomaly detection, as two common paradigms in VII, face different problems in practical applications. The former is that various and sufficient defects are difficult to obtain, while the latter is that specific defects cannot be located. To solve these problems, in this article, we focus on the few-shot semantic segmentation (FSS) method, which can locate unseen defects conditioned on a few annotations without retraining. Compared to common objects in natural images, the defects in VII are small. This brings two problems to current FSS methods: first, distortion of target semantics and second, many false positives for backgrounds. To alleviate these problems, we propose a small object few-shot segmentation (SOFS) model. The key idea for alleviating, first, is to avoid the resizing of the original image and correctly indicate the intensity of target semantics. SOFS achieves this idea via the nonresizing procedure and the prototype intensity downsampling of support annotations. To alleviate, second, we design an abnormal prior map in SOFS to guide the model in reducing false positives and propose a mixed normal dice loss to prevent the model from predicting false positives preferentially. SOFS can achieve FSS and few-shot anomaly detection determined by support masks. Diverse experiments substantiate the superior performance of SOFS.

Color is a prime example of categorical perception, yet it is unclear why and how color categories emerge. On the one hand, prelinguistic infants and several animals treat color categorically. On the other hand, recent modeling endeavors have successfully utilized communicative concepts as the driving force for color categories. Rather than modeling categories directly, we investigate the potential emergence of color categories as a result of acquiring visual skills. Specifically, we asked whether color is represented categorically in a convolutional neural network (CNN) trained to recognize objects in natural images. We systematically trained new output layers to the CNN for a color classification task and, probing novel colors, found borders that are largely invariant to the training colors. The border locations were confirmed using an evolutionary algorithm that relies on the principle of categorical perception. A psychophysical experiment on human observers, analogous to our primary CNN experiment, shows that the borders agree to a large degree with human category boundaries. These results provide evidence that the development of basic visual skills can contribute to the emergence of a categorical representation of color.

The Segment Anything Model (SAM) is widely used for segmenting a diverse range of objects in natural images from simple user prompts like points or bounding boxes. However, SAM's performance decreases substantially when applied to non-natural domains like microscopic imaging. Furthermore, due to SAM's interactive design, it requires a precise prompt for each image and object, which is un-feasible in many automated biomedical applications. Previous solutions adapt SAM by training millions of parameters via fine-tuning large parts of the model or of adapter layers. In contrast, we show that as little as 2,048 additional parameters are sufficient for turning SAM into a usecase specialist for a certain downstream task. Our novel PTSAM (prompt-tuned SAM) method uses prompt-tuning, a parameter-efficient fine-tuning technique, to adapt SAM for a specific task. We validate the performance of our approach on multiple microscopic and one medical dataset. Our results show that prompt-tuning only SAM's mask decoder already leads to a performance on-par with state-of-the-art techniques while requiring roughly 2,000× less trainable parameters. For addressing domain gaps, we find that additionally prompt-tuning SAM's image encoder is beneficial, further improving segmentation accuracy by up to 18% over state-of-the-art results. Since PTSAM can be reliably trained with as little as 16 annotated images, we find it particularly helpful for applications with limited training data and domain shifts.

The efficient separation of coal and gangue in the mining process is of great significance for improving coal mining efficiency and reducing environmental pollution. Automatic detection of coal and gangue is the key and foundation for the separation of coal and gangue. In this paper, we proposed a hierarchical framework for coal and gangue detection based on deep learning models. In this framework, the Gaussian pyramid principle is first used to construct multi-level training data, leading to the sets of coal and gangue image features with multiple scales. Then, the coal and gangue regional proposal networks (CG-RPN) are designed to determine the candidate regions of the target objects in the image. Next, convolution neural networks (CNNs) are constructed to recognize coal and gangue objects on the basis of extracted candidate regions. We performed our method on three different datasets. Experimental results showed that the proposed method improves the detection accuracy of coal and gangue objects by 0.8% compared with the previous methods, reaching up to 98.33%. In addition, our proposed method enables the detection of multiple coal and gangue objects in an individual image and solves the problem of queuing requirements in traditional methods.

We present an approach for detecting and estimating the 3D poses of objects\nin images that requires only an untextured CAD model and no training phase for\nnew objects. Our approach combines Deep Learning and 3D geometry: It relies on\nan embedding of local 3D geometry to match the CAD models to the input images.\nFor points at the surface of objects, this embedding can be computed directly\nfrom the CAD model; for image locations, we learn to predict it from the image\nitself. This establishes correspondences between 3D points on the CAD model and\n2D locations of the input images. However, many of these correspondences are\nambiguous as many points may have similar local geometries. We show that we can\nuse Mask-RCNN in a class-agnostic way to detect the new objects without\nretraining and thus drastically limit the number of possible correspondences.\nWe can then robustly estimate a 3D pose from these discriminative\ncorrespondences using a RANSAC- like algorithm. We demonstrate the performance\nof this approach on the T-LESS dataset, by using a small number of objects to\nlearn the embedding and testing it on the other objects. Our experiments show\nthat our method is on par or better than previous methods.\n

Segmentation and classification of objects in images is one of the most important and yet one of the most complex problems in computer vision. Many different things contribute to the complexity of this problem, but amongst those that are most significant are variations in object appearance due to various perspectives, changes in scene illumination and typically a large number of partially occluded objects in a scene. Humans have the ability to successfully deal with these complications and to recognize objects in scenes without seemingly much effort. However, to aid their recognition process, humans use different sources of information such as previous experiences, context of objects placed in a scene and rules learned in the past that suggest how the physical world should appear and behave. Computer vision is trying to replicate the same strategies used by human vision and apply them to computer systems with the aim of improving typical tasks of detection, localization and classification of image objects. In this work we propose a new model for natural image object classification using contextual information at the level of image segments. Context modeling is largely independent of appearance-based classification and proposed model enables simple upgrade of existing systems with information from global and/or local context. Context modeling is based on non-parametric use of appearance-based classification results which is a novel approach compared to previous systems that model context on a limited number of rules expressed with a fixed set of parameters. Model implementation resulted in a system that, in our simulations, showed stable improvement of the appearance-based object classification. DOI: http://dx.doi.org/10.5755/j01.itc.46.1.13610

Mild traumatic brain injury (mTBI) appears as low contrast lesions in magnetic resonance (MR) imaging. Standard automated detection approaches cannot detect the subtle changes caused by the lesions. The use of context has become integral for the detection of low contrast objects in images. Context is any information that can be used for object detection but is not directly due to the physical appearance of an object in an image. In this paper, new low-level static and dynamic context features are proposed and integrated into a discriminative voxel-level classifier to improve the detection of mTBI lesions. Visual features, including multiple texture measures, are used to give an initial estimate of a lesion. From the initial estimate novel proximity and directional distance, contextual features are calculated and used as features for another classifier. This feature takes advantage of spatial information given by the initial lesion estimate using only the visual features. Dynamic context is captured by the proposed posterior marginal edge distance context feature, which measures the distance from a hard estimate of the lesion at a previous time point. The approach is validated on a temporal mTBI rat model dataset and shown to have improved dice score and convergence compared to other state-of-the-art approaches. Analysis of feature importance and versatility of the approach on other datasets are also provided.

Most modern algorithms use convolutional neural networks to classify image data of different kinds. While this approach is a good method to differentiate between natural images of objects, big datasets are needed for the training process. Another drawback is the demand for high computational power. We introduce a new approach which involves classic feature vectors with structural information based on higher order Riesz transform. Following this way we create a framework specialized for texture data like images of rock cross-sections. The key advantages are faster computations and more versatile choices of the underlying machine learning tools while maintaining a comparable accuracy in comparison with state-of-the-art algorithms.

Automatic rib segmentation and labeling in computed tomography scans using a general framework for detection, recognition and segmentation of objects in volumetric data

The problem of detecting underwater targets from Electro-optical (EO) images is considered in this paper. A block-based log-likelihood ratio test has been developed for detection and segmentation of underwater mine-like objects in the EO images captured with a CCD-based image sensor. The main focus of this research is to develop a robust detection algorithm that can be used to detect low contrast and partial underwater objects from the EO imagery with low false alarm rate. The detection method involves identifying frames of interest (FOI) containing the potential targets. Once the FOI have been identified, regions of interest (ROI) within the FOI are segmented from the background. Performance of the detection method is tested in terms of probability of detection, false alarm rate, and receiver operating characteristic (ROC) curves for FOI in the selected data runs. The algorithm shows promising results in target detection and generation of good silhouettes for subsequent classification.