Intent-aware image cloning

To monitor objects of interest, such as wildlife and people, image-capturing devices are used to collect a large number of images with and without objects of interest. As we are recording valuable information about the behavior and activity of objects, the quality of images containing objects of interest should be better than that of images without objects of interest, even if the former exhibits more severe distortion than the latter. However, according to current methods, quality assessments produce the opposite results. In this study, we propose an end-to-end model, named DETR-IQA (detection transformer image quality assessment), which extends the capability to perform object detection and blind image quality assessment (IQA) simultaneously by adding IQA heads comprising simple multi-layer perceptrons at the top of the DETRs (detection transformers) decoder. Using IQA heads, DETR-IQA carried out blind IQAs based on the weighted fusion of the distortion degree of the region of objects of interest and the other regions of the image; the predicted quality score of images containing objects of interest was generally greater than that of images without objects of interest. Currently, the subjective quality score of all public datasets is in accordance with the distortion of images and does not consider objects of interest. We manually extracted the images in which the five predefined classes of objects were the main contents of the largest authentic distortion dataset, KonIQ-10k, which was used as the experimental dataset. The experimental results show that with slight degradation in object detection performance and simple IQA heads, the values of PLCC and SRCC were 0.785 and 0.727, respectively, and exceeded those of some deep learning-based IQA models that are specially designed for only performing IQA. With the negligible increase in the computation and complexity of object detection and without a decrease in inference speeds, DETR-IQA can perform object detection and IQA via multi-tasking and substantially reduce the workload.

In computer vision tasks such as, for example, object recognition, semantically accurate segmentation of a particular object of interest (OOI) is a critical step. Due to the OOI consisting of visually different fragments, traditional segmentation algorithms that are based on the identification of homogeneous regions usually do not perform well. In order to narrow this gap between low-level visual features and high-level semantics, some recent methods employ machine learning to generate more accurate models of the OOI. The main contribution of this paper is the inclusion of spatial relationships among the OOI fragments into the model. For this purpose, we employ Bayesian networks as a probabilistic approach for learning the spatial relationships which, in turn, becomes evidence that is used for the process of segmenting future instances of the OOI. The algorithm presented in this paper also uses multiple instance learning to obtain prototypical descriptions of each fragment of the OOI based on low-level visual features. The experimental results on both artificial and real image datasets indicate that the addition of spatial relationships improves segmentation performance.

Developing a spatial searching tool to enhance the search capabilities of large spatial repositories for Geographical Information System (GIS) update has attracted more and more attention. Typically, objects to be detected are represented by many local features or local parts. Testing images are processed by extracting local features which are then matched with the object’s model image. Most existing work that uses local features assumes that each of the local features is independent to each other. However, in many cases, this is not true. In this paper, a method of applying the local cooccurring patterns to disclose the cooccurring relationships between local features for object detection is presented. Features including colour features and edge-based shape features of the interested object are collected. To reveal the cooccurring patterns among multiple local features, a colour cooccurrence histogram is constructed and used to search objects of interest from target images. The method is demonstrated in detecting swimming pools from aerial images. Our experimental results show the feasibility of using this method for effectively reducing the labour work in finding man-made objects of interest from aerial images.

In this work, we present radiometric images for several concealed object experiments that were measured at TUBITAK — Marmara Research Center (MRC) facilities. The radiometric imaging system that collects the cosmic background radiation from the target is utilized on a 2D data collection grid to have the millimeter-wave image of the target. A detection algorithm is developed to point out the region of the hidden object beneath the cloth. The raw and processed passive millimeter wave images of various targets with concealed objects are shown after applying our detection algorithm. The resultant images have demonstrated that the passive imaging system together with our detection routine produces successful images of the targets.

Automatic colorization is generally classified into two groups: propagation-based methods and reference-based methods. In reference-based automatic colorization methods, color image(s) are used as reference(s) to reconstruct original color of a gray target image. The most important task here is to find the best matching pairs for all pixels between reference and target images in order to transfer color information from reference to target pixels. A lot of attractive local feature-based image matching methods have already been developed for the last two decades. Unfortunately, as far as we know, there are no optimal matching methods for automatic colorization because the requirements for pixel matching in automatic colorization are wholly different from those for traditional image matching. To design an efficient matching algorithm for automatic colorization, clustering pixel with low computational cost and generating descriptive feature vector are the most important challenges to be solved. In this paper, we present a novel method to address these two problems. In particular, our work concentrates on solving the second problem (designing a descriptive feature vector); namely, we will discuss how to learn a descriptive texture feature using scaled sparse texture feature combining with a nonlinear transformation to construct an optimal feature descriptor. Our experimental results show our proposed method outperforms the state-of-the-art methods in terms of robustness for color reconstruction for automatic colorization applications.

Proposal generation method for object detection in infrared image

The visual information of a surveillance system should provide extended and just-in-time perception in monitored environments. However, delivering visual information of surveillance from nonstationary providers, such as mobile cameras, to designated end-users over bandwidth-limited networks is a challenging task. In this paper, for transmitting visual objects of interest, an embedded spatial coding approach is proposed to progressively encode the presentation quality of visual objects under receivers' bandwidth limitation. First, different levels of importance for the objects of interest are characterized by the standard edge detection algorithm and several designated priorities for these objects are progressively assigned. The spatial coding approach then adopts a wavelet-based image coding algorithm based on the assigned priority of each object of interest. Therefore, each individual receiver can adaptively obtain the best visual information of the most important object under its available bandwidth. The proposed priority assignment and spatial coding mechanism has been extensively tested and analyzed. The experimental results show that, under different bandwidth scenarios and required acceptable signal-quality levels, the end-users can obtain the best visual quality of the important objects of interest. Therefore, the importance of the objects of interest in real-time surveillance video can be properly determined and the required transmission quantity can be effectively allocated.

Object density-based image segmentation and its applications in biomedical image analysis

The paper solves the problem of detecting exemplified objects in a color image. A solution provides the representation of similar objects in the same colors, and different objects in different colors. This is achieved by combining images of object examples and a target image into a single joint image, which is represented in sequential number 1, 2, ..., etc. colors. The mentioned effect is demonstrated by detecting irises and pupils in a test image. It is explained by the fact that: a) the joint image is approximated by a hierarchy of approximations in sequential color numbers; b) the hierarchy of approximations is described by a convex sequence of approximation errors (values of the total squared error ); c) due to the convexity, the approximation errors are reduced for all approximations of the joint image. In the last explanation item, it is applied the operation of combining hierarchically organized objects into a single object, which is introduced in this paper. To produce the required hierarchy of image approximations Ward's pixel clustering is used. Ward's method is generalized for image processing by parts (within pixel subsets) that provides generation of multiple proper approximation hierarchies and accelerates the calculations. To do so, the so- called split-and-merge pixel cluster CI-method is embedded into Ward's generalized method to provide a real-life minimization of the error for image approximation in a fixed number of colors.

This paper presents a new efficient enhanced automatic target recognition approach based on hybrid feature and intensity based methods. The proposed approach is based on comparison and matching between the acquired target image and the stored database images through edge detection, morphological operation, image transformation, image histogram matching and mutual information for both target and stored database images. Sobel operator is used to detect edges in both acquired target image and the stored database images. Hough transform is used as an image transformation which groups the extracted edge points in edged images to an appropriate set of lines which helps in features extraction and matching processes in both of target and stored database images. This gives an initial indication about realization and recognition between target image and its corresponding database image. Comparisons of image histogram and mutual information for both target and database images are finally used to emphasize the recognition of the target image and its verification with its corresponding database image. The proposed recognition approach passed through four stages which are: edge detection by Sobel edge detector, thinning as a morphological operation, Hough transformation, matching process using median deviation and finally histogram matching and measuring the mutual information between target and the available database images. The experimental results proved that the proposed hybrid target recognition approach gives a more accurate and successful recognition rate than other techniques which depend on using feature based methods or intensity based methods.

An optical illusion is an image of a real target perceived by the eye that is deceptive or misleading due to a physiological illusion or a specific visual trick. The recently developed metamaterials provide efficient approaches to generate a perfect optical illusion. However, all existing research on metamaterial illusions has been limited to theory and numerical simulations. Here, we propose the concept of a radar illusion, which can make the electromagnetic (EM) image of a target gathered by radar look like a different target, and we realize a radar illusion device experimentally to change the radar image of a metallic target into a dielectric target with predesigned size and material parameters. It is well known that the radar signatures of metallic and dielectric objects are significantly different. However, when a metallic target is enclosed by the proposed illusion device, its EM scattering characteristics will be identical to that of a predesigned dielectric object under the illumination of radar waves. Such an illusion device will confuse the radar, and hence the real EM properties of the metallic target cannot be perceived. We designed and fabricated the radar illusion device using artificial metamaterials in the microwave frequency, and good illusion performances are observed in the experimental results.

Osmosis is a transport phenomenon that is omnipresent in nature. It differs from diffusion by the fact that it allows nonconstant steady states. In our paper we lay the foundations of osmosis filtering for visual computing applications. We model filters with osmotic properties by means of linear drift-diffusion processes. They preserve the average grey value and the nonnegativity of the initial image. Most interestingly, however, we show how the nonconstant steady state of an osmosis evolution can be steered by its drift vector field. We interpret this behaviour as a data integration mechanism. In the integrable case, we characterise the steady state as a minimiser of a suitable energy functional. In the nonintegrable case, we can exploit osmosis as a framework to fuse incompatible data in a visually convincing way. Osmotic data fusion differs from gradient domain methods by its intrinsic invariance under multiplicative grey scale changes. The osmosis framework constitutes a novel class of methods that can be taylored to solve various problems in image processing, computer vision, and computer graphics. We demonstrate its versatility by proposing osmosis models for compact image respresentation, shadow removal, and seamless image cloning.Keywordsosmosisdrift–diffusionFokker–Planck equationdiffusion filtersgradient domain methodsshadow removalimage editing

In this paper, we present an approach for creating animatable 3D models from temporal monocular image acquisitions of non-rigid objects. During deformation, the object of interest is captured with only a single camera under full perspective projection. The aim of the presented framework is to obtain a shape deformation model in terms of joints and skinning weights that can finally be used for animating the model vertices. First, the monocular rigid shape estimation problem is solved by computing a template model of the object in rest pose from an image sequence. Next, the unknown external camera parameters and the deformation for each vertex are estimated alternately in a sequential approach. The resulting consistent non-rigid shape geometries are used to compute a kinematic skeleton control structure including skinning weights and optimized shape. For that, a completely data-driven optimization scheme is used, which iterates over three steps: (a) optimization of pose for each frame as well as joint parameters consistent over the entire sequence, (b) optimization of rest pose vertices to enhance the shape and (c) optimization of skinning weights for improved deformation characteristics. With experimental results on publicly available synthetic as well as real-world datasets, we demonstrate the quality of the proposed approach. The resulting models with fixed topology and rigged with skeleton and skinning weights can be animated in existing render engines.

The efficiency of spatial query processing is crucial for many applications such as location-based services. In spatial networks, queries like k-NN queries are all based on network distance evaluation. Classic solutions for these queries rely on network expansion and are not efficient enough for large networks. Some approaches have improved the query efficiency but brought considerable space cost for index. To address these problems, we propose a hierarchical graph partitioning based index named Partition Tree. It organizes the vertices of a spatial network into a hierarchy through a series of graph partitioning processes. Meanwhile precomputed distances are associated with this hierarchy to facilitate efficient query processing. Inspired by the observation that queries are usually invoked around objects of interest, we propose a query-oriented optimization on top of the Partition Tree. It uses a cost model to evaluate the influence of the object distribution and partitioning topology on the query efficiency. Then a cost-efficient graph partitioning method is developed based on this cost model. Experimental results on real datasets demonstrate that our proposed index and algorithms have superior performance over the state-of-the-art approaches and are scalable to large spatial networks.

In interactive human-robot path-planning, a capability for expressing the path topology provides a natural mechanism for describing task requirements. We propose a topology-aware RRT∗ algorithm that can explore in parallel any given set of topologies. The topological information used by the algorithm can either be assigned by the human prior to the planning or be selected from the human in posterior path selection. Theoretical analyses and experimental results are given to show that the optimal path of any topology can be found, including a winding topological constraint wherein the robot must circle one or more objects of interest.