Global Distribution Adjustment and Nonlinear Feature Transformation for Automatic Colorization

This study represents an innovative automatic method for black and white films colorization using texture features and a multilayer perceptron artificial neural network. In the proposed method, efforts are made to remove human interference in the process of colorization and replace it with an artificial neural network (ANN) which is trained using the features of the reference frame. Later, this network is employed for automatic colorization of the remained black and white frames. The reference frames of the black and white film are manually colored. Using a Gabor filter bank, texture features of all the pixels of the reference frame are extracted and used as the input feature vector of the ANN, while the output will be the color vector of the corresponding pixel. Finally, the next frames’ feature vectors are fed respectively to the trained neural network, and color vectors of those frames are the output. Applying AVI videos and using various color spaces, a series of experiments are conducted to evaluate the proposed colorization process. This method needs considerable time to provide a reasonable output, given rapidly changing scenes. Fortunately however, due to the high correlation between consecutive frames in typical video footage, the overall performance is promising regarding both visual appearance and the calculated MSE error. Apart from the application, we also aim to show the importance of the low level features in a mainly high level process, and the mapping ability of a neural network.

Advanced Chinese ink painting also includes work-brush flower and bird paintings with brilliant colors, in contrast to traditional ink paintings that often only use water, ink, and black and white. This serves as the foundation for our investigation into a generalized transfer problem involving ink and wash, or an ink painting coloring problem. Our goal is to automatically colorize black and white ink paintings using deep neural networks. This study can serve as a guide for coloring ink paintings and broaden the range of applications for ink painting style transfer. The high-level semantic information and low-level local features of ink paintings cannot be successfully extracted using the current generalized style transfer approach (colorization algorithm). The resulting images have muddy borders and low color saturation. In order to improve the accuracy and coherence of the coloring of ink paintings, we build training by combining the global and local features of ink paintings with the achievements of generative adversarial networks already made in the field of colorization. Comparative and objective evaluations of the experimental portion are made using metrics like peak signal-to-noise ratio (PSNR), structural similarity (SSIM), colorfulness, and user studies. Additionally, our approach beats the previous comparison approaches in terms of creative expression, color richness, and color overflow management.

Many studies have recently applied deep learning to the automatic colorization of line drawings. However, it is difficult to paint empty pupils using existing methods because the convolutional neural network are trained with pupils that have edges, which are generated from color images using image processing. Most actual line drawings have empty pupils that artists must paint in. In this paper, we propose a novel network model that transfers the pupil details in a reference color image to input line drawings with empty pupils. We also propose a method for accurately and automatically colorizing eyes. In this method, eye patches are extracted from a reference color image and automatically added to an input line drawing as color hints using our pupil position estimation network.

Intensity-based similarity measures assume that the original signal intensity of different modality images can provide statistically consistent information regarding the two modalities to be co-registered. In multi-modal registration problems, however, intensity-based similarity measures are often inadequate to identify an optimal transformation. Texture features can improve the performance of the multi-modal co-registration by providing more similar appearance representations of the two images to be co-registered, compared to the signal intensity representations. Furthermore, texture features extracted at different length scales (neighborhood sizes) can reveal similar underlying structural attributes between the images to be co-registered similarities that may not be discernible on the signal intensity representation alone. However one limitation of using texture features is that a number of them may be redundant and dependent and hence there is a need to identify non-redundant representations. Additionally it is not clear which features at which specific scales reveal similar attributes across the images to be co-registered. To address this problem, we introduced a novel approach for multimodal co-registration that employs new multi-scale image representations. Our approach comprises 4 distinct steps: (1) texure feature extraction at each length scale within both the target and template images, (2) independent component analysis (ICA) at each texture feature length scale, and (3) spectrally embedding (SE) the ICA components (ICs) obtained for the texture features at each length scale, and finally (4) identifying and combining the optimal length scales at which to perform the co-registration. To combine and co-register across different length scales, -mutual information (-MI) was applied in the high dimensional space of spectral embedding vectors to facilitate co-registration. To validate our multi-scale co-registration approach, we aligned 45 pairs of prostate MRI and histology images corresponding to the prostate with the objective of mapping extent of prostate cancer annotated by a pathologist on the pathology onto the pre-operative MRI. The registration results showed higher correlation between template and target images with average correlation ratio of 0.927 compared to 0.914 for intensity-based registration. Additionally an improvement in the dice similarity coefficient (DSC) of 13.6% was observed for the multi-scale registration compared to intensity-based registration and an 1.26% DSC improvement compared to registration involving the best individual scale.

The paper solves the problem of detecting exemplified objects in a color image. A solution provides the representation of similar objects in the same colors, and different objects in different colors. This is achieved by combining images of object examples and a target image into a single joint image, which is represented in sequential number 1, 2, ..., etc. colors. The mentioned effect is demonstrated by detecting irises and pupils in a test image. It is explained by the fact that: a) the joint image is approximated by a hierarchy of approximations in sequential color numbers; b) the hierarchy of approximations is described by a convex sequence of approximation errors (values of the total squared error ); c) due to the convexity, the approximation errors are reduced for all approximations of the joint image. In the last explanation item, it is applied the operation of combining hierarchically organized objects into a single object, which is introduced in this paper. To produce the required hierarchy of image approximations Ward's pixel clustering is used. Ward's method is generalized for image processing by parts (within pixel subsets) that provides generation of multiple proper approximation hierarchies and accelerates the calculations. To do so, the so- called split-and-merge pixel cluster CI-method is embedded into Ward's generalized method to provide a real-life minimization of the error for image approximation in a fixed number of colors.

Currently, gradient domain methods are popular for producing seamless cloning of a source image patch into a target image. However, structure conflicts between the source image patch and the target image may generate artifacts, preventing the general practices. In this paper, we tackle the challenge by incorporating the users' intent in outlining the source patch, where the boundary drawn generally has different appearances from the objects of interest. We first reveal that artifacts exist in the over-included region, the region outside the objects of interest in the source patch. Then we use the diversity from the boundary to approximately distinguish the objects from the over-included region, and design a new algorithm to make the target image adaptively take effects in blending. So the structure conflicts can be efficiently suppressed to remove the artifacts around the objects of interest in the composite result. Moreover, we develop an interpolation measure to composite the final image rather than solving a Poisson equation, and speed up the interpolation by treating pixels in clusters and using hierarchical sampling techniques. Our method is simple to use for instant and high-quality image cloning, in which users only need to outline a region of interested objects to process. Our experimental results have demonstrated the effectiveness of our cloning method.

Recent developments of video and sensing technology have led to large volumes of digital media data. Current media production relies on videos from the principal camera together with a wide variety of heterogeneous source of supporting data [photos, light detection and ranging point clouds, witness video camera, high dynamic range imaging, and depth imagery]. Registration of visual data acquired from various 2D and 3D sensing modalities is challenging because current matching and registration methods are not appropriate due to differences in structure, format, and noise characteristics for multimodal data. A combined 2D/3D visualization of this registered data allows an integrated overview of the entire data set. For such a visualization, a Web-based context presents several advantages. In this paper, we propose a unified framework for registration and visualization of this type of visual media data. A new feature description and matching method is proposed, adaptively considering local geometry, semiglobal geometry, and color information in the scene for more robust registration. The resulting registered 2D/3D multimodal visual data are too large to be downloaded and viewed directly via the Web browser, while maintaining an acceptable user experience. Thus, we employ hierarchical techniques for compression and restructuring to enable efficient transmission and visualization over the Web, leading to interactive visualization as registered point clouds, 2D images, and videos in the browser, improving on the current state-of-the-art techniques for Web-based visualization of big media data. This is the first unified 3D Web-based visualization of multimodal visual media production data sets. The proposed pipeline is tested on big multimodal data set typical of film and broadcast production, which are made publicly available. The<br/>proposed feature description method shows two times higher precision of feature matching and more stable registration performance than existing 3D feature descriptors.

Specific to the problem of feature description for colour image, a novel speeded-up robust features SURF descriptor of colour image-based quaternion CSURF-Q is proposed in this paper. Firstly, in order to represent the colour information of colour image effectively, the colour information among three channels is expressed by the three imaginary parts of quaternion. This can transform colour image to a pure quaternion matrix, which simplifies the representation process of image colour information. Secondly, the SURF features among three channels of colour image are extracted by using quaternion matrix. Because of the rotation invariance of quaternion norm, feature description vectors are generated as the form of quaternion norm. Finally, comparing CSURF-Q with SURF, FAIR-SURF, CSIFT, SIFT-CCH, colour-SURF and CHSIFT, the experiments prove that, when image appears affine changes, the correct matching of feature point pairs and time complexity are superior.

We sought to determine the joint influence of resolution, target range, and image contrast on the detection and identification of targets in simulated naturalistic scenes. Resolution requirements for target acquisition have been developed based on threshold values obtained using imaging systems, when target range was fixed, and image characteristics were determined by the system. Subsequent work has examined the influence of factors like target range and image contrast on target acquisition. We varied the resolution and contrast of static images in two experiments. Participants (soldiers) decided whether a human target was located in the scene (detection task) or whether a target was friendly or hostile (identification task). Target range was also varied (50-400 m). In Experiment 1, 30 participants saw color images with a single target exemplar. In Experiment 2, another 30 participants saw monochrome images containing different target exemplars. The effects of target range and image contrast were qualitatively different above and below 6 pixels per meter of target for both tasks in both experiments. Target detection and identification performance were a joint function of image resolution, range, and contrast for both color and monochrome images. The beneficial effects of increasing resolution for target acquisition performance are greater for closer (larger) targets.

In this paper, a landmark based localization system using a Kinect sensor is proposed and evaluated with the implemented system for precise and autonomous navigation of low cost robots. The proposed localization method finds the positions of landmark on the image plane and the depth value using color and depth images. The coordinates transforms are defined using the depth value. Using coordinate transformation, the position in the image plane is transformed to the position in the body frame. The ranges between the landmarks and the Kinect sensor are the norm of the landmark positions in body frame. The Kinect sensor position is computed using the tri-lateral whose inputs are the ranges and the known landmark positions. In addition, a new matching method using the pin hole model is proposed to reduce the mismatch between depth and color images. Furthermore, a height error compensation method using the relationship between the body frame and real world coordinates is proposed to reduce the effect of wrong leveling. The error analysis are also given to find out the effect of focal length, principal point and depth value to the range. The experiments using 2D bar code with the implemented system show that the position with less than 3cm error is obtained in enclosed space(3,500mm×3,000mm×2,500mm)

A comprehensive system for the analysis of portal images

Image colorization achieves more and more realistic results with the increasing power of recent deep learning techniques. It becomes more difficult to identify the synthetic colorized images by human eyes. In the literature, handcrafted-feature-based and convolutional neural network (CNN)-based forensic methods are proposed to distinguish between natural images (NIs) and colorized images (CIs). Although a recent CNN-based method achieves very good detection performance, an important issue (i.e., the blind detection problem) still remains and is not thoroughly studied. In this work, we focus on this challenging scenario of blind detection, i.e., no training sample is available from "unknown" colorization algorithm that we may encounter during the testing phase. This blind detection performance can be regarded as the generalization capability of a forensic detector. In this paper, we propose to first automatically construct negative samples through linear interpolation of paired natural and colorized images. Then, we progressively insert these negative samples into the original training dataset and continue to train the network. Experimental results demonstrate that our enhanced training can significantly improve the generalization performance of different CNN models.

In this article, new local matching measures, based on fuzzy similarity, for stereo matching of color images are proposed and evaluated. By formulating the individual similarity between a pair of pixels as a conjunction of the similarities of the respective color components, the field programmable gate array implementation problem becomes more computationally tractable, since the word-length of the numbers carrying the similarity information can be reduced compared to standard techniques (Sum of absolute differences and Sum of squared differences). It is also shown that combining information about color and local image structure (horizontal gradient component) in the matching measure and using a multi-scale measure is advantageous compared to using standard measures, in terms of percentage of correct matches and MSE for the erroneous matches. However, these improvements come at the prize of more complex implementations. Since the techniques are window-based, they share the typical drawbacks associated with other techniques of this type. This is expected, since the focus is on developing new local matching measure without addressing issues like post-processing of the resulting disparity maps. The techniques have been tested on two stereo color image pairs.

Many researchers have used the properties of the popular Elliptic Curve Cryptography(ECC) to devise a stronger and faster image encryption algorithm to assure the secrecy of images during online transmission. In this paper, a robust Elliptic curve based image encryption and authentication model for both grayscale and color images has been proposed. The model uses the secure Elliptic Curve Diffie-Hellman(ECDH) key exchange to compute a shared session key along with the improved ElGamal encoding scheme. 3D and 4D Arnold Cat maps are used to effectively scramble and transform the values of plain image pixels. A well-structured digital signature is used to verify the authenticity of the encrypted image prior to decryption. The model produces good-quality cipher images with an average entropy of 7.9993 for grayscale and 7.99925 for the individual components of color images. The model has high average NPCR of 99.6%, average UACI of 33.3% and low correlation for both grayscale and color images. The model has low computational costs with minimized point multiplication operations. The proposed model is robust with high resilience against statistical, differential, chosen-plaintext(CPA), known-plaintext(KPA) and occlusion attacks.

In this paper a new approach to reduction of a mixture of Gaussian and random impulse noises in color images is presented. The proposed filtering scheme is based on the application of the Bilateral Filter in order to address the problem of impulse noise reduction by determining the rate of region homogeneity, for calculating the weights needed for the Non-Local Means (NLM) averaging operation. Gaussian Mixture Model approach is applied for determining similarity between local image regions. The proposed solution is capable to successfully suppress the mixed noise of various intensities, at a lower computational cost than NLM method, due to the adaptive choice of size of search window for similar local neighborhoods. Experimental results prove that the introduced design yields better results than the Non-Local Means and Anisotropic Diffusion techniques in the case of color images contaminated by strong mixed Gaussian and impulsive noise.