Learning Affine Robust Binary Codes Based on Locality Preserving Hash

The widely used wavelet-thresholding techniques (DWT-H and DWT-S) have a near-optimal behavior that cannot be enhanced by any local denoising filter, but they cannot utilize the similarity of small-size image patches to enhance the denoising performance. Two of the latest improvements (WNLM and NLMW) introduced the Euclidean distance to measure the similarity of image patches, and then used the non-local meaning of similar patches for further denoising. Since the Euclidean distance is not a good similarity measurement, these two improvements are limited. In this study, we introduced the earth mover’s distance (EMD) as the similarity measure of small-scale patches within the wavelet sub-bands of noisy images. Moreover, at higher noise levels, we further incorporated joint bilateral filtering, which can filter both the spatial domain and the intensity domain of images. Denoising simulation experiments on BSDS500 demonstrated that our algorithm outperformed the DWT-H, DWT-S, WNLM, and NLMW algorithms by 4.197 dB, 3.326 dB, 2.097 dB, and 1.162 dB in terms of the average PSNR, and by 0.230, 0.213, 0.132, and 0.085 in terms of the average SSIM.

Dimensionality reduction is an important operation in information visualization, feature extraction, clustering, regression, and classification, especially for processing noisy high dimensional data. However, most existing approaches preserve either the global or the local structure of the data, but not both. Approaches that preserve only the global data structure, such as principal component analysis (PCA), are usually sensitive to outliers. Approaches that preserve only the local data structure, such as locality preserving projections, are usually unsupervised (and hence cannot use label information) and uses a fixed similarity graph. We propose a novel linear dimensionality reduction approach, supervised discriminative sparse PCA with adaptive neighbors (SDSPCAAN), to integrate neighborhood-free supervised discriminative sparse PCA and projected clustering with adaptive neighbors. As a result, both global and local data structures, as well as the label information, are used for better dimensionality reduction. Classification experiments on nine high-dimensional datasets validated the effectiveness and robustness of our proposed SDSPCAAN.

Non-local similarity images play a huge role in image denoising tasks. Many of the existing denoising algorithms have problems in that the edge information is too smooth, the reconstruction details are insufficient, and artifacts are easily generated while removing noise. In order to solve these shortcomings and improve the denoising accuracy, we propose a denoising algorithm based on non-local similarity and adaptive singular value threshold (ASVT). The algorithm consists of three basic steps: block matching grouping, ASVT denoising, and aggregation. First, similar image patches are grouped by block matching method, and each similar block group is used as a group matrix for each column of the matrix. Then, under the framework of image non-local similarity and low rank approximation, the denoising problem is transformed into low rank matrix approximation problem, which is solved by ASVT. Finally, all processed image patches are aggregated to produce an initial denoised image. In order to effectively avoid the influence of noise residual on denoising, the denoising result is further improved by the back projection strategy, and more detailed features are retained. Experimental results clearly show that the proposed algorithm is competitive with the current state-of-the-art denoising algorithms in terms of both quantitative measure and subjective visual quality and can retain more details and improve the smoothing problem.

The paper presents a new Convolutional Neural Network (CNN) architecture, called stacked stereo CNN, for computing disparity map from stereo images. In stacked stereo CNN, left and right image patches are stacked back-to-back and fed to a single tower CNN. This is in contrast to Siamese network where two towers are used, one for the left patch and other for the right patch. The proposed network is trained on a large set of similar and dissimilar image patches, which are generated from stereo images and their ground truth images from Middlebury stereo datasets. The network returns a dissimilarity score for a pair of image patch which is used to compute the cost volume. The cost volume is further refined using post processing steps before generating the final disparity map. The proposed network is evaluated on Middlebury datasets and achieves comparable results to the state-of-art algorithms.

3D reconstruction from monocular endoscopic images is a challenging task. State-of-the-art multi-view stereo (MVS) algorithms based on image patch similarity often fail to obtain a dense reconstruction from weakly-textured endoscopic images. In this paper, we present a novel deep-learning-based MVS algorithm that can produce a dense and accurate 3D reconstruction from a monocular endoscopic image sequence. Our method consists of three key steps. Firstly, a number of depth candidates are sampled around the depth prediction made by a pre-trained CNN. Secondly, each candidate is projected to the other images in the sequence, and the matching score is measured using a patch embedding network that maps each image patch into a compact embedding. Finally, the candidate with the highest score is selected for each pixel. Experiments on colonoscopy videos demonstrate that our patch embedding network outperforms zero-normalized cross-correlation and a state-of-the-art stereo matching network in terms of matching accuracy and that our MVS algorithm produces several degrees of magnitude denser reconstruction than the competing methods when same accuracy filtering is applied.

The key of image denoising algorithms is to preserve the details of the original image while denoising the noise in the image. The existing algorithms use the external information to better preserve the details of the image, but the use of external information needs the support of similar images or image patches. In this paper, an edge-aware image denoising algorithm is proposed to achieve the goal of preserving the details of original image while denoising and using only the characteristics of the noisy image. In general, image denoising algorithms use the noise prior to set parameters todenoise the noisy image. In this paper, it is found that the details of original image can be better preserved by combining the prior information of noise and the image edge features to set denoising parameters. The experimental results show that the proposed edge-aware image denoising algorithm can effectively improve the performance of block-matching and 3D filtering and patch group prior-based denoising algorithms and obtain higher peak signal-to-noise ratio and structural similarity values.

Blind image deconvolution, i.e., estimating both the latent image and the blur kernel from the only observed blurry image, is a severely ill-posed inverse problem. In this paper, we propose a blur kernel estimation method for blind motion deblurring using sparse representation and cross-scale self-similarity of image patches as priors to recover the latent sharp image from a single blurry image. Sparse representation indicates that image patches can always be represented well as a sparse linear combination of atoms in an appropriate dictionary. Cross-scale self-similarity results in that any image patch can in some way be well approximated by a number of other similar patches across different image scales. Our method is based on the observations that almost any image patch in a natural image has multiple similar patches in down-sampled versions of the image, and down-sampling produces image patches that are sharper than those in the blurry image itself. In our method, the dictionary for sparse representation is trained adaptively from sharper patches sampled from the down-sampled latent image estimate to make the similar patches of the latent sharp image well represented sparsely, and meanwhile, all patches from the latent image estimate are optimized to be as close to the sharper similar patches searched from the down-sampled version to enforce the sharp recovery of the latent image by constructing a non-local regularization. Experimental results on both simulated and real blurry images demonstrate that our method outperforms state-of-the-art blind deblurring methods.

Identifying the key genes related to tumors from gene expression data with a large number of features is important for the accurate classification of tumors and to make special treatment decisions. In recent years, unsupervised feature selection algorithms have attracted considerable attention in the field of gene selection as they can find the most discriminating subsets of genes, namely the potential information in biological data. Recent research also shows that maintaining the important structure of data is necessary for gene selection. However, most current feature selection methods merely capture the local structure of the original data while ignoring the importance of the global structure of the original data. We believe that the global structure and local structure of the original data are equally important, and so the selected genes should maintain the essential structure of the original data as far as possible. In this paper, we propose a new, adaptive, unsupervised feature selection scheme which not only reconstructs high-dimensional data into a low-dimensional space with the constraint of feature distance invariance but also employs -norm to enable a matrix with the ability to perform gene selection embedding into the local manifold structure-learning framework. Moreover, an effective algorithm is developed to solve the optimization problem based on the proposed scheme. Comparative experiments with some classical schemes on real tumor datasets demonstrate the effectiveness of the proposed method.

Image-based individual recognition of giant pandas in real wild scenes suffers from difficulties such as occlusion and multiple postures. Image painting is an important preprocessing step in solving the occlusion of giant panda images. A two-stage inpainting model of giant panda images based on partial convolutions and multi-scale features is proposed. In the coarse inpainting, the structural in-formation of the occluded part is restored, while the fine inpainting focuses on re-storing details such as textures and edges based on the coarse inpainting. A new PatchGAN with two discriminators in the coarse inpainting balances two-scale information guided by the WGAN-GP loss and L1 norm. The generator of the new PatchGAN uses partial-convolutions to avoid the propagation and influence of misinformation in the occluded area. In the fine inpainting, a new proposed module fusing multi-scale feature by contextual attention is added to the PatchGAN. The fine inpainting model learns and enhances the texture and details of the image output by the coarse inpainting through the global searching for mul-ti-scale similar image patches by multi-scale context attention. Thus, it can strengthen the semantic connection between occluded and real image regions. In order to achieve better multi-scale inpainting results, the perceived loss and style loss are added to the adversarial loss. Compared with state-of-art methods, the proposed method can effectively restore image textures and details while sup-pressing noise and artifacts from visual effects. The PSNR, SSIM and FIN of proposed method can achieve 35.69, 0.971 and 5.22 respectively, indicating that proposed method can obtain satisfied inpainting results.

Previous works about spatial information incorporation into a traditional bag-of-visual-words (BOVW) model mainly consider the spatial arrangement of an image, ignoring the rich textural information in land-use remote-sensing images. Hence, this article presents a 2-D wavelet decomposition (WD)-based BOVW model for land-use scene classification, since the 2-D wavelet decomposition method does well not only in textural feature extraction, but also in the multi-resolution representation of an image, which is favourable for the use of both spatial arrangement and textural information in land-use images. The proposed method exploits the textural structures of an image with colour information transformed into greyscale. Moreover, it works first by decomposing the greyscale image into different sub-images using 2-D discrete wavelet transform (DWT) and then by extracting local features of the greyscale image and all the decomposed images with dense regions in which a given image is evenly sampled by a regular grid with a specified grid space. After that, the method generates the corresponding visual vocabularies and computes histograms of visual word occurrences of local features found in each former image. Specifically, the soft-assignment or multi-assignment (MA) technique is employed, accounting for the impact of clustering on visual vocabulary creation that two similar image patches may be clustered into different clusters when increasing the size of visual vocabulary. The proposed method is evaluated on a ground truth image dataset of 21 land-use classes manually extracted from high-resolution remote-sensing images. Experimental results demonstrate that the proposed method significantly outperforms previous methods, such as the traditional BOVW model, the spatial pyramid representation-based BOVW method, the multi-resolution representation-based BOVW method, and so on, and even exceeds the best result obtained from the creator of the land-use dataset. Therefore, the proposed approach is very suitable for land-use scene classification tasks.

Recently, low-rank and sparse model-based dimensionality reduction (DR) methods have aroused lots of interest. In this paper, we propose an effective supervised DR technique named block-diagonal constrained low-rank and sparse-based embedding (BLSE). BLSE has two steps, i.e., block-diagonal constrained low-rank and sparse representation (BLSR) and block-diagonal constrained low-rank and sparse graph embedding (BLSGE). Firstly, the BLSR model is developed to reveal the intrinsic intra-class and inter-class adjacent relationships as well as the local neighborhood relations and global structure of data. Particularly, there are mainly three items considered in BLSR. First, a sparse constraint is required to discover the local data structure. Second, a low-rank criterion is incorporated to capture the global structure in data. Third, a block-diagonal regularization is imposed on the representation to promote discrimination between different classes. Based on BLSR, informative and discriminative intra-class and inter-class graphs are constructed. With the graphs, BLSGE seeks a low-dimensional embedding subspace by simultaneously minimizing the intra-class scatter and maximizing the inter-class scatter. Experiments on public benchmark face and object image datasets demonstrate the effectiveness of the proposed approach.

Over the years, many dimensionality reduction algorithms have been proposed for learning the structure of high dimensional data by linearly or non-linearly transforming it into a low-dimensional space. Some techniques can keep the local structure of data, while the others try to preserve the global structure. In this paper, we propose a linear dimensionality reduction technique that characterizes the local and global properties of data by firstly applying k-means algorithm on original data, and then finding the projection by simultaneously globally maximizing the between-cluster scatter matrix and locally minimizing the within-cluster scatter matrix, which actually keeps both local and global structure of data. Low complexity and structure preserving are two main advantages of the proposed technique. The experiments on both artificial and real data sets show the effectiveness and novelty of proposed algorithm in visualization and classification tasks.

The dimension reduction methods have been proved powerful and practical to extract latent features hidden in the signal for process monitoring. A novel nonlinear dimension reduction method called kernel orthogonal global-local preserving projections (KOGLPP) is proposed and applied for condition monitoring and fault detection. To overcome the shortcomings of kernel locality preserving projections (KLPP) and kernel principal component analysis (KPCA), the KOGLPP model aims at preserving the global and local data structures simultaneously by constructing a dual-objective optimization function, and a tuning parameter is introduced to adjust the trade-off between the global and local data structures. For the purpose of condition monitoring and fault detection, monitoring statistics are constructed in low dimensional feature space. As KOGLPP combines the advantages of both KPCA and KLPP, KOGLPP is also more powerful in extracting potential useful data characteristics. Finally, the effectiveness of the proposed nonlinear dimension reduction method is evaluated experimentally on a numerical example and a bearing test-rig. The results indicate its potential applications as an effective and reliable tool for condition monitoring and fault detection.

Feature extraction by learning Lorentzian metric tensor and its extensions

This paper proposes an image patch selection that finds similar patches in multiple images so that image denoising can suppress noise more effectively by exploiting the identified similar patches from the multi-view images. We encode all image patches in multi-view images into a low-dimensional space, and it allows for a denoiser to find similar patches effectively from the space. Our approach enables existing patch-based denoisers, which often find similar patches within an image window, to identify more similar patches by extending the limited search space into the entire space (i.e., all input images). We integrate our technique into state-of-the-art single-view denoising (block-matching and 3D filtering (BM3D)), and demonstrate that the BM3D combined with our approach is able to conduct multi-view image denoising effectively, without a major alteration to the existing algorithm.