Image classification using the surface-shape operator and multiscale features

This paper proposes a method of multiscale image analysis through an operator called the operator. By considering an image function as a surface function, the surface-shape operator describes shape of each pixel in terms of the topographic structure such as hill, dale, ridge, valley, etc. The surface-shape operator is used as a pre-processing for extracting the multiscale topographic structures of images in scale-space representation. Then, image features are extracted from the transformed images instead of the original ones. The surface-shape operator has invariant properties under monotonic and linear gray tone transformations, and is insensitive to additive noise modelled by linear functions, so the proposed method has a robustness with brightness variations and shading effects. We illustrate the usefulness of the proposed method with an application to image classification, and also demonstrate performances concerning the robustness with brightness variations and shading effects. The multiresolution simultaneous autoregressive (MRSAR) method is selected as a representative method of multiscale approaches for image classification used to-date for comparison purposes. Results show that the proposed method performs better and has much more robustness with brightness variations and shading effects than the MRSAR method.

One hundred and sixty-eight genotypes of cotton from the same growing region were used as a germplasm group to study the validity of different genetic distances in constructing cotton core subset. Mixed linear model approach was employed to unbiasedly predict genotypic values of 20 traits for eliminating the environmental effect. Six commonly used genetic distances (Euclidean, standardized Euclidean, Mahalanobis, city block, cosine and correlation distances) combining four commonly used hierarchical cluster methods (single distance, complete distance, unweighted pair-group average and Ward's methods) were used in the least distance stepwise sampling (LDSS) method for constructing different core subsets. The analyses of variance (ANOVA) of different evaluating parameters showed that the validities of cosine and correlation distances were inferior to those of Euclidean, standardized Euclidean, Mahalanobis and city block distances. Standardized Euclidean distance was slightly more effective than Euclidean, Mahalanobis and city block distances. The principal analysis validated standardized Euclidean distance in the course of constructing practical core subsets. The covariance matrix of accessions might be ill-conditioned when Mahalanobis distance was used to calculate genetic distance at low sampling percentages, which led to bias in small-sized core subset construction. The standardized Euclidean distance is recommended in core subset construction with LDSS method.

Deep learning technology has promoted the performance of defocus blur detection. However, blur detectors suffer from background clutter, scale ambiguity and blurred boundaries of the defocus blur regions. To conquer these issues, previous methods propose to use multi-scale image patches or images for blur detection, which costs much computation time. In this paper, we propose a deep neural network that takes a single-scale image as input to generate robust defocus blur detection. Specifically, we first extract multi-scale convolutional features by a feature extraction network. And then we resize the convolutional features of each layer by a fixed ratio to approximate convolutional features that extracted from a resized image with the same ratio. By approximation, it not only generates features extracted from a scaled image but also reduces the computation of feature extraction from multi-scale images. We concatenate the features extracted from the original image with the approximated features at the corresponding layers by convolutional layers to increase the blur distinguish ability. We gradually fuse the convolutional features from top-to-bottom by Conv-LSTMs to refine the blur predictions. We compare our method with nine state-of-the-art defocus blur detectors on two defocus blur detection benchmark datasets. Experiment results demonstrate the effectiveness of our proposed defocus blur detector.

Hellwig’s method is a multi-criteria decision-making technique designed to facilitate the ranking of alternatives based on their proximity to the ideal solution. Typically, this approach calculates distances using the Euclidean norm, assuming implicitly that the considered criteria are independent. However, in real-world situations, the assumption of criteria independence is rarely met. The paper aims to propose an extension of Hellwig’s method by incorporating the Mahalanobis distance. Substituting the Euclidean distance with the Mahalanobis distance has proven to be effective in handling correlations among criteria, especially in the context of asymmetrical relationships between criteria. Subsequently, we investigate the impact of the Euclidean and Mahalanobis distance measures on the several variants of Hellwig procedures, analyzing examples based on various illustrative data with 10 alternatives and 4 criteria. Additionally, we examine the influence of three normalization formulas in Hellwig’s aggregation procedures. The investigation results indicate that both the distance measure and normalization formulas have some impact on the final rankings. The evaluation and ranking of alternatives using the Euclidean distance measure are influenced by the normalization formula, albeit to a limited extent. In contrast, the Mahalanobis distance-based Hellwig’s method remains unaffected by the choice of normalization formulas. The study concludes that the ranking of alternatives is strongly dependent on the distance measure employed, whether it is Euclidean or Mahalanobis. The Mahalanobis distance-based Hellwig method is deemed a valuable tool for decision-makers in real-life situations. It enables the evaluation of alternatives by considering interactions between criteria, providing a more comprehensive perspective for decision-making.

The principle component analysis(PCA) is a common feature extraction method in machine learning and pattern recognition approaches. PCA has been used in many applications, and face recognition in which specific faces are recognizing in an images database is one of the popular applications. The default distance metric which has been used with PCA based-face recognition is Euclidean distance. In this study, we have tested the Mahalanobis distance instead of Euclidean, and PCA based on Mahalanobis distance suggested a better performance on our students images database with highest recognition rate. However, we proposed weighted and normalized Mahalanobis distance based PCA-face recognition(PCA_WNMD). The proposed algorithm (PCA_WNMD) showed an improvement in faces recognition rate when tested on our students images database compared to PCA based on Mahalanobis and default Euclidean distances.

Spectral and textural classification of single and multiple band digital images

It is important for Mars exploration rovers to achieve autonomous and safe mobility over rough terrain. Terrain classification can help rovers to select a safe terrain to traverse and avoid sinking and/or damaging the vehicle. Mars terrains are often classified using visual methods. However, the accuracy of terrain classification has been less than 90% in read operations. A high-accuracy vision-based method for Mars terrain classification is presented in this paper. By analyzing Mars terrain characteristics, novel image features, including multiscale gray gradient-grade features, multiscale edges strength-grade features, multiscale frequency-domain mean amplitude features, multiscale spectrum symmetry features, and multiscale spectrum amplitude-moment features, are proposed that are specifically targeted for terrain classification. Three classifiers, K-nearest neighbor (KNN), support vector machine (SVM), and random forests (RF), are adopted to classify the terrain using the proposed features. The Mars image dataset MSLNet that was collected by the Mars Science Laboratory (MSL, Curiosity) rover is used to conduct terrain classification experiments. The resolution of Mars images in the dataset is 256 × 256. Experimental results indicate that the RF classifies Mars terrain at the highest level of accuracy of 94.66%.

The distance-based regression model has many applications in analysis of multivariate response regression in various fields, such as ecology, genomics, genetics, human microbiomics, and neuroimaging. It yields a pseudo F test statistic that assesses the relation between the distance (dissimilarity) of the subjects and the predictors of interest. Despite its popularity in recent decades, the statistical properties of the pseudo F test statistic have not been revealed to our knowledge. This study derives the asymptotic properties of the pseudo F test statistic using spectral decomposition under the matrix normal assumption, when the utilized dissimilarity measure is the Euclidean or Mahalanobis distance. The pseudo F test statistic with the Euclidean distance has the same distribution as the quotient of two Chi-squared-type mixtures. The denominator and numerator of the quotient are approximated using a random variable of the form \(\xi\chi_d^2+\eta\) and the approximate error bound is given. The pseudo F test statistic with the Mahalanobis distance follows an F distribution. In simulation studies, the approximated distribution well matched the “exact” distribution obtained by the permutation procedure. The obtained distribution was further validated on H1N1 influenza data, aging human brain data, and embryonic imprint data.

In order to improve the detection and recognition ability of multiscale hyperspectral images with weak edges, a lossless compression algorithm of multiscale hyperspectral images with weak edges based on improved deep learning is proposed. Using deep-level feature information extraction method to extract fine-grained features of cancer cells from multiscale hyperspectral images with weak edges, constructing 3D surface structure reconstruction model of multiscale hyperspectral images with weak edges, combining with variational Bayesian filter detection method to decompose multiscale color structure features of multiscale hyperspectral images with weak edges, and constructing gradually refined output structure model of multiscale hyperspectral images with weak edges, Fuzzy image coarse-scale neural network learning method is used to realize block matching and edge detection of multi-scale hyperspectral images with weak edges. Ultra-fine-grained features of multi-scale hyperspectral images with weak edges are extracted. Deep learning algorithm is used to adaptively optimize the lossless compression process of multi-scale hyperspectral images with weak edges, and the optimal design of lossless compression algorithm for multi-scale hyperspectral images with weak edges is realized. The simulation results show that the adaptive convergence of lossless compression of multi-scale weak edge hyperspectral images with this method is good, the feature resolution is strong, and the hyperspectral feature detection and recognition ability is good.

In this paper, a novel approach for illumination normalization under varying lighting conditions is presented. Our approach utilizes the fact that discrete cosine transform (DCT) low-frequency coefficients correspond to illumination variations in a digital image. Under varying illuminations, the images captured may have low contrast; initially we apply histogram equalization on these for contrast stretching. Then the low-frequency DCT coefficients are scaled down to compensate the illumination variations. The value of scaling down factor and the number of low-frequency DCT coefficients, which are to be re-scaled, are obtained experimentally. The classification is done using k-nearest neighbor classification and nearest mean classification on the images obtained by inverse DCT on the processed coefficients. The correlation coefficient and Euclidean distance obtained using principal component analysis are used as distance metrics in classification. We have tested our face recognition method using Yale face database B. The results show that our method performs without any error (100% face recognition performance) even on the most extreme illumination variations. There are different schemes in the literature for illumination normalization under varying lighting conditions, but no one is claimed to give 100% recognition rate under all illumination variations for this database. The proposed technique is computationally efficient and can easily be implemented for real time face recognition system.

Visual change detection, aiming at segmentation of video frames into foreground and background regions, is one of the elementary tasks in computer vision and video analytics. The applications of change detection include anomaly detection, object tracking, traffic monitoring, human machine interaction, behavior analysis, action recognition, and visual surveillance. Some of the challenges in change detection include background fluctuations, illumination variation, weather changes, intermittent object motion, shadow, fast/slow object motion, camera motion, heterogeneous object shapes and real-time processing. Traditionally, this problem has been solved using hand-crafted features and background modelling techniques. In recent years, deep learning frameworks have been successfully adopted for robust change detection. This article aims to provide an empirical review of the state-of-the-art deep learning methods for change detection. More specifically, we present a detailed analysis of the technical characteristics of different model designs and experimental frameworks. We provide model design based categorization of the existing approaches, including the 2D-CNN, 3D-CNN, ConvLSTM, multi-scale features, residual connections, autoencoders and GAN based methods. Moreover, an empirical analysis of the evaluation settings adopted by the existing deep learning methods is presented. To the best of our knowledge, this is a first attempt to comparatively analyze the different evaluation frameworks used in the existing deep change detection methods. Finally, we point out the research needs, future directions and draw our own conclusions.

In recent years, the deep learning is applied to the field of traffic sign detection methods which achieves excellent performance. However, there are two main challenges in traffic sign detection to be solve urgently. For one thing, some traffic signs of small size are more difficult to detect than those of large size so that the small traffic signs are undetected. For another, some false signs are always detected because of interferences caused by the illumination variation, bad weather and some signs similar to the true traffic signs. Therefore, to solve the undetection and false detection, we first propose a cascaded R-CNN to obtain the multiscale features in pyramids. Each layer of the cascaded network except the first layer fuses the output bounding box of the previous one layer for joint training. This method contributes to the traffic sign detection. Then, we propose a multiscale attention method to obtain the weighted multiscale features by dot-product and softmax, which is summed to fine the features to highlight the traffic sign features and improve the accuracy of the traffic sign detection. Finally, we increase the number of difficult negative samples for dataset balance and data augmentation in the training to relieve the interference by complex environment and similar false traffic signs. The data augment method expands the German traffic sign training dataset by simulation of complex environment changes. We conduct numerous experiments to verify the effectiveness of our proposed algorithm. The accuracy and recall rate of our method are 98.7% and 90.5% in GTSDB, 99.7% and 83.62% in CCTSDB and 98.9% and 85.6% in Lisa dataset respectively.

In this paper we have proposed wavelet based edge detection algorithm that combines the coefficients of wavelet transforms on a series of scales. The outcome of this algorithm is edginess like information further used to obtain Independent Components using ICA algorithms. The combination of Multiscale wavelet based edge detection and Independent Component Analysis (ICA) is used for Face Recognition becomes a novel approach. The independent components obtained by ICA algorithms are used as feature vectors for classification. The Euclidean distance (L2) classifier is used for testing of images. The algorithm is tested on two different databases i.e Asian face database and Indian face database of face images for variation in illumination, facial expressions and facial poses up to 1800rotation angle. Encouraging results of this unique approach of face recognition has given future direction for research work in this area.

Representational similarity analysis (RSA) tests models of brain computation by investigating how neural activity patterns reflect experimental conditions. Instead of predicting activity patterns directly, the models predict the geometry of the representation, as defined by the representational dissimilarity matrix (RDM), which captures how similar or dissimilar different activity patterns associated by different experimental conditions are. RSA therefore first quantifies the representational geometry by calculating a dissimilarity measure for each pair of conditions, and then compares the estimated representational dissimilarities to those predicted by each model. Here we address two central challenges of RSA: First, dissimilarity measures such as the Euclidean, Mahalanobis, and correlation distance, are biased by measurement noise, which can lead to incorrect inferences. Unbiased dissimilarity estimates can be obtained by crossvalidation, at the price of increased variance. Second, the pairwise dissimilarity estimates are not statistically independent, and ignoring this dependency makes model comparison statistically suboptimal. We present an analytical expression for the mean and (co)variance of both biased and unbiased estimators of the squared Euclidean and Mahalanobis distance, allowing us to quantify the bias-variance trade-off. We also use the analytical expression of the covariance of the dissimilarity estimates to whiten the RDM estimation errors. This results in a new criterion for RDM similarity, the whitened unbiased RDM cosine similarity (WUC), which allows for near-optimal model selection combined with robustness to correlated measurement noise.

Attention module has achieved great success in numerous vision tasks. However, existing visual attention modules generally consider the features of a single-scale, and cannot make full use of their multi-scale contextual information. Meanwhile, the multi-scale spatial feature representation has demonstrated its outstanding performance in a wide range of applications. However, the multi-scale features are always represented in a layer-wise manner, i.e. it is impossible to know their contextual information at a granular level. Focusing on the above issue, a context-aware residual module for image classification is proposed in this paper. It consists of a novel multi-scale channel attention module MSCAM to learn refined channel weights by considering the visual features of its own scale and its surrounding fields, and a multi-scale spatial aware module MSSAM to further capture more spatial information. Either or both of the two modules can be plugged into any CNN-based backbone image classification architecture with a short residual connection to obtain the context-aware enhanced features. The experiments on public image recognition datasets including CIFAR10, CIFAR100，Tiny-ImageNet and ImageNet consistently demonstrate that our proposed modules significantly outperforms a wide-used state-of-the-art methods, e.g., ResNet and the lightweight networks of MobileNet and SqueezeeNet.