Exploiting the complementary strengths of multi-layer CNN features for image retrieval

Selective feature connection mechanism: Concatenating multi-layer CNN features with a feature selector

Imaging phantoms were scanned twice on 3 computed tomography scanners from 2 different manufactures with varying tube voltages and currents. Phantoms were segmented, and features were extracted using PyRadiomics and a pretrained CNN. After standardization the concordance correlation coefficient (CCC), mean feature variance, feature range, and the coefficient of variant were calculated to assess feature robustness. In addition, the cosine similarity was calculated for the vectorized activation maps for an exemplary phantom. For the in vivo comparison, the radiomics and CNN features of 30 patients with hepatocellular carcinoma (HCC) and 30 patients with hepatic colon carcinoma metastasis were compared. In total, 851 radiomics features and 256 CNN features were extracted for each phantom. For all phantoms, the global CCC of the CNN features was above 98%, whereas the highest CCC for the radiomics features was 36%. The mean feature variance and feature range was significantly lower for the CNN features. Using a coefficient of variant ≤0.2 as a threshold to define robust features and averaging across all phantoms 346 of 851 (41%) radiomics features and 196 of 256 (77%) CNN features were found to be robust. The cosine similarity was greater than 0.98 for all scanner and parameter variations. In the retrospective analysis, 122 of the 256 CNN (49%) features showed significant differences between HCC and hepatic colon metastasis. Convolutional neural network features were more stable compared with radiomics features against technical variations. Moreover, the possibility of tumor entity differentiation based on CNN features was shown. Combined with visualization methods, CNN features are expected to increase reproducibility of quantitative image representations. Further studies are warranted to investigate the impact of feature stability on radiological image-based prediction of clinical outcomes.

In this paper, we have proposed a novel feature representation on multi-scale receptive fields for objection recognition. The method is based on a modified convolutional neural networks (CNN), named network-in-network (NIN), which has shown a good performance in some computer vision tasks. However, applying NIN to some specific applications may encounter a few problems. First, the NIN removes the fully connected layers, which makes it unsuited to use in large-scale face recognition due to lack of an efficient feature representation, even though it brings a lot of performance benefits. Second, some lowerlayer features, which can make the feature representation more discriminative, is unused. In the pure forward architecture, these features are unseen to the classifier. To solve the two problems, we present a multi-scale receptive fields (MSRF) representation learning scheme. Based on a well trained NIN, we add a pathway to top layer and design a feature vector as final representation. In our experiments, we compare the result of our multi-scale receptive fields with standard NIN architecture. The results show our method can obtain a more explicit feature representation and improvements in performance.

Feature detectors and descriptors have been successfully used for various computer vision tasks, such as video object tracking and content-based image retrieval. Many methods use image gradients in different stages of the detection-description pipeline to describe local image structures. Recently, some, or all, of these stages have been replaced by convolutional neural networks (CNNs), in order to increase their performance. A detector is defined as a selection problem, which makes it more challenging to implement as a CNN. They are therefore generally defined as regressors, converting input images to score maps and keypoints can be selected with non-maximum suppression. This paper discusses and compares several recent methods that use CNNs for keypoint detection. Experiments are performed both on the CNN based approaches, as well as a selection of conventional methods. In addition to qualitative measures defined on keypoints and descriptors, the bag-of-words (BoW) model is used to implement an image retrieval application, in order to determine how the methods perform in practice. The results show that each type of features are best in different contexts.

Low-light image enhancement using Gaussian Process for features retrieval

Handwritten digit recognition, a task in computer vision, is critical for applications such as postal automation, banking, and digitization of forms. Traditional approaches have leveraged statistical models, but the rise of deep learning, particularly Convolutional Neural Networks (CNN) and Long-Short Term Memory (LSTM), has revolutionized the field. However, a comprehensive performance comparison of CNN and LSTM architectures in the context of handwritten digit classification remains underexplored. This study aimed to address this gap by evaluating and comparing CNN and LSTM models for the classification of handwritten digits. Two machine learning models – Convolutional Neural Network (CNN) and Long-Short Term Memory (LSTM) were trained with the preprocessed handwritten digit dataset. The CNN model was designed with multiple convolutional and pooling layers, along with dropout for regularization. The LSTM model was designed with LSTM layers to capture sequential patterns in the data, followed by a dense layer for classification. The models were implemented in python, evaluated and compared based on accuracy, precision, recall and F1-score. The evaluation and comparison results indicate that CNN achieved 99.31% accuracy, 99.0% precision, 99.0% recall, and a 99.0% F1-score, while LSTM achieved 98.90% accuracy, 99.0% precision, 99.0% recall, and a 99.0% F1-score. The results demonstrated that CNN outperformed LSTM in terms of accuracy and misclassification errors, making it the optimal choice for image-based handwritten digit recognition. This finding underscores the efficiency of CNN in addressing challenges related to digit recognition, contributing to the advancement of automated digit classification systems and improving the accuracy of image-based classification tasks.

Sex determination of silkworm pupae is important for silkworm industry. Multivariate analysis methods have been widely applied in hyperspectral imaging spectroscopy for classification. However, these methods require essential steps containing spectra preprocessing or feature extraction, which were not easy determined. Convolutional neural networks (CNNs), which have been employed in image recognition, could effectively learn interpretable presentations of the sample without the need of ad-hoc preprocessing steps. The species of silkworm pupae are usually up to hundreds. Conventional classifiers based on one species of silkworm pupae could not give high performance when explored to other species that not participating in the model building, resulting in bad generalization ability. In this study, a CNN model was trained to automatically identify the sex of silkworm pupae from different years and species based on the hyperspectral spectra. The results were compared with the frequently used conventional machine classifiers including support vector machine (SVM) and K nearest neighbors (KNN). The results showed that CNN outperformed SVM and KNN in terms of accuracy when applied to the raw spectra with 98.03%. However, the performance of CNN decreased to 95.09% when combined with the preprocessed data. Then principal component analysis (PCA) was adopted to reduce data dimensionality and extract features. CNN gave higher accuracy than SVM and KNN based on PCA. The discussion section revealed that CNN had high generalization ability that could classify silkworm pupae from different species with a rather well performance. It demonstrated that HSI technology in combination with CNN was useful in determining the sex of silkworm pupae.

Variety purity is an important indicator in seed quality detection. Different varieties of corn seeds may be mixed in the growth and development process, which affects the growth and yield of the seeds. Thus, it is necessary to find a fast and non-destructively method to detect the purity. In this paper, the feasibility of combining hyperspectral imaging with deep convolutional neural network (DCNN) was studied to classify four corn seed varieties. Firstly, the average spectra from the region of seed in endosperm side hyperspectral images over the wavelength range of 450–979 nm were extracted. Secondly, the performances of three models were compared, including DCNN, K nearest neighbors (KNN) and support vector machine (SVM). DCNN model has the 100% training accuracy rate, 94.4% testing accuracy rate and 93.3% validation accuracy rate, and outperforms KNN and SVM models in most cases. DCNN model also had the best performance in evaluation indexes (sensitivity, specificity and precision). Finally, the visual classification map was generated according to the results of DCNN. Results show that DCNN can be adopted in spectral data analysis for the variety classification of corn seed; and the classification performance can be improved effectively.

Seed freezing damage is an agricultural disaster, which has a great impact on seed quality assurance. The feasibility of combining hyperspectral imaging with deep convolutional neural network (DCNN) to classify different freeze-damaged corn seeds was studied in this paper. At first, the hyperspectral images of corn seeds subjected to five different freezing temperatures at 400–1000 nm were acquired, and then the average spectra were extracted from the region of embryo hyperspectral images over the wavelength range of 450–979 nm. Next, four models (K nearest neighbors (KNN), support vector machine (SVM), extreme learning machine (ELM), and DCNN) were developed for five-category (5 frozen conditions) and four-category (“no freezing,” “slight freezing,” “moderate freezing,” and “severe freezing”) classifications, and the values of the evaluation indexes (accuracy, sensitivity, specificity, and precision) were calculated for comparison. The results show that DCNN model had the most satisfactory result with accuracy rates of 100% (training set), 96.9% (validation set), and 97.5% (testing set) for five-category classification, with accuracy rates of 100% (training, validation, and testing set) for four-category classification, and DCNN model also had the best performance in the evaluation indexes. At last, the visual classification map was generated according to the results of DCNN. It shows that hyperspectral imaging and DCNN can provide a novel method to detect the freezing damage of corn seeds quickly and inexpensively.

Deep convolutional neural network (DCN) has become a common approach in visual robot self-localization. In a typical self-localization system, a DCN is trained as a visual place classifier from past visual experiences in the target environment. However, its classification performance can be deteriorated when it is tested in a different domain (e.g., times of day, weathers, seasons), due to domain shifts. Therefore, an efficient domain-adaptation (DA) approach to suppress perdomain DA cost would be desired. In this study, we address this issue with a novel “domain-shift localization (DSL)” technique that diagnosis the DCN classifier with the goal of localizing which region of the robot workspace is significantly affected by domain-shifts. In our approach, the DSL task is formulated as a fault-diagnosis (FD) problem, in which the deterioration of DCN-based self-localization for a given query image is viewed as an indicator of domain-shifts at the imaged region. In our contributions, we address the following non-trivial issues: (1) We address a subimage-level fine-grained DSL task given a typical coarse image-level DCN classifier, in which the target DCN system is queried with a region-of-interest (RoI) masked synthesized query image to diagnosis the RoI region; (2) We extend the DSL task to a relevance feedback (RF) framework, to perform a further query and return improved diagnosis results; and (3) We implement the proposed framework on 3D point cloud imagery-based self-localization and experimentally demonstrate the effectiveness of the proposed algorithm.

People with disabilities (hearing and listening impaired) have a vast social circle all over the world. For deaf and mute people, the sign language paves a way for the better communication which is the method that everyone understands and leads to effective communication among our diverse population. This study provides a comparative analysis for this complex task problems which are out of reach for a simple machine. As a result, there is a urgent need for the solution for such a problem, so the solution is integrating this problem with Machine Learning algorithms like Support Vector Machine, Convolutional Neural Network, K- Nearest neighbors. The results produced from Convolutional Neural Network are more accurate than those which is achieved by other several classifiers.

Nowadays, the information processing capabilities and resource storage capabilities of computers have been greatly improved, which also provides support for the neural network technology. Convolutional neural networks have good characterization capabilities in computer vision tasks, such as image recognition technology. Aiming at the problem of high similarity image recognition and classification in a specific field, this paper proposes a high similarity image recognition and classification algorithm fused with convolutional neural networks. First, we extract the image texture features, train different types, and different resolution image sets and determine the optimal texture different parameter values. Second, we decompose the image into subimages according to the texture difference, extract the energy features of each subimage, and perform classification. Then, the input image feature vector is converted into a one-dimensional vector through the alternating 5-layer convolution and 3-layer pooling of convolutional neural networks. On this basis, different sizes of convolution kernels are used to extract different convolutions of the image features, and then use convolution to achieve the feature fusion of different dimensional convolutions. Finally, through the increase in the number of training and the increase in the amount of data, the network parameters are continuously optimized to improve the classification accuracy in the training set and in the test set. The actual accuracy of the weights is verified, and the convolutional neural network model with the highest classification accuracy is obtained. In the experiment, two image data sets of gems and apples are selected as the experimental data to classify and identify gems and determine the origin of apples. The experimental results show that the average identification accuracy of the algorithm is more than 90%.

This article presents the artificial intelligence for object recognition by touching method with the tactile sensor, which has similar structural characteristics as the human hand. The object recognition uses a tactile glove sensor with small tactile sensors to spread throughout the glove, and there are touching points similar to human hands. Using the glove sensor to grasp the object that will obtain an image, there is tactile image. These images were used to compare recognition performance with two techniques: the Bag of Word (BoW) and the Convolutional Neural Network (CNN). In the experiment, the tactile glove performs 20 different objects. The BoW technique, the Scale Invariant Feature Transform (SIFT), has been used to feature extraction and then classified with K Nearest Neighbors (KNN) to evaluate the performance. The results illustrate that the accuracy of BoW and CNN is 70.80% and 97.20%, respectively. When comparing both techniques, it was found that the CNN had higher accuracy of about 26.40%. The results clearly show that CNN was more performance than BoW. Therefore, it is suitable to analyze tactile glove recognition, which can be applied to the recognition system of humanoid robots.

Hybrid Gabor Convolutional Networks

Convolutional Neural Networks (CNN) have dramatically boosted the performance of various computer vision tasks except visual tracking due to the lack of training data. In this paper, we pre-train a deep CNN offline to classify the 1 million images from 256 classes with very leaky non-saturating neurons for training acceleration, which is transformed to a discriminative classifier by adding an additional classification layer. In addition, we propose a novel approach for combining increasingly our CNN classifiers in a “cascade” structure through a modification of the AdaBoost framework, and then transfer the selected discriminative features from the ensemble of CNN classifiers to the robust visual tracking task, by updating online to robustly discard the background regions from promising object-like region to cope with appearance changes of the target. Extensive experimental evaluations on an open tracker benchmark demonstrate outstanding performance of our tracker by improving tracking success rate and tracking precision on an average of 9.2% and 13.9% at least over other state-of-the-art trackers.