Method for improving zero‐shot image classification

The supervised model based on deep learning has made great achievements in the field of image classification after training with a large number of labeled samples. However, there are many categories without or only with a few labeled training samples in practice, and some categories even have no training samples at all. The proposed zero-shot learning greatly reduces the dependence on labeled training samples for image classification models. Nevertheless, there are limitations in learning the similarity of visual features and semantic features with a predefined fixed metric (e.g., as Euclidean distance), as well as the problem of semantic gap in the mapping process. To address these problems, a new zero-shot image classification method based on an end-to-end learnable deep metric is proposed in this paper. First, the common space embedding is adopted to map the visual features and semantic features into a common space. Second, an end-to-end learnable deep metric, that is, the relation network is utilized to learn the similarity of visual features and semantic features. Finally, the invisible images are classified, according to the similarity score. Extensive experiments are carried out on four datasets and the results indicate the effectiveness of the proposed method.

In the image classification task, traditional model can only recognize annotated image samples, but class labels can't involve all the object categories. In order to reduce the dependence on the labels and recognize unannotated object samples, this paper proposes zero-shot image classification based on attribute. The binary attribute is used as the intermediate knowledge to migrate learned knowledge from training samples domain to test samples domain. Using the classification model of the multi-loss function based on ResNet-50 to predict the object attributes. Then, using an attribute matrix to represent the correspondence between the object class and the attribute. Finally, the result of attribute prediction is combined with the prior knowledge of the attribute matrix to get the category. Compared with the traditional image classification method, the attribute learning model is applied to the zero-shot image classification. The experimental data show that the method improves the recognition accuracy of the image and improves the flexibility of the image classification task, which lays the foundation for the multi-source domain adaptation induction problem.

Zero-shot image classification addresses the problem of non-overlapping categories between training and testing data. Human-annotated attributes are a commonly used form of auxiliary knowledge in zero-shot image classification. To assist experts in designing class attribute matrices, an interactive construction method is proposed, simplifying the tedious and unguided process. First, understandable attribute information is extracted from the training dataset images using a concept-based deep learning explainability method. Then, an interactive approach involving multi-view collaboration is employed to explore and analyze the importance of the extracted attributes. The system offers both global and local options to aid users in designing attribute values for testing dataset categories. Finally, the effectiveness and practicality of the design method are validated through case studies on the Animals with Attributes2 dataset and user evaluation experiments using the Likert scale, facilitating efficient and convenient class attribute construction for expert users.

Iris recognition has become a trend in the biometric discipline to identify an individual and is widely used compared to other biometrics and the traditional recognition system. Various factors, such as the obstruction of eyelids, eyelashes, shadows, specular reflection by eyeglasses, and other noises, can cause the occlusion of iris images. This study aims to propose a hybrid U-Net with ResNet segmentation method (HUREIS) in occlusion iris images for iris recognition. The HUREIS was compared with VGG-16 to determine its benchmark based on segmentation ac curacy and computational time. This study employed a MICHE I dataset that only considers occlusion images for eyelids and eyelashes for mobile devices. The findings showed that HUREIS performed the best dice score at 93.7% and Jaccard score at 88.3%, respectively. The performance of HUREIS is 5.4% better than VGG-16, with approximately 84% minimum computational time. The segmentation accuracy was compared with the state of-the-art methods based on the MICHE-I dataset and the U Net segmentation method to validate the effectiveness of the proposed iris segmentation method. Furthermore, this study evaluated the feature vector of HUREIS and VGG-16 to obtain the performance of iris classification using a support vector machine and decision tree C4.5 based on Euclidean distance and Cosine similarity. The Euclidean distance and cosine similarity were compared to determine the best image similarity score close to one. The results show that SVM for HUREIS achieved the optimal classification accuracy at 100% using Euclidean distance. Compared to decision tree C4.5, VGG-16 achieved 94.41% of accuracy using Euclidean distance.

Convolutional neural network (CNN) based methods have dominated the field of aerial scene classification for the past few years. While achieving remarkable success, CNN-based methods suffer from excessive parameters and notoriously rely on large amounts of training data. In this work, we introduce few-shot learning to the aerial scene classification problem. Few-shot learning aims to learn a model on base-set that can quickly adapt to unseen categories in novel-set, using only a few labeled samples. To this end, we proposed a meta-learning method for few-shot classification of aerial scene images. First, we train a feature extractor on all base categories to learn a representation of inputs. Then in the meta-training stage, the classifier is optimized in the metric space by cosine distance with a learnable scale parameter. At last, in the meta-testing stage, the query sample in the unseen category is predicted by the adapted classifier given a few support samples. We conduct extensive experiments on two challenging datasets: NWPU-RESISC45 and RSD46-WHU. The experimental results show that our method yields state-of-the-art performance. Furthermore, several ablation experiments are conducted to investigate the effects of dataset scale, the impact of different metrics and the number of support shots; the experiment results confirm that our model is specifically effective in few-shot settings.

Recent years have witnessed a surge of interest in graph-based transductive image classification. Existing simple graph-based transductive learning methods only model the pairwise relationship of images, however, and they are sensitive to the radius parameter used in similarity calculation. Hypergraph learning has been investigated to solve both difficulties. It models the high-order relationship of samples by using a hyperedge to link multiple samples. Nevertheless, the existing hypergraph learning methods face two problems, i.e., how to generate hyperedges and how to handle a large set of hyperedges. This paper proposes an adaptive hypergraph learning method for transductive image classification. In our method, we generate hyperedges by linking images and their nearest neighbors. By varying the size of the neighborhood, we are able to generate a set of hyperedges for each image and its visual neighbors. Our method simultaneously learns the labels of unlabeled images and the weights of hyperedges. In this way, we can automatically modulate the effects of different hyperedges. Thorough empirical studies show the effectiveness of our approach when compared with representative baselines.

As an improved method of k-nearest neighbor classification, the local mean-based nonparametric classifier had the ability to resist the effects of noise and classify unbalanced data. When selecting the nearest k samples and calculating the distance between the test samples and the local mean-based vectors, it always used Euclidean distance. However, for multi-dimensional data, using Euclidean distance which focused on the difference of the value to determine whether two vectors was similar was not so accurate. To solve this problem, a new local mean-based nonparametric classification method was proposed in this paper. It used the cosine distance which focused more on the difference of the dimension to select the k nearest neighbors and compute the distance between the test samples and the local mean-based vect ors. The new local mean-based nonparametric classification method was tested on the UCI datasets: Iris and Wine for different values of k in different test data set, the simulation results show that it outperforms the existing local mean-based nonparametric classifier.

Most hyperspectral image (HSI) classification methods assume that all classes in the test set are present during training. However, in real-world applications, acquiring labeled training samples is challenging. As a result, it is difficult for the training dataset to cover all possible land cover types, leading to the generalized zero-shot learning (GZSL) problem. Recently, vision-language models (VLMs) have provided rich semantic priors for land cover classes, offering promising potential for GZSL. However, two fundamental gaps hinder their application to HSI classification: the task paradigm gap, arising from the difference between image-level VLMs and the pixel-level HSI classification task; and the knowledge gap, due to the inconsistency between VLM features and HSI spectral–spatial representations. To bridge both gaps, a novel framework leveraging VLM semantic priors for GZSL in HSI classification is proposed, primarily using pseudo-labeling technique to provide knowledge for unseen classes. Specifically, a pseudo-label generation and enhancement module enables a paradigm transition from image-level understanding to pixel-level classification by incorporating HSI’s spatial information. A pseudo-label correction module then refines noisy labels using spectral cues to address the knowledge gap. Finally, a global learning strategy integrates pseudo-label distillation, supervised learning, and feature regularization to classify seen classes while enabling generalization to unseen ones. Experiments on benchmark HSI datasets demonstrate the proposed method’s superiority in generalized zero-shot classification. This work highlights the potential of VLMs in advancing HSI classification in practical applications.

Extracting image feature points and classification methods is the key of content-based image classification. In this paper, SIFT(Scale-invariant feature transform) algorithm is used to extract feature points, all feature points extracted are clustered by K-means clustering algorithm, and then BOW(bag of word) of each image is constructed. Finally, SVM(Support Vector Machine) is used to train a multi-class classifier to classify images. The SIFT algorithm has a strong tolerance for scaling, rotation, brightness changes, and noise. The k-means algorithm is simple in structure and fast in convergence. SVM can get better results in small sample training set and has excellent generalization ability. Experimental results show that the accuracy of image classification in the above methods is about 90%.

In this research, a new method for spectral-spatial classification of hyperspectral images based on hierarchical segmentation algorithm is introduced. Among the various spectral-spatial classification algorithms, the marker-based hierarchical segmentation algorithm has so far achieved the best results in combination with the support vector machine (SVM) classification algorithm. In the proposed method adopted in this research, the dimensions of the hyperspectral image were reduced via the minimum noise fraction (MNF) algorithm. Then, five spatial/texture features of wavelet transform, Gabor transform, mean, entropy, and contrast were extracted from the obtained bands. Later on, the multi-layer perceptron (MLP) neural network and SVM classification algorithms were applied to the obtained spectral and texture features, and their results were combined. The resulting map was then used to select the markers and to combine them with the marker-based hierarchical segmentation algorithm using the majority voting rule. The proposed method was applied to three hyperspectral images, Pavia, Telops, and Washington DC Mall. The results of the experiments demonstrate the superiority of the proposed method over the initial marker-based hierarchical algorithm. Quantitatively, it was better by 8, 12, and 9% for the Pavia, Telops, and Washington DC Mall datasets regarding the Kappa coefficient parameter, respectively.

In this paper we proposed a new method for texture classification of ultrasonic liver images based on Gabor wavelet. It is well known that Gabor wavelets attain maximum joint space-frequency resolution which is highly significant in the process of texture extraction in which the conflicting objectives of accuracy in texture representation and texture spatial localization are both important. This fact has been explored in our results as it shows that the classification rate obtained by Gabor wavelet is higher that those obtained using dyadic wavelets. The feature vector consists of 10 elements at each scale from Gabor wavelets which is relatively small compared to other methods. This has a significant impact on the speed of retrieval process. The proposed algorithm applied to discriminate ultrasonic liver images into three disease states that are normal liver, liver hepatitis and cirrhosis. In our experiment 45 liver sample images from each three disease states which already proven by needle biopsy were used. We achieved the sensitivity 85% in the distinction between normal and hepatitis liver images and 86% in the distinction between normal and cirrhosis liver images. Based on our experiments, the Gabor wavelet is more appropriate than dyadic wavelets for texture classification as it leads to higher classification accuracy.

The feature extraction-coding-pooling framework improves the performance of image classification because it generates robust and discriminative image representation. The proposed system uses saliency driven multi-scale nonlinear diffusion filtering with linear distance coding (LDC) method for image classification. The saliency driven multi-scale nonlinear diffusion filtering generates the images at small, mid and high scale and concatenation of information at these scales produces robust image classification using LDC. The class manifolds generated by this classification system are given as input to image retrieval system which finally retrieves all images those are relevant to supplied image.

In recent studies, superpixel segmentation has been integrated into hyperspectral (HS) image classification methods. However, the existing superpixel-based classification methods usually suffer from two serious problems. First, the accuracy and efficiency of current superpixel segmentation approaches cannot meet the demands of practical applications for HS images; second, conventional superpixel-based classification methods generally consider each generated superpixel as a unit for the image classification, which may help to reduce the computing time but result in a significant decrease of the classification accuracy. To solve the problems, we propose a fast region growing based superpixel segmentation (FRGSS) algorithm and a novel texture-adaptive superpixel integration strategy (TASIS) for the HS image classification. Experimental results on real Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) HS images demonstrate that the proposed FRGSS outperforms the state-of-the-art superpixel algorithm. In addition, the superiority of the TASIS is verified compared to the pixel-wise and the conventional superpixel-based classification methods.

Exploitation of the spectral capabilities of modern hyperspectral image demands efficient preprocessing and analyses methods. Analysts' choice of classifier and dimensionality reduction (DR) method and the harmony between them determine the accuracy of image classification. Multiple classifier system (MCS) has the potential to combine the relative advantages of several classifiers into a single image classification exercise for the hyperspectral image classification. In this paper, we propose an algorithmic extension of the MCS, named as dynamic classifier system (DCS), which exploits the context-based image and information class characteristics represented by multiple DR methods for hyperspectral image classification for land cover mapping. The proposed DCS algorithm pairs up optimal combinations of classifiers and DR methods specific to the hyperspectral image and performs image classifications based only on the identified combinations. Further, the impact of various trainable and nontrainable combination functions on the performance of the proposed DCS has been assessed. Image classifications were carried out on five multi-site airborne hyperspectral images using the proposed DCS and were compared with the MCS and SVM based supervised image classifications with and without DR. The results indicate the potential of the proposed DCS algorithm to increase the classification accuracy considerably over that of MCS or SVM supervised image classifications.

Medical Vision-Language Models VLMs show significant potential for aux-iliary diagnosis, especially given the continuous growth of medical image da-ta. However, adapting these models effectively with limited labeled data re-mains a challenge. This paper proposes a cross-modal adaptation method for few-shot medical image classification based on pre-trained VLMs. Our ap-proach leverages both image features and corresponding text features ex-tracted from the pre-trained models to train a classifier head. Furthermore, we employ the SHAP interpretability analysis method to select the most in-formative text features, thereby enhancing classification performance. We evaluated our method on the CheXpert5x200 dataset using MedCLIP and KAD as foundation models, comparing it against zero-shot classification and uni-modal adaptation using only image features . Results demonstrate that our approach significantly improves few-shot classification performance over the baselines. The SHAP-based feature selection provides additional gains. Ultimately, we present a general, simple, and efficient cross-modal ad-aptation strategy that enhances medical VLM performance using only a small number of image samples, contributing to more reliable AI-powered diagnos-tic tools.