Cell classification using convolutional neural networks in medical hyperspectral imagery

Variety purity is an important indicator in seed quality detection. Different varieties of corn seeds may be mixed in the growth and development process, which affects the growth and yield of the seeds. Thus, it is necessary to find a fast and non-destructively method to detect the purity. In this paper, the feasibility of combining hyperspectral imaging with deep convolutional neural network (DCNN) was studied to classify four corn seed varieties. Firstly, the average spectra from the region of seed in endosperm side hyperspectral images over the wavelength range of 450–979 nm were extracted. Secondly, the performances of three models were compared, including DCNN, K nearest neighbors (KNN) and support vector machine (SVM). DCNN model has the 100% training accuracy rate, 94.4% testing accuracy rate and 93.3% validation accuracy rate, and outperforms KNN and SVM models in most cases. DCNN model also had the best performance in evaluation indexes (sensitivity, specificity and precision). Finally, the visual classification map was generated according to the results of DCNN. Results show that DCNN can be adopted in spectral data analysis for the variety classification of corn seed; and the classification performance can be improved effectively.

Hyperspectral imaging has emerged as a novel imaging modality in the medical field, offering the ability to acquire images of biological tissues while simultaneously providing biochemical insights for in-depth tissue analysis. This approach facilitates early disease diagnosis, presenting advantages over traditional medical imaging techniques. Addressing challenges such as the computational burden of existing convolutional neural networks (CNNs) and imbalances in sample data, this paper introduces a lightweight GhostMRNet for the classification of microscopic hyperspectral images of human blood cells. The proposed model employs Ghost Modules to replace conventional convolutional layers and a cascading approach with small convolutional kernels for multiscale feature extraction, aiming to enhance feature extraction capabilities while reducing computational complexity. Additionally, an SE (Squeeze-and-Excitation) module is introduced to selectively allocate weights to features in each channel, emphasizing informative features and efficiently achieving spatial–spectral feature extraction in microscopic hyperspectral imaging. We evaluated the performance of the proposed GhostMRNet and compared it with other state-of-the-art models using two real medical hyperspectral image datasets. The experimental results demonstrate that GhostMRNet exhibits a superior performance, with an overall accuracy (OA), average accuracy (AA), and Kappa coefficient reaching 99.965%, 99.565%, and 0.9925, respectively. In conclusion, the proposed GhostMRNet achieves a superior classification performance at a smaller computational cost, thereby providing a novel approach for blood cell detection.

The accurate identification of melon seed varieties is essential for improving seed purity and the overall quality of melon production. In this study, hyperspectral imaging was used to identify six varieties of melon seeds. Both hyperspectral images and RGB images were generated during hyperspectral image acquisition. The spectral features of seeds were extracted from the hyperspectral images. The image features of the corresponding seeds were manually extracted from the RGB images. Five different datasets were formed using the spectral features and RGB images of the seeds, including seed spectral features, manually extracted seed image features, seed images, the fusion of seed spectral features with manually extracted features, and the fusion of seed spectral features with seed images. Logistic Regression (LR), Support Vector Classification (SVC), and Extreme Gradient Boosting (XGBoost) were used to establish classification models using spectral features and the manually extracted image features. Convolutional Neural Network (CNN) models were established using the five datasets. The results indicated that the CNN models achieved good performance in all five datasets, with classification accuracies exceeding 90% for the training, validation, and test sets. Also, CNN using the fused datasets obtained optimal performance, achieving classification accuracies exceeding 97% for the training, validation, and test sets. The results indicated that both spectral features and image features can be used to identify the six varieties of melon seeds, and their fusion of spectral features and image features can improve classification performance. These findings provide an alternative approach for melon seed variety identification, which can also be extended to other seed types.

Mapping multi-layered mangroves from multispectral, hyperspectral, and LiDAR data

Hyperspectral images (HSI) offer detailed spectral reflectance information about sensed objects through provision of information on hundreds of narrow spectral bands. HSI have a leading role in a broad range of applications, such as in forestry, agriculture, geology, and environmental sciences. The monitoring and management of agricultural lands is of great importance for meeting the nutritional and other needs of a rapidly and continuously increasing world population. In relation to this, classification of HSI is an effective way for creating land use and land cover maps quickly and accurately. In recent years, classification of HSI using convolutional neural networks (CNN), which is a sub-field of deep learning, has become a very popular research topic and several CNN architectures have been developed by researchers. The aim of this study was to investigate the classification performance of CNN model on agricultural HSI scenes. For this purpose, a 3D-2D CNN framework and a well-known support vector machine (SVM) model were compared using the Indian Pines and Salinas Scene datasets that contain crop and mixed vegetation classes. As a result of this study, it was confirmed that use of 3D-2D CNN offers superior performance for classifying agricultural HSI datasets.

ABSTRACTIn this letter, a new deep learning framework for spectral–spatial classification of hyperspectral images is presented. The proposed framework serves as an engine for merging the spatial and spectral features via suitable deep learning architecture: stacked autoencoders (SAEs) and deep convolutional neural networks (DCNNs) followed by a logistic regression (LR) classifier. In this framework, SAEs is aimed to get useful high-level features for the one-dimensional features which is suitable for the dimension reduction of spectral features, while DCNNs can learn rich features from the training data automatically and has achieved state-of-the-art performance in many image classification databases. Though the DCNNs has shown robustness to distortion, it only extracts features of the same scale, and hence is insufficient to tolerate large-scale variance of object. As a result, spatial pyramid pooling (SPP) is introduced into hyperspectral image classification for the first time by pooling the spatial feature maps of the top convolutional layers into a fixed-length feature. Experimental results with widely used hyperspectral data indicate that classifiers built in this deep learning-based framework provide competitive performance.

In recent years, growing interest in deep learning neural networks has raised a question on how they can be used for effective processing of high-dimensional datasets produced by hyperspectral imaging (HSI). HSI, traditionally viewed as being within the scope of remote sensing, is used in non-invasive substance classification. One of the areas of potential application is forensic science, where substance classification on the scenes is important. An example problem from that area—blood stain classification—is a case study for the evaluation of methods that process hyperspectral data. To investigate the deep learning classification performance for this problem we have performed experiments on a dataset which has not been previously tested using this kind of model. This dataset consists of several images with blood and blood-like substances like ketchup, tomato concentrate, artificial blood, etc. To test both the classic approach to hyperspectral classification and a more realistic application-oriented scenario, we have prepared two different sets of experiments. In the first one, Hyperspectral Transductive Classification (HTC), both a training and a test set come from the same image. In the second one, Hyperspectral Inductive Classification (HIC), a test set is derived from a different image, which is more challenging for classifiers but more useful from the point of view of forensic investigators. We conducted the study using several architectures like 1D, 2D and 3D convolutional neural networks (CNN), a recurrent neural network (RNN) and a multilayer perceptron (MLP). The performance of the models was compared with baseline results of Support Vector Machine (SVM). We have also presented a model evaluation method based on t-SNE and confusion matrix analysis that allows us to detect and eliminate some cases of model undertraining. Our results show that in the transductive case, all models, including the MLP and the SVM, have comparative performance, with no clear advantage of deep learning models. The Overall Accuracy range across all models is 98–100% for the easier image set, and 74–94% for the more difficult one. However, in a more challenging inductive case, selected deep learning architectures offer a significant advantage; their best Overall Accuracy is in the range of 57–71%, improving the baseline set by the non-deep models by up to 9 percentage points. We have presented a detailed analysis of results and a discussion, including a summary of conclusions for each tested architecture. An analysis of per-class errors shows that the score for each class is highly model-dependent. Considering this and the fact that the best performing models come from two different architecture families (3D CNN and RNN), our results suggest that tailoring the deep neural network architecture to hyperspectral data is still an open problem.

HLSK-CASMamba: hybrid large selective kernel and convolutional additive self-attention mamba for hyperspectral image classification

For hyperspectral image (HSI) classification, two branch networks generally use the convolution neural networks (CNNs) to extract the spatial features and the long short-term memory (LSTM) to learn the spectral features. However, CNN with a local kernel neglects the global properties of the whole HSI. LSTM doesn't consider the macroscopic and detailed information of spectra. In this paper, we propose a dual-view spectral and global spatial feature fusion network (DSGSF) to extract the spatial-spectral features for HSI classification, including a spatial subnetwork and a spectral subnetwork. In the spatial subnetwork, we propose a global spatial feature representation model based on the encoder-decoder structure with channel attention and spatial attention to learn the global spatial features. In the spectral subnetwork, we design a dual-view spectral feature aggregation model with view attention to learn the diversity of spectral features. By fusing the two subnetworks, we construct DSGSF to extract the spatial-spectral features of HSI with strong discriminating performance. Experimental results on three public datasets illustrate that the proposed method can achieve competitive results compared with the state-of-the-art methods. Code: https://github.com/RZWang-WH/DSGSF.

Hyperspectral data consists of continuous narrow spectral bands. Due to this, it has less spatial and high spectral information. Convolutional neural networks (CNNs) emerge as a highly contextual information model for remote sensing applications. Unfortunately, CNNs have constraints in their underlying network architecture in regards to the global correlation of spatial and spectral features, making them less reliable for mining and representing the sequential properties of spectral signatures. In this article, limpid size attention network (LSANet) is proposed, which contains 3D and 2D convolution blocks for enhancement of spatial-spectral features of the hyperspectral image (HSI). In addition, limpid attention block (LAB) is designed to provide a global correlation of the spectral and spatial features through LS attention. Furthermore, the computational costs of LS-attention are less compared to the multi-head self-attention (MHSA) of the classical vision transformer (ViT). In the ViT encoder a conditional position encoding (CPE) module is utilized that dynamically generates tokens from the feature maps to capture a richer contextual representation. The LSANet obtained overall accuracy (OA) of 98.78%, 98.67%, 97.52% and 89.45%, respectively, on the Indian Pines (IP), Pavia University (PU), Salina Valley (SV) and Botswana datasets. Our model’s quantitative and qualitative results are considerably better than the classical CNN and transformer-based methods.

Seed freezing damage is an agricultural disaster, which has a great impact on seed quality assurance. The feasibility of combining hyperspectral imaging with deep convolutional neural network (DCNN) to classify different freeze-damaged corn seeds was studied in this paper. At first, the hyperspectral images of corn seeds subjected to five different freezing temperatures at 400–1000 nm were acquired, and then the average spectra were extracted from the region of embryo hyperspectral images over the wavelength range of 450–979 nm. Next, four models (K nearest neighbors (KNN), support vector machine (SVM), extreme learning machine (ELM), and DCNN) were developed for five-category (5 frozen conditions) and four-category (“no freezing,” “slight freezing,” “moderate freezing,” and “severe freezing”) classifications, and the values of the evaluation indexes (accuracy, sensitivity, specificity, and precision) were calculated for comparison. The results show that DCNN model had the most satisfactory result with accuracy rates of 100% (training set), 96.9% (validation set), and 97.5% (testing set) for five-category classification, with accuracy rates of 100% (training, validation, and testing set) for four-category classification, and DCNN model also had the best performance in the evaluation indexes. At last, the visual classification map was generated according to the results of DCNN. It shows that hyperspectral imaging and DCNN can provide a novel method to detect the freezing damage of corn seeds quickly and inexpensively.

The Hyperspectral Images (HSI) are now being widely popular due to the evolution of satellite imagery and camera technology. Remote sensing has also gained popularity and it is also closely related to HSI. HSI possesses a wide variety of spatial and spectral features. However, HSI also has a consider-able amount of useless or redundant data. This redundant data causes a lot of trouble during classifications as it possesses a huge range in contrast to RGB. Traditional classification techniques do not apply efficiently to HSI. Even if somehow the traditional techniques are applied to it, the results produced are inefficient and undesirable. The Convolutional Neural Network (CNN), which are widely famous for the classification of images, have their fair share of trouble when dealing with HSI. 2D CNNs is not very efficient and 3D CNNs increases the computational complexity. To overcome these issues a new hybrid CNN approach is used which uses sigmoid activation function at the output layer, using a 2D CNN with 3D CNN to generate the desired output. Here, we are using HSI classification using hybrid CNN i.e., 2D and 3D. The dataset used is the Indian pines dataset sigmoid classifier for classification. And we gain the Overall accuracy 99.34 %, average accuracy 99.27%, kappa 99.25%.

Most of the existing hyperspectral image change detection methods based on deep learning utilize convolutional neural networks (CNN), recurrent neural networks (RNN), and long-short-term memory (LSTM). However, CNNs primarily focus on extracting spatial information from hyperspectral images, thereby failing to fully utilize channel and temporal information. In this paper, we propose a network called STSTNet that can jointly extract spatial, spectral, and temporal feature information. To achieve a richer feature representation space, we employ a spatial attention module to extract spatial feature information from feature maps, and a spectral transformer encoder structure to extract abundant spectral sequence information. Specifically, we design two temporal information extraction structures after each spatial information extraction structure or spectral information extraction structure to capture correlations and interactions within the hyperspectral images. Extensive experiments have been conducted on three hyperspectral datasets, and the experimental results demonstrate the network’s excellent performance.

Emotion recognition plays a pivotal role in advancing Human-Computer Interaction (HCI) by enabling systems to understand and respond to human emotional states. Among various modalities, Speech Emotion Recognition (SER) stands out as a non-invasive, cost-effective, and temporally efficient approach for detecting emotions. This study leverages the RAVDESS dataset, which provides high-quality audio recordings for emotion classification. The proposed methodology involves preprocessing audio signals to remove noise, extracting temporal and spectral features in both time and frequency domains, and implementing machine learning models for multi-class emotion classification. The study evaluates and compares the performance of several classification models, including Random Forest (RF), Multilayer Perceptron (MLP), Support Vector Machine (SVM), Convolutional Neural Networks (CNN), and Decision Trees (DT). Experimental results demonstrate promising accuracy levels, highlighting the potential of these machine learning techniques in SER. This research contributes to the integration of SER within Brain-Computer Interfaces (BCI) and other emotion-aware applications, paving the way for enhanced interactive systems. Keywords – Speech Emotion Recognition -Machine Learning - Random Forest -Multilayer Perceptron - Support Vector Machine - Convolutional Neural Networks - Decision Tree - RAVDESS Dataset

Hyperspectral image (HSI) classification is one of the most challenging problems in understanding HSI. Convolutional neural network(CNN), with the strong ability to extract features using the hidden layers in the network, has been introduced to solve this problem. However, several fully connected layers are always appended at the end of CNN, which dramatically reduced the efficiency of space utilization and make the classification algorithm hard to converge. Recently, a new network architecture called capsule network (CapsNet) was presented to improve the CNN. It uses groups of neurons as capsules to replace the neurons in traditional neural networks. Since the capsule can provide superior spectral features and spatial information extracted, its performance is better than the most advanced CNN in some fields. Motivated by this idea, a new remote sensing hyperspectral image classification algorithm called Conv-Caps is proposed to make full use of the advantages of both. We integrate spectral and spatial information into the proposed framework and combine Conv-Caps with Markov Random Field (MRF), which uses the graph cut expansion method to solve the classification task. The Caps-MRF method is further proposed. First, select an initial feature extractor,which a CNN without fully connected layers. Then, the initial recognition feature map is put into the newly designed CapsNet to obtain the probability map. Finally, the MRF model is used to calculate the subdivision labels. The presented method is trained with three real HSI datasets and is compared with the latest methods. We find the framework can produce competitive classification performance.