Automatic Segmentation of Indoor and Outdoor Scenes from Visual Lifelogging

Early and accurate identification of brain tumors is essential for determining effective treatment strategies and improving patient outcomes. Artificial intelligence (AI) and deep learning (DL) techniques have shown promise in automating diagnostic tasks based on magnetic resonance imaging (MRI). This study evaluates the performance of four pre-trained deep convolutional neural network (CNN) architectures for the automatic multi-class classification of brain tumors into four categories: Glioma, Meningioma, Pituitary, and No Tumor. The proposed approach utilizes the publicly accessible Brain Tumor MRI Msoud dataset, consisting of 7023 images, with 5712 provided for training and 1311 for testing. To assess the impact of data availability, subsets containing 25%, 50%, 75%, and 100% of the training data were used. A stratified five-fold cross-validation technique was applied. The CNN architectures evaluated include DeiT3_base_patch16_224, Xception41, Inception_v4, and Swin_Tiny_Patch4_Window7_224, all fine-tuned using transfer learning. The training pipeline incorporated advanced preprocessing and image data augmentation techniques to enhance robustness and mitigate overfitting. Among the models tested, Swin_Tiny_Patch4_Window7_224 achieved the highest classification Accuracy of 99.24% on the test set using 75% of the training data. This model demonstrated superior generalization across all tumor classes and effectively addressed class imbalance issues. Furthermore, we deployed and benchmarked the best-performing DL model on embedded AI platforms (Jetson AGX Xavier and Orin Nano), demonstrating their capability for real-time inference and highlighting their feasibility for edge-based clinical deployment. The results highlight the strong potential of pre-trained deep CNN and transformer-based architectures in medical image analysis. The proposed approach provides a scalable and energy-efficient solution for automated brain tumor diagnosis, facilitating the integration of AI into clinical workflows.

Abstract. In recent years, there has been a significant improvement in the detection, identification and classification of objects and images using Convolutional Neural Networks. To study the potential of the Convolutional Neural Network, in this paper three approaches are investigated to train classifiers based on Convolutional Neural Networks. These approaches allow Convolutional Neural Networks to be trained on datasets containing only a few hundred training samples, which results in a successful classification. Two of these approaches are based on the concept of transfer learning. In the first approach features, created by a pretrained Convolutional Neural Network, are used for a classification using a support vector machine. In the second approach a pretrained Convolutional Neural Network gets fine-tuned on a different data set. The third approach includes the design and training for flat Convolutional Neural Networks from the scratch. The evaluation of the proposed approaches is based on a data set provided by the IEEE Geoscience and Remote Sensing Society (GRSS) which contains RGB and LiDAR data of an urban area. In this work it is shown that these Convolutional Neural Networks lead to classification results with high accuracy both on RGB and LiDAR data. Features which are derived by RGB data transferred into LiDAR data by transfer learning lead to better results in classification in contrast to RGB data. Using a neural network which contains fewer layers than common neural networks leads to the best classification results. In this framework, it can furthermore be shown that the practical application of LiDAR images results in a better data basis for classification of vehicles than the use of RGB images.

Purpose: The aim of the article is to develop an algorithm for classifying cracks in the analyzed images using modern methods of deep machine learning and transfer learning based on pretrained convolutional neural network - Inception-ResNet-v2. Design/Methodology/Approach: Transfer learning based on the pretrained convolutional neural network was used to categorize the images. The fully conected layer of the Inception-ResNet-v2 network has been modified. The last layer was trained using a two-class (binary) linear SVM (Support Vector Machine). In total, 20,000 training cases (images) were used to train the fully connected layer within transfer learning process. The research analyzed the possibility of using the deep neural networks for quick and fully automatic identification of cracks / defects on the surface of analyzed parts. Findings: The results indicate that pretrained convolutional neural network using SVM to train a fully connected layer is a very effective solution for visual crack / fault detection. In the analyzed model, a positive classification was obtained at the level of 99.89%. Practical Implications: The model presented in the article can be used in quality control carried out by process monitoring. An effective model for identifying defective parts can be used in both logistics and production processes. Originality/Value: A novelty is the use of a freely available, deep neural network trained to classify 1000 categories of various images for binary categorization of faults (cracks). The algorithm was adjusted by replacing the primary, 1000-output fully connected layer in the Inception-ResNet-v2 network with a binary layer (2 categories). The fully connected layer has been trained using the classification version of the popular SVM learner, but thanks to the combination of this layer with the sophisticated fearure extraction ability of the pre-trained Inception-ResNet-v2 deep network, the resulting predictive model enables the classification of defects with a very high level of accuracy.

Here we investigate the application of convolutional neural networks (CNNs) to predict the survival time of patients with Glioblastoma Multiforme (GBM) brain tumor. Our dataset consists of T1-weighted high-resolution MRI images of just 68 GBM patients. We compare two analytic methods for predicting survival time. The first consists of training a small convolutional neural network (CNN) and the second uses extracted deep features from a pre-trained CNN. Our method is completely automated, except for tumor region segmentation. In addition, we utilize a snapshot ensemble approach to boost test accuracy when dealing with limited availability of medical images for CNN training purposes. Our approach achieves an accuracy of 72.06% using a trained small network and 66.18% using a pre-trained deep CNN. Our results compare favorably with the accuracy of 54.41% using histogram of oriented gradients (HOG) features and a non-neural network classifier.

Arabic handwritten text recognition is a complex and challenging research domain. This study proposes an offline Arabic handwritten word recognition system based on transfer learning. The system exploits four pre-trained convolutional neural network (CNN) architectures, namely VGG16, ResNet50, AlexNet, and InceptionV3. In addition, a specialized image recognition model derived from the ImageNet dataset is incorporated. A combination strategy is designed to combine transfer learning with specific fine-tuning techniques, aiming to improve recognition accuracy. The study is conducted on the IFN/ENIT dataset, which includes images of Tunisian City and village names. The results show that the proposed system achieves a recognition accuracy of 94.73%, which is significantly higher than the accuracy rates achieved by previous approaches. These results suggest that the proposed system is a promising approach for Arabic handwritten text recognition.

Deep learning based methods have recently been successfully explored in hyperspectral image classification field. However, training a deep learning model still requires a large number of labeled samples, which is usually impractical in hyperspectral images. In this paper, a simple but effective feature extraction method is proposed for hyperspectral image classification. Specifically, a pretrained deep convolutional neural network based on the ImageNet dataset is used to extract spatial features of a hyperspectral image. Recently, it is easy to obtain a pretrained convolutional neural network on the Internet. Note that the pretrained models are trained by using the ImageNet dataset. This means that the proposed method does not need labeled hyperspectral samples to train the deep model. Therefore, the proposed method alleviates the problem of lacking labeled samples and avoids the artificial design of feature extraction rules. Finally, the extracted features are stacked with spectral features as the input of a support vector machine classifier. The proposed method is conducted on three widely used hyperspectral image datasets. The experimental results demonstrate that the proposed method could outperform the conventional feature extraction methods and deep learning based methods.

Recently Pretrained Convolutional Neural Networks (CNNs) have proven its effectiveness in image extraction and classification. This powerful feature of CNNs in image processing is facilitated by machine learning to train and classify big data. Image capturing and security transformation are considered as a central necessity of remote sensing imagery of unmanned aerial vehicles (UAVs) and drones. This paper presents a novel artificial intelligent drone-based encrypted machine learning of image classification using a pertained CNN and image encryption-decryption by utilizing singular value decomposition (SVD) and XOR-Secret-Key block cipher cryptology. Initially, pretrained convolutional neural networks (CNN) are extensively used to extract and classify image features making advantage of machine learning training tools features. Training of partial set of image data can be performed to test, classify and label the untrained image data. Pretrained CNN can classify images into object categories. Afterward, the CNN the classified image output is transformed into a digital matrix using SVD and identifies its associated eigenvalues. These eigenvalues are then converted into a binary code. The image data encryption is implemented according to suggested keys. The first part applies the exclusive OR (XOR) operation of the eigenvalues with a selected cipher key. Meanwhile, the second part implements the XOR operation of the output of part one with a randomly generated key using Poisson distribution. The last step in the encryption will be obtained by generating a non-real SVD decomposition matrix; according to which a non-readable image will be resulted. The original image-matrix can be constructed by reversing the process using the security key-cipher block (Poisson Distribution Key and Stand-alone Cipher Code). Finally, SVD image processing results are demonstrated to verify the effectiveness and security of the applied approach that can be implemented for different images.

Deep convolutional neural networks (CNNs) enable learning trainable, highly representative and hierarchical image feature from sufficient training data which makes rapid progress in computer vision possible. There are currently three major techniques that successfully employ CNNs to medical image classification: training the CNN from scratch, using off-the-shelf pretrained CNN features, and transfer learning , i.e., fine-tuning CNN models pretrained from natural image dataset (such as large-scale annotated natural image database: ImageNet) to medical image tasks. In this chapter, we exploit three important factors of employing deep convolutional neural networks to computer-aided detection problems. First, we exploit and evaluate several different CNN architectures including from shallower to deeper CNNs: classical CifarNet, to recent AlexNet and state-of-the-art GoogLeNet and their variants. The studied models contain five thousand to 160 million parameters and vary in the numbers of layers. Second, we explore the influence of dataset scales and spatial image context configurations on medical image classification performance. Third, when and why transfer learning from the pretrained ImageNet CNN models (via fine-tuning) can be useful for medical imaging tasks are carefully examined. We study two specific computer-aided detection (CADe) problems, namely thoracoabdominal lymph node (LN) detection and interstitial lung disease (ILD) classification. We achieve the state-of-the-art performance on the mediastinal LN detection and report the first fivefold cross-validation classification results on predicting axial CT slices with ILD categories. Our extensive quantitative evaluation, CNN model analysis, and empirical insights can be helpful to the design of high-performance CAD systems for other medical imaging tasks, without loss of generality.

The analysis of natural images has been the topic of research in uncountable articles in computer vision and pattern recognition (e.g., natural images has been used as benchmarks for object recognition and image retrieval). However, despite the research progress in such field, there is a gap in the analysis of certain type of natural images, for instance, those in the context of animal behavior. In fact, biologists perform the analysis of natural images manually without the aid of techniques that were supposedly developed for this purpose. In this context, this paper presents a study on automated methods for the analysis of natural images of hummingbirds with the goal to assist biologists in the study of animal behavior. The automated analysis of hummingbird behavior is challenging mainly because of (1) the speed at which these birds move and interact; (2) the unpredictability of their trajectories; and (3) its camouflage skills. We report a comparative study of two deep learning approaches for the detection of hummingbirds in their nest. Two variants of transfer learning from convolutional neural networks (CNNs) are evaluated in real imagery for hummingbird behavior analysis. Transfer learning is adopted because not enough images are available for training a CNN from scratch, besides, transfer learning is less time consuming. Experimental results are encouraging, as acceptable classification performance is achieved with CNN-based features. Interestingly, a pretrained CNN without fine tunning and a standard classifier performed better in the considered data set.

Stress has been identified as one of the contributing reasons to vehicle crashes, which cost governments and society a large amount of money in terms of lost lives and productivity. Any alteration that creates physical, emotional, or physiological strain when driving is referred to as driving stress. Driving stress may vary depending on the different road conditions of driving. Understanding drivers’ discontent is one of the most important areas for improving intelligent transportation systems over the existing system. This study presents methods for analyzing and classifying EMG data collected during real-world driving tasks at different driving locations using a convolutional neural network (CNN). In this paper, there are 9 subjects (driver records) of at least 60 minutes duration. Developing CNN from scratch is difficult and it also demands specialized knowledge. As it was previously trained on the ImageNet dataset and could operate effectively with the small amount of training set, pre-trained CNN minimizes the effort of developing models from scratch. CNN is employed in the proposed work to classify driving stress levels by evaluating discriminatory patterns in spectrogram images. In the proposed work, the performance of pre-trained CNN SqueezeNet, GoogLeNet, and ResNet50 in identifying the level of stress (low, medium, and high) is compared. GoogLeNet performed best, with an accuracy of training and validation of 85.71% and 66.67%. Followed by ResNet50 with an accuracy of 71.43% and 66.67% and SqueezeNet with an accuracy of 71.43% and 55.56%, respectively.

Learning transferable features in deep convolutional neural networks for diagnosing unseen machine conditions

Purpose: To determine whether domain transfer learning can improve the performance of deep features extracted from digital mammograms using a pre-trained deep convolutional neural network (CNN) in the prediction of occult invasive disease for patients with ductal carcinoma in situ (DCIS) on core needle biopsy. Method: In this study, we collected digital mammography magnification views for 140 patients with DCIS at biopsy, 35 of which were subsequently upstaged to invasive cancer. We utilized a deep CNN model that was pre-trained on two natural image data sets (ImageNet and DTD) and one mammographic data set (INbreast) as the feature extractor, hypothesizing that these data sets are increasingly more similar to our target task and will lead to better representations of deep features to describe DCIS lesions. Through a statistical pooling strategy, three sets of deep features were extracted using the CNNs at different levels of convolutional layers from the lesion areas. A logistic regression classifier was then trained to predict which tumors contain occult invasive disease. The generalization performance was assessed and compared using repeated random sub-sampling validation and receiver operating characteristic (ROC) curve analysis. Result: The best performance of deep features was from CNN model pre-trained on INbreast, and the proposed classifier using this set of deep features was able to achieve a median classification performance of ROC-AUC equal to 0.75, which is significantly better (p<=0.05) than the performance of deep features extracted using ImageNet data set (ROCAUC = 0.68). Conclusion: Transfer learning is helpful for learning a better representation of deep features, and improves the prediction of occult invasive disease in DCIS.

Building collapses lead to high mortality rate and other damages. UAVs attached with optical cameras are widely available and can be used for collapsed building detection as they can cover a larger area within a given time with minimal cost. This paper is a systematic study on utilization of deep learning based approaches for collapsed building detection on images collected by UAVs. This study aims to identify suitable deep learning models with focus on optimization for real world application scenarios. Five different pre-trained Convolutional Neural Networks (CNN) are trained using transfer learning on our custom data set to assess the performance. The real novelty in this study pertains to suitably enhancing the architecture of these models by increasing the depth and width of the neural network models on top of feature extraction and fine tuning approaches, without any substantial increase in the additional computational tasks. Another prominent contribution is the construction of a new drone-based building image data set with more emphasis on real world correlation between collapsed and normal buildings from disaster hit areas. The enhanced CNN architectures of the five models applied to this rich data set resulted in an average 6.67% increase in accuracy as compared to the general fine tuning approaches. Through deeper fine tuning approach, model training time was also reduced by 6 seconds in average with moderate increase in trained model size.

ObjectiveIn this paper, we propose to evaluate the use of pre-trained convolutional neural networks (CNNs) as a features extractor followed by the Principal Component Analysis (PCA) to find the best discriminant features to perform classification using support vector machine (SVM) algorithm for neonatal sleep and wake states using Fluke® facial video frames. Using pre-trained CNNs as a feature extractor would hugely reduce the effort of collecting new neonatal data for training a neural network which could be computationally expensive. The features are extracted after fully connected layers (FCL’s), where we compare several pre-trained CNNs, e.g., VGG16, VGG19, InceptionV3, GoogLeNet, ResNet, and AlexNet.ResultsFrom around 2-h Fluke® video recording of seven neonates, we achieved a modest classification performance with an accuracy, sensitivity, and specificity of 65.3%, 69.8%, 61.0%, respectively with AlexNet using Fluke® (RGB) video frames. This indicates that using a pre-trained model as a feature extractor could not fully suffice for highly reliable sleep and wake classification in neonates. Therefore, in future work a dedicated neural network trained on neonatal data or a transfer learning approach is required.

Scene classification plays a crucial role in the travel and tourism industry in promoting travel. In this paper, we propose a fusion approach based on representation learning to perform scene classification using a basic convolutional neural network (CNN) and multiple pre-trained CNNs as feature extractors. We have used the Oliva and Torralba Scene (OT-Scene) dataset consisting of scene images of eight different categories. In this proposed approach we extract image representations from deep learning models such as basic CNN and pre-trained CNN, namely VGG19, MobileNetV2, and RestNet50. We then fuse these representations from several models into different combinations to achieve a more generalized representation or feature vectors. The fused feature vectors and the individual feature vectors are then fed into the artificial neural network (ANN), support vector machine (SVM), and random forest (RF) classifiers separately to compare the results. The results showed the highest scene classification accuracy of 97.4% from specific fused combinations.