Ensemble Transfer Learning Midcourse Guidance Algorithm for Velocity Maximization

Automated detection of leukemia by pretrained deep neural networks and transfer learning: A comparison

Common acoustic properties that different classes share degrades the performance of acoustic scene classification systems. This results in a phenomenon where a few confusing pairs of acoustic scenes dominate a significant proportion of all misclassified audio segments. In this article, we propose adopting a knowledge distillation framework that trains deep neural networks using soft labels. Soft labels, extracted from another pre-trained deep neural network, are used to reflect the similarity between different classes that share similar acoustic properties. We also propose utilizing specialist models to provide additional soft labels. Each specialist model in this study refers to a deep neural network that concentrates on discriminating a single pair of acoustic scenes that are frequently misclassified. Self multi-head attention is explored for training specialist deep neural networks to further concentrate on target pairs of classes. The goal of this article is to train a single deep neural network that demonstrates performance equivalent to, or higher than, an ensemble of multiple models, by distilling the knowledge from several models. Diverse experiments conducted using the detection and classification of acoustic scenes and events 2019 task 1-a dataset demonstrate that the knowledge distillation framework is effective in acoustic scene classification. Specialist models successfully decrease the number of misclassified audio segments in the target classes. The final single model with the proposed method that is trained by the proposed knowledge distillation from several models, including specialists trained using an attention mechanism, shows a classification accuracy of 77.63 %, higher than an ensemble of the baseline and multiple specialists.

Recently, pre-trained deep neural networks (DNNs) have outperformed traditional acoustic models based on Gaussian mixture models (GMMs) on a variety of large vocabulary speech recognition benchmarks. Deep neural nets have also achieved excellent results on various computer vision tasks using a random “dropout” procedure that drastically improves generalization error by randomly omitting a fraction of the hidden units in all layers. Since dropout helps avoid over-fitting, it has also been successful on a small-scale phone recognition task using larger neural nets. However, training deep neural net acoustic models for large vocabulary speech recognition takes a very long time and dropout is likely to only increase training time. Neural networks with rectified linear unit (ReLU) non-linearities have been highly successful for computer vision tasks and proved faster to train than standard sigmoid units, sometimes also improving discriminative performance. In this work, we show on a 50-hour English Broadcast News task that modified deep neural networks using ReLUs trained with dropout during frame level training provide an 4.2% relative improvement over a DNN trained with sigmoid units, and a 14.4% relative improvement over a strong GMM/HMM system. We were able to obtain our results with minimal human hyper-parameter tuning using publicly available Bayesian optimization code.

The unmanned aerial vehicles (UAVs) have been extensively used in providing intelligence such as target tracking. In our field experiments, a pre-trained deep neural network (DNN) is deployed on the UAV to identify a target from the captured video frames and enable UAV to keep tracking. However, tracking in real time by the DNN requires a lot of computational resources. This motivates us to consider offloading this type of machine learning (ML) tasks to a mobile edge computing (MEC) server. Specifically, we propose a novel hierarchical ML tasks distribution framework for the UAV tracking system, where the UAV is embedded with lower layers of the pre-trained convolutional neural network (CNN) model due to its limited computing capability, while the MEC server with rich computing resources will handle the higher layers of the CNN model. An optimization problem is formulated to minimize the CNN inference delay while taking into account the communications delay, computing time, and ML error. Insights are provided to understand the tradeoff between communications and ML computing in offloading decisions. Numerical results demonstrate the effectiveness of the proposed ML tasks distribution framework with the optimized offloading strategy.

Traditional Image Processing Techniques (IPT), used for automating the detection and classification of weld defects from radiography images, have their own limitations, which can be overcome by Deep Neural Networks (DNN). DNN produces considerably good results in fields which offer big dataset for it to train. DNN trained with small datasets by conventional methods produces less accurate results. This limits the use of DNN in many fields. This study focuses to overcome this limitation, by adopting transfer learning using Pre-trained deep convolutional neural networks. By this method, a weld defect radiographic image classifier, which can classify 14 types of weld defects, was constructed. 940 Image patches of weld defects were manually collected and labelled from GDXray database. Subsequently the features of this weld defect dataset were extracted using VGG16 and ResNet50 CNNs, both pre-trained on the ImageNet database. Then machine learning models such as Logistic Regression, Support Vector Machine (SVM) and Random Forest were trained on these extracted features. The Classifier based on SVM trained on features extracted by ResNet50 outperforms the other counter parts with an accuracy of 98%. In all these cases, transfer learning improves performance and reduces the training time and computational system requirements.

The contrast sensitivity function (CSF) is a fundamental signature of the visual system and has been measured extensively in numerous species. It is defined by measurements of the visibility threshold for sinusoidal gratings at all spatial frequencies. Here, we measured the CSF in artificial visual systems, namely deep neural networks (DNNs), using the same contrast discrimination paradigm as used in human psychophysical experiments. During training, networks are exclusively exposed to natural images, and the task is to identify which one of two input images has higher contrast. The contrast discriminator networks learn a linear classifier over frozen features of pretrained DNNs. At the testing stage, we measured the network's CSF by presenting sinusoidal gratings of different orientations and spatial frequencies. Our results demonstrate that the pretrained DNNs show the band-limited, inverted U-shaped CSF, that is characteristic of the human CSF. The exact shape of the DNNs' CSF appears to be task-dependent. The human CSF is better captured by scene segmentation DNNs than image classification ones (tested under identical settings such as the architecture and computational complexity). When the network was trained to discriminate the contrast in natural images from scratch (i.e., with no previous training task), a relatively flat CSF emerged, dissimilar to the human CSF. The environment of pretrained DNNs proved influential as well. Those with a diet of natural images, similar to human experience (e.g. objects or faces), obtain more human-like CSF. Pretrained DNNs with aerial pictures, such as a bird flying in the sky would see, obtain a shifted CSF towards higher spatial frequencies, similar to that of an eagle. In conclusion, the CSFs derived from the DNNs are the result of efficient processing of the natural world around us. Different visual environments have succinct effects on the shape of the CSFs.

Ultrasound computed tomography (USCT) receives increasing attention because of its capability to reconstruct quantitative information about the material property distribution as images with superior resolution. However, one roadblock for the wide adoption of relevant techniques is the high demand for computational resources and the long processing time for solving a large inverse problem in imaging. To alleviate the associated challenges, a two-stage inversion scheme is proposed: 1) the ultrasound scanning signals are first processed using a full waveform inversion (FWI) technique with a single iteration to rapidly create a model (image) with embedded wave speed distribution; 2) the corresponding image will be further improved by feeding into a pre-trained deep neural network. The deep learning models presented in this paper are built upon two architectures to instantaneously solve the associated inverse problems and to produce a high-resolution image in real-time. The first is based on 1D convolutional neural network (1D-CNN) layers with an autoencoder structure. The second implements additional layers and skip connections inspired by a U-Net architecture. The resultant superior reconstructions from both CNNs demonstrate that the proposed framework produces a high-resolution image from a rapidly-generated, low-resolution image in real-time, with dramatically improved results.

Gastric cancer is a significant health concern in East Asia, where early risk prediction is critical for prevention. However, the scarcity of single-hospital electronic health records (EHRs) data limits the applicability and generalizability of machine learning models. To address this challenge, we propose an ensemble transfer learning framework for gastric cancer prediction using structured EHRs in a data-scarce single-hospital setting. Three base models, Support Vector Machine (SVM), Random Forest, and Deep Neural Network (DNN), were pretrained on a large-scale national dataset from the Republic of Korean National Health Insurance Service (NHIS) and fine-tuned on a smaller institutional dataset from Kangdong Sacred Heart Hospital (KSHH). These fine-tuned models were combined via stacking ensemble learning with logistic regression as a meta-learner. The proposed model achieved strong performance with precision 0.78, recall 0.92, F1-score 0.83, accuracy 0.91, and AUC 0.93. For interpretability, permutation feature importance and Shapley Additive Explanations (SHAP) were applied. Smoking status, gender, and hypertensive disorder were identified as key predictors consistent with previous studies. This study demonstrates the successful application of transfer learning to overcome data scarcity in single-hospital structured EHRs. Furthermore, our stacking ensemble strategy outperformed the individual fine-tuned models, offering a generalizable framework for gastric cancer prediction in data-scarce clinical settings.

Automated image classification is an essential task of the computer vision field. The tagging of images into a set of predefined groups is referred to as image classification. The implementation of computer vision to automate image classification would be beneficial because manual image evaluation and identification can be time-consuming, particularly when there are many images of different classes. Deep learning approaches are proven to overperform existing machine learning techniques in many fields in recent years, and computer vision is one of the most notable examples. The very deep neural network (VDCNN) is a powerful deep learning model for image classification, and this paper examines it briefly using MNIST handwritten digit dataset. This dataset is used to prove the efficacy of very deep neural networks over other deep learning models. The proposed study aims to comprehend the very deep neural network architecture used to accomplish a handwritten digit recognition task. The feasibility of the proposed model is evaluated using mean accuracy, validation accuracy, and standard deviation. The study results of the very deep neural network model are compared to a convolutional neural network and convolutional neural network with batch normalization. According to the results of the comparison study, very deep neural networks achieve high accuracy of 99.1% for a handwritten dataset. The outcome of the proposed work is used to interpret how well a very deep neural network performs when compared to the other two models of deep neural networks. This proposed architecture may be used to automate the classification of handwritten digits dataset.

BackgroundDetection of ossification areas of hand bones in X-ray images is an important task, e.g. as a preprocessing step in automated bone age estimation. Deep neural networks have emerged recently as de facto standard detection methods, but their drawback is the need of large annotated datasets. Finetuning pre-trained networks is a viable alternative, but it is not clear a priori if training with small annotated datasets will be successful, as it depends on the problem at hand. In this paper, we show that pre-trained networks can be utilized to produce an effective detector of ossification areas in pediatric X-ray images of hands.Methods and findingsA publicly available Faster R-CNN network, pre-trained on the COCO dataset, was utilized and finetuned with 240 manually annotated radiographs from the RSNA Pediatric Bone Age Challenge, which comprises over 14.000 pediatric radiographs. The validation is done on another 89 radiographs from the dataset and the performance is measured by Intersection-over-Union (IoU). To understand the effect of the data size on the pre-trained network, subsampling was applied to the training data and the training was repeated. Additionally, the network was trained from scratch without any pre-trained weights. Finally, to understand whether the trained model could be useful, we compared the inference of the network to an annotation of an expert radiologist. The finetuned network was able to achieve an average precision (mAP@0.5IoU) of 92.92 ± 1.93. Apart from the wrist region, all ossification areas were able to benefit from more data. In contrast, the network trained from scratch was not able to produce any correct results. When compared to the annotations of the expert radiologist, the network was able to localize the regions quite well, as the F1-Score was on average 91.85 ± 1.06.ConclusionsBy finetuning a pre-trained deep neural network, with 240 annotated radiographs, we were able to successfully detect ossification areas in prediatric hand radiographs.

This research aims to evaluate the performance of neural networks in enhancing medical images and to explore the possibility of applying them in improving the quality of medical images for tumors, thereby enhancing diagnostic accuracy and treatment effectiveness. The study utilized a medical image of the lumbar spine MRI containing a tumor, to which Gaussian noise was added at levels of (0.01, 0.05, 0.10, 0.20). Noise removal techniques were applied using three linear filters: Mean Filter, Median Filter, and Gaussian Filter, in addition to the pre-trained deep neural network (DnCNN) for noise removal and image quality enhancement. To achieve these objectives, the deep learning tool library in MATLAB was used. Two evaluation methods were employed: the first was the visual inspection of the enhanced images, and the second included image quality improvement metrics such as PSNR, SSIM, and MSE. The results showed that the neural network (DnCNN) was able to improve the quality of medical images better than the other mentioned methods, achieving values of (66.7711, 0.6886, 0.0013) for (PSNR, SSIM, MSE) respectively at the low noise level of 0.01. It was observed that all the methods used faced challenges at higher noise levels. Thus, the results indicate that deep neural networks are effective in improving medical images and can be applied to enhance images related to spinal tumors, contributing to increased diagnostic accuracy and faster tumor detection, thereby improving healthcare. Keywords: Deep Neural Networks, Medical Images, Improving, DnCNN, Denoised تطبيقات الشبكات العصبية العميقة في تحسين الصور الطبية للأورام الشوكية الملخص يهدف هذا البحث إلى تقييم أداء الشبكات العصبية في تحسين الصور الطبية، واستكشاف إمكانية تطبيقها في مجال تحسين جودة الصور الطبية للأورام، مما يعزز دقة التشخيص وفاعلية العلاج، استخدمت الدراسة صورة طبية للعمود الفقري من النوع MRI ) ) في المنطقة القطنية تحتوي على ورم، حيث تم إضافة ضوضاء جاوس إلى الصورة بنسب ( 0.01، 0.05، 0.10، 0.20) تم استخدام تقنيات إزالة الضوضاء بواسطة ثلاثة فلاتر خطية، وهي Mean Filter وMedian Filter و Gaussian Filter ، بالإضافة إلى الشبكة العصبية العميقة ( DnCNN ) المدربة مسبقًا لإزالة الضوضاء وتحسين جودة الصور. لتحقيق هذه الأهداف، تم استخدام مكتبة أدوات التعلم العميق في MATLAB. كما تم استخدام طريقتين للتقييم: الأولى هي المعاينة البصرية للصور الناتجة عن التحسين، والثانية تشمل مقاييس تحسين جودة الصور مثل PSNR و SSIM و MSE . أظهرت النتائج أن الشبكة العصبية (DnCNN) قادرة على تحسين جودة الصور الطبية بشكل افضل من الطرق المذكورة الاخرى حيت حققت (66.7711 ، 0.6886 ، 0.0013 ) لكلا من (PSNR ،SSIM ، MSE ) على التوالي و عند نسبة ضوضاء المنخفظة 0.01)) . ولوحظ أن جميع الطرق المستخدمة تواجه تحديات عند مستويات الضوضاء المرتفعة. وبذلك تشير النتائج إلى أن الشبكات العصبية العميقة فعالة في تحسين الصور الطبية، وقابلة للتطبيق في تحسين الصور الخاصة بأورام العمود الفقري، مما يسهم في زيادة دقة التشخيص وسرعة اكتشاف الأورام، وبالتالي تحسين الرعاية الصحية. الكلمات المفتاحية : الشبكات العصبية العميقة ، الصور الطبية ، تحسين، DnCNN، ازالة الضوضاء .

Video hyperlinking represents a classical example of multimodal problems. Common approaches to such problems are early fusion of the initial modalities and crossmodal translation from one modality to the other. Recently, deep neural networks, especially deep autoencoders, have proven promising both for crossmodal translation and for early fusion via multimodal embedding. A particular architecture, bidirectional symmetrical deep neural networks, have been proven to yield improved multimodal embeddings over classical autoencoders, while also being able to perform crossmodal translation. In this work, we focus firstly at evaluating good single-modal continuous representations both for textual and for visual information. Word2Vec and paragraph vectors are evaluated for representing collections of words, such as parts of automatic transcripts and multiple visual concepts, while different deep convolutional neural networks are evaluated for directly embedding visual information, avoiding the creation of visual concepts. Secondly, we evaluate methods for multimodal fusion and crossmodal translation, with different single-modal pairs, in the task of video hyperlinking. Bidirectional (symmetrical) deep neural networks were shown to successfully tackle downsides of multimodal autoencoders and yield a superior multimodal representation. In this work, we extensively tests them in different settings, with different single-modal representations, within the context of video-hyperlinking. Our novel bidirectional symmetrical deep neural networks are compared to classical autoencoders and are shown to yield significantly improved multimodal embeddings that significantly (alpha=0.0001) outperform multimodal embeddings obtained by deep autoencoders with an absolute improvement in precision at 10 of 14.1% when embedding visual concepts and automatic transcripts and an absolute improvement of 4.3% when embedding automatic transcripts with features obtained with very deep convolutional neural networks, yielding 80% of precision at 10.

Existing agriculture cannot maximize crop production and selection due to complex interactions among environmental, management, and geographical factors. This paper presents a web application that forecasts crop yields and assesses crop suitability from structured agricultural data. The system provides two predictive modules: a Random Forest Regressor for forecasting crop production from ICRISAT historical data, and a pre-trained Deep Neural Network (DNN) for classifying crop suitability among Tamil Nadu districts from environmental and geographic inputs. An agent-based Explainable AI (XAI) workflow, powered by Large Language Models (Google Gemini and Groq’s Llama 3.1 via the phi-agent framework), automates the Random Forest pipeline—handling training, evaluation (MAE, R²), and feature importance extraction—while generating transparent, user-friendly explanations. DNN predictions are visualized on an interactive Folium map with accompanying LLM-generated explanations. This twin strategy offers both predictive output and actionable insights, making advanced analytics accessible to farmers and advisors via a Streamlit interface. The methodology and implementation show the system’s value for real-world agricultural decision support.

Typical deep neural networks (DNNs) are complex black-box models and their decision making process can be difficult to comprehend even for experienced machine learning practitioners. Therefore their use could be limited in mission-critical scenarios despite state-of-the-art performance on many challenging ML tasks. Through this work, we empower users to interpret DNNs with a post-hoc analysis protocol. We propose ProtoFac, an explainable matrix factorization technique that decomposes the latent representations at any selected layer in a pre-trained DNN as a collection of weighted prototypes, which are a small number of exemplars extracted from the original data (e.g. image patches, shapelets). Using the factorized weights and prototypes we build a surrogate model for interpretation by replacing the corresponding layer in the neural network. We identify a number of desired properties of ProtoFac including authenticity, interpretability, simplicity and propose the optimization objective and training procedure accordingly. The method is model-agnostic and can be applied to DNNs with varying architectures. It goes beyond per-sample feature-based explanation by providing prototypes as a condensed set of evidences used by the model for decision making. We applied ProtoFac to interpret pretrained DNNs for a variety of ML tasks including time series classification on electrocardiograms, and image classification. The result shows that ProtoFac is able to extract meaningful prototypes to explain the models' decisions while truthfully reflects the models' operation. We also evaluated human interpretability through Amazon Mechanical Turk (MTurk), showing that ProtoFac is able to produce interpretable and user-friendly explanations.

This paper is devoted to the efficiency analysis of detectors for the recognition of digits, which are mechanically stamped by a special machine on a steel cast billet. Such detectors are based on pre-trained deep neural networks. In the study, we analyze the performance of four different Faster R-CNN-based detectors. These neural networks have been trained and tested on our own training dataset obtained from the electro-metallurgical plant. According to the experiments, the best results are achieved by the Faster-RCNN Inception-Resnet v2 neural network detector. Its accuracy is about 98% on the test set.