Analysis of Image Captioning Approaches from a Deep Learning Perspective

With the recent increase in the number of news videos and in their value, their automatic indexing has been anticipated. Among various proposals for this problem, the use of textual data has been actively considered, and some applications of this type have actually been developed. Nonetheless, most of these approaches are textual data oriented, and a few of them consider the correspondences between their indexes (keywords) and image contents which are essentials for an image database.This paper proposes an automatic video indexing method which takes advantage of the characteristic of news videos: similar images have similar contents. The method tags indexes selectively according to typical image content. The method has been applied to actual news videos, and indexing has been successful for between 25% and 93% of “typical shots,” and for between 75% and 100% of “captionized shots.” © 2001 Scripta Technica, Syst Comp Jpn, 32(9): 32–41, 2001

The aim of this work is to distinguish between wild-type mice and Scn5a +/- mutant mice using short ECG signals. This mutation results in impaired cardiac sodium channel function and is associated with increased ventricular arrhythmogenic risk which can result in sudden cardiac death. Lead I and Lead II ECG signals from wild-type and Scn5a +/- mice are used and the mice are also grouped as female/male and young/old.We use our novel Symmetric Projection Attractor Reconstruction (SPAR) method to generate an attractor from the ECG signal using all of the available waveform data. We have previously manually extracted a variety of quantitative measures from the attractor and used machine learning to classify each animal as either wild-type or mutant. In this work, we take the attractor images and use these as input to a deep learning algorithm in order to perform the same classification. As there is only data available from 42 mice, we use a transfer learning approach in which a network that has been pretrained on millions of images is used as a starting point and the last few layers are changed in order to fine tune the network for the attractor images.The results for the transfer learning approach are not as good as for the manual features, which is not too surprising as the networks have not been trained on attractor images. However, this approach shows the potential for using deep learning for classification of attractor images.

To achieve more effective solution for large-scale image classification (i.e., classifying millions of images into thousands or even tens of thousands of object classes or categories), a deep multi-task learning algorithm is developed by seamlessly integrating deep CNNs with multi-task learning over the concept ontology, where the concept ontology is used to organize large numbers of object classes or categories hierarchically and determine the inter-related learning tasks automatically. Our deep multi-task learning algorithm can integrate the deep CNNs to learn more discriminative high-level features for image representation, and it can also leverage multi-task learning and inter-level relationship constraint to train more discriminative tree classifier over the concept ontology and control the inter-level error propagation effectively. In our deep multi-task learning algorithm, we can use back propagation to simultaneously refine both the relevant node classifiers (at different levels of the concept ontology) and the deep CNNs according to a joint objective function. The experimental results have demonstrated that our deep multi-task learning algorithm can achieve very competitive results on both the accuracy and the cost of feature extraction for large-scale image classification.

Parallel encoder–decoder framework for image captioning

Deep Learning (DL) techniques for Natural Language Processing have been evolving remarkably fast. Recently, the DL advances in language modeling, machine translation, and paragraph understanding are so prominent that the potential of DL in Software Engineering cannot be overlooked, especially in the field of program learning. To facilitate further research and applications of DL in this field, we provide a comprehensive review to categorize and investigate existing DL methods for source code modeling and generation. To address the limitations of the traditional source code models, we formulate common program learning tasks under an encoder-decoder framework. After that, we introduce recent DL mechanisms suitable to solve such problems. Then, we present the state-of-the-art practices and discuss their challenges with some recommendations for practitioners and researchers as well.

This study delves into the potential of utilizing deep learning (DL) techniques to analyze qualitative news dissemination for trading purposes. DL, renowned for its prowess in handling vast datasets and deciphering intricate patterns, holds promise in aiding investors seeking to enhance their trading strategies. Specifically, Long Short-Term Memory (LSTM) networks, known for their capacity to retain contextual information, are explored in this research. By employing DL models, we aim to forecast market sentiment based on news headlines, focusing on the Dow Jones industrial average from 2008 to 2020. Leveraging 25 daily news headlines, we extend our analysis to develop an algorithmic trading strategy. Through rigorous testing across two distinct cases over five-time steps, our study evaluates the effectiveness of DL-driven approaches in real-world trading scenarios. Furthermore, this research contributes to the growing body of literature on the intersection of deep learning and financial markets. By examining the application of DL in qualitative news analysis for trading purposes, we provide insights into the potential implications for investors and financial analysts. The findings offer valuable guidance for leveraging advanced computational techniques for decision-making in dynamic market environments. Additionally, this study underscores the importance of incorporating qualitative news data into trading strategies, highlighting the role of DL in extracting meaningful signals from unstructured textual information’s Overall, our findings shed light on the opportunities and challenges associated with harnessing DL for trading on news sentiment.

Simulation‐driven deep learning for rebar clutter elimination in ground‐penetrating radar images to detect backfill grout defects in segment linings

Deep learning algorithms can be used to design systems that report data on patients and deliver warnings to medical applications or electronic health information if there are changes in the patient's health. These systems could be created using deep Learning. This helps verify that patients get the proper effective care at the proper time for each specific patient. A healthcare decision support system was presented using the Internet of Things and deep learning methods. In the proposed system, we examined the capability of integrating deep learning technology into automatic diagnosis and IoT capabilities for faster message exchange over the Internet. We have selected the suitable Neural Network structure (number of best-hidden layers and activation function classes) to construct the e-health system. In addition, the e-health system relied on data from doctors to understand the Neural Network. In the validation method, the total evaluation of the proposed healthcare system for diagnostics provides dependability under various patient conditions. Based on evaluation and simulation findings, a dual hidden layer of feed-forward NN and its neurons store the tanh function more effectively than other NN. To overcome challenges, this study will integrate artificial intelligence with IoT. This study aims to determine the NN's optimal layer counts and activation function variations.

Text categorization is an effective activity that can be accomplished using a variety of classification algorithms. In machine learning, the classifier is built by learning the features of categories from a set of preset training data. Similarly, deep learning offers enormous benefits for text classification since they execute highly accurately with lower-level engineering and processing. This paper employs machine and deep learning techniques to classify textual data. Textual data contains much useless information that must be pre-processed. We clean the data, impute missing values, and eliminate the repeated columns. Next, we employ machine learning algorithms: logistic regression, random forest, K-nearest neighbors (KNN), and deep learning algorithms: long short-term memory (LSTM), artificial neural network (ANN), and gated recurrent unit (GRU) for classification. Results reveal that LSTM achieves 92% accuracy outperforming all other model and baseline studies.

Visual place recognition: A survey from deep learning perspective

Scene text recognition (STR) technology has a rapid development with the rise of deep learning. Recently, the encoder-decoder framework based on attention mechanism is widely used in STR for better recognition. However, the commonly used Long Short Term Memory (LSTM) network in the framework tends to ignore certain position or visual information. To address this problem, we propose a Position Information Enhanced Encoder-Decoder (PIEED) framework for scene text recognition, in which an addition position information enhancement (PIE) module is proposed to compensate the shortage of the LSTM network. Our module tends to retain more position information in the feature sequence, as well as the context information extracted by the LSTM network, which is helpful to improve the recognition accuracy of the text without context. Besides that, our fusion decoder can make full use of the output of the proposed module and the LSTM network, so as to independently learn and preserve useful features, which is helpful to improve the recognition accuracy while not increase the number of arguments. Our overall framework can be trained end-to-end only using images and ground truth. Extensive experiments on several benchmark datasets demonstrate that our proposed framework surpass state-of-the-art ones on both regular and irregular text recognition.

Sentiment analysis of news headlines is an important factor that investors consider when making investing decisions. We claim that the sentiment analysis of financial news headlines impacts stock market values. Hence financial news headline data are collected along with the stock market investment data for a period of time. Using Valence Aware Dictionary and Sentiment Reasoning (VADER) for sentiment analysis, the correlation between the stock market values and sentiments in news headlines is established. In our experiments, the data on stock market prices are collected from Yahoo Finance and Kaggle. Financial news headlines are collected from the Wall Street Journal, Washington Post, and Business-Standard website. To cope with such a massive volume of data and extract useful information, various embedding methods, such as Bag-of-words (BoW) and Term Frequency-Inverse Document Frequency (TF-IDF), are employed. These are then fed into machine learning models such as Naive Bayes and XGBoost as well as deep learning models such as Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM). Various natural language processing, andmachine and deep learning algorithms are considered in our study to achieve the desired outcomes and to attain superior accuracy than the current state-of-the-art. Our experimental study has shown that CNN (80.86%) and LSTM (84%) are the best performing models in relation to machine learning models, such as Support Vector Machine (SVM) (50.3%), Random Forest (67.93%), and Naive Bayes (59.79%). Moreover, two novel methods, BERT and RoBERTa, were applied with the expectation of better performance than all the other models, and they did exceptionally well by achieving an accuracy of 90% and 88%, respectively.

Predicting benefit from immune checkpoint inhibitors in patients with non-small-cell lung cancer by CT-based ensemble deep learning: a retrospective study

The diagnosis and classification of oral diseases are vital to overall health, yet traditional methods depend mainly on visual inspection, often ignoring the valuable context from textual symptom descriptions. While deep learning has advanced medical image analysis, most models are unimodal, processing only visual data. To address this gap, we propose VIT-MEDBERT, a multimodal transformer-based framework designed to classify oral diseases using both visual and textual data. Our approach uses a Vision Transformer (ViT) for processing oral cavity images and MEDBERT, a pre-trained language model, for interpreting symptom descriptions. These two modalities are fused to form a comprehensive representation for classification. A key contribution of this work is the introduction of OralSDv1, a custom-curated, expert-verified multi-modal dataset comprising 1163 paired samples of oral images and their corresponding symptom descriptions across six common oral conditions. Our experiments demonstrate that the VIT-MEDBERT model achieves superior performance compared to state-of-the-art single-modality models, attaining an accuracy of 90.0%, precision of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$9 0. 0 \%$</tex>, recall of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$8 9. 0 \%$</tex>, and an F1-score of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$8 9. 0 \%$</tex>. The results highlight the value of integrating textual symptom data with visual features, which offers enhanced robustness and interpretability in the classification of oral diseases.

The freshwater resource is invaluable and indispensable for any nation like the Republic of Korea. Recently, deep learning (DL), AI models have become more popular and applied frequently for surface water studies. The Segment Anything Model (SAM) has been developing sharply and takes an adaptable approach with the ability to perform zero-shot inference. Although, pre-trained SAM was trained with millions of images (a billion masks), applying it to remote sensing data reveals limitations of inaccurate results and unlabeled classes, particularly for the more complex and noise data from Synthetic-aperture Radar Images. Hence, we fine-tune the SAM model and other popular CNN models of YOLOv8, U-net(ResNet50), and DeepLab(ResNet50, EfficientNet) for lake semantic segmentation using multi-SAR RS datasets of Kompsat-5, ALOS-2, Sentinel-1, and a combination of the three datasets (data link) for model result comparisons. This study’s accuracy assessment showed the SAM was the most precise model (accuracy overall <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\approx ~0.95$ </tex-math></inline-formula>) followed by the DeepLab(ResNet50), YOLOv8, U-net(ResNet50), and DeepLab(EfficientNet) model. Almost all models segmented highly accurate lake areas fitted well with ground-truth masks, nevertheless, the SAM (code link) presented the most well-performance model. The YOLO deals well with larger datasets and requires deeper trains to gain higher accuracy outputs. In addition, this research investigated the responses of each DL model to the SAR dataset proving a need for fine-tuning model results on SAR RS for better lake segmentations.