Deep Fake Video Detection using Convolutional Neural Networks

In recent months, advancements in free deep learning-based software tools have made it easier to create convincing face swaps in videos, often leaving minimal traces of manipulation. This phenomenon is commonly known as "DeepFake" (DF) videos. While manipulations of digital videos using visual effects have been demonstrated for decades, recent progress in deep learning has significantly enhanced the realism of fake content and the ease with which it can be generated. These artificially intelligent tools, commonly referred to as AI-synthesized media or DF, have simplified the creation process. However, detecting DeepFake videos poses a significant challenge. Despite the simplicity of generating DF using AI tools, training algorithms to identify them is not straightforward. To address this issue, we have taken a step forward by employing Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN) for detection. Our system utilizes a CNN to extract features at the frame level. These features are then employed to train an RNN, which learns to classify whether a video has undergone manipulation. The system is capable of detecting temporal inconsistencies between frames introduced by the tools used in DF creation. To evaluate the effectiveness of our approach, we tested it against a large set of fake videos collected from a standard dataset. The results demonstrate the competitiveness of our system, achieved through a simple architecture.

This paper proposes a novel deep learning architecture involving combinations of Convolutional Neural Networks (CNN) layers and Recurrent neural networks (RNN) layers that can be used to perform segmentation and classification of 5 cardiac rhythms based on ECG recordings. The algorithm is developed in a sequence to sequence setting where the input is a sequence of five second ECG signal sliding windows and the output is a sequence of cardiac rhythm labels. The novel architecture processes as input both the spectrograms of the ECG signal as well as the heartbeats' signal waveform. Additionally, we are able to train the model in the presence of label noise. The model's performance and generalizability is verified on an external database different from the one we used to train. Experimental result shows this approach can achieve an average F1 scores of 0.89 (averaged across 5 classes). The proposed model also achieves comparable classification performance to existing state-of-the-art approach with considerably less number of training parameters.

With the rapid development of the e-commerce industry, online reviews of goods are a great help for consumers to make decisions. With the sharp increase in online order for goods and the explosion of product reviews, some merchants began to hire consumers to make fake purchases for profit, which led to the problem of identifying fake reviews. In this paper, we propose a method that uses feature engineering to eliminate the comments of false reviewers and combines convolutional neural network and recurrent neural network to classify and recognise reviews from the perspective of text. Traditional neural network models such as CNN, LSTM and BILSTM are compared with the hybrid model proposed by the text. The model is optimised by pre-training on the Baidu Baike commodity review database instead of the initial randomising word vector. The experimental results show that the combination of convolutional neural network and recurrent neural network can better extract the global and local features of false comments, and the model has a good effect. The updating of the pre-trained word vector makes the recognition effect of each model better.

Following recent advancements in deep learning and artificial intelligence, spoken language identification applications are playing an increasingly significant role in our day-to-day lives, especially in the domain of multi-lingual speech recognition. In this article, we propose a spoken language identification system that depends on the sequence of feature vectors. The proposed system uses a hybrid Convolutional Recurrent Neural Network (CRNN), which combines a Convolutional Neural Network (CNN) with a Recurrent Neural Network (RNN) network, for spoken language identification on seven languages, including Arabic, chosen from subsets of the Mozilla Common Voice (MCV) corpus. The proposed system exploits the advantages of both CNN and RNN architectures to construct the CRNN architecture. At the feature extraction stage, it compares the Gammatone Cepstral Coefficient (GTCC) feature and Mel Frequency Cepstral Coefficient (MFCC) feature, as well as a combination of both. Finally, the speech signals were represented as frames and used as the input for the CRNN architecture. After conducting experiments, the results of the proposed system indicate higher performance with combined GTCC and MFCC features compared to GTCC or MFCC features used individually. The average accuracy of the proposed system was 92.81% in the best experiment for spoken language identification. Furthermore, the system can learn language-specific patterns in various filter size representations of speech files.

According to the World Health Organization (WHO), in 2023 an estimated 466 million people (5.3 % of the global population) experienced hearing loss, including 432 million adults and 34 million children. This condition creates significant barriers in daily communication between deaf individuals and hearing communities. Advances in deep learning have enabled automatic sign language recognition, with Long Short-Term Memory (LSTM) networks being among the most widely used models. However, previous studies suggest that Gated Recurrent Units (GRU) can offer faster computation with comparable or even superior accuracy. This study presents a comparative performance analysis of GRU, LSTM, and Simple Recurrent Neural Networks (RNN) for recognizing 10 BISINDO (Indonesian Sign Language) gesture classes. Models were developed using a combination of Convolutional Neural Networks (CNN) for feature extraction and recurrent architectures for sequence modeling, with Mediapipe employed for landmark detection. Experimental results show that the GRU achieved 97.92 % training accuracy, 0.2804 training loss, 99.86 % test accuracy, 99.37 % precision, 99.33 % recall, 99.33 % F1-score, 99.99 % ROC-AUC, and an inference time of 87.30 ms, outperforming LSTM and Simple RNN in speed without compromising accuracy.

The study presents the results of the development and testing of deep learning neural network architectures, which demonstrate high accuracy rates in classifying neurophysiological data, in particular hemodynamic brain activation patterns obtained by functional near-infrared spectroscopy, during solving mathematical problems on spatial-numerical associations. The analyzed signal represents a multidimensional time series of oxyhemoglobin and deoxyhemoglobin dynamics. Taking the specificity of the fNIRS signal into account, a comparative analysis of 2 types of neural network architectures was carried out: (1) architectures based on recurrent neural networks: recurrent neural network with long short-term memory, recurrent neural network with long short-term memory with fully connected layers, bidirectional recurrent neural network with long short-term memory, convolutional recurrent neural network with long short-term memory; (2) architectures based on convolutional neural networks with 1D convolutions: convolutional neural network, fully convolutional neural network, residual neural network. Trained long short-term memory recurrent neural network architectures showed worse results in accuracy in comparison with 1D convolutional neural network architectures. Residual neural network (model_Resnet) demonstrated the highest accuracy rates in three experimental conditions more than 88% in detecting age-related differences in brain activation during spatial-numerical association tasks considering the individual characteristics of the respondents’ signal.

In this paper, alternative direction finding methods have been proposed using deep learning techniques. Firstly, Regeression and Classification models have created by using Convolutional Neural Networks (CNNs). In the second Convolutional Neural Networks and Recurrent Neural Networks (RNNs) have been utilized in the proposed methods. Despite having vast amount of direction finding studies, utilization of neural networks is scarce in literature and past works mostly only includes usage of CNNs. In this study, direction finding is performed by learning signals reaching multiple antenna arrays by networks. Created neural networks have been fed with different data formats and their performances against noisy and no-noise data have been shown. In addition, comparative analysis of the developed models were made in the similar Signal-to-Noise Ratio (SNR) range with the subspace based MUSIC algorithm, which is frequently used in direction finding.

Deep learning (also known as deep structured learning, hierarchical learning, or deep machine learning) is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using a deep graph with multiple processing layers, composed of multiple linear and nonlinear transformations. Deep learning has been characterized as a class of machine learning algorithms with the following characteristics [257]: • They use a cascade of many layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. The algorithms may be supervised or unsupervised and applications include pattern analysis (unsupervised) and classification (supervised). • They are based on the (unsupervised) learning of multiple levels of features or representations of the data. Higher-level features are derived from lower-level features to form a hierarchical representation. • They are part of the broader machine learning field of learning representations of data. • They learn multiple levels of representations that correspond to different levels of abstraction; the levels form a hierarchy of concepts. These definitions have in common: multiple layers of nonlinear processing units and the supervised or unsupervised learning of feature representations in each layer, with the layers forming a hierarchy from low-level to high-level features. Various deep learning architectures such as deep neural networks, convolutional deep neural networks, deep belief networks (DBN), and recurrent neural networks have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition, and bioinformatics where they have been shown to produce state-of-the-art results on various tasks. In this chapter we start in Section 7.1 with an introduction, giving a brief history of this field, the relevant literature, and its applications. Then we study some basic concepts of deep learning such as convolutional neural networks, recurrent neural networks, backpropagation algorithm, restricted Boltzmann machines, and deep learning networks in Section 7.2. Then we illustrate three examples for Apache Spark implementation for mobile big data (MBD), user moving pattern extraction, and combination with nonparametric Bayesian learning, respectively, in Sections 7.3 through 7.5. Finally, we have summary in Section 7.6.

Computer-aided detection and diagnosis in ECG signals for heart diseases are gaining increasing attention. However, developing and selecting the highly performing diagnostic model suitable for clinical implications is still challenging. In this paper, we proposed a combined network of convolutional neural network (CNN) and Recurrent Neural Network (RNN), designed for the classification of ECG heart signals for diagnostic purpose. The proposed network consists of 2 convolutional layers with <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$5\times 5$ </tex-math></inline-formula> kernels and ReLU activations, followed by 4 residual blocks, 2 bidirectional long short-term memory (biLSTM) layers, as well as 2 fully connected layers. Each residual block involved the structure of a Squeeze-and-Excitation Network (SENet) with lightweight property to recalibrate the feature map of the network. The last dense layer has 5 outputs, equivalent to the classes considered: non-ectopic beats, supraventricular ectopic beats, ventricular ectopic beats, fusion beats, and unknown beats. To train and evaluate the combined CNN and RNN, we transferred the knowledge acquired on beat classification tasks in 2017 PhysioNet/CinC Challenge to that in PhysionNet’s MIT-BIH dataset. The developed network achieved a recognition sensitivity of 95.90%, accuracy = 95.90% and specificity = 96.34% with classification time of single sample = 6.23 s in detecting 5 ECG classes. A comparative analysis proved the high performance of the proposed combined CNN and RNN against previous methods, demonstrating the potential of our proposed network in the analysis of beat patterns. The proposed model can be applied in cloud computing or implemented in mobile devices to evaluate cardiac health with the highest precision.

The development of deep learning technology has allowed the emergence of the phenomenon of deepfakes, which is the manipulation of digital videos that resemble real videos with a high level of realism. These technologies pose serious threats to privacy, digital security, and the spread of false information. As the quality of deepfake videos increases, the detection of this fake content becomes increasingly challenging. This study aims to design and evaluate a deepfake video detection model using a combination of Convolutional Neural Network (CNN) and Gated Recurrent Unit (GRU). CNN is used to extract spatial features from each video frame, while GRU is used to capture the temporal relationships between frames. The dataset used is Celeb-DF(v2), which is a benchmark dataset that contains real videos and high-quality deepfake videos. The CNN-GRU model was trained and tested on the dataset, and its performance was evaluated using accuracy, precision, recall, and F1-score metrics.

1 - Congruence of deep learning in biomedical engineering: future prospects and challenges

Human Activity Recognition (HAR) based on data from wearable sensors has become an attractive research topic thanks to its applications in different fields such as healthcare and smart environments. Recently, the advancement of deep learning with capability to perform automatically high-level feature extraction has achieved promising results. However, the performance of the deep learning models depends deeply on the characteristics of the datasets such as the number of classes, the inter-similarity and intra-variation. Therefore, directly comparing these models has become difficult since a wide variety of experimental protocols, evaluation metrics, and datasets are employed. In this paper, for the first time, a systematic evaluation of several deep learning models for HAR from wearable sensors is provided. In particular, three models named Convolutional Neural Network (CNN) [1], DeepConvLSTM - a combination of CNN and Long Short Term Memory (LSTM) [2], and SensCapsNet - a Capsule Neural Network for wearable sensor-based HAR [3] were implemented and evaluated on three benchmark datasets that are 19NonSens, CMDFall, and UCI-HAR dataset. Moreover, to have an intuitive explanation of deep learning models, a visualization of features learnt from these models is given. The evaluation codebase and results will be made publicly available for community use.

Image processing can be used to extract meaningful optical results from images. Here, from images of plasmonic structures, we combined convolutional neural networks with recurrent neural networks to extract the absorption spectra of structures. To provide the data required for the model, we performed 100,000 simulations with similar setups and random structures. In designing this deep network, we created a model that can predict the absorption response of any structure with a similar setup. We used convolutional neural networks to collect spatial information from the images, and then, we used that data and recurrent neural networks to teach the model to predict the relationship between the spatial information and the absorption spectrum. Our results show that this image processing method is accurate and can be used to replace time- and computationally-intensive numerical simulations. The trained model can predict the optical results in less than a second without the need for a strong computing system. This technique can be easily extended to cover different structures and extract any other optical properties.

An efficient approach for lane marking detection and classification by the combination of convolution neural network and recurrent neural network is proposed in this paper. First, convolution neural network is trained for lane marking features extraction, and then these convolution neural network features of continuous frames are transferred to recurrent neural network model for lane boundary detection and classification in the time domain. At last, a lane boundary fitting method based on dynamic programming is proposed to improve the lane detection accuracy and robustness. The method presented generates satisfactory results of lane detection and type classification under various traffic conditions according to the experimental results, which show that the approach provided in this paper outperforms traditional methods and the total lane markings classification reached 95.21% accuracy.

Deep fakes, synthetic media generated through artificial intelligence (AI) techniques, have become a growing concern due to their potential for misinformation, manipulation, and security threats. As the sophistication of deep fake technology continues to evolve, traditional methods of detection are becoming less effective, thus prompting the need for advanced AI-driven solutions. This article explores the various AI techniques employed in deep fake detection, such as deep learning, neural networks, and computer vision algorithms. A review of the latest advancements in AI technologies, including Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and Generative Adversarial Networks (GANs), is presented to highlight their roles in identifying deep fakes. Additionally, we explore challenges related to data availability, model robustness, and ethical implications. The paper concludes by proposing a hybrid detection framework that integrates multiple AI techniques to enhance the accuracy and efficiency of deep fake identification.