ProFake: Detecting Deepfakes in the Wild against Quality Degradation with Progressive Quality-adaptive Learning

AI-synthesized face-swapping videos, commonly known as DeepFakes, is an emerging problem threatening the trustworthiness of online information. The need to develop and evaluate DeepFake detection algorithms calls for large-scale datasets. However, current DeepFake datasets suffer from low visual quality and do not resemble DeepFake videos circulated on the Internet. We present a new large-scale challenging DeepFake video dataset, Celeb-DF, which contains 5, 639 high-quality DeepFake videos of celebrities generated using an improved synthesis process. We conduct a comprehensive evaluation of DeepFake detection methods and datasets to demonstrate the escalated level of challenges posed by Celeb-DF. Then we introduce Landmark Breaker, the first dedicated method to disrupt facial landmark extraction, and apply it to the obstruction of the generation of DeepFake videos. The experiments are conducted on three state-of-the-art facial landmark extractors using our Celeb-DF dataset.

In recent years, the multimedia forensics and security community has seen remarkable progress in multitask learning for DeepFake (<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i>, face forgery) detection. The prevailing approach has been to frame DeepFake detection as a binary classification problem augmented by manipulation-oriented auxiliary tasks. This scheme focuses on learning features specific to face manipulations with limited generalizability. In this paper, we delve deeper into semantics-oriented multitask learning for DeepFake detection, capturing the relationships among face semantics via joint embedding. We first propose an automated dataset expansion technique that broadens current face forgery datasets to support semantics-oriented DeepFake detection tasks at both the global face attribute and local face region levels. Furthermore, we resort to the joint embedding of face images and labels (depicted by text descriptions) for prediction. This approach eliminates the need for manually setting task-agnostic and task-specific parameters, which is typically required when predicting multiple labels directly from images. In addition, we employ bi-level optimization to dynamically balance the fidelity loss weightings of various tasks, making the training process fully automated. Extensive experiments on six DeepFake datasets show that our method improves the generalizability of DeepFake detection and renders some degree of model interpretation by providing human-understandable explanations.

The rapid advancement of generative adversarial networks (GANs) and other AI-driven synthesis techniques, deepfake videos have emerged as a significant threat to digital media integrity, enabling the creation of highly realistic but fake video content. These manipulated videos can be used maliciously in disinformation campaigns, identity theft, and other cybercrimes, making their detection a critical challenge. This paper presents a deep learning-based approach for deepfake video detection that leverages both spatial artifacts and temporal inconsistencies introduced during the manipulation process. The proposed method utilizes a hybrid architecture combining convolutional neural networks (CNNs) for frame-level feature extraction and recurrent neural networks (RNNs), specifically Long Short-Term Memory (LSTM) networks, to capture temporal dependencies across frames. We evaluate our method on widely used benchmark datasets including FaceForensics++, Celeb-DF, and DFDC, demonstrating superior accuracy, robustness, and generalization capability against multiple types of deepfake generation techniques. The experimental results validate the effectiveness of our approach in real-world scenarios, highlighting its potential for deployment in digital forensics, content moderation, and media verification systems. Keywords: Deepfake detection, Synthetic media, Deepfake forensics, AI-generated media,Face manipulation Video

The proliferation of the capability for producing and distributing deepfake videos threatens the integrity of systems of justice, democratic processes, and the general ability to critically assess evidence. This study sought to identify individual differences that predict one’s ability to detect these forgeries. It was hypothesized that measures of affect detection and political orientation would correlate with performance on a deepfake detection task. Within a sample (N = 173) of college undergraduates and participants from Amazon’s M-turk, affect detection ability was shown to correlate with deepfake detection ability, r(171) = .73, p < .001, and general orientation to the political left was shown to correlate with deepfake detection ability, r(171) = .42, p < .001. The results of this study serve to identify populations who are particularly susceptible to deception via deepfake video and to inform the development of interventions that may help defend against attempts to influence them.

In recent years, the abuse of a face swap technique called deepfake has raised enormous public concerns. So far, a large number of deepfake videos (known as deepfakes) have been crafted and uploaded to the internet, calling for effective countermeasures. One promising countermeasure against deepfakes is deepfake detection. Several deepfake datasets have been released to support the training and testing of deepfake detectors, such as DeepfakeDetection [1] and FaceForensics++ [23]. While this has greatly advanced deepfake detection, most of the real videos in these datasets are filmed with a few volunteer actors in limited scenes, and the fake videos are crafted by researchers using a few popular deepfake softwares. Detectors developed on these datasets may become less effective against real-world deepfakes on the internet. To better support detection against real-world deepfakes, in this paper, we introduce a new dataset WildDeepfake, which consists of 7,314 face sequences extracted from 707 deepfake videos collected completely from the internet. WildDeepfake is a small dataset that can be used, in addition to existing datasets, to develop and test the effectiveness of deepfake detectors against real-world deepfakes. We conduct a systematic evaluation of a set of baseline detection networks on both existing and our WildDeepfake datasets, and show that WildDeepfake is indeed a more challenging dataset, where the detection performance can decrease drastically. We also propose two (eg. 2D and 3D) Attention-based Deepfake Detection Networks (ADDNets) to leverage the attention masks on real/fake faces for improved detection. We empirically verify the effectiveness of ADDNets on both existing datasets and WildDeepfake. The dataset is available at: https://github.com/deepfakeinthewild/deepfake-in-the-wild.

Advances in generative models and manipulation techniques have given rise to digitally altered videos known as deepfakes. These videos are difficult to identify for both humans and machines. Modern detection methods exploit various weaknesses in deepfake videos, such as visual artifacts and inconsistent posing. In this paper, we describe a novel architecture called Eff-YNet designed to detect visual differences between altered and unaltered areas. The architecture combines an EfficientNet encoder and a U-Net with a classification branch into a model capable of both classifying and segmenting deepfake videos. The task of segmentation helps train the classifier and also produces useful segmentation masks. We also implement ResNet 3D to detect spatiotemporal inconsistencies. To test these models, we run experiments against the Deepfake Detection Challenge dataset and show improvements over baseline classification models. Furthermore, we find that an ensemble of these two approaches improves performance over a single approach alone.

Deepfake techniques can forge the visual or audio signals in the video, which leads to inconsistencies between visual and audio (VA) signals. Therefore, multimodal detection methods expose deepfake videos by extracting VA inconsistencies. Recently, deepfake technology has started VA collaborative forgery to obtain more realistic deepfake videos, which poses new challenges for extracting VA inconsistencies. Recent multimodal detection methods propose to first extract natural VA correspondences in real videos in a self-supervised manner, and then use the learned real correspondences as targets to guide the extraction of VA inconsistencies in the subsequent deepfake detection stage. However, the inherent VA relations are difficult to extract due to the modality gap, which leads to the limited auxiliary performance of the aforementioned self-supervised methods. In this paper, we propose Predictive Visual-audio Alignment Self-supervision for Multimodal Deepfake Detection (PVASS-MDD), which consists of PVASS auxiliary and MDD stages. In the PVASS auxiliary stage in real videos, we first devise a three-stream network to associate two augmented visual views with corresponding audio clues, leading to explore common VA correspondences based on cross-view learning. Secondly, we introduce a novel cross-modal predictive align module for eliminating VA gaps to provide inherent VA correspondences. In the MDD stage, we propose to the auxiliary loss to utilize the frozen PVASS network to align VA features of real videos, to better assist multimodal deepfake detector for capturing subtle VA inconsistencies. We conduct extensive experiments on existing widely used and latest multimodal deepfake datasets. Our method obtains a significant performance improvement compared to state-of-the-art methods.

In this paper we propose a novel human-centered approach for detecting forgery in face images, using dynamic prototypes as a form of visual explanations. Currently, most state-of-the-art deepfake detections are based on black-box models that process videos frame-by-frame for inference, and few closely examine their temporal inconsistencies. However, the existence of such temporal artifacts within deepfake videos is key in detecting and explaining deepfakes to a supervising human. To this end, we propose Dynamic Prototype Network (DPNet) - an interpretable and effective solution that utilizes dynamic representations (i.e., prototypes) to explain deepfake temporal artifacts. Extensive experimental results show that DPNet achieves competitive predictive performance, even on unseen testing datasets such as Google's DeepFakeDetection, DeeperForensics, and Celeb-DF, while providing easy referential explanations of deepfake dynamics. On top of DPNet's prototypical framework, we further formulate temporal logic specifications based on these dynamics to check our model's compliance to desired temporal behaviors, hence providing trustworthiness for such critical detection systems.

This article investigates deepfake detection by comparing the strategies used by young adults and seniors, focusing on identifying similarities and differences between the two age groups. Semi-structured interviews were conducted with 40 participants, including 20 young adults aged 21–35 and 20 seniors aged 55–70, to explore how they evaluate the authenticity of deepfake videos. The interview data were analyzed using qualitative content analysis to develop an in-depth understanding of age-related detection strategies. The findings reveal five common strategies employed by both groups: analysis of character appearance, scrutiny of non-human visual elements, assessment of audio cues, reliance on personal knowledge, and consultation with external sources. While both age groups draw on similar types of cues, seniors tend to rely more on accumulated life experience, whereas young adults place greater emphasis on information gained through digital media exposure. By highlighting these shared and divergent approaches, this study contributes to the deepfake detection literature and offers valuable insights into the factors shaping individuals’ strategies for identifying deepfake videos.

Deepfake technology, allowing manipulations of audiovisual content by means of artificial intelligence, is on the rise. This has sparked concerns about a weaponization of manipulated videos for malicious ends. A theory on deepfake detection is presented and three preregistered studies examined the detection of deepfakes in the political realm (featuring UK’s Prime Minister Boris Johnson, Studies 1–3, or former U.S. President Barack Obama, Study 2). Based on two system models of information processing as well as recent theory and research on fake news, individual differences in analytic thinking and political interest were examined as predictors of correctly detecting deepfakes. Analytic thinking (Studies 1 and 2) and political interest (Study 1) were positively associated with identifying deepfakes and negatively associated with the perceived accuracy of a fake news piece about a leaked video (whether or not the deepfake video itself was presented, Study 3). Implications for research and practice are discussed.

The advancement of generative models has made it easier to create highly realistic Deepfake videos. This accessibility has led to a surge in research on Deepfake detection to mitigate potential misuse. Typically, Deepfake detection models utilize binary backbones, even though the training dataset contains additional exploitable information, such as the Deepfake generation method employed for each video. However, recent findings suggest that inferring a binary class from a multi-class backbone yields superior performance compared to directly employing a binary backbone. Building upon this research, our article introduces two novel methods to infer a binary class from a multi-class backbone. The first method, named root dummies , leverages the dummy triplet loss, which employs fixed vectors (i.e., dummies) instead of mined positives and negatives in the triplet loss. By training the multi-class backbone with these dummies, we can easily infer a binary class during testing by adjusting the number of dummies (from six during training to two during inference). Through this approach, we achieve an accuracy improvement of 0.23% compared to the existing inference method, without requiring additional training. The second proposed method is transfer learning. It involves training a classifier, such as a support vector machine, to predict binary classes based on the image embeddings generated by the multi-class backbone. Although this method necessitates additional training, it further enhances the model’s performance, resulting in an accuracy increase of 1.79%. In summary, our proposed methods improve the accuracy of Deepfake detection by simply modifying the number of classes during training, making them suitable for integration into a variety of existing Deepfake training pipelines. Additionally, to foster reproducible research, we have made the source code of our solution publicly available at https://github.com/beuve/DmyT .

The widespread flow of deepfake videos across social media platforms has raised serious security concerns. Despite significant progress in traditional deepfake detection methods, several challenges persist, such as distortion from compression artefacts, loss of critical feature information, high execution time, suboptimal feature selection, poor handling of dynamic or multi-face content, data imbalance, limited labelled training data, and disruptions in feature distribution, which continue to degrade detection accuracy. To address these limitations, this paper presents a Secretary Bird Skill Optimisation Algorithm-based Pyramid Deep Belief Network (SB-PyDBNet) for robust multi-face deepfake recognition. Initially, frames from the input video are extracted. After that, the face is detected employing YOLO v3-Tiny and next, facial action unit detection is accomplished by Action Unit Network (AUNet). Thereafter, ResNet features, statistical features and Histogram of Oriented Gradients (HOG) are extracted. Finally, multi-face deepfake recognition is done utilising Pyramid Deep Belief Network (PyramidFDBNet), which is introduced by combining Deep Pyramidal Residual Network (PyramidNet) with Deep Belief Network (DBN), whereupon modification of layers is performed by the Fractional Calculus (FC) concept. Moreover, PyramidNet training is performed by the Secretary Bird Skill Optimisation Algorithm (SBSOA), which is devised by integrating the Secretary Bird Optimisation Algorithm (SBOA) with the Skill Optimisation Algorithm (SOA). Additionally, SBSOA_PyramidFDBNet acquired 92.378% accuracy, 93.790% True Negative Rate (TNR), and 91.194% True Positive Rate (TPR).

Altered and manipulated multimedia is increasingly present and widely distributed via social media platforms. Advanced video manipulation tools enable the generation of highly realistic-looking altered multimedia. While many methods have been presented to detect manipulations, most of them fail when evaluated with data outside of the datasets used in research environments. In order to address this problem, the Deepfake Detection Challenge (DFDC) provides a large dataset of videos containing realistic manipulations and an evaluation system that ensures that methods work quickly and accurately, even when faced with challenging data. In this paper, we introduce a method based on convolutional neural networks (CNNs) and recurrent neural networks (RNNs) that extracts visual and temporal features from faces present in videos to accurately detect manipulations. The method is evaluated with the DFDC dataset, providing competitive results compared to other techniques.

Deepfake technology, which uses artificial intelligence to create realistic fake images, audio, and videos, has raised significant concerns due to its potential for misuse and manipulation. The emergence of deepfake technology poses a significant threat to the integrity of digital content, necessitating robust detection mechanisms. This paper proposes a novel method for deepfake detection by combining Overlapping Multiple Dynamic Images (OMDI) and Inversed Overlapping Multiple Dynamic Images (I‐OMDI). Both representations capture temporal inconsistencies and subtle visual artifacts in fake videos by effectively utilizing spatial–temporal information. Our approach employs EfficientNetB7 as the backbone for feature extraction, enabling the model to distinguish between real and fake videos with high accuracy. By combining OMDI and I‐OMDI with a weighted average strategy, we amplify the strengths of each method. Specifically, we assign equal weights of 0.5 to OMDI and I‐OMDI based on their individual contributions to performance metrics. This balance yields substantial performance improvements across multiple datasets. When evaluated on the Celeb‐DF v2 and DFDC datasets, our proposed model achieves state‐of‐the‐art results, with an AUC score of 0.9952 on Celeb‐DF v2 and 0.9947 on DFDC. These results underscore the robustness of the combined OMDI and I‐OMDI methods in identifying deepfake videos. Furthermore, our model demonstrates superior performance compared to existing methods, including those by Tran et al. and Heo et al., underscoring its effectiveness in practical deepfake detection applications. © 2025 Institute of Electrical Engineers of Japan and Wiley Periodicals LLC.

Deepfake detection based on cross-domain local characteristic analysis with multi-domain transformer