Patch-based localized artifact enhancement for deepfake detection

Due to the abuse of deep forgery technology, the research on forgery detection methods has become increasingly urgent. The corresponding relationship between the frequency spectrum information and the spatial clues, which is often neglected by current methods, could be conducive to a more accurate and generalized forgery detection. Motivated by this inspiration, we propose a wavelet‐based texture mining and enhancement framework for face forgery detection. First, we introduce a frequency‐guided texture enhancement (FGTE) module that mining the high‐frequency information to improve the network’s extraction of effective texture features. Next, we propose a global–local feature refinement (GLFR) module to enhance the model’s leverage of both global semantic features and local texture features. Moreover, the interactive fusion module (IFM) is designed to fully incorporate the enhanced texture clues with spatial features. The proposed method has been extensively evaluated on five public datasets, such as FaceForensics++ (FF++), deepfake (DF) detection (DFD) challenge (DFDC), Celeb‐DFv2, DFDC preview (DFDC‐P), and DFD, for face forgery detection, yielding promising performance within and cross dataset experiments.

Learning a deep dual-level network for robust DeepFake detection

Thorax disease classification with attention guided convolutional neural network

Lunar craters and rilles are significant topographic features on the lunar surface that will play an essential role in future research on space energy resources and geological evolution. However, previous studies have shown low efficiency in detecting lunar impact craters and poor accuracy in detecting lunar rilles. There is no complete automated identification method for lunar features to explore space energy resources further. In this paper, we propose a new specific deep-learning method called high-resolution global–local networks (HR-GLNet) to explore craters and rilles and to discover space energy simultaneously. Based on the GLNet network, the ResNet structure in the global branch is replaced by HRNet, and the residual network and FPN are the local branches. Principal loss function and auxiliary loss function are used to aggregate global and local branches. In experiments, the model, combined with transfer learning methods, can accurately detect lunar craters, Mars craters, and lunar rilles. Compared with other networks, such as UNet, ERU-Net, HRNet, and GLNet, GL-HRNet has a higher accuracy (88.7 ± 8.9) and recall rate (80.1 ± 2.7) in lunar impact crater detection. In addition, the mean absolute error (MAE) of the GL-HRNet on global and local branches is 0.0612 and 0.0429, which are better than the GLNet in terms of segmentation accuracy and MAE. Finally, by analyzing the density distribution of lunar impact craters with a diameter of less than 5 km, it was found that: (i) small impact craters in a local area of the lunar north pole and highland (5°–85°E, 25°–50°S) show apparent high density, and (ii) the density of impact craters in the Orientale Basin is not significantly different from that in the surrounding areas, which is the direction for future geological research.

Due to the scarcity of fault samples in real-world industrial scenarios, few-shot fault diagnosis (FSFD) has attracted increasing attention, as it is vital for ensuring industrial safety and engineering reliability. However, existing metrics-based meta learning approaches generally calculate fault similarity relying on global pairwise, which insufficiently exploit fault information and thus limit discrimination among fault types. To address the insensitivity to buried fault information in complete processes, we propose a novel framework called the Local-Global Consistency Relation Network (LGCRN). The framework captures more comprehensive fault feature information through two components: the global branch, which focuses on macro-information involving all join process units, and the local branch, which emphasizes finer granularity from specific units. Additionally, a local-global alignment loss is utilized to optimize LGCRN, where self-cross relation improves the discriminability and stability of fault features by maintaining consistency between local and global branches. The self-cross relation comprises self-sample relation, ensuring consistency within the same sample at different granularities, and cross-sample relation, maintaining consistency across samples within the same class. Experiments on two benchmarks and a real semiconductor process demonstrate the feasibility and superiority of the proposed method, confirming its effectiveness in addressing limited-sample industrial fault diagnosis tasks and enabling timely maintenance crucial for operational safety in real-world scenarios.

Forensic deepfake audio detection using segmental speech features.

Accurate segmentation of 3D vascular structures is essential for various medical imaging applications. The dispersed nature of vascular structures leads to inherent spatial uncertainty and necessitates location awareness, yet most current 3D medical segmentation models rely on the patch-wise training strategy that usually loses this spatial context. In this study, we introduce the Coordinate-aware Modulated Mamba Network (COMMA) and contribute a manually labeled dataset of 570 cases, the largest publicly available 3D cerebrovascular dataset to date. COMMA leverages both entire and cropped patch data through global and local branches, ensuring robust and efficient spatial location awareness. Specifically, COMMA employs a channel-compressed Mamba (ccMamba) block to efficiently encode full-resolution image data, capturing long-range dependencies while optimizing computational costs. Additionally, we propose a coordinate-aware modulated (CaM) block to enhance interactions between the global and local branches, allowing the local branch to better perceive spatial information. We evaluate COMMA on six datasets, covering two imaging modalities and five types of vascular tissues. The results demonstrate COMMA's superior performance compared to state-of-the-art methods with computational efficiency, especially in segmenting small vessels. Ablation studies further highlight the importance of our proposed modules and spatial information. The code will be available at COMMA.

Person re-identification (Re-ID) aims at retrieving a person of interest across multiple cameras. With the advancement of deep network and increasing demand of intelligent video surveillance, it has gained significantly increased interest in the computer vision community. In this paper, we propose a simple yet effective Multi-Scale Horizontal (MSH) model for person Re-ID task. Firstly, the model consists of a novel Multi-branch network which adopted residual network ResNet50. There are two branches in our network: global branch and local branch. In local branch, the model slice a person into different parts in multi-scales. Secondly, we present a mix pooling method which considering both average and maximum pooling method. Finally, we employ triple loss and softmax loss as the loss function of the network. Experiments on two datasets (Market1501 and DukeMTMC-reID) demonstrate the advantage of the proposed model.

The current deep learning methods for copy-move forgery detection (CMFD) are mostly based on deep convolutional neural networks, which frequently discard a large amount of detail information throughout convolutional feature extraction and have poor long-range information extraction capabilities. The Transformer structure is adept at modeling global context information, but the patch-wise self-attention calculation still neglects the extraction of details in local regions that have been tampered with. A local-information-refined dual-branch network, LBRT (Local Branch Refinement Transformer), is designed in this study. It performs Transformer encoding on the global patches segmented from the image and local patches re-segmented from the global patches using a global modeling branch and a local refinement branch, respectively. The self-attention features from both branches are precisely fused, and the fused feature map is then up-sampled and decoded. Therefore, LBRT considers both global semantic information modeling and local detail information refinement. The experimental results show that LBRT outperforms several state-of-the-art CMFD methods on the USCISI dataset, CASIA CMFD dataset, and DEFACTO CMFD dataset.

Deepfake video detection has emerged as a critical challenge in the realm of artificial intelligence, given its implications for misinformation and digital security. This study evaluates the generalisation capabilities of the CoAtNet model—a hybrid convolution–transformer architecture—for deepfake detection across diverse datasets. Although CoAtNet has shown exceptional performance in several computer vision tasks, its potential for generalisation in cross-dataset scenarios remains underexplored. Thus, in this study, we explore CoAtNet’s generalisation ability by conducting an extensive series of experiments with a focus on discovering features and variations in deepfake videos. These experiments involve training the model using various input and processing configurations, followed by evaluating its performance on widely recognised public datasets. To the best of our knowledge, our proposed approach outperforms state-of-the-art models in terms of intra-dataset performance, with an AUC between 81.4% and 99.9%. Our model also achieves outstanding results in cross-dataset evaluations, with an AUC equal to 78%. This study demonstrates that CoAtNet achieves the best AUC for both intra-dataset and cross-dataset deepfake video detection, particularly on Celeb-DF, while also showing strong performance on DFDC.

Person re-identification with part prediction alignment

One of the strongest cues for retrieval of content information from images is shape. However, due to the wide range of transformations that an object might undergo, this is also the most difficult one to handle. It seems that shape retrieval is one of the major barriers nowadays on the way of image databases to become commonly used. Common approaches use global attributes (Faloutsos et al. [1]), feature points (Pentland et al. [2]), histograms (Jain and Vailaya [3]), or physical models of deformations (Del Bimbo and Pala [4]). We present an approach for shape retrieval from pictorial databases which is based on invariant features of the image. In particular we use a combination of semi-local multi-valued invariant signatures and global features. Spatial relations and global properties are used to eliminate non-relevant images before similarity is computed. Common approaches usually don't handle viewpoint transformations more complex than similarity and require the full shape in order to compute image features. The advantages of the proposed approach are its ability to handle images distorted by different viewpoint transformations, its ability to retrieve images even in situations in which part of the shape is missing (i.e., in case of occlusion or sketch-based queries), and its ability to support efficient indexing.

To solve the problems of image detail blurring and insufficient utilization of global information in the existing medical image super-resolution reconstruction, this paper proposes a dual-branch fusion network based on residual Transformer network and dynamic convolutional neural network (CTGFSR). The network consists of two branches, one is the global branch based on residual Transformer network, and the other is the local branch based on dynamic convolutional neural network. The global branch uses the self-attention mechanism of Transformer network, which can effectively mine the large-scale global information in the image and improve the overall quality of the image. The local branch uses the characteristic of dynamic convolution to adaptively adjust the convolution kernel parameters, which can enhance the feature extraction ability of convolutional neural network for multi-scale information and improve the detail restoration ability of the image without significantly increasing the network model size. The network uses residual skip connections to preserve the detail information in medical image super-resolution reconstruction. Finally, through the bidirectional gated attention mechanism, the two branches are fused to obtain the final super-resolution reconstruction image. This paper evaluates the performance of the network on two medical image datasets, namely ACDC abdominal MR related to medical image segmentation and L2R2022 lung CT related to registration. The experimental results show that compared with the mainstream super-resolution algorithms, CTGFSR has better overall performance. When the magnification factor is 2 or 4, compared with the convolutional neural network based CFIPC, PDCNCF, ESPCN, FSRCNN, VDSR and the Transformer network based ESRT, SwinIR, the structural similarity SSIM and peak signal-to-noise ratio PSNR have a certain improvement.

Despite the great progress in temporal action proposal generation, most state-of-the-art methods ignore the impact of action scales and the performance of short actions is still far from satisfaction. In this paper, we first analyze the sample imbalance issue in action proposal generation, and correspondingly devise a novel scale-invariant loss function to alleviate the insufficient learning of short actions. To further achieve proposal generation task, we adopt the pipeline of boundary evaluation and proposal completeness regression, and propose the Temporal Scale Invariant network. To better leverage the temporal context, boundary evaluation module generates action boundaries with high-precision-assured global branch and high-recall-assured local branch. Simultaneously, the proposal evaluation module is supervised with introduced scale-invariant loss, predicting accurate proposal completeness for different scales of actions. Comprehensive experiments are conducted on ActivityNet-1.3 and THUMOS14 benchmarks, where TSI achieves state-of-the-art performance. Especially, AUC performance of short actions is boosted from 36.53% to 39.63% compared with baseline.

Computerized tomography (CT) is a clinically primary technique to differentiate benign-malignant pulmonary nodules for lung cancer diagnosis. Early classification of pulmonary nodules is essential to slow down the degenerative process and reduce mortality. The interactive paradigm assisted by neural networks is considered to be an effective means for early lung cancer screening in large populations. However, some inherent characteristics of pulmonary nodules in high-resolution CT images, e.g., diverse shapes and sparse distribution over the lung fields, have been inducing inaccurate results. On the other hand, most existing methods with neural networks are dissatisfactory from a lack of transparency. In order to overcome these obstacles, a united framework is proposed, including the classification and feature visualization stages, to learn distinctive features and provide visual results. Specifically, a bilateral scheme is employed to synchronously extract and aggregate global-local features in the classification stage, where the global branch is constructed to perceive deep-level features and the local branch is built to focus on the refined details. Furthermore, an encoder is built to generate some features, and a decoder is constructed to simulate decision behavior, followed by the information bottleneck viewpoint to optimize the objective. Extensive experiments are performed to evaluate our framework on two publicly available datasets, namely, 1) the Lung Image Database Consortium and Image Database Resource Initiative (LIDC-IDRI) and 2) the Lung and Colon Histopathological Image Dataset (LC25000). For instance, our framework achieves 92.98% accuracy and presents additional visualizations on the LIDC. The experiment results show that our framework can obtain outstanding performance and is effective to facilitate explainability. It also demonstrates that this united framework is a serviceable tool and further has the scalability to be introduced into clinical research.