Bi-Source Reconstruction-Based Classification Network for Face Forgery Video Detection

Cervical cancer is one of the primary factors that endanger women’s health, and Thinprep cytologic test (TCT) is the common testing tool for the early diagnosis of cervical cancer. However, it is tedious and time-consuming for pathologists to assess and find abnormal cells in many TCT samples. Thus, automatic detection of abnormal cervical cells is highly demanded. Nevertheless, false positive cells are inevitable after automatic detection. It is still a burden for the pathologist if the false positive rate is high. To this end, here we propose a semi-supervised cervical cell diagnosis method that can significantly reduce the false positive rate. First, we incorporate a detection network to localize the suspicious abnormal cervical cells. Then, we design a semi-supervised classification network to identify whether the cervical cells are truly abnormal or not. To boost the performance of the semi-supervised classification network, and make full use of the localizing information derived from the detection network, we use the predicted bounding boxes of the detection network as an additional constraint for the attention masks from the classification network. Besides, we also develop a novel consistency constraint between the teacher and student models to guarantee the robustness of the network. Our experimental results show that our network can achieve satisfactory classification accuracy using only a limited number of labeled cells, and also greatly reduce the false positive rate in cervical cell detection.

A Convolutional neural network (CNN) has emerged as a widely used approach to computer vision tasks, including object classification and detection tasks. The high requirement for the model to be more computationally efficient on lower information and communication technology (ICT) resource, e.g., mobile terminals can benefit from model distillation. However, most existing distillation methods suffer from a significant accuracy reduction, which requires a large number of pre-training models or doesn't make good use of the more of the network information, e.g., in the middle layers, during the distillation. In this paper, we study how knowledge about traffic signs recognition could be transferred to smaller models by distillation while cutting channels. We present an optimized object detection network, which uses a Region Proposal Network (RPN) weighted loss and hard-soft distribution-wise distillation loss for structural differences between teacher and student networks. We validate the network on multiple real-world datasets, the experiments demonstrate that the classification accuracy can be improved by 9 % with about 16 times parameter reduction while the detection network performance could be increased by 10.6% using an optimized object detection network.

Electroencephalography (EEG) artifacts are very common in clinical diagnosis and can heavily impact diagnosis. Manual screening of artifact events is labor-intensive with little benefit. Therefore, exploring algorithms for automatic detection and classification of EEG artifacts can significantly assist clinical diagnosis. In this paper, we propose a learnable and explainable wavelet neural network (WaveNet) for EEG artifact detection and classification. The model is powered by the wavelet decomposition block based on invertible neural network, which can extract signal features without information loss, and a tree generator for building wavelet tree structure automatically. They provide the model with good feature extraction capabilities and explainability. To evaluate the model's performance more fairly, we introduce the base point level matching score (BASE) and the Event-Aligned Compensation Scoring (EACS) at the event level as two metrics for model performance evaluation. On the challenging Temple University EEG Artifact (TUAR) dataset, our model outperforms other baselines in terms of F1-score for both artifact detection and classification tasks. The case study also validates the model's ability to offer explainability for predictions based on frequency band energy, suggesting potential applications in clinical diagnosis.

Unsupervised anomaly detection defines an abnormal event as an event that does not conform to expected behavior. In the field of unsupervised anomaly detection, it is a pioneering work that leverages the difference between a future frame predicted by a generative adversarial network and its ground truth to detect an abnormal event. Based on the work, we improve the ability of video prediction framework to detect abnormal events by enhancing the difference between prediction results for normal and abnormal events. We incorporate super-resolution and self-attention mechanism to design a generative adversarial network. We propose an auto-encoder as a generator, which incorporates dense residual networks and self-attention. Moreover, we propose a new discriminator, which introduces self-attention on the basis of a relativistic discriminator. To predict a future frame with higher quality for normal events, we impose a constraint on the motion in video prediction by fusing optical flow and gradient difference between frames. We also introduce a perception constraint in video prediction to enrich the texture details of a frame. The AUC of our method on CUHK Avenue and Shanghai Tech datasets reaches 89.2% and 75.7% respectively, which is better than most existing methods. In addition, we propose a processing flow that can realize real-time anomaly detection in videos. The average running time of our video prediction framework is 37 frames per second. Among all real-time methods for abnormal event detection in videos, our method is competitive with the state-of-the-art methods.

Automatic event detection in cell videos is essential for monitoring cell populations in biomedicine. Deep learning methods have advantages over traditional approaches for cell event detection due to their ability to capture more discriminative features of cellular processes. Supervised deep learning methods, however, are inherently limited due to the scarcity of annotated data. Unsupervised deep learning methods have shown promise in general (non-cell) videos because they can learn the visual appearance and motion of regularly occurring events. Cell videos, however, can have rapid, irregular changes in cell appearance and motion, such as during cell division and death, which are often the events of most interest. We propose a novel unsupervised two-path input neural network architecture to capture these irregular events with three key elements: 1) a visual encoding path to capture regular spatiotemporal patterns of observed objects with convolutional long short-term memory units; 2) an event detection path to extract information related to irregular events with max-pooling layers; and 3) integration of the hidden states of the two paths to provide a comprehensive representation of the video that is used to simultaneously locate and classify cell events. We evaluated our network in detecting cell division in densely packed stem cells in phase-contrast microscopy videos. Our unsupervised method achieved higher or comparable accuracy to standard and state-of-the-art supervised methods.

Computer-aided polyp detection during colonoscopy is beneficial to reduce the risk of colorectal cancers. Deep learning techniques have made significant process in natural object detection. However, when applying those fully supervised methods to polyp detection, the performance is greatly depressed by the deficiency of labeled data. In this paper, we propose a novel bootstrapping method for polyp detection in colonoscopy videos by augmenting training data with temporal consistency. For a detection network that is trained on a small set of annotated polyp images, we fine-tune it with new samples selected from the test video itself, in order to more effectively represent the polyp morphology of current video. A strategy of selecting new samples is proposed by considering temporal consistency in the test video. Evaluated on 11954 endoscopic frames of the CVC-ClinicVideoDB dataset, our method yields great improvement on polyp detection for several detection networks, and achieves state-of-the-art performance on the benchmark dataset.

Despite recent breakthroughs in object detection with static images, extending state-of-the-art object detectors from image to video is challenging. The detection accuracy suffers from degenerated object appearances in videos, e.g., occlusion, video defocus, motion blur, etc. In this paper, we present a new framework called Missing Recover Recurrent Neural Networks (MR-RNN) for improving object detection in videos, which captures temporal information to recover the missing object. First, We detect objects in consecutive frames to obtain the bounding boxes and their confidence scores. The detector is trained for every frame of the video. Then we feed these detections into a Recurrent Neural Network (LSTM [8] or BiLSTM [4]) to capture temporal information. This method is tested on a large-scale vehicle dataset, “DETRAC”. Our approach achieves Average Precision (AP) of 68.90 based on SSD detector, an improvement of 2.68 over the SSD detector. Experimental results show that our method successfully detects many objects which are missed by basic detectors.

The verification of video authenticity has become progressively more challenging with the rapid advancements in video synthesis technologies. However, current detection approaches predominantly depend on intra-frame spatial artifacts and temporal inconsistencies, restricting their capacity to fully exploit the spatio-temporal characteristics of manipulated videos. To address this problem, we propose the Spatial and Temporal Feature Aggregation Network (STFANet), which employs a two-path structure to extract spatial and temporal features independently. These extracted features are subsequently integrated to construct high-fidelity spatio-temporal representations. Additionally, we incorporate a Vision Transformer module to capture global dependencies within the feature maps, enhancing the overall feature representation. Extensive experiments validate the efficacy of the proposed approach in detecting facial forgery in videos. Performance evaluations on benchmark datasets, including FaceForensics++ and Celeb-DF, confirm the effectiveness of our method, yielding AUC scores of 0.9933 and 0.9829, respectively. Furthermore, we investigate the impact of feature aggregation at different stages on the generated feature maps, revealing significant improvements in the quality of spatio-temporal representations.

Salient object detection in video using deep non-local neural networks

FTCF: Full temporal cross fusion network for violence detection in videos

Among the catastrophic natural events posing hazards to human lives and infrastructures, fire is the phenomenon causing more frequent damages. Thanks to the spread of smart cameras, video fire detection is gaining more attention as a solution to monitor wide outdoor areas where no specific sensors for smoke detection are available. However, state-of-the-art fire detectors assure a satisfactory Recall but exhibit a high false-positive rate that renders the application practically unusable. In this paper, we propose FLAME, an efficient and adaptive classification framework to address fire detection from videos. The framework integrates a state-of-the-art deep neural network for frame-wise object detection, in an automatic video analysis tool. The advantages of our approach are twofold. On the one side, we exploit advances in image detector technology to ensure a high Recall. On the other side, we design a model-based motion analysis that improves the system’s Precision by filtering out fire candidates occurring in the scene’s background or whose movements differ from those of the fire. The proposed technique, able to be executed in real-time on embedded systems, has proven to surpass the methods considered for comparison on a recent literature dataset representing several scenarios. The code and the dataset used for designing the system have been made publicly available by the authors at (https://mivia.unisa.it/large-fire-dataset-with-negative-samples-lfdn/).

Accurate pedestrian detection in highly crowded surveillance videos is a challenging task, since the regions of pedestrians in the videos may be largely occluded by other pedestrians. In this paper, we propose an effective part-based deep network cascade (HsNet) to solve this problem. In this model, the part-based scheme effectively restrains the appearance variations of pedestrians caused by heavy occlusion. The deep network captures discriminative information of visible body parts. In addition, the cascade architecture enables very fast detection. We make experiments on one of the largest surveillance video dataset, namely TRECVid SED Pedestrian Dataset (SED-PD). It is shown that in highly crowded surveillance videos, our proposed method achieves very competitive performance compared with state-of-the-art methods. More importantly, our method is significantly faster.

Detecting and understanding interactions between students and teachers in classroom is an important criterion for computer vision-based educational assistive systems. Recently, deep long-range spatial dependencies modeling techniques, such as non-local networks, have been proven to be very effective for such tasks. Yet, regarding global context generation, we analyze that the non-local operation only compares pixels using their values, which cannot pertain to structural information. In this paper, we first extend the non-local module to corporate locality attributes. We further observe that each query is treated uniformly to generate the attention map. Hence, we incorporate distance-wise representations with an efficient implementation into the non-local formulas. The proposed locality and relative distance-aware non-local module is integrated into an object detection architecture namely Libra-RCNN and is evaluated through our experiments on a pre-access hand-raising gesture dataset. Our straightforward modification achieves 0.8% and 2.8% higher performance compared to the baseline Libra-RCNN model, in terms of mAP 0.5 and mAP 0.75 respectively.

learning-based image processing models, which are among recent data processing techniques [7].

Visual-language collaborative multimodal transformer network for group activity detection in surveillance videos