Temporal context and representative feature learning for weakly supervised video anomaly detection

Recent studies have introduced end-to-end TTS, which integrates the production of context and acoustic features in statistical parametric speech synthesis. As a result, a single neural network replaced laborious feature engineering with automated feature learning. However, little is known about what types of context information end-to-end TTS extracts from text input before synthesizing speech, and the previous knowledge about context features is barely utilized. In this work, we first point out the model similarity between end-to-end TTS and parametric TTS. Based on the similarity, we evaluate the quality of encoder outputs from an end-to-end TTS system against eight criteria that are derived from a standard set of context information used in parametric TTS. We conduct experiments using an evaluation procedure that has been newly developed in the machine learning literature for quantitative analysis of neural representations, while adapting it to the TTS domain. Experimental results show that the encoder outputs reflect both linguistic and phonetic contexts, such as vowel reduction at phoneme level, lexical stress at syllable level, and part-of-speech at word level, possibly due to the joint optimization of context and acoustic features.

In complex tracking scenarios of multitarget occlusion, especially when the target is seriously occluded, feature loss can lead to tracking failure. In this article, we develop a multitarget occlusion tracking algorithm based on a 3-D spatio-temporal context graph model, which can fully utilize the targets’ RGB-D information. Specifically, we first extract target’s 3-D features containing appearance, motion, spatial, and temporal features. Then, the spa’Io-temporal context features between different targets are also be acquired. Furthermore, the attribute relational graph (ARG) model with target’s 3-D spatio-temporal context features is constructed, which combines spatial features in the current frame and temporal features between consecutive frames. Moreover, if the target is determined to be occluded according to the matching results in the current frame, the probability relaxation method is used to assign the optimal tracking label to clustering blocks in occlusion area by matching these ARG models, which can address the multitarget cross occlusion problem. Extensive experiments on the Princeton Tracking Benchmark Datasets clearly demonstrate that our method can achieve superior performance against the typical algorithms proposed in recent years.

Click through rate (CTR) prediction of image ads is the core task of online display advertising systems, and logistic regression (LR) has been frequently applied as the prediction model. However, LR model lacks the ability of extracting complex and intrinsic nonlinear features from handcrafted high-dimensional image features, which limits its effectiveness. To solve this issue, in this paper, we introduce a novel deep neural network (DNN) based model that directly predicts the CTR of an image ad based on raw image pixels and other basic features in one step. The DNN model employs convolution layers to automatically extract representative visual features from images, and nonlinear CTR features are then learned from visual features and other contextual features by using fully-connected layers. Empirical evaluations on a real world dataset with over 50 million records demonstrate the effectiveness and efficiency of this method.

Reward-predicting distractor orientations support contextual cueing: Persistent effects in homogeneous distractor contexts

Compositional visual question answering (Compositional VQA) needs to provide an answer to a compositional question, which requires the model to have advanced capabilities of multi-modal semantic understanding and logical reasoning. However, current VQA models mainly concentrate on enriching the visual representations of images and neglect the redundancy in the enriched information to bring some negative impacts. To enhance the value and availability of semantic features, we propose a novel core-to-global reasoning (CTGR) model for compositional VQA. The model first extracts both global features and core features from image and question through a feature embedding module. Then, to enhance the value of semantic features, we propose an information filtering module to align visual features and text features at the core semantic level and to filter out the redundancy carried by image and question features at the global semantic level, which can further strengthen cross-modal correlations. Besides, we design a novel core-to-global reasoning mechanism for multimodal fusion, which integrates content features from core learning and context features from global features for accurate answer predictions. Finally, extensive experimental results on GQA, GQA-sub, VQA2.0 and Visual7W demonstrate the effectiveness and superiority of CTGR.

Accurate multivariate traffic flow forecasting is critical for intelligent transportation systems yet remains challenging due to the complex interplay of temporal dynamics and spatial interactions. While Transformer-based models have shown promise in capturing long-range temporal dependencies, most existing approaches compress multidimensional observations into flattened sequences—thereby neglecting explicit modeling of cross-dimensional (i.e., spatial or inter-variable) relationships, which are essential for capturing traffic propagation, network-wide congestion, and node-specific behaviors. To address this limitation, we propose TSAformer, a novel Transformer architecture that explicitly preserves and jointly models time and dimension as dual structural axes. TSAformer begins with a multimodal input embedding layer that encodes raw traffic values alongside temporal context (time-of-day and day-of-week) and node-specific positional features, ensuring rich semantic representation. The core of TSAformer is the Two-Stage Attention (TSA) module, which first models intra-dimensional temporal evolution via time-axis self-attention then captures inter-dimensional spatial interactions through a lightweight routing mechanism—avoiding quadratic complexity while enabling all-to-all cross-node communication. Built upon TSA, a hierarchical encoder–decoder (HED) structure further enhances forecasting by modeling traffic patterns across multiple temporal scales, from fine-grained fluctuations to macroscopic trends, and fusing predictions via cross-scale attention. Extensive experiments on three real-world traffic datasets—including urban road networks and highway systems—demonstrate that TSAformer consistently outperforms state-of-the-art baselines across short-term and long-term forecasting horizons. Notably, it achieves top-ranked performance in 36 out of 58 critical evaluation scenarios, including peak-hour and event-driven congestion prediction. By explicitly modeling both temporal and dimensional dependencies without structural compromise, TSAformer provides a scalable, interpretable, and high-performance solution for spatiotemporal traffic forecasting.

Speech enhancement in noisy and reverberant conditions remains a challenging task. In this work, a time-frequency masking based method for speech enhancement with multi -channel data using convolutional neural networks (CNN) is proposed, where the CNN is trained to estimate the ideal ratio mask by discriminating directional speech source from diffuse or spatially uncorrelated noise. The proposed method operates on, frame-by-frame, the magnitude and phase components of the short-time Fourier transform coefficients of all frequency sub-bands and microphones. The avoidance of temporal context and explicit feature extraction makes the proposed method suitable for online implementation. In contrast to most speech enhancement methods that utilize multi -channel data, the proposed method does not require information about the spatial position of the desired speech source. Through experimental evaluation with both simulated and real data, we show the robustness of the proposed method to unseen acoustic conditions as well as varying noise levels.

Graph convolutional neural network for multi-scale feature learning

Benefiting from the powerful discriminative feature learning capability of convolutional neural networks (CNNs), deep learning techniques have achieved remarkable performance improvement for the task of salient object detection (SOD) in recent years. However, most existing deep SOD models do not fully exploit informative contextual features, which often leads to suboptimal detection performance in the presence of a cluttered background. This paper presents a context-aware attention module that detects salient objects by simultaneously constructing connections between each image pixel and its local and global contextual pixels. Specifically, each pixel and its neighbors bidirectionally exchange semantic information by computing their correlation coefficients, and this process aggregates contextual attention features both locally and globally. In addition, an attention-guided hierarchical network architecture is designed to capture fine-grained spatial details by transmitting contextual information from deeper to shallower network layers in a top-down manner. Extensive experiments on six public SOD datasets show that our proposed model demonstrates superior SOD performance against most of the current state-of-the-art models under different evaluation metrics.

Office documents have emerged as a prevalent attack vector, with adversaries increasingly embedding executable payloads and malicious macros to evade signature‐based detection mechanisms. To address these challenges, this study presents a hybrid Transformer–CNN semantic behavioral modeling framework for Office malware detection. The proposed approach jointly captures semantic and contextual features from behavioral traces obtained through sandbox analysis of 140 753 real‐world malicious Office samples. A composite feature selection strategy—combining Mutual Information, Random Forest Importance, and Chi‐Square metrics—is employed to retain the most discriminative attributes. Furthermore, a domain‐aware suspiciousness scoring mechanism integrates adversarial knowledge to improve explainability and contextual interpretation. By combining the Transformer's capability to model long‐range dependencies with the CNN's strength in identifying localized behavioral patterns, the system effectively captures diverse malware behaviors. Experimental results demonstrate a detection accuracy of 98.37%, outperforming CNN and baseline transformers in precision, recall, and F1‐score. In addition, interpretability studies of the learned features demonstrate that the model offers both resilience and explainability, contributing to a stronger foundation for future Office‐based malware defense systems.

In the past three years, the stereo matching method based on 3D CNNs has achieved surprising results and has received more and more attention. However, most stereo matching approaches aim to improve prediction accuracy by constructing and aggregating cost volumes through extensive 3D convolutions, which not only does not fully utilize the geometric information, but also overlooks the computational speed. Thus, achieving high-accuracy, high-efficiency stereo matching has become challenging. In this paper, we present a rapid and precise stereo matching network named CGW based on 3D CNNs, which simultaneously achieves real-time functioning, considerable accuracy, and a strong generalization capability. The network is divided into two parts. The first part constructs the geometric attention cube through a lightweight feature extraction network and a lightweight 3D regularization network. The second part filters the context features using the geometric attention cube to obtain the context geometric cube, and finally, the disparity is predicted and refined to obtain the final disparity. We adopted MobileNetV3 as an efficient backbone for feature extraction and designed 3D depthwise separable convolutions with residual structures to replace traditional 3D convolutions for constructing the cost volume and performing cost aggregation, aiming to reduce the model size and improve the computational speed. Additionally, we designed the context geometric attention (CGA) module and embedded it into the lightweight 3D regularization network, as well as designed the Warped Disparity Refinement (WDR) network to further improve the disparity prediction accuracy. CGA effectively guides cost aggregation by integrating rich contextual and geometric information, while also providing feedback for feature learning to guide more efficient context feature extraction. WDR constructs a warping cost volume using the obtained initial disparity, combined with image features, the initial disparity map, and reconstruction errors, to optimize the disparity. According to the initial disparity, it searches for the accurate disparity within a refined range. By narrowing the search range, WDR simplifies the task for the network to locate the correct disparity (residual), while simultaneously improving the computational efficiency. Experiments conducted on multiple benchmark datasets showed that, compared to other fast methods, CGW has advantages in both speed and accuracy and exhibits better generalization performance.

Emotion recognition aims to discern the emotional state of subjects within an image, relying on subject-centric and contextual visual cues. Current approaches typically follow a two-stage pipeline: first localize subjects by off-the-shelf detectors, then perform emotion classification through the late fusion of subject and context features. However, the complicated paradigm suffers from disjoint training stages and limited fine-grained interaction between subject-context elements. To address the challenge, we present a single-stage emotion recognition approach, employing a Decoupled Subject-Context Transformer (DSCT), for simultaneous subject localization and emotion classification. Rather than compartmentalizing training stages, we jointly leverage box and emotion signals as supervision to enrich subject-centric feature learning. Furthermore, we introduce DSCT to facilitate interactions between fine-grained subject-context cues in a ''decouple-then-fuse'' manner. The decoupled query tokens-subject queries and context queries-gradually intertwine across layers within DSCT, during which spatial and semantic relations are exploited and aggregated. We evaluate our single-stage framework on two widely used context-aware emotion recognition datasets, CAER-S and EMOTIC. Our approach surpasses two-stage alternatives with fewer parameter numbers, achieving a 3.39% accuracy improvement and a 6.46% average precision gain on CAER-S and EMOTIC datasets, respectively. Code and models are available at: https://github.com/Sampson-Lee/DSCT.

Esophageal cancer has become one of the important cancers that seriously threaten human life and health, and its incidence and mortality rate are still among the top malignant tumors. Histopathological image analysis is the gold standard for diagnosing different differentiation types of esophageal cancer. The grading accuracy and interpretability of the auxiliary diagnostic model for esophageal cancer are seriously affected by small interclass differences, imbalanced data distribution, and poor model interpretability. Therefore, we focused on developing the category imbalance attention block network (CIABNet) model to try to solve the previous problems. First, the quantitative metrics and model visualization results are integrated to transfer knowledge from the source domain images to better identify the regions of interest (ROI) in the target domain of esophageal cancer. Second, in order to pay attention to the subtle interclass differences, we propose the concatenate fusion attention block, which can focus on the contextual local feature relationships and the changes of channel attention weights among different regions simultaneously. Third, we proposed a category imbalance attention module, which treats each esophageal cancer differentiation class fairly based on aggregating different intensity information at multiple scales and explores more representative regional features for each class, which effectively mitigates the negative impact of category imbalance. Finally, we use feature map visualization to focus on interpreting whether the ROIs are the same or similar between the model and pathologists, thus better improving the interpretability of the model. The experimental results show that the CIABNet model outperforms other state-of-the-art models, which achieves the most advanced results in classifying the differentiation types of esophageal cancer with an average classification accuracy of 92.24%, an average precision of 93.52%, an average recall of 90.31%, an average F1 value of 91.73%, and an average AUC value of 97.43%. In addition, the CIABNet model has essentially similar or identical to the ROI of pathologists in identifying histopathological images of esophageal cancer. Our experimental results prove that our proposed computer-aided diagnostic algorithm shows great potential in histopathological images of multi-differentiated types of esophageal cancer.

This paper addresses the problem of obtaining a blur-based segmentation map from a single image affected by motion or defocus blur. Since traditional hand-designed priors have fundamental limitations, we utilise deep neural networks to learn features related to blur and enable a pixel-level blur classification. Our approach mitigates the ambiguities present in blur detection task by introducing joint learning of global context and local features into the framework. Specifically, we train two sub-networks to perform the task at global (image) and local (patch) levels. We aggregate the pixel-level probabilities estimated by two networks and feed them to a MRF based framework which returns a refined and dense segmentation-map of the image with respect to blur. We also demonstrate via both qualitative and quantitative evaluation, that our approach performs favorably against state-of-the-art blur detection or segmentation works, and show its utility to applications of automatic image matting and blur magnification.

The content-based remote sensing image retrieval (CBRSIR) has attracted increasing attention with the number of remote sensing (RS) images growing explosively. Benefiting from the strong capacity of the deep convolutional neural network (DCNN), the performance of CBRSIR has been improved in recent years. Although great successes have been obtained, learning the RS images' representative features and enhancing the retrieval efficiency for the large-scale CBRSIR tasks are still two challenging problems. In this article, we propose a new CBRSIR method named feature and hash (FAH) learning, which consists of a deep feature learning model (DFLM) and an adversarial hash learning model (AHLM). The DFLM aims at learning the RS images' dense features to guarantee the retrieval precision. In the DFLM, the DCNN and the proposed feature aggregation are integrated to capture the multiscale features. Then, the discrimination of the obtained features can be highlighted by the attention map in the developed attention branch. The AHLM maps the dense features onto the compact hash codes so that the retrieval efficiency can be improved. The AHLM contains a hash learning submodel and an adversarial regularization submodel. In particular, the hash learning submodel learns the real-valued hash codes that are similarity preserved by semantic supervisions. The adversarial regularization submodel regularizes the real-valued hash codes to learn the discrete uniform distribution with possible values 0 and 1. In this way, the hash codes are coding-balanced and the quantization errors are reduced. Encouraging experimental results counted on three public benchmark data sets demonstrate that our FAH can achieve competitive performance in the CBRSIR task compared with many existing hash learning methods.