Cross-Modal Subspace Clustering via Deep Canonical Correlation Analysis

In complex underwater environments, the single mode of a single sensor cannot meet the precision requirement of object identification, and multisource fusion is currently the mainstream research approach. Deep canonical correlation analysis is an efficient feature fusion method but suffers from problems such as not strong scalability and low efficiency. Therefore, an improved deep canonical correlation analysis fusion method is proposed for underwater multisource sensor data containing noise. First, a denoising autoencoder is used for denoising and to reduce the data dimension to extract new feature expressions of raw data. Second, given that underwater acoustic data can be characterized as 1-dimensional time series, a 1-dimensional convolutional neural network is used to improve the deep canonical correlation analysis model, and multilayer convolution and pooling are implemented to decrease the number of parameters and increase the efficiency. To improve the scalability and robustness of the model, a stochastic decorrelation loss function is used to optimize the objective function, which reduces the algorithm complexity from O(n³) to O(n²). The comparison experiment of the proposed algorithm and other typical algorithms on MNIST containing noise and underwater multisource data in different scenes shows that the proposed algorithm is superior to others regardless of the efficiency or precision of target classification.

Batch process monitoring is of great importance to ensure the stable operation during the process running. However, traditional deep learning methods have certain limitations when dealing with complex data structures and dynamic features that are prominent in industrial batch processes. This paper proposes a GraphSAGE-LSTM-based deep canonical correlation analysis (GSA-LSTM-CCA) to enhance the performance of feature learning for batch process monitoring. First, the k-nearest neighbor method is used to construct a graph for the variable-unfolding batch process data as a supplementary structural information for the original spatiotemporal correlations. Then, GraphSAGE and LSTM are used to extract the structural features and time-varying features of data, respectively. Finally, the learned features are weighted and fused to construct a residual generator, and CCA is utilized to carry out process monitoring. The effectiveness and superiority of the proposed method are verified in an actual injection molding process.

Scene change detection is the process of identifying the differences between the multi-temporal image scenes at the semantic level, which has significant potential in the application of urban development and land management. In this paper, we propose a novel deep convolution canonical correlation analysis neural network (DCCANet) architecture, which could consider the spectral-spatial-temporal correlation feature for scene change detection in remote sensing images. For this purpose, we put together the convolutional neural networks (CNNs) and the deep canonical correlation analysis (DCCA) into the end-to-end network. The CNN could get the spectral-spatial feature information for scene representation, while the later could enhance the temporal correlation by nonlinear high- dimensional transformations between the multi-temporal image scenes for scene change detection. Experiments with high-resolution remote sensing image scene datasets demonstrated that our proposed approach can get a better performance in scene classification and change detection.

Linkage influence of energy market on financial market by multiscale complexity synchronization

Inference of hidden common driver dynamics by anisotropic self-organizing neural networks.

This paper investigates attention fatigue detection and multi-grain emotional AI classification for student mental health through an educational brain-computer interface (BCI), with a focus on integrating biomechanical principles to enhance understanding and application. Recognizing the growing importance of students’ mental health and well-being, this study introduces a domain generalization approach in transfer learning to improve cross-subject BCI model accuracy, addressing the challenges of individual variability. The proposed model utilizes only seven electrodes and achieves a 90% accuracy rate in differentiating between two cognitive-behavioral tasks. A truncated weighting algorithm is employed to optimize electrode combinations, enabling effective generalization across subjects. To tackle the practical challenges of emotion recognition in educational settings, the study reduces data sampling points by identifying key brain regions and frequency bands associated with emotions. Machine learning algorithms, including support vector machines (SVM), Bayesian networks, and K-nearest neighbor (KNN), further enhance recognition accuracy. By integrating eye movement and electroencephalography (EEG) signals using deep canonical correlation analysis, the model achieves cumulative accuracy improvements of 15% and 12% compared to unimodal EEG and eye movement data, respectively, across 12 subjects. Incorporating biomechanical principles, the study also examines the mechanical properties of neural tissues and their influence on signal propagation. By analyzing the viscoelastic behavior of brain tissue and its impact on EEG signal transmission, the research provides insights into how mechanical stress and strain affect neural activity. This biomechanical perspective enhances the understanding of individual variability in EEG signals and contributes to the development of more robust and personalized BCI models. The integration of biomechanics with AI-driven emotion classification and attention fatigue detection offers a comprehensive approach to improving student mental health and educational outcomes. This fusion approach demonstrates superior performance in both emotion classification and attention fatigue detection, offering substantial potential for real-time interventions in student mental health and the enhancement of educational outcomes.

Takashima, Naoki Li, Frédéric Grzegorzek, Marcin Shirahama, KimiakiMusic-emotion retrieval is important for treatments of mood disorders and depression based on music. Although existing approaches only investigate one-way retrieval from music to emotions, an approach from emotion to music might be needed for proper application of music-based therapies. This can be achieved by sensor-based music retrieval which firstly recognises a specific emotion based on physiological data acquired by wearable sensors and then identifies music suitable for that emotion. In this paper, we propose Cross-modal Music-emotion Retrieval (CMR) as the first step to achieve retrieval of music based on sensor data. Our approach uses Deep Canonical Correlation Analysis (DeepCCA) which projects music samples and their associated emotion sequences into a common space using deep neural networks, and maximises the correlation between projected music samples and emotion sequences using CCA. Our experiments show the superiority of our approach for CMR compared to one-way retrieval.

Conventional Speech Emotion Recognition (SER) approaches put more emphasis on extracting magnitude spectrum-based features, such as Mel Frequency Cepstral Coefficients (MFCCs), and Mel spectrogram. However, phase information is ignored due to signal processing difficulties such as the phase wrapping issue. This work develops a multichannel Convolution Neural Network-Bidirectional Long Short Term Memory (CNN-BLSTM) architectures with an attention mechanism for speaker-independent SER by considering phase and magnitude spectrum-based features. The phase-based features are extracted using the Modified Group Delay Function (MODGD). The obtained phase features are combined with MFCC features. The CNN-BLSTM network extract learned representation from magnitude and phase features. The learned representation from MFCCs and MODGD are combined and given as an input to the Support Vector Machine (SVM) for classification. The Deep Canonical Correlation Analysis (DCCA) is used to maximize the correlation between magnitude and phase information to improve the conventional SER system’s performance. Here the IEMOCAP database is used for performance analysis. The experimental results show improvement over MFCC features and existing approaches for unimodal SER. In this work, we also developed real-time Web server application for the proposed architecture.

Age-related macular degeneration (AMD) is a prevalent retinal degenerative disease among the elderly and is a major cause of irreversible vision loss worldwide. Although color fundus photography (CFP) and optical coherence tomography (OCT) are widely used for AMD diagnosis, information from a single modal is inadequate to fully capture the complex pathological features of AMD. To address this, this study proposes an innovative multi-modal deep learning framework that fine-tunes pre-trained single-modal retinal models for efficient application in multi-modal AMD categorization tasks. Specifically, two independent vision transformer models are used to extract features from CFP and OCT images, followed by deep canonical correlation analysis (DCCA) to perform nonlinear mapping and fusion of features from both modalities, maximizing cross-modal feature correlation. Moreover, to reduce the computational complexity of multi-modal integration, we introduce the low-rank adaptation (LoRA) technique, which uses low-rank decomposition of parameter matrices, achieving superior performance compared to full fine-tuning with only about 0.49% of the trainable parameters. Experimental results on the public dataset MMC-AMD validate the framework's effectiveness. The proposed model achieves an overall F1-score of 0.948, AUC-ROC of 0.991, and accuracy of 0.949, significantly outperforming existing single-modal and multi-modal baseline models, particularly excelling in recognizing complex pathological categories.

This paper tackles a fine-grained action recognition task that aims to identify pitch-type-specific habits in baseball pitching motions. This task is highly challenging because it requires distinguishing subtle differences among pitch types within the common motion of “throwing.” Although it scarcely addressed in previous studies, it holds great potential for practical applications such as sports performance analysis. We propose a novel multimodal approach that integrates skeleton and RGB modalities. First, the Segment Anything Model (SAM) is employed to remove background information from RGB videos, thereby mitigating background bias. Next, Deep Canonical Correlation Analysis (DCCA) is applied to align the skeleton and RGB features. By further decomposing the RGB features into structural and textural components, the proposed method enhances discriminative performance. Experiments conducted on a custom pitching dataset collected from three pitchers demonstrate that our method outperforms conventional approaches in classification accuracy and effectively captures subtle pitching habits.

With the rapid growth of multimedia content on the Internet, multimedia retrieval has been extensively studied for decades. Inspired by the progress of deep neural network (DNN) in single-media retrieval, the researchers have applied the DNN to cross-media retrieval. Most of the existing methods cast low-level features of cross-media data onto a unified feature space. However, most of these feature spaces usually do not have explicit semantics which results in an unsatisfactory retrieval result. By considering the above issue, cross-media joint representation by hierarchical learning with DSCCA-LR is proposed in this paper which is able to explore jointly the correlation and semantic information. It combines Deep Canonical Correlation Analysis (DCCA) and Logistic Regression (LR) to learn two types of semantic features of of heterogeneous data and concatenate them to form common features for cross modal retrieval. Extensive experiments on two datasets show that the proposed approach achieves superior retrieval accuracy comparing to several state-of-the-art cross-media retrieval methods.

The rapid advancements in generative AI have led to text-to-video models creating highly realistic content, raising serious concerns about misinformation spread through synthetic videos. As these AI videos become more convincing, they threaten information integrity across social media, news, and digital communications. Using AI-generated videos, bad actors can now create false narratives, manipulate public opinion, and influence critical processes like elections. This technology's democratization means that sophisticated disinformation campaigns are no longer limited to well-resourced actors, creating an urgent need for reliable detection methods and human-machine cooperation to maintain public trust in visual information across our digital transformation landscape. The accessibility of these tools to a broader audience amplifies the potential for widespread misinformation, making robust detection systems crucial for maintaining social media integrity.Through our research into video generation models, we identified that state-of-the-art systems like diffusion transformers operate on patches of noisy latent spaces. We deliberately mirrored this architecture in our classifier design, enabling it to analyze videos using the same fundamental structural unit generation models used to create them. This architectural alignment allows our system to adapt to emerging generation techniques while maintaining detection efficacy.We designed an explainable video classifier using deep learning and neural networks that detect AI-generated content and show evidence for its decisions. The classifier uses three main parts: a convolutional encoder that turns video frames into latent representations, a patch vectorizer that breaks these representations into analyzable chunks, and a transformer that processes these chunks to make the final decision. This human-centered computing design lets us efficiently process videos while maintaining explainability through Integrated Gradients, which reveal which input parts influenced the model's decisions.We use integrated gradients to show which parts of a video led to the model's decision. This method looks at how the model's decision changes as we move from a blank video to the actual video, showing us which pixels matter most for classification. These pixel-level maps provide clear evidence of why the model thinks a video is AI-generated or real, providing transparency critical for building trust in automated content verification systems.We will test our model on the GenVideo dataset, a comprehensive collection of videos labeled as real or AI-generated from diverse sources, including Stable Diffusion, Sora, Kinetic 400, and MSRVTT. This large-scale data analytics evaluation will check how well it classifies videos and explains its decisions, helping determine if the model can work as a practical tool for machine learning-based content verification, considering that wrong AI classifications could harm content creators' reputations.Our work adds to the growing field of explainable AI in content authentication and shows why we need clear evidence when making high-stakes decisions about video content. Future work will look at detecting hybrid videos (real videos with AI elements added) and making our visual explanations more useful for human decision-makers in content verification. The insights gained will inform the development of more sophisticated detection systems capable of addressing evolving challenges in digital content authenticity.

The prevalent approach for unsupervised text style transfer is disentanglement between content and style. However, it is difficult to completely separate style information from the content. Other approaches allow the latent text representation to contain style and the target style to affect the generated output more than the latent representation does. In both approaches, however, it is impossible to adjust the strength of the style in the generated output. Moreover, those previous approaches typically perform both the sentence reconstruction and style control tasks in a single model, which complicates the overall architecture. In this paper, we address these issues by separating the model into a sentence reconstruction module and a style module. We use the Transformer-based autoencoder model for sentence reconstruction and the adaptive style embedding is learned directly in the style module. Because of this separation, each module can better focus on its own task. Moreover, we can vary the style strength of the generated sentence by changing the style of the embedding expression. Therefore, our approach not only controls the strength of the style, but also simplifies the model architecture. Experimental results show that our approach achieves better style transfer performance and content preservation than previous approaches.

Although multi-view learning has made significant progress over the past few decades, it is still challenging due to the difficulty in modeling complex correlations among different views, especially under the context of view missing. To address the challenge, we propose a novel framework termed Cross Partial Multi-View Networks (CPM-Nets), which aims to fully and flexibly take advantage of multiple partial views. We first provide a formal definition of completeness and versatility for multi-view representation and then theoretically prove the versatility of the learned latent representations. For completeness, the task of learning latent multi-view representation is specifically translated to a degradation process by mimicking data transmission, such that the optimal tradeoff between consistency and complementarity across different views can be implicitly achieved. Equipped with adversarial strategy, our model stably imputes missing views, encoding information from all views for each sample to be encoded into latent representation to further enhance the completeness. Furthermore, a nonparametric classification loss is introduced to produce structured representations and prevent overfitting, which endows the algorithm with promising generalization under view-missing cases. Extensive experimental results validate the effectiveness of our algorithm over existing state of the arts for classification, representation learning and data imputation.

SUM: Serialized Updating and Matching for text-based person retrieval