MolUNet++: Adaptive-grained explicit substructure and interaction aware molecular representation learning

Studies on generalization of learned visuomotor perturbations have generally focused on whether learning is coded in extrinsic or intrinsic reference frames. This dichotomy, however, is challenged by recent findings showing that learning is represented in a mixed reference frame. Overlooked in this framework is how learning appears to consist of multiple processes, such as explicit reaiming and implicit motor adaptation. Therefore, the proposed mixed representation may simply reflect the superposition of explicit and implicit generalization functions, each represented in different reference frames. Here we characterized the individual generalization functions of explicit and implicit learning in relative isolation to determine whether their combination could predict the overall generalization function when both processes are in operation. We modified the form of feedback in a visuomotor rotation task in an attempt to isolate explicit and implicit learning and tested generalization across new limb postures to dissociate the extrinsic/intrinsic representations. We found that the amplitude of explicit generalization was reduced with postural change and was only marginally shifted, resembling an extrinsic representation. In contrast, implicit generalization maintained its amplitude but was significantly shifted, resembling a mixed representation. A linear combination of individual explicit and implicit generalization functions accounted for nearly 85% of the variance associated with the generalization function in a typical visuomotor rotation task, where both processes are in operation. This suggests that each form of learning results from a mixed representation with distinct extrinsic and intrinsic contributions and the combination of these features shapes the generalization pattern observed at novel limb postures. NEW & NOTEWORTHY Generalization following learning in visuomotor adaptation tasks can reflect how the brain represents what it learns. In this study, we isolated explicit and implicit forms of learning and showed that they are derived from a mixed reference frame representation with distinct extrinsic and intrinsic contributions. Furthermore, we showed that the overall generalization pattern at novel workspaces is due to the superposition of independent generalization effects developed by explicit and implicit learning processes.

SADGFeat: Learning local features with layer spatial attention and domain generalization

Learning outcomes for residency training are defined in competency frameworks such as the CanMEDS framework, which ultimately aim to better prepare residents for their future tasks. Although residents' training relies heavily on learning through participation in the workplace under the supervision of a specialist, it remains unclear how the CanMEDS framework informs practice-based learning and daily interactions between residents and supervisors. This study aimed to explore how the CanMEDS framework informs residents' practice-based training and interactions with supervisors. Constructivist grounded theory guided iterative data collection and analyses. Data were collected by direct observations of residents and supervisors, combined with formal and field interviews. We progressively arrived at an explanatory theory by coding and interpreting the data, building provisional theories and through continuous conversations. Data analysis drew on sensitising insights from communities of practice theory, which provided this study with a social learning perspective. CanMEDS roles occurred in an integrated fashion and usually remained implicit during interactions. The language of CanMEDS was not adopted in clinical practice, which seemed to impede explicit learning interactions. The CanMEDS framework seemed only one of many factors of influence in practice-based training: patient records and other documents were highly influential in daily activities and did not always correspond with CanMEDS roles. Additionally, the position of residents seemed too peripheral to allow them to learn certain aspects of the Health Advocate and Leader roles. The CanMEDS framework did not really guide supervisors' and residents' practice or interactions. It was not explicitly used as a common language in which to talk about resident performance and roles. Therefore, the extent to which CanMEDS actually helps improve residents' learning trajectories and conversations between residents and supervisors about residents' progress remains questionable. This study highlights the fact that the reification of competency frameworks into the complexity of practice-based learning is not a straightforward exercise.

The current paradigm of deep learning models for the joint representation of molecules and text primarily relies on 1D or 2D molecular formats, neglecting significant 3D structural information that offers valuable physical insight. This narrow focus inhibits the models' versatility and adaptability across a wide range of modalities. Conversely, the limited research focusing on explicit 3D representation tends to overlook textual data within the biomedical domain. We present a unified pre-trained language model, MolLM, that concurrently captures 2D and 3D molecular information alongside biomedical text. MolLM consists of a text Transformer encoder and a molecular Transformer encoder, designed to encode both 2D and 3D molecular structures. To support MolLM's self-supervised pre-training, we constructed 160K molecule-text pairings. Employing contrastive learning as a supervisory signal for learning, MolLM demonstrates robust molecular representation capabilities across four downstream tasks, including cross-modal molecule and text matching, property prediction, captioning, and text-prompted molecular editing. Through ablation, we demonstrate that the inclusion of explicit 3D representations improves performance in these downstream tasks. Our code, data, pre-trained model weights, and examples of using our model are all available at https://github.com/gersteinlab/MolLM. In particular, we provide Jupyter Notebooks offering step-by-step guidance on how to use MolLM to extract embeddings for both molecules and text.

The rise of Artificial Intelligence (AI) has produced prophets and prophecies announcing that the age of artificial consciousness is near. Not only does the mere idea that any machine could ever possess the full potential of human consciousness suggest that AI could replace the role of God in the future, it also puts into question the fundamental human right to freedom and dignity. This position paper takes the stand that, in the light of all we currently know about brain evolution and the never-stopping formation of adaptive neural circuitry for learning, memory, decision making and, ultimately, fully conscious reasoning and creativity in the human species, the idea of an artificial consciousness appears misconceived. The paper highlights some of the major reasons why. While awareness to external stimuli for processes such as perception, recognition, and operational problem solving is under the direct control of functionally specific brain networks associated with sensory and cognitive functions across animal species, consciousness is a unique property of the human mind. Potentiated by brain evolution, consciousness has come to be when humans became able to represent, and reflect on, the Self in relation to past, present and future, and to project these representations into possible worlds by drawing and other forms of conceptual and creative expression. Epigenetically determined, shaped by experience, capable of representing real and non-real world states, consciousness is enabled by context-dependent adaptive brain circuits that have evolved on the grounds of self-organizing functional interactions at different levels of integration in a from-local-to global functional brain design. The evolution of the latter being continuous, the limits of consciousness are unpredictable. If cracking the computational code to human consciousness were possible, the resulting algorithms would have to be able to generate temporal activity patterns simulating long-distance signal reverberation across the brain and the de-correlation of spatial signal contents from their temporal signatures. In the light of scientific evidence for complex interactions between implicit (non-conscious) and explicit (conscious) representations in learning, memory, and the construction of conscious representation, the code would have to be capable of making all implicit processing explicit. Algorithms would have to be capable of a progressive, less and less arbitrary selection of temporal activity patterns in a continuously developing neural network structure akin to that of the human brain, from synapses to higher cognitive functional integration. The code would have to possess the self-organizing capacities that generate the brain signatures of phenomenal consciousness. In the biological brain, consolidation or extinction of these temporal brain signatures is driven by external event probabilities according to the principles of Hebbian learning. Consciousness is constantly fed by such learning, capable of generating stable representations despite an incommensurable amount of variability in input data, across time and across individuals, for life-long integration of experience data. Artificial consciousness would require probabilistic adaptive computations capable of emulating all the dynamics of human learning and memory that enable human intelligence and creativity. No AI is likely to ever have such potential.

In this paper, we have proposed a novel feature representation on multi-scale receptive fields for objection recognition. The method is based on a modified convolutional neural networks (CNN), named network-in-network (NIN), which has shown a good performance in some computer vision tasks. However, applying NIN to some specific applications may encounter a few problems. First, the NIN removes the fully connected layers, which makes it unsuited to use in large-scale face recognition due to lack of an efficient feature representation, even though it brings a lot of performance benefits. Second, some lowerlayer features, which can make the feature representation more discriminative, is unused. In the pure forward architecture, these features are unseen to the classifier. To solve the two problems, we present a multi-scale receptive fields (MSRF) representation learning scheme. Based on a well trained NIN, we add a pathway to top layer and design a feature vector as final representation. In our experiments, we compare the result of our multi-scale receptive fields with standard NIN architecture. The results show our method can obtain a more explicit feature representation and improvements in performance.

Advances in small molecule representations and AI-driven drug research: bridging the gap between theory and application.

Structural magnetic resonance imaging (sMRI) reveals the structural organization of the brain. Learning general brain representations from sMRI is an enduring topic in neuroscience. Previous deep learning models neglect that the brain, as the core of cognition, is distinct from other organs whose primary attribute is anatomy. Capturing the high-level representation associated with inter-individual cognitive variability is key to appropriately represent the brain. Given that this cognition-related information is subtle, mixed, and distributed in the brain structure, sMRI-based models need to both capture fine-grained details and understand how they relate to the overall global structure. Additionally, it is also necessary to explicitly express the cognitive information that implicitly embedded in local-global image features. Therefore, we propose MCPATS, a brain representation learning framework that combines Multi-task Collaborative Pre-training (MCP) and Adaptive Token Selection (ATS). First, we develop MCP, including mask-reconstruction to understand global context, distort-restoration to capture fine-grained local details, adversarial learning to integrate features at different granularities, and age-prediction, using age as a surrogate for cognition to explicitly encode cognition-related information from local-global image features. This co-training allows progressive learning of implicit and explicit cognition-related representations. Then, we develop ATS based on mutual attention for downstream use of the learned representation. During fine-tuning, the ATS highlights discriminative features and reduces the impact of irrelevant information. MCPATS was validated on three different public datasets for brain disease diagnosis, outperforming competing methods and achieving accurate diagnosis. Further, we performed detailed analysis to confirm that the MCPATS-learned representation captures cognition-related information.

Machine learning as a tool for chemical space exploration broadens horizons to work with known and unknown molecules. At its core lies molecular representation, an essential key to improve learning about structure-property relationships. Recently, contrastive frameworks have been showing impressive results for representation learning in diverse domains. Therefore, this paper proposes a contrastive framework that embraces multimodal molecular data. Specifically, our approach jointly trains a graph encoder and an encoder for the simplified molecular-input line-entry system (SMILES) string to perform the contrastive learning objective. Since SMILES is the basis of our method, i.e., we built the molecular graph from the SMILES, we call our framework as SMILES Contrastive Learning (SMICLR). When stacking a nonlinear regressor on the SMICLR's pretrained encoder and fine-tuning the entire model, we reduced the prediction error by, on average, 44% and 25% for the energetic and electronic properties of the QM9 data set, respectively, over the supervised baseline. We further improved our framework's performance when applying data augmentations in each molecular-input representation. Moreover, SMICLR demonstrated competitive representation learning results in an unsupervised setting.

Molecular-text multimodal alignment is crucial for drug discovery and molecular design. However, existing methods often struggle to simultaneously model the semantic consistency between 1D/2D/3D molecular representations and natural language descriptions. This paper proposes TriFusion-Mol, an end-to-end trimodal molecular-text representation learning framework that significantly enhances the transferability and discriminability of cross-modal representations through stepwise alignment, modality-consistency constraints, and modality-attention fusion. Specifically, TriFusion-Mol first conducts text-aligned pretraining at the unimodal level for SMILES, molecular graphs, and 3D conformations separately, incorporating a dynamic hard negative mining strategy (batch-hard + memory queue) to ensure each modality learns a robust semantic space. Subsequently, the model explicitly constrains the consistency of 1D/2D/3D representations through a multimodal consistency loss to prevent embedding space collapse. Finally, we design a modality-attention fusion module to adaptively integrate multimodal information and realign it with text, achieving a unified molecular semantic representation. This study demonstrates that progressive alignment with modality-adaptive fusion effectively constructs robust multimodal molecular representations, providing a novel technical pathway for intelligent molecular design.

The accurate prediction of suitable chiral stationary phases (CSPs) for resolving the enantiomers of a given compound poses a significant challenge in chiral chromatography. Previous attempts at developing machine learning models for structure-based CSP prediction have primarily relied on 1D SMILES strings [the simplified molecular-input line-entry system (SMILES) is a specification in the form of a line notation for describing the structure of chemical species using short ASCII strings] or 2D graphical representations of molecular structures and have met with only limited success. In this study, we apply the recently developed 3D molecular conformation representation learning algorithm, which uses rapid conformational analysis and point clouds of atom positions in the 3D space, enabling efficient chemical structure-based machine learning. By harnessing the power of the rapid 3D molecular representation learning and a data set comprising over 300,000 chromatographic enantioseparation records sourced from the literature, our models afford notable improvements for the chemical structure-based choice of appropriate CSP for enantioseparation, paving the way for more efficient and informed decision-making in the field of chiral chromatography.

The application of deep learning to a diverse array of research problems has accelerated progress across many fields, bringing conventional paradigms to a new intelligent era. Just as the roles of instrumentation in the old chemical revolutions, we reinforce the necessity for integrating deep learning in molecular systems engineering and design as a transformative catalyst towards the next chemical revolution. To meet such research needs, we summarize advances and progress across several key elements of molecular systems: molecular representation, property estimation, representation learning, and synthesis planning. We further spotlight recent advances and promising directions for several deep learning architectures, methods, and optimization platforms. Our perspective is of interest to both computational and experimental researchers as it aims to chart a path forward for cross-disciplinary collaborations on synthesizing knowledge from available chemical data and guiding experimental efforts.

Accurate molecular property prediction is fundamental to modern drug discovery and materials design. However, prevailing computational methods are often insufficient, as they rely on single-granularity structural representations that fail to capture the hierarchical complexity of molecular systems. To address this challenge, we propose a new approach to molecular representation learning that incorporates structural information across multiple scales. We design DFusMol (Dual Fusion with Global and Local Attention), a novel framework inspired by multi-modal learning. DFusMol employs graph encoders to capture features from both atomic-level molecular graphs and motif-level graphs derived from chemical rules. A customized global-local attention mechanism then blends these diverse features to build comprehensive molecular representations. Experiments on nine public benchmark datasets reveal that DFusMol delivers top-tier predictive performance across all tasks, outperforming state-of-the-art self-supervised learning models on six of them. By effectively integrating atomic- and motif-level information, DFusMol provides an innovative and efficient solution for molecular property prediction, enhancing representation learning methodologies and demonstrating strong potential for applications in drug design and lead compound screening.

The accurate modeling and prediction of small molecule properties and bioactivities depend on the critical choice of molecular representation. Decades of informatics-driven research have relied on expert-designed molecular descriptors to establish quantitative structure-activity and structure-property relationships for drug discovery. Now, advances in deep learning make it possible to efficiently and compactly learn molecular representations directly from data. In this review, we discuss how active research in molecular deep learning can address limitations of current descriptors and fingerprints while creating new opportunities in cheminformatics and virtual screening. We provide a concise overview of the role of representations in cheminformatics, key concepts in deep learning, and argue that learning representations provides a way forward to improve the predictive modeling of small molecule bioactivities and properties.

Accurate prediction of molecular toxicity is vital for drug development. Most mainstream methods rely on fingerprints or graph-based feature extraction, the emergence of large language models (LLMs) offers new prospects for molecular representation learning in toxicity prediction. Although several studies attempt to leverage LLMs to integrate molecular sequence data for pretraining molecular representations, certain limitations remain. Current LLM-based approaches usually utilize solely on class embedding features, overlooking the rich information in sequence embedding. Moreover, integrating pre-trained molecular representations with multi-modal molecular data may further enhance performance in toxicity prediction. To address these challenges, we propose BFGTP, a BERT-guided two-stage molecular representation learning framework for toxicity prediction. Firstly, we design independent encoders for molecular descriptions of three modalities, where the fingerprint encoder with dual level attention mechanisms effectively integrates multi-category fingerprints. Then, the two-stage guide strategy is introduced to fully utilize the prior knowledge of LLMs, employing contrastive learning to align and fuse the tri-modal representations and knowledge distillation to align predicted value distributions. BFGTP ultimately combines fingerprint and graph representations to predict molecular toxicity. Experiments on seven toxicity datasets show that BFGTP outperforms baselines, achieving the highest AUC on five datasets and the best average performance across five evaluation metrics. Ablation studies, t-SNE visualization and case study confirm the effectiveness of BFGTP's components and its ability to capture meaningful molecular representations.