MolAR: Bringing Chemical Structures to Life with Augmented Reality and Machine Learning

Visualizing 3D molecular structures is crucial to understanding and predicting their chemical behavior. However, static 2D hand-drawn skeletal structures remain the preferred method of chemical communication. Here, we combine cutting-edge technologies in augmented reality (AR), machine learning, and computational chemistry to develop MolAR, an open-source mobile application for visualizing molecules in AR directly from their hand-drawn chemical structures. Users can also visualize any molecule or protein directly from its name or protein data bank ID and compute chemical properties in real time via quantum chemistry cloud computing. MolAR provides an easily accessible platform for the scientific community to visualize and interact with 3D molecular structures in an immersive and engaging way.

Extended-Reality Technologies: An Overview of Emerging Applications in Medical Education and Clinical Care.

Lectins (Ls) play an important role in many diseases such as different types of cancer, parasitic infections and other diseases. Interestingly, the Protein Data Bank (PDB) contains +3000 protein 3D structures with unknown function. Thus, we can in principle, discover new Ls mining non-annotated structures from PDB or other sources. However, there are no general models to predict new biologically relevant Ls based on 3D chemical structures. We used the MARCH-INSIDE software to calculate the Markov-Shannon 3D electrostatic entropy parameters for the complex networks of protein structure of 2200 different protein 3D structures, including 1200 Ls. We have performed a Linear Discriminant Analysis (LDA) using these parameters as inputs in order to seek a new Quantitative Structure-Activity Relationship (QSAR) model, which is able to discriminate 3D structure of Ls from other proteins. We implemented this predictor in the web server named LECTINPred, freely available at http://bio-aims.udc.es/LECTINPred.php. This web server showed the following goodness-of-fit statistics: Sensitivity=96.7 % (for Ls), Specificity=87.6 % (non-active proteins), and Accuracy=92.5 % (for all proteins), considering altogether both the training and external prediction series. In mode 2, users can carry out an automatic retrieval of protein structures from PDB. We illustrated the use of this server, in operation mode 1, performing a data mining of PDB. We predicted Ls scores for +2000 proteins with unknown function and selected the top-scored ones as possible lectins. In operation mode 2, LECTINPred can also upload 3D structural models generated with structure-prediction tools like LOMETS or PHYRE2. The new Ls are expected to be of relevance as cancer biomarkers or useful in parasite vaccine design.

Structural genomics provides an important approach for characterizing and understanding systems biology. As a step toward better integrating protein three-dimensional (3D) structural information in cancer systems biology, we have constructed a Human Cancer Pathway Protein Interaction Network (HCPIN) by analysis of several classical cancer-associated signaling pathways and their physical protein-protein interactions. Many well known cancer-associated proteins play central roles as "hubs" or "bottlenecks" in the HCPIN. At least half of HCPIN proteins are either directly associated with or interact with multiple signaling pathways. Although some 45% of residues in these proteins are in sequence segments that meet criteria sufficient for approximate homology modeling (Basic Local Alignment Search Tool (BLAST) E-value <10(-6)), only approximately 20% of residues in these proteins are structurally covered using high accuracy homology modeling criteria (i.e. BLAST E-value <10(-6) and at least 80% sequence identity) or by actual experimental structures. The HCPIN Website provides a comprehensive description of this biomedically important multipathway network together with experimental and homology models of HCPIN proteins useful for cancer biology research. To complement and enrich cancer systems biology, the Northeast Structural Genomics Consortium is targeting >1000 human proteins and protein domains from the HCPIN for sample production and 3D structure determination. The long range goal of this effort is to provide a comprehensive 3D structure-function database for human cancer-associated proteins and protein complexes in the context of their interaction networks. The network-based target selection (BioNet) approach described here is an example of a general strategy for targeting co-functioning proteins by structural genomics projects.

BackgroundStructural information about epitopes, particularly the three-dimensional (3D) structures of antigens in complex with immune receptors, presents a valuable source of data for immunology. This information is available in the Protein Data Bank (PDB) and provided in curated form by the Immune Epitope Database and Analysis Resource (IEDB). With continued growth in these data and the importance in understanding molecular level interactions of immunological interest there is a need for new specialized molecular visualization and analysis tools.ResultsThe EpitopeViewer is a platform-independent Java application for the visualization of the three-dimensional structure and sequence of epitopes and analyses of their interactions with antigen-specific receptors of the immune system (antibodies, T cell receptors and MHC molecules). The viewer renders both 3D views and two-dimensional plots of intermolecular interactions between the antigen and receptor(s) by reading curated data from the IEDB and/or calculated on-the-fly from atom coordinates from the PDB. The 3D views and associated interactions can be saved for future use and publication. The EpitopeViewer can be accessed from the IEDB Web site through the quick link 'Browse Records by 3D Structure.'ConclusionThe EpitopeViewer is designed and been tested for use by immunologists with little or no training in molecular graphics. The EpitopeViewer can be launched from most popular Web browsers without user intervention. A Java Runtime Environment (RJE) 1.4.2 or higher is required.

As commodity virtual reality and augmented reality hardware becomes more accessible, the opportunity to use these systems for learning and training will increase. This study provides an exploratory look at performance differences for a simple memory matching task across four different technologies that could easily be used for learning and training. We compare time and number of attempts to successfully complete a memory matching game across virtual reality, augmented reality, a large touchscreen table-top display and a real environment. The results indicate that participants took more time to complete the task in both the augmented reality and real conditions. Augmented reality and real environments were statistically different than the fastest two conditions, which occurred in the virtual reality and table-top touch display conditions.

X‐ray structure of <i>Saccharomyces cerevisiae</i> homologous mitochondrial matrix factor 1 (Hmf1)

ConoMode is a database for complex three-dimensional (3D) structures of conopeptides binding with their target proteins. Conopeptides, a large family of peptides from the venom of marine snails of the Conus genus, have exceptionally diverse sequences, and their high specificity to block ion channels makes them crucial as drug leads and tools for physiological studies. ConoMode is a specialized archive for the collection of 3D coordinate data for the conopeptides and their binding target proteins from published literature and the Protein Data Bank. These 3D structures can be determined using experimental methods such as X-ray crystallography and electron microscopy and computational methods including docking, homology modeling and molecular dynamics simulations. The binding modes for the conopeptides determined using computational modeling must be validated based on experimental data. The 3D coordinate data from ConoMode can be searched, visualized, downloaded and uploaded. Currently, ConoMode manages 19 conopeptide sequences (from 10 Conus species), 15 protein sequences and 37 3D structures. ConoMode utilizes a modern technical framework to provide a good user experience on mobile devices with touch interaction features. Furthermore, the database is fully optimized for unstructured data and flexible data models.Database URL: http://conomode.qnlm.ac/conomode/conomode/index

The perceptual behavior of consumers on a product displayed in the market has a vital role in analyzing the importance given to that product. Therefore, various strategies have been developed to understand this consumer behavior in the selection of products. Immersive technologies like virtual, augmented, and mixed reality are among them. With the foremost feature of immersion in the virtual world and interaction of users with virtual objects, virtual reality, and augmented reality have unlocked their potential in research and a user-friendly tool for analyzing consumer behavior. In addition to these technologies, mixed reality also has a significant role in investigating consumer behavior. Studies on immersive technologies in food applications are vast, hence this review focuses on the applications of virtual, augmented, and mixed reality in the food selection behavior of consumers. The behavioral studies are elicited to develop new products based on consumer needs, to understand the shopping behavior in supermarkets for real-time usage, and to know the influence of emotions in a selection of products. The findings suggest that virtual, augmented, and mixed reality induce immersion of the users in food selection behavioral studies. Information on the technological advancements in the tools used for bringing immersion and interaction are discussed for its futuristic applications in food. Though immersive technology gives users a realistic virtual environment experience, its application in food systems is in the budding stage. More research on human response studies would contribute to its innovative and inevitable application in the future.

Inherent to the field of computer-aided surgery (CAS) is the necessity to handle sophisticated equipment in the operating room; an undesired side-effect of this development is the fact that the surgeon’s attention is drawn from the operating field since surgical progress is partially monitored on the computer’s screen. The overlay of computer-generated graphics over a real-world scene, also referred to as augmented reality (AR), provides a possibility to solve this problem. The considerable technical problems associated with this approach such as viewing of the scenery within a common focal range on the head-mounted display (HMD) or latency in display on the HMD have, however, kept AR from widespread usage in CAS. In this paper, the concept of the Varioscope AR, a lightweight head-mounted operating microscope used as a HMD, is introduced. The registration of the patient to the preoperative image data as well as preoperative planning take place on VISIT, a surgical navigation system developed at our hospital. Tracking of the HMD and stereoscopic visualisation take place on a separate POSIX.4 compliant realtime operating system running on PC hardware. While being in a very early stage of laboratory testing, we were able to overcome the technical problems described above; our work resulted in an AR visualisation system with an update rate of 6 Hz and a latency below 130 ms. First tests with 2D/3D registration have shown a match between 3D world coordinates and 2D HMD display coordinates in the range of 1.7 pixels. It integrates seamlessly into a surgical navigation system and provides a common focus for both virtual and real world objects. On the basis of our current results, we conclude that the Varioscope AR with the realtime visualisation unit is a major step towards the introduction of AR into clinical routine.

Phantom motor execution facilitated by machine learning and augmented reality as treatment for phantom limb pain: a single group, clinical trial in patients with chronic intractable phantom limb pain

Nowadays, Augmented Reality flourishes in educational settings. Yet, little is known about teachers’ and children’s views of Augmented Reality applications in Preschool. This paper explores 71 preschoolers’ opinions of Augmented Reality teaching integrated into a traditional learning activity. Additionally, five educators’ views of Augmented Reality applications in Preschool are captured. Mixed methods with questionnaires and semi-structured interviews were used. The questionnaires record children’s preferences regarding their favorite learning activity between traditional and the Augmented Reality one. Additionally, they explore the activity preschoolers would like to repeat and found most enjoyable: playful. Regarding quantitative data analysis, independent/paired samples t-tests and chi-square test along with bootstrapping with 1000 samples were used. As for the qualitative data collection, educators’ semi-structured interviews focused on three axes: (a) children’s motivation and engagement in Augmented Reality activities, (b) Augmented Reality’s potential to promote skills, and (c) Augmented Reality as a teaching tool in preschool. The emerging results are: Preschoolers prefer more Augmented Reality activities than traditional ones. There are no statistically significant gender differences in preferences for Augmented Reality activities. Educators regard Augmented Reality technology as an innovative, beneficial teaching approach in preschool. However, they express concern regarding the promotion of collaboration among preschoolers via Augmented Reality.

Integrating Augmented Reality in the Assembly Domain - Fundamentals, Benefits and Applications

The goal of this review was to identify and summarize technology-assisted methods that assess nonverbal behaviors known to influence the quality of the therapeutic alliance between health care providers and patients in clinical, research, or educational settings. Strong therapeutic alliances in health care provider-patient relationships contribute to positive patient outcomes. Previous studies have shown that nonverbal behaviors from providers, such as tone of voice, facial expression, eye contact, and physical positioning, help build strong therapeutic relationships. Technological advances have created opportunities to automate detection and feedback about nonverbal behaviors in clinical interactions without the intervention of highly trained instructors. Automated detection and feedback will increase opportunities to develop better nonverbal communication, which is expected to improve the quality of the therapeutic alliance. Participants were current or aspiring health care providers who routinely would be expected to have patient encounters in clinical, educational, or research settings. Interactions could include actual patient interactions, model patients, virtual patients, or simulated patients. Included studies involved technology-assisted methods to collect or analyze at least 1 nonverbal behavior. This review included qualitative and quantitative studies and review articles. PubMed, Embase, CINAHL, ERIC, Scopus, and Google Scholar were searched for published and unpublished literature in English between 2010 and 2023. Two reviewers independently completed title and abstract screening, full-text review, and data extraction. Any conflicts that arose were resolved by a third reviewer. Extraction was completed by 2 reviewers. Twenty-five sources were included. The most frequently measured behaviors included gaze frequency or duration on the patient's face (52% of studies) and facial expression (48%), with 24% of studies measuring social touch or speech patterns, 20% measuring posture and gestures, 12% measuring proximity to patient, 8% measuring synchrony, and 4% measuring voice. Technologies for data collection included audio or video recordings and other technologies such as eye-tracking glasses or multi-functional headsets, used alone or integrated into augmented reality or virtual reality systems. Challenges associated with analyzing data included recognition of patients' and providers' voices, facial features, and body parts, and classifying body posture and movement as well as facial expression and movements. Studies used commercial, open source, or customized software to meet each challenge. Many incorporated machine learning or algorithms associated with artificial intelligence (AI), which required significant training data. Technology-assisted methods for analyzing voice and social touch were limited, whereas there were several available analysis techniques for analyzing synchrony, gestures, posture, and proximity to patients. The use of technology in collection and analysis of provider-patient interactions to enhance nonverbal behaviors and communication skills is rapidly developing, with multiple options available. Future efforts should focus on technology-assisted methods to analyze voice and social touch, and should transparently report contributing factors and training data for machine learning models or AI-associated algorithms. Development of systems will require significant testing to ensure accessibility to the intended audience and broadly accurate and meaningful nonverbal behaviors. OSF ( https://osf.io/kedmf ).

Virtual reality and augmented reality have the potential to enhance and widespread practical learning environments in professional courses efficiently in a cost-efficient manner by limiting the costs of real resources substituting them with fixed costs from.VR/AR applications with virtual resources. There are advantages in the learning process, as practical, active and visual learning methods are more efficient and virtual and augmented reality can digitalize these procedures and replicate them at scale with different degrees of virtualization. In this work we aim to provide a framework that allows the creation of VR and AR experiences for learning or training proposes in a serious environment adding gamification elements to keep user engaged in the learning/training process. In the process gamification adaptation to VR/AR environment is demonstrated in real applications. The learning tasks in this approach are not necessarily changed or take advantage of new forms of interactions and guidance but aim to be replicated in a blend between virtual and real environments. In this regard, we hope to advance gamification application to account for more elements, such as VR/AR interaction, digital twins and digital aids in a learning application. In this article we detail possible scenarios for the application of virtual reality and augmented reality combined with machine learning in serious games and learning scenarios.