Perceptual-Similarity-Aware Deep Speaker Representation Learning for Multi-Speaker Generative Modeling

Increasing numbers of software vulnerabilities are discovered every year whether they are reported publicly or discovered internally in proprietary code. These vulnerabilities can pose serious risk of exploit and result in system compromise, information leaks, or denial of service. We leveraged the wealth of C and C++ open-source code available to develop a largescale function-level vulnerability detection system using machine learning. To supplement existing labeled vulnerability datasets, we compiled a vast dataset of millions of open-source functions and labeled it with carefully-selected findings from three different static analyzers that indicate potential exploits. Using these datasets, we developed a fast and scalable vulnerability detection tool based on deep feature representation learning that directly interprets lexed source code. We evaluated our tool on code from both real software packages and the NIST SATE IV benchmark dataset. Our results demonstrate that deep feature representation learning on source code is a promising approach for automated software vulnerability detection.

A Comprehensive Survey on Deep Graph Representation Learning

Event Abstract Back to Event Quality of synthetic speech and auditory working memory performance: neuroergonomic perspectives from fNIRS Adrian Curtin1, 2* and Hasan Ayaz1, 3, 4 1 School of Biomedical Engineering, Science and Health Systems, Drexel University, United States 2 Shanghai Jiao Tong University, School of Biomedical Engineering, China 3 University of Pennsylvania, Department of Family and Community Health, United States 4 Children's Hospital of Philadelphia, The Division of General Pediatrics, United States The way in which we effortlessly understand speech makes it relatively easy to overlook the complexity of the neural mechanisms which support its operation. This perceived ease of speech comprehension has inspired researchers and interface designers seek to incorporate additional auditory channels of information to consumer and industrial devices, especially in circumstances where the user of the device is otherwise engaged in manual or visual tasks such as handsfree communication while driving. However, previous research has demonstrated that there are unconscious cognitive costs associated with simple speech comprehension which can be exacerbated by task performance(Treffner & Barrett, 2004) and audio quality(Delogu, Conte, & Sementina, 1998; Francis & Nusbaum, 2009). In a previous exploratory work, we have observed performance deficits and neural activation differences due to low-quality synthetic speech during speech comprehension tasks(Curtin & Ayaz, 2017), and here we extend this exploration by examining how the quality of synthetic speech affects auditory working memory (AWM) performance and neural load using functional Near Infrared Spectroscopy (fNIRS), a continuous and noninvasive neuroimaging technique capable of investigating brain dynamics under naturalistic and applied settings (Ayaz et al., 2013). Working memory is an important executive function in which a limited storage system is updated and maintained during the execution of other processing tasks. As a limited resource, tasks which require significant working memory resources are especially sensitive to distortions and other factors which may competitively demand additional attention(Banich, 2009), hence the development of systems and user interfaces must carefully consider added cognitive burdens imposed as part of the design process. Beyond design considerations, the wide variety of situations which require auditory working memory (AWD) and the way in which degraded audio quality adversely can affect these has important implications in day-to-day life whether the degraded audio signal is due to environmental noise, hearing loss, distortion produced by hearing aid devices, or low-quality synthetic speech. Here we present a neuroergonomic evaluation on the effects of synthetic speech quality during an auditory working memory task while functional Near-Infrared Spectroscopy (fNIRS) was used to noninvasively monitor both the dorsolateral-Prefrontal Cortex (dlPFC) and the left-Temporal-Parietal Junction region (TPJ) which includes the primary auditory cortex, Broca’s area and Wernicke’s area. Sixteen right-handed healthy controls with no history of neurological or mental illness were recruited to participate in this study. Participants completed an auditory version of the N-Back task using vocalized letters as stimuli while the audio quality was manipulated by using low-quality and high-quality synthetic voices. Results show that reaction time was significantly increased by the presentation of low-quality synthetic speech under both the 0-Back and 2-Back conditions. Additionally, increased working memory load was associated with behavioral performance decreases in terms of accuracy (Figure 1, F(1,225.12)=8.34,p<0.005) and reaction time (Figure 2, F(1,22.46)=30.32,p<0.001) and neural differences as measured by changes in oxygenated hemoglobin [HbO] as measured by fNIRS in both the left-dlPFC (F(1_200.43)=33.2,p<0.001) and the left-TPJ (Figure 3, F(1,160)=8.32,p<0.005), near Wernicke’s Area. Our results suggest both that the interpretation of low-quality voices results in significant behavioral deficits in terms of reaction time (F(1,22.33)=29.91, p<0.001) and that increased AWM load is reflected by changes in activation in not only areas of executive function, but also associative auditory areas. These preliminary results highlight the potential significant impact of neuroimaging for usability and user interface research. Various characteristics of audio user interface can negatively impact cognitive workload in ways that should be accounted for during the design of complex systems or interfaces. Wearable and mobile brain function monitoring can be used to detect these otherwise unobserved characteristics. Figure 1: Auditory N-Back accuracy vs Workload and Synthetic Audio Quality. Figure 2: Mean reaction time during Auditory N-Back vs Workload and Synthetic Audio Quality. Figure 3: Mean [HbO] changes in Left-Temporal region vs Working Memory Load and Synthetic Audio Quality Figure 1 Figure 2 Figure 3 Acknowledgements The authors would like to thank Dr. Paul Crawford for his help with this study. This study is made possible by a research award from the Intel Corporation. The content of the information herein does not necessarily reflect the position or the policy of the sponsor and no official endorsement should be inferred. Keywords: auditory working memory, fNIRS, n-back, synthetic speech, Prefrontal Cortex, Auditory Cortex Conference: 2nd International Neuroergonomics Conference, Philadelphia, PA, United States, 27 Jun - 29 Jun, 2018. Presentation Type: Poster Presentation Topic: Neuroergonomics Citation: Curtin A and Ayaz H (2019). Quality of synthetic speech and auditory working memory performance: neuroergonomic perspectives from fNIRS. Conference Abstract: 2nd International Neuroergonomics Conference. doi: 10.3389/conf.fnhum.2018.227.00039 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 03 Apr 2018; Published Online: 27 Sep 2019. * Correspondence: Mr. Adrian Curtin, School of Biomedical Engineering, Science and Health Systems, Drexel University, Philadelphia, PA, United States, abc48@drexel.edu Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Adrian Curtin Hasan Ayaz Google Adrian Curtin Hasan Ayaz Google Scholar Adrian Curtin Hasan Ayaz PubMed Adrian Curtin Hasan Ayaz Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.

Most existing network representation learning algorithms focus on network structures for learning. However, network structure is only one kind of view and feature for various networks, and it cannot fully reflect all characteristics of networks. In fact, network vertices usually contain rich text information, which can be well utilized to learn text-enhanced network representations. Meanwhile, Matrix-Forest Index (MFI) has shown its high effectiveness and stability in link prediction tasks compared with other algorithms of link prediction. Both MFI and Inductive Matrix Completion (IMC) are not well applied with algorithmic frameworks of typical representation learning methods. Therefore, we proposed a novel semi-supervised algorithm, tri-party deep network representation learning using inductive matrix completion (TDNR). Based on inductive matrix completion algorithm, TDNR incorporates text features, the link certainty degrees of existing edges and the future link probabilities of non-existing edges into network representations. The experimental results demonstrated that TFNR outperforms other baselines on three real-world datasets. The visualizations of TDNR show that proposed algorithm is more discriminative than other unsupervised approaches.

A deep multi-task representation learning method for time series classification and retrieval

State representation learning for control: An overview

Image classification serves as the cornerstone of computer vision, traditionally achieved through discriminative models based on deep neural networks. Recent advancements have introduced classification methods derived from generative models, which offer the advantage of zero-shot classification. However, these methods suffer from two main drawbacks: high computational overhead and inferior performance compared to discriminative models. Inspired by the coordinated cognitive processes of rapid-slow pathway interactions in the human brain during visual signal recognition, we propose the Diffusion-Based Discriminative Model Enhancement Framework (DBMEF). This framework seamlessly integrates discriminative and generative models in a training-free manner, leveraging discriminative models for initial predictions and endowing deep neural networks with rethinking capabilities via diffusion models. Consequently, DBMEF can effectively enhance the classification accuracy and generalization capability of discriminative models in a plug-and-play manner. We have conducted extensive experiments across 17 prevalent deep model architectures with different training methods, including both CNN-based models such as ResNet and Transformer-based models like ViT, to demonstrate the effectiveness of the proposed DBMEF.Specifically, the framework yields a 1.51% performance improvement for ResNet-50 on the ImageNet dataset and 3.02% on the ImageNet-A dataset. In conclusion, our research introduces a novel paradigm for image classification, demonstrating stable improvements across different datasets and neural networks.

AIMAFE: Autism spectrum disorder identification with multi-atlas deep feature representation and ensemble learning

Voice cloning is used to generate synthetic speech that mimics vocal characteristics of human talkers. This experiment used voice cloning to compare human and synthetic speech for intelligibility, human-likeness, and perceptual similarity, all tested in young adults with normal hearing. Masked-sentence recognition was evaluated using speech produced by five human talkers and their synthetically generated voice clones presented in speech-shaped noise at −6 dB signal-to-noise ratio. There were two types of sentences: semantically meaningful and nonsense. Human and automatic speech recognition scoring was used to evaluate performance. Participants were asked to rate human-likeness and determine whether pairs of sentences were produced by the same versus different people. As expected, sentence-recognition scores were worse for nonsense sentences compared to meaningful sentences, but they were similar for speech produced by human talkers and voice clones. Human-likeness scores were also similar for speech produced by human talkers and their voice clones. Participants were very good at identifying differences between voices but were less accurate at distinguishing between human/clone pairs, often leaning towards thinking they were produced by the same person. Reliability scoring by automatic speech recognition agreed with human reliability scoring for 98% of keywords and was minimally dependent on the context of the target sentences. Results provide preliminary support for the use of voice clones when evaluating the recognition of human and synthetic speech. More generally, voice synthesis and automatic speech recognition are promising tools for evaluating speech recognition in human listeners.

We propose a cross-domain latent modulation mechanism within a variational autoencoders (VAE) framework to enable improved transfer learning. Our key idea is to procure deep representations from one data domain and use it as perturbation to the reparameterization of the latent variable in another domain. Specifically, deep representations of the source and target domains are first extracted by a unified inference model and aligned by employing gradient reversal. Second, the learned deep representations are cross-modulated to the latent encoding of the alternate domain. The consistency between the reconstruction from the modulated latent encoding and the generation using deep representation samples is then enforced in order to produce inter-class alignment in the latent space. We apply the proposed model to a number of transfer learning tasks including unsupervised domain adaptation and image-to-image translation. Experimental results show that our model gives competitive performance.

With the rapid advancement of DNA sequencing, metagenomics and metatranscriptomics have made great progress, which deepen our understanding on the human microbiome and its impact on human health and diseases. The microbiome, which is characterized by small samples, high dimensions and complicated relationships with hosts, refers to the species, genes and genomes of the microbiota, as well as the products of the microbiota and the host environment. In fact, many machine learning methods have been used to conduct Microbiome-Wide Association Studies which can link the microbiome with the phenotypes, such as the status of human health and diseases. However, existing methods such as Support Vector Machines (SVMs) have some limitations on deep representation learning with deep architectures which can promote the reuse of features and potentially lead to progressively more abstract features at higher layers of representations. Recently, Deep Neural Networks (DNNs), a kind of deep learning models, are widely used for metagenomic data analysis and can perform well on representation learning. But they are considered as a black box and sufferring from criticisms due to theirs lacking of interpretability. Thus, it is interesting to explore other deep learning models for metagenomic data analysis. In this work, we introduce a deep learning model called Deep Forest to study the microbiome associations and we also present an ensemble method for feature selection. Experimental results show that Deep Forest outperforms the traditional machine learning methods. In addition, compared to DNNs, Deep Forest has better interpretability and less hyperparameters.

Learning deep discriminative representations with pseudo supervision for image clustering

Generative models offer advantageous characteristics for classification tasks, such as the availability of unsupervised data and calibrated confidence. In contrast, discriminative models have advantages in terms of their potential to outperform their generative counterparts and the simplicity of their model structures and learning algorithms. In this article, we propose a method to train a hybrid of discriminative and generative models in a single neural network (NN), which exhibits the characteristics of both models. The key idea is the Gaussian-coupled softmax layer, which is a fully connected layer with a softmax activation function coupled with Gaussian distributions. This layer can be embedded into an NN-based classifier and allows the classifier to estimate both the class posterior distribution and the input data distribution. We demonstrate that the proposed hybrid model can be applied to semi-supervised learning and confidence calibration.

Despite the continuous evolution and significant improvement of cybersecurity mechanisms, malware threats remain one of the most important concerns in cyberspace. Meanwhile, Android malware plays a big role in these ever-growing threats. In recent years, deep learning has become the dominant machine learning technique for malware detection and continues to make outstanding achievements. Deep graph representation learning is the task of embedding graph-structured data into a low-dimensional space using deep learning models. Recently, autoencoders have proven to be an effective way for deep representation learning. However, it is not straightforward to apply the idea of autoencoder to graph-structured data because of their irregular structure. In this paper, we present DroidMalGNN, a novel deep learning technique that combines autoencoders with graph neural networks (GNNs) to detect Android malware in an end-to-end manner. DroidMalGNN represents each Android application with an attributed function call graph (AFCG) that allows it to model complex relationships between data. For more efficiency, DroidMalGNN performs graph representation learning in a supervised manner where two autoencoders are trained with benign and malicious AFCGs separately. In this way, it generates two informative embedding vectors for each AFCG in a low-dimensional space and feeds them into a dense neural network to classify the AFCG as benign or malicious. Our experimental results show that DroidMalGNN can achieve good detection performance in terms of different evaluation measures.

Nowadays, with the success and fast growth of social media communities and mobile devices, people are encouraged to share their multimedia data online. Analyzing and summarizing data into useful information thus becomes increasingly important. For online photo sharing services like Flickr, when users are uploading a batch of daily photos at a time, the tags users provided tend to be rather vague, containing only a small amount of information. For better photo application and understanding, we attempt to automatically discover semantic-rich (hidden) aspects of photos merely by looking at image contents. In this paper, we propose an effective model, which is a combination of LDA model and deep learning representations, to realize the idea of automatic aspect discovery. We then discuss the properties of this aspect discovery model through experiments on event summarization task. In those experiments, we show the high diversity and high quality of aspects discovered by our proposed method. Meanwhile, we conduct an user study to evaluate the quality of the summarized results. Moreover, the proposed method can be further extended to human attribute discovery for a given event. We automatically discover different aspects on our Olympic Games data (e.g. football, ice skating).