• Home
  • Search
  • Deep Gaussian process based multi-speaker speech synthesis with latent speaker representation
  • Cite Icon7
  • https://doi.org/10.1016/j.specom.2021.07.001Copy DOI Icon

Deep Gaussian process based multi-speaker speech synthesis with latent speaker representation

Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

This paper proposes deep Gaussian process (DGP)-based frameworks for multi-speaker speech synthesis and speaker representation learning. A DGP has a deep architecture of Bayesian kernel regression, and it has been reported that DGP-based single speaker speech synthesis outperforms deep neural network (DNN)-based ones in the framework of statistical parametric speech synthesis. By extending this method to multiple speakers, it is expected that higher speech quality can be achieved with a smaller number of training utterances from each speaker. To apply DGPs to multi-speaker speech synthesis, we propose two methods: one using DGP with one-hot speaker codes, and the other using a deep Gaussian process latent variable model (DGPLVM). The DGP with one-hot speaker codes uses additional GP layers to transform speaker codes into latent speaker representations. The DGPLVM directly models the distribution of latent speaker representations and learns it jointly with acoustic model parameters. In this method, acoustic speaker similarity is expressed in terms of the similarity of the speaker representations, and thus, the voices of similar speakers are efficiently modeled. We experimentally evaluated the performance of the proposed methods in comparison with those of conventional DNN and variational autoencoder (VAE)-based frameworks, in terms of acoustic feature distortion and subjective speech quality. The experimental results demonstrate that (1) the proposed DGP-based and DGPLVM-based methods improve subjective speech quality compared with a feed-forward DNN-based method, and (2) even when the amount of training data for target speakers is limited, the DGPLVM-based method outperforms other methods, including the VAE-based one. Additionally, (3) by using a speaker representation randomly sampled from the learned speaker space, the DGPLVM-based method can generate voices of non-existent speakers.

Similar Papers
  • Conference Article
  • Citations3

Multi-Speaker Text-to-Speech Synthesis Using Deep Gaussian Processes

  • Oct 25, 2020
  • Kentaro Mitsui +2
  • Conference Article
  • Citations2

Waveform-Based Speaker Representations for Speech Synthesis

  • Sep 02, 2018
  • Moquan Wan +2
  • Conference Article
  • Citations49

An investigation of augmenting speaker representations to improve speaker normalisation for DNN-based speech recognition

  • Apr 01, 2015
  • Hengguan Huang +1
  • Conference Article
  • Citations17

Deep Speaker Representation Using Orthogonal Decomposition and Recombination for Speaker Verification

  • May 01, 2019
  • Insoo Kim +3
  • Conference Article
  • Citations7

Utterance-Level Sequential Modeling for Deep Gaussian Process Based Speech Synthesis Using Simple Recurrent Unit

  • Apr 11, 2020
  • Tomoki Koriyama +1
  • Conference Article
  • Citations5

A Training Method Using DNN-guided Layerwise Pretraining for Deep Gaussian Processes

  • May 01, 2019
  • Tomoki Koriyama +1
  • Research Article
  • Citations32

Empirical Assessment of Deep Gaussian Process Surrogate Models for Engineering Problems

  • Sep 18, 2020
  • Journal of Aircraft
  • Dushhyanth Rajaram +6
  • Conference Article
  • Citations1

Automatically Designed Deep Gaussian Process for Turbomachinery Application

  • Jun 07, 2021
  • Yuan Jin +2
  • Research Article
  • Citations32

A new optical density granulometry-based descriptor for the classification of prostate histological images using shallow and deep Gaussian processes

  • Jul 04, 2019
  • Computer Methods and Programs in Biomedicine
  • Ángel E Esteban +5
  • Research Article

Text Line Recognition of Dai Language using Statistical Characteristics of Texture Analysis and Deep Gaussian Process

  • May 18, 2021
  • International Journal of Circuits, Systems and Signal Processing
  • Jingying Zhao +4
  • PDF
  • Research Article
  • Citations17

Deep state-space Gaussian processes

  • Sep 23, 2021
  • Statistics and Computing
  • Zheng Zhao +2
  • Research Article

Abstract 5012: Deep Gaussian process with uncertainty estimation Improves microsatellite instability prediction based on whole slide image: A retrospective multicenter and multiethnic cohort study

  • Jun 15, 2022
  • Cancer Research
  • Sunho Park +4
  • Book Chapter
  • Citations3

Multi-view Deep Gaussian Processes

  • Jan 01, 2018
  • Shiliang Sun +1
  • Research Article
  • Citations66

Bayesian optimization using deep Gaussian processes with applications to aerospace system design

  • Jun 12, 2020
  • Optimization and Engineering
  • Ali Hebbal +4
  • Research Article
  • Citations84

Multi-View Representation Learning With Deep Gaussian Processes.

  • Jun 10, 2020
  • IEEE Transactions on Pattern Analysis and Machine Intelligence
  • Shiliang Sun +2
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.