Adversarial Learning to Reduce Sources of Variability in Speech Applications

Path-based multi-hop reasoning over knowledge graph for answering questions via adversarial reinforcement learning

Speech technology is rapidly developing and has improved a lot over the last few years. Nevertheless, speech-enabled applications have not yet become mainstream software. Furthermore, there is a lack of proven design methods and methodologies specifically concerning speech applications. So far the application of speech technology has only been a limited success. This Paper describes a project done at CMG Trade Transport & Industry BV. It is called SWAMP and is an example of the application of speech technology in human-computer interaction. The reasoning model behind the speech interface is based on the Belief Desire Intention (BDI) model for rational agents. Other important tools that were used to build the speech user interface are the Microsoft Speech API 5 and CLIPS.

The notion of a grammar as a relation between sets of strings is quite ancient. In the 4th century BCE, Pāṇini—Pynini’s eponym—proposed a sophisticated computational system (see Penn and Kiparsky 2012) in which Sanskrit words and phrases are constructed by repeated application of rewrite rules. These rules consist of a target (for example, in Aṣṭdhyāyā 6.1.77, [i, i:, u:,ṛ, ṛ.:]), a replacement (the corresponding semivowels), and a context in which it is applied (before another vowel). Priscian, writing in the 6th century CE, proposes a system of rewrite rules generating inflectional variants of Latin verbs (Matthews 1972). In the 19th century, historical linguists—particularly those now known as the Neogrammarians—employed rewrite rules to describe language change. For instance, Grimm’s Law correctly predicts that the initial *p in the Proto-Indo-European word meaning ‘father’ (cf. Latin pater) surfaces as [f] in Germanic (e.g., Old High German fater). In the 20th century, linguists adapted rewrite rules—some of which closely resemble historical sound changes—to sketch the morphophonemics of languages such as Menomini (Bloomfield 1939), Russian (Jakobson 1948), and Modern Hebrew (Chomsky 1951), and began to formalize rewrite systems. One particularly influential formal approach is the phonological rule notation popularized by Chomsky and Halle (1968). Subsequent work, reviewed below, establishes strong connections between the notation used by Chomsky and Halle and the regular languages,1 provides algorithms to compile rewrite rules into finite-state transducers, and employs them for speech and language technology applications. This chapter formalizes rewrite rules and shows how they are compiled into finite transducers, combined into rule cascades, and applied to input strings. Their use is then illustrated with applications in speech and language technology, including grapheme-to-phoneme conversion, morphological generation, and text normalization; the latter two applications are discussed in greater detail in later chapters.

Adversarial contrastive learning (ACL) is considered an effective way to improve the robustness of pre-trained models. In contrastive learning, a projector which consists of multilayer perceptron (MLP) will project high dimension 3D point cloud feature into low dimension for calculating contrastive loss during contrastive pretraining.We propose a novel method for generating high-quality 3D adversarial examples for adversarial training, which leverages the virtual adversarial loss with the feature representations prior to projection in a contrastive learning framework. To train the self-supervised contrastive learning framework adversarially, we introduce our robust aware loss function. Additionally, we show that incorporating high difference points using the Difference of Normal (DoN) operator as an additional input for adversarial self-supervised contrastive learning can significantly enhance the adversarial robustness of the pre-trained model. Our proposed method, POINTACL, is evaluated on several downstream tasks, including 3D classification and 3D segmentation using multiple datasets. Our experimental results demonstrate that POINTACL achieves state-of-the-art performance in terms of robust accuracy when compared to other contrastive adversarial learning methods.

Many data mining applications potentially operate in an adversarial environment where adversaries adapt their behavior to evade detection. Typically adversaries alter data under their control to cause a large divergence of distribution between training and test data. Existing state-of-the-art adversarial learning techniques try to address this problem in which there is only a single type of adversary. In practice, a learner often has to face multiple types of adversaries that may employ different attack tactics. In this paper, we tackle the challenges of multiple types of adversaries with a nested Stackelberg game framework. We demonstrate the effectiveness of our framework with extensive empirical results on both synthetic and real data sets. Our results demonstrate that the nested game framework offers more reliable defense against multiple types of attackers.

La ejecución contencioso-administrativa resulta infravalorada en su esfuerzo resolutorio en contraposición a la tutela judicial declarativa. Ello resulta particularmente evidente con ocasión de impugnaciones de procesos selectivos, ámbito en el que suele subestimarse la carga ínsita a una humilde pretensión, la cual, en ocasiones, contará con fuerza para estremecer los pilares del proceso selectivo con el que cabe relacionarla. No sobrará aquilatar el fallo en orden a los aspectos que hayan de derivar de él ni desechar la posible operatividad de la tutela cautelar con el fin de evitar las perniciosas consecuencias que quepa anticipar ante los decursos procedimentales y procesales vinculados con la depuración del asunto. Como si fuésemos niños hurgando con un lápiz en una telaraña, no debemos olvidar que la vibración que provoquemos tenderá de forma natural a extenderse, desde el punto en el que el lápiz contactó con tal estructura hasta ocupar toda su extensión.

Written text is based on an orthographic representation of words, i.e. linear sequences of letters. Modern speech technology (automatic speech recognition and text-to-speech synthesis) is based on phonetic units representing realization of sounds. A mapping between the orthographic form and phonetic forms representing the pronunciation is thus required. This may be obtained by creating pronunciation lexica and/or rule-based systems for grapheme-to-phoneme conversion. Traditionally, this mapping has been obtained manually, based on phonetic and linguistic knowledge. This approach has a number of drawbacks: i) the pronunciations represent typical pronunciations and will have a limited capacity for describing pronunciation variation due to speaking style and dialectical/accent variations; ii) if multiple pronunciation variants are included, it does not indicate which variants are more significant for the specific application; iii) the description is based on phonetic-knowledge and does not take into account that the units used in speech technology may deviate from the phonetic interpretation; and iv) the description is limited to units with a linguistic interpretation. The paper will present and discuss methods for modeling pronunciation and pronunciation variation specifically for applications in speech technology.

Pathological diagnosis assists in saving human lives, but such models are annotation hungry and pathological images are notably expensive to annotate. Contrastive learning could be a promising solution that relies only on the unlabeled training data to generate informative representations. However, the majority of current methods in contrastive learning have the following two issues: (1) positive samples produced through random augmentation are less challenging, and (2) false negative pairs problem caused by negative sampling bias. To alleviate the above issues, we propose a novel contrastive learning method called Cluster-Aware Adversarial Contrastive Learning (CA2CL). Specifically, a mixed data augmentation technique is provided to learn more transferable representations by generating more discriminative sample pairs. Furthermore, to mitigate the effects of inherent false negative pairs, we adopt a cluster-aware loss to identify similarities between instances and incorporate them into the process of contrastive learning. Finally, we generate challenging contrastive data pairs by adversarial learning, and adversarially learn robust representations in the representation space without the labeled training data, which aims to maximize the similarity between the augmented sample and the related adversarial sample. Our proposed CA2CL is evaluated on two public datasets: NCT-CRC-HE and PCam for the fine-tuning and linear evaluation tasks and on two other public datasets: GlaS and CARG for the detection and segmentation tasks, respectively. Extensive experimental results demonstrate the superior performance improvement of our method over several Self-supervised learning (SSL) methods and ImageNet pretraining particularly in scenarios with limited data availability for all four tasks.

Active consistency network for multi-source domain generalization in brain tumor segmentation

A lo largo del siglo XVI, los tlaxcaltecas crearon un nutrido corpus de documentos legales, ceremonias y pinturas en los que ensalzaron su papel como conquistadores aliados ya cristianizados, y con el que afianzaron una memoria colectiva vinculada a su destacada participación en la conquista de México-Tenochtitlan. Un ejemplo de dicho corpus se encuentra en los Cantares mexicanos (ca. 1597), cancionero novohispano en náhuatl clásico conservado en la Biblioteca Nacional de México. Se trata de los dos cantares titulados Tlaxcaltecayotl, que narran episodios asociados a la guerra contra Tenochtitlan desde el punto de vista tlaxcalteca. Dadas las características performativas y el formato de ambos cantos, el presente artículo tiene como objetivo analizar los aspectos históricos y escénicos de uno de ellos, dispuesto en los folios 54r y 55v del manuscrito, para comprender sus diferentes intencionalidades como dispositivo de la memoria cultural tlaxcalteca relacionada con el fin del Imperio mexica, en el que también se insertaron otros actores de la élite nahua del siglo XVI. Para dicho análisis, se considera una nueva traducción al español.

El estudio no intenta desarrollar una teoría, pedagógica interdisciplinaria, sino desentrañar, tanto un nivel de investigación como didáctico, las posibilidades y limitaciones que dicho intento implica. Los resultados, al tiempo que significan una valiosa ayuda para que la Pedagogía encuentre sus propios cauces de formalización científica a través de la investigación interdisciplinaria pone de manifiesto los riesgos de reduccionismo en los que se puede caer: a) Quedarse en un estéril formalismo. b) Olvidar que el eje integrador de una teoría pedagógica interdisciplinaria es el hombre educable. c) Limitar el hombre a-un producto cultural, a un simple conglomerado de elementos, en suma, a un modelo arbitrario de hombre, derivando, consecuentemente, en un concepto incorrecto de educación. Autores:José-Vicente Merino Fernández. Universidad Complutense de Madrid.Descriptores: interdisciplinaridad multidisciplinaridad pedagogía sistema

This book addresses different aspects of the research field and a wide range of topics in speech signal processing, speech recognition and language processing. The chapters are divided in three different sections: Speech Signal Modeling, Speech Recognition and Applications. The chapters in the first section cover some essential topics in speech signal processing used for building speech recognition as well as for speech synthesis systems: speech feature enhancement, speech feature vector dimensionality reduction, segmentation of speech frames into phonetic segments. The chapters of the second part cover speech recognition methods and techniques used to read speech from various speech databases and broadcast news recognition for English and non-English languages. The third section of the book presents various speech technology applications used for body conducted speech recognition, hearing impairment, multimodal interfaces and facial expression recognition.

The last several years have been an exciting time at AT&T in the field of advanced speech applications for telecommunications: technical progress and platform/processor advances have enabled the identification, development and testing of a range of new services. During this period AT&T brought together, under a single corporate roof, a research laboratory committed to advancing speech technology, business organizations building platforms to leverage this technology for telecommunications applications, and yet other business organizations with responsibility for deploying speech enabled services to facilitate the use and reduce the cost of telecommunications services for both consumers and businesses. While this period of our corporate history has drawn to a close, we look back to provide an overview of how technical progress, platform advances and network services needs and opportunities interacted to make speech technology an everyday experience for millions of people, and some of the lessons we learned along the way.

With this comprehensive and accessible introduction to the field, you will gain all the skills and knowledge needed to work with current and future audio, speech, and hearing processing technologies. Topics covered include mobile telephony, human-computer interfacing through speech, medical applications of speech and hearing technology, electronic music, audio compression and reproduction, big data audio systems and the analysis of sounds in the environment. All of this is supported by numerous practical illustrations, exercises, and hands-on MATLAB examples on topics as diverse as psychoacoustics (including some auditory illusions), voice changers, speech compression, signal analysis and visualisation, stereo processing, low-frequency ultrasonic scanning, and machine learning techniques for big data. With its pragmatic and application driven focus, and concise explanations, this is an essential resource for anyone who wants to rapidly gain a practical understanding of speech and audio processing and technology.

This study investigates gender-based phonological variations in the acoustic production of paired monophthongs among Pahari native speakers of Pakistani English (PakE). It addresses the research questions concerning how male and female speakers differ in the acoustic characteristics of English monophthongs within the PakE context. The study builds upon the theoretical frameworks of acoustic phonetics and sociophonetics, combining them to analyse vowel acoustics while considering sociocultural influences precisely. Ten participants, evenly divided between males and females, provided speech samples for specific monophthong pairs. A standardised list of 10 monosyllabic words in pairs was used as stimuli. The analysis of collected speech samples employed Praat software for formant and duration measurements, facilitated by R Software for graphical representation. Results indicate that male speakers exhibit higher F1 and F2 values in several monophthong pairs, while female speakers tend to have longer durations. These gender-related phonological distinctions contribute to understanding PakE's phonological landscape, bridging the gap between language, gender, and social identity. The study has implications for pronunciation teaching and speech technology in PakE, offering a nuanced exploration of the sociophonetic dynamics at play. Future research can delve further into sociolinguistic factors influencing these phonological distinctions and their social motivations. This research enhances the academic understanding of PakE's phonological complexity and practical applications in language pedagogy and speech technology. Keywords: Gender -based phonological variation; Acoustic monophthong production; Pakistani English (PakE); Sociophonetics; Speech technology