Auditory and vibrotactile interactions in perception of timbre acoustic features

The ratio of direct to reverberant sound energy (D/R) has been shown to be a primary acoustic cue to perceived sound source distance. Because it is unclear exactly how D/R might be encoded in the auditory system, a variety of more physiologically plausible correlates to D/R have been identified, including: spectral variance, interaural correlation, and temporal cues. Here, following recent neural work by Kuwada and Kim [ARO, 2014], we describe a new correlate to D/R and perceived distance related to the amplitude modulation (AM) depth of the signal at the listener's location. This cue is caused by the change in the modulation transfer characteristics of the room as a function of source distance. Results from an apparent distance estimation task confirm the efficacy of this AM depth cue in a reverberant soundfield (approximate broadband T60 = 3 s), when level cues are made ineffective. Distance estimates to near sources (≤2m) were found to be more accurate when the source signal (1-octave band of noise centered at 4 kHz) had AM (32 Hz, 100% depth), and this facilitation was only observed in reverberation. The facilitation was most evident for monaural input, indicating that the AM depth cue is likely processed monaurally.

The impact of amplitude modulation frequency in harmonic motion imaging on inclusion characterization.

The present study tested the existence of modulation filters by transposing the concept of excitation pattern to the amplitude modulation (AM) domain. In the first experiment, AM rate-discrimination thresholds (DLFs) for white-noise carriers with AM reference rates of 4, 16, and 64 Hz were measured in five normal-hearing listeners. AM depth was either (i) fixed at 0.2, 0.6, or 1.0, or (ii) varied randomly between 0.2 and 1.0. Randomization of the modulation depth occurred either between trials or between the two intervals of the same trial. In a second experiment, detection thresholds of slow (1 or 4 Hz) frequency modulation (FMDLs) applied to the AM carrier (of identical rates as in the previous experiment) were measured, with and without the presence of a slow sinusoidal modulation applied to the depth of the AM carrier. Overall, the data show that DLFs are less affected than FMDLs by variations in the depth of AM. The results are discussed in light of current models of AM perception.

This study investigates the degree of similarity of three different auditory cortical areas with respect to the coding of periodic stimuli. Simultaneous single- and multiunit recordings in response to periodic stimuli were made from primary auditory cortex (AI), anterior auditory field (AAF), and secondary auditory cortex (AII) in the cat to addresses the following questions: is there, within each cortical area, a difference in the temporal coding of periodic click trains, amplitude-modulated (AM) noise bursts, and AM tone bursts? Is there a difference in this coding between the three cortical fields? Is the coding based on the temporal modulation transfer function (tMTF) and on the all-order interspike-interval (ISI) histogram the same? Is the perceptual distinction between rhythm and roughness for AM stimuli related to a temporal versus spatial representation of AM frequency in auditory cortex? Are interarea differences in temporal response properties related to differences in frequency tuning? The results showed that: 1) AM stimuli produce much higher best modulation frequencies (BMFs) and limiting rates than periodic click trains. 2) For periodic click trains and AM noise, the BMFs and limiting rates were not significantly different for the three areas. However, for AM tones the BMF and limiting rates were about a factor 2 lower in AAF compared with the other areas. 3) The representation of stimulus periodicity in ISIs resulted in significantly lower mean BMFs and limiting rates compared with those estimated from the tMTFs. The difference was relatively small for periodic click trains but quite large for both AM stimuli, especially in AI and AII. 4) Modulation frequencies <20 Hz were represented in the ISIs, suggesting that rhythm is coded in auditory cortex in temporal fashion. 5) In general only a modest interdependence of spectral- and temporal-response properties in AI and AII was found. The BMFs were correlated positively with characteristic frequency in AAF. The limiting rate was positively correlated with the frequency-tuning curve bandwidth in AI and AII but not in AAF. Only in AAF was a correlation between BMF and minimum latency was found. Thus whereas differences were found in the frequency-tuning curve bandwidth and minimum response latencies among the three areas, the coding of periodic stimuli in these areas was fairly similar with the exception of the very poor representation of AM tones in AII. This suggests a strong parallel processing organization in auditory cortex.

Electromagnetic interference in the permeability of saquinavir across the blood–brain barrier using nanoparticulate carriers

Sid Bacon was a pioneer in studies of the extent to which the detection of target amplitude modulation (AM) of a carrier is affected by additional (masker) amplitude modulation applied to the same carrier (within-channel modulation masking) or to a different carrier (across-channel modulation masking). Here, these two types of modulation masking were compared for normal-hearing and hearing-impaired subjects. The target was either 4-Hz or 16-Hz sinusoidal AM of a 4000-Hz carrier. The target AM depth was fixed. The masker AM was applied either to the same carrier or to a carrier at 3179 or 2518 Hz. The masker AM rate was 0.25, 0.5, 1, 2, or 4 times the target rate. The masker AM depth was varied adaptively to determine the value needed just to mask the target AM. Preliminary results indicate that within-channel modulation-masking patterns are similar for normal-hearing and hearing-impaired subjects, suggesting that the hypothetical modulation filters are not affected by hearing loss. However, the amount of across-channel modulation masking is lower for normal-hearing than for hearing-impaired subjects, presumably because of the reduced frequency selectivity of the latter. The increased across-channel masking for the hearing-impaired subjects may contribute to their difficulties in understanding speech in background sounds.

Children's speech understanding is vulnerable to indoor noise and reverberation: e.g., from classrooms. It is unknown how they develop the ability to use temporal acoustic cues, specifically amplitude modulation (AM) and voice onset time (VOT), which are important for perceiving distorted speech. Through three experiments, we investigated the typical development of AM depth detection in vowels (experiment I), categorical perception of VOT (experiment II), and consonant identification (experiment III) in quiet and in speech-shaped noise (SSN) and mild reverberation in 6- to 14-year-old children. Our findings suggested that AM depth detection using a naturally produced vowel at the rate of the fundamental frequency was particularly difficult for children and with acoustic distortions. While the VOT cue salience was monotonically attenuated with increasing signal-to-noise ratio of SSN, its utility for consonant discrimination was completely removed even under mild reverberation. The reverberant energy decay in distorting critical temporal cues provided further evidence that may explain the error patterns observed in consonant identification. By 11-14 years of age, children approached adult-like performance in consonant discrimination and identification under adverse acoustics, emphasizing the need for good acoustics for younger children as they develop auditory skills to process distorted speech in everyday listening environments.

Magic angle sample spinning (MASS) averages the inhomogeneous magnetic field due to the susceptibility contrast in porous media by modulating the local magnetic field (frequency modulation). Molecular diffusion introduces a homogeneous broadening, which modulates the amplitude of the signal (amplitude modulation). The depth of amplitude modulation is determined by the interplay of molecular diffusion and MASS averaging and contains rich information on local magnetic fields, through which the spatial structure of the sample may be obtained. In this paper, we present two methods to quantify the amplitude modulation: a phase suppressed method and a more conventional two-dimensional (2D)-exchange method. The phase suppressed method directly observes the amplitude modulation in the time domain. The approximate equations are derived to extract the physical information from the time domain data. The conventional 2D-exchange spectrum contains both the frequency modulation and the amplitude modulation terms. The amplitude modulation introduces cross peaks in the 2D spectrum. The integration of the 2D spectrum along lines parallel to the main diagonal line will give a one-dimensional spectrum, that is the Fourier transform of the amplitude modulation.

Auditory steady-state response (ASSR) is a translational biomarker for several neurological and psychiatric disorders, such as hearing loss, schizophrenia, bipolar disorder, autism, etc. The ASSR is sinusoidal electroencephalography (EEG)/magnetoencephalography (MEG) responses induced by periodically presented auditory stimuli. Traditional frequency analysis assumes ASSR is a stationary response, which can be analyzed using linear analysis approaches, such as Fourier analysis or Wavelet. However, recent studies have reported that the human steady-state responses are dynamic and can be modulated by the subject’s attention, wakefulness state, mental load, and mental fatigue. The amplitude modulations on the measured oscillatory responses can result in the spectral broadening or frequency splitting on the Fourier spectrum, owing to the trigonometric product-to-sum formula. Accordingly, in this study, we analyzed the human ASSR by the combination of canonical correlation analysis (CCA) and Holo-Hilbert spectral analysis (HHSA). The CCA was used to extract ASSR-related signal features, and the HHSA was used to decompose the extracted ASSR responses into amplitude modulation (AM) components and frequency modulation (FM) components, in which the FM frequency represents the fast-changing intra-mode frequency and the AM frequency represents the slow-changing inter-mode frequency. In this paper, we aimed to study the AM and FM spectra of ASSR responses in a 37 Hz steady-state auditory stimulation. Twenty-five healthy subjects were recruited for this study, and each subject was requested to participate in two auditory stimulation sessions, including one right-ear and one left-ear monaural steady-state auditory stimulation. With the HHSA, both the 37 Hz (fundamental frequency) and the 74 Hz (first harmonic frequency) auditory responses were successfully extracted. Examining the AM spectra, the 37 Hz and the 74 Hz auditory responses were modulated by distinct AM spectra, each with at least three composite frequencies. In contrast to the results of traditional Fourier spectra, frequency splitting was seen at 37 Hz, and a spectral peak was obscured at 74 Hz in Fourier spectra. The proposed method effectively corrects the frequency splitting problem resulting from time-varying amplitude changes. Our results have validated the HHSA as a useful tool for steady-state response (SSR) studies so that the misleading or wrong interpretation caused by amplitude modulation in the traditional Fourier spectrum can be avoided.

Dissociation of psychophysical and EEG steady-state response measures of cross-modal temporal correspondence for amplitude modulated acoustic and vibrotactile stimulation

This study tests the hypothesis that amplitude modulation (AM) detection will be better under conditions where basilar membrane (BM) response growth is expected to be linear rather than compressive. This hypothesis was tested by (1) comparing AM detection for a tonal carrier as a function of carrier level for subjects with and without cochlear hearing impairment (HI), and by (2) comparing AM detection for carriers presented with and without an ipsilateral notched-noise precursor, under the assumption that the precursor linearizes BM responses. Average AM detection thresholds were approximately 5 dB better for subjects with HI than for subjects with normal hearing (NH) at moderate-level carriers. Average AM detection for low-to-moderate level carriers was approximately 2 dB better with the precursor than without the precursor for subjects with NH, whereas precursor effects were absent or smaller for subjects with HI. Although effect sizes were small and individual differences were noted, group differences are consistent with better AM detection for conditions where BM responses are less compressive due to cochlear hearing loss or due to a reduction in cochlear gain. These findings suggest the auditory system may quickly adjust to the local soundscape to increase effective AM depth and improve signal-to-noise ratios.

BackgroundThe Montreal Cognitive Assessment (MoCA) is a commonly used screening tool for cognitive impairment. Despite translation into multiple languages to facilitate broader use globally, there are inherent education and cultural biases that result in variations in cognitive screening accuracies. Acoustic voice features are emerging as a more education, language and culturally agnostic indicator of cognitive status, but as a surrogate to the MoCA has not been adequately explored. This pilot study aimed to examine the association between spectral acoustic features extracted by two functionals and MoCA total scores.MethodWe included 80 participants from the Boston University Alzheimer's Disease Research Center, whose responses to a picture‐description task were digitally recorded and administered the MoCA. Using openSMILE, each recording was divided into 20‐ms frames, using a sliding window that advanced by 10 ms for every segment. For each segment, we extracted 26 RASTA‐style filtered auditory spectrum (bands 1‐26) low‐level descriptors (LLD). Two functionals were applied to the 26 LLD to summarize information across each recording: the mean (mean value of LLD for all segments in each recording) and “upleveltime75” (percentage of time the signal exceeds 75% of the feature range above the minimum). Linear regression models were used to assess the association of these acoustic features with MoCA total scores, adjusting for age, sex, and education.ResultParticipant demographics included age: mean 68.7, SD 9.27 years; 81.25% college graduate or higher; 60.0% women. Among these participants, 29 were cognitively impaired. The mean MoCA score was 26.49 (SD = 2.54). Among the 26 upleveltime75 functional‐based spectral features, 5 showed significant negative associations with MoCA. The strongest effect was observed for band 25 (beta = –0.77, SE = 0.26, p = 0.0038). In contrast, no significant associations were observed for the acoustic features generated by the mean functional.ConclusionThese results suggest that digital voice contains cognitive‐related signals and show promise as a potential globally appropriate cognitive screening tool given the ease in which it can be collected and the availability of automated open‐source tools for analysis. Further exploration analyzing voice recordings across different languages and cultural/education strata are warranted.

The world&#x0027;s biggest pandemic, COVID-19, has shown its lethal impact on human life. The current diagnostics methods are reverse transcription-polymerase chain reaction (RT-PCR) and rapid diagnostic assays have several bottlenecks in terms of the nature of sample collection as it needs some laboratory experts and careful handling of the potentially infectious samples. However, one of the non-invasive ways of diagnostics is to focus on speech modality, which has been paid less attention, during the detection of COVID-19. Hence in this work, the speech features, particularly temporal and spectral features have been studied for COVID-19 detection. The temporal features used in this work are Short-Time Energy, Long-Term Log Energy Variation (LTLEV) Zero Crossing Count (ZCC) and Pitch etc. On the other hand, the spectral features used herein are Power Spectral Density, Average Power, Mel-Frequency Cepstral Coefficients, Group delay spectrum, Spectral Entropy etc. Such spectral and temporal speech features have not been analyzed in the identification of COVID-19 symptoms to the best of authors knowledge. Further, this paper has shown the impact of COVID-19 on a real time human voice, analyzed using speech processing techniques, and shown their efficacy in detecting COVID-19. These features are safe, comparatively faster, cost-effective, and require fewer complexities. Our article will motivate the scientific community to use such features for further research in the collective battle against COVID-19.

BackgroundDepression is major global public health problems among university students. Currently, the evaluation and monitoring of depression predominantly depend on subjective and self-reported methods. There is an urgent necessity to develop objective means of identifying depression. Acoustic features, which convey emotional information, have the potential to enhance the objectivity of depression assessments. This study aimed to investigate the feasibility of utilizing acoustic features for the objective and automated identification and characterization of depression among Chinese university students.MethodsA cross-sectional study was undertaken involving 103 students with depression and 103 controls matched for age, gender, and education. Participants' voices were recorded using a smartphone as they read neutral texts. Acoustic analysis and feature extraction were performed using the OpenSMILE toolkit, yielding 523 features encompassing spectral, glottal, and prosodic characteristics. These extracted acoustic features were utilized for discriminant analysis between depression and control groups. Pearson correlation analyses were conducted to evaluate the relationship between acoustic features and Patient Health Questionnaire-9 (PHQ-9) scores. Five machine learning algorithms including Linear Discriminant Analysis (LDA), Logistic Regression, Support Vector Classification, Naive Bayes, and Random Forest were used to perform the classification. For training and testing, ten-fold cross-validation was employed. Model performance was assessed using receiver operating characteristic (ROC) curve, area under the curve (AUC), precision, accuracy, recall, and F1 score. Shapley Additive exPlanations (SHAP) method was used for model interpretation.ResultsIn depression group, 32 acoustic features (25 spectral features, 5 prosodic features and 2 glottal features) showed significant alterations compared with controls. Further, 27 acoustic features (10 spectral features, 3 prosodic features, and 1 glottal features) were significantly correlated with depression severity. Among five machine learning algorithms, LDA model demonstrated the highest classification performance, with an AUC of 0.771. SHAP analysis suggested that Mel-frequency cepstral coefficients (MFCC) features contributed most to the model's classification efficacy.ConclusionsThe integration of acoustic features and LDA model demonstrates a high accuracy in distinguishing depression among Chinese university students, suggesting its potential utility in rapid and large-scale depression screening. MFCC may serve as objective and valid features for the automated identification of depression on Chinese university campuses.

Human auditory perception and speech intelligibility have been successfully described based on the two concepts of spectral masking and amplitude modulation (AM) masking. The power-spectrum model (PSM) [Patterson and Moore (1986). Frequency Selectivity in Hearing, pp. 123-177] accounts for effects of spectral masking and critical bandwidth, while the envelope power-spectrum model (EPSM) [Ewert and Dau (2000). J. Acoust. Soc. Am. 108, 1181-1196] has been successfully applied to AM masking and discrimination. Both models extract the long-term (envelope) power to calculate signal-to-noise ratios (SNR). Recently, the EPSM has been applied to speech intelligibility (SI) considering the short-term envelope SNR on various time scales (multi-resolution speech-based envelope power-spectrum model; mr-sEPSM) to account for SI in fluctuating noise [Jørgensen, Ewert, and Dau (2013). J. Acoust. Soc. Am. 134, 436-446]. Here, a generalized auditory model is suggested combining the classical PSM and the mr-sEPSM to jointly account for psychoacoustics and speech intelligibility. The model was extended to consider the local AM depth in conditions with slowly varying signal levels, and the relative role of long-term and short-term SNR was assessed. The suggested generalized power-spectrum model is shown to account for a large variety of psychoacoustic data and to predict speech intelligibility in various types of background noise.