The Period-Modulated Harmonic Locked Loop (PM-HLL): A low-effort algorithm for rapid time-domain multi-periodicity estimation

This chapter takes a new look at an old riddle by discussing putative parallelisms between speech and music processing, as well as music to speech transfer effects. Interestingly, both speech and music rely on comparable acoustic information (i.e. timing, pitch, and timbre) and share, at least partially, perceptive, cognitive, motivational, and motor components. Importantly, in this context, the chapter introduces a cortical framework of speech processing, as well as the perceptual and cognitive operations underlying speech learning mechanisms. It then reviews previous literature pinpointing the influence of music training on segmental and suprasegmental aspects of speech processing as well as on cognitive functions. Two complementary hypotheses underlying such transfer effects—cascade effects and multidimensional processes—are proposed. The issue of shared versus distinct neural networks for music and speech processing is also discussed. Finally, the chapter integrates the often observed perceptual and cognitive advantages of musicians in a holistic framework by considering their interrelations during an ecological, valid, word-learning task.

Effective management of transportation networks requires accurate travel time information that is largely determined by the quality of collected real-time traffic data. In connected vehicle (CV) environments, wherein equipped vehicles may be the primary source of reliable travel time data, accuracy and reliability of travel time estimates present a challenge due to the low market penetration at the early deployment stages of CV technology. The absence of ground truth data presents another challenge for quantifying the accuracy and reliability of travel time estimates. Therefore, CV infrastructure should be well planned in transportation networks to achieve acceptable and reliable estimates of travel times. Recent research shows that the accuracy of travel time estimates is influenced by traffic density, CV market penetration, and transmission range. These factors also impact the vehicle-to-vehicle and vehicle-to-infrastructure communication stability in a transportation network. This suggests correlation between the accuracy and reliability of travel time estimates and the communication stability. This study develops regression models to measure the accuracy and reliability of travel time estimates as a function of communication stability. Such models can help transportation planners assess the anticipated accuracy and reliability of travel time estimates in CV environments, as well as make better infrastructure investment decisions to ensure an acceptable level of accuracy and reliability of travel time estimates.

In this paper, we propose a fundamental frequency (F0) estimation algorithm for noisy speech based on exploring the local time-frequency (TF) segment. Our algorithm is motivated by the fact that the full band speech signal is redundant for pitch perception. We assume that in the same time region, the TF segment least affected by noise interference is more reliable for F0 estimation than the full band spectrum. Our algorithm consists of two main stages. Firstly, the overall TF plane is divided into overlapped TF segments, and then the F0 candidates are estimated from each single TF segment. Secondly, the optimal F0 value is selected from F0 candidates based on signal to noise ratio (SNR) estimation and dynamic programming. The experimental results show that the proposed algorithm outperforms several non-parametric state-of-the-art F0 estimation techniques.

Time of death dependent criteria in vitreous humor—Accuracy of estimating the time since death

Fundamental frequency estimation is one of the most important issues in the field of speech processing. An accurate estimate of the fundamental frequency plays a key role in the field of speech and music analysis. So far, various methods have been proposed in the time- and frequency-domain. However, the main challenge is the strong noises in speech signals. In this paper, to improve the accuracy of fundamental frequency estimation, we propose a method for optimal combination of fundamental frequency estimation methods, in noisy signals. In this method, to discriminate voiced frames from unvoiced frames in a better way, the Voiced/Unvoiced (V/U) scores of four pitch detection methods are combined linearly. These methods are: Autocorrelation, Yin, YAAPT and SWIPE. After identifying the Voiced/Unvoiced label of each frame, the fundamental frequency (F <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sub> ) of the frame is estimated using the SWIPE method. The optimal coefficients for linear combination are determined using the regularized least squares method with Tikhonov regularization. To evaluate the proposed method, 10 speech files (5 female and 5 male voices) are selected from the PTDB-TUG standard database and the results are presented in terms of SDFPE, GPE, VDE, PTE and FFE standard error criteria. The results indicate that our proposed method relatively reduced the aforementioned criteria (averaged in various SNRs) by 27.13%, 22.14%, 17.40% and 26.74% respectively, which demonstrate the effectiveness of the proposed method in comparison to state-of-the-art methods.

Fundamental frequency estimation is one of the most important issues in the field of speech processing. An accurate estimate of the fundamental frequency plays a key role in the field of speech and music analysis. So far, various methods have been proposed in the time- and frequency-domain. However, the main challenge is the strong noises in speech signals. In this paper, to improve the accuracy of fundamental frequency estimation, we propose a method for optimal nonlinear combination of fundamental frequency estimation methods, in noisy signals. In this method, to discriminate voiced frames from unvoiced frames in a better way, the Voiced/Unvoiced (V/U) scores of four pitch detection methods are combined with nonlinear fusion. These methods are: Autocorrelation (AC), Yin, YAAPT and SWIPE. After identifying the Voiced/Unvoiced label of each frame, the fundamental frequency (F <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sub> ) of the frame is estimated using the SWIPE method. The optimal function for nonlinear combination is determined using Multi-Layer Perceptron (MLP) neural network (NN). To evaluate the proposed method, 10 speech files (5 female and 5 male voices) are selected from the PTDB-TUG standard database and the results are presented in terms of GPE, VDE, PTE and FFE standard error criteria. The results indicate that our proposed method relatively reduced the aforementioned criteria (averaged in various SNRs) by 25.06%, 20.92%, 13.94%, and 25.94% respectively, which demonstrate the effectiveness of the proposed method in comparison to state-of-the-art methods.

The fundamental frequency (F 0) represents pitch in speech that determines prosodic characteristics of speech and is needed in various tasks for speech analysis and synthesis.Despite decades of research on this topic, F 0 estimation at low signal-to-noise ratios (SNRs) in unexpected noise conditions remains difficult.This work proposes a new approach to noise robust F 0 estimation using a recurrent neural network (RNN) trained in a supervised manner.Recent studies employ deep neural networks (DNNs) for F 0 tracking as a frame-by-frame classification task into quantised frequency states but we propose waveform-tosinusoid regression instead to achieve both noise robustness and accurate estimation with increased frequency resolution.Experimental results with PTDB-TUG corpus contaminated by additive noise (NOISEX-92) demonstrate that the proposed method improves gross pitch error (GPE) rate and fine pitch error (FPE) by more than 35 % at SNRs between -10 dB and +10 dB compared with well-known noise robust F 0 tracker, PEFAC.Furthermore, the proposed method also outperforms state-of-the-art DNN-based approaches by more than 15 % in terms of both FPE and GPE rate over the preceding SNR range.

The recent surge in open-source Multimodal Large Language Models (MLLM) frameworks, such as LLaVA, provides a convenient kickoff for artificial intelligence developers and researchers. However, most of the MLLM frameworks take vision as the main input modality, and provide limited in-depth support for the modality of speech, audio, and music. This situation hinders the development of audio-language models, and forces researchers to spend a lot of effort on code writing and hyperparameter tuning. We present SLAM-LLM, an open-source deep learning framework designed to train customized MLLMs, focused on speech, language, audio, and music processing. SLAM-LLM provides a modular configuration of different encoders, projectors, LLMs, and parameter-efficient fine-tuning plugins. SLAM-LLM also includes detailed training and inference recipes for mainstream tasks, along with high-performance checkpoints like LLM-based Automatic Speech Recognition (ASR), Automated Audio Captioning (AAC), and Music Captioning (MC). Some of these recipes have already reached or are nearing state-of-the-art performance, and some relevant techniques have also been accepted by academic papers. We hope SLAM-LLM will accelerate iteration, development, data engineering, and model training for researchers. We are committed to continually pushing forward audio-based MLLMs through this open-source framework, and call on the community to contribute to the LLM-based speech, audio and music processing.

In environments with many concurrent sound sources, it is often possible to hear out individual sounds and identify their sources. To accomplish this perceptual organization of the sound sources, listeners analyze the mixture of sounds to estimate the constituent components that correspond to one acoustic sound source. Signal components that are likely to come from the same source are grouped into an auditory object, that is segregated from the acoustic complex. The grouping of signal components into sources is based on identifying shared features of the components, like common spectral cues (the same fundamental frequency or frequency modulation), temporal cues (the same temporal envelope), or spatial cues (the same interaural differences). When signal components’ onsets fall within such a time window that they are perceived as being synchronous, the components are combined to form a new auditory object, that is monitored across time on cues that sustain their grouping. Given that spatial cues are inherent to every sound and already present at the onset, and that they enhance detection of a sound by a spatial release from masking, it is unclear whether and how much temporal perception depends on the presence of spatial cues. The research presented here addressed the contribution of the binaural hearing system, which processes a sound’s spatial cues, to the processing of its temporal cues, and the contribution of the sounds’ temporal cues to the processing of their spatial cues. In order to assess the contribution of spatial cues to temporal perception, experiments were conducted in which the temporal position of a target sound’s onset was controlled by the subjects to find its desired position relative to a regular series of reference sound onsets. Manipulation of the target’s interaural differences, its sensation level, and the target and reference sounds’ durations allowed comparison of their effects on temporal perception of the target’s onset. The main finding was that temporal perception precision degraded below a critical level, depending on the absolute target and masker levels and the difference in spatial configuration between target and masker that facilitated perception at much lower signal-to-noise ratios. The implication of a sound’s signal-to-noise ratio getting below this critical level is that, when the temporal window for grouping signal components’ onsets is enlarged due to the degraded precision, there is an increased probability for signal components to be grouped incorrectly into an auditory object, or to be prevented from grouping into a new auditory object and remain part of the acoustic complex until other evidence is gathered for the existence of their source. This may be a contributing factor to the decrease of speech intelligibility in noisy environments, where precise temporal perception of transients is required for the processing of speech signals. In order to assess the contribution of temporal cues to binaural processing, thresholds were measured for discrimination between the spatial configurations of two sounds that had identical spectral range and onset and offset times, but differed in their temporal envelope structure and had equally-sized directional information to opposite sides of the medial plane. The measured minimum interaural differences in time or level at which the sounds’ spatial configurations could successfully be discriminated represented the ability to process binaural cues, and allowed comparison between the effects of the various manipulations of bandwidths, center frequencies, and temporal envelopes to the sounds. The main findings were that discrimination based on interaural level differences was good for all tested conditions (although absolute thresholds depended on the specific conditions), and that discrimination based on interaural time differences was only possible when the temporal envelopes of the sounds were sufficiently different. The binaural system must, therefore, have been capable of processing the interaural differences within short periods of time when one of the sounds was dominant in the monaural temporal envelopes of the combined signals at each ear. Analysis of the monaural temporal envelopes influences the ability to use the binaural information to separate individual signal components, and allow tracking across time of the signals components with identical spatial cues to form a perceptual unity, the auditory object. In conditions where discrimination was not possible, the temporal envelopes of the individual sounds were too similar and such periods did not occur systematically in the temporal envelopes of the combined sounds. This finding suggests that a monaural analysis of a composite signal’s temporal envelope is a prerequisite for the binaural processing of the constituent signal components’ interaural time differences. The implication is that, if the signals’ temporal envelopes are too similar, their signal components may not be correctly grouped into separate and individually lateralized auditory objects, but remain fused as a single object with an averaged spatial image.

The stochastic resonance (SR) phenomenon induced by a multiplicative periodic signal in a logistic growth model with correlated noises is studied by using the theory of signal-to-noise ratio (SNR) in the adiabatic limit. The expressions of the SNR are obtained. The effects of multiplicative noise intensity α and additive noise intensity D, and correlated intensity λ on the SNR are discussed respectively. It is found that the existence of a maximum in the SNR is the identifying characteristic of the SR phenomena. In comparison with the SR induced by additive periodic signal, some new features are found: (1) When SNR as a function of λ for fixed ratio of α and D, the varying of α can induce a stochastic multi-resonance, and can induce a re-entrant transition of the peaks in SNR vs λ; (2) There exhibits a doubly critical phenomenon for SNR vs D and λ, i.e., the increasing of D (or λ) can induce the critical phenomenon for SNR with respect to λ (or D); (3) The doubly stochastic resonance effect appears when α and D are simultaneously varying in SNR, i.e., the increment of one noise intensity can help the SR on another noise intensity come forth.

Sparse Signal Decomposition for Periodic Signal Mixtures

Single User, Transmission Control Protocol Downstream Throughput (TCPDST) probability models in an IEEE802.11b WLAN have been developed, validated and evaluated for performance. Measurement of single user TCPDST were taken using Tamosoft throughput test while that of signal to noise ratio (SNR) were taken using inSSIDer 2.1. The Tamosoft throughput tests were conducted using different quality of service (QoS) traffic. These QoS traffic (which were sent through an infrastructure based network) correspond to different wireless multimedia tags. Measurements were taken in free space, small offices and open corridor environments. By assuming a normal distribution, single user TCPDST Cumulative distribution function (CDF) probability models were developed for different signal categories namely: (i) all the SNR considered, (ii) strong signals only, (iii) grey signals only and (iv) weak signals only. The models were validated and their performances evaluated using root mean square (RMS) errors. RMS errors were computed by comparing model predicted values with validation data. The RMS errors for single user CDF all signals model was 0.1466%. RMS errors for strong signals models, grey signals model and weak signals model respectively were 0.1466%, 0.6756% and 0.1233% indicating acceptable performances. All signals, strong signals, grey signals and weak signals CDF probability models predicted probabilities of obtaining TCPDST values greater than 5Mbps as 74.79%, 90.55%, 13.00% and 4.77% respectively while probabilities of obtaining TCPDST values less than 2Mbps were predicted as 4.91%, 0.00%, 18.98% and 52.41% respectively. These probability models will provide additional useful information needed to design efficient distributed data networks.&#x0D; Keywords: Throughput, TCP, WLAN, probability models

Spectral refinement (SR) offers a computationally in-expensive means of generating a refined (higher resolution) signal spectrum by linearly combining the spectra of shorter, contiguous signal segments. The benefit of this method has previously been demonstrated on the problem of fundamental frequency (F0) estimation in speech processing – specifically for the improved estimation of very low F0. One drawback of SR is, however, the poorer detection of voicing onsets due to the Heisenberg-Gabor limit on time and frequency resolution. This may also lead to degraded performance in noisy conditions. Transitioning between long- and short-time windows for the spectral analysis may offer a good trade-off in these situations. This contribution presents a method to adaptively switch between short- and long-time windows (and, correspondingly, between the short-term and the refined spectrum) for voicing detection and F0 estimation. The improvements in voicing detection and F0 estimation due to this adaptive switching is conclusively demonstrated on audio signals in clean and corrupted conditions.

Central auditory system responses from children while listening to speech in noise

In this paper, a novel Doppler shift estimation method for frequency diverse array (FDA) radar based on graph signal processing (GSP) theory is proposed and investigated. First, a well-designed graph signal model for a monostatic linear FDA is formulated. Subsequently, spectral decomposition is conducted on the constructed signal model utilizing graph Fourier transform (GFT) techniques, enabling the extraction of the target’s Doppler shift parameter through spectral peak search. A comprehensive series of simulation experiments demonstrates that the proposed method can achieve the accurate estimation of target parameters even under low signal-to-noise ratio (SNR) conditions. Furthermore, the proposed method exhibits superior performance compared to the MUSIC algorithm, offering enhanced resolution and estimation accuracy. Additionally, the method is highly amenable to parallel processing, significantly reducing the computational burden associated with traditional procedures.