A Minimal Personalization of Dynamic Binaural Synthesis with Mixed Structural Modeling and Scattering Delay Networks

Purpose – The purpose of this paper is to present a system for customized binaural audio delivery based on the extraction of relevant features from a 2-D representation of the listener’s pinna. Design/methodology/approach – The most significant pinna contours are extracted by means of multi-flash imaging, and they provide values for the parameters of a structural head-related transfer function (HRTF) model. The HRTF model spatializes a given sound file according to the listener’s head orientation, tracked by sensor-equipped headphones, with respect to the virtual sound source. Findings – A preliminary localization test shows that the model is able to statically render the elevation of a virtual sound source better than non-individual HRTFs. Research limitations/implications – Results encourage a deeper analysis of the psychoacoustic impact that the individualized HRTF model has on perceived elevation of virtual sound sources. Practical implications – The model has low complexity and is suitable for implementation on mobile devices. The resulting hardware/software package will hopefully allow an easy and low-tech fruition of custom spatial audio to any user. Originality/value – The authors show that custom binaural audio can be successfully deployed without the need of cumbersome subjective measurements.

The use of audio systems that employ binaural synthesis with head tracking has become increasingly popular, particularly in virtual reality gaming systems. The binaural synthesis process uses the Head-Related Transfer Functions (HRTF) as an input required to assign the directions of arrival to sounds coming from virtual sound sources in the created virtual environments. Generic HRTFs are often used for this purpose to accommodate all potential listeners. The hypothesis of the research is that the use of individual HRTF in binaural synthesis instead of generic HRTF leads to improved accuracy and quality of virtual sound source localization, thus enhancing the user experience. A novel methodology is proposed that involves the use of dynamic virtual sound sources. In the experiments, the test participants were asked to determine the direction of a dynamic virtual sound source in both the horizontal and vertical planes using both generic and individual HRTFs. The gathered data are statistically analyzed, and the accuracy of localization is assessed with respect to the type of HRTF used. The individual HRTFs of the test participants are measured using a novel and efficient method that is accessible to a broad range of users.

이 연구는 중앙면 상에서 주성분 분석법을 이용하여 시간 및 주파수 영역에서 머리전달함수의 모형화 기법들을 다룬다. 시간영역의 머리전달함수, 복소수 값의 머리전달함수, 확장된 머리전달함수, 로그 크기의 머리전달함수에 기반하여 각각 주성분 분석법을 수행하여 얻은 네 가지 머리전달함수 모형들에 대해서 최소자승오차 관점에서 모형화 성능을 비교하고, 모형들간의 이론적인 관계를 살펴보는 것이 이 연구의 목적이다. 모형화에 사용되는 기저함수의 수가 동일하다면, 시간영역에서의 머리전달함수 혹은 확장된 머리전달함수에 기반한 모형이 복소수 값의 머리전달함수에 기반한 모형보다 최소자승오차 관점에서 더 효율적인 모형화 성능을 지닌다. 시간영역에서의 머리전달함수에 기반한 모형과 확장된 머리전달함수에 기반한 모형은 이론적으로 동일한 모형이며 서로 푸리에 변환 관계가 있다. 로그 크기의 머리전달함수에 기반한 모형은 다른 모형들과 모형화 성능 및 이론적인 관계를 비교할 수가 없는데, 이는 로그 크기의 머리전달함수에 기반한 모형은 머리전달함수의 크기 정보만을 로그 크기로 다루는 반면에 다른 모형들은 선형 크기로 머리전달함수의 크기와 위상정보를 모두 다루기 때문이다. This study deals with modeling of head-related transfer functions(HRTFs) using principal components analysis(PCA) in the time and frequency domains. Four PCA models based on head-related impulse responses(HRIRs), complex-valued HRTFs, augmented HRTFs, and log-magnitudes of HRTFs are investigated. The objective of this study is to compare modeling performances of the PCA models in the least-squares sense and to show the theoretical relationship between the PCA models. In terms of the number of principal components needed for modeling, the PCA model based on HRIR or augmented HRTFs showed more efficient modeling performance than the PCA model based on complex-valued HRTFs. The PCA model based on HRIRs in the time domain and that based on augmented HRTFs in the frequency domain are shown to be theoretically equivalent. Modeling performance of the PCA model based on log-magnitudes of HRTFs cannot be compared with that of other PCA models because the PCA model deals with log-scaled magnitude components only, whereas the other PCA models consider both magnitude and phase components in linear scale.

Human beings have a remarkable ability to determine the direction of arrival of a sound and to separate sounds of interest. Replicating this ability is a challenging problem in audio signal processing. In this paper we present a model for the head related transfer function (HRTF) developed with the localisation objective in mind. This is achieved by splitting the 3D localisation cues in terms of two functions which can be independently evaluated. We illustrate the theory for calculating these functions and validate the results against actual HRTF data. We find the model to be a close match for a significant number of potential source locations.

Head-related transfer function (HRTF) plays an important role in the construction of 3D auditory display. This paper presents an individual HRTF modeling method using deep neural networks based on spatial principal component analysis. The HRTFs are represented by a small set of spatial principal components combined with frequency and individual-dependent weights. By estimating the spatial principal components using deep neural networks and mapping the corresponding weights to a quantity of anthropometric parameters, we predict individual HRTFs in arbitrary spatial directions. The objective and subjective experiments evaluate the HRTFs generated by the proposed method, the principal component analysis (PCA) method, and the generic method. The results show that the HRTFs generated by the proposed method and PCA method perform better than the generic method. For most frequencies the spectral distortion of the proposed method is significantly smaller than the PCA method in the high frequencies but significantly larger in the low frequencies. The evaluation of the localization model shows the PCA method is better than the proposed method. The subjective localization experiments show that the PCA and the proposed methods have similar performances in most conditions. Both the objective and subjective experiments show that the proposed method can predict HRTFs in arbitrary spatial directions.

Four head-related transfer function (HRTF) data sets were compared to determine the effect of HRTF measurement methodology on the localization of spatialized auditory stimuli. Spatial audio interfaces typically require HRTF data sets to generate the spatialized auditory stimuli. HRTF measurement is accomplished using a variety of techniques that can require several nearly arbitrary decisions about methodology. The effects of these choices upon the resulting spatial audio interface are unclear. Sixteen participants completed a sound localization task that included real-world, broadband stimuli spatialized at eight locations on the horizontal plane. Four different HRTF data sets were utilized to spatialize the stimuli: two publicly available HRTF data sets and two data sets obtained using different in-house measurement systems. All HRTFs were obtained from the Knowles Electronics Mannequin for Acoustics Research. Unsigned localization error and proportion of front/back reversals did not differ significantly across HRTF data sets. Poorest accuracy was observed in locations near the medial (front/back) axis of the listener, mainly because of the relatively large proportions of reversals at these locations. This study suggests that the particular generalized HRTF data set chosen for spatialization is of minimal importance to the localizability of the resulting stimuli. This result will inform the design of many spatial audio interfaces that are based upon generalized HRTFs, including wayfaring devices, communication systems, and virtual reality systems.

BackgroundIn order to present virtual sound sources via headphones spatially, head-related transfer functions (HRTFs) can be applied to audio signals. In this so-called binaural virtual acoustics, the spatial perception may be degraded if the HRTFs deviate from the true HRTFs of the listener.ObjectiveIn this study, participants wearing virtual reality (VR) headsets performed a listening test on the 3D audio perception of virtual audiovisual scenes, thus enabling us to investigate the necessity and influence of the individualization of HRTFs. Two hypotheses were investigated: first, general HRTFs lead to limitations of 3D audio perception in VR and second, the localization model for stationary localization errors is transferable to nonindividualized HRTFs in more complex environments such as VR.MethodsFor the evaluation, 39 subjects rated individualized and nonindividualized HRTFs in an audiovisual virtual scene on the basis of 5 perceptual qualities: localizability, front-back position, externalization, tone color, and realism. The VR listening experiment consisted of 2 tests: in the first test, subjects evaluated their own and the general HRTF from the Massachusetts Institute of Technology Knowles Electronics Manikin for Acoustic Research database and in the second test, their own and 2 other nonindividualized HRTFs from the Acoustics Research Institute HRTF database. For the experiment, 2 subject-specific, nonindividualized HRTFs with a minimal and maximal localization error deviation were selected according to the localization model in sagittal planes.ResultsWith the Wilcoxon signed-rank test for the first test, analysis of variance for the second test, and a sample size of 78, the results were significant in all perceptual qualities, except for the front-back position between own and minimal deviant nonindividualized HRTF (P=.06).ConclusionsBoth hypotheses have been accepted. Sounds filtered by individualized HRTFs are considered easier to localize, easier to externalize, more natural in timbre, and thus more realistic compared to sounds filtered by nonindividualized HRTFs.

Recent work in human-machine interaction and audiology has indicated the effectiveness of using spatialized bone conduction (BC) hearing in augmented reality (AR) environments. For broader applications, individualization methods are preferred, as they require simpler computation and measurement, despite the fact that spatial hearing for BC is not yet well investigated at individual levels. Accordingly, this study examines different individualization techniques for spatial audio using BC applied to a navigation task with a moving virtual sound object in external noise, which simulates a common situation in AR environments. The tested methods include one based on individually measured head-related transfer functions (HRTFs), generic HRTFs, and a modified version based on these. Subjects are instructed to follow a virtual sound source presented from the BC headphones while walking. The virtual sound source is updated so that the source is located ahead of the subject according to the subject’s location and head rotation tracked by the motion capture system. The subjects’ performance are evaluated as the error of the subjects’ actual path from the simulated pathway. Results are discussed on the comparison between individualization methods and in regards to recent reports on sound localization in BC hearing without the listener’s body movement.

Head-Related Transfer Function (HRTF) measurement and extraction are important tasks for personalized-spatial audio. Many laboratories have their own apparatuses for data-collection but few studies have compared their results to a common subject or have modeled inter-dataset variances. We present a Bayesian fusion method based on Gaussian process (GP) modeling of joint spatial-frequency HRTFs over different spherical-measurement grids. Neumann KU-100 dummy HRTFs from 7 labs in the "Club Fritz" study are compared and fused to each other based on learning a set of transformations from the GP data-likelihood and covariance assumptions; parameter and hyperparameter training is automatic. Experimental results show that fused models for horizontal and median-plane HRTFs generalize the datasets better than pre-transformed ones.

The understanding and simplified modeling of the Head Related Transfer Function (HRTF) is very important for applications in virtual audio reality. However HRTF depends on many factors such as the shape of head and pinnae, torso, refraction by the body, and others. In this work we investigated the influences of a thin-elastic layered structure of a head on the HRTF. The analytical solution is applied to a six-layer model of the head. The elastic properties of layers are taken into account in the modeling. The results are compared with the results of spherical rigid model and experiments. The numerical results showed the new peaks of HRTF caused by elastic scattering and re-reflecting from layers. However this effect is very small and can be neglected at high frequency.

This paper presents a computationally efficient auralization method, which combines a wavelet-based model for head related transfer functions (HRTFs) with the grouping of such functions for close directions. The proposed method results in a significant reduction of the processing time of the auralization process, by modeling the HRTFs using wavelet transforms and grouping them in regions around the listener, without compromising significantly the identification of a source sound direction. Simulation results obtained with the proposed auralization process are shown in the paper.

This paper introduces a continuous functional model for head-related transfer functions (HRTFs) in the horizontal auditory scene. The approach uses a separable representation consisting of a Fourier-Bessel series expansion for the spectral components and a conventional Fourier series expansion for the spatial components. Being independent of the data, these two sets of basis functions remain unchanged for all subjects and measurement setups. Hence, the model can transform an individualized HRTF to a subject specific set of coefficients. A continuous functional model is also developed in the time domain. We show the efficient model performance in approximating experimental measurements by using the HRTF measurements from a KEMAR manikin and the synthetic data from the spherical head model. The statistical results are determined from a 50-subject HRTF data set. We also corroborate the predictive capability of the proposed model. The model has near optimal performance, which can be ascertained by comparison with the standard principle component analysis (PCA) and discrete Karhunen-Loeve expansion (KLE) methods at the measurement points and for a given number of parameters.

This paper presents a deep-learning technique for modeling Head-related Transfer Function (HRTF) using a large data-set from IRCAM [7] of 49 subjects with 115 directions/subject. We use a log-spectral distortion metric that we have used to optimize the performance of the deep-learning model. The presented model first uses a sparse autoencoder (AE) for capturing the salient representation (peaks, and notches for elevation) from the user-captured HRTFs, in its latent representation, and then uses a generalized regression neural network (GRNN) to approximate the latent representation of the corresponding HRTFs at arbitrary angles. The sparse AE then decodes the GRNN output to reconstruct the HRTF. The proposed approach is shown to outperform, objectively using the log-spectral distortion, the state-of-the-art linear-least-squares optimum solution (viz., derived from Principal Component Analysis) to perform better generalization over the HRTFs in the IRCAM dataset. Towards this, Bayesian optimization has been performed to tune the hyper-parameters of the sparse AE and the GRNN.

It is widely acknowledged that individualized head-related transfer function (HRTF) measurements are needed to adequately capture all of the 3D spatial hearing cues. However, many perceptual studies have shown that localization accuracy in the lateral dimension is only minimally decreased by the use of non-individualized head-related transfer functions. This evidence supports the idea that the individualized components of an HRTF could be isolated from those that are more general in nature. In the present study we decomposed the HRTF at each location into average, lateral and intraconic spectral components, along with an ITD in an effort to isolate the sound localization cues that are responsible for the inter-individual differences in localization performance. HRTFs for a given listener were then reconstructed systematically with components that were both individualized and non-individualized in nature, and the effect of each modification was analyzed via a virtual localization test where brief 250 ms noise bursts were rendered with the modified HRTFs. Results indicate that the cues important for individualization of HRTFs are contained almost exclusively in the intraconic portion of the HRTF spectra and localization is only minimally affected by introducing non-individualized cues into the other HRTF components. These results provide new insights into what specific inter-individual differences in head-related acoustical features are most relevant to sound localization, and provide a framework for how future human-machine interfaces might be more effectively generalized and/or individualized.

Understanding and simplified modeling of the head related transfer function (HRTF) holds the key to many applications in spatial audio. We develop an analytical solution to the problem of scattering of sound from a sphere in the vicinity of an infinite plane. Using this solution we study the influence of a nearby scattering rigid surface, on a spherical model for the HRTF.