DNN-based HRTF individualization for accurate spectral cues using a compact PRTF

Four head-related transfer function (HRTF) data sets were compared to determine the effect of HRTF measurement methodology on the localization of spatialized auditory stimuli. Spatial audio interfaces typically require HRTF data sets to generate the spatialized auditory stimuli. HRTF measurement is accomplished using a variety of techniques that can require several nearly arbitrary decisions about methodology. The effects of these choices upon the resulting spatial audio interface are unclear. Sixteen participants completed a sound localization task that included real-world, broadband stimuli spatialized at eight locations on the horizontal plane. Four different HRTF data sets were utilized to spatialize the stimuli: two publicly available HRTF data sets and two data sets obtained using different in-house measurement systems. All HRTFs were obtained from the Knowles Electronics Mannequin for Acoustics Research. Unsigned localization error and proportion of front/back reversals did not differ significantly across HRTF data sets. Poorest accuracy was observed in locations near the medial (front/back) axis of the listener, mainly because of the relatively large proportions of reversals at these locations. This study suggests that the particular generalized HRTF data set chosen for spatialization is of minimal importance to the localizability of the resulting stimuli. This result will inform the design of many spatial audio interfaces that are based upon generalized HRTFs, including wayfaring devices, communication systems, and virtual reality systems.

Head-related transfer function (HRTF) plays an important role in the construction of 3D auditory display. This paper presents an individual HRTF modeling method using deep neural networks based on spatial principal component analysis. The HRTFs are represented by a small set of spatial principal components combined with frequency and individual-dependent weights. By estimating the spatial principal components using deep neural networks and mapping the corresponding weights to a quantity of anthropometric parameters, we predict individual HRTFs in arbitrary spatial directions. The objective and subjective experiments evaluate the HRTFs generated by the proposed method, the principal component analysis (PCA) method, and the generic method. The results show that the HRTFs generated by the proposed method and PCA method perform better than the generic method. For most frequencies the spectral distortion of the proposed method is significantly smaller than the PCA method in the high frequencies but significantly larger in the low frequencies. The evaluation of the localization model shows the PCA method is better than the proposed method. The subjective localization experiments show that the PCA and the proposed methods have similar performances in most conditions. Both the objective and subjective experiments show that the proposed method can predict HRTFs in arbitrary spatial directions.

이 연구는 중앙면 상에서 주성분 분석법을 이용하여 시간 및 주파수 영역에서 머리전달함수의 모형화 기법들을 다룬다. 시간영역의 머리전달함수, 복소수 값의 머리전달함수, 확장된 머리전달함수, 로그 크기의 머리전달함수에 기반하여 각각 주성분 분석법을 수행하여 얻은 네 가지 머리전달함수 모형들에 대해서 최소자승오차 관점에서 모형화 성능을 비교하고, 모형들간의 이론적인 관계를 살펴보는 것이 이 연구의 목적이다. 모형화에 사용되는 기저함수의 수가 동일하다면, 시간영역에서의 머리전달함수 혹은 확장된 머리전달함수에 기반한 모형이 복소수 값의 머리전달함수에 기반한 모형보다 최소자승오차 관점에서 더 효율적인 모형화 성능을 지닌다. 시간영역에서의 머리전달함수에 기반한 모형과 확장된 머리전달함수에 기반한 모형은 이론적으로 동일한 모형이며 서로 푸리에 변환 관계가 있다. 로그 크기의 머리전달함수에 기반한 모형은 다른 모형들과 모형화 성능 및 이론적인 관계를 비교할 수가 없는데, 이는 로그 크기의 머리전달함수에 기반한 모형은 머리전달함수의 크기 정보만을 로그 크기로 다루는 반면에 다른 모형들은 선형 크기로 머리전달함수의 크기와 위상정보를 모두 다루기 때문이다. This study deals with modeling of head-related transfer functions(HRTFs) using principal components analysis(PCA) in the time and frequency domains. Four PCA models based on head-related impulse responses(HRIRs), complex-valued HRTFs, augmented HRTFs, and log-magnitudes of HRTFs are investigated. The objective of this study is to compare modeling performances of the PCA models in the least-squares sense and to show the theoretical relationship between the PCA models. In terms of the number of principal components needed for modeling, the PCA model based on HRIR or augmented HRTFs showed more efficient modeling performance than the PCA model based on complex-valued HRTFs. The PCA model based on HRIRs in the time domain and that based on augmented HRTFs in the frequency domain are shown to be theoretically equivalent. Modeling performance of the PCA model based on log-magnitudes of HRTFs cannot be compared with that of other PCA models because the PCA model deals with log-scaled magnitude components only, whereas the other PCA models consider both magnitude and phase components in linear scale.

Applications in the creation of virtual auditory spaces (VAS) and sonification require individualized head related transfer functions (HRTFs) for perceptual fidelity. HRTFs exhibit significant variation from person to person due to differences between their pinnae, and their body sizes. We propose and preliminarily implement a simple HRTF customization based on the use of a published database of HRTFs (Algazi et al., 2001) that also contains geometrical measurements of subject pinnae. We measure some of these features via simple image processing, and select the HRTF that has features most closely corresponding to the individual's features. This selection procedure is implemented along with the virtual auditory system described in (Zotkin et al., 2002), and listener tests conducted comparing the customized HRTF and a fixed HRTF. Despite the simplicity of the method, tests reveal average improvement in localization accuracy of about 25 percent, though performance improvement varies with source location and individuals.

Efficient prediction of individual head-related transfer functions based on 3D meshes

The use of audio systems that employ binaural synthesis with head tracking has become increasingly popular, particularly in virtual reality gaming systems. The binaural synthesis process uses the Head-Related Transfer Functions (HRTF) as an input required to assign the directions of arrival to sounds coming from virtual sound sources in the created virtual environments. Generic HRTFs are often used for this purpose to accommodate all potential listeners. The hypothesis of the research is that the use of individual HRTF in binaural synthesis instead of generic HRTF leads to improved accuracy and quality of virtual sound source localization, thus enhancing the user experience. A novel methodology is proposed that involves the use of dynamic virtual sound sources. In the experiments, the test participants were asked to determine the direction of a dynamic virtual sound source in both the horizontal and vertical planes using both generic and individual HRTFs. The gathered data are statistically analyzed, and the accuracy of localization is assessed with respect to the type of HRTF used. The individual HRTFs of the test participants are measured using a novel and efficient method that is accessible to a broad range of users.

"© 2017 IEEE. Personal use of this material is permitted. Permissíon from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertisíng or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works."

Toward orthogonal non-individualised head-related transfer functions for forward and backward directional sound: cluster analysis and an experimental study

In this paper we analyze several auditory localization neural models that are based on head related transfer functions (HRTFs). HRTFs represent the combined directional-spectral response of the pinnae head and torso. The role of HRTFs in such modeling has hitherto been underestimated despite substantial experimental evidence to its relevance in spatial hearing, especially in determining direction of highfrequency sound sources. In the first section we suggest a neural model that links the physiology of binaural processing to a neural network that extracts spectral ratios. These ratios correspond to HRTFs ratios and can provide auditory directional cues. Next, we compare several methods of matching HRTFs ratios using discriminative matching measure (DMM). We consider several solutions to the matching problem from a neural signal processing viewpoint. We compare correlation based approaches with DMM optimization approach and with a non-linear approach based on neural back-propagation algorithm. All three models can be implemented by neural networks. Finally, we include experimental results of simulations that are conducted using these methods. Experiments show that the back-propagation based neural network yields the best results in terms of DMM both for narrow-band and broad band excitation. The back-propagation neural network is also superior in matching noisy HRTF ratio vectors. [Neurol Res 2001; 23: 489-500]

Individualization of head-related transfer functions (HRTFs) can improve the quality of binaural applications with respect to the localization accuracy, coloration, and other aspects. Using anthropometric features (AFs) of the head, neck, and pinna for individualization is a promising approach to avoid elaborate acoustic measurements or numerical simulations. Previous studies on HRTF individualization analyzed the link between AFs and technical HRTF features. However, the perceptual relevance of specific errors might not always be clear. Hence, the effects of AFs on perceived perceptual qualities with respect to the overall difference, coloration, and localization error are directly explored. To this end, a listening test was conducted in which subjects rated differences between their own HRTF and a set of nonindividual HRTFs. Based on these data, a machine learning model was developed to predict the perceived differences using ratios of a subject's individual AFs and those of presented nonindividual AFs. Results show that perceived differences can be predicted well and the HRTFs recommended by the models provide a clear improvement over generic or randomly selected HRTFs. In addition, the most relevant AFs for the prediction of each type of error were determined. The developed models are available under a free cultural license.

Adaptation to altered or non-individual head-related transfer functions (HRTFs) has been widely studied in individuals with normal hearing. Studies have shown that after an adaptation period, individuals can significantly improve their sound localization performance with altered cues. Recently, it has been shown that after training with one set of HRTFs, improvements can also be observed with another set, a phenomenon referred to as generalization. This study provides significantly more evidence of this phenomenon by including a larger number of participant groups and a longer training period compared to prior work. The experiment involved four groups of participants: two control groups (one with individual HRTFs, one with non-individual HRTFs) only performing localization tests and two training groups (one with individual HRTFs, one with non-individual HRTFs) completing both localization tests and training sessions. In all cases, the stimulus consisted of a male speech sentence reproduced binaurally. Both training groups showed significant and comparable improvements in localization performance. Moreover, generalization effects were observed and found to be similar between the two training groups.

3D audio real-time rendering is of great importance for virtual reality (VR) application, especially on mobile devices. However, the limited computational power makes it hard to implement fast generation of spatial sound using head-related transfer function (HRTF). To accelerate the procedure of head-related impulse response (HRIR) convolution, we propose an efficient rendering method using the parameterized HRTF. The HRIRs are preprocessed into the parameterized HRTFs with a delayed parameter, so as to realize the fast convolutions in 3D audio synthesis. We found our method could reduce the calculation time of 3D audio rendering significantly. The results were 0.9 times and 0.68 times on average the play duration of the MPEG standard sequences, with Qualcomm Snapdragon 821 and Samsung Exynos 7420. And the subjective test showed that the real-time rendered sounds had the similar audio quality and location perception to the 3D audio rendered with the raw HRIRs.

International audience

This paper presents a deep-learning technique for modeling Head-related Transfer Function (HRTF) using a large data-set from IRCAM [7] of 49 subjects with 115 directions/subject. We use a log-spectral distortion metric that we have used to optimize the performance of the deep-learning model. The presented model first uses a sparse autoencoder (AE) for capturing the salient representation (peaks, and notches for elevation) from the user-captured HRTFs, in its latent representation, and then uses a generalized regression neural network (GRNN) to approximate the latent representation of the corresponding HRTFs at arbitrary angles. The sparse AE then decodes the GRNN output to reconstruct the HRTF. The proposed approach is shown to outperform, objectively using the log-spectral distortion, the state-of-the-art linear-least-squares optimum solution (viz., derived from Principal Component Analysis) to perform better generalization over the HRTFs in the IRCAM dataset. Towards this, Bayesian optimization has been performed to tune the hyper-parameters of the sparse AE and the GRNN.

High performance audio and video codecs and ever increasing bandwidths of data communications networks have enabled multi-platform delivery of high definition media content via transmission channels such as terrestrial broadcast, broadband IP networks and mobile telecommunications systems. Media content consumption becomes ubiquitous and more personalized than ever before and audio is often rendered through headphones. Attentions should be paid to sound quality, and therefore high definition virtual auditory space (VAS) is sought after as an ideal solution. Stereophonic and surround sound are still commonplace while spatial audio is in its infancy to meet the demand of 2.5/3 D high definition video. Binaural headphone rendering offers ultimate spatial audio and source localization; however, individual head-related transfer functions (HRTFs), head movement tracking and suitably synthesized signals are needed to achieve its full potential. Most audio recordings to date are intended and hence best for loudspeaker rendering. Good cross-feed filters can be an interim solution to improve headphone user experience. This paper proposes a universal cross-feed filter. HRTFs at specific source locations were first considered to derive the cross-feed filters. User preferred HRTF parameters were also identified with a variety of representative soundtracks. The transfer functions were measured and simplified to derive user preferred universal cross-feed filters for all. Validation tests confirmed improved listening experience.