Song Recognition Based on Audio Finger printing

Automatic highlights extraction for drama video using music emotion and human face features

Previous studies on Lamb wave touchscreen (LWT) were carried out based on the assumption that the unknown touch had the consistent parameters with acoustic fingerprints in the reference database. The adaptability of LWT to the variations in touch force and touch area was investigated in this study for the first time. The automatic collection of the databases of acoustic fingerprints was realized with an experimental prototype of LWT employing three pairs of transmitter–receivers. The self-adaptive updated weight coefficient of the used transmitter–receiver pairs was employed to successfully improve the accuracy of the localization model established based on a learning method. The performance of the improved method in locating single- and two-touch actions with the reference database of different parameters was carefully evaluated. The robustness of the LWT to the variation of the touch force varied with the touch area. Moreover, it was feasible to locate touch actions of large area with reference databases of small touch areas as long as the unknown touch and the reference databases met the condition of equivalent averaged stress.

In this paper, we propose a novel method for the detection of road accidents by analyzing audio streams for road surveillance application. In the last decade, due to the increase in a number of people and transportation vehicles, traffic accidents have become one of a major public issue worldwide. The vast number of injuries and death due to road traffic accident reveals the story of a global crisis of road safety. A total of 52,160 road traffic accidents (RTA), 1130 injuries and 85 fatalities were registered during the year 2000, in the state of Qatar. An increase in the number of transportation vehicles around cities has risen the need for more security and safety in public environments. The most obvious reason for a person's death during accidents is the absence or prolong response time of the first aid facility, which is due to the delay in the information of the accident being reached to the police, hospital or ambulance team. In the last couple of years, several surveillance systems have been proposed based on image and video processing for automatically detecting road accidents and car crashes to ensure a quick response by emergency teams. However, in some situations such as adverse weather conditions or cluttered environments, the visual information is not sufficient enough, whereas analyzing the audio tracks can significantly improve the overall reliability of surveillance systems. In this paper we propose a novel method that automatically identifies hazardous situations such as tire skidding and car crashes in presence of background noises, by analyzing the audio streams. Previous studies show that methods for the detection, estimation, and classification of nonstationary signals can be enhanced by utilizing the time-frequency (TF) characteristics of such signals. The TF-based techniques have been proved to outperform classical techniques based on either time or frequency domains, in analyzing real-life nonstationary signals. Time-frequency distributions (TFDs) give additional information about signals that cannot be extracted from either the time domain or frequency domain representations, i.e. the instantaneous frequency of the components of a signal. In order to utilize this extra information provided by TF domain representation, the proposed approach extracts TF image features from quadratic time-frequency distributions (QTFDs), for the detection of audio anomalies. The extended modified-B distribution (EMBD) is utilized to transform a 1-dimensional audio signal into 2-dimensional TF representation which is interpreted as an image. The image descriptors based features are then extracted from the TF representation to classify visually the audio signals into background or abnormal activity patterns in the TF domain. The proposed features are based on Haralick's texture features extracted from the TF representation of audio signals considered and processed as a textured image. These features are used to characterize and hence classify audio signals into M classes. This research study demonstrates that a TF image pattern recognition approach offers significant advantages over standard signal classification methods that utilize either t-domain only or f-domain only features. The proposed method has been experimentally validated on a large open source database of sounds, including several kinds of background noise. The events to be recognized are superimposed on different background sounds of roads and traffic jam. The obtained results are compared with a recent study, utilizing the same large and complex data set of audio signals, and the same experimental setup. The overall classification results confirm the superior performance of the proposed approach with accuracy improvement up to 6%.

Although multiple music recordings may sound identical to a human listener, the underlying representations of sound may differ due to the variations in their audio encoding and/or transmission methods. In contrast to the existing audio-fingerprinting techniques, which establishes the fingerprint of each source music to identify unknown, ldquodistortedrdquo audio clips, this paper proposes an unsupervised clustering framework to identify unknown, ldquodistortedrdquo audio clips derived from the same source music. By grouping together audio data derived from the same music, the human effort required to label music data can be dramatically reduced. This work develops methods to measure the similarities between audio clips and use hierarchical agglomerative clustering to group together audio clips that are similar to one another. Also proposed is a method based on the Rand Index to determine the optimal number of clusters automatically in relation to the number of source music recordings.

In this work, we re-think the task of speech enhancement in unconstrained real-world environments. Current state- of-the-art methods use only the audio stream and are limited in their performance in a wide range of real-world noises. Recent works using lip movements as additional cues improve the quality of generated speech over "audio-only " methods. But, these methods cannot be used for several applications where the visual stream is unreliable or completely absent. We propose a new paradigm for speech enhancement by exploiting recent breakthroughs in speech- driven lip synthesis. Using one such model as a teacher network, we train a robust student network to produce accurate lip movements that mask away the noise, thus acting as a "visual noise filter". The intelligibility of the speech enhanced by our pseudo-lip approach is comparable ( <; 3% difference) to the case of using real lips. This implies that we can exploit the advantages of using lip movements even in the absence of a real video stream. We rigorously evaluate our model using quantitative metrics as well as human evaluations. Additional ablation studies and a demo video on our website containing qualitative comparisons and results clearly illustrate the effectiveness of our approach.

Speech systems work reasonably well under homogeneous acoustic environmental conditions but become fragile in practical applications involving real-world environments (e.g., in-car, broadcast news, digital archives, etc.) where the audio stream contains multi-environment characteristics. To date, most approaches dealing with environmental noise in speech systems are based on assumptions concerning the noise, rather than exploring and characterizing the nature of the noise. In this chapter, we present our recent advances in the formulation and development of an in-vehicle environmental sniffing framework previously presented in [1,2,3,4]. The system is comprised of different components to detect, classify and track acoustic environmental conditions. The first goal of the framework is to seek out detailed information about the environmental characteristics instead of just detecting environmental change points. The second goal is to organize this knowledge in an effective manner to allow intelligent decisions to direct subsequent speech processing systems. After presenting our proposed in-vehicle environmental sniffing framework, we consider future directions and present discussion on supervised versus unsupervised noise clustering, and closed-set versus open-set noise classification.

Augmented Reality (AR) can seamlessly combine a real scene viewed by a user and a virtual component generated by a computer. This work introduces a system architecture integrating augmented reality technology with state-of-art computer vison techniques such as image semantic segmentation and style colorization. The proposed production system, ARNature, is able to superimpose a virtual scene, audio, and other enhancements in real time over a realworld environment for enhancing tourism experience. ARNature: A lot times, tourists have limited money and time to experience a tourist site during different seasons or weather conditions. The visitors are able to go on an augmented reality journey using an AR device, such as HoloLens, tablet or cellphones, and interact with real objects in a natural scene. Different enhancements of a tourism site are digitally overlaid onto visitors’ direct field of vision in real time. In addition, a voice module can be used to play music and provide additional information. Related algorithms, system design, and simulation results for a prototype ARNature system are presented. Furthermore, a no-reference image quality measure, Naturalness Image Quality Evaluator (NIQE), was utilized to evaluate the immersiveness and naturalness of ARNature. The results demonstrate that ARNature has the ability to enhance tourist experience in a truly immersive manner.

Detection and recognition of loud sounds and characteristic noises can significantly increase the level of safety and ensure timely response to various emergency situations. Audio event detection is the first step in recognizing audio signals in a continuous audio input stream. This article presents a number of problems that are associated with the development of sound event detection systems, such as the deviation for each environment and each sound category, overlapping audio events, unreliable training data, etc. Both methods for detecting monophonic impulsive audio event and polyphonic sound event detection methods which are used in the state-of-the-art sound event detection systems are presented. Such systems are presented in Detection and Classification of Acoustic Scenes and Events (DCASE) challenges and workshops, which take place every year. Beside a majority of works focusing on the improving overall performance in terms of accuracy many other aspects have also been studied. Several systems presented at DCASE 2021 task 4 were considered, and based on their analysis, there was a conclusion about possible future for sound event detection systems. Also the actual directions in the development of modern audio analytics systems are presented, including the study and use of various architectures of neural networks, the use of several data augmentation techniques, such as universal sound separation, etc.

We introduce a novel use of wearable technology to reconstruct and output high dynamic range (HDR) audio signals. The purpose of designing an end-to-end HDR audio wearable device is to create an audio device capable of replicating and even extending the dynamic range of human hearing. We also introduce a different method of visualizing audio by way of a rotary Sequential Wave Imprinting Machine (SWIM). The results show that real-time HDR audio processing and visualization are possible at the standard audio sampling rate of 44.1 kHz. Additionally, the quality of the recorded audio improved from the originally captured data after HDR processing. We further analyze the dependence of different audio gain combinations as well as the influence of the number of audio input channels on the quality of the HDR signal reconstruction in various acoustic environments.

Automatic Speaker Verification (ASV) is being implemented in many applications, where maximum security and robustness against attacks must be guaranteed. One of the most challenging attacks that an ASV system can face is the so called logical access attack, in which the attacker has the possibility to directly inject a compromised audio sample into the system. The development of countermeasures for this kind of attack has received little attention to date. When the injected audio is identical to a sample previously seen by the system, current audio fingerprinting techniques can detect most of these attacks. However, we show that, with trivial modifications that do not require any special signal processing knowledge, the audio can bypass these countermeasures while keeping the ability of the ASV system to authenticate the user. To address this issue, we propose an alternative method and validate it against a variety of audio perturbations. It generalizes previous fingerprinting techniques and acquire robustness against changes of tempo.

While a reliable protection against illegal copies does not exists today, tracking of illegal copies and prove of ownership are important detection functions, which can be realized by using passive security mechanisms of digital watermarking. Recent research has identified many watermarking algorithms for all common media types ranging from printed matter to multimedia files. The main topics of interest concentrates on transparency and robustness: The watermark must not reduce media quality and should be detectable after most common media operations and attacks. Algorithm security is discussed with regards only to key space most of the times, while especially for customer identification known as active fingerprinting specialized attacks like coalition attacks are known. Digital fingerprinting raises the additional problem that we produce different copies for each customer. Attackers can compare several fingerprinted copies to find and destroy the embedded identification string by altering the data in those places where a difference was detected. Few approaches have been introduced for image and video watermarking schemes, but there are no observations for audio fingerprinting techniques. In our paper we discuss methods for secure customer identification by digital fingerprinting for audio data. We describe first two algorithms by Boneh et al. [BoSh95] and Schwenk et al. [DBS+99] and then combine these schemes with an audio watermarking algorithm for practical evaluation of their coalition resistance to detect illegal copies. We provide test results and evaluate the security against different types of coalition attacks.

This work presents an adapted version of the Computational Geometry Algorithm (CGA) used for the development of audio-based applications and services. The CGA algorithm analyses an audio stream and produces a unique set of points that can be considered to be the audio data “fingerprint”. It is shown that this fingerprint is coding-independent, a fact that can render the proposed algorithm suitable for multiple purposes, including the categorisation of content identity and the identification of audio clips, hence providing support for the realisation of audio sorting/searching tasks and services. Additionally, based on specific novel applications and services, the overall algorithmic performance and efficiency characteristics of the CGA algorithm are discussed and analysed.