GOL-SFSTS based few-shot learning mechanical anomaly detection using multi-channel audio signal

Generally, a virtual reality (VR) system cannot provide the realistic sound formed by the multi-channel audio signals to a user with the stereo headphone environment. In addition, the VR system has a gap between the visual scene and the sound because it supplies the audio signal having only a constant sound scene without respect to the change of the user's position. To solve these problems, we introduce the sound scene control of binaural sound. Binaural sound is a stereo realistic sound that can be generated by convolving a 10.1 channel audio signal with a head related transfer function (HRTF) coefficients which features all the paths from the multi-channel speaker layout to the ears in free space. However, since a binaural sound is generated using a fixed multi-channel layout and HRTF coefficients, the binaural sound has a constant sound scene and cannot reflect the user's movements. So, we apply the sound scene control scheme that modifies the binaural sound to allow the user's movement in the VR system. Initially, a multichannel layout is re-created according to the user's azimuth change and the original multi-channel signal is mapped to a new ultra multi-channel layout using a constant power panning law. Secondly, the sound level of the new multichannel audio signal is controlled by the user's distance change, using the characteristics of the sound level inversely proportional to distance. Finally, the final multi-channel audio signal is convolved with the HRTF coefficients to produce a binaural sound with the controlled sound scene. As a result, the proposed realistic audio sound generation method allows the current VR system to provide true VR service without distinction between the visual scenes and sounds.

Usually multimedia data have to be compressed before transmitting, and higher compression rate, or equivalently lower bitrate, relieves the load of communication channels but impacts negatively the quality. We investigate the bitrate lower bound for perceptually lossless compression of a major type of multimedia—multichannel audio signals. This bound equals to the perceptible information rate of the signals. Traditionally, Perceptual Entropy (PE), based primarily on monaural hearing measures the perceptual information rate of individual channels. But PE cannot measure the spatial information captured by binaural hearing, thus is not suitable for estimating Spatial Audio Coding (SAC) bitrate bound. To measure this spatial information, we build a Binaural Cue Physiological Perception Model (BCPPM) on the ground of binaural hearing, which represents spatial information in the physical and physiological layers. This model enables computing Spatial Perceptual Entropy (SPE), the lower bitrate bound for SAC. For real-world stereo audio signals of various types, our experiments indicate that SPE reliably estimates their spatial information rate. Therefore, “SPE plus PE” gives lower bitrate bounds for communicating multichannel audio signals with transparent quality.

Provided are an apparatus and a method for reproducing a surround wave field using wave field synthesis. The apparatus includes an audio signal analyzer for analyzing a received multi-channel audio signal to check the number of audio signal channels, and extracting a sound source signal for each checked channel from the multi-channel audio signal; a wave field synthesis renderer for localizing the extracted sound source signal for each channel at a virtual sound image outside a narrow space using wave field synthesis so that the extracted sound source signal is suitable for the number of the checked audio signal channels; and an audio reproducer for reproducing the localized virtual sound source signal.

A system and method of audio processing provides enhanced speech recognition. Audio input is received at a plurality of microphones. The multi-channel audio signal from the microphones may be processed by a beamforming network to generate a single-channel enhanced audio signal, on which voice activity is detected. Audio signals from the microphones are additionally processed by an adaptable noise cancellation filter having variable filter coefficients to generate a noise-suppressed audio signal. The variable filter coefficients are updated during periods of voice inactivity. A speech recognition engine may apply a speech recognition algorithm to the noise-suppressed audio signal and generate an appropriate output. The operation of the speech recognition engine and the adaptable noise cancellation filter may advantageously be controlled based on voice activity detected in the single-channel enhanced audio signal from the beamforming network.

Few-shot learning (FSL) is designed to equip models with the capability to quickly adapt to new, unseen domains in open-world scenarios. However, there is a notable discrepancy between the multitude of new concepts encountered in the open world and the limited scale of existing FSL studies, which focus predominantly on a small number of novel classes. This limitation hinders the practical implementation of FSL in real-world situations. To address this issue, we introduce a novel problem called Few-Shot Learning with Many Novel Classes (FSL-MNC), which expands the number of novel classes more than 500 times compared to traditional FSL settings. This new challenge presents two main difficulties: increased computational load during meta-training and reduced classification accuracy due to the larger number of classes during meta-testing. To tackle these problems, we introduce the Simple Hierarchy Pipeline (SHA-Pipeline). In response to the inefficiency of traditional Episode Meta-Learning (EML) protocols, we redesign a more efficient meta-training strategy to manage the increased number of novel classes. Moreover, to distinguish distinct semantic features across a broad array of novel classes, we effectively reconstruct and utilize class hierarchy information during meta-testing. Our experiments demonstrate that the SHA-Pipeline substantially outperforms both the ProtoNet baseline and current leading alternatives across various numbers of novel classes.

Sounds provide us with vast amounts of information about surrounding objects and can even remind us visual images of them. Is it possible to implement this noteworthy human ability on machines? In this paper, we study a new task that consists of predicting image recognition results in the form of semantic segmentation with given multichannel audio signals. Our approach uses a convolutional neural network that is designed to directly output semantic segmentation results by taking audio features as its inputs. A bilinear feature fusion scheme is incorporated that efficiently models underlying higher-order interactions between audio and visual sources. Experimental evaluations with both synthetic and real sound datasets show that our approach can recover the desired segmented images reasonably well.

A Survey on Anomaly Detection with Few-Shot Learning

With the advancement of technologies like 5G, cloud computing, and microservices, the complexity of network management systems and the variety of technical components have greatly increased. This rise in complexity has rendered traditional operations and maintenance methods inadequate for current monitoring and maintenance demands. Consequently, artificial intelligence for IT operations (AIOps), which harnesses AI and big data technologies, has emerged as a solution. AIOps plays a crucial role in enhancing service quality and customer satisfaction, boosting engineering productivity, and reducing operational costs. This article delves into the primary tasks involved in AIOps, such as anomaly detection, and log fault analysis and classification. A significant challenge identified in many AIOps tasks is the scarcity of fault sample data, indicating a natural alignment of these tasks with few-shot learning. Inspired by model-agnostic meta-learning (MAML), we propose a new anomaly detector, MAML-KAD, for application in various AIOps tasks. Observations confirm that meta-learning algorithms effectively enhance AIOps tasks, showcasing the wide-ranging application prospects of meta-learning algorithms in the field of AIOps. Moreover, we introduced an AIOps platform that embeds meta-learning within its diagnostic core and features streamlined log collection, caching, and alerting to automate the AIOps workflow.

Sound Source Localization (SSL) is critical in industrial environments for monitoring equipment status and ensuring operator safety. Current SSL methods face a trade-off between accuracy and model complexity. Therefore, developing precise and lightweight SSL methods for devices with limited computational resources remains an urgent necessity. This paper introduces Latent Space Kernel Least Mean Square (LS-KLMS), a novel lightweight online learning algorithm designed for accurate and efficient SSL. LS-KLMS integrates Latent Space Model (LSM) with the Kernel Least Mean Square (KLMS) framework, enabling the transformation of high-dimensional audio data into a low-dimensional latent space and subsequently mapping it into a Reproducing Kernel Hilbert Space (RKHS). Within RKHS, linear filtering establishes nonlinear relationships between multi-channel audio signals and source coordinates, maintaining the lightweight and online learning capabilities inherent to Kernel Adaptive Filtering (KAF). An SSL system utilizing eight <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathbf{6 4}$</tex>-channel microphone arrays is deployed in a real-world hydropower station to evaluate the performance of LS-KLMS against popular existing SSL methods. Experimental results demonstrate that LS-KLMS achieves superior localization accuracy while maintaining a lightweight structure and exhibiting stable, rapid convergence. These attributes make LS-KLMS a promising solution for accurate and efficient SSL in resource-limited industrial applications.

In this paper, we propose a novel method for the detection of road accidents by analyzing audio streams for road surveillance application. In the last decade, due to the increase in a number of people and transportation vehicles, traffic accidents have become one of a major public issue worldwide. The vast number of injuries and death due to road traffic accident reveals the story of a global crisis of road safety. A total of 52,160 road traffic accidents (RTA), 1130 injuries and 85 fatalities were registered during the year 2000, in the state of Qatar. An increase in the number of transportation vehicles around cities has risen the need for more security and safety in public environments. The most obvious reason for a person's death during accidents is the absence or prolong response time of the first aid facility, which is due to the delay in the information of the accident being reached to the police, hospital or ambulance team. In the last couple of years, several surveillance systems have been proposed based on image and video processing for automatically detecting road accidents and car crashes to ensure a quick response by emergency teams. However, in some situations such as adverse weather conditions or cluttered environments, the visual information is not sufficient enough, whereas analyzing the audio tracks can significantly improve the overall reliability of surveillance systems. In this paper we propose a novel method that automatically identifies hazardous situations such as tire skidding and car crashes in presence of background noises, by analyzing the audio streams. Previous studies show that methods for the detection, estimation, and classification of nonstationary signals can be enhanced by utilizing the time-frequency (TF) characteristics of such signals. The TF-based techniques have been proved to outperform classical techniques based on either time or frequency domains, in analyzing real-life nonstationary signals. Time-frequency distributions (TFDs) give additional information about signals that cannot be extracted from either the time domain or frequency domain representations, i.e. the instantaneous frequency of the components of a signal. In order to utilize this extra information provided by TF domain representation, the proposed approach extracts TF image features from quadratic time-frequency distributions (QTFDs), for the detection of audio anomalies. The extended modified-B distribution (EMBD) is utilized to transform a 1-dimensional audio signal into 2-dimensional TF representation which is interpreted as an image. The image descriptors based features are then extracted from the TF representation to classify visually the audio signals into background or abnormal activity patterns in the TF domain. The proposed features are based on Haralick's texture features extracted from the TF representation of audio signals considered and processed as a textured image. These features are used to characterize and hence classify audio signals into M classes. This research study demonstrates that a TF image pattern recognition approach offers significant advantages over standard signal classification methods that utilize either t-domain only or f-domain only features. The proposed method has been experimentally validated on a large open source database of sounds, including several kinds of background noise. The events to be recognized are superimposed on different background sounds of roads and traffic jam. The obtained results are compared with a recent study, utilizing the same large and complex data set of audio signals, and the same experimental setup. The overall classification results confirm the superior performance of the proposed approach with accuracy improvement up to 6%.

An system, method, and method of encoding/decoding a multi-channel audio signal, including a decoding level generation unit producing decoding-level information that helps a bitstream including a number of audio channel signals and space information to be decoded into a number of audio channel signals, wherein the space information includes information about magnitude differences and/or similarities between channels, and an audio decoder decoding the bitstream according to the decoding-level information. Accordingly, even a single input bitstream can be decoded into a suitable number of channels depending on the type of a speaker configuration used. Scalable channel decoding can be achieved by partially decoding an input bitstream. In the scalable channel decoding, a decoder may set decoding levels and outputs audio channel signals according to the decoding levels, thereby reducing decoding complexity.

Sound source localization in dynamic environments with multiple moving speakers presents significant challenges due to reverberation, noise, and unknown source counts. To address these issues, this paper proposes an integrated deep-learning framework combining spatial spectrum estimation with blind source detection. The method employs a causal convolution–recurrent network (SRP-DPCRN) to extract robust spatial features from multichannel audio signals under adverse acoustic conditions. Subsequently, an iterative attention-based detection network (IASDNet) automatically identifies active sources from the estimated spatial spectrum without requiring prior knowledge of source quantity. Evaluated on both simulated datasets and the real-recorded LOCATA benchmark, the proposed system demonstrates superior performance in multi-source tracking scenarios, achieving an average detection accuracy of 96% with mean angular error below 3.5 degrees. The results confirm that joint optimization of feature learning and source counting provides an effective solution for blind localization in practical applications, significantly outperforming conventional and deep-learning baselines.

We propose in this paper a backward-compatible multichannel audio codec. This codec represents a multichannel audio input signal by a down mix and parametric data. In order to enable backward compatibility, it is necessary to have the possibility of exerting control over the down-mixing procedure. At the same time, in order to achieve a high coding efficiency, both signal and perceptual redundancies should be exploited. In this paper, we describe a codec that unifies the above-mentioned conditions: backward compatibility and exploitation of both signal and perceptual redundancies. The codec combines a high audio quality and a low parameter bit rate. Moreover, its design is flexible, examples of which are the scalability of the audio quality to (in principle) transparency and the possibility to preserve the correlation structure of the original input signals by using synthetic signals. A stereo backward compatible version of the proposed codec is used as a component of the recently standardized MPEG Surround multichannel audio codec.

This paper proposes a new multi-channel network traffic anomaly detection method combined with the idea of multi-scale decomposition and multi-channel detection theory. It can be learned that anomalies could change the characteristics of traffic data at different scales. Traditional anomaly detection methods usually work on each scale independently thus mainly focused on temporally correlated traffic. With the fully exploration on internal frequency-time correlations within multiple scales, this method first obtained the multi-scale decomposition of original traffic data using Ensemble Empirical Mode Decomposition (EEMD), then it is combined with a multi-channel Generalized Likelihood Ratio Test (GLRT) detector, for anomaly detection and decision-making. It can be verified with experiments that this method performs better than other traditional methods, thus gives a new sight on the anomaly detection with different types of traffic data.

For multi-channel signals with a wide range of amplitude and frequency, a real-time, high-speed digital heterogeneous microsystem is proposed in this work based on FPGA and application processor, with both time and frequency domain processing capabilities, while the size is only 16.2cm *14.4cm. The system can provide real-time time and frequency domain data to the application processor unit(APU) in the form of interruption or query. The system uses serial peripheral interface(SPI) to communicate with analog-to-digital converter(ADC) and enhanced local bus controller(eLBC) to communicate with APU to achieve high-speed data transmission between different modules. Windowing fast Fourier transform(FFT) is used in this system, which improves the accuracy of the single frequency signal. A specially designed ping-pong buffer is used in the system so that the data can be accessed by the APU at any time. The file system acceleration is proposed to speed up data storage. The system can achieve real-time, high-speed multi-channel discrete signal common mode detection and DC blocking, preprocessing, windowing FFT, data storage. Experimental results show the bandwidth between FPGA and APU can reach 66Mb/s. And files writing speed can reach 1.07MB/s. For signals in the range of 100HZ~9 KHz, the acquiring and processing error is within 4.5%.