Robotic sound-source localisation architecture using cross-correlation and recurrent neural networks

Parsing is a key task in computer science, with applications in compilers, natural language processing, syntactic pattern matching, and formal language theory. With the recent development of deep learning techniques, several artificial intelligence applications, especially in natural language processing, have combined traditional parsing methods with neural networks to drive the search in the parsing space, resulting in hybrid architectures using both symbolic and distributed representations. In this article, we show that existing symbolic parsing algorithms for context-free languages can cross the border and be entirely formulated over distributed representations. To this end, we introduce a version of the traditional Cocke–Younger–Kasami (CYK) algorithm, called distributed (D)-CYK, which is entirely defined over distributed representations. D-CYK uses matrix multiplication on real number matrices of a size independent of the length of the input string. These operations are compatible with recurrent neural networks. Preliminary experiments show that D-CYK approximates the original CYK algorithm. By showing that CYK can be entirely performed on distributed representations, we open the way to the definition of recurrent layer neural networks that can process general context-free languages.

ABSTRACTA cognitive map is an internal model of the external world and contains the spatial representation of the surrounding environment. The existence of the cognitive map was first identified in rats; rats can navigate to their desired destination using cognitive maps while dealing with environmental uncertainty. We performed a mobile robot navigation experiment where obstacles were randomly placed using hierarchical recurrent neural network (HRNN) with multiple timescales. The HRNN was trained to navigate the mobile robot to the destination indicated by a snapshot image. After the training, the HRNN was able to successfully avoid the obstacles and navigate to the destination from any location in the environment. Analysis of the internal states of the HRNN showed that the module with fast timescale handles obstacle avoidance and the one with slow timescale has spatial representation corresponding to the spatial position of the destination. Moreover, in the experiment wherein the novel path appeared, the trained HRNN performed shortcut behavior. The shortcut behavior shows that the HRNN performed navigation using the self-organized spatial representation in the slow recurrent neural network. This indicates that training of goal-oriented navigation, i.e. the navigation motivated by a snapshot image of the destination results in the self-organization of cognitive map-like representation.

Front-back reversals (FBRs) in sound-source localization tasks due to cone-of-confusion errors on the azimuth plane occur with some regularity, and their occurrence is listener-dependent. There are fewer FBRs for wideband, high-frequency sounds than for low-frequency sounds presumably because the sources of low-frequency sounds are localized on the basis of interaural differences (interaural time and level differences), which can lead to ambiguous responses. Spectral cues can aid in determining sound-source locations for wideband, high-frequency sounds, and such spectral cues do not lead to ambiguous responses. However, to what extent spectral features might aid sound-source localization is still not known. This paper explores conditions in which the spectral profile of two-octave wide noise bands, whose sources were localized on the azimuth plane, were randomly varied. The experiment demonstrated that such spectral profile randomization increased FBRs for high-frequency noise bands, presumably because whatever spectral features are used for sound-source localization were no longer as useful for resolving FBRs, and listeners relied on interaural differences for sound-source localization, which led to response ambiguities. Additionally, head rotation decreased FBRs in all cases, even when FBRs increased due to spectral profile randomization. In all cases, the occurrence of FBRs was listener-dependent.

Brain Computer Interfaces (BCI) is the generic denomination of systems aiming to establish communication between a human being and an automated system, based on the electric brain signals detected through a variety of modalities. Among these, electroencephalographic signals (EEG) have received considerable attention due to several factors arising on practical scenarios, such as noninvasiveness, portability, and relative cost, without lost on accuracy and generalization. In this chapter we discuss the characteristics of a typical phenomenon associated to motor imagery and mental tasks experiments, known as event related synchronization and desynchronization (ERD/ERS), as well as its energy distribution in the time-frequency space. The typical behavior of ERD/ERS phenomenon has led proposal of different approaches oriented to the solution of the identification problem. In this work, an architecture based on adaptive neuro-fuzzy inference systems (ANFIS) assembled to a recurrent neural network, applied to the problem of mental tasks temporal classification, is presented. The electroencephalographic signals (EEG) are pre-processed through band-pass filtering in order to separate the set of energy signals in alpha and beta bands. The energy in each band is represented by fuzzy sets obtained through an ANFIS system, and the temporal sequence corresponding to the combination to be detected, associated to the specific mental task, is entered into a recurrent neural network. Experimentation using EEG signals corresponding to mental tasks exercises, obtained from a database available to the international community for research purposes, is reported. Two recurrent neural networks are used for comparison purposes: Elman network, and a fully connected recurrent neural network (FCRNN) trained by RTRL-EKF (real time recurrent learning – extended Kalman filter). A classification rate of 88.12 % in average was obtained through the FCRNN during the generalization stage.KeywordsIndependent Component AnalysisMotor ImageryRecurrent Neural NetworkAdaptive Neuro Fuzzy Inference SystemMental TaskThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

REFEREED PAPERS

Audition is one of our most important modalities and is widely used to communicate and sense the environment around us. We present an auditory robotic system capable of computing the angle of incidence (azimuth) of a sound source on the horizontal plane. The system is based on some principles drawn from the mammalian auditory system and using a recurrent neural network (RNN) is able to dynamically track a sound source as it changes azimuthally within the environment. The RNN is used to enable fast tracking responses to the overall system. The development of a hybrid system incorporating cross-correlation and recurrent neural networks is shown to be an effective mechanism for the control of a robot tracking sound sources azimuthally.KeywordsSound SourceAuditory CortexOutput ActivationRecurrent Neural NetworkInput ActivationThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This article proposes an estimation method of power consumption for building air conditioning systems with multiple indoor units per outdoor unit. In the proposed method, a recurrent neural network is used to estimate the power consumption from current states of the air conditioning system. First, power consumption was estimated using data measured in summer and winter. As input data at this time, the preset temperature, the outside air temperature, the on-off state, and the number of people in the room that can be used for prediction by using the building operation plan and the weather forecast data were used. By this, it is verified the method for prediction of power consumption. As a result, it was possible to estimate the power consumption in summer or winter by learning the data combining summer and winter. Next, as verification toward prediction of power consumption, estimation was performed using simulated data in which the set temperature, operating conditions, and the number of people in the room were artificially created. As a result, it was shown that the power consumption can be estimated even if the transition of the input data differs from day to day, and the proposed prediction method can be used to predict the power consumption.

End-to-End Sound Source Localization using YAMNET and Recurrent Neural Network

This study introduces a new method for extracting sound from pictures by utilizing machine learning. Lately, there has been a lot of excitement around multi-modal learning because of its ability to reveal valuable information from various sources, like images and sound. Our research is centered on using the unique qualities of visual and auditory signals to predict sound content from pictures. This opens up possibilities for enhancing accessibility, creating content, and providing immersive user experiences. We start by exploring previous research in multi-modal learning, audio-visual processing, and tasks like image captioning and sound source localization. Based on this background, we introduce an approach that merges convolutional neural networks (CNNs) for image analysis with recurrent neural networks (RNNs) or transformers for sequence interpretation. The system is educated on a collection of matched images and associated audio tracks, allowing it to grasp the intricate connections between visual and auditory data. In our study, we carefully assessed the performance of our proposed method by using well-known metrics. We measure how well our method works by comparing it to other methods and showing that it can accurately and quickly extract audio from images. We also show through qualitative analysis that our model can create clear audio representations from a variety of visual inputs. After a thorough discussion, we analyze the findings, pointing out both the advantages and drawbacks of our method. We pinpoint potential areas for further study, such as delving into more advanced structures and incorporating semantic data to enhance audio extraction. To sum up, this study adds to the expanding field of multi-modal learning by introducing a promising model for extracting audio from images through machine learning. Our results emphasize the potential of this technology to improve accessibility, inspire creativity, and increase user engagement in different fields. Key Words: Audio Extraction, Machine Learning, Computer Vision, Deep Learning, Convolutional Neural Networks

Learning the underlying dynamics of periodic and aperiodic systems evolving over space and time provides substantial challenges across scientific fields. Despite recent advances in machine learning (ML), our understanding of its limits to learning, identifying, and emulating real physical systems remains incomplete. With the increasing complexity of dynamical systems and higher power consumption, ML systems take inspiration from the complex learning mechanisms of the mammalian brain, for biologically plausible learning. As we delve into the designing of brain- inspired ML systems, we deal with the challenges of the learning procedures, network sizes, energy efficiency and training complexities involved with it. It's crucial to characterize their performance in learning and controlling unknown dynamical systems to address them. We address these issues by designing sparsity aware learning mechanisms in feedback driven recurrent neural networks (RNNs), incorporate biologically plausible neuron models, and integrate computational neuroscience to build data driven AI systems. We explore brain-inspired neuronal dynamics to build AI systems that can mimic and classify dynamical systems. We propose full-FORCE (FF) SNN, Reservoir-in-reservoir (R-i-R) architecture, and SNN based applications learning, We pro- pose sparsity aware learning in RNNs, full-FORCE based sparsity regularization in recurrent spiking neural networks (RSNN) and an RSNN based BLE application for dynamical systems learning. Our proposed learning methods and architectures exhibit strong performance, noise robustness and energy efficiency in simulated and real-world adaptive system emulation. We evaluate bio signals, motion trajectories, Lorenz systems, real-world dynamic variable datasets, and other dynamical systems in this dissertation.

On-line path planning for autonomous mobile robots using recurrent neural networks

A digital discrete high-order fast bandpass filter on the TMS320F28377D processor for mobile robot navigation is developed and investigated. The application of the developed band-pass filter in beacon laser navigation systems is proposed in order to increase the speed of mobile robot coordinate computation when solving the fast convolution problem. A fast convolution algorithm based on a digital continuous band-pass Butterworth filter, which is optimal in terms of minimum computation time, is presented. The developed 22-order Butterworth filter is implemented on a digital signal processor TMS320F28377D using a double buffer with parallel recording and processing of data from ADC. The peculiarity of the bandpass filter implementation is its real-time operation for searching signals of navigation beacon lasers in a given frequency band. Two DMA direct memory access modules, CLA accelerator module of the TMS320F28377D processor are used to increase the speed of data processing. The correlation function calculated as a result of convolution of the original signal with the reference signal of a given frequency and filtered by a low-pass filter using the moving average method was obtained. The maximum of the calculated correlation function shows the time delay between the beginning of the beacon pulse transmission and the reception time at the photodiode. This delay time shows the azimuth to the beacon. Laser beacon system (LSM) for robot navigation has the following advantages compared to radio equipment: lower range measurement error, small size of optical systems compared to the size of radar antennas, and high angle resolution due to the narrow beam of sounding radiation. High noise immunity of LSM, including to deliberate interference, allows solving navigation tasks in conditions of suppression of global navigation systems (such as GPS and GLONASS), when the means of autonomous navigation cannot fully provide data to the robot control system. The convolution execution time on the TMS320F28377D processor is 1.8 ms.

Agriculture monitoring is a key task for producers, governments and decision makers. The analysis of multitemporal remote sensing data provides a cost-effective way to perform this task. Recurrent Neural Networks (RNNs) have been successfully used in temporal modeling problems, while Convolutional Neural Networks (CNNS) are the state-of-the-art in image classification, mainly due to their ability to capture spatial context. In this work, we propose the use of a hybrid network architecture for crop mapping that combines RNNs and CNNs. We evaluate this architecture experimentally upon a Sentinel-1A database from a tropical region in Brazil. The ability of recurrent networks to model temporal context is compared with the conventional image stacking approach. The impact of using CNN learned features rather than context aware handcrafted features is also investigated. In our analysis the hybrid architecture achieved better average class accuracy than alternative approaches based on image stacking and GLCM features.

In this paper, a recurrent neural network is used to develop a dynamic controller for mobile robots. The advantage of the control approach is that no knowledge about the robot model is required. This property is very useful in practical situations, where the exact knowledge about the robot parameters is almost unattainable. The proposed approach has been experimentally tested on an Omnidirectional RoboCup Player available at the Robotics Lab of the University of Stuttgart.

Sentiment analysis involves classifying text into positive, negative and neutral classes according to the emotions expressed in the text. Extensive study has been carried out in performing sentiment analysis using the traditional ‘bag of words’ approach which involves feature selection, where the input is given to classifiers such as Naive Bayes and SVMs. A relatively new approach to sentiment analysis involves using a deep learning model. In this approach, a recently discovered technique called word embedding is used, following which the input is fed into a deep neural network architecture. As sentiment analysis using deep learning is a relatively unexplored domain, we plan to perform in-depth analysis into this field and implement a state of the art model which will achieve optimal accuracy. The proposed methodology will use a hybrid architecture, which consists of CNNs (Convolutional Neural Networks) and RNNs (Recurrent Neural Networks), to implement the deep learning model on the SAR14 and Stanford Sentiment Treebank data sets.