Decoding Hand Trajectory from Primary Motor Cortex ECoG Using Time Delay Neural Network

Developing electrophysiological recordings of brain neuronal activity and their analysis provide a basis for exploring the structure of brain function and nervous system investigation. The recorded signals are typically a combination of spikes and noise. High amounts of background noise and possibility of electric signaling recording from several neurons adjacent to the recording site have led scientists to develop neuronal signal processing tools such as spike sorting to facilitate brain data analysis. Spike sorting plays a pivotal role in understanding the electrophysiological activity of neuronal networks. This process prepares recorded data for interpretations of neurons interactions and understanding the overall structure of brain functions. Spike sorting consists of three steps: spike detection, feature extraction, and spike clustering. There are several methods to implement each of spike sorting steps. This paper provides a systematic comparison of various spike sorting sub-techniques applied to real extracellularly recorded data from a rat brain basolateral amygdala. An efficient sorted data resulted from careful choice of spike sorting sub-methods leads to better interpretation of the brain structures connectivity under different conditions, which is a very sensitive concept in diagnosis and treatment of neurological disorders. Here, spike detection is performed by appropriate choice of threshold level via three different approaches. Feature extraction is done through PCA and Kernel PCA methods, which Kernel PCA outperforms. We have applied four different algorithms for spike clustering including K-means, Fuzzy C-means, Bayesian and Fuzzy maximum likelihood estimation. As one requirement of most clustering algorithms, optimal number of clusters is achieved through validity indices for each method. Finally, the sorting results are evaluated using inter-spike interval histograms.

In this research, we study the effect of feature selection in the spike detection and sorting accuracy. We introduce a new feature representation for neural spikes from multichannel recordings. The features selection plays a significant role in analyzing the response of brain neurons. The more precise selection of features leads to a more accurate spike sorting, which can group spikes more precisely into clusters based on the similarity of spikes. Proper spike sorting will enable the association between spikes and neurons. Different with other threshold-based methods, the cepstrum of spike signals is employed in our method to select the candidates of spike features. To choose the best features among different candidates, the Kolmogorov-Smirnov (KS) test is utilized. Then, we rely on the super paramagnetic method to cluster the neural spikes based on KS features. Simulation results demonstrate that the proposed method not only achieve more accurate clustering results but also reduce computational burden, which implies that it can be applied into real-time spike analysis.

Spike sorting is a fundamental process for decoding neural activity, involving preprocessing, spike detection, feature extraction, clustering, and validation. However, conventional spike sorting methods are highly fragmented, labor-intensive, and heavily reliant on expert manual curation, limiting their scalability and reproducibility. This challenge has become more pressing with advances in neural recording technology, such as high-density Neuropixels for large-scale neural recording or flexible electrodes for long-term stable recording over months to years. The volume and complexity of these datasets make manual curation infeasible, requiring an automated and scalable solution. Here, we introduce SpikeAgent, a multimodal large language model (LLM)-based AI agent that automates and standardizes the entire spike sorting pipeline. Unlike traditional approaches, SpikeAgent integrates multiple LLM backends, coding functions, and established algorithms, autonomously performing spike sorting with reasoning-based decision-making and real-time interaction with intermediate results. It generates interpretable reports, providing transparent justifications for each sorting decision, enhancing transparency and reliability. We benchmarked SpikeAgent against human experts across various neural recording technology, demonstrating its versatility and ability to achieve curation consistency that are equal to, or even higher than human experts. It also drastically reduces the expertise barrier and accelerates the curation and validation time by orders of magnitude. Moreover, it enables automated interpretability of the neural spiking data, which cannot be achieved by any conventional methods. SpikeAgent presents a paradigm shift in processing signals for neuroscience and brain-computer interfaces, while laying the ground for AI agent-augmented science across various domains.

This paper investigated the speaker independent vowel recognition for Malay children using the Time Delay Neural Network (TDNN). Due to less research done on the children speech recognition, the temporal structure of the children speech was not fully understood. This 2 hidden layers TDNN was proposed to discriminate 6 Malay vowels: /a/, /e/, /ə/, /i/, /o/ and /u/. The speech database consisted of vowel sounds from 360 children speakers. Cepstral coefficient was normalized for the input of TDNN. The frame rate of the TDNN was tested with 10ms, 20ms, and 30ms. It was found out that the 30ms frame rate produced the highest vowel recognition accuracy with 81.92%. The TDNN also showed higher speech recognition rate compared to the previous studies that used Multilayer Perceptron.

Prediction of hourly solar radiation using a novel hybrid model of ARMA and TDNN

Recently, x-vector [1] has been a successful and popular approach for speaker verification, which employs a time delay neural network (TDNN) and statistics pooling to extract speaker characterizing embedding from variable-length utterances. Improvement upon the x-vector has been an active research area, and enormous neural networks have been elaborately designed based on the x-vector, e.g., extended TDNN (E-TDNN) [2], factorized TDNN (F-TDNN) [3], and densely connected TDNN (D-TDNN) [4]. In this work, we try to identify the optimal architectures from a TDNN based search space employing neural architecture search (NAS), named SpeechNAS. Leveraging the recent advances in the speaker recognition, such as high-order statistics pooling, multi-branch mechanism, D-TDNN and angular additive margin softmax (AAM) loss with a minimum hyper-spherical energy (MHE), SpeechNAS automatically discovers five network architectures, from SpeechNAS-1 to SpeechNAS-5, of various numbers of parameters and GFLOPs on the large-scale text-independent speaker recognition dataset VoxCelebl. Our derived best neural network achieves an equal error rate (EER) of 1.02% on the standard test set of VoxCelebl, which surpasses previous TDNN based state-of-the-art approaches by a large margin.

In this paper, an evaluation of various discriminant neural networks classifiers for text- independent speaker verification problem is presented. Each person to be verified has a personalized neural network model. A new classifier called neural tree network (NTN) is also examined for this application. The memoryless feedforward neural network architecture makes decisions based on static features. Time delay neural network (TDNNs) have proved to be an efficient way to handle the dynamic nature of speech. Furthermore, a model called recurrent time delay neural networks (RTDNNs), obtained through a local feedback connection at the first hidden layer level of TDNNs is investigated. The training is carried out by backpropagation for sequence algorithm. The database used is a subset of the TIMIT database consisting of 38 speakers from the same dialect region. The NTN is compared with the MLP, TDNN, and RTDNN. It is shown that NTN is found to perform better than the other neural networks classifiers. Also, a little bit performance improvement was achieved due to the addition of temporal information for text-independent speaker verification problem using TDNNs and RTDNNs. Finally, we described the experimental results obtained using different neural network models.© (1994) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

Spike sorting, i.e., the separation of the firing activity of different neurons from extracellular measurements, is a crucial but often error-prone step in the analysis of neuronal responses. Usually, three different problems have to be solved: the detection of spikes in the extracellular recordings, the estimation of the number of neurons and their prototypical (template) spike waveforms, and the assignment of individual spikes to those putative neurons. If the template spike waveforms are known, template matching can be used to solve the detection and classification problem. Here, we show that for the colored Gaussian noise case the optimal template matching is given by a form of linear filtering, which can be derived via linear discriminant analysis. This provides a Bayesian interpretation for the well-known matched filter output. Moreover, with this approach it is possible to compute a spike detection threshold analytically. The method can be implemented by a linear filter bank derived from the templates, and can be used for online spike sorting of multielectrode recordings. It may also be applicable to detection and classification problems of transient signals in general. Its application significantly decreases the error rate on two publicly available spike-sorting benchmark data sets in comparison to state-of-the-art template matching procedures. Finally, we explore the possibility to resolve overlapping spikes using the template matching outputs and show that they can be resolved with high accuracy.

This thesis was to solve the problem that tremendous amount of data of neural signal seriously confined the number of transmission channels with a limited bandwidth in a digital telemetry system. Although a spike-detection algorithm designed for data reduction had been reported, the signal in between the spikes was lost. Moreover, various non-ideal factors greatly reduced the accuracy of spike detection. This thesis used the commercialized data-compression algorithm, MPEG Layer 3 (MP3), as our data reduction scheme and evaluated its performance on spike sorting. The evaluation results showed that MP3 kept at least 80% accuracy of unit isolation. For reducing the consumption of hardware resource by MP3, MP3 algorithm was further simplified and the corresponding performance was checked. All the algorithm simplification carried out in this thesis compromised barely discernible performance. At last this thesis discussed a method for multi-channel signal analysis, ICA. The effect of MP3 on ICA after the highest level of simplification was simulated. The results showed that the miss probabilities of MP3 were considerably low, regardless of the encoding bit rate. According to all the results mentioned above, we preliminarily validated MP3 a proper neural signal compressor.

Cardiovascular pathologies cause 23.5% of human deaths, worldwide. An auto-diagnostic system monitoring heart activity, which can identify the early symptoms of cardiac illnesses, might reduce the death rate caused by these problems. Phonocardiography (PCG) is one of the possible techniques able to detect heart problems. Nevertheless, acoustic signal enhancement is required since it is exposed to various disturbances coming from different sources. The most common denoising enhancement is based on the Wavelet Transform (WT). However, the WT is highly susceptible to variations in the noise frequency distribution. This paper proposes a new adaptive denoising algorithm, which combines WT and Time Delay Neural Networks (TDNN). The acquired signal is decomposed by means of the WT using the coif five-wavelet basis at the tenth decomposition level and then provided as input to the TDNN. Besides the advantage of adaptive thresholding, the reason for using TDNNs is their capacity of estimating the Inverse Wavelet Transform (IWT). The best parameters of the TDNN were found for a NN consisting of 25 neurons in the first and 15 in the second layer and the delay block of 12 samples. The method was evaluated on several pathological heart sounds and on signals recorded in a noisy environment. The performance of the developed system with respect to other wavelet-based denoising approaches was validated by the online questionnaire.

Neural networks with time delays offer the potential of providing massive parallelism and adaptation, and can provide excellent discrimination for temporal signal processing tasks. In a time delay neural network (TDNN), the shift-invariance capability required in speech recognition is achieved by explicitly making time-shifted copies of the inputs and internal responses and linking their corresponding weights. In this paper, we propose a limited feedback TDNN (LFB-TDNN), which feedbacks only those errors with significant magnitudes over a limited time span during the semi-batch back-propagation learning. Simulations show that the LFB-TDNN achieves favorable performance in isolated speech recognition tasks when compared with the standard TDNN, the discrete density and the continuous density hidden Markov models.

Time-delayed neural networks (TDNNs) can be used to learn the dynamics of an unknown system from input-output data. In many cases a model of the system is also available in the form of a system of ODES, derived either from first principles or using heuristic arguments. In such cases a functional comparison can be made between the dynamic behaviour of the model with that of the trained TDNN for arbitrary inputs. We show that Volterra kernels for both the model system and the TDNN can be obtained, and thus the system responses compared, in a manner that is independent of the input. The techniques of structural bilinearisation of ODES and Volterra series expansion of a TDNN, are demonstrated by application to the Hodgkin-Huxley set of equations.

A simple method for efficient spike detection in multiunit recordings

Presents a writer independent system for on-line handwriting recognition which can handle both cursive script and hand-print. The pen trajectory is recorded by a touch sensitive pad, such as those used by note-pad computers. The input to the system contains the pen trajectory information, encoded as a time-ordered sequence of feature vectors. Features include X and Y coordinates, pen-lifts, speed, direction and curvature of the pen trajectory. A time delay neural network with local connections and shared weights is used to estimate a posteriori probabilities for characters in a word. A hidden Markov model segments the word into characters in a way which optimizes the global word score, taking a dictionary into account. A geometrical normalization scheme and a fast but efficient dictionary search are also presented. Trained on 20000 unconstrained cursive words from 59 writers and using a 25000 word dictionary the authors reached a 89% character and 80% word recognition rate on test data from a disjoint set of writers. >

The work is devoted to resolving the contradiction between the accuracy of modelling nonlinear dynamics and the speed of model construction under conditions of limited computing resources. The purposeof the work is to reduce the time for building time delay neural networks while ensuring a given accuracy in the tasks of identifying nonlinear dynamic objects with continuous characteristics. This goal is achieved by developing a method for pre-training neural networks that reflect the basic characteristics of the subject area. The scientific novelty of the work is the development of a method for identifying nonlinear dynamic objectsin the form of time delay neural networks based on the use of a set of basic pre-trained neural networks that reflect the typical properties of the subject area. In contrast to the traditional approach to pre-training, the developed method allows building models of lower complexity.A formal criterion is proposed for determining the moment of termination of the neural network pre-training, the use of which allows avoiding retraining of the base model and ensuring a significant reduction in the model training time on the target data set. The practical utility of the work lies in the development of an algorithm for the method of pre-training time delay neural networks in the tasks of identifying nonlinear dynamic objects with continuous characteristics, which allows to significantlyreduce the training time of neural networks without losing model accuracy. The value of this study is todetermine the area of effective use of the proposed method, namely, when the general and target datasets do not have significant differences and the target dataset is of sufficient size to reflect the properties of the research object.