An implantable VLSI architecture for real time spike sorting in cortically controlled Brain Machine Interfaces

Unsupervised automatic online spike sorting using reward-based online clustering

Objective. Spike sorting is a set of techniques used to analyze extracellular neural recordings, attributing individual spikes to individual neurons. This field has gained significant interest in neuroscience due to advances in implantable microelectrode arrays, capable of recording thousands of neurons simultaneously. High-density electrodes, combined with efficient and accurate spike sorting systems, are essential for various applications, including brain machine interfaces (BMIs), experimental neural prosthetics, real-time neurological disorder monitoring, and neuroscience research. However, given the resource constraints of modern applications, relying solely on algorithmic innovation is not enough. Instead, a co-optimization approach that combines hardware and spike sorting algorithms must be taken to develop neural recording systems suitable for resource-constrained environments, such as wearable devices and BMIs. This co-design requires careful consideration when selecting appropriate spike-sorting algorithms that match specific hardware and use cases. Approach. We investigated the recent literature on spike sorting, both in terms of hardware advancements and algorithms innovations. Moreover, we dedicated special attention to identifying suitable algorithm-hardware combinations, and their respective real-world applicabilities. Main results. In this review, we first examined the current progress in algorithms, and described the recent departure from the conventional ‘3-step’ algorithms in favor of more advanced template matching or machine-learning-based techniques. Next, we explored innovative hardware options, including application-specific integrated circuits, field-programmable gate arrays, and in-memory computing devices (IMCs). Additionally, the challenges and future opportunities for spike sorting are discussed. Significance. This comprehensive review systematically summarizes the latest spike sorting techniques and demonstrates how they enable researchers to overcome traditional obstacles and unlock novel applications. Our goal is for this work to serve as a roadmap for future researchers seeking to identify the most appropriate spike sorting implementations for various experimental settings. By doing so, we aim to facilitate the advancement of this exciting field and promote the development of innovative solutions that drive progress in neural engineering research.

Brain-machine interfaces (BMIs) based on extracellular recordings with microelectrodes provide means of observing the activities of neurons that orchestrate fundamental brain function, and are therefore powerful tools for exploring the function of the brain. Due to physical restrictions and risks for post-surgical complications, wired BMIs are not suitable for long-term studies in freely behaving animals. Wireless BMIs ideally solve these problems, but they call for low-complexity techniques for data compression that ensure maximum utilization of the wireless link and energy resources, as well as minimum heat dissipation in the surrounding tissues. In this paper, we analyze the performances of various system architectures that involve spike detection, spike alignment and spike compression. Performance is analyzed in terms of spike reconstruction and spike sorting performance after wireless transmission of the compressed spike waveforms. Compression is performed with transform coding, using five different compression bases, one of which we pay special attention to. That basis is a fixed basis derived, by singular value decomposition, from a large assembly of experimentally obtained spike waveforms, and therefore represents a generic basis specially suitable for compressing spike waveforms. Our results show that a compression factor of 99.8%, compared to transmitting the raw acquired data, can be achieved using the fixed generic compression basis without compromising performance in spike reconstruction and spike sorting. Besides illustrating the relative performances of various system architectures and compression bases, our findings show that compression of spikes with a fixed generic compression basis derived from spike data provides better performance than compression with downsampling or the Haar basis, given that no optimization procedures are implemented for compression coefficients, and the performance is similar to that obtained when the optimal SVD based basis is used.

While many brain machine interface (BMI) systems have been presented in the literature, most of these systems present the user with an always on interface with no way to shut the interface down when not needed. This paper proposes two extensions of previous BMI work to create an asynchronous BMI in which the system only produces outputs when needed. The first classifies incoming signals into not only task related states, but also an idle state. A refinement of this system utilizes a Markov Model (MM) of the task to impose order on the sequence of states produced by the system. This MM filter improves the accuracy of the system an average of 16%.

Transmitting large amounts of data sensed by multi-electrode array devices is widely considered to be a challenging problem in designing implantable neural recording systems. Spike sorting is an important step to reducing the data bandwidth before wireless data transmission. The feasibility of spike sorting algorithms in scaled CMOS technologies, which typically operate on low frequency neural spikes, is determined largely by its power consumption, a dominant portion of which is leakage power. Our goal is to explore energy saving architectures for online spike sorting without sacrificing performance. In the face of non-stationary neural data, training is not always a feasible option. We present a low-power architecture for real-time online spike sorting that does not require any training period and has the capability to quickly respond to the changing spike shapes.

Limitations of many Brain Machine Interface (BMI) systems using invasive electrodes include reliance on single neurons and decoding limited to kinematics only. This study investigates whether force-related information is present in the local field potential (LFP) recorded with deep brain electrodes using data from 14 patients with Parkinson's disease. A classifier based on logistic regression (LR) is developed to classify various force stages, using 10-fold cross validation. Least Absolute and Shrinkage Operator (Lasso) is then employed in order to identify the features with the most predictivity. The results show that force-related information is present in the LFP, and it is possible to distinguish between various force stages using certain frequency-domain (delta, beta, gamma) and time-domain (mobility) features in real-time.

Unsupervised spike sorting, a vital processing step in real-time brain-implantable microsystems, is faced with the prominent challenge of managing nonstationarity in neural signals. In long-term recordings, spike waveforms gradually change and new source neurons are likely to become activated. Adaptive spike sorters combined with on-implant training units effectively process the nonstationary signals at the cost of high hardware resource utilization. On the other hand, static approaches, while being hardware-friendly, are subjected to decreased processing performance in such recordings where the neural signal characteristics gradually change. To strike a balance between the hardware cost and processing performance, this study proposes a hardware-efficient novelty-aware spike sorting approach that is capable of dealing with both variated spike waveforms and spike waveforms generated from new source neurons. Its improved hardware efficiency compared to adaptive ones and capability of dealing with nonstationary signals make it attractive for implantable applications. The proposed novelty-aware spike sorting especially would be a good fit for brain-computer interfaces where long-term, real-time interaction with the brain is required, and the available on-implant hardware resources are limited. Our unsupervised spike sorting benefits from a novelty detection process to deal with neural signal variations. It tracks the spike features so that in case of detecting an unexpected change (novelty detection) both on and off-implant parameters are updated to preserve the performance in new state. To make the proposed approach agile enough to be suitable for brain implants, the on-implant computations are reduced while the computational burden is realized off-implant. The performance of our proposed approach is evaluated using both synthetic and real datasets. The results demonstrate that, in the mean, it is capable of detecting 94.31% of novel spikes (wave-drifted or emerged spikes) with a classification accuracy (CA) of 96.31%. Moreover, an FPGA prototype of the on-implant circuit is implemented and tested. It is shown that in comparison to the OSORT algorithm, a pivotal spike sorting method, our spike sorting provides a higher CA at significantly lower hardware resources. The proposed circuit is also implemented in a 180-nm standard CMOS process, achieving a power consumption of 1.78[Formula: see text][Formula: see text] per channel and a chip area of 0.07[Formula: see text]mm2 per channel.

Error-related potentials (ErrPs) have been proposed as a means for improving brain–computer interface (BCI) performance by either correcting an incorrect action performed by the BCI or label data for continuous adaptation of the BCI to improve the performance. The latter approach could be relevant within stroke rehabilitation where BCI calibration time could be minimized by using a generalized classifier that is continuously being individualized throughout the rehabilitation session. This may be achieved if data are correctly labelled. Therefore, the aims of this study were: (1) classify single-trial ErrPs produced by individuals with stroke, (2) investigate test–retest reliability, and (3) compare different classifier calibration schemes with different classification methods (artificial neural network, ANN, and linear discriminant analysis, LDA) with waveform features as input for meaningful physiological interpretability. Twenty-five individuals with stroke operated a sham BCI on two separate days where they attempted to perform a movement after which they received feedback (error/correct) while continuous EEG was recorded. The EEG was divided into epochs: ErrPs and NonErrPs. The epochs were classified with a multi-layer perceptron ANN based on temporal features or the entire epoch. Additionally, the features were classified with shrinkage LDA. The features were waveforms of the ErrPs and NonErrPs from the sensorimotor cortex to improve the explainability and interpretation of the output of the classifiers. Three calibration schemes were tested: within-day, between-day, and across-participant. Using within-day calibration, 90% of the data were correctly classified with the entire epoch as input to the ANN; it decreased to 86% and 69% when using temporal features as input to ANN and LDA, respectively. There was poor test–retest reliability between the two days, and the other calibration schemes led to accuracies in the range of 63–72% with LDA performing the best. There was no association between the individuals’ impairment level and classification accuracies. The results show that ErrPs can be classified in individuals with stroke, but that user- and session-specific calibration is needed for optimal ErrP decoding with this approach. The use of ErrP/NonErrP waveform features makes it possible to have a physiological meaningful interpretation of the output of the classifiers. The results may have implications for labelling data continuously in BCIs for stroke rehabilitation and thus potentially improve the BCI performance.

This paper presents a review and analysis of embroidered textile antennas for wireless body-centric communication systems. Especially, the paper concentrates on recent advancements in embroidered on-body radio frequency identification (RFID) antennas and wireless brain machine interface (BMI) systems and their specific antenna and wireless power transfer challenges.

The purpose of this study was to explore voice activity detection (VAD) in a subject with implanted electrocorticographic (ECoG) electrodes. Accurate VAD is an important preliminary step before decoding and reconstructing speech from ECoG. For this study we used ECoG signals recorded while a subject performed a picture naming task. We extracted time-domain features from the raw ECoG and spectral features from the ECoG high gamma band (70-110Hz). The RelieF algorithm was used for selecting a subset of features to use with seven machine learning algorithms for classification. With this approach we were able to detect voice activity from ECoG signals, achieving a high accuracy using the 100 best features from all electrodes (96%) or only 12 features from the two best electrodes (94%) using the support vector machines or a linear regression classifier. These findings may contribute to the development of ECoG-based brain machine interface (BMI) systems for rehabilitating individuals with communication impairments.KeywordsVoice activity detectionelectrocorticographybrain machine interfacemachine learning

Neural decoding and its applications to brain computer interfaces (BCI) are essential for understanding the association between neural activity and behavior. A prerequisite for many decoding approaches is spike sorting, the assignment of action potentials (spikes) to individual neurons. Current spike sorting algorithms, however, can be inaccurate and do not properly model uncertainty of spike assignments, therefore discarding information that could potentially improve decoding performance. Recent advances in high-density probes (e.g., Neuropixels) and computational methods now allow for extracting a rich set of spike features from unsorted data; these features can in turn be used to directly decode behavioral correlates. To this end, we propose a spike sorting-free decoding method that directly models the distribution of extracted spike features using a mixture of Gaussians (MoG) encoding the uncertainty of spike assignments, without aiming to solve the spike clustering problem explicitly. We allow the mixing proportion of the MoG to change over time in response to the behavior and develop variational inference methods to fit the resulting model and to perform decoding. We benchmark our method with an extensive suite of recordings from different animals and probe geometries, demonstrating that our proposed decoder can consistently outperform current methods based on thresholding (i.e. multi-unit activity) and spike sorting. Open source code is available at https://github.com/yzhang511/density_decoding.

Brain-Machine Interfaces (BMI) establish a new information communication and control channel between the brain and peripheral devices, which independent of the spinal cord and peripheral nervous system. In recent years, it has become a hotspot research in the field of international intelligent science. However, the current BMI usually belong to synchronous system, which requires subjects to perform specific tasks at specific program setting, which greatly limits the development of BMI system. The development of asynchronous BMI has attracted extensive attention of researchers, whose study focus on the detection of idle state. Hence, this paper proposed a novel asynchronous BMI detection method for idle and working state recognition, which is based on Hurst exponent and Support Vector Machine (SVM). In this work, an asynchronous BMI system based on facial expression paradigm was constructed to control the assembly robot. The method of Hurst exponent and SVM was used to recognize the asynchronous state and different facial expressions. Then, offline and online experiments were constructed to verify the effective of proposed method. The experimental results demonstrated that there is a large difference in the Hurst exponent between different facial expressions state and the idle state, and the average accuracy of SVM classification results can reach more than 90%.

Objective. The OSort algorithm, a pivotal unsupervised spike sorting method, has been implemented in dedicated hardware devices for real-time spike sorting. However, due to the inherent complexity of neural recording environments, OSort still grapples with numerous transient cluster occurrences during the practical sorting process. This leads to substantial memory usage, heavy computational load, and complex hardware architectures, especially in noisy recordings and multi-channel systems. Approach. This study introduces an optimized OSort algorithm (opt-OSort) which utilizes correlation coefficient (CC), instead of Euclidean distance as classification criterion. The CC method not only bolsters the robustness of spike classification amidst the diverse and ever-changing conditions of physiological and recording noise environments, but also can finish the entire sorting procedure within a fixed number of cluster slots, thus preventing a large number of transient clusters. Moreover, the opt-OSort incorporates two configurable validation loops to efficiently reject cluster outliers and track recording variations caused by electrode drifting in real-time. Main results. The opt-OSort significantly reduces transient cluster occurrences by two orders of magnitude and decreases memory usage by 2.5–80 times in the number of pre-allocated transient clusters compared with other hardware implementations of OSort. The opt-OSort maintains an accuracy comparable to offline OSort and other commonly-used algorithms, with a sorting time of 0.68 µs as measured by the hardware-implemented system in both simulated datasets and experimental data. The opt-OSort’s ability to handle variations in neural activity caused by electrode drifting is also demonstrated. Significance. These results present a rapid, precise, and robust spike sorting solution suitable for integration into low-power, portable, closed-loop neural control systems and brain–computer interfaces.

This article presents a novel brain-machine interface system that uses electroencephalogram (EEG) and functional electric stimulation (FES) to support neurorehabilitation. Although repetitive facilitation exercise is a remarkable neurorehabilitation method, improving its success rate has proven difficult. This is because therapists are unable to detect patient motor commands and are thus, unable to stimulate the affected part of the body at the same time the motor commands are traveling along neuronal pathways. Here, we investigated the feasibility of both detecting the motor signal and, within milliseconds, stimulating the affected limb via a brain-machine interface system that combines the EEG and FES. The system extracts the timing of the motor commands from the EEG data using a support vector machine algorithm. Three healthy individuals participated in three experiments to: 1) decide the appropriate electric current value for stimulation; 2) determine the appropriate discrimination timing for the motor command detection algorithm; and 3) validate the effect using the system. The validation experiments were carried out under three experimental conditions-no system, fixed FES, and EEG-based FES. The results showed that for all three participants, wrist accelerations were significantly higher for EEG-based FES than for no system or fixed FES. We conclude that the EEG-based FES system detects the timing of motor commands and simultaneously stimulates a patient's affected limb in response.

Protocol for machine learning-based classification of neuronal cell types from spike waveforms