Strategies for High-Performance Resource-Efficient Compression of Neural Spike Recordings

Objective. Spike sorting is a set of techniques used to analyze extracellular neural recordings, attributing individual spikes to individual neurons. This field has gained significant interest in neuroscience due to advances in implantable microelectrode arrays, capable of recording thousands of neurons simultaneously. High-density electrodes, combined with efficient and accurate spike sorting systems, are essential for various applications, including brain machine interfaces (BMIs), experimental neural prosthetics, real-time neurological disorder monitoring, and neuroscience research. However, given the resource constraints of modern applications, relying solely on algorithmic innovation is not enough. Instead, a co-optimization approach that combines hardware and spike sorting algorithms must be taken to develop neural recording systems suitable for resource-constrained environments, such as wearable devices and BMIs. This co-design requires careful consideration when selecting appropriate spike-sorting algorithms that match specific hardware and use cases. Approach. We investigated the recent literature on spike sorting, both in terms of hardware advancements and algorithms innovations. Moreover, we dedicated special attention to identifying suitable algorithm-hardware combinations, and their respective real-world applicabilities. Main results. In this review, we first examined the current progress in algorithms, and described the recent departure from the conventional ‘3-step’ algorithms in favor of more advanced template matching or machine-learning-based techniques. Next, we explored innovative hardware options, including application-specific integrated circuits, field-programmable gate arrays, and in-memory computing devices (IMCs). Additionally, the challenges and future opportunities for spike sorting are discussed. Significance. This comprehensive review systematically summarizes the latest spike sorting techniques and demonstrates how they enable researchers to overcome traditional obstacles and unlock novel applications. Our goal is for this work to serve as a roadmap for future researchers seeking to identify the most appropriate spike sorting implementations for various experimental settings. By doing so, we aim to facilitate the advancement of this exciting field and promote the development of innovative solutions that drive progress in neural engineering research.

BackgroundPrincipal component analysis (PCA) has been widely employed for automatic neuronal spike sorting. Calculating principal components (PCs) is computationally expensive, and requires complex numerical operations and large memory resources. Substantial hardware resources are therefore needed for hardware implementations of PCA. General Hebbian algorithm (GHA) has been proposed for calculating PCs of neuronal spikes in our previous work, which eliminates the needs of computationally expensive covariance analysis and eigenvalue decomposition in conventional PCA algorithms. However, large memory resources are still inherently required for storing a large volume of aligned spikes for training PCs. The large size memory will consume large hardware resources and contribute significant power dissipation, which make GHA difficult to be implemented in portable or implantable multi-channel recording micro-systems.MethodIn this paper, we present a new algorithm for PCA-based spike sorting based on GHA, namely stream-based Hebbian eigenfilter, which eliminates the inherent memory requirements of GHA while keeping the accuracy of spike sorting by utilizing the pseudo-stationarity of neuronal spikes. Because of the reduction of large hardware storage requirements, the proposed algorithm can lead to ultra-low hardware resources and power consumption of hardware implementations, which is critical for the future multi-channel micro-systems. Both clinical and synthetic neural recording data sets were employed for evaluating the accuracy of the stream-based Hebbian eigenfilter. The performance of spike sorting using stream-based eigenfilter and the computational complexity of the eigenfilter were rigorously evaluated and compared with conventional PCA algorithms. Field programmable logic arrays (FPGAs) were employed to implement the proposed algorithm, evaluate the hardware implementations and demonstrate the reduction in both power consumption and hardware memories achieved by the streaming computingResults and discussionResults demonstrate that the stream-based eigenfilter can achieve the same accuracy and is 10 times more computationally efficient when compared with conventional PCA algorithms. Hardware evaluations show that 90.3% logic resources, 95.1% power consumption and 86.8% computing latency can be reduced by the stream-based eigenfilter when compared with PCA hardware. By utilizing the streaming method, 92% memory resources and 67% power consumption can be saved when compared with the direct implementation of GHA.ConclusionStream-based Hebbian eigenfilter presents a novel approach to enable real-time spike sorting with reduced computational complexity and hardware costs. This new design can be further utilized for multi-channel neuro-physiological experiments or chronic implants.

Brain Machine Interface (BMI) systems demand real-time spike sorting to instantaneously decode the spike trains of simultaneously recorded cortical neurons. Real-time spike sorting, however, requires extensive computational power that is not feasible to implement in implantable BMI architectures, thereby requiring transmission of high-bandwidth raw neural data to an external computer. In this work, we describe a miniaturized, low power, programmable hardware module capable of performing this task within the resource constraints of an implantable chip. The module computes a sparse representation of the spike waveforms followed by "smart" thresholding. This cascade restricts the sparse representation to a subset of projections that preserve the discriminative features of neuron-specific spike waveforms. In addition, it further reduces telemetry bandwidth making it feasible to wirelessly transmit only the important biological information to the outside world, thereby improving the efficiency, practicality and viability of BMI systems in clinical applications.

The scale of in vivo electrophysiology has expanded in recent years, with simultaneous recordings across thousands of electrodes now becoming routine. These advances have enabled a wide range of discoveries, but they also impose substantial computational demands. Spike sorting, the procedure that extracts spikes from extracellular voltage measurements, remains a major bottleneck: a dataset collected in a few hours can take days to spike sort on a single machine, and the field lacks rigorous validation of the many spike sorting algorithms and preprocessing steps that are in use. Advancing the speed and accuracy of spike sorting is essential to fully realize the potential of large-scale electrophysiology. Here, we present an end-to-end spike sorting pipeline that leverages parallelization to scale to large datasets. The same workflow can run reproducibly on individual workstations, high-performance computing clusters, or cloud environments, with computing resources tailored to each processing step to reduce costs and execution times. In addition, we introduce a benchmarking pipeline, also optimized for parallel processing, that enables systematic comparison of multiple sorting pipelines. Using this framework, we show that Kilosort4, a widely used spike sorting algorithm, outperforms Kilosort2.5 (). We also show that 7× lossy compression, which substantially reduces the cost of data storage, has minimal impact on spike sorting performance. Together, these pipelines address the urgent need for scalable and transparent spike sorting of electrophysiology data, preparing the field for the coming flood of multi-thousand-channel experiments.

Spike sorting, i.e., the separation of the firing activity of different neurons from extracellular measurements, is a crucial but often error-prone step in the analysis of neuronal responses. Usually, three different problems have to be solved: the detection of spikes in the extracellular recordings, the estimation of the number of neurons and their prototypical (template) spike waveforms, and the assignment of individual spikes to those putative neurons. If the template spike waveforms are known, template matching can be used to solve the detection and classification problem. Here, we show that for the colored Gaussian noise case the optimal template matching is given by a form of linear filtering, which can be derived via linear discriminant analysis. This provides a Bayesian interpretation for the well-known matched filter output. Moreover, with this approach it is possible to compute a spike detection threshold analytically. The method can be implemented by a linear filter bank derived from the templates, and can be used for online spike sorting of multielectrode recordings. It may also be applicable to detection and classification problems of transient signals in general. Its application significantly decreases the error rate on two publicly available spike-sorting benchmark data sets in comparison to state-of-the-art template matching procedures. Finally, we explore the possibility to resolve overlapping spikes using the template matching outputs and show that they can be resolved with high accuracy.

Unsupervised spike sorting, a vital processing step in real-time brain-implantable microsystems, is faced with the prominent challenge of managing nonstationarity in neural signals. In long-term recordings, spike waveforms gradually change and new source neurons are likely to become activated. Adaptive spike sorters combined with on-implant training units effectively process the nonstationary signals at the cost of high hardware resource utilization. On the other hand, static approaches, while being hardware-friendly, are subjected to decreased processing performance in such recordings where the neural signal characteristics gradually change. To strike a balance between the hardware cost and processing performance, this study proposes a hardware-efficient novelty-aware spike sorting approach that is capable of dealing with both variated spike waveforms and spike waveforms generated from new source neurons. Its improved hardware efficiency compared to adaptive ones and capability of dealing with nonstationary signals make it attractive for implantable applications. The proposed novelty-aware spike sorting especially would be a good fit for brain-computer interfaces where long-term, real-time interaction with the brain is required, and the available on-implant hardware resources are limited. Our unsupervised spike sorting benefits from a novelty detection process to deal with neural signal variations. It tracks the spike features so that in case of detecting an unexpected change (novelty detection) both on and off-implant parameters are updated to preserve the performance in new state. To make the proposed approach agile enough to be suitable for brain implants, the on-implant computations are reduced while the computational burden is realized off-implant. The performance of our proposed approach is evaluated using both synthetic and real datasets. The results demonstrate that, in the mean, it is capable of detecting 94.31% of novel spikes (wave-drifted or emerged spikes) with a classification accuracy (CA) of 96.31%. Moreover, an FPGA prototype of the on-implant circuit is implemented and tested. It is shown that in comparison to the OSORT algorithm, a pivotal spike sorting method, our spike sorting provides a higher CA at significantly lower hardware resources. The proposed circuit is also implemented in a 180-nm standard CMOS process, achieving a power consumption of 1.78[Formula: see text][Formula: see text] per channel and a chip area of 0.07[Formula: see text]mm2 per channel.

Physical principles of brain–computer interfaces and their applications for rehabilitation, robotics and control of human brain states

Neural decoding and its applications to brain computer interfaces (BCI) are essential for understanding the association between neural activity and behavior. A prerequisite for many decoding approaches is spike sorting, the assignment of action potentials (spikes) to individual neurons. Current spike sorting algorithms, however, can be inaccurate and do not properly model uncertainty of spike assignments, therefore discarding information that could potentially improve decoding performance. Recent advances in high-density probes (e.g., Neuropixels) and computational methods now allow for extracting a rich set of spike features from unsorted data; these features can in turn be used to directly decode behavioral correlates. To this end, we propose a spike sorting-free decoding method that directly models the distribution of extracted spike features using a mixture of Gaussians (MoG) encoding the uncertainty of spike assignments, without aiming to solve the spike clustering problem explicitly. We allow the mixing proportion of the MoG to change over time in response to the behavior and develop variational inference methods to fit the resulting model and to perform decoding. We benchmark our method with an extensive suite of recordings from different animals and probe geometries, demonstrating that our proposed decoder can consistently outperform current methods based on thresholding (i.e. multi-unit activity) and spike sorting. Open source code is available at https://github.com/yzhang511/density_decoding.

Recordings made directly from the nervous system are a key tool in experimental electrophysiology and the development of bioelectronic medicines. Analysis of these recordings involves the identification of signals from individual neurons, a process known as spike sorting. A critical and limiting feature of spike sorting is the need to align individual spikes in time. However, electrophysiological recordings are made in extremely noisy environments that seriously limit the performance of the spike-alignment process. We present a new centroid-based method and demonstrate its effectiveness using deterministic models of nerve signals. We show that spike alignment in the presence of noise is possible with a 30 dB reduction in minimum SNR compared with the conventional methods. We present a mathematical analysis of the centroid method, characterizing its fundamental operation and performance. Furthermore, we show that the centroid method lends itself particularly well to hardware realization, and we present results from a low-power implementation that operates on an FPGA, consuming ten times less power than conventional techniques - an important property for implanted devices. Our centroid method enables the accurate alignment of spikes in sub-0 dB SNR recordings and has the potential to enable the analysis of spikes in a wider range of environments than has been previously possible. Our method thus has the potential to influence significantly the design of electrophysiological recording systems in the future.

A brain-computer interface (BCI) can be defined as any system that can track the person's intent which is embedded in his/her brain activity and, from it alone, translate the intention into commands of a computer. Among the brain signal monitoring systems best suited for this challenging task, electroencephalography (EEG) and magnetoencephalography (MEG) are the most realistic, since both are non-invasive, EEG is portable and MEG could provide more specific information that could be later exploited also through EEG signals. The first two BCI steps require set up of the appropriate experimental protocol while recording the brain signal and then to extract interesting features from the recorded cerebral activity. To provide information useful in these BCI stages, our aim is to provide an overview of a new procedure we recently developed, named functional source separation (FSS). As it comes from the blind source separation algorithms, it exploits the most valuable information provided by the electrophysiological techniques, i.e. the waveform signal properties, remaining blind to the biophysical nature of the signal sources. FSS returns the single trial source activity, estimates the time course of a neuronal pool along different experimental states on the basis of a specific functional requirement in a specific time period, and uses the simulated annealing as the optimization procedure allowing the exploit of functional constraints non-differentiable. Moreover, a minor section is included, devoted to information acquired by MEG in stroke patients, to guide BCI applications aiming at sustaining motor behaviour in these patients. Relevant BCI features - spatial and time-frequency properties - are in fact altered by a stroke in the regions devoted to hand control. Moreover, a method to investigate the relationship between sensory and motor hand cortical network activities is described, providing information useful to develop BCI feedback control systems. This review provides a description of the FSS technique, a promising tool for the BCI community for online electrophysiological feature extraction, and offers interesting information to develop BCI applications to sustain hand control in stroke patients.

Spike sorting is a technique in the field of neuroscience applied to identify and classify the action potentials (spikes) of neurons in extracellular recordings. This method is important considering that the simultaneous activity of several neurons can be recorded through the use of electrodes, but the recorded signals are often mixed. Extracting the characteristics of individual spikes provides valuable information about the identity of neurons that generate them, the type of cells involved (inhibitory/excitatory), encoding of information in spike rates or spike times, and so on. Therefore, unmixing the spikes recorded on an electrode via spike sorting is crucial. Here, we apply a novel method to extract features of mixed spikes, namely the Superlet transform, which enables the computation of spectral characteristics with a higher resolution. We use machine learning to determine which features of the Superlet spectrum contain the most information about the shapes of individual spikes, thereby enabling their unmixing during spike sorting.

Extracellular spikes recorded in the visual cortex (Area 17/18, V1) are commonly classified into either regular-spiking (RS) or fast-spiking (FS). Using multi-electrode arrays positioned in cat V1 and a broadband stimulus, we show that there is also a distinct class with positive-spiking (PS) waveforms. PS units were associated mainly with non-oriented receptive fields while RS and FS units had orientation-selective receptive fields. We suggest that PS units are recordings of axons originating from the thalamus. This conclusion was reinforced by our finding that we could record PS units after cortical silencing, but not record RS and FS units. The importance of our findings is that we were able to correlate spike shapes with receptive field characteristics with high precision using multi-electrode extracellular recording techniques. This allows considerable increases in the amount of information that can be extracted from future cortical experiments. Extracellular spike waveforms from recordings in the visual cortex have been classified into either regular-spiking (RS) or fast-spiking (FS) units. While both these types of spike waveforms are negative-dominant, we show that there are also distinct classes of spike waveforms in visual Area 17/18 (V1) of anaesthetised cats with positive-dominant waveforms, which are not regularly reported. The spatial receptive fields (RFs) of these different spike waveform types were estimated, which objectively revealed the existence of oriented and non-oriented RFs. We found that units with positive-dominant spikes, which have been associated with recordings from axons in the literature, had mostly non-oriented RFs (84%), which are similar to the centre-surround RFs observed in the dorsal lateral geniculate nucleus (dLGN). Thus, we hypothesise that these positive-dominant waveforms may be recordings from dLGN afferents. We recorded from V1 before and after the application of muscimol (a cortical silencer) and found that the positive-dominant spikes (PS) remained while the RS and FS cells did not. We also noted that the PS units had spiking characteristics normally associated with dLGN units (i.e. higher response spike rates, lower response latencies and higher proportion of burst spikes). Our findings show quantitatively that it is possible to correlate the RF properties of cortical neurons with particular spike waveforms. This has implications for how extracellular recordings should be interpreted and complex experiments can now be contemplated that would have been very challenging previously, such as assessing the feedforward connectivity between brain areas in the same location of cortical tissue.

Accurate spike sorting is an important issue for neuroscientific and neuroprosthetic applications. The sorting of spikes depends on the features extracted from the neural waveforms, and a better sorting performance usually comes with a higher sampling rate (SR). However for the long duration experiments on free-moving subjects, the miniaturized and wireless neural recording ICs are the current trend, and the compromise on sorting accuracy is usually made by a lower SR for the lower power consumption. In this paper, we implement an on-chip spike sorting processor with integrated interpolation hardware in order to improve the performance in terms of power versus accuracy. According to the fabrication results in 90nm process, if the interpolation is appropriately performed during the spike sorting, the system operated at the SR of 12.5 k samples per second (sps) can outperform the one not having interpolation at 25 ksps on both accuracy and power.

Spike sorting is one of the cornerstones of extracellular electrophysiology. By leveraging advanced signal processing and data analysis techniques, spike sorting makes it possible to detect, isolate, and map single neuron spiking activity from both in vivo and in vitro extracellular electrophysiological recordings. A crucial step of any spike sorting pipeline is to reduce the dimensionality of the recorded spike waveform data. Reducing the dimensionality of the processed data is a near-universal practice, fundamentally motivated by the use of clustering algorithms responsible to detect, isolate, and sort the recorded putative neurons. In this paper, we propose and illustrate on both synthetic and experimental data that employing the nonlinear dimensionality reduction technique Uniform Manifold Approximation and Projection (UMAP) can drastically improve the performance, efficiency, robustness, and scalability of spike sorting pipelines without increasing their computational cost. We show how replacing the linear or ad hoc, expert-defined, supervised nonlinear dimensionality reduction methods commonly used in spike sorting pipelines by the unsupervised, mathematically grounded, nonlinear dimensionality reduction method provided by UMAP drastically increases the number of correctly sorted neurons, makes the identification of quieter, seldom spiking neurons more reliable, enables deeper and more precise explorations and analysis of the neural code, and paves new ways toward more efficient and end-to-end automatable spike sorting pipelines of large-scale extracellular neural recording as those produced by high-density multielectrode arrays.

Spike sorting of neural data from single electrode recordings is a hard problem in machine learning that relies on significant input by human experts. We approach the task of learning to detect and classify spike waveforms in additive noise using two stages of large margin kernel classification and probability regression. Controlled numerical experiments using spike and noise data extracted from neural recordings indicate significant improvements in detection and classification accuracy over linear amplitude- and template-based spike sorting techniques.