Efficient and reliable spike sorting from neural recordings with UMAP-based unsupervised nonlinear dimensionality reduction.

Identifying collective variables (CVs) for chemical reactions is essential to reduce the 3N-dimensional energy landscape into lower dimensional basins and barriers of interest. However, in condensed phase processes, the nonmeaningful motions of bulk solvent often overpower the ability of dimensionality reduction methods to identify correlated motions that underpin collective variables. Yet solvent can play important indirect or direct roles in reactivity, and much can be lost through treatments that remove or dampen solvent motion. This has been amply demonstrated within principal component analysis (PCA), although less is known about the behavior of nonlinear dimensionality reduction methods, e.g., uniform manifold approximation and projection (UMAP), that have become recently utilized. The latter presents an interesting alternative to linear methods though often at the expense of interpretability. This work presents distance-attenuated projection methods of atomic coordinates that facilitate the application of both PCA and UMAP to identify collective variables in the presence of explicit solvent and further the specific identity of solvent molecules that participate in chemical reactions. The performance of both methods is examined in detail for two reactions where the explicit solvent plays very different roles within the collective variables. When applied to raw molecular dynamics data in solution, both PCA and UMAP representations are dominated by bulk solvent motions. On the other hand, when applied to data preprocessed by our attenuated projection methods, both PCA and UMAP identify the appropriate collective variables (though varying sensitivity is observed due to the presence of explicit solvent that results from the projection method). Importantly, this approach allows identification of specific solvent molecules that are relevant to the CVs and their importance.

Dimension reduction has become an important tool for dealing with high dimensional data. Locally linear embedding (LLE) is a nonlinear dimension reduction method which can preserve local configurations of nearest neighbors. However, finding the nearest neighbors requires the definition of a distance measure, which is a critical step in LLE. In this paper, the Rank-order distance measure is used to substitute the traditional Euclidean distance measure in order to find better nearest neighbor candidates for preserving local configurations of the manifolds. The Rank-order distance between the data points is calculated using their neighbors’ ranking orders, and is shown to be able to improve the clustering of high dimensional data. The proposed method is called Rank-order based LLE (RLLE). The RLLE method is evaluated by comparing with the original LLE, ISO-LLE and IED-LLE on two handwritten datasets. It is shown that the effectiveness of a distance measure in the LLE method is closely related to whether it can be used to find good nearest neighbors. The experimental results show that the proposed RLLE method can improve the process of dimension reduction effectively, and C-index is another good candidate for evaluating the dimension reduction results.

Detection of maize seed germination rate based on improved locally linear embedding

Objective. Spike sorting is a set of techniques used to analyze extracellular neural recordings, attributing individual spikes to individual neurons. This field has gained significant interest in neuroscience due to advances in implantable microelectrode arrays, capable of recording thousands of neurons simultaneously. High-density electrodes, combined with efficient and accurate spike sorting systems, are essential for various applications, including brain machine interfaces (BMIs), experimental neural prosthetics, real-time neurological disorder monitoring, and neuroscience research. However, given the resource constraints of modern applications, relying solely on algorithmic innovation is not enough. Instead, a co-optimization approach that combines hardware and spike sorting algorithms must be taken to develop neural recording systems suitable for resource-constrained environments, such as wearable devices and BMIs. This co-design requires careful consideration when selecting appropriate spike-sorting algorithms that match specific hardware and use cases. Approach. We investigated the recent literature on spike sorting, both in terms of hardware advancements and algorithms innovations. Moreover, we dedicated special attention to identifying suitable algorithm-hardware combinations, and their respective real-world applicabilities. Main results. In this review, we first examined the current progress in algorithms, and described the recent departure from the conventional ‘3-step’ algorithms in favor of more advanced template matching or machine-learning-based techniques. Next, we explored innovative hardware options, including application-specific integrated circuits, field-programmable gate arrays, and in-memory computing devices (IMCs). Additionally, the challenges and future opportunities for spike sorting are discussed. Significance. This comprehensive review systematically summarizes the latest spike sorting techniques and demonstrates how they enable researchers to overcome traditional obstacles and unlock novel applications. Our goal is for this work to serve as a roadmap for future researchers seeking to identify the most appropriate spike sorting implementations for various experimental settings. By doing so, we aim to facilitate the advancement of this exciting field and promote the development of innovative solutions that drive progress in neural engineering research.

Spike sorting is a fundamental process for decoding neural activity, involving preprocessing, spike detection, feature extraction, clustering, and validation. However, conventional spike sorting methods are highly fragmented, labor-intensive, and heavily reliant on expert manual curation, limiting their scalability and reproducibility. This challenge has become more pressing with advances in neural recording technology, such as high-density Neuropixels for large-scale neural recording or flexible electrodes for long-term stable recording over months to years. The volume and complexity of these datasets make manual curation infeasible, requiring an automated and scalable solution. Here, we introduce SpikeAgent, a multimodal large language model (LLM)-based AI agent that automates and standardizes the entire spike sorting pipeline. Unlike traditional approaches, SpikeAgent integrates multiple LLM backends, coding functions, and established algorithms, autonomously performing spike sorting with reasoning-based decision-making and real-time interaction with intermediate results. It generates interpretable reports, providing transparent justifications for each sorting decision, enhancing transparency and reliability. We benchmarked SpikeAgent against human experts across various neural recording technology, demonstrating its versatility and ability to achieve curation consistency that are equal to, or even higher than human experts. It also drastically reduces the expertise barrier and accelerates the curation and validation time by orders of magnitude. Moreover, it enables automated interpretability of the neural spiking data, which cannot be achieved by any conventional methods. SpikeAgent presents a paradigm shift in processing signals for neuroscience and brain-computer interfaces, while laying the ground for AI agent-augmented science across various domains.

Locally linear embedding (LLE) is a method for nonlinear dimensionality reduction, which calculates a low dimensional embedding with the property that nearby points in the high dimensional space remain nearby and similarly co-located with respect to one another in the low dimensional space [1]. LLE algorithm needs to set up a free parameter, the number of nearest neighbors k . This parameter has a strong influence in the transformation. In this paper is proposed a cost function that quantifies the quality of the embedding results and computes an appropriate k . Quality measure is tested on artificial and real-world data sets, which allow us to visually confirm whether the embedding was correctly calculated.

Recent advances in the analysis of high-dimensional signal data have triggered an increasing interest in geometry-based methods for nonlinear dimensionality reduction (NDR). In many applications, high-dimensional datasets typically contain redundant information, and NDR methods are important for an efficient analysis of their properties. During the last few years, concepts from differential geometry were used to create a new range of NDR methods. In the construction of such geometry-based strategies, a natural question is to understand their interaction with classical and modern signal processing tools (convolution transforms, Fourier analysis, wavelet functions). In particular, an important task is the analysis of the incurred geometrical deformation when applying signal transforms to the elements of a dataset. In this paper, we propose the concepts of modulation maps and modulation manifolds for the construction of particular datasets relevant in signal processing and NDR. We consider numerical methods for analyzing geometrical properties of the modulation manifolds, with a particular focus on their scalar and mean curvature. In our numerical examples, we apply the resulting geometry-based analysis to simple test cases, where we present geometrical and topological effects of relevance in manifold learning.

We present a method for automated spike sorting for recordings with high-density, large-scale multielectrode arrays. Exploiting the dense sampling of single neurons by multiple electrodes, an efficient, low-dimensional representation of detected spikes consisting of estimated spatial spike locations and dominant spike shape features is exploited for fast and reliable clustering into single units. Millions of events can be sorted in minutes, and the method is parallelized and scales better than quadratically with the number of detected spikes. Performance is demonstrated using recordings with a 4,096-channel array and validated using anatomical imaging, optogeneticstimulation, and model-based quality control. A comparison with semi-automated, shape-based spike sorting exposes significant limitations of conventional methods. Our approach demonstrates that it is feasible to reliably isolate the activity of up to thousands of neurons and that dense, multi-channel probes substantially aid reliable spike sorting.

Variable selection based on locally linear embedding mapping for near-infrared spectral analysis

Divergent Projection Analysis for Unsupervised Dimensionality Reduction

One of the most effective ways to reduce the computational complexity of nonlinear dimensionality reduction is hierarchical partitioning of the space with the subsequent approximation of calculations. In this paper, the efficiency of two approaches to space partitioning, the partitioning of input and output spaces, is analyzed. In addition, a method for nonlinear dimensionality reduction is proposed. It is based on construction of a partitioning tree of the input multidimensional space and an iterative procedure of the gradient descent with the approximation carried out on the nodes of the constructed space partitioning tree. In the method proposed, the relative position of the corrected objects and partitioning tree nodes in both input and output spaces is taken into account in the approximation. The method developed was analyzed based on publicly available datasets.

We propose an unsupervised compressed sensing (CS)-based framework to compress, recover, and cluster neural action potentials. This framework can be easily integrated into high-density multi-electrode neural recording VLSI systems. Embedding spectral clustering and group structures in dictionary learning, we extend the proposed framework to unsupervised spike sorting without prior label information. Additionally, we incorporate group sparsity concepts in the dictionary learning to enable the framework for multi-channel neural recordings, as in tetrodes. To further improve spike sorting success rates in the CS framework, we embed template matching in sparse coding to jointly predict clusters of spikes. Our experimental results demonstrate that the proposed CS-based framework can achieve a high compression ratio (8:1 to 20:1), with a high quality reconstruction performance (>8 dB) and a high spike sorting accuracy (>90%).

Single-cell RNA-seq (scRNA-seq) is a powerful tool for analyzing heterogeneous and functionally diverse cell population. Visualizing scRNA-seq data can help us effectively extract meaningful biological information and identify novel cell subtypes. Currently, the most popular methods for scRNA-seq visualization are principal component analysis (PCA) and t-distributed stochastic neighbor embedding (t-SNE). While PCA is an unsupervised dimension reduction technique, t-SNE incorporates cluster information into pairwise probability, and then maximizes the Kullback–Leibler divergence. Uniform Manifold Approximation and Projection (UMAP) is another recently developed visualization method similar to t-SNE. However, one limitation with UMAP and t-SNE is that they can only capture the local structure of the data, the global structure of the data is not faithfully preserved. In this manuscript, we propose a semisupervised principal component analysis (ssPCA) approach for scRNA-seq visualization. The proposed approach incorporates cluster-labels into dimension reduction and discovers principal components that maximize both data variance and cluster dependence. ssPCA must have cluster-labels as its input. Therefore, it is most useful for visualizing clusters from a scRNA-seq clustering software. Our experiments with simulation and real scRNA-seq data demonstrate that ssPCA is able to preserve both local and global structures of the data, and uncover the transition and progressions in the data, if they exist. In addition, ssPCA is convex and has a global optimal solution. It is also robust and computationally efficient, making it viable for scRNA-seq cluster visualization.

High-grade gliomas are aggressive primary brain tumors that affect the surrounding brain tissue, which can lead to neurological deficits, epilepsy, cognitive deficits, and tumor progression. Research has focused on glutamatergic alterations, but the understanding of the underlying network mechanisms is still scarce. To investigate how gliomas might affect neural network integrity and capacity for spiking activity, we induced network bursting events in neocortical slices from F98 glioma-bearing rats. Network activity was recorded using a 64 × 64 high-density multielectrode array to assess the network's capacity for extended periods of high activity. We describe glioma cell implantation, brain slice preparation, and detailed data analysis of electrophysiological recordings, focusing on computational methods for characterization of the network-wide bursting events. Spike sorting was performed using Kilosort4 and employed to identify network bursts. Burst origins were estimated from multi-unit activity, and network burst propagation speed was assessed. Network-level functional connectivity was analyzed using metrics derived from graphs based on the spike time tiling coefficient. Our results indicate reduced network integrity in glioma-bearing slices. The combination of a high-density multielectrode array with Kilosort4 is advantageous over conventional recording techniques in terms of detecting network bursting and plotting center-of-activity trajectories. This protocol provides a robust platform for dissecting network activity and establishes a basis for future mechanistic studies. We also discuss the limitations and experimental challenges of our preclinical approach to investigating glioma-associated effects on surrounding neuronal networks.

Dimensionality reduction is an unsupervised learning task aimed at creating a low-dimensional summary and/or extracting the most salient features of a dataset. Principal component analysis is a linear dimensionality reduction method in the sense that each principal component is a linear combination of the input variables. To allow features that are nonlinear functions of the input variables, many nonlinear dimensionality reduction (NLDR) methods have been proposed. In this article, we propose novel NLDR methods based on bottleneck deep autoencoders. Our contributions are 2-fold: (1) We introduce a monotonicity constraint into bottleneck deep autoencoders for estimating a single nonlinear component and propose two methods for fitting the model. (2) We propose a new, forward stepwise deep learning architecture for estimating multiple nonlinear components. The former helps extract interpretable, monotone components when the assumption of monotonicity holds, and the latter helps evaluate reconstruction errors in the original data space for a range of components. We conduct numerical studies to compare different model fitting methods and use two real data examples from the studies of human immune responses to HIV to illustrate the proposed methods. Supplementary materials for this article are available online.