Comparing representational geometries using whitened unbiased-distance-matrix similarity

Representational similarity analysis (RSA) has been shown to be an effective framework to characterize brain-activity profiles and deep neural network activations as representational geometry by computing the pairwise distances of the response patterns as a representational dissimilarity matrix (RDM). However, how to properly analyze and visualize the representational geometry as dynamics over the time course from stimulus onset to offset is not well understood. In this work, we formulated the pipeline to understand representational dynamics with RDM movies and Procrustes-aligned Multidimensional Scaling (pMDS), and applied it to neural recording of monkey IT cortex. Our results suggest that the the multidimensional scaling alignment can genuinely capture the dynamics of the category-specific representation spaces with multiple visualization possibilities, and that object categorization may be hierarchical, multi-staged, and oscillatory (or recurrent).

Representational similarity analysis (RSA) is a key element in the multivariate pattern analysis toolkit. The central construct of the method is the representational dissimilarity matrix (RDM), which can be generated for datasets from different modalities (neuroimaging, behavior, and computational models) and directly correlated in order to evaluate their second-order similarity. Given the inherent noisiness of neuroimaging signals it is important to evaluate the reliability of neuroimaging RDMs in order to determine whether these comparisons are meaningful. Recently, multivariate noise normalization (NNM) has been proposed as a widely applicable method for boosting signal estimates for RSA, regardless of choice of dissimilarity metrics, based on evidence that the analysis improves the within-subject reliability of RDMs (Guggenmos et al. 2018; Walther et al. 2016). We revisited this issue with three fMRI datasets and evaluated the impact of NNM on within- and between-subject reliability and RSA effect sizes using multiple dissimilarity metrics. We also assessed its impact across regions of interest from the same dataset, its interaction with spatial smoothing, and compared it to GLMdenoise, which has also been proposed as a method that improves signal estimates for RSA (Charest et al. 2018). We found that across these tests the impact of NNM was highly variable, as also seems to be the case for other analysis choices. Overall, we suggest being conservative before adding steps and complexities to the (pre)processing pipeline for RSA.

One hundred and sixty-eight genotypes of cotton from the same growing region were used as a germplasm group to study the validity of different genetic distances in constructing cotton core subset. Mixed linear model approach was employed to unbiasedly predict genotypic values of 20 traits for eliminating the environmental effect. Six commonly used genetic distances (Euclidean, standardized Euclidean, Mahalanobis, city block, cosine and correlation distances) combining four commonly used hierarchical cluster methods (single distance, complete distance, unweighted pair-group average and Ward's methods) were used in the least distance stepwise sampling (LDSS) method for constructing different core subsets. The analyses of variance (ANOVA) of different evaluating parameters showed that the validities of cosine and correlation distances were inferior to those of Euclidean, standardized Euclidean, Mahalanobis and city block distances. Standardized Euclidean distance was slightly more effective than Euclidean, Mahalanobis and city block distances. The principal analysis validated standardized Euclidean distance in the course of constructing practical core subsets. The covariance matrix of accessions might be ill-conditioned when Mahalanobis distance was used to calculate genetic distance at low sampling percentages, which led to bias in small-sized core subset construction. The standardized Euclidean distance is recommended in core subset construction with LDSS method.

Perceptual and cognitive content is thought to be represented in the brain by patterns of activity across populations of neurons. In order to test whether a computational model can explain a given population code and whether corresponding codes in man and monkey convey the same information, we need to quantitatively relate population-code representations. Here I give a brief introduction to representational similarity analysis, a particular approach to this problem. A population code is characterized by a representational dissimilarity matrix (RDM), which contains a dissimilarity for each pair of activity patterns elicited by a given stimulus set. The RDM encapsulates which distinctions the representation emphasizes and which it deemphasizes. By analyzing correlations between RDMs we can test models and compare different species. Moreover, we can study how representations are transformed across stages of processing and how they relate to behavioral measures of object similarity. We use an example from object vision to illustrate the method's potential to bridge major divides that have hampered progress in systems neuroscience.

The principle component analysis(PCA) is a common feature extraction method in machine learning and pattern recognition approaches. PCA has been used in many applications, and face recognition in which specific faces are recognizing in an images database is one of the popular applications. The default distance metric which has been used with PCA based-face recognition is Euclidean distance. In this study, we have tested the Mahalanobis distance instead of Euclidean, and PCA based on Mahalanobis distance suggested a better performance on our students images database with highest recognition rate. However, we proposed weighted and normalized Mahalanobis distance based PCA-face recognition(PCA_WNMD). The proposed algorithm (PCA_WNMD) showed an improvement in faces recognition rate when tested on our students images database compared to PCA based on Mahalanobis and default Euclidean distances.

The question of how the human brain represents information, and if insights from this knowledge help us in formulating better deep learning algorithms, is still open. This paper attempts to address these questions by investigating the brain responses when participants viewed images of objects belonging to a variety of categories. We adopt the representational similarity analysis (RSA) framework wherein the brain responses are projected into a representational dissimilarity (RD) space so that responses from multiple subjects can be assessed on a common basis. We performed RSA of the brain responses of 15 participants on a set of 92 images (from the Algonauts challenge data set) Cichy et al. (The Algonauts project: a platform for communication between the sciences of biological and artificial intelligence 2019 [4]). The results reveal that human brain responses exhibit appropriate qualitative similarities observed in the image space. We then evaluated various deep networks such as VGGnet, AlexNet, and Inception in Siamese configuration with RD-loss function and identified hidden layers that had maximal similarity to human RD matrices (RDM). The best layers then were subjected to further analysis by looking at the image pairs corresponding to high and low similarity values both in the early and late visual areas. Our results point out that while deep neural network (DNN) responses mimicking early visual areas seem to indicate both feature- and category-based similarities, those mimicking the late visual areas are mostly categorical and contextual. It seems that while deep networks are capable of mimicking the similarity distributions observed in human brain responses, it is still not clear what the networks are actually learning while trying to minimize the representational dissimilarity loss function. We suggest the use of alternative dissimilarity measures to possibly redress this problem. We also discuss the possibility of adopting a dissimilarity metric for training future deep networks.

Hellwig’s method is a multi-criteria decision-making technique designed to facilitate the ranking of alternatives based on their proximity to the ideal solution. Typically, this approach calculates distances using the Euclidean norm, assuming implicitly that the considered criteria are independent. However, in real-world situations, the assumption of criteria independence is rarely met. The paper aims to propose an extension of Hellwig’s method by incorporating the Mahalanobis distance. Substituting the Euclidean distance with the Mahalanobis distance has proven to be effective in handling correlations among criteria, especially in the context of asymmetrical relationships between criteria. Subsequently, we investigate the impact of the Euclidean and Mahalanobis distance measures on the several variants of Hellwig procedures, analyzing examples based on various illustrative data with 10 alternatives and 4 criteria. Additionally, we examine the influence of three normalization formulas in Hellwig’s aggregation procedures. The investigation results indicate that both the distance measure and normalization formulas have some impact on the final rankings. The evaluation and ranking of alternatives using the Euclidean distance measure are influenced by the normalization formula, albeit to a limited extent. In contrast, the Mahalanobis distance-based Hellwig’s method remains unaffected by the choice of normalization formulas. The study concludes that the ranking of alternatives is strongly dependent on the distance measure employed, whether it is Euclidean or Mahalanobis. The Mahalanobis distance-based Hellwig method is deemed a valuable tool for decision-makers in real-life situations. It enables the evaluation of alternatives by considering interactions between criteria, providing a more comprehensive perspective for decision-making.

The distance-based regression model has many applications in analysis of multivariate response regression in various fields, such as ecology, genomics, genetics, human microbiomics, and neuroimaging. It yields a pseudo F test statistic that assesses the relation between the distance (dissimilarity) of the subjects and the predictors of interest. Despite its popularity in recent decades, the statistical properties of the pseudo F test statistic have not been revealed to our knowledge. This study derives the asymptotic properties of the pseudo F test statistic using spectral decomposition under the matrix normal assumption, when the utilized dissimilarity measure is the Euclidean or Mahalanobis distance. The pseudo F test statistic with the Euclidean distance has the same distribution as the quotient of two Chi-squared-type mixtures. The denominator and numerator of the quotient are approximated using a random variable of the form \(\xi\chi_d^2+\eta\) and the approximate error bound is given. The pseudo F test statistic with the Mahalanobis distance follows an F distribution. In simulation studies, the approximated distribution well matched the “exact” distribution obtained by the permutation procedure. The obtained distribution was further validated on H1N1 influenza data, aging human brain data, and embryonic imprint data.

Representational Similarity Analysis (RSA) has emerged as a popular method for relating representational spaces from human brain activity, behavioral data, and computational models. RSA is based on the comparison of representational (dis-)similarity matrices (RDMs or RSMs), which characterize the pairwise (dis-)similarities of all conditions across all features (e.g. fMRI voxels or units of a model). However, classical RSA treats each feature as equally important. This ‘equal weights’ assumption contrasts with the flexibility of multivariate decoding, which reweights individual features for predicting a target variable. As a consequence, classical RSA may lead researchers to underestimate the correspondence between a model and a brain region and, in case of model comparison, may lead them to select an inferior model. The aim of this work is twofold: First, we sought to broadly test feature-reweighted RSA (FR-RSA) applied to computational models and reveal the extent to which reweighting model features improves RSM correspondence and affects model selection. Previous work suggested that reweighting can improve model selection in RSA but it has remained unclear to what extent these results generalize across datasets and data modalities. To draw more general conclusions, we utilized a range of publicly available datasets and three popular deep neural networks (DNNs). Second, we propose voxel-reweighted RSA, a novel use case of FR-RSA that reweights fMRI voxels, mirroring the rationale of multivariate decoding of optimally combining voxel activity patterns. We found that reweighting individual model units markedly improved the fit between model RSMs and target RSMs derived from several fMRI and behavioral datasets and affected model selection, highlighting the importance of considering FR-RSA. For voxel-reweighted RSA, improvements in RSM correspondence were even more pronounced, demonstrating the utility of this novel approach. We additionally show that classical noise ceilings can be exceeded when FR-RSA is applied and propose an updated approach for their computation. Taken together, our results broadly validate the use of FR-RSA for improving the fit between computational models, brain, and behavioral data, allowing us to better adjudicate between competing computational models. Further, our results suggest that FR-RSA applied to brain measurement channels could become an important new method to assess the correspondence between representational spaces.

Rooted Mahalanobis distance based Gustafson-Kessel fuzzy C-means

Despite the tight coupling between sensory and motor processing for fine manipulation in humans, it is not yet totally clear which specific properties of the fingers are mapped in the precentral and postcentral gyrus. We used fMRI to compare the morphology, connectivity, and encoding of the motor and tactile finger representations (FRs) in the precentral and postcentral gyrus of 25 5-fingered participants (8 females). Multivoxel pattern and structural and functional connectivity analyses demonstrated the existence of distinct motor and tactile FRs within both the precentral and postcentral gyrus, integrating finger-specific motor and tactile information. Using representational similarity analysis, we found that the motor and tactile FRs in the sensorimotor cortex were described by the perceived structure of the hand better than by the actual hand anatomy or other functional models (finger kinematics, muscles synergies). We then studied a polydactyly individual (i.e., with a congenital 6-fingered hand) showing superior manipulation abilities and divergent anatomic-functional hand properties. The perceived hand model was still the best model for tactile representations in the precentral and postcentral gyrus, while finger kinematics better described motor representations in the precentral gyrus. We suggest that, under normal conditions (i.e., in subjects with a standard hand anatomy), the sensorimotor representations of the 5 fingers in humans converge toward a model of perceived hand anatomy, deviating from the real hand structure, as the best synthesis between functional and structural features of the hand.SIGNIFICANCE STATEMENT Distinct motor and tactile finger representations exist in both the precentral and postcentral gyrus, supported by a finger-specific pattern of anatomic and functional connectivity across modalities. At the representational level, finger representations reflect the perceived structure of the hand, which might result from an adapting process harmonizing (i.e., uniformizing) the encoding of hand function and structure in the precentral and postcentral gyrus. The same analyses performed in an extremely rare polydactyly subject showed that the emergence of such representational geometry is also found in neuromechanical variants with different hand anatomy and function. However, the harmonization process across the precentral and postcentral gyrus might not be possible because of divergent functional-structural properties of the hand and associated superior manipulation abilities.

SMOTE (synthetic minority over-sampling technique) has been used the most as a solution to the problem of imbalanced data.SMOTE selects the nearest neighbor based on Euclidean distance.However, Euclidean distance has the disadvantage of not considering the correlation between variables.In particular, the Mahalanobis distance has the advantage of considering the covariance of variables.But if there are outliers, they usually influence calculating the Mahalanobis distance.To solve this problem, we use the Mahalanobis distance by estimating the covariance matrix using MCD (minimum covariance determinant).Then apply Mahalanobis distance based on MCD to SMOTE to create new data.Therefore, we showed that in most cases this method provided high performance indicators for classifying imbalanced data.

Fuzzy C-means (FCM) is a kind of classic cluster method, which has been widely used in various fields, such as image segmentation and data mining. Euclidean distance is a frequently used distance metric in FCM, but it is only suitable for data with spherical structure. As an extension of Euclidean distance, Mahalanobis distance has been used in Gustafson–Kessel FCM and its variants to tackle ellipsoidal data. For the convenience of optimizing, most existing Mahalanobis distance-based FCM algorithms only focus on squared Mahalanobis distance. However, squared Mahalanobis distance may not be the best distance metric for FCM because it is easy to enlarge the influence of outliers. In this article, we propose a novel <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\ell _{2,p}$</tex-math></inline-formula> -norm and Mahalanobis distance-based FCM model, abbreviated as LM-FCM, which can help FCM improve the ability of tackling ellipsoidal clusters and outliers. Then, in order to reduce computational complexity, we propose a more simplified model, abbreviated as SLM-FCM. Furthermore, we develop an iteratively reweighted optimization algorithm to optimize the proposed models and provide a rigorous monotonous convergence proof. Finally, compared with the existing state-of-the-art FCM algorithms, we conduct extensive experiments on both synthetic and real-world datasets to manifest the superior clustering performance and robustness of the proposed algorithms.

Spherical SOM, an improved version of a kind of neutral network SOM, has successfully been applied to data analysis in a variety of fields achieving effective results. However, distance measure of commercial spherical SOM is limited to the Euclidean distance and it is not suitable enough to the analysis of biased data such as blood test results. The Mahalanobis distance is said to be effective for the analysis of such medical data. Therefore it is expected that better results should be obtained if spherical SOM with Mahalanobis distance is applied to the analysis of medical data. In this paper, we take blood test items as multi-dimensional vectors and convert the input data into Mahalanobis distance with the aim of developing an automated diagnosis system by spherical SOM with Mahalanobis distance as pseudo input data. Conversion of the input data into Mahalanobis distance ensures correct evaluations of the biased data of blood test results at the same time allowing automated diagnosis based on doctors’ intuitions and experiences. We have grouped subjects of diagnosis whose features were not well discriminated by conventional Mahalanobis distance and have administered detailed discrimination by the group and obtained better discrimination rates. While in the conventional studies TP rates for the following five categories, no liverrelated problem, hepatoma (liver cancer), acute hepatitis, chronic hepatitis and liver cirrhosis, were 100%, 70%, 100%, 80% and 60% respectively, they were 96%, 80%, 71%, 86% and 91% respectively with the proposed method. Significant results were obtained overall except for acute hepatitis.

Introduction Large language models (LLMs) have the potential to realize accurate risk stratification and disease prediction by integrating multimodal data, such as electronic health records, medical images, and genomic profiles. However, in complex tasks like atherosclerosis risk estimation, the design of the prompt is critical to bring out the performance of LLMs. Several researches have demonstrated that in-context learning, where the prompt contains some cases, improves the performance of LLMs when the contained cases are carefully selected. Because manual selection usually requires enormous costs, automatic case selection has been desired. Methods This study aims to improve the effectiveness of in-context learning in LLM for atherosclerosis risk prediction by introducing a strategy for case selection. Our method involves two key components: feature selection and case selection. In the feature selection phase, we compute the mutual information between the clinical features and atherosclerosis risk using the database to extract the essential clinical features. This feature selection removes the unimportant features that disturb the performance of the case selection mechanism. Then, in the case selection phase, we select several cases from the database that are the most similar to the patient being diagnosed. This study compares three metrics to compute the similarity score: - Mahalanobis distance (trained via large margin nearest neighbor classification; LMNN [1]) - Cosine similarity (between the vector representations in KNN-augmented in-context example selection; KATE [2]) - Euclidean distance (between the prompt features) This combined strategy supplies the LLM with informative cases and thereby enhances the diagnostic performance. Results We used a fine-tuned Llama3-8B model for our target language and analyzed a dataset of 117,709 cases. Each case contained 96 features that were collected from annual health check-up data. From this dataset, 1,000 cases each were randomly selected for validation and test datasets. The remaining data were used as the database in our system. We considered a binary classification task and assigned the positive label when the Cardio-Ankle Vascular Index (CAVI) was greater than 8.0. The proportion of selected features was adjusted in 10% increments to maximize the F1 score on the validation dataset. The result shows that our proposed method, the strategic feature and case selections, significantly improved the F1 score for the test dataset by around 2% compared with zero-shot prompt and random selection (see the attached table). Among three similarity metrics, Euclidean distance yielded the highest F1 scores. We also observed that the feature selection phase with the adjusted number of selected features improved the F1 score in almost all settings. Conclusion Our strategy for feature and case selections improved the performance of in-context learning for LLM-based atherosclerosis risk prediction.