A Spatial Relationship Aware Dataset for Robotics

Study question Can a generalizable computer vision foundation model be trained using self-supervised learning on time-lapse embryo images to perform multiple clinically relevant downstream tasks? Summary answer We developed FEMI, a foundation model trained on eight million time-lapse images, to perform multiple clinical tasks, including blastocyst quality scoring, ploidy prediction, and segmentation. What is known already In vitro fertilization success critically depends on choosing viable embryos, a process hampered by current limited diagnostic tools, high costs, and ethical concerns. In recent years, various medical fields have increasingly explored foundation models using vision transformer architectures. These models, trained in a self-supervised manner on vast unlabeled image datasets, perform various clinically relevant tasks. Once trained on a massive unlabeled dataset, the encoder portion of the foundation model can be extracted and subsequently fine-tuned with a labeled dataset to perform various clinically relevant tasks like classification, regression, and segmentation. Study design, size, duration The foundation model was trained on 8 million Embryoscope® (E-SD) and Embryoscope+® (E+) images spanning clinics in the United States, Canada, and Europe. Datasets contained various additional information, such as ploidy status, blastocyst scores (numerical values from 3 to 14 based on Zhan et al. 2020), and segmentation masks (for trophectoderm [TE], inner cell mass [ICM], and zona pellucida [ZP]) that were used for training downstream tasks. Participants/materials, setting, methods A masked autoencoder architecture is trained on time-lapse images to create a foundation model (FEMI). The encoder from the autoencoder is extracted and fine-tuned on three image-based downstream tasks: ploidy prediction, blastocyst scoring, and embryo component segmentation. The performance of the fine-tuned foundation model is compared to VGG16 architectures, specifically trained for each downstream task. We evaluated performances using the area under the receiver-operating-characteristic (AUROC), mean absolute error (MAE), and mean intersection over union (mIoU). Main results and the role of chance The training dataset for the downstream tasks consisted of image data and labels, including PGT-A ploidy results (euploid or aneuploid), blastocyst scores (3-14), and segmentation masks. No clinical data like maternal age was used in order to assess FEMI’s learning capabilities from only time-lapse images against VGG16 model baselines. Both FEMI and baseline models used the same data splits for consistent comparisons. For ploidy prediction, models trained on a Weill Cornell Medicine (WCM) dataset were validated using WCM, Florida, and Spain data. FEMI achieved a 0.610 ± 0.004 AUROC, outperforming the baseline’s 0.590 ± 0.005. In blastocyst score prediction, FEMI attained a superior MAE of 0.099 ± 0.002 compared to the baseline’s 0.118 ± 0.001. For embryo segmentation (TE, ICM, ZP), a publicly available dataset from Simon Fraser University was split for training and validation. FEMI exceeded the baseline in mIoU for TE segmentation (0.738 ± 0.002 vs. 0.726 ± 0.002) and was comparable in ICM (0.832 ± 0.008 vs. 0.821 ± 0.011) and ZP segmentation (0.779 ± 0.002 vs. 0.778 ± 0.010). Limitations, reasons for caution The current version of FEMI is trained on a subset of our available datasets and can be improved through further training. Moreover, the downstream ploidy and blastocyst score models may be biased by errors in PGT results and the subjectivity of embryologists, respectively. Wider implications of the findings A generalizable foundational model for IVF time-lapse imaging aids clinicians in embryo selection by providing various clinical insights for better decision-making. FEMI, once fully trained, will be publicly accessible to the scientific community allowing for researchers to fine-tune FEMI on their own clinic-specific applications with their own image datasets. Trial registration number not applicable

Inefficient traffic signal control system is one of the most important causes of traffic congestion in the cities of developing countries such as Bangladesh, India, Kenya, etc. This can be mitigated by adopting a decentralized traffic-responsive signal system, where vehicle detection is performed on the road through different image-based deep learning architectures amenable to limited-resource embedded platforms as available in developing countries. Deep learning architectures currently available in this regard demand high computational resources to achieve higher inference speed and better accuracy. Besides, the few existing limited-resource deep learning architectural alternatives neither attain higher inference speed nor substantial accuracy due to not overcoming the inherent limitations. To this extent, in this study, we propose a novel limited-resource deep learning architecture, namely DhakaNet, for real-time vehicle detection in on-road (street-view) traffic images. Our proposed architecture leverages enhancing Cross-Stage Partial Network and Path Aggregation Network to build the backbone and head networks, respectively. Besides, we develop a novel multi-scale attention module to extract multi-scale meaningful features from the images, where the developed multi-scale attention module boosts the detection accuracy at the cost of small overhead. Rigorous experimental evaluation of our proposed DhakaNet over three benchmark street-view traffic datasets such as DhakaAI, IITM-HeTra-A, and IITM-HeTra-B shows up to 51% faster inference speed at a similar accuracy, or up to 13% higher accuracy at a similar inference speed compared to other state-of-the-art limited-resource deep learning architectural alternatives.

Introduction: Tumor single-cell data provide unique insights into the cellular dynamics of cancer progression and therapeutic outcomes. Inspired by advancements in natural language processing (NLP), foundation models are increasingly used in single-cell analyses to uncover biological insights and identify clinical biomarkers. However, the performance of these models on cancer-specific tasks remains underexplored. We present an extensive evaluation of foundation models trained on millions of single-cell profiles for cancer-focused investigations. Methods: We evaluated nine predictive models-three traditional methods and six foundation models (four Geneformer and two scGPT models), across various downstream tasks. Traditional models used embeddings from high variable genes (HVG), principal components (PCA), and generative model embedding (scVI), each paired with a random forest classifier for downstream classification. The evaluated foundation models varied in architecture sizes, pretraining dataset sizes (30M to 95M cell profiles), and training datasets, which were either generic or cancer-specific. We assessed the quality of representations by measuring their predictive performance on lung cancer treatment status (TKI treated vs. treatment-naive patients) and breast cancer subtypes (ER+ vs. TNBC). We examined how model characteristics influenced their performance in these domains for all tasks. Results: Zero-shot evaluations of the foundation models showed superior single-cell data clustering, measured by average silhouette width (ASW), compared to traditional methods (HVG, PCA, scVI), suggesting that pre-trained embeddings from millions of single-cell profiles may offer enhanced representations. We thus evaluated the learned representation on downstream predictive tasks, showing that foundation models with a larger pretraining dataset (95M cells), cancer-specific training, larger input size (4096 genes), and deeper model (12 layers) achieved the best performance among foundation models (F1=0.97) for predicting treatment status in lung cancer. In contrast, smaller and generic foundation models had significantly lower performance, however, additional self-supervised training on task-specific data rescued their performance to match larger foundation models. Similar trends were observed in predicting breast cancer subtypes. Notably, the best foundation model’s performance was not significantly different from the best traditional method (PCA) in predicting breast cancer subtypes. Conclusions: These results highlight key factors driving foundation model performance on cancer-specific tasks. While current foundation models show inconsistent contributions to cancer-specific questions, observed performance trends highlight their potential to enhance single-cell data representation and drive significant advancements in cancer research and precision medicine. Citation Format: Haitham Elmarakeby, Ahmed Roman, Shreya Johri, Eliezer M. Van Allen. Evaluation of single-cell foundation models for cancer outcome predictions [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2025; Part 1 (Regular Abstracts); 2025 Apr 25-30; Chicago, IL. Philadelphia (PA): AACR; Cancer Res 2025;85(8_Suppl_1):Abstract nr 5084.

ABSTRACTFoundation Models (FMs) have made significant strides in both industrial and scientific domains. In this paper, we evaluate the performance of FMs for single‐cell sequencing data analysis through comprehensive experiments across eight downstream tasks pertinent to single‐cell data. Overall, the top FMs include scGPT, Geneformer, and CellFM by considering model performances and user accessibility among ten single‐cell FMs. However, by comparing these FMs with task‐specific methods, we found that single‐cell FMs may not consistently excel than task‐specific methods in all tasks, which challenges the necessity of developing foundation models for single‐cell analysis. In addition, we evaluated the effects of hyperparameters, initial settings, and stability for training single‐cell FMs based on a proposed scEval framework, and provide guidelines for pre‐training and fine‐tuning to enhance the performances of single‐cell FMs. Our work summarizes the current state of single‐cell FMs, points to their constraints and avenues for future development, and offers a freely available evaluation pipeline to benchmark new models and improve method development.

ObjectiveTo assess the performance of the current foundation models in histopathology. Patients and MethodsThe assessment involves a comprehensive evaluation of some foundation models, such as the CLIP derivatives, namely PLIP and BiomedCLIP, which were fine-tuned on data scraped from the internet. The comparison is performed against simpler and nonfoundational histology models that are trained on well-curated data, eg, the cancer genome atlas. All models are evaluated on 8 datasets, 4 of which are internal histology datasets collected and curated at Mayo Clinic, and 4 well-known public datasets: PANDA, BRACS, CAMELYON16, and DigestPath. Evaluation metrics include accuracy and macro-averaged F1 score, using a majority vote among top-k (eg, MV@5) at the whole slide image/patch levels. Moreover, all models are evaluated in classification settings. This detailed analysis allows for a deep understanding of each model’s performance across various datasets. ResultsIn various evaluation tasks, domain-specific (and nonfoundational) models like DinoSSLPath and KimiaNet outperform general-purpose foundation models. The DinoSSLPath excels in whole slide image-level retrieval for internal colorectal cancer and liver datasets with MV@5 macro-averaged F1 scores of 63% and 74%, respectively. The KimiaNet leads in breast and skin cancer tasks with respective Top-1 and MV@5 scores of 56% and 70%, respectively and scores 75% on the public CAMELYON16 dataset. Similar trends are observed in patch-level metrics, highlighting the advantage of using specialized datasets like the cancer genome atlas for histopathological analysis. ConclusionTo enable effective vision-language foundation models in biomedicine, high-quality, multi-modal medical datasets are essential. These datasets serve as the substrate for training models capable of translating research into clinical practice. Of importance, the alignment (correspondence) between textual and visual data—often diagnostic—is critical and requires validation by domain experts. Thus, advancing foundation models in this field necessitates collaborative efforts in data curation and validation.

BackgroundThe assessment of cortical thickness through structural magnetic resonance imaging (MRI) reveals various cortical atrophy patterns, delivering crucial insights into the extent of neurodegeneration. Numerous studies have focused on these thickness values to construct models for diverse neurodegeneration‐related tasks, such as classifying disease groups, staging progression, or predicting progression speed. However, these tasks require substantial training datasets tailored to each specific task, necessitating the development of new task‐specific models. We aimed to establish a foundational model using pre‐trained BERT, integrating context‐aware deep latent representations derived from a large cohort of unlabeled cortical atrophy data, to address these challenges.MethodWe collected 10,450 T1‐weighted MRI images within the ADNI dataset. Cortical thickness was derived using FreeSurfer across 68 regions based on the Desikan‐Killiany atlas. Emplying the BERT model, we applied its self‐attention capabilities to encode deep bidrectional representations of the data. The model is pre‐trained with 8,360 unlabeled cortical thickness data through self‐supervised learning, predicting randomly masked thickness values. Subsequently, we fine‐tuned this pre‐trained model with only one additional output layer using a supervised approach, employing independent data subsets for three challenging downstream classification tasks: (1) mild cognitive impairment (MCI) and Alzheimer’s dementia (AD), (2) MCI converters (C‐MCI) and non‐converters (N‐MCI) within 3 years, and (3) Aβ‐positive and negative cases. During fine‐tuning, demographic information encoded by one layer was also incorporated for classification purposes.ResultTable 1 provides demographic details of the subset data for fine‐tuning, showing comparable trends in each set of information, except for MMSE in the MCI and AD comparison. Table 2 outlines our model's performance on three downstream classification tasks. Despite training with a relatively limited amount of data, our model demonstrated competitive performance with state‐of‐the‐art methods, achieving 85% accuracy on the classification of MCI and AD, 85% of C‐MCI and N‐MCI, and 69% of Aβ‐positive and negative.ConclusionThis study introduces the novel cortical atrophy foundation model, demonstrating competitive performance even with a limited dataset across diverse challenging tasks. The model's performance in predictive tasks highlights its potential applicability for various nerodegeneration‐related challenges, paving the way for advanced AI applications within the realm of neurodegenerative diseases.

An automatic depression detection (ADD) system using spoken language offers the opportunity to develop practical, low-cost tools to detect symptoms early. However, limited data availability, privacy concerns, and transcription efforts pose significant challenges. Recent advancements in foundational models, capable of understanding and processing multimodal inputs, present opportunities for enhancing ADD systems. This study explores various speech foundation models to investigate their impact on ADD. We leverage Whisper and MMS for automatic transcription and integrate speech and text embeddings into a language model optimized with low-rank adaptation (LoRA). In addition, we examine the effects of fine-tuning strategies and prompt formats on model performance. We used English and Bengali datasets to demonstrate the potential of our method in ADD, even with moderate-quality transcriptions. The best speech and language foundation models outperform baseline models on both datasets.

Depth estimation in single image aims to find the depth of different individual objects and the position of an object. Based on the Make3D dataset, first image segmentation is done using GMM and to describe their main features and attributes, the feature extraction is done using PCA, finally depth map will be generated with the help of FCM algorithm. Depth map provide as an excellent method to visualize distance of the surfaces of scene objects from a viewpoint. The platform used here is MATLAB R2018a. In our method, the final depth map is represented using different color where yellow color represents far objects and blue color indicates nearest objects. In the aforementioned perspective, the proposed model establishes a relationship between depths of different individual objects. It is very important for several computer vision applications like robot, obstacle avoidance, object recognition, self-driving car etc. The major finding of our study is to find the depth and position of different individual objects.

Sucrose-induced obesity impairs novel object recognition learning in young rats

Underwater debris detection using YOLOv12 with enhanced feature extraction using R-ELAN and FlashAttention network

An automated road defect detection system is a key part of intelligent traffic infrastructure maintenance. Existing object detection models have slow inference speed and low detection accuracy. This problem is more serious under shadows, oil stains, and occlusion from natural environments. Detection accuracy of existing models drops sharply in complex environments. This paper proposes a lightweight and high-accuracy crack detection framework based on YOLOv8s, named Crack-YOLO. It is designed to solve slow speed and low accuracy in road defect detection under complex conditions. The model replaces the original convolution modules in YOLOv8 with Context-Guided (CG) modules. It uses the designed C2f_DynamicConv to replace static convolution kernels. The original detection head is replaced with an Adaptive Spatial Feature Fusion (ASFF) head. Experiments and analysis are conducted on four datasets: CrackVariety, CrackTree200, Crack500, and CFD. Experimental results show that Crack-YOLO greatly outperforms YOLOv8s in both detection speed and detection accuracy. On the self-built CrackVariety dataset, the mAP@0.5 reaches 71.4%. The inference speed reaches 416 FPS. Compared with the baseline YOLOv8s model, accuracy increases by 31.0%. The speed increases by nearly two times. In addition, the model is successfully deployed on a Raspberry Pi 5 edge device. It is integrated with the ASTM D6433 standard. After detecting road defects, the Pavement Condition Index (PCI) is calculated automatically. This proves its practicality on devices with limited computing resources.

Foundation models are pretrained on massive datasets. However, collecting medical datasets is expensive and time-consuming, and raises privacy concerns. Here we show that synthetic data generated via conditioning with disease labels can be leveraged for building high-performing medical foundation models. We pretrained a retinal foundation model, first with approximately one million synthetic retinal images with physiological structures and feature distribution consistent with real counterparts, and then with only 16.7% of the 904,170 real-world colour fundus photography images required in a recently reported retinal foundation model (RETFound). The data-efficient model performed as well or better than RETFound across nine public datasets and four diagnostic tasks; and for diabetic-retinopathy grading, it used only 40% of the expert-annotated training data used by RETFound. We also support the generalizability of the data-efficient strategy by building a classifier for the detection of tuberculosis on chest X-ray images. The text-conditioned generation of synthetic data may enhance the performance and generalization of medical foundation models.

The location and intensity of non-resolved space object can be obtained in space-based surveillance, however, the material, size, and status of the object are lost. Spectrum represents the inherent property difference of the object, which can be used as an important means for feature extraction and recognition of non-resolved space object. According to the influence factors, including material, structure, background and orbit, a mathematical model for spectral properties of space object is established. Based on the model, inverse calculation method for feature extraction and recognition of space object is proposed. Taking the HuanJing-1 satellite scale model as an example, experimental verification for feature extraction and recognition of space object in typical parameters is made. Experimental results demonstrate the validity of the modeling method.

We propose a new scheme for object recognition in robot vision. The proposed scheme uses the HSI model as the input. A gradient algorithm is used to obtain the edge estimation of objects. The edge points are regarded as stimulus and the other points as nonstimulus. The stimulus context of all the possible positions of objects is coded in stimulus vectors. Then a HMM-based vision system for scene analyse and object recognition is presented. This method has been tested using our mobile service robot called ARNOLD and real data of Columbia Object Image Library (COIL-20). Through these experiments, we have demonstrated the generalisation capabilities and robustness of object recognition and classification.

Pancreatic ductal adenocarcinoma (PDAC) remains one of the most lethal malignancies, with prognosis strongly influenced by the presence of lymph node metastasis (LNM). However, preoperative LNM assessment from computed tomography (CT) is limited by low sensitivity, high inter-observer variability, and substantial heterogeneity across imaging protocols. This retrospective multi-center study (546 patients from three institutions) introduces a privacy-preserving deep learning framework that integrates large-scale CT foundation model pre-training with heterogeneity-aware federated optimization to improve LNM detection in PDAC. A CT Vision Foundation Model, pre-trained on 148,000 volumetric CT scans using contrastive self-supervised learning, is fine-tuned to generate transferable 3D representations for patient-level LNM classification. To enable decentralized model training while mitigating inter-institutional variability, we extend federated aggregation to jointly account for label-distribution discrepancies and representation-level divergence across clients. The centralized model achieved a balanced accuracy of 0.601 and a diagnostic odds ratio (DOR) of 3.45, outperforming classical machine learning baselines and prior PDAC LNM approaches. Under federated settings, the proposed heterogeneity-aware strategy consistently outperformed standard FedAvg, recovering a substantial proportion of the centralized model's performance while preserving strict data privacy. In particular, it improved balanced accuracy by 12.6% over FedAvg and demonstrated superior discriminative ability across all participating cohorts. These findings indicate that combining foundation model pre-training with discrepancy-aware federated learning enhances generalization, robustness, and clinical relevance for multi-center PDAC LNM detection. The proposed framework offers a scalable and privacy-preserving pathway for deploying deep learning models across distributed healthcare systems.