Generative data-engine foundation model for universal few-shot 2D vascular image segmentation.

The digitization of histology slides has revolutionized pathology, providing massive datasets for cancer diagnosis and research. Contrastive self-supervised and vision-language models have been shown to effectively mine large pathology datasets to learn discriminative representations. On the other hand, generative models, capable of synthesizing realistic and diverse images, present a compelling solution to address unique problems in pathology that involve synthesizing images; overcoming annotated data scarcity, enabling privacy-preserving data sharing, and performing inherently generative tasks, such as virtual staining. We introduce PixCell, the first diffusion-based generative foundation model for histopathology. We train PixCell on PanCan-30M, a vast, diverse dataset derived from 69,184 H&E-stained whole slide images covering various cancer types. We employ a progressive training strategy and a self-supervision-based conditioning that allows us to scale up training without any annotated data. PixCell generates diverse and high-quality images across multiple cancer types, which we find can be used in place of real data to train a self-supervised discriminative model. Synthetic images shared between institutions are subject to fewer regulatory barriers than would be the case with real clinical images. Furthermore, we showcase the ability to precisely control image generation using a small set of annotated images, which can be used for both data augmentation and educational purposes. Testing on a cell segmentation task, a mask-guided PixCell enables targeted data augmentation, improving downstream performance. Finally, we demonstrate PixCell’s ability to use H&E structural staining to infer results from molecular marker studies; we use this capability to infer IHC staining from H&E images. Our trained models are publicly released to accelerate research in computational pathology.

As an important part of the industrialization process, fully automated instrument monitoring and identification are experiencing an increasingly wide range of applications in industrial production, autonomous driving, and medical experimentation. However, digital instruments usually have multi-digit features, meaning that the numeric information on the screen is usually a multi-digit number greater than 10. Therefore, the accuracy of recognition with traditional algorithms such as threshold segmentation and template matching is low, and thus instrument monitoring still relies heavily on human labor at present. However, manual monitoring is costly and not suitable for risky experimental environments such as those involving radiation and contamination. The development of deep neural networks has opened up new possibilities for fully automated instrument monitoring; however, neural networks generally require large training datasets, costly data collection, and annotation. To solve the above problems, this paper proposes a new instrument monitoring method based on few-shot learning (FLIMM). FLIMM improves the average accuracy (ACC) of the model to 99% with only 16 original images via effective data augmentation method. Meanwhile, due to the controllability of simulated image generation, FLIMM can automatically generate annotation information for simulated numbers, which greatly reduces the cost of data collection and annotation.

Accurate detection of vessel boundaries is particularly important for a precise extraction of vasculatures in magnetic resonance angiography (MRA). In this paper, we propose the use of weighted local variance (WLV)-based edge detection scheme for vessel boundary detection in MRA. The proposed method is robust against changes of intensity contrast of edges and capable of giving high detection responses on low contrast edges. These robustness and capabilities are essential for detecting the boundaries of vessels in low contrast regions of images, which can contain intensity inhomogeneity, such as bias field, interferences induced from other tissues, or fluctuation of the speed related vessel intensity. The performance of the WLV-based edge detection scheme is studied and shown to be able to return strong and consistent detection responses on low contrast edges in the experiments. The proposed edge detection scheme can be embedded naturally in the active contour models for vascular segmentation. The WLV-based vascular segmentation method is tested using MRA image volumes. It is experimentally shown that the WLV-based edge detection approach can achieve high-quality segmentation of vasculatures in MRA images.

Image segmentation for object to extract data from ultrasound acquired is an essential preprocessing step for the effective diagnosis. Various image segmentation methods have been studied. In this study, interactive image segmentation method by graph cut algorithm is proposed to develop a variety of applications of vascular ultrasound imaging and diagnostics. General imaging and vascular ultrasound imaging segmentation by entering constrain condition such as foreground and background. In the future it will be able to develop new ultrasound diagnostics.

PurposeThis study aims to enhance surgical safety by developing a method for vascular segmentation in laparoscopic surgery videos with limited visibility. We introduce an adaptive sensitivity-fisher regularization (ASFR) approach to adapt neural networks, initially trained on non-medical datasets, for vascular segmentation in laparoscopic videos.MethodsOur approach utilizes heterogeneous transfer learning by integrating fisher information and sensitivity analysis to mitigate catastrophic forgetting and overfitting caused by limited annotated data in laparoscopic videos. We calculate fisher information to identify and preserve critical model parameters while using sensitivity measures to guide adjustment for new task.ResultsThe fine-tuned models demonstrated high accuracy in vascular segmentation across various complex video sequences, including those with obscured vessels. For both invisible and visible vessels, our method achieved an average Dice score of 41.3. In addition to outperforming traditional transfer learning approaches, our method exhibited strong adaptability across multiple advanced video segmentation architectures. ConclusionThis study introduces a novel heterogeneous transfer learning approach, ASFR, which significantly enhances the precision of vascular segmentation in laparoscopic videos. ASFR effectively addresses critical challenges in surgical image analysis and paves the way for broader applications in laparoscopic surgery, promising improved patient outcomes and increased surgical efficiency.

Vascular segmentation plays an important role in medical image analysis. A novel technique for the automaticextraction of vascular trees from 2D medical images is presented, which combines Hessian-based multiscale filtering and a modified level set method. In the proposed algorithm, the morphological top-hat transformation is firstly adopted to attenuate background. Then Hessian-based multiscale filtering is used to enhance vascular structures by combining Hessian matrix with Gaussian convolution to tune the filtering response to the specific scales. Because Gaussian convolution tends to blur vessel boundaries, which makes scale selection inaccurate, an improved level set method is finally proposed to extract vascular structures by introducing an external constrained term related to the standard deviation of Gaussian function into the traditional level set. Our approach was tested on synthetic images with vascular-like structures and 2D slices extracted from real 3D abdomen magnetic resonance angiography (MRA) images along the coronal plane. The segmentation rates for synthetic images are above 95%. The results for MRA images demonstrate that the proposed method can extract most of the vascular structures successfully and accurately in visualization. Therefore, the proposed method is effective for the vascular tree extraction in medical images.

Clinical assessment of vasculatures is essential for the detection and treatment of vascular diseases which can be potentially fatal. To facilitate clinical assessment of blood vessels, there is a growing need of developing computer assisted vessel segmentation schemes based on magnetic resonance angiographic (MRA) images. A vast number of approaches have been proposed in the past decade for the segmentation of vascular structures in MRA images. These approaches were devised according to different assumptions on the shape of blood vessels and different underlying prior knowledge about the desired imaging modalities. The development of these approaches aims at delivering more accurate and robust segmentation results. Nonetheless, these approaches face different technical challenges that prohibit them from being widely employed in the clinical environment. The challenges include significant contrast variation of vessel boundaries in MRA images, the excessive computation time required by some algorithms and the complicated geometry of vascular structures. These challenges motivate us to propose three novel edge detection and vascular segmentation methods. In the first proposed method, vessel segmentation is performed grounded on the edge detection responses given by the weighted local variance-based edge detector. This detector is robust against large intensity contrast changes and capable of returning accurate detection responses on low contrast edges. Our second method is an efficient implementation of a well founded vessel detection approach. The proposed efficient implementation is a thousand times faster than the conventional implementation without segmentation performance deterioration. The third method is a curvilinear structure descriptor which is robust against the disturbance induced by closely located objects. Preliminary experimental results show that the proposed methods are very suitable for vascular segmentation in MRA images.

Realistic image synthesis is to generate an image that is perceptually indistinguishable from an actual image. Generating realistic looking images with large variations (e.g., large spatial deformations and large pose change), however, is very challenging. Handing large variations as well as preserving appearance needs to be taken into account in the realistic looking image generation. In this paper, we propose a novel realistic looking image synthesis method, especially in large change demands. To do that, we devise generative guiding blocks. The proposed generative guiding block includes realistic appearance preserving discriminator and naturalistic variation transforming discriminator. By taking the proposed generative guiding blocks into generative model, the latent features at the layer of generative model are enhanced to synthesize both realistic looking- and target variation- image. With qualitative and quantitative evaluation in experiments, we demonstrated the effectiveness of the proposed generative guiding blocks, compared to the state-of-the-arts.

Scalable learning for bridging the species gap in image-based plant phenotyping

Basing on morphological characteristics of retinal vascular structure and its changing, in order to realize the early diagnosis and quantitative analysis of the severity of diabetes, cardiovascular disease, and fundus disease ect, we propose a retinal vascular image multi-scale segmentation method based on hybrid model in this paper. First, combing the statistical principle and image enhancement method, the retinal vascular image was segmented and extracted. Second, a hybrid model consisting of a Gaussian model and two exponential models for vascular fitting was developed. Then, the K-means clustering method is used to estimate initial parameters, and the estimated parameters are iteratively processed to solve model parameters; Finally, the retinal vascular image is segmented according to the maximum a posteriori criterion to extract vessels. The experimental results on DRIVE database show that our proposed segmentation method can extract retinal vascular network effectively, and the segmentation accuracy is 94.62%. The proposed segmentation method can thus help the ophthalmologists in efficient retinal image analysis and fruitful treatment to the patient community.

Coronary artery disease (CAD) remains the leading cause of death globally, highlighting the critical need for accurate diagnostic tools in medical imaging. Traditional segmentation methods for coronary angiograms often struggle with vessel discontinuity and inaccuracies, impeding effective diagnosis and treatment planning. To address these challenges, we developed the Local Adaptive Segmentation Framework (LASF), enhancing the YOLOv8 architecture with dilation and erosion algorithms to improve the continuity and precision of vascular image segmentation. We further enriched the ARCADE dataset by meticulously annotating both proximal and distal vascular segments, thus broadening the dataset's applicability for training robust segmentation models. Our comparative analyses reveal that LASF outperforms well-known models such as UNet and DeepLabV3Plus, demonstrating superior metrics in precision, recall, and F1-score across various testing scenarios. These enhancements ensure more reliable and accurate segmentation, critical for clinical applications. LASF represents a significant advancement in the segmentation of vascular images within coronary angiograms. By effectively addressing the common issues of vessel discontinuity and segmentation accuracy, LASF stands to improve the clinical management of CAD, offering a promising tool for enhancing diagnostic accuracy and patient outcomes in medical settings.

In this paper, we introduce multi-symplectic Lagrangian variational integrators for solving Chan-Vese active contour models in image segmentation. Energy functionals are discretized firstly, and numerical schemes are derived from discrete Euler-Lagrange equations based on discrete variational principle. Lagrangian variational integrators preserve native differential structure-multi-symplecticity, that makes the numerical methods have a satisfied behavior. Experiments are performed on the benchmark images from literature. We further evaluated the methods in a segmentation database containing 1023 images. It shows that the proposed numerical schemes attain relatively faster convergence rates and better segmentation accuracy. Comparisons with the standard explicit Euler method of the original Chan-Vese model and other fast numerical optimization methods show that the proposed methods have better stability, higher accuracy, and are more robust when dealing with a large number of pictures. This study provides an example for further research to improve the performance of other existing image segmentation methods based on active contour models.

Convolutional neural networks (CNNs), as a typical deep learning technique, have been widely used in image segmentation, but they often require a large amount of annotated data. However, the number of available pixel-wise labeled medical images is extremely small, and this prevents the application of CNNs in many medical image segmentation tasks. We proposed a self-paced learning with diversity SPLD) framework to boost the performances of medical image segmentation models with a limited amount of annotated data. Self-paced learning (SPL) is a learning regime that selects training samples in order from the easiest to the most difficult for model training. In addition, we took the diversity of the data into consideration. The proposed self-paced learning with diversity by query-by-committee (SPLD-QBC) algorithm dynamically and diversely selects the appropriate training data to boost the performance of an image segmentation model. SPLD-QBC incorporates the query-by-committee (QBC) technique for data selection and affinity propagation for optimizing the data diversity. By dynamically selecting the optimal sequence of training samples from different probability distributions, the segmentation models achieved improved performances. To verify the effectiveness of the proposed SPLD-QBC framework, we conducted experiments on three medical image segmentation tasks with five different datasets. The experimental results indicated that the proposed SPLD-QBC algorithm significantly improved upon the segmentation performances of the baseline models and resulted in a higher Dice score, surface distance and mean intersection over union (mIoU). The proposed SPLD significantly boosts the segmentation performances of models and is easily embedded into CNN-based image segmentation models.

HomeRadiology: Imaging CancerVol. 2, No. 2 Previous Research HighlightsFree AccessA New Approach for Automated Image Segmentation of Organs at Risk in Cervical CancerMeredith Jones, Lacey R. McNallyMeredith Jones, Lacey R. McNallyMeredith JonesLacey R. McNallyPublished Online:Mar 27 2020https://doi.org/10.1148/rycan.2020204010MoreSectionsPDF ToolsImage ViewerAdd to favoritesCiteTrack CitationsPermissionsReprints ShareShare onFacebookTwitterLinked InEmail Take-Away Points■ Major Focus: Development of an automated multiclass image segmentation model to identify organs at risk (OARs) on CT scans in patients with locally advanced cervical cancer, prior to radiation therapy.■ Key Results: The convolutional neural network model outperformed model-based, intensity-based, and atlas-based segmentation algorithms, with comparable results to other deep learning segmentation models while also decreasing the time needed for image segmentation by several thousand-fold.■ Impact: Reduction of intra- and interuser variability imposed by manual OAR delineation can lead to more accurate and efficient radiation therapy planning.The recent article by Liu et al developed an automated multiclass image segmentation model to identify organs at risk (OARs) on CT scans. To plan effective and safe radiation therapy in the context of cervical cancer, a CT scan is often acquired prior to the first treatment. Currently, a radiation oncologist will segment the CT images by hand, which is a time-consuming process, subject to intra- and interobserver variability, to identify OARs. To overcome these potential limitations and biases, automated image segmentation techniques can be utilized to improve accuracy and speed of annotating each image. A major problem in creation of deep learning image segmentation models for medical images is the lack of available datasets for model training and validation. Liu et al developed a two-dimensional image segmentation model based off the established convolutional neural network architecture, U-Net. U-Net was utilized because it is feasible to train from scratch with a small training set.This study looked at the CT scans of 105 patients with locally advanced cervical cancer. OAR segmentation for cervical cancer is a particularly challenging problem due to variability in organ size and frequent unclear boundaries. Even an experienced radiologist will have trouble. Seven organs were identified as OARs for cervical cancer radiation therapy: bladder, bone marrow, femoral head left, femoral head right, rectum, small intestine, and spinal cord.The model uses context aggregation blocks where U-Net has convolution layers. These context aggregation blocks use dilation convolution layers with different dilated rates and normal convolution layers with different kernel sizes to yield a low false-positive rate and accurate feature extraction in a large receptive field. A squeeze-extract block was used to ensure proper consideration of organs that are small. A class-weighted cross-entropy loss function was also used to overcome the imbalance in the datasets created by the differences in size and shape of each OAR.A ground truth was available based on the radiation oncologist’s annotations, allowing a Dice similarity coefficient (DSC) and 95th Hausdorff distance (HD) to be calculated and used as an evaluation metric. The DSC and the HD metrics indicated a comparable performance to other deep learning image segmentation models in most of the organs analyzed; inferior results were seen in the rectum and superior results in the intestines. All results were analyzed by an oncologist who determined that greater than 90% of results produced by this model were highly acceptable to be used during radiation therapy planning. To make the model more robust and generalizable, it is necessary to acquire a larger dataset with scans from different sources. Translation of this model to three dimensional may ultimately improve clinical translation.Highlighted ArticleLiu Z, Liu X, Xiao B, et al. Segmentation of organs-at-risk in cervical cancer CT images with a convolutional neural network. Physica Medica 2020;69:184-191. doi: https://doi.org/10.1016/j.ejmp.2019.12.008Highlighted ArticleLiu Z, Liu X, Xiao B, et al. Segmentation of organs-at-risk in cervical cancer CT images with a convolutional neural network. Physica Medica 2020;69:184-191. doi: https://doi.org/10.1016/j.ejmp.2019.12.008 Crossref, Medline, Google ScholarArticle HistoryPublished online: Mar 27 2020 FiguresReferencesRelatedDetailsCited ByJournal of Biomimetics, Biomaterials and Biomedical Engineering, Vol. 49Mathematical Problems in Engineering, Vol. 2020Recommended Articles nnU-Net: Further Automating Biomedical Image AutosegmentationRadiology: Imaging Cancer2021Volume: 3Issue: 1Deep Learning: A Primer for RadiologistsRadioGraphics2017Volume: 37Issue: 7pp. 2113-2131Generative Adversarial Networks: A Primer for RadiologistsRadioGraphics2021Volume: 41Issue: 3pp. 840-857Current Applications and Future Impact of Machine Learning in RadiologyRadiology2018Volume: 288Issue: 2pp. 318-328Deep Generative Adversarial Networks: Applications in Musculoskeletal ImagingRadiology: Artificial Intelligence2021Volume: 3Issue: 3See More RSNA Education Exhibits A Multidisciplinary Approach for Program Development with Artificial Intelligence in Pancreatic Cancer: How We Fit InDigital Posters2019Abnormal Changes After Radiation Therapy: What Radiologists Should KnowDigital Posters2018Artificial Intelligence in Diagnostic Imaging: Current Applications and Future PerspectiveDigital Posters2019 RSNA Case Collection Acute Radiation EnteropathyRSNA Case Collection2021Small Bowel LymphomaRSNA Case Collection2021Hirschsprung DiseaseRSNA Case Collection2021 Vol. 2, No. 2 Metrics Downloaded 310 times Altmetric Score

ObjectivesPosition- and posture-dependent deformation of the vascular system is a relatively unexplored field. The goal of this scoping review was to create an overview of existing vascular imaging modalities in different body positions and postures and address the subsequent changes in vascular anatomy.MethodsScopus, Medline, and Cochrane were searched for literature published between January 1, 2000, and June 30, 2022, incorporating the following categories: image modality, anatomy, orientation, and outcomes.ResultsOut of 2446 screened articles, we included 108. The majority of papers used ultrasound (US, n = 74) in different body positions and postures with diameter and cross-sectional area (CSA) as outcome measures. Magnetic resonance imaging (n = 22) and computed tomography (n = 8) were less frequently used but allowed for investigation of other geometrical measures such as vessel curvature and length. The venous system proved more sensitive to postural changes than the arterial system, which was seen as increasing diameters of veins below the level of the heart when going from supine to prone to standing positions, and vice versa.ConclusionsThe influence of body positions and postures on vasculature was predominantly explored with US for vessel diameter and CSA. Posture-induced deformation and additional geometrical features that may be of interest for the (endovascular) treatment of vascular pathologies have been limitedly reported, such as length and curvature of an atherosclerotic popliteal artery during bending of the knee after stent placement. The most important clinical implications of positional changes are found in diagnosis, surgical planning, and follow-up after stent placement.Clinical relevance statementThis scoping review presents the current state and opportunities of position- and posture-dependent imaging of vascular structures using various imaging modalities that are relevant in the fields of clinical diagnosis, surgical planning, and follow-up after stent placement.Key Points• The influence of body positions and postures on the vasculature was predominantly investigated with US for vessel diameter and cross-sectional area.• Research into geometrical deformation, such as vessel length and curvature adaptation, that may be of interest for the (endovascular) treatment of vascular pathologies is limited in different positions and postures.• The most important clinical implications of postural changes are found in diagnosis, surgical planning, and follow-up after stent placement.