Interactive exemplar-based segmentation toolkit for biomedical image analysis

“BAU-Insectv2” represents a novel agricultural dataset tailored for deep learning applications and biomedical image analysis focused on plant-insect interactions. This dataset encompasses a diverse collection of high-resolution images capturing intricate details of plant-insect interactions across various agricultural settings. Leveraging deep learning methodologies, this study aims to employ convolutional neural networks (CNN) and advanced image analysis techniques for precise insect detection, classification, and understanding of insect-related patterns within agricultural ecosystems. We mainly focus on addressing insect-related issues in South Asian crop cultivation. The dataset's extensive scope and high-quality imagery provide a robust foundation for developing and validating models capable of accurately identifying and analyzing diverse plant insects. The dataset's utility extends to biomedical image analysis, fostering interdisciplinary research avenues across agriculture and biomedical sciences. This dataset holds significant promise for advancing research in agricultural pest management, ecosystem dynamics, and biomedical image analysis techniques.

Shape information plays an important role in biomedical image analysis because of the strong shape characteristics of biological structures. It is often used as a prior to constrain or refine the intermediate shape information derived from low-level image features. In this paper, we give an overview of the sparse shape composition based prior modeling method and its various applications of biomedical image analysis. Instead of learning a generative shape model, it incorporates shape priors on-the-fly through the sparse shape composition. Particularly, a shape instance derived from low level image features is refined by a sparse linear combination of a sparse set of shapes in the repository. We also design three strategies to improve run-time efficiency. 1) When the shape repository contains a large number of instances, K-SVD can be used to learn a more compact but still informative shape dictionary. 2) If the derived shape instance has a large number of vertices, which often appears in 3D problems, affinity propagation method can be used to partition the surface into small subregions. 3) When there are multiple structures, hierarchical scheme is employed to model them simultaneously. These strategies decrease the scale of the sparse optimization problem and thus speed up the algorithm. Our method is applied on different biomedical image analysis problems, including localization, tracking and segmentation of anatomical structures. In all of these applications, this method achieves promising results.

Nuclear forensics relies on different signatures to identify the source of nuclear material. Such signatures include crystalline structure, chemical composition, and particle morphology. One way to quantify morphology in electron microscope imagery is through image segmentation, where pixels are assigned to several partitions (or groups) that correspond to particles, grains, and other objects of interest within the image. Once pixels are assigned to segments, it is relatively straightforward to quantify other quantities of interest, such as grain size, circularity, etc. However, the range and diversity of microscope images make it difficult to obtain an accurate segmentation automatically. The accuracy of segmentation can be improved through supervised learning, but this requires many images to be manually segmented. Another way to improve the accuracy is to use interactive segmentation. Interactive segmentation requires a human to provide image-specific user input to improve performance. However, the amount of user input (effort) is generally far less than is required for supervised learning. In this paper, we investigate several parallelization strategies to automatically explore the user input parameter space of interactive segmentation algorithms across a large number of images. Specifically, we investigate four different parallelization mechanisms in a high-performance computing (HPC) environment and use the Amdahl fraction to evaluate efficiency on multiple processor cores across multiple nodes. Ultimately, the parallelization strategy that was most efficient utilized the message passing interface integrated with the segmentation and quantification code. This strategy had an Amdahl fraction of 0.985 and a performance of about 0.251 s/image. These results indicate that the parameter space of interactive segmentation algorithms can be efficiently explored using HPC. This opens the door to future work where user input is reduced and where interactive image segmentation algorithms are automatically applied to large image sets.

(Hyper)-graphical models in biomedical image analysis.

Cell nucleus segmentation is a fundamental task in biomedical image analysis. Generating realistic cell nucleus data with ground truth masks can help tackle difficulties such as insufficient training data for deep learning models and the need to deal with cases (e.g., tightly clumped nuclei). Known nucleus generation methods generated individual nucleus masks from parametric models or based on direct transformations of real masks. It is difficult for these methods to capture and simulate the distributions of real nuclei and interactions among hard nuclei. In this paper, we propose a new three-stage coarse-to-fine nucleus generation method for 2D and 3D nucleus segmentation. The first stage simulates the positions and sizes of nuclei; the second stage simulates the shapes of nuclei and interactions among clumped nuclei; the third stage simulates the textures of nuclei. We evaluate our method on 2D and 3D cell nucleus image datasets. Experimental results show that our new nucleus generation method considerably helps improve cell nucleus segmentation performance and outperforms known nucleus generation methods for nucleus segmentation with a small amount of training data.

Interactive segmentation has an important role in facilitating the annotation process of future LiDAR datasets. Existing approaches sequentially segment individual objects at each LiDAR scan, repeating the process throughout the entire sequence, which is redundant and ineffective. In this work, we propose interactive 4D segmentation, a new paradigm that allows segmenting multiple objects on multiple LiDAR scans simultaneously, and Interactive4D, the first interactive 4D segmentation model that segments multiple objects on superimposed consecutive LiDAR scans in a single iteration by utilizing the sequential nature of LiDAR data. While performing interactive segmentation, our model leverages the entire spacetime volume, leading to more efficient segmentation. Operating on the 4D volume, it directly provides consistent instance IDs over time and also simplifies tracking annotations. Moreover, we show that click simulations are crucial for successful model training on LiDAR point clouds. To this end, we design a click simulation strategy that is better suited for the characteristics of LiDAR data. To demonstrate its accuracy and effectiveness, we evaluate Interactive4D on multiple LiDAR datasets, where Interactive4D achieves a new state-of-the-art by a large margin. We publicly release the code and models at https://vision.rwth-aachen.de/Interactive4D.

Biomedical image analysis has become critically important to the public health and welfare. However, analyzing biomedical images is time-consuming and labor-intensive, and has long been performed manually by highly trained human experts. As a result, there has been an increasing interest in applying machine learning to automate biomedical image analysis. Recent progress in deep learning research has catalyzed the development of machine learning in learning discriminative features from data with minimum human intervention. Many deep learning models have been designed and achieved superior performance in various data analysis applications. This chapter starts with the basic of deep learning models and some practical strategies for handling biomedical image applications with limited data. After that, case studies of deep feature extraction for gene expression pattern image annotations, imaging data completion for brain disease diagnosis, and segmentation of infant brain tissue images are discussed to demonstrate the effectiveness of deep learning in biomedical image analysis.

Evaluation of the benchmark datasets for testing the efficacy of deep convolutional neural networks

Exploring the complementarity of THz pulse imaging and DCE-MRIs: Toward a unified multi-channel classification and a deep learning framework

New software toolkits for comprehensive visualization and analysis of three-dimensional multimodal biomedical images.

Interactive segmentation has recently been explored to effectively and efficiently harvest high-quality segmentation masks by iteratively incorporating user hints. While iterative in nature, most existing interactive segmentation methods tend to ignore the dynamics of successive interactions and take each interaction independently. We here propose to model iterative interactive image segmentation with a Markov decision process (MDP) and solve it with reinforcement learning (RL) where each voxel is treated as an agent. Considering the large exploration space for voxel-wise prediction and the dependence among neighboring voxels for the segmentation tasks, multi-agent reinforcement learning is adopted, where the voxel-level policy is shared among agents. Considering that boundary voxels are more important for segmentation, we further introduce a boundary-aware reward, which consists of a global reward in the form of relative cross-entropy gain, to update the policy in a constrained direction, and a boundary reward in the form of relative weight, to emphasize the correctness of boundary predictions. To combine the advantages of different types of interactions, i. e., simple and efficient for point-clicking, and stable and robust for scribbles, we propose a supervoxel-clicking based interaction design. Experimental results on four benchmark datasets have shown that the proposed method significantly outperforms the state-of-the-arts, with the advantage of fewer interactions, higher accuracy, and enhanced robustness.

We present Spotlight, an automated user guidance technique for improving quality and efficiency of interactive segmentation tasks. Spotlight augments interactive segmentation algorithms by automatically highlighting areas in need of attention to the user during the interaction phase. We employ a 3D Livewire algorithm as our base segmentation method where the user quickly provides a minimal initial contour seeding. The quality of the initial segmentation is then evaluated based on three different metrics that probe the contour edge strength, contour stability and object connectivity. The result of this evaluation is fed into a novel algorithm that autonomously suggests regions that require user intervention. Essentially, Spotlight flags potentially problematic image regions in a prioritized fashion based on an optimization process for improving the final 3D segmentation. We present a variety of qualitative and quantitative examples demonstrating Spotlight’s intuitive use and proven utility in reducing user input by increasing automation.KeywordsUser GuidanceMedical Image AnalysisSlice PlaneInteractive SegmentationObject ConnectivityThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Real-time 3D interactive segmentation of echocardiographic data through user-based deformation of B-spline explicit active surfaces

MIDeepSeg: Minimally interactive segmentation of unseen objects from medical images using deep learning.

Special collection on biological and biomedical image analysis