Research on Medical Image Segmentation Based on SAM and Its Future Prospects.

447 Background: Mortality from bladder cancer (BC) increases exponentially once it invades the muscle. At the population level, accurate delineation of these patients is challenging. Methods: To develop and validate a natural language processing (NLP) model for automatically identifying muscle-invasive BC (MIBC) patients, aiding in population-based BC research. All patients with a CPT code for transurethral resection of bladder tumor (TURBT) (N = 76,060) were selected from the Department of Veterans Affairs (VA) Corporate Data Warehouse database. A sample of 600 patients (with 2,337 full-text notes) who had TURBT and confirmed pathology results were selected for NLP model development (500 patients) and validation (100 patients). Muscle-invasion (yes/no), unknown, or no cancer, were confirmed by detailed chart review of pathology notes. The NLP performance was assessed by calculating the sensitivity, positive predictive value (PPV), and overall accuracy at the individual note and patient levels. Results: In the validation cohort, the NLP model had overall accuracy of 88% and 92% at the note and patient levels. Specifically, PPV and specificity for predicting muscle-invasion on note level were 83% and 70%, respectively. The model classified non-muscle invasive BC (NMIBC) with 98% sensitivity at both the note and patient levels. Although the sensitivity for MIBC was 70% for note-level determination, the sensitivity was 86% when evaluated at the patient level. When applying the model to 71,200 patients VA-wide, the model classified 13,642 (19%) as having MIBC and 47,595 (66%) as NMIBC. The NLP model was able to identify invasion status for 96% TURBT patients at the population level. Inherent limitations include relatively small training set given the size of the VA population. Conclusions: This NLP model for identifying muscle-invasion at the population level had high accuracy. The NLP model may be a practical and accurate tool for efficiently identifying BC invasion status and may potentially aid in population-based BC research in the VA.

Understanding treatment patterns and effectiveness for patients with metastatic prostate cancer (mPCa) is dependent on accurate assessment of metastatic status. The objective was to develop a natural language processing (NLP) model for identifying patients with mPCa and evaluate the model's performance against chart-reviewed data and an International Classification of Diseases (ICD) 9/10 code-based method. In total, 139,057 radiology reports on 6,211 unique patients from the Department of Veterans Affairs were used. The gold standard was metastases by detailed chart review of radiology reports. NLP performance was assessed by sensitivity, specificity, positive predictive value, negative predictive value, and date of metastases detection. Receiver operating characteristic curves was used to assess model performance. When compared with chart review, the NLP model had high sensitivity and specificity (85% and 96%, respectively). The NLP model was able to predict patient-level metastasis status with a sensitivity of 91% and specificity of 81%, whereas sensitivity and specificity using ICD9/10 billing codes were 73% and 86%, respectively. For the NLP model, date of metastases detection was exactly concordant and within < 1 week in 55% and 58% of patients, compared with 8% and 17%, respectively, using the ICD9/10 billing codes method. The area under the curve for the NLP model was 0.911. A limitation is the NLP model was developed on the basis of a subset of patients with mPCa and may not be generalizable to all patients with mPCa. This population-level NLP model for identifying patients with mPCa was more accurate than using ICD9/10 billing codes when compared with chart-reviewed data. Upon further validation, this model may allow for efficient population-level identification of patients with mPCa.

In recent years, generative adversarial networks (GANs) have gained tremendous popularity for various imaging related tasks such as artificial image generation to support AI training. GANs are especially useful for medical imaging-related tasks where training datasets are usually limited in size and heavily imbalanced against the diseased class. We present a systematic review, following the PRISMA guidelines, of recent GAN architectures used for medical image analysis to help the readers in making an informed decision before employing GANs in developing medical image classification and segmentation models. We have extracted 54 papers that highlight the capabilities and application of GANs in medical imaging from January 2015 to August 2020 and inclusion criteria for meta-analysis. Our results show four main architectures of GAN that are used for segmentation or classification in medical imaging. We provide a comprehensive overview of recent trends in the application of GANs in clinical diagnosis through medical image segmentation and classification and ultimately share experiences for task-based GAN implementations.

To develop and evaluate an internally validated natural language processing (NLP) model to determine guideline adherence of antibiotic decisions in the emergency department (ED) for skin and soft tissue infections (SSTIs). This cross-sectional pilot study developed and applied a random forest (RF)/NLP model to classify clinical narratives of patients with skin infections as requiring either methicillin-resistant Staphylococcus aureus (MRSA) or non-MRSA antibiotic coverage. Conducted at a quaternary care children's hospital with an annual volume of 50,000 ED visits, our study included patients aged 1 to 18 years presenting for SSTIs to the ED from July 1, 2018, through June 30, 2023. Main outcomes included the NLP model's sensitivity, specificity, and receiver operator characteristic (ROC) curve compared to gold standard manual physician review of the medical records. A total of 1588 patients were part of the training data set, with an additional 423 patients utilized for validation. The RF model achieved an area under the curve (AUC) of 0.99, 97.0% sensitivity, and 94.9% specificity. In the validation data set, the model achieved 96.6% sensitivity and 90.1% specificity. Performance remained strong despite absent/missing patient history in certain narratives. The NLP model demonstrated that automated analysis of clinical narratives for determining guideline adherence is feasible. Despite missing data in narratives, the model's high performance suggests potential for broader application. Given the rise in antibiotic-resistant infections and the role of judicious antibiotic use, developing automated systems through NLP can significantly contribute to health care delivery and patient safety. The methodology of this study provides a feasible, sustainable path for similar applications in the emergency setting.

The highly structured nature of medical reports makes them feasible for automated large-scale patient identification. This study aimed to develop a natural language processing (NLP) model to retrospectively retrieve patients with presence and history of carotid stenosis (CS) using their ultrasound reports. Ultrasound reports from our institution between January 2016 and December 2017 were selected. To process the texts, we developed a parser to divide the raw text into fields. For baseline method, we used bag-of-n-grams and term frequency inverse document frequency as the features and used linear classifiers. Logistic regression was performed as the baseline model. Convolution and recurrent neural networks (CNN; RNN) with attention mechanism were applied to the dataset to improve the classification accuracy. We had 1220 ultrasound reports for training and 307 for testing, totaling to 1527 reports. For predicting history of CS, both CNN and RNN-attention models had a significantly higher specificity than logistic regression. In addition, RNN-attention also had a significantly higher F1 score and accuracy. For predicting presence of carotid stenosis, all models achieved above 93% accuracy. RNN-attention achieved a 95.4% accuracy, although the difference with logistic regression was not statistically significant. RNN-attention had a statistically significant higher specificity than logistic regression. We developed linear, CNN, and RNN models to predict history and presence of CS from ultrasound reports. We have demonstrated NLP to be an efficient, accurate approach for large-scale retrospective patient identification, with applications in long-term follow-up of patients and clinical research studies. • Natural language processing models using both linear classifiers and neural networks can achieve a good performance, with an overall accuracy above 90% in predicting history and presence of carotid stenosis. • Convolution and recurrent neural networks, especially with additional features including field awareness and attention mechanism, have superior performance than traditional linear classifiers. • NLP is shown to be an efficient approach for large-scale retrospective patient identification, with applications in long-term follow-up of patients and further clinical research studies.

Deep learning has achieved great success in the field of computer vision, and the precision in image classification and image detection has surpassed humans. Therefore, this paper combines deep learning and medical image segmentation, focusing on how to improve the accuracy and speed of segmentation algorithm of medical exercise rehabilitation image. Aiming at the shortcomings of traditional medical image recognition methods, a medical exercise rehabilitation image segmentation algorithm based on hierarchical features of convolutional neural networks is proposed, this paper calls it as hierarchical features of convolutional neural networks (HFCNN). The algorithm firstly samples the convolution output of multiple layers in the convolutional neural network to a unified scale and combines them to construct a hierarchical feature. This hierarchical feature combines the structural information of objects contained in the shallow layer of the network with the semantic information of objects contained in the deep layers of the network, so it has a strong ability to express. Secondly, the image can be segmented into multiple super pixels by the super pixel segmentation algorithm. The classifier is trained using the hierarchical features of the super pixel, and then the classification result is mapped back to the pixel. Finally, a fully connected conditional random field algorithm including one-potential potential energy and paired potential energy is constructed. The corresponding energy function is used to smooth the classification result of the pixel, and the regional consistency and continuity of the pixel mark are improved. Compared with many classical convolutional neural network algorithms, this algorithm not only accelerates the network convergence speed, shortens the training time, but also significantly improves the accuracy of segmentation algorithm of medical exercise rehabilitation image, showing good practical value.

DSEU-net: A novel deep supervision SEU-net for medical ultrasound image segmentation

108 Background: Classification of disease response is an essential task in cancer research and needs to be done at scale. Automating this process can improve efficiency in the generation of real-world evidence, potentially leading to better patient outcomes. We aim to develop and evaluate Natural Language Processing (NLP) models for this task. Methods: Using 6203 computed tomography (CT) and 1358 magnetic resonance imaging (MRI) reports from 587 patients with lung cancer of all stages seen at the National Cancer Centre Singapore (NCCS), we trained four NLP models (BioBERT, RadBERT-RoBERTA, BioClinicalBERT, GatorTron) to classify the reports into one of four categories: no evidence of disease, stable disease, partial response or disease progression. Model output was compared against human-curated ground truth and performance was evaluated by accuracy. Results: Of the 4 models, GatorTron performed the best (accuracy = 97.1%), followed by RadBERT-RoBERTA (accuracy = 96.2%), BioBERT (accuracy = 94.2%), with BioClinicalBERT being last (accuracy = 90.4%). NLP Model runtimes for the dataset were relatively short, with BioBERT and BioClinicalBERT taking 3 minutes per epoch, RadBERT-RoBERTA taking 6 minutes per epoch, and GatorTron taking 10 minutes per epoch on a single central processing unit (CPU). Conclusions: We have demonstrated the effectiveness of NLP models for classifying disease responses in radiology reports of lung cancer patients. This has the potential to help derive progression-free survival for real-world evidence generation.

In recent years, advancements in artificial intelligence, especially in natural language processing (NLP) models, have progressed rapidly. These models demonstrate remarkable results through training on large datasets and extensive architectures. However, the output process is often a black box, and the decision-making process remains unclear. Our research focuses on the internal representations, specifically the latent variables, generated by NLP models. In earlier work, we explored the latent variables and spaces produced by Sentence-BERT using image generation models. This approach aimed to visualize these spaces by converting discrete textual embeddings into images, introducing continuity and revealing novel relationships. This paper presents the development of an image generation model that uses a common decoder for both GPT-2 and Sentence-BERT, aimed at examining how differences in model architecture affect their latent spaces. We also investigate the impact of training dataset differences by comparing models trained in English and Japanese. Our findings indicate that while the models often generate similar outputs, significant differences emerge in sentences containing multiple elements, attributable to the differing focuses and objectives of the models. Our goal is to understand these latent spaces and contribute to the development of explainable AI.

Natural language processing with deep learning for medical adverse event detection from free-text medical narratives: A case study of detecting total hip replacement dislocation

The American Society of Anesthesiologist’s Physical Status (ASA-PS) classification system assesses comorbidities before sedation and analgesia, but inconsistencies among raters have hindered its objective use. This study aimed to develop natural language processing (NLP) models to classify ASA-PS using pre-anesthesia evaluation summaries, comparing their performance to human physicians. Data from 717,389 surgical cases in a tertiary hospital (October 2004–May 2023) was split into training, tuning, and test datasets. Board-certified anesthesiologists created reference labels for tuning and test datasets. The NLP models, including ClinicalBigBird, BioClinicalBERT, and Generative Pretrained Transformer 4, were validated against anesthesiologists. The ClinicalBigBird model achieved an area under the receiver operating characteristic curve of 0.915. It outperformed board-certified anesthesiologists with a specificity of 0.901 vs. 0.897, precision of 0.732 vs. 0.715, and F1-score of 0.716 vs. 0.713 (all p <0.01). This approach will facilitate automatic and objective ASA-PS classification, thereby streamlining the clinical workflow.

One unintended consequence of the Electronic Health Records (EHR) implementation is the overuse of content-importing technology, such as copy-and-paste, that creates “bloated” notes containing large amounts of textual redundancy. Despite the rising interest in applying machine learning models to learn from real-patient data, it is unclear how the phenomenon of note bloat might affect the Natural Language Processing (NLP) models derived from these notes. Therefore, in this work we examine the impact of redundancy on deep learning-based NLP models, considering four clinical prediction tasks using a publicly available EHR database. We applied two deduplication methods to the hospital notes, identifying large quantities of redundancy, and found that removing the redundancy usually has little negative impact on downstream performances, and can in certain circumstances assist models to achieve significantly better results. We also showed it is possible to attack model predictions by simply adding note duplicates, causing changes of correct predictions made by trained models into wrong predictions. In conclusion, we demonstrated that EHR text redundancy substantively affects NLP models for clinical prediction tasks, showing that the awareness of clinical contexts and robust modeling methods are important to create effective and reliable NLP systems in healthcare contexts.

AUA guidelines for shared decision making (SDM) in prostate cancer recommend discussion of five content areas in consultations: (1) cancer severity (tumor risk (TR), pathology results (PR)); (2) life expectancy (LE); (3) cancer prognosis (CP); (4) baseline urinary and erectile function (UF and EF); and (5) treatment side effects (erectile dysfunction (ED), urinary incontinence (UI), and irritative urinary symptoms (LUTS)). However, patient retention of information after the visit and inconsistent risk communication by physicians are barriers to informed SDM. We sought to develop natural language processing (NLP) models based on recorded consultations to provide key information to patients and audit quality of physician communication. We used 50 consultation transcripts to train and validate NLP models to identify sentences related to key concepts. We then tested whether communication quality across entire consultations could be determined by sentences with the highest model-predicted topic concordance in 20 separate consultation transcripts. Our development dataset included 28,927 total sentences, with 75% reserved for training and 25% for internal validation. The Random Forest model had the highest accuracy for identifying topic-concordant sentences, with area under the curve 0.98, 0.94, 0.89, 0.92, 0.84, 0.96, 0.98, 0.97, and 0.99 for TR, PR, LE, CP, UF, EF, ED, UI, and LUTS compared with manual coding across all concepts in the internal validation dataset. In 20 separate consultations, the top 10 model-identified sentences correctly graded communication quality across entire consultations with accuracies of 100%, 90%, 95%, 95%, 80%, 95%, 85%, 100%, and 95% for TR, PR, LE, CP, UF, EF, ED, UI, and LUTS compared with manual coding, respectively. NLP models accurately capture key information and grade quality of physician communication in prostate cancer consultations, providing the foundation for scalable quality assessment of risk communication.

The rapid development of science and technology has been accompanied by an exponential growth in peer-reviewed scientific publications. At the same time, the review of each paper is a laborious process that must be carried out by subject matter experts. Thus, providing high-quality reviews of this growing number of papers is a significant challenge. In this work, we ask the question “can we automate scientific reviewing? ”, discussing the possibility of using natural language processing (NLP) models to generate peer reviews for scientific papers. Because it is non-trivial to define what a “good” review is in the first place, we first discuss possible evaluation metrics that could be used to judge success in this task. We then focus on the machine learning domain and collect a dataset of papers in the domain, annotate them with different aspects of content covered in each review, and train targeted summarization models that take in papers as input and generate reviews as output. Comprehensive experimental results on the test set show that while system-generated reviews are comprehensive, touching upon more aspects of the paper than human-written reviews, the generated texts are less constructive and less factual than human-written reviews for all aspects except the explanation of the core ideas of the papers, which are largely factually correct. Given these results, we pose eight challenges in the pursuit of a good review generation system together with potential solutions, which, hopefully, will inspire more future research in this direction. We make relevant resource publicly available for use by future research: https://github. com/neulab/ReviewAdvisor. In addition, while our conclusion is that the technology is not yet ready for use in high-stakes review settings we provide a system demo, ReviewAdvisor (http://review.nlpedia.ai/), showing the current capabilities and failings of state-of-the-art NLP models at this task (see demo screenshot in A.2). A review of this paper written by the system proposed in this paper can be found in A.1.

Cerebral hemorrhage is a critical medical condition that necessitates a rapid and precise diagnosis for timely medical intervention, including emergency operation. Computed tomography (CT) is essential for identifying cerebral hemorrhage, but its effectiveness is limited by the availability of experienced radiologists, especially in resource-constrained regions or when shorthanded during holidays or at night. Despite advancements in artificial intelligence-driven diagnostic tools, most require technical expertise. This poses a challenge for widespread adoption in radiological imaging. The introduction of advanced natural language processing (NLP) models such as GPT-4, which can annotate and analyze images without extensive algorithmic training, offers a potential solution. This study investigates GPT-4's capability to identify and annotate cerebral hemorrhages in cranial CT scans. It represents a novel application of NLP models in radiological imaging. In this retrospective analysis, we collected 208 CT scans with 6 types of cerebral hemorrhages at Ren Ji Hospital, Shanghai Jiao Tong University School of Medicine, between January and September 2023. All CT images were mixed together and sequentially numbered, so each CT image had its own corresponding number. A random sequence from 1 to 208 was generated, and all CT images were inputted into GPT-4 for analysis in the order of the random sequence. The outputs were subsequently examined using Photoshop and evaluated by experienced radiologists on a 4-point scale to assess identification completeness, accuracy, and success. The overall identification completeness percentage for the 6 types of cerebral hemorrhages was 72.6% (SD 18.6%). Specifically, GPT-4 achieved higher identification completeness in epidural and intraparenchymal hemorrhages (89.0%, SD 19.1% and 86.9%, SD 17.7%, respectively), yet its identification completeness percentage in chronic subdural hemorrhages was very low (37.3%, SD 37.5%). The misidentification percentages for complex hemorrhages (54.0%, SD 28.0%), epidural hemorrhages (50.2%, SD 22.7%), and subarachnoid hemorrhages (50.5%, SD 29.2%) were relatively high, whereas they were relatively low for acute subdural hemorrhages (32.6%, SD 26.3%), chronic subdural hemorrhages (40.3%, SD 27.2%), and intraparenchymal hemorrhages (26.2%, SD 23.8%). The identification completeness percentages in both massive and minor bleeding showed no significant difference (P=.06). However, the misidentification percentage in recognizing massive bleeding was significantly lower than that for minor bleeding (P=.04). The identification completeness percentages and misidentification percentages for cerebral hemorrhages at different locations showed no significant differences (all P>.05). Lastly, radiologists showed relative acceptance regarding identification completeness (3.60, SD 0.54), accuracy (3.30, SD 0.65), and success (3.38, SD 0.64). GPT-4, a standout among NLP models, exhibits both promising capabilities and certain limitations in the realm of radiological imaging, particularly when it comes to identifying cerebral hemorrhages in CT scans. This opens up new directions and insights for the future development of NLP models in radiology. ClinicalTrials.gov NCT06230419; https://clinicaltrials.gov/study/NCT06230419.