Predicting CVSS Metric via Description Interpretation

447 Background: Mortality from bladder cancer (BC) increases exponentially once it invades the muscle. At the population level, accurate delineation of these patients is challenging. Methods: To develop and validate a natural language processing (NLP) model for automatically identifying muscle-invasive BC (MIBC) patients, aiding in population-based BC research. All patients with a CPT code for transurethral resection of bladder tumor (TURBT) (N = 76,060) were selected from the Department of Veterans Affairs (VA) Corporate Data Warehouse database. A sample of 600 patients (with 2,337 full-text notes) who had TURBT and confirmed pathology results were selected for NLP model development (500 patients) and validation (100 patients). Muscle-invasion (yes/no), unknown, or no cancer, were confirmed by detailed chart review of pathology notes. The NLP performance was assessed by calculating the sensitivity, positive predictive value (PPV), and overall accuracy at the individual note and patient levels. Results: In the validation cohort, the NLP model had overall accuracy of 88% and 92% at the note and patient levels. Specifically, PPV and specificity for predicting muscle-invasion on note level were 83% and 70%, respectively. The model classified non-muscle invasive BC (NMIBC) with 98% sensitivity at both the note and patient levels. Although the sensitivity for MIBC was 70% for note-level determination, the sensitivity was 86% when evaluated at the patient level. When applying the model to 71,200 patients VA-wide, the model classified 13,642 (19%) as having MIBC and 47,595 (66%) as NMIBC. The NLP model was able to identify invasion status for 96% TURBT patients at the population level. Inherent limitations include relatively small training set given the size of the VA population. Conclusions: This NLP model for identifying muscle-invasion at the population level had high accuracy. The NLP model may be a practical and accurate tool for efficiently identifying BC invasion status and may potentially aid in population-based BC research in the VA.

Purpose The continuous influx of software vulnerabilities poses a significant challenge to organizations, necessitating effective resource allocation for threat mitigation. A key factor in this process is assessing the severity of vulnerabilities to prioritize which issues require immediate attention. This paper aims to automate the prediction of common vulnerability scoring system (CVSS) metrics from textual descriptions of vulnerabilities, reducing the reliance on manual expert analysis. Design/methodology/approach This study applies machine learning and natural language processing techniques, particularly convolutional neural networks (CNNs), to predict CVSS base metrics such as attack vectors, attack complexity and required privileges. The CNN models are trained on vulnerability descriptions and evaluated for their accuracy in predicting these metrics, which are then used to compute overall severity base scores. Findings The CNN models demonstrated high accuracy in predicting CVSS base metrics from textual descriptions. The predicted severity base scores closely align with those provided by human experts, showing the model’s potential to streamline the vulnerability assessment process. Practical implications Automating CVSS metric prediction could significantly reduce the time and effort required for vulnerability severity assessment. This would enable security teams to quickly identify and prioritize critical vulnerabilities, improving response times in cybersecurity management. Originality/value This research provides an innovative approach to vulnerability management by automating CVSS metric prediction, reducing the need for manual expert analysis and therefore accelerating security assessments.

Based on the analysis of vulnerability data for web resources and the CVSS metric, the distribution of the average CVSS (Common Vulnerability Scoring System standard for calculating a numerical vulnerability score on a ten-point scale) score for the websites of theRepublicofBelaruswas studied. The hypothesis on the distribution of the CVSS vulnerability assessment according to Poisson's law was tested by chi-square criteria. It was found that about 10% of web resources from the original general of samples of 19000 size have a critical averaged assessment level of vulnerability. As part of this work an universal system for collecting technical information about active web resources on the Internet from public directories and registries has been developed. Specific search templates have been developed using RegExp JavaScript expressions to detect the versions of technologies that were used to create websites. Based on this data the percentage distribution of used technologies, top-level domains and the geographical location of the servers were calculated. Proposed system can be adapted to any unique conditions required by information security specialists to conduct a security audit of web resources.

Understanding treatment patterns and effectiveness for patients with metastatic prostate cancer (mPCa) is dependent on accurate assessment of metastatic status. The objective was to develop a natural language processing (NLP) model for identifying patients with mPCa and evaluate the model's performance against chart-reviewed data and an International Classification of Diseases (ICD) 9/10 code-based method. In total, 139,057 radiology reports on 6,211 unique patients from the Department of Veterans Affairs were used. The gold standard was metastases by detailed chart review of radiology reports. NLP performance was assessed by sensitivity, specificity, positive predictive value, negative predictive value, and date of metastases detection. Receiver operating characteristic curves was used to assess model performance. When compared with chart review, the NLP model had high sensitivity and specificity (85% and 96%, respectively). The NLP model was able to predict patient-level metastasis status with a sensitivity of 91% and specificity of 81%, whereas sensitivity and specificity using ICD9/10 billing codes were 73% and 86%, respectively. For the NLP model, date of metastases detection was exactly concordant and within < 1 week in 55% and 58% of patients, compared with 8% and 17%, respectively, using the ICD9/10 billing codes method. The area under the curve for the NLP model was 0.911. A limitation is the NLP model was developed on the basis of a subset of patients with mPCa and may not be generalizable to all patients with mPCa. This population-level NLP model for identifying patients with mPCa was more accurate than using ICD9/10 billing codes when compared with chart-reviewed data. Upon further validation, this model may allow for efficient population-level identification of patients with mPCa.

The highly structured nature of medical reports makes them feasible for automated large-scale patient identification. This study aimed to develop a natural language processing (NLP) model to retrospectively retrieve patients with presence and history of carotid stenosis (CS) using their ultrasound reports. Ultrasound reports from our institution between January 2016 and December 2017 were selected. To process the texts, we developed a parser to divide the raw text into fields. For baseline method, we used bag-of-n-grams and term frequency inverse document frequency as the features and used linear classifiers. Logistic regression was performed as the baseline model. Convolution and recurrent neural networks (CNN; RNN) with attention mechanism were applied to the dataset to improve the classification accuracy. We had 1220 ultrasound reports for training and 307 for testing, totaling to 1527 reports. For predicting history of CS, both CNN and RNN-attention models had a significantly higher specificity than logistic regression. In addition, RNN-attention also had a significantly higher F1 score and accuracy. For predicting presence of carotid stenosis, all models achieved above 93% accuracy. RNN-attention achieved a 95.4% accuracy, although the difference with logistic regression was not statistically significant. RNN-attention had a statistically significant higher specificity than logistic regression. We developed linear, CNN, and RNN models to predict history and presence of CS from ultrasound reports. We have demonstrated NLP to be an efficient, accurate approach for large-scale retrospective patient identification, with applications in long-term follow-up of patients and clinical research studies. • Natural language processing models using both linear classifiers and neural networks can achieve a good performance, with an overall accuracy above 90% in predicting history and presence of carotid stenosis. • Convolution and recurrent neural networks, especially with additional features including field awareness and attention mechanism, have superior performance than traditional linear classifiers. • NLP is shown to be an efficient approach for large-scale retrospective patient identification, with applications in long-term follow-up of patients and further clinical research studies.

To develop and evaluate an internally validated natural language processing (NLP) model to determine guideline adherence of antibiotic decisions in the emergency department (ED) for skin and soft tissue infections (SSTIs). This cross-sectional pilot study developed and applied a random forest (RF)/NLP model to classify clinical narratives of patients with skin infections as requiring either methicillin-resistant Staphylococcus aureus (MRSA) or non-MRSA antibiotic coverage. Conducted at a quaternary care children's hospital with an annual volume of 50,000 ED visits, our study included patients aged 1 to 18 years presenting for SSTIs to the ED from July 1, 2018, through June 30, 2023. Main outcomes included the NLP model's sensitivity, specificity, and receiver operator characteristic (ROC) curve compared to gold standard manual physician review of the medical records. A total of 1588 patients were part of the training data set, with an additional 423 patients utilized for validation. The RF model achieved an area under the curve (AUC) of 0.99, 97.0% sensitivity, and 94.9% specificity. In the validation data set, the model achieved 96.6% sensitivity and 90.1% specificity. Performance remained strong despite absent/missing patient history in certain narratives. The NLP model demonstrated that automated analysis of clinical narratives for determining guideline adherence is feasible. Despite missing data in narratives, the model's high performance suggests potential for broader application. Given the rise in antibiotic-resistant infections and the role of judicious antibiotic use, developing automated systems through NLP can significantly contribute to health care delivery and patient safety. The methodology of this study provides a feasible, sustainable path for similar applications in the emergency setting.

Computing the probability of vulnerability exploitation in Bayesian attack graphs (BAGs) is a key process for the network security assessment. The conditional probability of vulnerability exploitation could be obtained from the exploitability of the NIST's Common Vulnerability Scoring System (CVSS). However, the method which N. Poolsappasit et al. proposed for computing conditional probability could be used only in the CVSS metric version v2.0, and can't be used in other two versions. In this paper, we present two methods for computing the conditional probability based on CVSS's other two metric versions, version 1.0 and version 3.0, respectively. Based on the CVSS, the conditional probability computation of vulnerability exploitation is complete by combining the method of N. Poolsappasit et al.

108 Background: Classification of disease response is an essential task in cancer research and needs to be done at scale. Automating this process can improve efficiency in the generation of real-world evidence, potentially leading to better patient outcomes. We aim to develop and evaluate Natural Language Processing (NLP) models for this task. Methods: Using 6203 computed tomography (CT) and 1358 magnetic resonance imaging (MRI) reports from 587 patients with lung cancer of all stages seen at the National Cancer Centre Singapore (NCCS), we trained four NLP models (BioBERT, RadBERT-RoBERTA, BioClinicalBERT, GatorTron) to classify the reports into one of four categories: no evidence of disease, stable disease, partial response or disease progression. Model output was compared against human-curated ground truth and performance was evaluated by accuracy. Results: Of the 4 models, GatorTron performed the best (accuracy = 97.1%), followed by RadBERT-RoBERTA (accuracy = 96.2%), BioBERT (accuracy = 94.2%), with BioClinicalBERT being last (accuracy = 90.4%). NLP Model runtimes for the dataset were relatively short, with BioBERT and BioClinicalBERT taking 3 minutes per epoch, RadBERT-RoBERTA taking 6 minutes per epoch, and GatorTron taking 10 minutes per epoch on a single central processing unit (CPU). Conclusions: We have demonstrated the effectiveness of NLP models for classifying disease responses in radiology reports of lung cancer patients. This has the potential to help derive progression-free survival for real-world evidence generation.

The American Society of Anesthesiologist’s Physical Status (ASA-PS) classification system assesses comorbidities before sedation and analgesia, but inconsistencies among raters have hindered its objective use. This study aimed to develop natural language processing (NLP) models to classify ASA-PS using pre-anesthesia evaluation summaries, comparing their performance to human physicians. Data from 717,389 surgical cases in a tertiary hospital (October 2004–May 2023) was split into training, tuning, and test datasets. Board-certified anesthesiologists created reference labels for tuning and test datasets. The NLP models, including ClinicalBigBird, BioClinicalBERT, and Generative Pretrained Transformer 4, were validated against anesthesiologists. The ClinicalBigBird model achieved an area under the receiver operating characteristic curve of 0.915. It outperformed board-certified anesthesiologists with a specificity of 0.901 vs. 0.897, precision of 0.732 vs. 0.715, and F1-score of 0.716 vs. 0.713 (all p <0.01). This approach will facilitate automatic and objective ASA-PS classification, thereby streamlining the clinical workflow.

One unintended consequence of the Electronic Health Records (EHR) implementation is the overuse of content-importing technology, such as copy-and-paste, that creates “bloated” notes containing large amounts of textual redundancy. Despite the rising interest in applying machine learning models to learn from real-patient data, it is unclear how the phenomenon of note bloat might affect the Natural Language Processing (NLP) models derived from these notes. Therefore, in this work we examine the impact of redundancy on deep learning-based NLP models, considering four clinical prediction tasks using a publicly available EHR database. We applied two deduplication methods to the hospital notes, identifying large quantities of redundancy, and found that removing the redundancy usually has little negative impact on downstream performances, and can in certain circumstances assist models to achieve significantly better results. We also showed it is possible to attack model predictions by simply adding note duplicates, causing changes of correct predictions made by trained models into wrong predictions. In conclusion, we demonstrated that EHR text redundancy substantively affects NLP models for clinical prediction tasks, showing that the awareness of clinical contexts and robust modeling methods are important to create effective and reliable NLP systems in healthcare contexts.

BackgroundAntibiotics are the most commonly prescribed drugs for children and frequently inappropriately prescribed. Outpatient antimicrobial stewardship interventions aim to reduce inappropriate antibiotic use. Previous work has relied on diagnosis coding for case identification which may be inaccurate. In this study, we sought to develop automated methods for analyzing note text to identify cases of acute otitis media (AOM) based on clinical documentation.MethodsWe conducted a cross-sectional retrospective chart review and sampled encounters from 7/1/2018 – 6/30/2019 for patients < 5 years old presenting for a problem-focused visit. Complete note text and limited structured data were extracted for 12 randomly selected weekdays (one from each month during the study period). An additional weekday was randomly selected for validation. The primary outcome was correctly identifying encounters where AOM was present. Human review was considered the “gold standard” and was compared to ICD codes, a natural language processing (NLP) model, and a recursive partitioning (RP) model.ResultsA total of 2,724 encounters were included in the training cohort and 793 in the validation cohort. ICD codes and NLP had good performance overall with sensitivity 91.2% and 93.1% respectively in the training cohort. However, NLP had a significant drop-off in performance in the validation cohort (sensitivity: 83.4%). The RP model had the highest sensitivity (97.2% training cohort; 94.1% validation cohort) out of the 3 methods.Figure 1. Details of encounters included in the training and validation cohorts.Table 1. Performance of ICD coding, a natural language processing (NLP) model, and a recursive partitioning (RP) model for identifying cases of acute otitis media (AOM)ConclusionNatural language processing of outpatient pediatric visit documentation can be used successfully to create models accurately identifying cases of AOM based on clinical documentation. Combining NLP and structured data can improve automated case detection, leading to more accurate assessment of antibiotic prescribing practices. These techniques may be valuable in optimizing outpatient antimicrobial stewardship efforts.DisclosuresAll Authors: No reported disclosures

Natural language processing with deep learning for medical adverse event detection from free-text medical narratives: A case study of detecting total hip replacement dislocation

Data security is a critical component of Accounting Information Systems (AIS), considering the sensitivity of financial information that must be safeguarded against cyber threats. This study aims to analyze security risks within an AIS platform by utilizing vulnerability scan data collected from the domain https://kiis.ibik.ac.id. Three open-source security tools—OpenVAS, OWASP ZAP, and NMAP—were used to detect potential system vulnerabilities. The research identifies and classifies these vulnerabilities based on severity levels and CVSS (Common Vulnerability Scoring System) scores. The findings reveal multiple medium and low-level vulnerabilities, including open TCP ports, missing anti-clickjacking headers, and improper content security policies, which could expose the system to threats such as cross-site scripting (XSS), clickjacking, and unauthorized access. The study recommends implementing essential security headers, closing unused ports, and conducting continuous system monitoring to enhance AIS resilience. These insights highlight the importance of proactive cybersecurity measures in protecting financial data integrity within modern accounting systems. Keywords: accounting information system, data security, vulnerability analysis, CVSS, cybersecurity risk

This study focuses on the problem of attack quantification in distribution automation systems (DASs) and proposes a quantitative model of attacks based on the common vulnerability scoring system (CVSS) and attack trees (ATs) to conduct a quantitative and systematic evaluation of attacks on a DAS. In the DAS security architecture, AT nodes are traversed and used to represent the attack path. The CVSS is used to quantify the attack sequence, which is the leaf node in an AT. This paper proposes a method to calculate each attack path probability and find the maximum attack path probability in DASs based on attacker behavior. The AT model is suitable for DAS hierarchical features in architecture. The experimental results show that the proposed model can reduce the influence of subjective factors on attack quantification, improve the probability of predicting attacks on the DASs, generate attack paths, better identify attack characteristics, and determine the attack path and quantification probability. The quantitative results of the model’s evaluation can find the most vulnerable component of a DAS and provide an important reference for developing targeted defensive measures in DASs.

Internet of Vehicles (IoV) is an intelligent vehicular technology that allows vehicles to communicate with each other via internet. Communications and the Internet of Things (IoT) enable cutting-edge technologies including such self-driving cars. In the existing systems, there is a maximum communication delay while transmitting the messages. The proposed system uses hybrid Co-operative, Vehicular Communication Management Framework called CAMINO (CA). Further it uses, energy efficient fast message routing protocol with Common Vulnerability Scoring System (CVSS) methodology for improving the communication delay, throughput. It improves security while transmitting the messages through networks. In this research, we present a unique intelligent vehicular infrastructure communication management framework. This framework includes additional stability for both short and long-range mobile communications. It also includes built-in cooperative intelligent transport system (C-ITS) capabilities for experimental verification in real-world contexts. In addition, an energy efficient-fast message distribution routing protocol (EE-FMDRP) has been presented. This combines the benefits between both temporal and direction oriented routing methods. This has been suggested for distributing information from the origin ends to the predetermined objective in a quick, accurate, and effective manner in the event of an emergency. The critical value scale score(CVSS) employ ratings to measure the assault probability in Markov chains. Probabilities of chained transitions allow us to statistically evaluate the integrity of a group of IoV assets. Thus the proposed method helps to enhance the vehicular systems. The CAMINO with energy efficient fast protocol using CVSS (CA- EEFP-CVSS) method outperforms in terms of shortest transmission latency achieves 2.6 sec, highest throughput 11.6%, and lowest energy usage 17% and PDR 95.78%.