A review on deep neural networks for ICD coding

International Classification of Diseases (ICD) coding plays a pivotal role in healthcare systems with its provision of a standard method for classifying medical diagnoses, treatments, and procedures. However, the process of manually applying ICD codes to clinical records is both time-consuming and error-prone, particularly considering the large magnitude of medical terminologies and the periodic changes to the coding system. This work introduces a Hierarchical Multi-Head Attention Network (HMHAN) that aims to automate ICD coding using domain-related embeddings with an attention mechanism. The proposed method uses BioClinicalBERT for feature extraction from clinical text and then a two-level attention mechanism to learn hierarchical dependencies between labels. BioClinicalBERT is pre-trained on large biomedical and clinical corpora that enable it to capture complex contextual relationships specific to medical language more effectively. The multi-head attention mechanism enables the model to focus on different parts of the input text simultaneously, learning intricate associations between medical terms and corresponding ICD codes at various levels. This method uses SMOTE (Synthetic Minority Oversampling Technique) based multi-label resampling to solve class imbalance. SMOTE generates synthetic examples for underrepresented classes, allowing the model to learn better from imbalanced data without overfitting. For this work, MIMIC-IV dataset of de-identified radiology reports and corresponding ICD codes are used. The performance of the model is assessed with F1 score, Hamming loss, and ROC-AUC metrics. Results obtained from the model with an F1 score of 0.91, Hamming loss of 0.07, and ROC-AUC of 0.92 show promising research directions to automate the ICD coding process. This system will improve the effectiveness of healthcare workflows by automating ICD code generation for advanced clinical care.

DRCNNTLe: A deep recurrent convolutional neural network with transfer learning through pre-trained embeddings for automated ICD coding.

Assessing the Accuracy of Emergency Department International Classification of Diseases Coding

Diagnostic coding of abuse related fractures at two children's emergency departments

To determine whether the discharge diagnosis codes used within tertiary hospitals accurately identified patients with Giant Cell Arteritis (GCA), as determined by expert opinion at 6 months. The study was performed across three major hospitals within Perth, Western Australia. Patients with an International Classification of Diseases (ICD) code for GCA (M31.5 and M31.6) at discharge on first inpatient presentation were identified. Review of case notes, discharge summaries, letters, pathology, and imaging results were undertaken. The percentage of patients with an ICD code for GCA at initial discharge with confirmed GCA by expert opinion at 6 months (as the anchor diagnosis) was calculated. As validation of the anchor diagnosis, the percentage who fulfilled the 2022 ACR/EULAR criteria was also calculated, the number of hospital discharges per patient with an ICD code for GCA was calculated, to determine if multiple discharges with an ICD code for GCA increased the specificity of the ICD coding. 93 out of 157 admissions with an ICD for GCA were identified as a first inpatient presentation. At 6 months follow up, 65.6%, 95 CI [55.0%, 75.1%] had confirmed GCA by expert opinion. 88.2%, 95 CI [79.8%, 93.9%] met the 2022 ACR/EULAR criteria for GCA. The specificity of the ICD coding increased with increasing number of discharges– 67.4% with single episode, 80% with two episodes, and 100% with three or more episodes (p = 0.1373). Only 43.8%, 95 CI [26.4%, 62.3%] of patients who did not have GCA had an alternative diagnosis provided. 31.3%, 95 CI [16.1%, 50.0%] of patients who did not have GCA still have the GCA diagnosis listed in their subsequent clinical records and discharge summaries. Only 2/3rds of those patients with an initial discharge ICD code for GCA were found to have confirmed GCA at follow-up. This poor correlation between the ICD coding and confirmed diagnosis of GCA will impact the quality of data extracted from administrative health datasets for epidemiological and longitudinal studies. Selecting patients with two or more GCA coded episodes could improve the homogeneity of the cohort for recruitment into GCA studies, but a larger sample size study is required.

International Classification of Diseases(ICD) is an authoritative health care classification system of different diseases and conditions for clinical and management purposes. Considering the complicated and dedicated process to assign correct codes to each patient admission based on overall diagnosis, we propose a hierarchical deep learning model with attention mechanism which can automatically assign ICD diagnostic codes given written diagnosis. We utilize character-aware neural language models to generate hidden representations of written diagnosis descriptions and ICD codes, and design an attention mechanism to address the mismatch between the numbers of descriptions and corresponding codes. Our experimental results show the strong potential of automated ICD coding from diagnosis descriptions. Our best model achieves 0.53 and 0.90 of F1 score and area under curve of receiver operating characteristic respectively. The result outperforms those achieved using character-unaware encoding method or without attention mechanism. It indicates that our proposed deep learning model can code automatically in a reasonable way and provide a framework for computer-auxiliary ICD coding.

Widespread lack of awareness and limited real-world prevalence evidence have impeded cachexia care and research. We hypothesized that healthcare professionals may identify the term cachexia, leading to International Classification of Diseases (ICD) coding for this term, with or without records of body weight loss for diagnosing cancer cachexia, and that frequently, ICD coding does not accurately reflect weight data. We assessed cachexia prevalence in patients diagnosed with cancer, using the Clinical Data Warehouse of a French hospital containing two types of real-world digitized data: (a) 'structured': coded diagnoses and electronic records; (b) 'unstructured': uncoded clinical narratives/reports: discharge summaries, procedure results, letters. Two sequential searches covering 2018-2023 (1) determined the prevalence of cachexia in all patients with a diagnosis of cancer using ICD-10 code R64 (cachexia), electronic records of body weight and unstructured narrative data; and (2) examined data of cancers of high-prevalence cachexia: colorectal, pancreatic and bronchial/lung cancers, determining (a) prevalence of cachexia by these criteria; (b) extent to which a diagnosis of cachexia was supported by weight loss data; and (c) extent to which the diagnosis of cachexia was not made despite objective weight data indicating its presence. A total of 76 547 of 737 906 patients had cancer of any primary type; 1856 (2.42%) of these had a cachexia diagnosis: 620 identified by ICD code, 1507 by unstructured data, including 271 by both. Of 6946 patients with colorectal, pancreatic or bronchial/lung cancer, 2033 patients (29.3%) were identified with cachexia by structured and/or unstructured data; both approaches were required to discover cachexia cases. From structured data an ICD-R64 search found only 254 patients described by the term cachexia, of which 127 had weight data supporting the diagnosis in the electronic medical record. An additional 1340 patients with BMI < 20 kg/m2 or weight loss > 5% were not coded as cachectic. Unstructured data for the three cancers identified 439 additional cachexic patients. (1) Standard ICD code-searching underestimated cachexia prevalence in all patients and those with high-prevalence-cachexia cancers; (2) Many cachexia cases were not diagnosed, although data indicated its presence; (3) Many cachexia cases diagnosed by judgement were not supported by data; (4) Increasing provider awareness of cancer cachexia definitions would likely improve cachexia care and research.

In epidemiologic studies, patients with head and neck squamous cell carcinoma (HNSCC) are classified mainly by the International Classification of Diseases (ICD) codes. However, some patients are of an unclear subsite, the “gray zone” cases, which could reflect ICD coding error, absence of primary subsite, or extensive primary tumors that cross over multiple subsites of the oral cavity and oropharynx. Patients with gray zone squamous cell carcinomas were compared with patients with oral cavity squamous cell carcinoma (OSCC) or oropharyngeal squamous cell carcinoma (OPSCC) and stratified by human papillomavirus (HPV) status that was determined by p16 immunostaining or HPV serology. Comparisons consisted of clinicodemographic features and prognostic outcomes presented by Kaplan-Meier curves and Cox proportional hazards regression models, reported as hazard ratios. There were 158 consecutive patients with gray zone HNSCC diagnosed at the Princess Margaret Cancer Center between 2006 and 2017: 66 had subsite coding discrepancies against the clinician’s documentation (“discrepant” cases; e.g., the diagnosis by the clinician was OSCC, while the classification by ICD coding was OPSCC), while 92 were squamous cell carcinoma of unknown primary of the head and neck (SCCUPHN) after complete diagnostic workup. Comparators included 721 consecutive OSCC and 938 OPSCC adult cases. All HPV-positive cohorts (OPSCC, discrepant, and SCCUPHN) had similar clinicodemographic characteristics and better 3- and 5-y overall survival and disease-free survival than their HPV-negative counterparts. In contrast, HPV-negative discrepant cases had prognostic outcomes most similar to HPV-negative OPSCC cases, while HPV-negative SCCUPHN had survival outcomes most similar to those of patients with OSCC in this study. HPV-positive status can improve the classification of patients with unclear or discrepant oral/oropharyngeal subsite, an improvement over classification systems that are solely clinician defined or conducted through ICD coding. However, due to clinical practice, we could not make definitive reclassification for patients with HPV-negative gray zone HNSCC.

With the vigorous development of MOOC open online courses, low completion rate of courses has become an important side effect factor which is affecting the promotion and application of MOOC courses in worldwide. Through analysis of edX open data sets, it is found that loss of learners is one of the factors which are leading to low completion rate of MOOC courses, and also there is a close correlation between learning behavior and final scores with loss of learners. Based on this challenge, some research results show that low completion rate of MOOC courses can be greatly improved by predicting the relationship between learning behavior and scores. This article focuses on application of Deep Neural Network (DNN) in prediction and analysis of MOOC learners' learning behavior. First, data analysis of course performance and learning behavior are discussed, then prediction model based on DNN is given for learning behavior and final scores. Second, experiment is done for DNN model, the results show that DNN can be used for analysis and prediction of complex correlation between learning behavior and final scores, thus which can realize more accurate prediction, reduce loss of learner, and carry out individualized teaching intervention to potential learners.

Online dynamic stability analysis plays a key role in modern power systems and is becoming even more vital with the penetration of Distributed Energy Resources (DERs). Identification of an accurate lower-order model for the ‘external system” makes it possible to carry out the stability analysis of a study system faster than the analysis for the original detailed system. However, the identified lower-order model becomes invalid when the operating state changes in the power system. ‘Black box’ representation of the ‘external system’ with the application of Deep Neural Networks (DNN) is widely discussed in the literature. In such equivalent models, the DNN response changes based on two aspects i.e. the operating point of the power system and the location/type of the fault. In this paper, a clustering algorithm is proposed to identify the clusters of the ‘external system’ models for different states of the power system, and a DNN is trained for each cluster. The clustering algorithm is based on the pre-fault steady-state power flow from the ‘external system’ to the ‘study system’, which is a reflection of changes in the system state. Mean Absolute Error (MAE) between responses of the actual fault and the predicted fault is used to evaluate the accuracy of the identified system. The proposed technique is easy to implement and requires a lesser number of DNN models compared to identifying an individual DNN model for each operating point to represent the ‘external system’.

Neural transfer learning for assigning diagnosis codes to EMRs

With the rapid development of the Internet of Things (IoT) and communication technology, deep neural network (DNN) applications, such as medical imaging, speech transcription, handwritten text recognition, have been widely used in IoT devices. However, due to resource constraints on these devices, e.g., limited memory capacity, weak computing capacity, and low battery capacity, IoT devices cannot support complicated DNN operation effectively and, thus, fail to fulfill the requirements of Quality of Service of mobile users. One promising approach is to migrate the DNN model to a remote cloud server to reduce the computing burden on IoT devices. Unfortunately, it still suffers from high delay and low bandwidth when communicating with cloud servers. Although the transmission delay of the edge server is low, its computing capacity lacks scalability and elasticity.To make matters worse, in the real world, the wireless connection between IoT devices and the cloud is intermittent, which can cause offloading failures during large-scale DNN data transmission. In this article, we describe a DNN model migration framework to overcome the abovementioned challenges, which consists of three parts: DNN model preprocessing, partition-offloading plan, and partition-uploading plan. Accordingly, we introduce the operation of the DNN migration and the available methods for each part. In addition, we improve the DNN partition-uploading plan in a multiuser edge-cloud collaborative computing environment. Finally, we highlight the important challenges of achieving more efficient DNN migration and point out the unresolved issues of DNN migration, which may shed light on future research directions.

Sedimentary basins, integral to Earth's geological history and energy resource exploration, undergo complex changes driven by sedimentation, subsidence and geological processes. Gravity anomaly inversion is a crucial technique offering insights into subsurface structures and density variations. Our study addresses the challenge of complex subsurface structure assessment by leveraging deep neural networks to invert observed gravity anomalies. Optimization approaches traditionally incorporate known density distributions obtained from borehole data or geological logging for inverting basement depth in sedimentary basins using observed gravity anomalies. Our study explores the application of deep neural networks in accurate architectural assessment of sedimentary basins and demonstrates their significance in mineral and hydrocarbon exploration. Recent years have witnessed a surge in the use of machine learning in geophysics, with deep learning models playing a pivotal role. Integrating deep neural networks, such as the feedforward neural networks, has revolutionized subsurface density distribution and basement depth estimation. This study introduces a deep neural network specifically tailored for inverting observed gravity anomalies to estimate two‐dimensional basement relief topographies in sedimentary basins. To enhance computational efficiency, a one‐dimensional discrete cosine transform based discretization approach is employed. Synthetic data, generated using non‐Gaussian fractals, compensates for the scarcity of true datasets for training the deep neural network model. The algorithm's robustness is validated through noise introduction with comparisons against an efficient and traditional global optimization‐based approach. Gravity anomalies of real sedimentary basins further validate the algorithm's efficacy, establishing it as a promising methodology for accurate and efficient subsurface imaging in geological exploration.

Recently, data-driven approaches motivated by modern deep learning have been applied to optical communications in place of traditional model-based counterparts. The application of deep neural networks (DNN) allows flexible statistical analysis of complicated fiber-optic systems without relying on any specific physical models. Due to the inherent nonlinearity in DNN, various equalizers based on DNN have shown significant potential to mitigate fiber nonlinearity. In this paper, we propose turbo equalization (TEQ) based on DNN as a new alternative framework to deal with nonlinear fiber impairments. The proposed DNN-TEQ is constructed with nested deep residual networks (ResNet) to train extrinsic likelihood given soft-information feedback from channel decoding. Through extrinsic information transfer (EXIT) analysis, we verify that our DNN-TEQ can accelerate decoding convergence to achieve a significant gain in achievable throughput by 0.61 b/s/Hz. We also demonstrate that optimizing irregular low-density parity-check (LDPC) codes based on the EXIT chart of the DNN-TEQ can improve achievable rates by up to 0.12 b/s/Hz.

The rapid advancements and widespread application of deep neural networks (DNNs), coupled with their reliance on sensitive and private data, have sparked growing concerns regarding data privacy and the ''right to be forgotten''. To address these concerns, machine unlearning has been proposed to efficiently eliminate the influence of specific training data from trained DNNs. However, existing machine unlearning methods struggle with the large number of parameters in trained DNNs, which lead to slow execution and high memory consumption, making them impractical for large-scale models. In this paper, we shift our focus to the small set of weights in the final classification layer of DNNs, which are defined as as ''prototypes'' for different classes. Our key observation is that the prototype associated with the unlearned training data undergoes a significant shift, whereas prototypes of unrelated classes exhibit only minor changes when comparing the prototypes of original and retrained models. Based on this observation, we propose a novel machine unlearning approach that efficiently achieves machine unlearning by directly adjusting the prototypes of DNNs. We first introduce Naive Prototype Surgery (Naive PS), a fast and simplified method that uses a closed-form solution to approximate unlearning effect by directly adjusting the prototype associated with the unlearned data. Next, we propose Prototype Surgery (PS), which incorporates soft label information to fine-tune the prototypes of all classes, to achieve a more effective unlearning. Both methods achieve data unlearning by only modifying the prototypes in the DNNs, thus avoiding the challenges posed by the large number of model parameters. Extensive experiments on four datasets demonstrate that our methods significantly accelerate the unlearning process while achieving comparable results to five existing methods in terms of both unlearning performance and privacy guarantee.