Quantile-based interpretable neural network models: Mortality forecasting and actuarial simulations

Cause-of-death mortality forecasting, a key topic in public health and actuarial science, is a challenging task due to the difficulty of modeling that accounts for dependencies among causes of death. While several cause-of-death mortality models have been proposed to address this difficulty, little attention has been paid to improving their predictive performance. Recently, purely data-driven approaches using tensor decomposition methods have been introduced to cause-of-death mortality modeling, demonstrating strong out-of-sample predictive performance compared to existing models. However, these methods have difficulties in the interpretability of multi-rank tensor components to achieve strong predictive performance. In response, we propose a novel tensor-based cause-of-death mortality model by replacing the tensor decomposition with a convolutional autoencoder with a one-dimensional latent layer that provides a Lee-Carter-like time-series factor; the model also provides the age sensitivity of cause-specific log mortality to the time-series factor. Due to the representational capability of the neural network, our model achieves better predictive performance compared to the existing tensor decomposition-based models, despite the simplified latent layer for model interpretability.

Automatic creation of polarity dictionaries is an important issue, as explanations of prediction models are often required in the financial industry. This paper proposes a novel method of developing an interpretable and predictable neural network model. The neural network model we built can extract polarity scores of concepts from documents. Furthermore, we can detect pairwise interactions between concepts, and create polarity concept dictionaries using our neural network model. The model was built using vector representations of words and polarity scores for about 100 words provided by financial professionals, and we obtained about a hundred times more polarity scores for unknown words through backpropagation. First, we analyze the properties of our method from a theoretical point of view. We then confirm its capabilities by conducting simulations of assigning polarity scores to unknown words and detecting interactions using artificial data. We subsequently estimate sentiment tags using real financial textual datasets. Compared with other conventional methods, the proposed approach can forecast sentiments with higher F1 scores. Finally, we develop a polarity concept dictionary based on Yahoo! Finance board textual data.

Theory-Guided Machine Learning Finds Geometric Structure-Property Relationships for Chemisorption on Subsurface Alloys

AI modeling CDM can improve the quality and efficiency of clinical practice or provide secondary opinion consultations for patients with limited medical resources to address healthcare disparities. In this study, we developed an interpretable AI model to select radiotherapy treatment options, that is, whole-brain radiation therapy (WBRT) versus stereotactic radiosurgery (SRS), for patients with brain metastases. A total of 232 patients with brain metastases treated by radiation therapy from 2018 to 2023 were obtained. CT/MR images with contoured target lesions and organs-at-risk (OARs) as well as non-image-based clinical parameters were extracted and digitized as inputs to the model. These parameters included (1) tumor size, shape, location, and proximity of lesions to OARs; (2) age; (3) the number of brain metastases; (4) Eastern Cooperative Oncology Group (ECOG) performance status; (5) presence of neurologic symptoms; (6) if surgery was performed (either pre/post-op RT); (7) newly diagnosed cancer with brain metastases (de-novo) versus re-treatment (either local or distant in the brain); (8) primary cancer histology; (9) presence of extracranial metastases; (10) extent of extracranial disease (progression vs. stable); and (11) receipt of systemic therapy. One vanilla and two interpretable 3D convolutional neural networks (CNN) models were developed. The vanilla one-path model (VM-1) uses only images as input, while the two interpretable models use both images and clinical parameters as inputs with two (IM-2) and 11 (IM-11) independent paths, respectively. This novel design allowed the model to calculate a class activation score for each input to interpret its relative weighting and importance in decision-making. The actual radiotherapy treatment (WBRT or SRS) used for the patients was used as ground truth for model training. The model performance was assessed by Stratified-10-fold cross-validation, with each fold consisting of selected 184 training, 24 validation, and 24 testing subjects. A total of 232 brain metastases patients treated by WBRT or SRS were evaluated, including 80 WBRT and 152 SRS patients. Based on the images alone, the VM-1 model prescribed correctly for 143 (94%) SRS and 67 (84%) WBRT cases. Based on both images and clinical parameters, the IM-2 model prescribed correctly for 149 (98%) SRS and 74 (93%) WBRT cases. IM-11 provided the most interpretability with a relative weighting for each input as follows: CT image (59.5%), ECOG performance status (7.5%), re-treatment (5%), extracranial metastases (1.5%), number of brain metastases (9.5%), neurologic symptoms (3%), pre/post-surgery (2%), primary cancer histology (2%), age (1%), progressive extracranial disease (6%), and receipt of systemic therapy (4.5%), reflecting the importance of all these inputs in clinical decision-making. Interpretable CNN models were successfully developed to use CT/MR images and non-image-based clinical parameters to predict the treatment selection between WBRT and SRS for brain metastases patients. The interpretability makes the model more transparent, carrying profound importance for the prospective integration of these models into routine clinical practice, particularly for informing real-time clinical decision-making.

The rapid proliferation of artificial intelligence (AI) technologies across various industries and decision-making processes has undeniably transformed the way of approaching complex problems and tasks. AI systems have proven their prowess in areas such as healthcare, finance, and autonomous systems, revolutionizing how decisions are made. Nevertheless, this proliferation of AI has raised critical concerns regarding the transparency, accountability, and fairness of these systems, as many of the state-of-the-art AI models often resemble complex black boxes. These intricate models, particularly deep learning neural networks, harbor non-linear relationships that are difficult for human users to decipher, thereby raising concerns about bias, fairness, and overall trustworthiness in AI-driven decisions. The urgency of this issue is underscored by the realization that AI should not merely be accurate; it should also be interpretable. Explainable AI (XAI) has emerged as a vital field of research, emphasizing the development of models and techniques that render AI systems comprehensible and transparent in their decision-making processes. This paper investigates into the relevance and significance of XAI across various domains, including healthcare, finance, and autonomous systems, where the ability to understand the rationale behind AI decisions is paramount. In healthcare, where AI assists in diagnosis and treatment, the interpretability of AI models is crucial for clinicians to make informed decisions. In finance, applications like credit scoring and investment analysis demand transparent AI to ensure fairness and accountability. In the realm of autonomous systems, transparency is indispensable to guarantee safety and compliance with regulations. Moreover, government agencies in areas such as law enforcement and social services require interpretable AI to maintain ethical standards and accountability. This paper also highlights the diverse array of research efforts in the XAI domain, spanning from model-specific interpretability methods to more general approaches aimed at unveiling complex AI models. Interpretable models like decision trees and rule-based systems have gained attention for their inherent transparency, while integrating explanation layers into deep neural networks strives to balance accuracy with interpretability. The study emphasizes the significance of this burgeoning field in bridging the gap between AI's advanced capabilities and human users' need for comprehensible AI systems. It seeks to contribute to this field by exploring the design, development, and practical applications of interpretable AI models and techniques, with the ultimate goal of enhancing the trust and understanding of AI-driven decisions.

Learning What Makes Catalysts Good

Interpretable machine learning models for gene expression datasets are important for understanding the decision-making process of a classifier and gaining insights on the underlying molecular processes of genetic conditions. Interpretable models can potentially support early diagnosis before full disease manifestation. This is particularly important yet, challenging for mental health. We hypothesise this is due to extreme heterogeneity issues which may be overcome and explained by personalised modelling techniques. Thus far, most machine learning methods applied to gene expression datasets, including deep neural networks, lack personalised interpretability. This paper proposes a new methodology named personalised constrained neuro fuzzy inference (PCNFI) for learning personalised rules from high dimensional datasets which are structurally and semantically interpretable. Case studies on two mental health related datasets (schizophrenia and bipolar disorders) have shown that the relatively short and simple personalised fuzzy rules provided enhanced interpretability as well as better classification performance compared to other commonly used machine learning methods. Performance test on a cancer dataset also showed that PCNFI matches previous benchmarks. Insights from our approach also indicated the importance of two genes (ATRX and TSPAN2) as possible biomarkers for early differentiation of ultra-high risk, bipolar and healthy individuals. These genes are linked to cognitive ability and impulsive behaviour. Our findings suggest a significant starting point for further research into the biological role of cognitive and impulsivity-related differences. With potential applications across bio-medical research, the proposed PCNFI method is promising for diagnosis, prognosis, and the design of personalised treatment plans for better outcomes in the future.

1. In this study, neural network (NN) and response surface (RS) models were developed to investigate the response [average daily gain (ADG) and feed efficiency (FE)] of young broiler chickens to dietary protein and lysine. For this purpose, data on their responses to dietary protein and lysine were extracted from the literature and separate NN and RS models were constructed. 2. Comparison between the NN and RS models revealed higher accuracy of prediction with the NN models compared to the RS models. In terms of R 2 values, the NN models developed for both ADG (R 2 = 0.923) and FE (R 2 = 0.904) were far superior to the RS models (R 2 for ADG = 0.511; R 2 for FE = 0.67). This suggests that the NN models can serve as an alternative option to conventional regression approaches including use of RS models. 3. Optimisation of the NN models developed for response to protein and lysine showed that diets containing 220.7 (g/kg of diet) protein and 12.85 (g/kg of diet) lysine maximise ADG, whereas maximum FE is achieved with diets containing 241.3 and 13.12 (g/kg) protein and lysine, respectively. Based on the optimisation results, optimal dietary protein and lysine concentrations for maximum FE in broiler chickens during the starting period are higher than for ADG.

Urbanization is a great challenge for modern societies, promising better access to economic opportunities, but widening socioeconomic inequalities. Accurately tracking this process as it unfolds has been challenging for traditional data collection methods, but remote sensing information offers an alternative way to gather a more complete view of these societal changes. By feeding neural networks with satellite images, the socioeconomic information associated with that area can be recovered. However, these models lack the ability to explain how visual features contained in a sample trigger a given prediction. Here, we close this gap by predicting socioeconomic status across France from aerial images and interpreting class activation mappings in terms of urban topology. We show that trained models disregard the spatial correlations existing between urban class and socioeconomic status to derive their predictions. These results pave the way to build more interpretable models, which may help to better track and understand urbanization and its consequences. Neural network models can predict the socioeconomic wealth of an area from aerial views, but fall short of explaining how visual features trigger a given prediction. The authors develop a pipeline for projecting class activation maps onto the underlying urban topology, to help interpret such predictions.

Hybrid modeling approaches for predicting COVID-19 mortality: A comparative study across USA, France, and India

The molecular regulatory network (MRN) within a cell determines cellular states and transitions between them. Thus, modeling of MRNs is crucial, but this usually requires extensive analysis of time-series measurements, which is extremely difficult to obtain from biological experiments. However, single-cell measurement data such as single-cell RNA-sequencing databases have recently provided a new insight into resolving this problem by ordering thousands of cells in pseudo-time according to their differential gene expressions. Neural network modeling can be employed by using temporal data as learning data. In contrast, Boolean network modeling of MRNs has a growing interest, as it is a parameter-free logical modeling and thereby robust to noisy data while still capturing essential dynamics of biological networks. In this study, we propose a Boolean feedforward neural network (FFN) modeling by combining neural network and Boolean network modeling approach to reconstruct a practical and useful MRN model from large temporal data. Furthermore, analyzing the reconstructed MRN model can enable us to identify control targets for potential cellular state conversion. Here, we show the usefulness of Boolean FFN modeling by demonstrating its applicability through a toy model and biological networks.

Background: Data mining (DM) is an approach used in extracting valuable information from environmental processes. This research depicts a DM approach used in extracting some information from influent and effluent wastewater characteristic data of a waste stabilization pond (WSP) in Birjand, a city in Eastern Iran. Methods: Multiple regression (MR) and neural network (NN) models were examined using influent characteristics (pH, Biochemical oxygen demand [BOD 5 ], temperature, chemical oxygen demand [COD], total suspended solids [TSS], total dissolved solid [TDS], electrical conductivity [EC] and turbidity) as the regression input vectors. Models were adjusted to input attributes, effluent BOD 5 (BODout) and COD (CODout). The models performances were estimated by 10-fold external cross-validation. An internal 5-fold cross-validation was also used for the training data set in NN model. The models were compared using regression error characteristic (REC) plot and other statistical measures such as relative absolute error (RAE). Sensitivity analysis was also applied to extract useful knowledge from NN model. Results: NN models (with RAE = 78.71 ± 1.16 for BODout and 83.67 ± 1.35 for CODout) and MR models (with RAE = 84.40% ± 1.07 for BODout and 88.07 ± 0.80 for CODout) indicate different performances and the former was better (P < 0.05) for the prediction of both effluent BOD 5 and COD parameters. For the prediction of CODout the NN model with hidden layer size (H) = 4 and decay factor = 0.75 ± 0.03 presented the best predictive results. For BODout the H and decay factor were found to be 4 and 0.73 ± 0.03, respectively. TDS was found as the most descriptive influent wastewater characteristics for the prediction of the WSP performance. The REC plots confirmed the NN model performance superiority for both BOD and COD effluent prediction. Conclusion: Modeling the performance of WSP systems using NN models along with sensitivity analysis can offer better understanding on exploring the most significant parameters for the prediction of system performance. The findings of this study could build the foundation for prospective work on the characterization of WSP operations and optimization of their performances with a view to conducting statistical approaches.

Abstract: The first part of the motivations behind the demand for explainable and interpretable models, emphasizing the ethical, legal, and practical implications of deploying black-box models in critical domains. The discussion extends to the societal impact of decisions made by these models and the importance of building trust among end-users. The second part explores the existing techniques and methodologies designed to enhance the explain ability and interpretability of ML models. From traditional linear models to complex deep neural networks, delve into methods. Special attention is given to recent advancements, including state-of-the-art attention mechanisms and layer-wise relevance propagation in neural networks. Search significance: The primary goal of any machine learning model is to demonstrate high ability by effectively capturing patterns, relationships, and trends within the training data and generalizing well to new, unseen data. A model with high ability is capable of making accurate predictions, which is crucial for its practical utility and effectiveness in various applications. Interpretability is essential for building trust and acceptance of machine learning models. An interpretable model allows stakeholders to comprehend why a particular prediction was made, which is crucial for model deployment, regulatory compliance, and ethical considerations. Method: the approach involving a weighted sum essentially involves performing multiplication, while subtraction is employed for addition and sorting. When considering candidate keywords, we’ve previously explored how they are generated and presented. The weighted sum of a fourdimensional feature constitutes the vector, which, however, is altered in the course of the process. The necessity for weights arises, as the four characteristics possess different parsing capabilities. This encompasses both keywords and keywords. The greater the ability to differentiate, the more effective the manual identification process can be. In reality, manually performing the identification of a weight vector for the domain proves to be excessively burdensome due to its complexity. Result: From the result Random Forest is in 1st rank whereas DNN 5 Layers is in lowest rank Keywords: machine learning; explain ability; interpretability; fairness; sensitivity; black-box

Computer-aided Prediction of the ZrO2 Nanoparticles' Effects on Tensile Strength and Percentage of Water Absorption of Concrete Specimens

Background: Nowadays, forecasting models based on nonparametric models have been developed in many branches of science such as mathematics and economics. However, the relatively complicated structure of these models has made them less practical in medical sciences. Objectives: In this study, we investigated the application of a nonparametric regression model to predict the psychological symptoms appearing six months after mild traumatic brain injury. We also made a comparison between the performance of nonparametric regression and artificial neural network models in predicting the psychological symptoms. Methods: In a six-month period during 2015 - 2017, information of 100 mild traumatic patients was included in a prospective cohort study. The data were then divided randomly into two groups of training (n = 50) and testing (n = 50) for 100 times. In the training group, the focus was on 100 artificial neural network and nonparametric regression models. However, in the testing group, a comparison was made between the values obtained using the two final models. To compare the models, the ROC curve and the accuracy rate were finally applied. Results: According to the obtained results, the nonparametric regression model showed an accuracy rate of 91.25% while the neural network model had an accuracy rate of 85.34%. In the experimental set for both neural network and nonparametric regression models, the area under the ROC curve appeared to be 81.51% and 85.73 %, respectively. Conclusions: The nonparametric regression model appeared to be more powerful than the neural network model in predicting psychological symptoms.