VDNN: virtualized deep neural networks for scalable, memory-efficient neural network design

Objective: Congestive heart failure is a problem affecting millions of Americans. A continuous, non-invasive, telemonitoring device that can accurately monitor cardiac metrics could greatly help this population, reducing unnecessary hospitalizations and cost. Approach: Machine learning (ML) algorithms trained on electrical-impedance tomography (EIT) data are presented for portable cardiac monitoring. The approach was validated on a simulated thorax and a measured tank experiment. A highly detailed 4D chest model (finite element method mesh and conductivity profiles) was developed utilizing the 4D XCAT phantom to provide realistic data. The ML algorithms were trained using databases that assumed the presence of poorly contacting electrodes without any assumptions of knowing which electrodes would be bad in the experiment. The trained ML algorithms were compared to EIT evaluated with and without removing bad electrodes. Main results: A regression support vector machine and a deep neural network (DNN) were found to be the most accurate and robust to poorly contacting electrodes while not needing to know which electrodes were in poor contact in the simulated and measured experiments, respectively. Significance: Although the ML algorithms are not always better than EIT (with bad electrodes removed), the comparable results without needing a priori knowledge of which electrodes are bad is seen as a very promising feature. An evaluation of computational costs showed that the DNN required comparable computational power to the other methods while requiring less memory, which could make the DNNs an attractive algorithm for a low-power, portable system. This work represents an important validation of the method using measured data, and model development, which is needed to apply this method on real clinical data. Additionally, the developed 4D simulated thorax model could be an important tool within the EIT community.

Water is a big challenge not only in India but in many countries of the world. Machine learning and forecasting model has been employed towards water demand and ground water level prediction. But in terms of water scarcity, much less work has been carried out by employing machine learning algorithms like 'artificial neural network' (ANN) and 'grey forecasting' model for forecasting water scarcity and none has focused on historical data like water availability, water consumption for a particular area and stress value for predicting water scarcity. So accordingly, we here have developed a water scarcity prediction system based on historical data by employing 'deep neural networks' which is an advanced form of 'artificial neural networks'. We have also compared 'deep neural network' with existing machine learning algorithms such as support vector machine (SVM), logistic regression and Naive Bayes. From the analysis of algorithms based on dataset, deep neural networks have been found as the best prediction model for water scarcity.

Graphics processing units (GPUs) have been widely adopted to accelerate the training of deep neural networks (DNNs). Although the computational performance of GPUs has been improving steadily, the memory size of modern GPUs is still quite limited, which restricts the sizes of DNNs that can be trained on GPUs, and hence raises serious challenges. This paper introduces a framework, referred to as moDNN (memory optimal DNN training on GPUs), to optimize the memory usage in DNN training. moDNN supports automatic tuning of DNN training code to match any given memory budget (not smaller than the theoretical lower bound). By taking full advantage of overlapping computations and data transfers, we develop new heuristics to judiciously schedule data offloading and prefetching transfers, together with convolution algorithm selection, to optimize memory usage. We further devise a new sub-batch size selection method which also greatly reduces memory usage. moDNN can save memory usage up to 59×, compared with an ideal case which assumes that the GPU memory is sufficient to hold all data. When executing moDNN on a GPU with 12 GB memory, the training time is increased by only 3 percent, which is much shorter than that incurred by the best known approach, vDNN. Furthermore, we propose an optimization strategy for moDNN on multiple GPUs again by utilizing the idea of overlapping data transfers and GPU computations. The results show that 3.7× speedup is attained on four GPUs.

Summary A meticulous interpretation of steady-state or unsteady-state relative permeability (Kr) experimental data is required to determine a complete set of Kr curves. In this work, different machine learning (ML) models were developed to assist in a faster estimation of these curves from steady-state drainage coreflooding experimental runs. These ML algorithms include gradient boosting (GB), random forest (RF), extreme gradient boosting (XGB), and deep neural network (DNN) with a main focus on and comparison of the two latter algorithms (XGB and DNN). Based on existing mathematical models, a leading-edge framework was developed where a large database of Kr and capillary pressure (Pc) curves were generated. This database was used to perform thousands of coreflood simulation runs representing oil-water drainage steady-state experiments. The results obtained from these simulation runs, mainly pressure drop along with other conventional core analysis data, were used to estimate analytical Kr curves based on Darcy’s law. These analytically estimated Kr curves along with the previously generated Pc curves were fed as features into the ML model. The entire data set was split into 80% for training and 20% for testing. The k-fold cross-validation technique was applied to increase the model’s accuracy by splitting 80% of the training data into 10 folds. In this manner, for each of the 10 experiments, nine folds were used for training and the remaining fold was used for model validation. Once the model was trained and validated, it was subjected to blind testing on the remaining 20% of the data set. The ML model learns to capture fluid flow behavior inside the core from the training data set. In terms of applicability of these ML models, two sets of experimental data were needed as input; the first was the analytically estimated Kr curves from the steady-state drainage coreflooding experiments, while the other was the Pc curves estimated from centrifuge or mercury injection capillary pressure (MICP) measurements. The trained/tested model was then able to estimate Kr curves based on the experimental results fed as input. Furthermore, to test the performance of the ML model when only one set of experimental data is available to an end user, a recurrent neural network (RNN) algorithm was trained/tested to predict Kr curves in the absence of Pc curves as an input. The performance of the three developed models (XGB, DNN, and RNN) was assessed using the values of the coefficient of determination (R2) along with the loss calculated during training/validation of the model. The respective crossplots along with comparisons of ground truth vs. artificial intelligence (AI)-predicted curves indicated that the model is capable of making accurate predictions with an error percentage between 0.2% and 0.6% on history-matching experimental data for all three tested ML techniques. This implies that the AI-based model exhibits better efficiency and reliability in determining Kr curves when compared to conventional methods. The developed ML models by no means replace the need to conduct drainage coreflooding or centrifuge experiments but act as an alternative to existing commercial platforms that are used to interpret experimental data to predict Kr curves. The two main advantages of the developed ML models are their capability of predicting Kr curves within a matter of a few minutes as well as with limited intervention from the end user. The results also include a comparison between classical ML approaches, shallow neural networks, and DNNs in terms of accuracy in predicting the final Kr curves. The research presented here is an extension of the state-of-the-art framework proposed by Mathew et al. (2021). However, the two main aspects of the current study are the application of deep learning for the prediction of Kr curves and the application of feature engineering. The latter not only reduces the training/testing time for the ML models but also enables the end user to obtain the final predictions with the least set of experimental data. The various models discussed in this research work currently focus on the prediction of Kr curves for drainage steady-state experiments; however, the work can be extended to capture the imbibition cycle as well.

Cardiovascular diseases (CVDs) have been identified as the leading cause of mortality worldwide. Electrocardiogram (ECG) is a fundamental diagnostic tool used for the diagnosis and detection of these diseases. The new technological tools can help enhance the effectiveness of ECGs. Machine learning (ML) is widely acknowledged as a highly effective approach in the realm of computer-aided diagnostics. This article presents a review of the effectiveness of ML algorithms and deep-learning algorithms in diagnosing, identifying, and classifying CVDs using ECG data. The review identified relevant studies published in the 5 major databases: PubMed, Web of Science (WoS), Scopus, Springer, and IEEE Xplore; between 2021 and 2023, a total of 30 were chosen for the comprehensive quantitative and qualitative. The study demonstrated that different datasets are available online with data related to CVDs. The various ML techniques are employed for the purpose of classification. Based on our investigation, it has been observed that deep learning-based neural network algorithms, such as convolutional neural networks and deep neural networks, have demonstrated superior performance in the detection of entire record data. Furthermore, deep learning showcases its efficacy even when confronted with a scarcity of data. ML approaches utilizing ECG data exhibit a notable proficiency in the realm of diagnosis, hence holding the potential to mitigate the occurrence of disease-related consequences at advanced stages.

Background: MRI forms an imperative part of the diagnostic and treatment protocol for primary brain tumors and metastasis. Though conventional T1W MRI forms the basis for diagnosis at present, it faces several limitations. Machine learning (ML) algorithms require less expertise and provide better diagnostic accuracy. Methods: A systematic review of PubMed, Google Scholar, and Cochrane databases along with registries through 1980-2021 was done. Original articles in English evaluating Conventional MRI or ML algorithms. Data was extracted by 2 reviewers and meta-analysis was performed using bivariate regression model. Results: The study protocol was registered under PROSPERO. Twelve studies with 1247 participants were included for systematic analysis and three studies for meta-analysis. ML algorithms had better aggregate sensitivity and specificity (80%, 83.14%) than Conventional MRI (81.84%, 74.78%).The pooled sensitivity, specificity, DOR for the studies were 0.926 (95% CI, 0.840-0.926), 0.991 (95% CI, 0.955-0.998) and 1446.946 (312.634-6692.646) with AUC=0.904 under HSROC. On subgroup analysis, MRS and Random Forest Model had highest sensitivity and specificity (100%,100%;100%,100%), DSC MRI and Deep Neural Network had highest AUC (0.98,0.986). Conclusions: ML algorithm has superior diagnostic performance and faster diagnostic capability once trained than conventional imaging for brain tumors. It has immense potential to be the standard of care in the future.

The paper offers a design of an optical non-orthogonal multiple access (O-NOMA) system with Rayleigh fading channels by analysing the bit error rate (BER) and power spectral density (PSD) using machine learning (ML)-based and traditional signal detection methods. The primary focus is on various QAM modulation orders, such as 32-QAM, 64-QAM, 256-QAM, and 512-QAM. The research compares the performance of different detectors including deep neural networks (DNN), convolutional neural networks (CNN), recurrent neural networks (RNN), and conventional techniques. Among these, the DNN always yields better BER performance up to 10−5 BER at SNR values 12 dB lower than conventional techniques. For 32-QAM, in particular, the DNN provides a gain of 10.3 dB compared to conventional techniques, while similar gains are realized for higher QAM orders. PSD analysis also shows that the DNN is compact in terms of its spectral footprint, achieving out-of-band emissions as low as −690 dB, essential for reducing interference and maximizing spectral efficiency. Such results highlight the promise of using ML-based detection – particularly DNN – as an efficient and effective method for future, high-capacity optical NOMA systems functioning over a wide range of modulation schemes.

One of the most important threats to today’s civilization is terrorism. Terrorism not only disturbs the law and order situations in a society but also affects the quality of lives of humans and makes them suppressed physically and emotionally and deprives them of enjoying life. The more the civilizations have advanced, the more the people are working towards exploring different mechanisms to protect the mankind from terrorism. Different techniques have been used as counterterrorism to protect the lives of individuals in society and to improve the quality of life in general. Machine learning methods have been recently explored to develop techniques for counterterrorism based on artificial intelligence (AI). Since deep learning has recently gained more popularity in machine learning domain, in this paper, these techniques are explored to understand the behavior of terrorist activities. Five different models based on deep neural network (DNN) are created to understand the behavior of terrorist activities such as is the attack going to be successful or not? Or whether the attack is going to be suicide or not? Or what type of weapon is going to be used in the attack? Or what type of attack is going to be carried out? Or what region is going to be attacked? The models are implemented in single-layer neural network (NN), five-layer DNN, and three traditional machine learning algorithms, i.e., logistic regression, SVM, and Naïve Bayes. The performance of the DNN is compared with NN and the three machine learning algorithms, and it is demonstrated that the performance in DNN is more than 95% in terms of accuracy, precision, recall, and F1-Score, while ANN and traditional machine learning algorithms have achieved a maximum of 83% accuracy. This concludes that DNN is a suitable model to be used for predicting the behavior of terrorist activities. Our experiments also demonstrate that the dataset for terrorist activities is big data; therefore, a DNN is a suitable model to process big data and understand the underlying patterns in the dataset.

PurposeChronic diseases are considered as one of the serious concerns and threats to public health across the globe. Diseases such as chronic diabetes mellitus (CDM), cardio vasculardisease (CVD) and chronic kidney disease (CKD) are major chronic diseases responsible for millions of death. Each of these diseases is considered as a risk factor for the other two diseases. Therefore, noteworthy attention is being paid to reduce the risk of these diseases. A gigantic amount of medical data is generated in digital form from smart healthcare appliances in the current era. Although numerous machine learning (ML) algorithms are proposed for the early prediction of chronic diseases, these algorithmic models are neither generalized nor adaptive when the model is imposed on new disease datasets. Hence, these algorithms have to process a huge amount of disease data iteratively until the model converges. This limitation may make it difficult for ML models to fit and produce imprecise results. A single algorithm may not yield accurate results. Nonetheless, an ensemble of classifiers built from multiple models, that works based on a voting principle has been successfully applied to solve many classification tasks. The purpose of this paper is to make early prediction of chronic diseases using hybrid generative regression based deep intelligence network (HGRDIN) model.Design/methodology/approachIn the proposed paper generative regression (GR) model is used in combination with deep neural network (DNN) for the early prediction of chronic disease. The GR model will obtain prior knowledge about the labelled data by analyzing the correlation between features and class labels. Hence, the weight assignment process of DNN is influenced by the relationship between attributes rather than random assignment. The knowledge obtained through these processes is passed as input to the DNN network for further prediction. Since the inference about the input data instances is drawn at the DNN through the GR model, the model is named as hybrid generative regression-based deep intelligence network (HGRDIN).FindingsThe credibility of the implemented approach is rigorously validated using various parameters such as accuracy, precision, recall, F score and area under the curve (AUC) score. During the training phase, the proposed algorithm is constantly regularized using the elastic net regularization technique and also hyper-tuned using the various parameters such as momentum and learning rate to minimize the misprediction rate. The experimental results illustrate that the proposed approach predicted the chronic disease with a minimal error by avoiding the possible overfitting and local minima problems. The result obtained with the proposed approach is also compared with the various traditional approaches.Research limitations/implicationsUsually, the diagnostic data are multi-dimension in nature where the performance of the ML algorithm will degrade due to the data overfitting, curse of dimensionality issues. The result obtained through the experiment has achieved an average accuracy of 95%. Hence, analysis can be made further to improve predictive accuracy by overcoming the curse of dimensionality issues.Practical implicationsThe proposed ML model can mimic the behavior of the doctor's brain. These algorithms have the capability to replace clinical tasks. The accurate result obtained through the innovative algorithms can free the physician from the mundane care and practices so that the physician can focus more on the complex issues.Social implicationsUtilizing the proposed predictive model at the decision-making level for the early prediction of the disease is considered as a promising change towards the healthcare sector. The global burden of chronic disease can be reduced at an exceptional level through these approaches.Originality/valueIn the proposed HGRDIN model, the concept of transfer learning approach is used where the knowledge acquired through the GR process is applied on DNN that identified the possible relationship between the dependent and independent feature variables by mapping the chronic data instances to its corresponding target class before it is being passed as input to the DNN network. Hence, the result of the experiments illustrated that the proposed approach obtained superior performance in terms of various validation parameters than the existing conventional techniques.

With the advancement of machine learning methods, audio sentiment analysis has become an active research area in recent years. For example, business organizations are interested in persuasion tactics from vocal cues and acoustic measures in speech. A typical approach is to find a set of acoustic features from audio data that can indicate or predict a customer's attitude, opinion, or emotion state. For audio signals, acoustic features have been widely used in many machine learning applications, such as music classification, language recognition, emotion recognition, and so on. For emotion recognition, previous work shows that pitch and speech rate features are important features. This thesis work focuses on determining sentiment from call center audio records, each containing a conversation between a sales representative and a customer. The sentiment of an audio record is considered positive if the conversation ended with an appointment being made, and is negative otherwise. In this project, a data processing and machine learning pipeline for this problem has been developed. It consists of three major steps: 1) an audio record is split into segments by speaker turns; 2) acoustic features are extracted from each segment; and 3) classification models are trained on the acoustic features to predict sentiment. Different set of features have been used and different machine learning methods, including classical machine learning algorithms and deep neural networks, have been implemented in the pipeline. In our deep neural network method, the feature vectors of audio segments are stacked in temporal order into a feature matrix, which is fed into deep convolution neural networks as input. Experimental results based on real data shows that acoustic features, such as Mel frequency cepstral coefficients, timbre and Chroma features, are good indicators for sentiment. Temporal information in an audio record can be captured by deep convolutional neural networks for improved prediction accuracy.

The probability of network attacks is increasing daily due to the continuous development of tools and techniques that bypass the firewall and other network security boundaries. This motivates the researcher towards the upgradation and advancement in adaptive Artificial Intelligence (AI) based intrusion detection systems (IDS). The traditional machine learning (ML) based IDS has its limitations due to noise and the unexplainable nature of the decision‐making that took place during the implementation of the ML algorithm for categorization of attacked or normal data packets that arrive in the network. To address these issues, this research proposes an ML algorithm with the understanding of decisions through Explainable artificial intelligence (XAI). The dataset used for experimentation is IoTID20, which extracts data in a real‐time Internet of Things (IoT) environment. This paper compares the accuracy results of different ML algorithms such as Random Forest (RF), Extreme Gradient Boosting (XGBoost), Multilayer Perceptron –Neural Networks (MLP‐NN), Deep Neural Networks (DN), and Decision Tree (DT). During the classification of attacks, the Random Forest shows an accuracy of 93.80%, the XGBoost shows an accuracy of 97.30%, the Decision Tree shows an accuracy of 99.99%, the MLP Classifier—Neural Network shows an accuracy of 95.90%, and the Deep Neural Network (DNN) shows an accuracy of 94.60%. These algorithms are also analyzed with Precision, Recall, and F1‐Score. The proposed method of incorporating XAI increases the automation process with high accuracy and explainability of the decision to categorize anomalous and normal packets in an IoT environment. This remarkable achievement provides better intuitions in IDS with good protection in resistance to novel and unknown attacks.

Maize kernel traits such as kernel length, kernel width, and kernel number determine the total kernel weight and, consequently, maize yield. Therefore, the measurement of kernel traits is important for maize breeding and the evaluation of maize yield. There are a few methods that allow the extraction of ear and kernel features through image processing. We evaluated the potential of deep convolutional neural networks and binary machine learning (ML) algorithms (logistic regression (LR), support vector machine (SVM), AdaBoost (ADB), Classification tree (CART), and the K-Neighbor (kNN)) for accurate maize kernel abortion detection and classification. The algorithms were trained using 75% of 66 total images, and the remaining 25% was used for testing their performance. Confusion matrix, classification accuracy, and precision were the major metrics in evaluating the performance of the algorithms. The SVM and LR algorithms were highly accurate and precise (100%) under all the abortion statuses, while the remaining algorithms had a performance greater than 95%. Deep convolutional neural networks were further evaluated using different activation and optimization techniques. The best performance (100% accuracy) was reached using the rectifier linear unit (ReLu) activation procedure and the Adam optimization technique. Maize ear with abortion were accurately detected by all tested algorithms with minimum training and testing time compared to ear without abortion. The findings suggest that deep convolutional neural networks can be used to detect the maize ear abortion status supplemented with the binary machine learning algorithms in maize breading programs. By using a convolution neural network (CNN) method, more data (big data) can be collected and processed for hundreds of maize ears, accelerating the phenotyping process.

to meet the extensive computational load presented by the rapidly growing Machine Learning (ML) and Artificial Intelligence (AI) algorithms such as Deep Neural Networks (DNNs) and Convolutional Neural Networks (CNNs). In order to obtain hardware solutions to meet the low-latency and high-throughput computational demands from these algorithms, Non-Von Neumann computing architectures such as In-memory Computing (IMC)/ Processing-in-memory (PIM) are being extensively researched and experimented with. In this survey paper, we analyze and review pioneer IMC/PIM works designed to accelerate ML algorithms such as DNNs and CNNs. We investigate different architectural aspects and dimensions of these works and provide our comparative evaluations. Furthermore, we discuss challenges and limitations in IMC research and also present feasible directions based on our observations and insight.

We have seen in the past decade a sharp increase in the extent that companies use data to optimize their businesses. Variously called the `Big Data' or `Data Science' revolution, this has been characterized by massive amounts of data, including unstructured and nontraditional data like text and images, and the use of fast and flexible Machine Learning (ML) algorithms in analysis. With recent improvements in Deep Neural Networks (DNNs) and related methods, application of high-performance ML algorithms has become more automatic and robust to different data scenarios. That has led to the rapid rise of an Artificial Intelligence (AI) that works by combining many ML algorithms together – each targeting a straightforward prediction task – to solve complex problems. We will define a framework for thinking about the ingredients of this new ML-driven AI. Having an understanding of the pieces that make up these systems and how they fit together is important for those who will be building businesses around this technology. Those studying the economics of AI can use these definitions to remove ambiguity from the conversation on AI's projected productivity impacts and data requirements. Finally, this framework should help clarify the role for AI in the practice of modern business analytics and economic measurement.

Automated image classification is an essential task of the computer vision field. The tagging of images into a set of predefined groups is referred to as image classification. The implementation of computer vision to automate image classification would be beneficial because manual image evaluation and identification can be time-consuming, particularly when there are many images of different classes. Deep learning approaches are proven to overperform existing machine learning techniques in many fields in recent years, and computer vision is one of the most notable examples. The very deep neural network (VDCNN) is a powerful deep learning model for image classification, and this paper examines it briefly using MNIST handwritten digit dataset. This dataset is used to prove the efficacy of very deep neural networks over other deep learning models. The proposed study aims to comprehend the very deep neural network architecture used to accomplish a handwritten digit recognition task. The feasibility of the proposed model is evaluated using mean accuracy, validation accuracy, and standard deviation. The study results of the very deep neural network model are compared to a convolutional neural network and convolutional neural network with batch normalization. According to the results of the comparison study, very deep neural networks achieve high accuracy of 99.1% for a handwritten dataset. The outcome of the proposed work is used to interpret how well a very deep neural network performs when compared to the other two models of deep neural networks. This proposed architecture may be used to automate the classification of handwritten digits dataset.