Deep Learning-Based Multifloor Indoor Tracking Scheme Using Smartphone Sensors

Electricity load forecasting is a crucial undertaking within all the deregulated markets globally. Among the research challenges on a global scale, the investigation of deep transfer learning (DTL) in the field of electricity load forecasting represents a fundamental effort that can inform artificial intelligence applications in general. In this paper, a comprehensive study is reported regarding day-ahead electricity load forecasting. For this purpose, three sequence-to-sequence (Seq2seq) deep learning (DL) models are used, namely the multilayer perceptron (MLP), the convolutional neural network (CNN) and the ensemble learning model (ELM), which consists of the weighted combination of the outputs of MLP and CNN models. Also, the study focuses on the development of different forecasting strategies based on DTL, emphasizing the way the datasets are trained and fine-tuned for higher forecasting accuracy. In order to implement the forecasting strategies using deep learning models, load datasets from three Greek islands, Rhodes, Lesvos, and Chios, are used. The main purpose is to apply DTL for day-ahead predictions (1–24 h) for each month of the year for the Chios dataset after training and fine-tuning the models using the datasets of the three islands in various combinations. Four DTL strategies are illustrated. In the first strategy (DTL Case 1), each of the three DL models is trained using only the Lesvos dataset, while fine-tuning is performed on the dataset of Chios island, in order to create day-ahead predictions for the Chios load. In the second strategy (DTL Case 2), data from both Lesvos and Rhodes concurrently are used for the DL model training period, and fine-tuning is performed on the data from Chios. The third DTL strategy (DTL Case 3) involves the training of the DL models using the Lesvos dataset, and the testing period is performed directly on the Chios dataset without fine-tuning. The fourth strategy is a multi-task deep learning (MTDL) approach, which has been extensively studied in recent years. In MTDL, the three DL models are trained simultaneously on all three datasets and the final predictions are made on the unknown part of the dataset of Chios. The results obtained demonstrate that DTL can be applied with high efficiency for day-ahead load forecasting. Specifically, DTL Case 1 and 2 outperformed MTDL in terms of load prediction accuracy. Regarding the DL models, all three exhibit very high prediction accuracy, especially in the two cases with fine-tuning. The ELM excels compared to the single models. More specifically, for conducting day-ahead predictions, it is concluded that the MLP model presents the best monthly forecasts with MAPE values of 6.24% and 6.01% for the first two cases, the CNN model presents the best monthly forecasts with MAPE values of 5.57% and 5.60%, respectively, and the ELM model achieves the best monthly forecasts with MAPE values of 5.29% and 5.31%, respectively, indicating the very high accuracy it can achieve.

This paper provides a hybrid approach between Wi-Fi and pedestrian dead reckoning (PDR) for the iPhones in indoor environments, and addresses the following two problems. First, since Apple Inc. no longer provides public information about currently-connected Wi-Fi (e.g., service set identifier, received signal strength and channel), Wi-Fi based pedestrian tracking apps that run on the iPhones are more restricted compared to other ones on the Android platforms. Second, even the PDR approach provides such a great way for self-localization, it suffers from accumulated errors of inertial sensors embedded in the smartphones. We propose a conversion function from a Wi-Fi status value to a proximity for the localization purpose. Then, a mobile iPhone collects mobility information from inertial measurements unit (IMU), inputs to the PDR, and combines with Wi-Fi proximity in order to perform accurate self-localization and tracking. Moreover, we improve the PDR by reducing drifting effects caused by the IMU biases. Experiment results show that the proposed scheme is effective and has low complexity, while bringing the benefits from smartphone IMU.

Human activity recognition is a very active research on pervasive computing and mobile health application. Many human activity systems based on inertial measurement unit (IMU) sensor data were proposed in the past few years. These systems mainly use IMU sensor placed on he torso and limbs to collect data and utilize supervised machine learning algorithms on sensor data. One main issue of these systems is that wearing multiple on-body IMU sensors may bring inconvenience to users' daily life. The other issue of these exiting methods is that an activity recognition model that is trained on a specific subject does not work well when being applied to predict another subject's activities since IMU activity data always carry information that is specific to the human subject who conducts the activities. In our work, inspired by the principle of domain adaption, we proposed a new deep-learning activity recognition model based on an adversarial network which can remove the subject-specific information within the IMU activity data and extract subject-independent features shared by the data collected on different subjects. We also for the first time use data collected from insole based IMU sensors on 8 participants for 5 common activities to build a new real world human activity dataset which can minimize the inconvenience for users to wear. We conducted experiments with our new real-world dataset. Results show that our subject independent activity recognition model outperforms state-of-art supervised learning techniques and eliminates the effects of individual differences between subjects successfully. The average recognition accuracy under the leave-one-out (L1O) condition achieves 99.0% which is higher than the performance of traditional human activity recognition system based on CNNs.

PurposeTo conduct a head-to-head comparison between deep learning (DL) and radiomics models across institutions for predicting microvascular invasion (MVI) in hepatocellular carcinoma (HCC) and to investigate the model robustness and generalizability through rigorous internal and external validation.MethodsThis retrospective study included 2304 preoperative images of 576 HCC lesions from two centers, with MVI status determined by postoperative histopathology. We developed DL and radiomics models for predicting the presence of MVI using B-mode ultrasound, contrast-enhanced ultrasound (CEUS) at the arterial, portal, and delayed phases, and a combined modality (B + CEUS). For radiomics, we constructed models with enlarged vs. original regions of interest (ROIs). A cross-validation approach was performed by training models on one center’s dataset and validating the other, and vice versa. This allowed assessment of the validity of different ultrasound modalities and the cross-center robustness of the models. The optimal model combined with alpha-fetoprotein (AFP) was also validated. The head-to-head comparison was based on the area under the receiver operating characteristic curve (AUC).ResultsThirteen DL models and 25 radiomics models using different ultrasound modalities were constructed and compared. B + CEUS was the optimal modality for both DL and radiomics models. The DL model achieved AUCs of 0.802–0.818 internally and 0.667–0.688 externally across the two centers, whereas radiomics achieved AUCs of 0.749–0.869 internally and 0.646–0.697 externally. The radiomics models showed overall improvement with enlarged ROIs (P < 0.05 for both CEUS and B + CEUS modalities). The DL models showed good cross-institutional robustness (P > 0.05 for all modalities, 1.6–2.1% differences in AUC for the optimal modality), whereas the radiomics models had relatively limited robustness across the two centers (12% drop-off in AUC for the optimal modality). Adding AFP improved the DL models (P < 0.05 externally) and well maintained the robustness, but did not benefit the radiomics model (P > 0.05).ConclusionCross-institutional validation indicated that DL demonstrated better robustness than radiomics for preoperative MVI prediction in patients with HCC, representing a promising solution to non-standardized ultrasound examination procedures.

Abstract. With the continuous improvement of the hardware level of the inertial measurement unit (IMU), indoor pedestrian dead reckoning (PDR) using an inertial device has been paid more and more attention. Typical PDR system position estimation is based on acceleration obtained from accelerometers to measure the step count, estimate step length and generate the position with the heading received from angular sensors (magnetometers and gyroscopes). Unfortunately, collected signals are very responsive to the alignment of sensor devices, built-in instrumental errors and distortions from the surrounding environment. In our work, a pedestrian positioning method using step detection based on a shoe-mounted inertial unit is arranged and put to the test, and the final results are analyzed. The extended Kalman filter (EKF) provides estimation of the errors which are acquired by the XSENS IMU sensor biases. The EKF is revised with acceleration and angular rate computations by the ZUPT (zero velocity update) and ZARU (zero angular rate update) algorithms. The step detection associated with these two solutions is the perfect choice to calculate the current position and distance walked and to estimate the IMU sensors' collected errors by using EKF. The test with a shoe-mounted IMU device was performed and analyzed in order to check the performance of the recommended method. The combined PDR final results were compared to GPS/Beidou postprocessing kinematic results (outdoor environment) and to a real route which was prepared and calculated for an indoor environment. After the comparison, the results show that the accuracy of the regular-speed walking under ZUPT and ZARU combination in the case of outdoor positioning did not go beyond 0.19 m (SD) and for indoor positioning accuracy did not exceed 0.22 m (SD). The authors are conscious that built-in drift errors coming from accelerometers and gyroscopes, as well as the final position obtained by XSENS IMU, are only stable for a short time period. Based on this consideration, our future work will be focused on supporting the methods presented with radio technologies (WiFi) or image-based solutions to correct all IMU imperfections.

COVID-19 is highly infectious and causes acute respiratory disease. Machine learning (ML) and deep learning (DL) models are vital in detecting disease from computerized chest tomography (CT) scans. The DL models outperformed the ML models. For COVID-19 detection from CT scan images, DL models are used as end-to-end models. Thus, the performance of the model is evaluated for the quality of the extracted feature and classification accuracy. There are four contributions included in this work. First, this research is motivated by studying the quality of the extracted feature from the DL by feeding these extracted to an ML model. In other words, we proposed comparing the end-to-end DL model performance against the approach of using DL for feature extraction and ML for the classification of COVID-19 CT scan images. Second, we proposed studying the effect of fusing extracted features from image descriptors, e.g., Scale-Invariant Feature Transform (SIFT), with extracted features from DL models. Third, we proposed a new Convolutional Neural Network (CNN) to be trained from scratch and then compared to the deep transfer learning on the same classification problem. Finally, we studied the performance gap between classic ML models against ensemble learning models. The proposed framework is evaluated using a CT dataset, where the obtained results are evaluated using five different metrics The obtained results revealed that using the proposed CNN model is better than using the well-known DL model for the purpose of feature extraction. Moreover, using a DL model for feature extraction and an ML model for the classification task achieved better results in comparison to using an end-to-end DL model for detecting COVID-19 CT scan images. Of note, the accuracy rate of the former method improved by using ensemble learning models instead of the classic ML models. The proposed method achieved the best accuracy rate of 99.39%.

Introduction: Magnetic resonance imaging (MRI) is the most common tool to examine glioblastoma. Preoperative MRI can be used to initial diagnosis, and surgery planning, while follow-up MRIs can be used to evaluate treatment responses, identify recurrency, and detect side effects. The follow-up MRIs are usually taken after the first-line therapy, such as maximal safe resection. In the past, radiologists manually segment the tumor regions from normal brain tissue on follow-up MRIs, which is time-consuming, error-prone, and challenging. Several deep-learning (DL) models have been developed utilizing preoperative images, but their performance has yet to be evaluated on follow-up MRIs. In this research, we built the largest follow-up MRI cohort (311 patients) to assess these DL models and their generalizability and performance on independent preoperative and follow-up images. We also made the first follow-up-based deep learning models for this specific task. Methods: All evaluation deep learning models (10 models) were trained by the Brain Tumor Segmentation challenge 2020 (BraTS’20) and evaluated by fifty pairs of preoperative and follow-up scans from our institution. The segmentation form our institution is evaluated by board certified radiologist. MRIs in the BraTS’20 dataset were all preoperative scans. After the evaluation, we randomly assigned 264 patients’ scans from our institution to the training dataset and 47 patients’ scans to the testing dataset. We compared three types of models in our follow-up deep learning model, including 1) UNet-3D, 2) UNet-3D+transfer-learning, and 3) UNet-3D+transfer-learning+baysian-learning. The benchmark for all models was the Dice similarity coefficient (DSC). DSC can measure the spatial overlap between model prediction and ground truth. The value of DSC is between zero to one. Zero means no overlap and one indicates complete overlap. Results: Our study demonstrates that the BraTS'20 trained models' performance decreased by 13.05% in independent preoperative MRI scans and 19.04% in follow-up MRI scans. The most significant mismatch regions were FLAIR hyperintense regions (3.68% drop in independent preoperative scans and 10.61% drop in independent follow-up scans) and Non-enhancing core (5.20% drop in independent preoperative scans and 11.99% drop in independent follow-up scans). Our best model can achieve the best DSC among three tumor regions compared to all evaluation models (FLAIR hyperintense regions: DSC 0.77 V.S. DSC 0.58; Enhancing tumor region: DSC 0.87 V.S. DSC 0.68; Non-enhancing tumor region: DSC 0.92 V.S. DSC 0.55). Conclusion: Maximal safe resection induced brain structure change, decreasing the performance of the preoperative-based DL model. Implementing a follow-up MRI-based segmentation model is essential to make accurate and generalizable results to address structural changes after maximal safe resection. Our follow-up DL model demonstrates the DSC score can be recovered. We commit to further developing the tool to assist radiologists in handling follow-up MRIs for glioblastoma patients. Citation Format: Kang Lin Hsieh, Tanjida Kabir, Luis Nunez-Rubiano, Yu-Chun Hsu, Yu Cai, Juan Rodriguez Quintero, Octavio Arevalo, Kangyi Zhao, Jackie Zhang, Jiguang Zhu Zhu, Roy Riascos, Xioaqian Jiang, Shayan Shams. A confident and operator-independent deep segmentation model to measure residual tumor volume in the follow-up MRIs for glioblastoma [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2023; Part 2 (Clinical Trials and Late-Breaking Research); 2023 Apr 14-19; Orlando, FL. Philadelphia (PA): AACR; Cancer Res 2023;83(8_Suppl):Abstract nr LB067.

Pedestrian dead reckoning (PDR) is a self-contained positioning technology and has been a significant research topic in recent years. Pedestrian-stride-length estimation is the core part of the PDR system and directly affects the performance of the PDR. The current stride-length-estimation method is difficult to adapt to changes in pedestrian walking speed, which leads to a rapid increase in the error of the PDR. In this paper, a new deep-learning model based on long short-term memory (LSTM) and Transformer, LT-StrideNet, is proposed to estimate pedestrian-stride length. Next, a shank-mounted PDR framework is built based on the proposed stride-length-estimation method. In the PDR framework, the detection of pedestrian stride is achieved by peak detection with a dynamic threshold. An extended Kalman filter (EKF) model is adopted to fuse the gyroscope, accelerometer, and magnetometer. The experimental results show that the proposed stride-length-estimation method can effectively adapt to changes in pedestrian walking speed, and our PDR framework has excellent positioning performance.

We developed a multimodal ultrasound (US) deep learning (DL) fusion model to automatically classify early fibrosis in patients with chronic kidney disease (CKD). This prospective study included patients with CKD who underwent continuous gray-scale US, superb microvascular imaging, and strain elastography from May to November 2022. According to the pathological tubular atrophy and interstitial fibrosis score, patients were divided into minimal and mild groups (affected area ≤10% and 11 − 25% of the total cortical volume, respectively). The dataset was divided into training (70%) and test (30%) sets. A DL model combining the features of the three US modes was developed to predict early fibrosis in patients with CKD. We compared these findings with the area under the receiver operating characteristic curve (AUC) of the clinical model by analyzing the receiver operating characteristic curve in the test set. The AUC of single-mode DL based on gray-scale US, superb microvascular imaging, and strain elastography was 0.682, 0.745, and 0.648, respectively, while that of the multimodal US DL model was 0.86. The accuracy, specificity, and sensitivity of the multimodal US DL model were 0.779, 0.767, and 0.796, respectively, and the negative and positive predictive values were 0.842 and 0.706, respectively. The AUC of the multimodal US DL model was significantly better than that of the single-mode DL and clinical models. The DL algorithm developed using multimodal US images can effectively predict early fibrosis in patients with CKD with significantly greater accuracy than single-mode DL or clinical models.

We aimed at comparing intratumoral and peritumoral deep learning, radiomics, and fusion models in predicting KRAS mutations in rectal cancer using endorectal ultrasound imaging. This study included 304 patients with rectal cancer from Fujian Medical University Union Hospital. The patients were randomly divided into a training group (213 patients) and a test group (91 patients) at a 7:3 ratio. Radiomics and deep learning models were established using primary tumor and peritumoral images. In the optimally performing regions-of-interest, two fusion strategies, a feature-based and a decision-based model, were employed to build the fusion models. The Shapley additive explanation (SHAP) method was used to evaluate the significance of features in the optimal radiomics, deep learning, and fusion models. The performance of each model was assessed using the area under the receiver operating characteristic curve (AUC) and decision curve analysis (DCA). In the test cohort, both the radiomics and deep learning models exhibited optimal performance with a 10-pixel patch extension, yielding AUC values of 0.824 and 0.856, respectively. The feature-based DLRexpand10_FB model attained the highest AUC (0.896) across all study sets. In addition, the DLRexpand10_FB model demonstrated excellent sensitivity, specificity, and DCA. SHAP analysis underscored the deep learning feature (DL_1) as the most significant factor in the hybrid model. The feature-based fusion model DLRexpand10_FB can be employed to predict KRAS gene mutations based on pretreatment endorectal ultrasound images of rectal cancer. The integration of peritumoral regions enhanced the predictive performance of both the radiomics and deep learning models.

Deep learning breast cancer risk models demonstrate improved accuracy compared with traditional risk models but have not been prospectively tested. We compared the accuracy of a deep learning risk score derived from the patient's prior mammogram to traditional risk scores to prospectively identify patients with cancer in a cohort due for screening. We collected data on 119 139 bilateral screening mammograms in 57 617 consecutive patients screened at 5 facilities between September 18, 2017, and February 1, 2021. Patient demographics were retrieved from electronic medical records, cancer outcomes determined through regional tumor registry linkage, and comparisons made across risk models using Wilcoxon and Pearson χ2 2-sided tests. Deep learning, Tyrer-Cuzick, and National Cancer Institute Breast Cancer Risk Assessment Tool (NCI BCRAT) risk models were compared with respect to performance metrics and area under the receiver operating characteristic curves. Cancers detected per thousand patients screened were higher in patients at increased risk by the deep learning model (8.6, 95% confidence interval [CI] = 7.9 to 9.4) compared with Tyrer-Cuzick (4.4, 95% CI = 3.9 to 4.9) and NCI BCRAT (3.8, 95% CI = 3.3 to 4.3) models (P &lt; .001). Area under the receiver operating characteristic curves of the deep learning model (0.68, 95% CI = 0.66 to 0.70) was higher compared with Tyrer-Cuzick (0.57, 95% CI = 0.54 to 0.60) and NCI BCRAT (0.57, 95% CI = 0.54 to 0.60) models. Simulated screening of the top 50th percentile risk by the deep learning model captured statistically significantly more patients with cancer compared with Tyrer-Cuzick and NCI BCRAT models (P &lt; .001). A deep learning model to assess breast cancer risk can support feasible and effective risk-based screening and is superior to traditional models to identify patients destined to develop cancer in large screening cohorts.

The past few years have seen a surge of applying Deep Learning (DL) models for a wide array of tasks such as image classification, object detection, machine translation, etc. While DL models provide an opportunity to solve otherwise intractable tasks, their adoption relies on them being optimized to meet latency and resource requirements. Benchmarking is a key step in this process but has been hampered in part due to the lack of representative and up-to-date benchmarking suites. This is exacerbated by the fast-evolving pace of DL models. This paper proposes DLBricks, a composable benchmark generation design that reduces the effort of developing, maintaining, and running DL benchmarks on CPUs. DLBricks decomposes DL models into a set of unique runnable networks and constructs the original model's performance using the performance of the generated benchmarks. DLBricks leverages two key observations: DL layers are the performance building blocks of DL models and layers are extensively repeated within and across DL models. Since benchmarks are generated automatically and the benchmarking time is minimized, DLBricks can keep up-to-date with the latest proposed models, relieving the pressure of selecting representative DL models. Moreover, DLBricks allows users to represent proprietary models within benchmark suites. We evaluate DLBricks using $50$ MXNet models spanning $5$ DL tasks on $4$ representative CPU systems. We show that DLBricks provides an accurate performance estimate for the DL models and reduces the benchmarking time across systems (e.g. within $95\%$ accuracy and up to $4.4\times$ benchmarking time speedup on Amazon EC2 c5.xlarge).

Prediction of wake and output power for an offshore wind turbine under various wind conditions is essential to maximize energy generation and minimize wake effects. In previous studies, deep learning (DL) models developed using high-fidelity computational fluid dynamics (CFD) simulation results have demonstrated high accuracy and efficiency in predicting the wake characteristics and output power of offshore wind turbines. However, these existing DL models were typically developed for specific types of wind turbines, which cannot be directly used for a new turbine type. Development of a new DL model relying solely on simulation data from a new wind turbine would waste the valuable information contained in previous datasets and existing DL models, leading to increased computational costs for data collection. To address this issue, this paper aims to leverage a previously established simulation dataset and a pre-trained DL model for a wind turbine to develop a new model for another type of wind turbine. The proposed method requires only a small number of new simulations for model development using transfer learning with fine-tuning and model-agnostic meta-learning (MAML), thereby enhancing computational efficiency. The results demonstrate that the models developed using fine-tuning and MAML achieve lower prediction errors and less model training time compared to those trained directly on new wind turbine data. Furthermore, the model trained using MAML outperforms the fine-tuning model when the dataset size is extremely small, while the fine-tuning model has lower prediction errors with relatively larger datasets.

To develop and evaluate static ultrasound images of ovaries-based radiomics and deep learning models for predicting antral follicle count (AFC) and anti-Müllerian hormone (AMH). A retrospective analysis was performed on 395 infertility patients who completed AFC through transvaginal ultrasonography and AMH test in two branches of the hospital. The data set was divided into training cohort (n = 210, Branch A), internal test cohort (n = 91, Branch A), and external test cohort (n = 94, Branch B). The AFC results of static ultrasound images (AFCstatic) and dynamic scan (AFCdynamic) were analysed for correlation and consistency. The radiomics, deep learning and combined models (Rad-DL model) were developed to quantitatively predict AFC and AMH respectively, and the ordered 3-class classification (AFCdynamic≤5,6≤AFCdynamic≤19 and 20≤AFCdynamic) prediction was further carried out. There was a high degree of correlation and consistency between AFCstatic and AFCdynamic in both branches. In the quantitative prediction of AFCdynamic, the R2 of the radiomics, deep learning and Rad-DL models in the external test cohort were 0.572, 0.541, 0.583, respectively. In the quantitative prediction of AMH, the R2 of the radiomics, deep learning and Rad-DL models in the external test cohort were 0.420, 0.352, 0.509, respectively. For the ordered 3-class classification prediction, the radiomics, deep learning and Rad-DL models demonstrated a good discrimination performance with overall AUCs of 0.816 (95%CI, 0.747–0.887), 0.843 (95%CI, 0.772–0.901), and 0.881 (95%CI, 0.816–0.931) in the external test cohort, respectively. Radiomics and deep learning models developed by using static ultrasound images of ovaries may quantitatively predict AFCdynamic to a certain extent, and accurately screen infertility patients with different ovarian reserve levels. This study lays the foundation for further break through the rigid antral follicle count and explore an intelligent non-invasive ovarian reserve evaluation method.

This paper presents a Data Science-oriented application for image classification tasks that is able to automatically: a) gather images needed for training Deep Learning (DL) models with a built-in search engine crawler; b) remove duplicate images; c) sort images using built-in pre-trained DL models or user’s own trained DL model; d) apply data augmentation; e) train a DL classification model; f) evaluate the performance of a DL model and system by using an accuracy calculator as well as the Accuracy Per Consumption (APC), Accuracy Per Energy Cost (APEC), Time to closest APC (TTCAPC) and Time to closest APEC (TTCAPEC) metrics calculators. Experimental results show that the proposed Computer Vision application has several unique features and advantages, proving to be efficient regarding execution time and much easier to use when compared to similar applications.