Distillation-SAM: Knowledge Distillation Based Auto-prompt Embedding Learning for Surgical Image Segmentation.

Despite recent advances in deep learning, the rise of edge devices, and the exponential growth of Internet of Things (IoT) connected devices undermine the performance of deep learning models. It is clear that the future of computing is moving to edge devices. Autonomous vehicles and self-driving cars have leveraged the power of computer vision, especially object detection to navigate through traffic safely. Nevertheless, to be able to drive on all types of roads, these new vehicles have to be equipped with a road anomaly detection system, which strikes the need for small deep learning models of detecting road anomalies that can be deployed on these vehicles for safer driving experience. However, the current deep learning models are not practical on embedded devices due to the heavy resource requirements of the models, as such cannot be deployed on embedded devices. This paper proposes a theoretical approach to building a lightweight model from a cumbersome pothole detection model that is suitable on edge devices using knowledge distillation. It presents the theoretical approach of knowledge distillation, why it is a better technique of model compression compared to the rest. It shows that a cumbersome model can be made lightweight without sacrificing accuracy and with a reduced time complexity and faster training time.

Accurate monitoring of wheat phenology is critical for ensuring wheat production. Recent advances in deep learning have enabled the automated detection of wheat phenology in the field. In particular, deep learning models using multi-temporal image series have addressed the challenge of low accuracy in models that only use spatial features by incorporating dynamic aspects of the wheat growth process. However, utilizing multi-temporal image series introduces challenges such as model parameter redundancy, complex inference processes, and difficulties in real-time deployment. To address these issues, this study presents an optimization method for deriving wheat phenology from single-temporal images (WPDSI) that combines knowledge distillation and multi-layer attention transfer. The proposed approach employs knowledge distillation. In this framework, a teacher model extracts spatiotemporal features from multi-temporal image-series and generates soft labels to guide a student model trained on single-temporal images. This reduces model complexity and input data requirements. Multi-layer attention transfer allows the student model to inherit feature representations from multiple layers of the teacher model. This enhances its ability to capture key phenological characteristics and supports interpretability through attention mechanisms. The proposed method achieves an overall accuracy (OA) of 0.927, comparable to models trained on multi-temporal image series. Furthermore, the model demonstrates strong generalization on unseen datasets, enhancing real-time performance and computational efficiency while maintaining high accuracy, providing a practical solution for deriving wheat phenology in the field. The dataset is available at https://github.com/phenology-detection/WPDSI.

Emotion detection, hence, is an important step toward making human-computer interaction a more enhanced process, where systems are made capable of identifying and responding to the emotional state of users. In fact, multimodal emotion detection systems in which both auditory and visual information are fused are emerging, and these approaches toward expressive emotional states are complementary and robust. Multimodal systems enhance the quality of interacting and, through many applications, can diagnose emotional disorders, monitor automotive safety, and improve human-robot interactions. In nature, the high-dimensional space and dynamic threats have resulted in obtaining low accuracy and high computational cost in applying the traditional models based on single-modality data. On the other hand, multimodal systems explore the synergy between audio and visual data, giving better performance and higher accuracy in inferring subtle emotional expressions. The latest improvement was done on these systems using recent advancements in transfer learning and deep learning techniques.That being said, this research Proposal devises a multimodal emotion recognition system integrating speech and face information through transfer learning for improved accuracy and robustness. Serving this purpose, the objectives of this research entail the effective comparison among different transfer-learning strategies, including the impact of pre-trained models in speech-based emotion recognition, and to introduce the role of voice activity detection in the process. Advanced neural network architectures like Spatial Transformer Networks and bidirectional LSTM in facial emotion recognition will also be tested. Early and late fusion strategies will also be used to find the best strategy for combining speech and facial data.This research will target several challenges that involve the complexity of data, balancing of the model performance-robustness balance, computational limitations, and standardization of evaluations in developing a working and robust emotion recognition system to enhance digital interaction and apply in practical areas. The goal is to create a system that oversteps the limitation of single-modality models through state-of-the-art advances in deep learning, as well as front-line improvements in transfer learning, in the manner of emotion detection performance.

Recent advancements in deep learning techniques tailored for autonomous driving in Indian road conditions are crucial for revolutionizing transportation systems. Indian roads present unique challenges, including unpredictable traffic patterns, diverse road infrastructures, and challenging weather conditions. Deep learning is pivotal in addressing these challenges, focusing on perception, decision-making, and control. Various architectures, including Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and Deep Reinforcement Learning (DRL), are analyzed for their efficacy in autonomous driving. Sensor fusion techniques enhance perception capabilities, such as lidar, radar, and camera data integration. Additionally, advancements in semantic segmentation and object detection algorithms improve scene understanding and obstacle recognition. Collaborative efforts between academia, industry, and government agencies are essential for accelerating the deployment of autonomous driving technology in Indian road conditions. In conclusion, leveraging cutting-edge deep learning methodologies enables safe and efficient navigation of autonomous vehicles, ushering in a transformative future in transportation systems.

The rapid advancements in deep learning (DL) and artificial intelligence (AI) have led to transformative applications across various domains. Edge devices, including mobile and embedded systems, are crucial for the widespread adoption of machine intelligence. However, training and deploying deep neural networks (DNNs) require substantial computational resources, resulting in high energy consumption. This energy often stems from non-renewable sources, leading to significant carbon dioxide emissions and environmental damage. With the emergence of large-scale models for AI-generated content (AIGC), such as large language models and diffusion models, the energy consumption issue has intensified, creating an urgent demand for sustainable AI solutions. These challenges motivate my research on energy-efficient deep learning toward sustainable AI. To build sustainable AI systems, this thesis works on promoting AI sustainability and using AI for sustainability purposes. To promote AI sustainability, a hardware-software co-design approach is adopted, integrating both bottom-up and top-down strategies for practical, energy-efficient deep learning deployments.The first part of the thesis introduces the bottom-up approach, emphasizing AI algorithm-aware efficient system design. Specifically, edge devices utilize dynamic power management strategies, such as dynamic voltage and frequency scaling (DVFS), to manage energy consumption by adjusting voltage-frequency (VF) levels. System-level configurations, including CPU cores, CPU VF levels, and GPU VF levels, significantly impact both latency and power consumption during DNN inference. Developing effective strategies for these configurations enables system-level optimizations, providing a path to deal with the efficiency challenge. The second part of the thesis presents the top-down approach, which focuses on designing hardware-driven efficient AI algorithms. Given the hardware constraints in AI application environments, this approach explores efficient compression techniques, such as weight pruning and knowledge distillation, to create lightweight models with reduced computational demands, thereby enhancing efficiency. The third part of the thesis examines the efficiency challenges posed by the evolution of large-scale models, which have greatly enhanced the capabilities of AIGC for text and image generation. While these models offer substantial generative potential, they also exacerbate the efficiency dilemma. This section explores strategies to optimize the efficiency of these large-scale models based on their unique characteristics. Beyond efficiency, the deployment of AI in real-world applications, such as autonomous vehicles, underscores the importance of prioritizing safety. The fourth part of the thesis addresses issues of AI fairness, trustworthiness, and security to ensure sustainable AI. This section investigates the potential to reverse-engineer adversarial attacks to infer the adversary's underlying information. In conclusion, this thesis works on building energy-efficient DL for sustainable AI. The objective of the work is not only to enhance the efficiency of AI systems but also to address broader concerns of fairness, trust, and security, paving the way for responsible and sustainable AI deployment in real-world scenarios.--Author's abstract

Quantization-aware training for low precision photonic neural networks

Recent advances in deep learning bring impressive performance for multimedia applications. Hence, compressing and deploying these applications on resource-limited edge devices via model compression becomes attractive. Knowledge distillation (KD) is one of the most popular model compression techniques. However, most well-behaved KD approaches require the original dataset, which is usually unavailable due to privacy issues, while existing data-free KD methods perform much worse than data-required counterparts. In this paper, we analyze previous data-free KD methods from the data perspective and point out that using a single pre-trained model limits the performance of these approaches. We then propose a Data-Free Ensemble knowledge Distillation (DFED) framework, which contains a student network, a generator network, and multiple pre-trained teacher networks. During training, the student mimics behaviors of the ensemble of teachers using samples synthesized by a generator, which aims to enlarge the prediction discrepancy between the student and teachers. A moment matching loss term assists the generator training by minimizing the distance between activations of synthesized samples and real samples. We evaluate DFED on three popular image classification datasets. Results demonstrate that our method achieves significant performance improvements compared with previous works. We also design an ablation study to verify the effectiveness of each component of the proposed framework.

Prostate cancer is generally graded by pathologists based on hematoxylin and eosin (H and E) stained slides. Because of the large size of the tumor areas in radical prostatectomies (RP), this task can be tedious and error prone with known high interobserver variability. Recent advancements in deep learning have enabled development of automated systems that may assist pathologists in prostate diagnostics. As prostate cancer originates from glandular tissue, an important prerequisite for development of such algorithms is the possibility to automatically differentiate between glandular tissue and other tissues. In this paper, we propose a method for automatically segmenting epithelial tissue in digitally scanned prostatectomy slides based on deep learning. We collected 30 single-center whole mount tissue sections, with reported Gleason growth patterns ranging from 3 to 5, from 27 patients that underwent RP. Two different network architectures, U-Net and regular fully convolutional networks with varying depths, were trained using a set of sparsely annotated slides. We evaluated the trained networks on exhaustively annotated regions from a separate test set. The test set contained both healthy and cancerous epithelium with different Gleason growth patterns. The results show the effectiveness of our approach given a pixel-based AUC score of 0.97. Our method contains no prior assumptions on glandular morphology, does not directly rely on the presence of lumina, and all features are learned by the network itself. The generated segmentation can be used to highlight regions of interest for pathologists and to improve cancer annotations to further enhance an automatic cancer grading system.

Breast cancer is the most prevalent cancer globally and presents a significant threat to human health. The low survival rate for advanced stages highlights the importance of early and accurate diagnosis. Pathology images, providing insights into cancer's cellular aggressiveness, are crucial for breast cancer diagnosis. Advances in deep learning have transformed automated medical image diagnosis. However, traditional deep learning methods, reliant on large labeled datasets, face challenges with the typically small size of medical image datasets. To overcome this, our study combines deep transfer learning (DTL) with convolutional neural networks(CNN). We utilize and enhance a ResNet50 model pretrained on a large dataset. Additionally, we apply four different deep transfer learning methods to feature extraction and classification of the BreakHis dataset, aiming to determine if breast cancer is benign or malignant. Ultimately, our system achieved the best average accuracy rates under four different magnifications: 98.57%, 96.83%, 97.76%, and 97.16%. These results demonstrate the accuracy and potential of deep transfer learning in classifying breast cancer histopathological images and its promising application in tumor prediction.

The rapid growth of genomic data presents new challenges and opportunities in the field of bioinformatics. Deep learning has emerged as a powerful tool for analyzing large and complex genomic datasets, offering unprecedented capabilities for identifying patterns, predicting gene functions, and understanding genetic variations. This article explores the latest advancements in deep learning techniques applied to genomic data analysis. It discusses key approaches such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), and autoencoders, and their applications in various aspects of genomics, including gene expression analysis, variant calling, and drug discovery. The article also highlights the integration of deep learning with other emerging technologies, such as CRISPR, and its potential to revolutionize personalized medicine and genetic research..

Advancements in deep learning technologies and an increase in medical data have enhanced the accuracy of disease diagnosis and treatment strategies. Notably, significant progress has been made in the use of deep learning-based time-series prediction models for short-term disease onset prediction and analysis of important features. However, research on explainable deep learning for long-term disease progression, such as cancer and chronic diseases, still faces challenges. The difficulty in estimating explainable gradual disease progression from observable patient test data is a key factor. To address this issue, we propose a new approach called the “deep state-space analysis framework.” This framework utilizes sequentially obtained electronic health records (EHRs) to estimate and visualize temporal changes in the latent states of patients related to disease progression. It enables the clustering of latent patient states according to the severity of disease progression and identifies key factors leading to a poor prognosis with medication. To validate our framework, a detailed analysis of data from 12,695 patients with cancer was conducted. The estimated transitions of the latent states capture the clinical status of the patients and their continuous temporal changes. Furthermore, anemia was identified as a poor prognostic factor during state transitions in patients with cancer. Significant features were also confirmed, such as immune cell abnormalities, which are poor prognostic factors in patients treated with Nivolumab, Osimertinib, and Afatinib. This technological innovation deepens our understanding of disease progression and supports early treatment adjustments, prognostic evaluations, and the formulation of optimal long-term strategies. With the advancements in deep learning, its application in healthcare has even greater potential.

Deep learning techniques have recently demonstrated remarkable success in numerous domains. Typically, the success of these deep learning models is measured in terms of performance metrics such as accuracy and mean average precision (mAP). Generally, a model's high performance is highly valued, but it frequently comes at the expense of substantial energy costs and carbon footprint emissions during the model building step. Massive emission of CO2 has a deleterious impact on life on earth in general and is a serious ethical concern that is largely ignored in deep learning research. In this article, we mainly focus on environmental costs and the means of mitigating carbon footprints in deep learning models, with a particular focus on models created using knowledge distillation (KD). Deep learning models typically contain a large number of parameters, resulting in a 'heavy' model. A heavy model scores high on performance metrics but is incompatible with mobile and edge computing devices. Model compression techniques such as knowledge distillation enable the creation of lightweight, deployable models for these low-resource devices. KD generates lighter models and typically performs with slightly less accuracy than the heavier teacher model (model accuracy by the teacher model on CIFAR 10, CIFAR 100, and TinyImageNet is 95.04%, 76.03%, and 63.39%; model accuracy by KD is 91.78%, 69.7%, and 60.49%). Although the distillation process makes models deployable on low-resource devices, they were found to consume an exorbitant amount of energy and have a substantial carbon footprint (15.8, 17.9, and 13.5 times more carbon compared to the corresponding teacher model). The enormous environmental cost is primarily attributable to the tuning of the hyperparameter, Temperature (τ). In this article, we propose measuring the environmental costs of deep learning work (in terms of GFLOPS in millions, energy consumption in kWh, and CO2 equivalent in grams). In order to create lightweight models with low environmental costs, we propose a straightforward yet effective method for selecting a hyperparameter (τ) using a stochastic approach for each training batch fed into the models. We applied knowledge distillation (including its data-free variant) to problems involving image classification and object detection. To evaluate the robustness of our method, we ran experiments on various datasets (CIFAR 10, CIFAR 100, Tiny ImageNet, and PASCAL VOC) and models (ResNet18, MobileNetV2, Wrn-40-2). Our novel approach reduces the environmental costs by a large margin by eliminating the requirement of expensive hyperparameter tuning without sacrificing performance. Empirical results on the CIFAR 10 dataset show that the stochastic technique achieves an accuracy of 91.67%, whereas tuning achieves an accuracy of 91.78%-however, the stochastic approach reduces the energy consumption and CO2 equivalent each by a factor of 19. Similar results have been obtained with CIFAR 100 and TinyImageNet dataset. This pattern is also observed in object detection classification on the PASCAL VOC dataset, where the tuning technique performs similarly to the stochastic technique, with a difference of 0.03% mAP favoring the stochastic technique while reducing the energy consumptions and CO2 emission each by a factor of 18.5.

Deep learning belongs to the field of artificial intelligence, where machines perform tasks that typically require some kind of human intelligence. Deep learning tries to achieve this by drawing inspiration from the learning of a human brain. Similar to the basic structure of a brain, which consists of (billions of) neurons and connections between them, a deep learning algorithm consists of an artificial neural network, which resembles the biological brain structure. Mimicking the learning process of humans with their senses, deep learning networks are fed with (sensory) data, like texts, images, videos or sounds. These networks outperform the state-of-the-art methods in different tasks and, because of this, the whole field saw an exponential growth during the last years. This growth resulted in way over 10,000 publications per year in the last years. For example, the search engine PubMed alone, which covers only a sub-set of all publications in the medical field, provides already over 11,000 results in Q3 2020 for the search term ‘deep learning’, and around 90% of these results are from the last three years. Consequently, a complete overview over the field of deep learning is already impossible to obtain and, in the near future, it will potentially become difficult to obtain an overview over a subfield. However, there are several review articles about deep learning, which are focused on specific scientific fields or applications, for example deep learning advances in computer vision or in specific tasks like object detection. With these surveys as a foundation, the aim of this contribution is to provide a first high-level, categorized meta-survey of selected reviews on deep learning across different scientific disciplines and outline the research impact that they already have during a short period of time. The categories (computer vision, language processing, medical informatics and additional works) have been chosen according to the underlying data sources (image, language, medical, mixed). In addition, we review the common architectures, methods, pros, cons, evaluations, challenges and future directions for every sub-category.

Accurate prostate volume estimation is crucial for effective prostate disease management. Ultrasound (US) imaging, particularly transrectal ultrasound, offers a cost-effective and rapid assessment. However, US images often suffer from artifacts and poor contrast, making prostate volume estimation challenging. This review explores recent advancements in deep learning (DL) techniques for automatic prostate segmentation in US images as a primary step toward prostate volume estimation. We examine various DL architectures, including traditional U-Net modifications and innovative designs incorporating residual connections, multi-directional image data, and attention mechanisms. Additionally, we discuss pre-processing methods to enhance image quality, the integration of shape information, and strategies to improve the consistency and robustness of DL models. The effectiveness of these techniques is evaluated through metrics such as the Dice Similarity Coefficient, Jaccard Index, and Hausdorff Distance. The review highlights the potential of DL in improving prostate volume estimation accuracy and reducing clinical workload while also identifying areas for future research to enhance model performance and generalizability.

The integration of IDS and Internet of Things (IoT) with deep learning plays a significant role in safety. Security has a strong role to play. Application of the IoT network decreases the time complexity and resources. In the traditional intrusion detection systems (IDS), this research work implements the cutting-edge methodologies in the IoT environment. This research is based on analysis, conception, testing and execution. Detection of intrusions can be performed by using the advanced deep learning system and multiagent. The NSL-KDD dataset is used to test the IoT system. The IoT system is used to test the IoT system. In order to detect attacks from intruders of transport layer, efficiency result rely on advanced deep learning idea. In order to increase the system performance, multi -agent algorithms could be employed to train communications agencies and to optimize the feedback training process. Advanced deep learning techniques such as CNN will be researched to boost system performance. The testing part an IoT includes data simulator which will be used to generate in continuous of research work finding with deep learning algorithms of suitable IDS in IoT network environment of current scenario without time complexity.