Optimal explainable vision transformer framework for skin cancer diagnosis with neural architecture search feature learning

ETNAS: An energy consumption task-driven neural architecture search

Defect recognition is an important task in intelligent manufacturing. Due to the subjectivity of human annotation, the collected defect data usually contains a lot of noise and unpredictable uncertainties, which have a great negative influence on defect recognition. It is a significant challenge to discover an effective defect recognition model with satisfactory uncertainty processing ability. A natural way is to automatically search for an efficient deep model, which can be realized by neural architecture search (NAS). To achieve this, we propose an efficient fuzzy NAS framework for defect recognition, where the searched architecture can effectively handle uncertain information from the given datasets. Specifically, we first design a fuzzy search space and the related encoding strategy for fuzzy NAS. Then, we propose a comparator-based evolutionary search approach, where an online end-to-end comparator is learned to directly determine the selection of candidate architectures from the evolutionary population. The comparator works in an end-to-end way and it transforms the complex ranking problem of evaluating architectures into a simple classification task, which overcomes the rank disorder issue suffered from traditional performance predictors. A series of experimental results demonstrate that the architecture with fewer #Params (1.22 M) search by fuzzy neural architecture search framework for defect recognition method achieves higher accuracy (92.26%) compared to the state-of-the-art results (i.e., DARTS-PV) on the ELPV dataset, as well as competitive results (accuracy = 76.4%, #Params = 1.04 M) on the CODEBRIM dataset. Experimental results show the effectiveness and efficiency of our proposed method in handling uncertain problems.

New illumination and imaging techniques are continually being developed for cancer diagnosis. They need to be evaluated in the framework of a specific diagnostic problem. In this work, we evaluate the usefulness of fluorescence illumination within the framework of skin cancer diagnosis. This illumination provides monochrome images that encode certain information about deep layers of the skin, which can be particularly interesting for the diagnosis of skin lesions such as basal cell carcinoma. A broad study of candidate diagnostic features extracted from fluorescence images and evaluated within the framework of the posed diagnostic problem was conducted. Afterward, we used both a genetic algorithm (GA) and forward and backward scanning methods for feature selection and evaluated the diagnostic results by using the K-nearest neighbors (KNN) classifier. This work validates the fluorescence illumination technique for skin cancer diagnosis, indicating concrete image processing techniques that best target the diagnostic problem, and shows that the GA approach obtains the best classification results.

Convolutional neural networks (artificial intelligence, AI) are rapidly appearing within the field of dermatology, with diagnostic accuracy matching that of dermatologists. As technologies become available for use by both the health professionals and the general public, their uptake in healthcare will become more acceptable. National Health Service England recognizes the potential of AI for healthcare but emphasizes that patient-centred care should be at the forefront of these technological advancements. To obtain opinions of patients on the use of AI in a dermatology setting, when aiding the diagnosis of skin cancers. A cross-sectional 14-point questionnaire was handed out to patients attending dermatology outpatient skin cancer clinics in two UK hospitals, between March and August 2018. In total, 603 patient questionnaires were completed. Nearly half (47%; n = 282) of respondents were not concerned if AI technology was used by a skin specialist to aid skin cancer diagnosis. However, the majority (81%; n = 491) of respondents, considered it important for a dermatologist to examine and confirm a diagnosis and to be present for discussion of a cancer diagnosis. Although the majority of respondents were not reluctant about the use of AI for skin cancer diagnosis, respondents still considered it important that dermatologists are involved in the diagnosis and/or confirmation of skin cancer. Furthermore, the study results demonstrate that personal interaction with a clinician is important. This is in keeping with proposals that AI be used as an adjunctive technology to increase accuracy of skin cancer diagnoses, but not as a substitute for a dermatologist.

Fault detection and diagnosis for non-linear processes empowered by dynamic neural networks

Accurate dynamic reserve estimation is essential for effective reservoir development, particularly in fractured vuggy carbonate reservoirs characterized by complex pore structures, multiple spatial scales, and pronounced heterogeneity. Traditional reserve evaluation methods often struggle to account for the coupled behavior of pores, fractures, and vugs, leading to limited reliability. In this study, a modified material balance equation is proposed that explicitly considers the contributions of matrix pores, fractures, and vugs, as well as the influence of varying aquifer multiples. To validate the model, physical experiments were conducted using cores with different fracture–vug configurations under five distinct aquifer multiples. A field case analysis was also performed using production data from representative wells in the Fuman Oilfield. The results demonstrate that the proposed model achieves a fitting accuracy exceeding 94%, effectively capturing the dynamics of fractured vuggy systems with active water drive. The model enables quantitative evaluation of single-well reserves and aquifer multiples, providing a reliable basis for estimating effective recoverable reserves. Furthermore, by comparing simulated formation pressures (excluding aquifer effects) with actual static pressures, the contribution of external aquifer support to reservoir energy can be quantitatively assessed. This approach offers a practical and robust framework for reserve estimation, pressure diagnosis, and development strategy optimization in strongly water-driven fractured vuggy reservoirs.

Skin cancer is considered to be the most common human malignancy. Around 5 million new cases of skin cancer are recorded in the United States annually. Early identification and evaluation of skin lesions are of great clinical significance, but the disproportionate dermatologist-patient ratio poses a significant problem in most developing nations. Therefore a novel deep architecture, named as SkiNet, is proposed to provide faster screening solution and assistance to newly trained physicians in the process of clinical diagnosis of skin cancer. The main motive behind SkiNet’s design and development is to provide a white box solution, addressing a critical problem of trust and interpretability which is crucial for the wider adoption of Computer-aided diagnosis systems by medical practitioners. The proposed SkiNet is a two-stage pipeline wherein the lesion segmentation is followed by the lesion classification. Monte Carlo dropout and test time augmentation techniques have been employed in the proposed method to estimate epistemic and aleatoric uncertainty. A novel segmentation model named Bayesian MultiResUNet is used to estimate the uncertainty on the predicted segmentation map. Saliency-based methods like XRAI, Grad-CAM and Guided Backprop are explored to provide post-hoc explanations of the deep learning models. The ISIC-2018 dataset is used to perform the experimentation and ablation studies. The results establish the robustness of the proposed model on the traditional benchmarks while addressing the black-box nature of such models to alleviate the skepticism of medical practitioners by incorporating transparency and confidence to the model’s prediction.

Deep Neural Networks are often vulnerable to adversarial attacks. Neural Architecture Search (NAS), one of the tools for developing novel deep neural architectures, demonstrates superior performance in prediction accuracy in various machine learning applications. However, the performance of a neural architecture discovered by NAS against adversarial attacks has not been sufficiently studied, especially under the regime of knowledge distillation. Given the presence of a robust teacher, we investigate if NAS would produce a robust neural architecture by inheriting robustness from the teacher. In this paper, we propose Robust Neural Architecture Search by Cross-Layer knowledge distillation (RNAS-CL), a novel NAS algorithm that improves the robustness of NAS by learning from a robust teacher through cross-layer knowledge distillation. Unlike previous knowledge distillation methods that encourage close student-teacher output only in the last layer, RNAS-CL automatically searches for the best teacher layer to supervise each student layer. Experimental results demonstrate the effectiveness of RNAS-CL and show that RNAS-CL produces compact and adversarially robust neural architectures. Our results point to new approaches for finding compact and robust neural architecture for many applications. The code of RNAS-CL is available at https://github.com/Statistical-Deep-Learning/RNAS-CL.

Improving the performance of deep neural networks (DNNs) is important to both the compiler and neural architecture search (NAS) communities. Compilers apply program transformations in order to exploit hardware parallelism and memory hierarchy. However, legality concerns mean they fail to exploit the natural robustness of neural networks. In contrast, NAS techniques mutate networks by operations such as the grouping or bottlenecking of convolutions, exploiting the resilience of DNNs. In this work, we express such neural architecture operations as program transformations whose legality depends on a notion of representational capacity. This allows them to be combined with existing transformations into a unified optimization framework. This unification allows us to express existing NAS operations as combinations of simpler transformations. Crucially, it allows us to generate and explore new tensor convolutions. We prototyped the combined framework in TVM and were able to find optimizations across different DNNs, that significantly reduce inference time - over 3× in the majority of cases. Furthermore, our scheme dramatically reduces NAS search time.

Multi-objective neural architecture search (NAS) for super-resolution image restoration (SRIR) targets models that simultaneously deliver high-fidelity reconstructions and respect strict computational budgets—requirements that single-objective searches routinely overlook. In response, we introduce BASS, a versatile multi-branch search space, together with a hybrid optimisation framework that couples NSGA-III’s global exploration with fine-grained local search. The local phase is preliminarily instantiated with hill climbing, tabu search and simulated annealing, enabling a systematic comparison of their ability to refine candidate architectures. We cast the task as a tri-objective problem that maximises predicted PSNR while minimising floating-point operations and parameter count, ensuring that discovered networks remain both accurate and efficient. Extensive experiments show that hybridising NSGA-III with local search accelerates convergence and consistently yields superior architectures; the hill-climbing variant offers the best overall trade-off. The resulting BASSN models attain competitive peak signal-to-noise ratios across ×2, ×3 and ×4 upscaling factors while fitting diverse resource envelopes, providing a practical and scalable route toward deployable SRIR solutions.

Differentiable architecture search has demonstrated promising results in automatically designing neural network architectures with desired properties, such as high accuracy and low FLOPs. However, it suffers from a cumbersome training process, and the injection of constraints in the search phase often relies on some hand-crafted heuristic regularizers, the design of which typically requires tremendous human effort. In this paper, to address these critical challenges, we present FGA-NAS, an efficient method for resource-constrained architecture search. First, to reduce the computational cost and improve search flexibility, we propose a novel condensed search space that merges multiple parallel-placed candidates into a single one. Second, to enable the gradient-based optimization for neural architecture search (NAS) under multiple combinatorial constraints, we decompose the constrained NAS into a few simple sub-problems without introducing any heuristics by using the ADMM algorithm [1]. Then, the constrained NAS can be resolved by alternately solving the simple sub-problems. Experimental results on ImageNet show that our method can discover efficient and accurate neural network architectures that achieve the state-of-the-art by only using 0.2 GPU days.

Decision of the model complexity is a significant challenge in contemporary data-driven modeling applications. Designing neural architecture involves the process of determining the optimal model complexity for deep neural networks (DNNs) models in order to uncover relationships in real-world data patterns. Consequently, optimizing DNN architectures is crucial for enhancing the practical approximation performance of DNN models to real-world data. This study implements a data-driven neuroevolution scheme for the optimal neural architecture search (NAS) and demonstrates a data-driven engineering application for cooling load estimation. The proposed neuroevolution scheme aims at evolving to the best generalizing DNN model that well suits the modeling complexity requirements of the dataset. To this end, the objective function for the neural architecture optimization process is simplified to the mean square error of the test dataset, which enables to reduce the risk of insufficient generalization during NAS. By employing this objective function for evolution field optimization (EFO), the proposed neuroevolution process can automatically achieve the optimal model complexity, preventing overfitting and underfitting cases, and thereby attaining almost the best generalization for the dataset. For this purpose, this approach combines parametric learning with the backpropagation algorithm and structural learning with EFO-based neural architecture search to address data-driven, optimal complexity DNN model generation problems. Effectiveness of the method is demonstrated in the cooling load estimation problem of residential buildings, and performances of the optimal DNN models with four objective functions are analyzed. The design of objective function for the best generalizing model is also elaborated.

This paper proposes a neural architecture search (NAS) method for split computing. Split computing is an emerging machine-learning inference technique that addresses the privacy and latency challenges of deploying deep learning in IoT systems. In split computing, neural network models are separated and cooperatively processed using edge servers and IoT devices via networks. Thus, the architecture of the neural network model significantly impacts the communication payload size, model accuracy, and computational load. In this paper, we address the challenge of optimizing neural network architecture for split computing. To this end, we proposed NASC (NAS for Split Computing), which jointly explores optimal model architecture and a split point to achieve higher accuracy while meeting latency requirements (i.e., smaller total latency of computation and communication than a certain threshold). NASC employs a one-shot NAS that does not require repeating model training for a computationally efficient architecture search. Our performance evaluation using hardware (HW)-NAS-Bench of benchmark data demonstrates that the proposed NASC can improve the “communication latency and model accuracy” trade-off, i.e., reduce the latency by approximately 40–60% from the baseline, with slight accuracy degradation.

Deep learning has been applied in the field of network intrusion detection and has yielded good results. In malicious network traffic classification tasks, many studies have achieved good performance with respect to the accuracy and recall rate of classification through self-designed models. In deep learning, the design of the model architecture greatly influences the results. However, the design of the network model architecture usually requires substantial professional knowledge. At present, the focus of research in the field of traffic monitoring is often directed elsewhere. Therefore, in the classification task of the network intrusion detection field, there is much room for improvement in the design and optimization of the model architecture. A neural architecture search (NAS) can automatically search the architecture of the model under the premise of a given optimization goal. For this reason, we propose a model that can perform NAS in the field of network traffic classification and search for the optimal architecture suitable for traffic detection based on the network traffic dataset. Each layer of our depth model is constructed according to the principle of maximum coding rate attenuation, which has strong consistency and symmetry in structure. Compared with some manually designed network architectures, classification indicators, such as Top-1 accuracy and F1 score, are also greatly improved while ensuring the lightweight nature of the model. In addition, we introduce a surrogate model in the search task. Compared to using the traditional NAS model to search the network traffic classification model, our NAS model greatly improves the search efficiency under the premise of ensuring that the results are not substantially different. We also manually adjust some operations in the search space of the architecture search to find a set of model operations that are more suitable for traffic classification. Finally, we apply the searched model to other traffic datasets to verify the universality of the model. Compared with several common network models in the traffic field, the searched model (NAS-Net) performs better, and the classification effect is more accurate.

Neural Architecture Search (NAS) has emerged as a crucial method for automating the design of deep learning models. Despite its potential, NAS frequently requires substantial computational and hardware resources. To mitigate these challenges, transferable NAS (TNAS) has been introduced, leveraging prior NAS results to enhance performance on new tasks. However, existing methods largely focus on knowledge transfer within identical neural search spaces, overlooking the potential for cross-domain transferability. Motivated by this gap, we explore evolutionary TNAS across heterogeneous search spaces by learning common neural representations. In particular, we introduce a novel approach that encodes both operational and topological information of neural architectures into a unified sequence using a simple tokenizer. This sequence is then processed by a variational auto-encoder, with a Transformer-based encoder to capture rich neural representations and a decoder that reconstructs the original sequence. By utilizing these latent representations, we further establish an inter-domain mapping that acts as a bridge, enabling effective explicit solution transfer among diverse search spaces to enhance the evolutionary NAS process. To harness this capability, we develop an evolutionary sequential transfer optimization approach that transfers knowledge during population initialization, providing both flexibility and adaptability. To the best of our knowledge, this work serves as the first attempt in the literature exploring evolutionary TNAS across diverse spaces. Moreover, we demonstrate the utility of our method through comprehensive empirical studies using different architecture spaces, including NAS-Bench-101, NAS-Bench-201, and the DARTS search space. Our results show that the proposed method significantly enhances the adaptability and performance of NAS across varied domains.