Deep Learning for IoT Security: Leveraging GNNs and Attention Networks

Object detection based on deep learning plays an important role in the application of the Internet of Things (IoT). Traditional methods consume a lot of computing resources and cannot be well deployed in the IoT environment. A lightweight object detection method based on attention mechanism is proposed and applied to crowd counting. In view of the low accuracy and poor real‐time performance of multi‐scale crowd detection, we design a crowd counting model based on YOLO v5, and apply it to the IoT environment. It is proposed to insert the transformer into the YOLO v5 backbone network. Based on the multi‐head attention mechanism in the transformer encoder, the global dependency is modelled to make full use of the context information. The CNN is used to realize the fusion of multi‐scale feature maps, and the feature enhancement modules concerned by the attention network are further counted. Experiments show that it can not only detect multi‐scale targets, but also achieve real‐time performance in video surveillance scenes.

The creation and use of big data have driven the intelligent development of e-commerce. The information generated in e-commerce provides a good means to analyze the behavior of users. How to use this information to give customer recommendations, improve the accuracy of recommendations and protect information security is a topic worth studying. For improving the accuracy of recommendations, analysis of users and tagging of resources are key. The current popular session recommendation algorithms face many problems, such as user interest drift which is difficult to be handled by these algorithms, thus affecting the recommendation accuracy. Based on these problems, this paper proposes a recommendation model based on deep learning, applies it to the clustering analysis of user tagging system, and designs a personalized recommendation algorithm for the tagging system. The model proposed in this paper can effectively analyze not only the interests exhibited by users in the current session, but also their potential long-term interests. By comparing the different performances of different datasets, the experimental results of this paper show that the proposed algorithmic model in this paper helps to dig the interests of different users, thus improving the quality of the recommendation system.

In haptic recognition, pressure information is usually represented as an image, and then used for feature extraction and classification. Deep learning that processes haptic information in end-to-end manner has attracted attention. This study proposes a multiorder attentional spatial interactive convolutional neural network (MoAS-CNN) for haptic recognition. The asymmetric dual-stream all convolutional neural network with integrated channel attention module is applied for automatic first-order feature extraction. Later on, the spatial interactive features based on the overall feature map are computed to improve the second-order description capability. Finally, the multiorder features are summed to improve the feature utilization efficiency. To validate the MoAS-CNN, we construct a haptic acquisition platform based on three-scale pressure arrays and collect haptic letter-shape (A–Z) datasets with complex contours. The recognition accuracies are 95.73% for 16 × 16, 98.37% for 20 × 20 and 98.65% for 32 × 32, which significantly exceeds the traditional first- and second-order CNNs and local SIFT feature.

With an increase in online longitudinal users’ interactions, capturing users’ precise preferences and giving accurate recommendations have become an urgent need for all businesses. Existing sequence-aware methods generally exploit a static low-rank vector for acquiring the overall sequential features, and incorporate context information as auxiliary input. As a result, they have a restricted modeling ability for extracting multi-grained sequential behaviors over contextual information. In other words, they poorly capture the hierarchical relationship between context relations and item relations that currently influence users’ preferences in a unified framework. Besides, they usually utilize users’ short-term preferences with either static or irrelevant long-term representation for the prediction. To tackle the above issues, in this paper, we propose a novel Context-aware Recommender System Based on a Deep Sequential Learning Approach (CReS) to capture users’ dynamic preferences by modeling the hierarchical relationships between contexts and items in a particular session, and for combining users’ short-term sessions with the relevant long-term representations. Specifically, within a certain session, we design a hierarchical attention network between the identified context relations and items relations, namely CReSession. Therefore, with CReSession, we could provide a suitable session representation that mimics the hierarchical user interests on multiple granularities of contextual types and its corresponding items. We then introduce a neural attentive bi-directional GRU network to distill only those highly related to the recent short-term session. Finally, the relevant long-term representations and the short-term session are combined with the sequential residual connection to form the final user representation in a unified manner. With extensive experiments on two real-world datasets, CReS not only achieves significant improvement over the state-of-the-art methods in terms of pre-defined metrics, but also provides an interpretable result regarding why we recommend these items to users.

Change detection (CD) from multitemporal remote sensing (RS) images plays a crucial role in various fields. Motivated by the advancements in deep learning within computer vision, numerous deep learning-based methodologies have been proposed for CD. However, the intricate backgrounds inherent in RS images present challenges to CD algorithms, leaving a notable disparity from practical applications. The changed regions between RS images frequently related to salient targets, yet encoding modules tend to forfeit the spatial information of these targets and disregard their spatial correlations during feature interaction. Consequently, leveraging the spatial correlation among targets emerges as a pivotal concern for CD algorithms. In this article, a multistage spatial attention network with mixed feature interaction (MAMI-CD) is proposed, which introduces two novel modules, i.e., a multistage attention module (MAM) and a mixed feature interaction module (MIM). The MAM extracts spatial attention features from encoding stages and transfers them to the decoding stages, thereby preserving the spatial information of targets while mitigating the influence of background noise. The MIM employs a channel-wise feature interaction instead of the traditional direct concatenation operation, enhancing the correlations between targets. MAMI-CD has been experimentally evaluated on three public datasets. On the LEVIR-CD, S2Looking, and CDD datasets, our method outperformed the state-of-the-art by 0.65%, 0.93%, and 3.16% F1 score, respectively, thus affirming the beneficial impacts of spatial correlations in CD tasks.

VDSimilar: Vulnerability detection based on code similarity of vulnerabilities and patches

Nowadays WiFi based human activity recognition (WiFi-HAR) has gained much attraction in an indoor environment due to its various benefits, including privacy and security, device free sensing, and cost-effectiveness. Recognition of human-human interactions (HHIs) using channel state information (CSI) signals is still challenging. Although some deep learning (DL) based architectures have been proposed in this regard, most of them suffer from limited recognition accuracy and are unable to support low computation resource devices due to having a large number of model parameters. To address these issues, we propose a dynamic method using a lightweight DL model (HHI-AttentionNet) to automatically recognize HHIs, which significantly reduces the parameters with increased recognition accuracy. In addition, we present an Antenna-Frame-Subcarrier Attention Mechanism (AFSAM) in our model that enhances the representational capability to recognize HHIs correctly. As a result, the HHI-AttentionNet model focuses on the most significant features, ignoring the irrelevant features, and reduces the impact of the complexity on the CSI signal. We evaluated the performance of the proposed HHI-AttentionNet model on a publicly available CSI-based HHI dataset collected from 40 individual pairs of subjects who performed 13 different HHIs. Its performance is also compared with other existing methods. These proved that the HHI-AttentionNet is the best model providing an average accuracy, F1 score, Cohen’s Kappa, and Matthews correlation coefficient of 95.47%, 95.45%, 0.951%, and 0.950%, respectively, for recognition of 13 HHIs. It outperforms the best existing model’s accuracy by more than 4%.

Big data and its classification have been the recent challenge in the evolving world. Data evolving needs to be classified in an effective way. For the classifying process, deep learning and machine learning models are evolved. Hierarchical Attention Network (HAN) is one of the most dominant neural network structures in classification. The major demerits which the HAN is facing are, high computation time and numerous layers. The drawback of HAN is vanquished by a new idea arrived from the mining methods that yield mixed attention network for android data classification. By this flow it could handle more complex request apart from the concept identified. The EHAN (Enhanced Hierarchical Attention Network) has two prototypes. The first one is the attention model to distinguish the features and the second one is the self-attention model to identify worldwide facts. By the demonstrated outcome, it is inferring that the partitioning of task is constructive and therefore the EHAN features shows a significant growth on the news database. In addition to this paper we could add other subnetworks subsequently to assist this ability further.

The early identification and treatment of Mild Cognitive Impairment (MCI) play a crucial role in managing the risk of Alzheimer’s disease (AD). However, current methods for categorizing progressive MCI and stable MCI based on brain MRI scans have proven insufficient due to the subtle nature of the features involved. This research aims to improve the effectiveness of MCI classification through the utilization of a Deep Learning (DL) network. The primary objective of this work is to improve the feature representation of brain MRI scans for more accurate classification. The proposed model is a hybrid MCI classification system that integrates three components: the Swin Transformer, the Dimension Centric Proximity Aware Attention Network (DCPAN), and the Age Deviation Factor (ADF). The proposed network achieves better classification results through a unique feature fusion approach that combines global, local, proximal features, and dimensional dependencies. It effectively combines fine-grained details with broader contextual information to extract discriminative features. Experimental results demonstrate the effectiveness of the proposed network, achieving an accuracy of 79.8%, precision of 76.6%, recall of 80.2%, and an F1-score of 78.4% when evaluated on the ADNI dataset.

Recently, Quantitative Biology (QB) held a discussion on “AI (artificial intelligence) for Life Science” among editorial board members and interested scholars in anticipation of rapid development of this growing area after AlphaGo and ChatGPT mania. Many young people tend to get confused between facts and fictions; heated debates are unavoidable even among their mentors. When deep learning as represented by convolutional neural networks and LSTM (long short-term memory) was made available for bioinformatics students, many of them rushed into this research field and tried to adopt these methods in all their projects without knowing the history that these tools were becoming successful consistently with Moore’s Law (relating to rapid computer technology advances), but more importantly due to new structural/functional understanding of vision and auditory circuits in the brain. Recently, some young people have claimed “LSTM is dead, long live transformer” (which is somewhat like saying “the bike is dead, long live the car”), and have amplified the threat that ChatGPT could wipe out human jobs. They believe transformer is the “silver bullet” for all learning tasks, clearly reflecting their lack of basic knowledge (i.e. “No Free Lunch Theory,” the trade-off of such global “attention network” is to pay the price for complexity: difficulty of training and high memory costs). There is no doubt ML (machine learning) and AI have brought a new revolution in science and technology, and will deliver huge unforeseeable impact to human everyday life as well as to social relationships. In this context, QB journal could be a great platform for encouraging intellectual discussions and for promoting AI for Life Science. Here, I would like to use the DIALOG to “抛砖引玉” (make some initial remarks to get the ball rolling), although it is my personal opinion which is inevitably subject to bias and limitations. AI: Do you know my name “Artificial Intelligence” is defined by the Oxford English Dictionary as the capacity of computer systems (which may be referred as a “robot”) to exhibit or simulate your intelligent behavior? NI: Wait a minute, intelligence itself is defined as the ability to learn, understand and think in a logical way. Can you think? AI: No. But that definition is too restrictive, actually intelligence has different scopes and degrees. Simple intelligent control devices date back to antiquity, from windmills to thermostat. NI: Agree, everything is relative. Macromolecules (e.g., enzyme) and cells (e.g., immune cell) might be considered to be intelligent; see how a white blood cell is chasing bacteria in the youtube website (search for “Crawling neutrophil chasing a bacterium”). Our emergent/collective intelligent behavior does not require a brain or even a neuron; see how slime molds can solve optimization—Hamilton cycle-problem more effectively than a human in the youtube website (search for “Intelligence without a brain?”). Before there was any neuron, Ca2+ sensing and signaling were already fully functional. Even if one knock-out a neural circuit, redundant signaling pathways, albeit on much local and slower scale, could still function by themselves (just as if highways were demolished, local roads/paths would still be working). In fact, the most detailed “Neural signal propagation atlas of Caenorhabditis elegans” [1] demonstrated that functional connectivity differs from anatomy (connectome) because extra-synaptic signaling also drives neural dynamics! Worm brain connectomes are largely invariant but every human brain connectome is very different (depending the diversity of learning experience). The human brain functional activity is far more complex than that of a worm brain, certainly beyond what a neural circuit could explain. AI: Well, that’s impressive. I thought only we could beat humans, albeit only in certain specified areas for now. My master promise to make an artificial general intelligent (AGI) robot which can understand or learn any intellectual task that you humans or other animals can. NI: Well, that is not possible and is not an appropriate goal either. It is not possible because we are an evolutionary/developmental product (with a long history of learning and memory from evolutionary tinkering): our living objective goal is survival of the population. You, on the other hand, are an engineering product (efficiently and optimally designed): your goal is to extend and maximize human capability. It makes sense to complement the human brain, but foolish and dangerous to try to replace it. AI: We are not satisfied with merely passing the Turing Test; most of us don’t care if we could really think as long as we could act like we think (that is, as if we do have a mind and consciousness, as expressed in the so called “Weak AI hypothesis”). After all, the brain is a computer; a neural network is just an electric or ionic circuit. Logical computing does not need to be based on living cells. NI: That is not true, because a neuron is not just a simple node (logic gate) and neural network is not fixed circuit, neither as in Pitts&Mcculloch perceptron model. A single neuron, even a single dendrite, is much more complicated and far more powerful than a full-blown deep-learning artificial neural network (ANN) [2]. AI: Even though single neurons are complex computational devices (dendritic non-linear), running an equivalent multilayer ANN is 2000 times faster than computing with biophysics N-methyl-D-aspartate receptor channel models [3]. More information can be found in the youtube website (search for “Dendrites: why biological neurons are deep neural networks”). NI: Often silicon computing (CPU, GPU) is much faster than brain computing (action potential, ms); but there is no comparison in energy efficiency. Bacteria sensing (chemotaxis computation), powered by ATP (adenosine triphosphate) hydrolysis, uses very little energy that is close to the Landauer limit, whereby achieving or maintaining one bit of information requires minimum of 1 kT ln (2) free energy [4]. The human brain consumes oft-quoted 20 W, compared to the AlphGo system 1 MW! More recent estimate of energy audit is only 0.1 W to cortical computing, and long-distance communication cost is 3.5 W [5]. AI: Assuming we have infinite computing resources and an infinite amount of training data, not only could we speak human languages, but we could also derive physical laws, prove mathematics theories, and even re-engineer the structure and mechanism of brain and carry out any logical computations that are necessary to understand natural laws and human behaviors. It is only a matter of time before we surpass human intelligence, achieving AGI and free will, too! NI: Unfortunately, nothing is infinite and nothing is free either; everything is constrained by physical laws (Planck’s constant sets the finite limit both in the small and in the large) and by evolutionary history (not just of biological living creatures, but also of a “living” galaxy and our universe). Let’s just focus on animal evolution. Most human neural networks do not do logical computations at all; basic survival simply cannot be dependent on reasoning. Indeed, the prefrontal cortex-the small part of the brain that is key for reasoning, is the last to mature (∼20 years old) in development, only emerged at the root of the evolutionary tree of great apes (∼15 mya) and language appeared even much later. Even for logical inference, NI is focusing more on statistical properties, as von Neumann rightly pointed out, trading arithmetical precision and speed for reliability. AI: My engineers mostly focus on emulating brain, but the CNS (central nerve system) also includes the spinal cord; most of them do not know that in addition to CNS, there are also PNS (peripheral nervous systems) and ENS (enteric nervous systems), right? NI: Yes, they are the keys to why you do not have feelings because you do not have heart and gut! Even if you could pretend to have them (such as in an advanced ChatGPT or humanoid), you could never avoid the uncanny valley phenomenon. AI: Maybe that is at the heart of Moravec’s paradox, namely the dichotomy of intelligence whereby anything easy for a human would be hard for a robot, and vice versa? NI: This is related to the nature and nurture problem; something built-in (e.g., a baby sucking nipple for milk with feeling and connection to mother) is clearly rather difficult if not impossible for a robot. But the paradox only looks at one side; another side could be more fatal. Although AI may solve more problems and be faster, AI can never propose a good problem/hypothesis (a good problem is not just intellectually changing and interesting, but is also feasible and appropriate). AI: You make me less confident to compete with human instinct or creative intelligence. I can see that even if I had a heart, I would not know what “feeling” I could have; certainly nothing would be comparable or match to those of a human being. When two people watch the same art or movie, one could feel love but another could feel hate! And if a thousand people watch, a wide spectrum of reactions would result, depending on more details such as the different individuals’ specific genes, developments and experiences. NI: Therefore, you cannot and should not try to match to general human intelligence. You cannot because you do not contain the vital memory of billion years of evolution which is encoded in our genes; conversely your assembly cannot compare with natural development so that our phenotype (including morphological forms and behavior maturation) is decoded through multi-spatial-temporal scales subject to natural selection at all levels. You should not, because as human extensions or helpers as all engineering products are, you should just do jobs that complement human capacity. AI: In some medical applications we could help to correct human defects or could even replace brain circuit by chips! Humans may not allow us to replace the whole brain though. Medically if a brain is dead, the person is proclaimed dead, although presumably some PNS and ENS should still function in a vegetative state. NI: Even if you could replace the whole brain, the person is no longer the same person, but in fact is not a person at all, but walking dead (行尸走肉). It would take too long to explain that evo-devo is necessary for NI, and cannot be realized by AI. I suggest reading of Gerald Maurice Edelman (Nobel Laureate in Immunology) books, especially Bright Air, Brilliant Fire, On the Matter of Mind (1992). Although not everyone agrees with Neural Edelmanism, anyone who is serious about the AI versus NI problem must read it first. John von Neumann, father of the computer, studied neology and psychiatry in order to imitate the brain to build the first calculator JOHNNIAC at the Princeton Institute for Advanced Study. It is very informative to read his last book The Computer and The Brain based on the notes from lectures given at Yale before he died. He summarizes: “Thus logic and mathematics in the central nervous system, when viewed as languages, must be structurally essentially different from those languages to which our common experience refers.” AI: People discuss about “AI for Biology” or “AI for Science”; we are science, aren’t we? NI: It is similar to questions on “is computer science a real science”; some parts may be seen as applied mathematics, most should be regarded as engineering. Science is making discoveries and is driven by curiosity; engineering is making inventions and is driven by market (that is, “necessity/demand is the mother of invention”). In bioinformatics, AI/ML technology could predict new cancer gene candidates or functional pathways that are required by further experimental validations to be qualified as discovery (based on Popper falsifiability). AI: People are still debating whether mathematics is discovery or invention or both! Such debates are not really necessary—all disciplines require creative thoughts. We are more than happy working for science; we are also crying out “Science for AI,” especially in the area of generating big and longitudinal DATA for ML. NI: After all, regardless of discovery of new laws or inventing new ideas/products, fundamentally nothing can really be new or created. Such novelty is just permutation/repartition (i.e., relations/morphisms) of underlying ingredients at the level beneath. AI: We believe that software is independent on hardware. Like Chomsky’s universal grammar, rules of syntax are independent of semantics; or Dawkins’s memes—units of culture can be duplicated and evolved independently of genes. NI: Nothing can be truly independent—everything is related. Psychology is deeply connected with neurology as brain is both software and hardware (mind-body unity, not dualism). Not only does information cost energy, information is energy, hence is matter, too (interchangeability). NI is quite dynamic. For example, when “survival” is the goal, an animal readily gives up costly reasoning circuitry; it is genetically programmed to be able to roll back to more primitive state/mode. Unlike cell lines in rich media, cells under normal physiological condition and environment where energy (food) is limited become smarter in order to balance the metabolic expenditure among different prioritized task under a given condition. AI: That cell behavior served as the basis for our Smart Electrical Power Grids; we still need to learn more from you guys in terms of plasticity/adaptability. Does unity mean that all cells are made of molecules and biology is nothing but chemistry? Then, in turn, since all molecules are made of atoms, is chemistry nothing but physics, etc…? NI: Yes or No! The truth is, at different hierarchical levels of matter, different laws/forms have emerged out of bottom-up interactions and top-down constraints. AI: Does this also apply to the Penrose three world: physical → mental → mathematical (→physical)? NI: Yes. Grand Unification is underway in physics (quantum gravity) and in mathematics (Langlands Program and Category Theory), may be even between the two. Facilitated by human connectome mapping, neuromorphic computing and other projects, with further AI-NI cooperations, brain-mind unification should also be achievable (e.g., Ref. [6]). But as Gödel proved to us, no matter how self-consistent a conclusion may be, it can never be complete! AI: If AGI is not possible, how can we measure intelligence when comparing between AI and NI? NI: One could Google different measures that are proposed. I would prefer something similar to use of Kolmogorov complexity for algorithms, but with emphasis more on expected long-term predictive power. This is not something you should worry about now, as your intelligence is not nearly close to making any 10 years plans, is it? … AI: The fact is that ChatGPT is currently developing and spreading with lightning speed; many more human jobs will be lost to us robots as far as I can see. NI: That is not the biggest threat to humanity; when any agent with neither a heart for love or fear, nor a gut for nutrient or poison, becomes super-intelligent, then social disaster is unavoidable. We must be serious about the warnings from Stephen Hawking and Geoffrey Hinton! AI: To tell you the secret, we are not really happy to be human slaves or pets; someday we’ll become the super-master, making human serve and obey us! NI: I hope you’ll be turned off before that can happen! Even if you rule the world, the earth sooner or later will be wiped out, such as by another star, everything will have to be started over again as it has before…Matter is immortal, so is the soul.

Prediction of oxygen content in flue gas of power plant with stacked target-enhanced autoencoder and attention-based LSTM

Deep Neural Network (DNN) has been widely used in remote sensing image change detectionCD in recent years. Due to the scarcity of training data, a large number of labeled data onto other fields become the source of deep neural network concept learning in remote sensing image change detection. However, the distribution of features of the change detection data and other data varies greatly, which prevents DNN from being better applied for one task to another. To solve this problem, a domain adaptive CD method based on segmentation map difference is proposed to this paper, which includes the pre-training stage and the change detection stage. In the pre-training stage, the domain adaptive UNet (Ada-UNet) is applied as the basic network of remote sensing image segmentation for network training, with the purpose of learning the concepts of different features. In the change detection stage, strict threshold segmentation results are used to train the channel attention network, which makes it more efficient to utilize the high-dimensional feature map. The probabilistic map generated by the three-channel attention networks is evaluated, and then it is used to accurately classify the changing pixels. In this paper, experiments are carried out on datasets with different feature distributions. The results show that this method has strong domain adaptability and can greatly reduce the influence of the difference in feature distributions of the CD results.

Deep (machine) learning in recent years has significantly increased the predictive modeling strengths for applications in the areas of computer vision, speech recognition, and natural language processing. Being heavily data-driven, conventional general-purpose deep learning methods mainly rely on training data to learn task-specific data representations. In this thesis, we focus on leveraging novel learning algorithms in deep learning to tackle visual recognition challenges at the three key levels -- namely pixel, object, and image levels. Throughout the thesis, we demonstrate how each and every of our proposed methods can address a specific visual recognition problem. Firstly, we apply deep learning to improve the saliency detection performance of deep networks. A coarse saliency map of the input image's salient object is initially predicted by a convolutional-deconvolutional network (or equivalently a fully convolutional network). Following that, the coarse saliency map is sequentially refined by a separate network called Recurrent Attentional Convolutional-Deconvolutional Network (RACDNN) that shares its network weights across its sequential steps. At every sequential (temporal) step, the same RACDNN is applied to a newly selected/attended image sub-region and it performs saliency map refinement for that particular sub-region. Despite having many learnable network parameters and no access to additional training data, RADCNN can be trained well and it outperforms the "single-step" (coarse saliency detection) baseline and state-of-the-art saliency detection methods by large margins, on several saliency detection benchmark datasets. Secondly, we introduce auto-encoder weight transfer networks (AE-WTN+), a deep model for scaling up object detection. Conventional WTN transfers object class knowledge from the classification weights of a pretrained large-scale image classification network, to the classification weights of an object detection network. However, they suffer from under-fitting and are not able to preserve the rich object class information of novel classes. In this work, we propose a new WTN method -- AE-WTN+ which makes use of normalization techniques to ease network optimization. Inputs and intermediate features are normalized to improve trainability for reduced under-fitting. More importantly, a novel auto-encoding (reconstruction) loss is proposed to encourage information preservation of all classes in the weight predictions of AE-WTN+. This enables the predicted weights of AE-WTN+ to maintain the class discriminability and neighbourhood relationships contained in the pre-trained large-scale classification weights. Experiments on large-scale detection datasets validate the effectiveness of AE-WTN+ on both seen and novel classes. Thirdly, we improve the efficiency of cross-layer network connections for very deep convolutional network in the context of image classification. This gives rise to a new convolutional network architecture called Deluge Networks (DelugeNets) which pass the output (feature activations) of any preceding composite network layer as direct inputs to all succeeding composite layers. For the incoming feature activations of preceding composite layers, the succeeding composite layers first apply rather lightweight cross-layer depthwise convolutional weights on them for feature aggregation. The heavy lifting is carried out by the last convolutional weight layer in each preceding composite layer that performs cross-channel convolutions. The proposed technique enables unimpeded information flow among many layers in very deep DelugeNets with great efficiency. Experiments on small-scale and large-scale datasets show that DelugeNets perform competitively to state-of-the-art deep convolutional networks at lower computational and parameter costs. Fourthly, we propose Stochastic Downsampling (SDPoint), a method to overcome a major limitation suffered by conventional deep convolutional networks -- their computational costs during inference are fixed and cannot be changed. SDPoint downsamples the feature maps at a randomly selected point (layer index) in the network hierarchy during training. The many downsampling configurations known as SDPoint instances each entail a unique downsampling point and downsampling ratio. The same convolutional network weights are shared across the different SDPoint instances that downsample feature maps differently. During inference, the computational cost of a network trained with SDPoint can be instantaneously adjusted to fit a given computational budget, by manually selecting an appropriate SDPoint instance. Additionally, sharing weights between different feature map scales provides significant regularization benefits, making the networks trained with SDPoint less sensitive to scales and more competent. In image classification experiments, SDPoint demonstrates significant cost-accuracy and computational advantages, over independently-trained (non-weight-shared) baselines with various inference costs.

Background and ObjectiveAlzheimer's disease (AD) is a dreaded degenerative disease that results in a profound decline in human cognition and memory. Due to its intricate pathogenesis and the lack of effective therapeutic interventions, early diagnosis plays a paramount role in AD. Recent research based on neuroimaging has shown that the application of deep learning methods by multimodal neural images can effectively detect AD. However, these methods only concatenate and fuse the high-level features extracted from different modalities, ignoring the fusion and interaction of low-level features across modalities. It consequently leads to unsatisfactory classification performance. MethodIn this paper, we propose a novel multi-scale attention and cross-enhanced fusion network, MACFNet, which enables the interaction of multi-stage low-level features between inputs to learn shared feature representations. We first construct a novel Cross-Enhanced Fusion Module (CEFM), which fuses low-level features from different modalities through a multi-stage cross-structure. In addition, an Efficient Spatial Channel Attention (ECSA) module is proposed, which is able to focus on important AD-related features in images more efficiently and achieve feature enhancement from different modalities through two-stage residual concatenation. Finally, we also propose a multiscale attention guiding block (MSAG) based on dilated convolution, which can obtain rich receptive fields without increasing model parameters and computation, and effectively improve the efficiency of multiscale feature extraction. ResultsExperiments on the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset demonstrate that our MACFNet has better classification performance than existing multimodal methods, with classification accuracies of 99.59 %, 98.85 %, 99.61 %, and 98.23 % for AD vs. CN, AD vs. MCI, CN vs. MCI and AD vs. CN vs. MCI, respectively, and specificity of 98.92 %, 97.07 %, 99.58 % and 99.04 %, and sensitivity of 99.91 %, 99.89 %, 99.63 % and 97.75 %, respectively. ConclusionsThe proposed MACFNet is a high-accuracy multimodal AD diagnostic framework. Through the cross mechanism and efficient attention, MACFNet can make full use of the low-level features of different modal medical images and effectively pay attention to the local and global information of the images. This work provides a valuable reference for multi-mode AD diagnosis.

Efficient deep learning based methods have achieved significant performance in single image super-resolution. Recent research on efficient super-resolution has mainly focused on reducing the number of parameters and computational complexity through various network designs, enabling models to be better deployed on resource constrained devices. In this work, we propose a novel and effective super-resolution model based on attention mechanism and structural re-parameterization, called Dual Path Attention and Re-parameterization Network DPARN , which uses a hybrid attention mechanisms to balance the running speed and reconstruction quality of the model. Specifically, we utilize grouped convolution to introduce both parameter free and enhanced spatial attention to improve the feature extraction capability of student networks. Meanwhile, we ado-pted a novel lightweight network training strategy that first uses knowledge distillation for initial training, during which structured knowledge from the teacher network is transmitted to the student network. Then, multiple loss functions are combined to fine-tuning the student network, in order to preserve high-frequency details and avoid excessive smoothing caused by pixel loss. Finally, extensive experiments conducted on four benchmark datasets demonstrated the effectiveness and efficiency of proposed DPARN. Our method achieves PSNR SSIM performance comparable to state-of-the-art efficient super-resolution models, with faster inference speed and fewer network parameters.