A Novel Hierarchical Convolutional Neural Network for Question Answering over Paragraphs

In this paper, we detail our submission to the 7th year BioASQ competition. We present our approach for Task-7b, Phase B, Exact Answering Task. These Question Answering (QA) tasks include Factoid, Yes/No, List Type Question answering. Our system is based on a contextual word embedding model. We have used a Bidirectional Encoder Representations from Transformers (BERT) based system, fined tuned for biomedical question answering task using BioBERT. In the third test batch set, our system achieved the highest ‘MRR’ score for Factoid Question Answering task. Also, for List type question answering task our system achieved the highest recall score in the fourth test batch set. Along with our detailed approach, we present the results for our submissions, and also highlight identified downsides for our current approach and ways to improve them in our future experiments.

Factoid question answering (QA) has recently benefited from the development of deep learning (DL) systems. Neural network models outperform traditional approaches in domains where large datasets exist, such as SQuAD (ca. 100,000 questions) for Wikipedia articles. However, these systems have not yet been applied to QA in more specific domains, such as biomedicine, because datasets are generally too small to train a DL system from scratch. For example, the BioASQ dataset for biomedical QA comprises less then 900 factoid (single answer) and list (multiple answers) QA instances. In this work, we adapt a neural QA system trained on a large open-domain dataset (SQuAD, source) to a biomedical dataset (BioASQ, target) by employing various transfer learning techniques. Our network architecture is based on a state-of-the-art QA system, extended with biomedical word embeddings and a novel mechanism to answer list questions. In contrast to existing biomedical QA systems, our system does not rely on domain-specific ontologies, parsers or entity taggers, which are expensive to create. Despite this fact, our systems achieve state-of-the-art results on factoid questions and competitive results on list questions.

This study introduces a Hierarchical Convolutional Neural Network (HCNN) for the classification and severity estimation of Mixed Eccentricity (ME) faults in Salient Pole Synchronous Machines (SPSMs). The HCNN extracts patterns from raw voltage and current waveforms, automatically learning fault features without manual preprocessing. Time Delay (TD) embeddings were specifically incorporated into the HCNN to reconstruct underlying system dynamics and effectively capture temporal dependencies and nonlinearities inherent in the input signals. Conventional methods typically rely on extensive signal preprocessing to identify these subtle fault patterns but often struggle due to the sensitivity of preprocessing techniques to noise and overlapping harmonic components. The HCNN achieved perfect classification accuracy for ME faults, and regression accuracies of $90.16 \%$ and $99.93 \%$ were obtained for severity estimation using the voltage and current-based input scenarios, respectively. The results indicate that HCNNs can effectively detect and quantify ME faults in SPSMs.

A system for approximate number discrimination has been shown to arise in at least two types of hierarchical neural network models—a generative Deep Belief Network (DBN) and a Hierarchical Convolutional Neural Network (HCNN) trained to classify natural objects. Here, we investigate whether the same two network architectures can learn to recognise exact numerosity. A clear difference in performance could be traced to the specificity of the unit responses that emerged in the last hidden layer of each network. In the DBN, the emergence of a layer of monotonic ‘summation units’ was sufficient to produce classification behaviour consistent with the behavioural signature of the approximate number system. In the HCNN, a layer of units uniquely tuned to the transition between particular numerosities effectively encoded a thermometer-like ‘numerosity code’ that ensured near-perfect classification accuracy. The results support the notion that parallel pattern-recognition mechanisms may give rise to exact and approximate number concepts, both of which may contribute to the learning of symbolic numbers and arithmetic.

Natural language processing systems are often downstream of unreliable inputs: machine translation, optical character recognition, or speech recognition. For instance, virtual assistants can only answer your questions after understanding your speech. We investigate and mitigate the effects of noise from Automatic Speech Recognition systems on two factoid Question Answering (QA) tasks. Integrating confidences into the model and forced decoding of unknown words are empirically shown to improve the accuracy of downstream neural QA systems. We create and train models on a synthetic corpus of over 500,000 noisy sentences and evaluate on two human corpora from Quizbowl and Jeopardy! competitions.

The ghost phenomenon in synthetic aperture radar (SAR) imaging is primarily caused by azimuth or range ambiguities, which cause difficulties in SAR target detection application. To mitigate this influence, we propose a ship target detection method in spaceborne SAR imagery, using a hierarchical convolutional neural network (H-CNN). Based on the nature of ghost replicas and typical target classes, a two-stage CNN model is built to detect ship targets against sea clutter and the ghost. First, regions of interest (ROIs) were extracted from a large imaged scene during the coarse-detection stage. Unwanted ghost replicas represented major residual interference sources in ROIs, therefore, the other CNN process was executed during the fine-detection stage. Finally, comparative experiments and analyses, using Sentinel-1 SAR data and various assessment criteria, were conducted to validate H-CNN. Our results showed that the proposed method can outperform the conventional constant false-alarm rate technique and CNN-based models.

This paper compares some existing evaluation metrics for factoid question answering (QA) from the viewpoint of stability and sensitivity, using the NTCIR-4 QAC2 Japanese factoid QA tasks and the Buckley/Voorhees stability method and Voorhees/Buckley swap method. Our main findings are: (1) For QA evaluation with ranked lists containing up to five answers, the fraction of questions with a correct answer within top 5 (NQcorrect5) and that with a correct answer at rank 1 (NQcorrect1) are not as stable and sensitive as reciprocal rank. (2) Q-measure, which can handle multiple correct answers and answer correctness levels, is at least as stable and sensitive as reciprocal rank, provided that a mild gain value assignment is used. Emphasizing answer correctness levels tends to hurt stability and sensitivity, while handling multiple correct answers improves them. As our experimental methods are language-independent, we believe that these findings apply to QA in languages other than Japanese as well.

Recently, a self-attention network (SAN) is developed as an effective strategy to extract features from attention areas for image classification. However, for hyperspectral image (HSI), the lack of position supervision of object regions and inefficient similarity computation lead to unsatisfactory classification performance on mixed pixels. To alleviate the above two problems for HSI image classification, we propose a novel hierarchical homogeneity-attention network (H2AN) in this article. First, we design a homogeneity-attention block (HAB) to depict the feature correlation with the homogeneous mask. Using the supervision of homogeneity mask, we can calculate the attention guided by the predefined number of homogeneity embeddings, which can reduce the heavy computation instead of the global search in self-attention block (SAB) of SAN. Second, we propose a hierarchical convolutional neural network (HCNN) inserting the HAB into different levels of network cells for highly efficient feature extraction of target regions, named H2AN. Because of the transferring of homogeneity property from shallow layer to deep layer, our H2AN outperforms the state-of-the-art methods in qualitative and quantitative experiments on three typical datasets.

Predicting bridge longitudinal displacement from monitored operational loads with hierarchical CNN for condition assessment

Gastrointestinal (GI) conditions are widespread and significantly impact the quality of life and healthcare. Stool appearance is a valuable GI diagnostic indicator, and with the advent of the Internet of Things (IoT), daily monitoring of excreta from a toilet is emerging as a promising digital health tool. This article describes a stool image analysis approach that classifies two physiologically relevant stool characteristics: 1) form and 2) color for an IoT-based smart toilet. We constructed a stool image data set with 3275 images, spanning all seven types of the Bristol stool-form scale (BSFS), a widely used metric for stool consistency and a variety of colors. We used ground-truth data obtained through the annotation of our data set by two gastroenterologists and developed a stool-color card to standardize the labeling of stool colors. We addressed two limitations associated with the application of computer-vision techniques to a smart-toilet system: 1) uneven separability between different stool-form categories and 2) class imbalance in the data set. We present results on hierarchical convolutional neural network (CNN) architectures for training a stool-form classifier and on perceptual color quantization coupled with machine-learning techniques to optimize the color-feature space for the classification of stool color. We utilized an edge–cloud approach to pursue an optimal balance between accuracy and latency and for the classification of stool form, we achieved a balanced accuracy of 84.4% and 84.2% reduction in latency compared to a cloud-model only. For color classification, the logistic-regression (LR) classifier provided 80.8% balanced accuracy and 47.3% reduction in communication latency.

Robust RGB-D tracking via compact CNN features

With the increasing daily workload of physicians, computer-aided diagnosis (CAD) systems based on deep learning play an increasingly important role in pattern recognition of diagnostic medical images. In this paper, we propose a framework based on hierarchical convolutional neural networks (CNNs) for automatic detection and classification of focal liver lesions (FLLs) in multi-phasic computed tomography (CT). A total of 616 nodules, composed of three types of malignant lesions (hepatocellular carcinoma, intrahepatic cholangiocarcinoma, and metastasis) and benign lesions (hemangioma, focal nodular hyperplasia, and cyst), were randomly divided into training and test sets at an approximate ratio of 3:1. To evaluate the performance of our model, other commonly adopted CNN models and two physicians were included for comparison. Our model achieved the best results to detect FLLs, with an average test precision of 82.8%, recall of 93.4%, and F1-score of 87.8%. Our model initially classified FLLs into malignant and benign and then classified them into more detailed classes. For the binary and six-class classification, our model achieved average accuracy results of 82.5 and73.4%, respectively, which were better than the other three classification neural networks. Interestingly, the classification performance of the model was placed between a junior physician and a senior physician. Overall, this preliminary study demonstrates that our proposed multi-modality and multi-scale CNN structure can locate and classify FLLs accurately in a limited dataset, and would help inexperienced physicians to reach a diagnosis in clinical practice.

Introduction: Accurate and automated assessment of anterior cruciate ligament (ACL) injuries in MR images is essential for athlete healthcare and rapid diagnosis of knee injuries. However, challenges such as the small size of the ligament, variations in MR image quality, and complex anatomical structures complicate the classification process.Methods: In this study, we propose a hierarchical deep learning model for the detection and classification of ACL injuries. The model consists of two main phases: ACL segmentation and injury classification. In the first phase, we employ an encoder-decoder architecture with attention mechanisms to accurately identify the ACL region in MR images, while suppressing background noise. Skip connections are used to preserve spatial details and improve segmentation accuracy. In the second phase, the segmented ACL region is input into a hierarchical convolutional neural network (CNN) for classification. Dense blocks are incorporated to maximize feature reuse, while max-pooling and global average pooling (GAP) layers help to reduce overfitting and improve feature extraction.Results: The proposed method was evaluated on a knee MRI dataset and compared with other state-of-the-art approaches. Our model demonstrated high accuracy in both segmentation and classification tasks, owing to the integration of attention mechanisms and hierarchical feature extraction.Conclusion: This approach offers a robust solution for the automated assessment of ACL injuries, providing clinicians and sports medicine specialists with a reliable tool for more efficient and accurate diagnosis.

With 5G millimeter wave communications, the resulting radiation reflects on most visible objects, creating rich multipath environments. The radiation is thus significantly shaped by the obstacles it interacts with, carrying latent information regarding the relative positions of the transmitter, the obstacles, and the mobile receiver. Through a pre-estabilhed codebook of beamforming patterns transmitted by a base station, the concept of beamformed fingerprints for mobile devices’ outdoor positioning has been previously proposed. In this paper, a tailored hierarchical convolutional neural network is proposed to further leverage the structure in the aforementioned hidden information. Average errors of down to 3.3 meters are obtained on a simulation environment based on realistic outdoor scenarios, containing mostly non-line-of-sight positions, making it a very competitive and promising alternative for outdoor positioning.

With the rapid-development of AI technology, artificial intelligence algorithms for the aerospace applications have shown very good simulation performance in many areas. Among the spaceborne application fields, star identification can be seen as a typical pattern recognition process. It’s also the key part of attitude determination of the satellites, which requires the algorithm to be robust and efficient due to the limited computing and storing resources of the spaceborne computers. Nevertheless, most of the previous algorithms are not possible to be applied in practical due to the reasons above. This article proposes a strategy of constructing ‘net-structure’ images of stars to build the datasets for training and testing. Besides, a hierarchical convolutional neural network(CNN) with a small size is also designed. It performs good results on robustness and efficiency in the experiments. In the end, a method of fusing the Conv layers and the batch normalization (BN)layers is also adopted to further accelerate the algorithm.