How does Color Constancy Affect Target Recognition and Instance Segmentation?

A Real-World Size Organization of Object Responses in Occipitotemporal Cortex

Deep Neural Networks (DNN) are now widely used in computer vision, due to their recent success in large-scale image classification. Despite the impressive performance of DNN in supervised optical character recognition (OCR) for handwritten documents, they typically have not been used for the application of reading historical manuscripts. The main reason is the lack of availability of large datasets and of understanding about the amount of data required to achieve good results in supervised character classification tasks.In this paper, we show how to train DNN with a limited amount of data for the application of OCR of historical manuscripts. We investigate different strategies for data augmentation for palimpsests and analyze the effect of different methods on DNN performance. The method was tested on the well-known Archimedes palimpsest image dataset using 30 exemplars per character. We found that using the random mask overlay method of augmentation we can achieve up to 82% accuracy for classifying characters of the Archimedes palimpsest.

Introduction: Patients with non-ischemic DCM are at risk of life-threatening ventricular arrhythmias. Improved patient selection for ICD implantation is warranted. Deep neural networks (DNN) can discover complex ECG features without the need for hand-crafted feature extraction. This study aimed to distinguish patients at risk of arrhythmias and detect and visualize ECG characteristics that attribute to the DNN classification. Methods: We trained a DNN model on baseline raw ECG samples of adult patients with DCM for the composite outcome of sustained ventricular arrhythmia, (aborted) sudden cardiac death and appropriate ICD therapy in the University Medical Centre Utrecht. Gradient-Weighted Class Activation Mapping was used for visual interpretation of the network. DNN performance was validated both internally and in an external cohort from the Maastricht University Medical University Center. Results: 26.628 ECGs were used for pre-training the network. In total, 317 patients with DCM were included in the primary cohort with a median age of 52 years (IQR 42 - 61), median LVEF of 25% (IQR 20-33), NYHA II and III (32% and 25%). 233 patients were ICD recipients. Subsequently 84 patients reached the outcome (median follow-up duration of 3.96 years (IQR 1.94 - 7.00)). The DNN focused on subtle but predictive differences in the terminal QRS-segment, the T-wave, and the P-wave (figure). The AUC of the DNN was 0.70 in cross-validation. External validation was similar, with an AUC of 0.66. Conclusion: In this study we trained and validated a deep neural network for prediction of life-threatening arrhythmic events. Even though the discriminatory changes on ECG segments were very subtle, the trained DNN was able to predict life-threatening events, focusing on the terminal QRS-segment, the T-wave, and the P-wave. Future studies are warranted to elucidate the identified ECG abnormalities and understand their electrophysiological substrates for LTA prediction in DCM.

The recent surge of interest in Deep Neural Networks (DNNs) has led to increasingly complex networks that tax computational and memory resources. Many DNNs presently use 16-bit or 32-bit floating point operations. Significant performance and power gains can be obtained when DNN accelerators support low-precision numerical formats. Despite considerable research, there is still a knowledge gap on how low-precision operations can be realized for both DNN training and inference. In this work, we propose a DNN architecture, Deep Positron, with posit numerical format operating successfully at ≤8 bits for inference. We propose a precision-adaptable FPGA soft core for exact multiply-and-accumulate for uniform comparison across three numerical formats, fixed, floating-point and posit. Preliminary results demonstrate that 8-bit posit has better accuracy than 8-bit fixed or floating-point for three different low-dimensional datasets. Moreover, the accuracy is comparable to 32-bit floating-point on a Xilinx Virtex-7 FPGA device. The trade-offs between DNN performance and hardware resources, i.e. latency, power, and resource utilization, show that posit outperforms in accuracy and latency at 8-bit and below.

Advanced Driver Assistance System (ADAS) is a very important part of an up to date vehicle. For achieving highlevel objectives in such ADAS functionality like LKA (lane keeping assistance), LDW (lane departure warning system), FCW (forward collision warning) the quality of the algorithms under the hood must be extremely high. In the last few years, it is common that these algorithms are based on DNNs (deep neural networks) applied to the tasks of semantic and instance segmentation, 2D/3D object detection and visual object tracking. Recent state-of-the-art DNN models as usual solve only one single task from the listed above and running several neural networks is rather computationally expensive and even impossible due to the lack of the GPU memory. One of the approaches used to overcome such a problem is a shared backbone (also called feature extractor or encoder). The backbone consumes most of the computing resources thus the model with a shared backbone achieves better inference performance. Unfortunately, the training procedure for a shared backbone model has several difficulties. The first one is the lack of datasets with all the required and uniform annotation types. The second problem is a more sophisticated backpropagation procedure. In this paper, we consider several methods for multi-task neural network training and present the results of such training procedures on several public datasets with dissimilar annotation types. The shared backbone is applied to the following three tasks performed simultaneously on the road scene: semantic segmentation, 2D object detection and 3D object detection. While the performance of the DNNs with shared backbone increased significantly, we obtained the quality evaluation results, which are quite close to the original separate state-of-the-art DNNs and even outperforms them in some evaluation indices.

Deep neural networks (DNNs) are increasingly deployed in various fields. Despite their spectacular advances, DNNs are known to suffer from adversarial vulnerabilities. The robustness of DNNs is then threatened by leading them to misclassifications with unexpected inputs (adversarial examples). The fuzzing technique frequently used for testing traditional software has recently been adopted to evaluate the robustness of DNNs. Current DNN fuzzing techniques focus on image classification DNNs and generate test cases by mutations, e.g., image transformations and adversarial perturbations. However, mutation-based test cases usually lack diversity and have distribution deflection from the original DNN input space, which impacts the evaluation of DNNs. In this paper, we propose a generation-based fuzzing frame-work FuzzGAN to detect adversarial flaws existing in DNNs. We integrate the testing purpose and the guidance of the neuron coverage into the original objectives of auxiliary classifier generative adversarial networks. Hence, FuzzGAN learns the representation of a DNN's input space and generates test cases without the limitation of any concrete seed input. We evaluate the performance of FuzzGAN on two DNN models that have classical network structures and are trained on public datasets. The experiment results demonstrate that FuzzGAN can generate realistic, diverse and valid test cases and achieve high neuron coverage. Moreover, these test cases can be used to improve the performance of the target DNN through adversarial retraining.

Accurate prediction of energy demand is crucial for improving services, reducing costs, and optimizing operations in energy systems. Deep neural networks (DNNs) have emerged as a popular method for energy demand forecasting. However, the performance of DNNs can be affected by data quality and hyperparameter selection. To address these concerns, this study proposes a novel energy demand forecasting technique that combines DNNs with an enhanced Giza pyramid construction methodology. The aim of this study is to provide a more reliable and effective approach for forecasting energy demand. The DNNs are employed to capture the complex relationships between input and output variables, while the Giza pyramids algorithm is utilized to optimal selection of hyperparameters of the network. Real-world energy demand data is used to evaluate the proposed approach, comparing it with state-of-the-art baseline models. The research methodology involves assessing the suggested approach using real-world energy demand information and conducting a comparative analysis with cutting-edge baseline models, including modified BP neural network (MBPNN), Neural Network based Genetic Algorithm (NNGA), and reinforcement learning and deep neural network (RLDNN). The IGPCA/CNN method outperforms other methods in energy prediction accuracy across short-term, medium-term, and long-term time scales. It achieves an MSE score of 0.564, lower than MBPNN, NNGA, and RLDNN. In medium-term prediction, it achieves an MSE score of 0.587, better than MBPNN, NNGA, and RLDNN. In long-term prediction, it achieves an MSE score of 0.629, lower than MBPNN and RLDNN. Further analysis and validation experiments are needed to ensure robustness and generalizability. Comparing the method with other state-of-the-art approaches can provide a comprehensive understanding of its superiority. The performance of the models is evaluated based on reliability and effectiveness in energy demand forecasting. The major conclusion of this study is that the proposed approach outperforms the initial models in accurately forecasting energy demand. The combination of DNNs and the improved Giza pyramid construction methodology results in enhanced performance, demonstrating superior reliability and effectiveness compared to other models. The study highlights the significance of accurate energy demand prediction for optimizing energy systems and reducing costs.

Deep neural network (DNN) can extract high-dimensional feature of images for computer vision tasks including Optical Coherence Tomography (OCT) images classification. However, OCT images are usually processed by DNN just like natural images, thus the performance of DNN is not satisfactory. We present an end-to-end DNN targeting OCT images classification. Considering the characteristic of OCT images, we introduce attention mechanism into classifier to extract more specific feature of OCT images. Our network demonstrates its capacity to enhance the features that represent the disease region. Our method achieves the state-of-the-art performance with average accuracy of 99.5% and F1-score of 0.995 on the OCT images dataset.

Where is my attention? An explainable AI exploration in water detection from SAR imagery

White Matter Hyperintensities (WMH) are imaging biomarkers which indicate cerebral microangiopathy, a risk factor for stroke and vascular dementia. When training Deep Neural Networks (DNN) to segment WMH, data pooling may be used to increase the training dataset size. However, it is not yet fully understood how pooling of heterogeneous data influences the segmentation performance. In this contribution, we investigate the impact of sampling ratios between different datasets with varying data quality and lesion volumes. We observe systematic changes in DNN performance and segmented lesion volume depending on the sampling ratio. If properly chosen, a single DNN can accurately segment and quantify both large and small lesions on different quality test data without loss of performance compared with a specialized DNN.

The recent success of brain-inspired deep neural networks (DNNs) in solving complex, high-level visual tasks has led to rising expectations for their potential to match the human visual system. However, DNNs exhibit idiosyncrasies that suggest their visual representation and processing might be substantially different from human vision. One limitation of DNNs is that they are vulnerable to adversarial examples, input images on which subtle, carefully designed noises are added to fool a machine classifier. The robustness of the human visual system against adversarial examples is potentially of great importance as it could uncover a key mechanistic feature that machine vision is yet to incorporate. In this study, we compare the visual representations of white- and black-box adversarial examples in DNNs and humans by leveraging functional magnetic resonance imaging (fMRI). We find a small but significant difference in representation patterns for different (i.e. white- versus black- box) types of adversarial examples for both humans and DNNs. However, human performance on categorical judgment is not degraded by noise regardless of the type unlike DNN. These results suggest that adversarial examples may be differentially represented in the human visual system, but unable to affect the perceptual experience.

3D Facial Expression Recognition (FER) is an active research topic due to its multi-fields human machine applications. We expose in this paper a new approach for Data Augmentation (DA) in order to improve 3D FER using Deep Neural Networks (DNN). Our main contribution consists in using the Coherent Point Drift (CPD) non-rigid registration to generate additional 3D facial data conveying various expressions mainly the prototypical expressions: Happiness, Sadness, Fear, Surprise, Disgust, and Anger. We start by choosing a set of different references defined by arbitrarily selected neutral faces. We apply then the CPD non-rigid registration between each selected neutral face and each 3D facial model conveying various expressions from the whole BU-3DFE database. Thus, we augment the dataset by a factor equal to the used references. Afterwards, we estimate the 3D elastic deformation between the reference (3D neutral face) and the target (3D face with expression) in order to generate consequently various 3D expressions by switching the reference and the target within the registration process. Afterwards, we gather the produced 3D expressions to increase the size of our dataset. Finally, we exploit a DNN architecture to evaluate our proposed DA method. The used DA is effective and increases our DNN performance. Experimental results operated on the whole BU-3DFE database shows promising results reaching 94.88%.

Deep Neural Networks (DNNs) are able to solve many complex tasks with less engineering effort and better performance.However, these networks often use data for training and evaluation without investigating its representation, i.e. the form of the used data.In the present paper, we analyze the impact of data representations on the performance of DNNs using energy time series forecasting.Based on an overview of exemplary data representations, we select four exemplary data representations and evaluate them using two different DNN architectures and three forecasting horizons on real-world energy time series.The results show that, depending on the forecast horizon, the same data representations can have a positive or negative impact on the accuracy of DNNs.

Deep neural networks (DNNs) are tremendously applied in artificial intelligence field. While the performance of DNNs is continuously improved by more complicated and deeper structures, the feasibility of deployment on embedded system remains as a critical problem. Neuromorphic system designs based on resistive random-access memory (ReRAM) provide an opportunity for power-efficient DNN employment. However, it encounters the challenge of limited programming resolution. A quantized training method is proposed in this paper to enhance the performance of neuromorphic systems based on ReRAM. Different from previous methods in which a dedicated regularization term is used in loss function to constrain parameter distribution, our quantized training method deals with training and quantization at the same time to alleviate the impact of limited parameter precision. Models with discrete parameters obtained after training can be directly mapped onto ReRAM devices. We implement experiments on image recognition tasks using multi-layer perceptron (MLP) and convolution neural network (CNN). The results verify that the quantized training method can approximate the accuracy of full-precision training, e.g., a two-layer MLP based on binary ReRAM only decreases the classification accuracy by 0.25% for MNIST dataset. In addition, we carefully investigate and present the importance of layer size under the ReRAM's low programming resolution, the different parameter resolution demands for convolution layer and fully connected layer, and system robustness to ReRAM variations after quantized training. The codes are available at https://github.com/qingyangqing/quantized-rram-net.

FPGAs have been rapidly adopted for acceleration of Deep Neural Networks (DNNs) with improved latency and energy efficiency compared to CPU and GPU-based implementations. High-level synthesis (HLS) is an effective design flow for DNNs due to improved productivity, debugging, and design space exploration ability. However, optimizing large neural networks under resource constraints for FPGAs is still a key challenge. In this paper, we present a series of effective design techniques for implementing DNNs on FPGAs with high performance and energy efficiency. These include the use of configurable DNN IPs, performance and resource modeling, resource allocation across DNN layers, and DNN reduction and re-training. We showcase several design solutions including Long-term Recurrent Convolution Network (LRCN) for video captioning, Inception module for FaceNet face recognition, as well as Long Short-Term Memory (LSTM) for sound recognition. These and other similar DNN solutions are ideal implementations to be deployed in vision or sound based IoT applications.