DNN Feature Map Compression using Learned Representation over GF(2)

Deep neural network clustering is superior to the conventional clustering methods due to deep feature extraction and nonlinear dimensionality reduction. Nevertheless, deep neural network leads to a rough representation regarding the inherent relationship of the data points. Therefore, it is still difficult for deep neural network to exploit the effective structure for direct clustering. To address this issue, we propose a robust embedded deep K-means clustering (RED-KC) method. The proposed RED-KC approach utilizes the δ-norm metric to constrain the feature mapping process of the auto-encoder network, so that data are mapped to a latent feature space, which is more conducive to the robust clustering. Compared to the existing auto-encoder networks with the fixed prior, the proposed RED-KC is adaptive during the process of feature mapping. More importantly, the proposed RED-KC embeds the clustering process with the auto-encoder network, such that deep feature extraction and clustering can be performed simultaneously. Accordingly, a direct and efficient clustering could be obtained within only one step to avoid the inconvenience of multiple separate stages, namely, losing pivotal information and correlation. Consequently, extensive experiments are provided to validate the effectiveness of the proposed approach.

Scene text detection has attracted great interest from the computer vision and pattern recognition communities since text information plays an important role in image indexing and scene understanding. Deep neural networks have become popular for the task of scene text detection, especially for their ability to learn strong text features. However, existing deep learning based state-of-the-art scene text detection methods detect texts only from a single feature map which is unable to capture semantic information at all scales. In this paper, we propose a novel deep learning based model that leverages the pyramid structure of feature maps for accurate scene text detection. We also design a deep convolutional neural network model for non-maximum suppression. In addition, we develop a novel loss function and training method for end-to-end training. The experimental results validate that our end-to-end system is simple, fast, and achieves high accuracy on standard datasets, namely, ICDAR 2015 and MSRA-TD500. We also create a dataset for scene text detection.

Feature mapping using far-field microphones for distant speech recognition

Video hyperlinking represents a classical example of multimodal problems. Common approaches to such problems are early fusion of the initial modalities and crossmodal translation from one modality to the other. Recently, deep neural networks, especially deep autoencoders, have proven promising both for crossmodal translation and for early fusion via multimodal embedding. A particular architecture, bidirectional symmetrical deep neural networks, have been proven to yield improved multimodal embeddings over classical autoencoders, while also being able to perform crossmodal translation. In this work, we focus firstly at evaluating good single-modal continuous representations both for textual and for visual information. Word2Vec and paragraph vectors are evaluated for representing collections of words, such as parts of automatic transcripts and multiple visual concepts, while different deep convolutional neural networks are evaluated for directly embedding visual information, avoiding the creation of visual concepts. Secondly, we evaluate methods for multimodal fusion and crossmodal translation, with different single-modal pairs, in the task of video hyperlinking. Bidirectional (symmetrical) deep neural networks were shown to successfully tackle downsides of multimodal autoencoders and yield a superior multimodal representation. In this work, we extensively tests them in different settings, with different single-modal representations, within the context of video-hyperlinking. Our novel bidirectional symmetrical deep neural networks are compared to classical autoencoders and are shown to yield significantly improved multimodal embeddings that significantly (alpha=0.0001) outperform multimodal embeddings obtained by deep autoencoders with an absolute improvement in precision at 10 of 14.1% when embedding visual concepts and automatic transcripts and an absolute improvement of 4.3% when embedding automatic transcripts with features obtained with very deep convolutional neural networks, yielding 80% of precision at 10.

The demand on devices and systems empowered by biometric verification and identification mechanisms has been increasing in recent years as they have become a significant part of our lives. Palm vein biometric is an emerging technology that has drawn considerable attention from researchers and scientists over the last decade. In the present work, a novel feature extraction methodology named GPWLD combing the Gabor features with positional Weber's local descriptor (PWLD) features is proposed. WLD is a highly representative micro-pattern descriptor that performs well against noise and illumination changes in images. However, it lacks the ability to capture the vein pattern features at different orientations. Moreover, its descriptor packs the local information content into a single histogram that does not take the spatial locality of micro-patterns into consideration. To solve these two issues, firstly, the palm vein image is passed through Gabor filter with different orientations to capture the salient rotational features found in the output feature maps. Secondly, the spatiality is achieved by uniformly dividing each feature map into several blocks. Next, Weber's law feature descriptor (WLD) histogram is computed for each block in every feature map. Finally, these histograms are concatenated to compose the final feature vector. Due to the high dimensionality of the final feature vector, Principal component analysis (PCA) algorithm is utilized for feature size reduction. In the classification stage, a deep neural network (DNN) comprising an optimized stacked autoencoder (SAE) with Bayesian optimization and a softmax layer is used. Optimization of the SAE is carried out by using Bayesian optimization to find the optimal SAE structure and the options of the training algorithm. The Experimental results verify the discriminative power of the extracted features and the accuracy of the proposed DNN. For both Identification and verification experiments, the proposed method attains higher identification rate and lower EER than state-of-the-art methods.

At present, many cities have established sophisticated metro transportation systems, and metro passenger traffic accounts for about 50 to 80 percent of the total passenger traffic. Unfortunately, there are hundreds of incidents of train delays and even casualties caused by foreign objects in metros of various cities every year. The detection of foreign objects between metro doors and Platform Screen Doors (PSDs) has become an urgent problem to be solved. The main contribution of this paper is that this is the first attempt to use deep neural networks to solve the problem of foreign object detection between the PSDs and metro doors. In this paper, we first introduce the object detection framework called You Only Look Once (YOLO series) and Single Shot MultiBox Detector (SSD). We further explore the performance of YOLO v3 and SSD by implementing their models to do the foreign object detecting task in our dataset (from real-world data). The experimental results demonstrate that deep neural networks achieve good detection accuracy in the task of foreign object detection between the PSDs and metro doors. In addition, the deep neural network can also give the category information of foreign objects, which greatly reducing the time of artificial judgment and relieving traffic pressure. We hope our paper will serve as a solid baseline and help ease future research in foreign object detection between the PSDs and metro doors.

Motivated by the development of deep convolution neural networks (DCNNs), the aircraft detection from remote sensing images has gained tremendous progress. However, due to complex background and multi-scale characteristics, it remains a challenge in remote sensing detection. In this paper, we propose a two-stage aircraft detection method based on deep neural networks, which integrates Deconvolution operation with Position Attention mechanism (DPANet). Specifically, considering that remote sensing images are taken from the top-down perspective, which leads to significant external structural characteristic, we introduce a deconvolution module to capture the external structural feature representation of aircraft during the feature map generation process. Moreover, aiming at reducing the error detections caused by complex background in remote sensing, we propose a position attention module in the second stage. By calculating the feature similarity between any two pixels of the target feature map, DPANet can extract the complicated internal structure feature representation of aircraft, which improve the ability to distinguish background and aircraft. By integrating the deconvolution and position attention modules, DPANet can provide better representation ability for the structural characteristic of aircraft in remote sensing images. Experimental results show that the proposed method can effectively reduce the error detections and improve the accuracy of the aircraft detection.

Convolutional neural networks (CNNs) are widely used and have achieved great success in computer vision and speech processing applications. However, deploying the large-scale CNN model in the embedded system is subject to the constraints of computation and memory. An optimized block-floating-point (BFP) arithmetic is adopted in our accelerator for efficient inference of deep neural networks in this paper. The feature maps and model parameters are represented in 16-bit and 8-bit formats, respectively, in the off-chip memory, which can reduce memory and off-chip bandwidth requirements by 50% and 75% compared to the 32-bit FP counterpart. The proposed 8-bit BFP arithmetic with optimized rounding and shifting-operation-based quantization schemes improves the energy and hardware efficiency by three times. One CNN model can be deployed in our accelerator without retraining at the cost of an accuracy loss of not more than 0.12%. The proposed reconfigurable accelerator with three parallelism dimensions, ping-pong off-chip DDR3 memory access, and an optimized on-chip buffer group is implemented on the Xilinx VC709 evaluation board. Our accelerator achieves a performance of 760.83 GOP/s and 82.88 GOP/s/W under a 200-MHz working frequency, significantly outperforming previous accelerators.

Quantization Friendly MobileNet (QF-MobileNet) Architecture for Vision Based Applications on Embedded Platforms

Deep Neural Networks (DNNs) have made significant improvements to reach the desired accuracy to be employed in a wide variety of Machine Learning (ML) applications. Recently the Google Brain's team demonstrated the ability of Capsule Networks (CapsNets) to encode and learn spatial correlations between different input features, thereby obtaining superior learning capabilities compared to traditional (i.e., non-capsule based) DNNs. However, designing CapsNets using conventional methods is a tedious job and incurs significant training effort. Recent studies have shown that powerful methods to automatically select the best/optimal DNN model configuration for a given set of applications and a training dataset are based on the Neural Architecture Search (NAS) algorithms. Moreover, due to their extreme computational and memory requirements, DNNs are employed using the specialized hardware accelerators in IoT-Edge/CPS devices. In this paper, we propose NASCaps, an automated framework for the hardware-aware NAS of different types of DNNs, covering both traditional convolutional DNNs and CapsNets. We study the efficacy of deploying a multi-objective Genetic Algorithm (e.g., based on the NSGA-II algorithm). The proposed framework can jointly optimize the network accuracy and the corresponding hardware efficiency, expressed in terms of energy, memory, and latency of a given hardware accelerator executing the DNN inference. Besides supporting the traditional DNN layers, our framework is the first to model and supports the specialized capsule layers and dynamic routing in the NAS-flow. We evaluate our framework on different datasets, generating different network configurations, and demonstrate the tradeoffs between the different output metrics. We will open-source the complete framework and configurations of the Pareto-optimal architectures at https://github.com/ehw-fit/nascaps.

Single Shot MultiBox Detector (SSD) method using multi-scale feature maps for object detection, showing outstanding performance in object detection task. However, as a one-stage detection method, it’s difficult for SSD methods to quickly notice significant areas of objects in the image. In the SSD network structure, feature maps of different scales are used to independently predict object, and there is a lack of interaction between low-level feature maps and high-level feature maps. In this paper we propose an enhanced SSD method using interactive multi-scale attention features (MA-SSD). Our method uses the attention mechanism to generate attention features of multiple scales and adds it to the original detection branch of the SSD method, which effectively enhances the feature representation ability and improves the detection accuracy. At the same time, the feature of different detection scales interacts with each other, and all the detection branches in our method have a parallel structure, which ensures the detection efficiency. Our proposed method achieves competitive performance on the public dataset PascalVOC.

Currently, deep learning (DL) algorithms perform best in image classification and object detection tasks. Consequently, they are frequently used to address most problems involving computer vision. In this sense, the pervasive presence of smartphones and IoT devices has created a need to make this artificial intelligence portable. Given that deep neural network (DNN) models consist of millions of parameters, emerging research efforts have focused on enabling offline DL execution on low-resource devices, such as within the TinyML paradigm. This study analyzes the state-of-the-art of DL embedded in smartphones to develop an app for children that can recognize 3D geometric shapes without needing an internet connection. Alongside a systematic literature review, we conduct experiments with several pre-trained and lightweight models, which were subsequently evaluated using parametric statistical tests. While DL on smartphones is an underexplored area, it is expected to evolve significantly. Among the classification models tested, DenseNet169 demonstrated the highest accuracy (81%), whereas the MobileNet variants were faster and closer to real-time performance (30 FPS). In detection tasks, the EfficientDet-Lite and YOLOv8 models were evaluated, with EfficientDet-Lite being less accurate but faster (50 ms) compared to YOLOv8 (4 seconds). Although the field of DL on smartphones still requires further development, current lightweight models and frameworks offer significant opportunities for practical application.

This paper implements a new text region recognition algorithm that can accurately localize image text regions in natural image with complex background. The method is mainly based on the anchor mechanism of the faster R-CNN, taking into account the special features of the text area relative to other object detect tasks, so as to convert the text area detection task in the image into a general object detection task for the small area text. In this way, we can detect the text proposal directly in the convolutional feature map of the neural network, and it can simultaneously predict the text/non- text score of the proposal and the coordinates of each proposal in the image. Then we propose a text line construction algorithm that can combine the text regions into complete text line blocks, thus greatly improving the accuracy and reliability of our text detection model. Our text detector also works accurately in multi-scale and multi-lingual text detection tasks. It achieves 0.86 F-measure and 0.78 F-measure on the ICDAR 2011 and ICDAR 2013 benchmarks, which also confirms the accuracy of our model.

As the core technologies represented by Internet of Things (IoT) technology and computer vision technology booming, object detection algorithms based on deep learning have received high attention in various fields. Object detection is not only a core problem in the field of computer vision, but also a prerequisite and basis for many computer vision tasks, and has important applications in the fields of autonomous driving and video surveillance. The task of object detection is to find all the targets of interest in an image and determine their category and location. Object detection includes two tasks: object classification and object localization. However, object classification focuses on the most discriminative part of the feature map, and object localization requires a feature map focusing on the whole region of the object. Therefore, in order to improve the detection accuracy, we propose a CBAM-RetinaNet network model.CBAM (Convolutional Block Attention Module) is a lightweight convolutional attention module, which combines channel and spatial attention mechanism modules. The channel attention module focuses on the meaningful information in the input image, which would lead to an increase in the accuracy of object classification. The spatial attention module focuses on the location information of the object, which may improve the accuracy of object localization. The experiment results show that the proposed achieves better object detection performance on the PASCAL VOC2007 dataset, and the accuracy is 2% higher than that of the RetinaNet model, which has good experimental results.

Attention is an important field to explore the importance of each convolutional kernel channel/weight. The existing attention methods mostly use the Squeeze-and-Excitation (SE) technology to extract the global nonlinear feature vectors as the weights of corresponding feature maps. However, the pooling operators and fully-connected layers used in SE technology extract global features at the cost of much valuable information loss and the parameter amount increase. Actually, the feature map containing full information is a ready-made and better attention for other feature maps in the same layer. Simultaneously the products of feature maps will bring powerful non-linearity. Seeing this, Kernel Product (KP) technology is proposed to simply get useful nonlinear attention. To verify the effectiveness of KP, the proposed KP module is employed on Selective Kernel Networks (SKNets) to take the place of the original SE technology. The variety of SKNets is called Kernel Product Networks (KPNets) in this paper. In addition, identity mapping is used to solve the non-convergence problem in very deep neural networks. The KPNets are evaluated on ImageNet-1k, CIFAR-10, and CIFAR-100. The experiment results show that KPNets outperform many state-of-the-art methods and get a similar but more efficient performance than its SKNets with counterpart.