Fully automated carbonate petrography using deep convolutional neural networks

Nowadays Convolutional Neural Networks (CNNs) are being employed in a wide range of industrial technologies for a variety of sectors, such as medical, automotive, aviation, agriculture, space, etc. This paper reviews the state-of-the-art in both the field of CNNs for image classification and object detection and Autonomous Driving Systems (ADSs) in a synergetic way. Layer-based details of CNNs along with parameter and floating-point operation number calculations are outlined. Using an evolutionary approach, the majority of the outstanding image classification CNNs, published in the open literature, is introduced with a focus on their accuracy performance, parameter number, model size, and inference speed, highlighting the progressive developments in convolutional operations. Results of a novel investigation of the convolution types and operations commonly used in CNNs are presented, including a timing analysis aimed at assessing their impact on CNN performance. This extensive experimental study provides new insight into the behaviour of each convolution type in terms of training time, inference time, and layer level decomposition. Building blocks for CNN-based object detection are also discussed, such as backbone networks and baseline types, and then representative state-of-the-art designs are outlined. Experimental results from the literature are summarised for each of the reviewed models. This is followed by an overview of recent ADSs related works and current industry activities, aiming to bridge academic research and industry practice on CNNs and ADSs. Design approaches targeted at solving problems of automakers in achieving real-time implementations are also proposed based on a discussion of design constraints, human vs. machine evaluations and trade-off analysis of accuracy vs. size. Current technologies, promising directions, and expectations from the literature on ADSs are introduced including a comprehensive trade-off analysis from a human-machine perspective.

Object detection is a major branch and fundamental task in computer vision, aiming to localize, identify and classify even the smallest objects of interest in images. Features can be extracted efficiently by deep convolutional neural networks (CNNs) as the backbone for real-time or near real-time object detection performance than the hand-crafted-based traditional methods. In the past few years, the advent of transformer-based models with robust self-attention mechanisms has not only raised object detection performance to a higher level but has also enabled it to produce excellent results. Many object detection tasks in the real world require that 3D information about the object be obtained, thus strengthening active research in 3D object detection. However, the algorithms for detecting 3D objects are not easy to propagate in real-world applications due to many factors, making reconstruction of 2D object detection algorithms to 3D object detection algorithms the suitable alternative. Therefore, we review the evolution of 2D object detection algorithms for digital imaging applications, focusing on their developments, models, applications, datasets, evaluation metrics, strengths and weaknesses, for better understanding of their landmarks and contributions to the advancement of the field.

Convolutional neural networks (CNNs) have gained a huge attention for real-world artificial intelligence (AI) applications such as image classification and object detection. On the other hand, for better accuracy, the size of the CNNs’ parameters (weights) has been increasing, which in turn makes it difficult to enable on-device CNN inferences in resource-constrained edge devices. Though weight pruning and 5-bit quantization methods have shown promising results, it is still challenging to deploy large CNN models in edge devices. In this paper, we propose an encoding and hardware-based decoding technique which can be applied to 5-bit quantized weight data for on-device CNN inferences in resource-constrained edge devices. Given 5-bit quantized weight data, we employ arithmetic coding with range scaling for lossless weight compression, which is performed offline. When executing on-device inferences with underlying CNN accelerators, our hardware decoder enables a fast in-situ weight decompression with small latency overhead. According to our evaluation results with five widely used CNN models, our arithmetic coding-based encoding method applied to 5-bit quantized weights shows a better compression ratio by 9.6× while also reducing the memory data transfer energy consumption by 89.2%, on average, as compared to the case of uncompressed 32-bit floating-point weights. When applying our technique to pruned weights, we obtain better compression ratios by 57.5×–112.2× while reducing energy consumption by 98.3%–99.1% as compared to the case of 32-bit floating-point weights. In addition, by pipelining the weight decoding and transfer with the CNN execution, the latency overhead of our weight decoding with 16 decoding unit (DU) hardware is only 0.16%–5.48% and 0.16%–0.91% for non-pruned and pruned weights, respectively. Moreover, our proposed technique with 4-DU decoder hardware reduces system-level energy consumption by 1.1%–9.3%.

The performance of convolutional neural networks (CNNs) in image classification and object detection has been remarkable, even though they contain millions and billions of parameters. This over‐parameterization of CNN makes them both memory‐intensive and computationally complex and exhaustive. This greatly hinders the application of CNNs in resource‐constrained environments such as Internet of things (IoT) and edge devices. This poses a critical challenge for CNNs in deploying these powerful computer vision tools to mobile devices, which needs immediate attention. In this study, we have proposed a novel technique based on non‐convex optimization, max‐norm regularization. The max‐norm will structurally prune the number of parameters without compromising the model's performance. The proximal gradient descent algorithm is used for network optimization while using this non‐convex regularizer. The max‐norm is combined with the channel pruning to achieve more sparse CNN networks. Later, the pruned network can be easily deployed in the resource‐constrained application environment. The proposed technique is tested on several benchmark datasets for validation. In addition, in this study, the sparsified CNNs are used for biomedical image analysis using the BRAIN MRI dataset. This sparsely trained CNN model can later serve as the best lightweight model applicable in the IoT healthcare sector for detecting and classifying three types of brain tumours, one of the most life‐threatening diseases whose early detection can save the costly lives of human beings. This is the first paper to propose the novel max‐norm regularizer to enforce sparse learning through CNNs. The paper provides a detailed analysis of convex and non‐convex regularizers before presenting the proposed novel max‐norm regularizer. Finally, the paper compares the proposed max‐norm regularizer with existing regularization methods using state‐of‐the‐art CNN models.

Brain tumor implies development of strange cells in brain. In cutting edge stages, brain tumor is most risky infection which can't be relieved. Thus, it ought to be identified in the beginning phases with the assistance of MRI (Magnetic Resonance Image). So, there will be more changes to the patient to endure. Quite possibly, the most functional and significant technique is to utilize Deep Neural Network (DNN). In this paper, a Deep Convolutional Neural Network (DCNN) based on VGG16 model has been developed to identify a tumor through brain Magnetic Resonance Imaging (MRI) dataset. In the clinical field, the strategies of machine learning (ML) and data mining hold a critical stand. It is effectively used to achieve the efficiency and exact location of tumor. The proposed technique involves automatic segmentation method based on Deep Convolution neural network (DCNN). It is layer based segmentation and classification technique. Different levels are engaged with the proposed technique, first step is data collection and then pre-processing & average filtering is done, after that segmentation, feature extraction and classification via DCNN is performed. In this paper, a fresh technique for classification of brain MRI images has been proposed using DCNN based softmax Classifier. The proposed system has been compared with the existing ones. The results are really encouraging as the loss is reduced and accuracy is increased. The proposed system achieved the training accuracy of 98.9% and validation accuracy of 100%. The training loss is reduced up to 0.0230 and the validation loss reduced up to 0.0109.

The convolution neural networks have revolutionized the computer vision domain. It has proven to be a dominant technology to carry out tasks such as image classification, semantic segmentation, and object detection. The convolution neural networks surpass the performance of the existing algorithms such as SIFT, HOG, etcetera. Where, instead of manually engineering the features, supervised learning help to learn the essential low-level and high-level features necessary for classifications. The convolution neural networks have become a popular tool to counter computer vision problems. However, it is computationally, and memory intensive to train and deploy the network because of the model size of a deep convolution neural networks. However, the research in the field of design space exploration (DSE) of neural networks and compression techniques to develop compact architectures, have made convolution neural networks memory and computationally efficient. These techniques have also improved the feasibility of convolution neural network for deployment on embedded targets. The paper explores the concept of compact convolution filters to reduce the number of parameters in a convolution neural network. The intuition behind the approach is that replacing convolution filters with a stack of compact convolution filters helps in developing a compact architecture with competitive accuracy. This paper explores the fire module a compact convolution filter and proposes a method of recreating a state-of-the-art architecture VGG-16 using the fire modules to develop a compact architecture, which is further trained on the CIFAR-10 dataset and deployed on a real-time embedded platform known as Bluebox 2.0 by NXP using RTMaps software framework.

Purpose This study aims to automate the visual inspection of piling sheets in water channel construction using artificial intelligence (AI). By employing image classification and object detection techniques, the research focuses on extracting and analysing geometric features to enhance the accuracy and efficiency of the inspection process. It also addresses key challenges associated with the unique characteristics of construction materials and the limited variability of available inspection datasets. Design/methodology/approach Convolutional neural networks (CNNs) with varying complexities are employed for image classification, across four and six classes, and for object detection of piling sheets in water channel environments. A dataset provided by Witteveen + Bos is preprocessed to generate training sets, and the CNN architectures are optimized for enhanced performance. The accuracy and efficiency of the proposed models are evaluated and compared against traditional manual inspection methods. Findings The AI-driven approach significantly reduces processing time, evaluating 40,000 images in just 11.9 h, compared to approximately one month using manual assessment. The 4-class classification model achieves an accuracy of 96%, while the 6-class model attains 72%. The object detection model produces a mean average precision (mAP) of 79%. These results meet the performance standards set by the Dutch company Witteveen + Bos, which demonstrate the effectiveness of AI in automating the inspection of piling sheets. Originality/value This study introduces a novel AI-based approach for assessing piling sheets, demonstrating substantial improvements over traditional inspection methods. It introduces a systematic evaluation of various CNN architectures and hyperparameters to optimize the models specifically for piling sheet inspection rather than relying on off-the-shelf solutions. The use of CNNs for both image classification and object detection adheres to relevant Dutch engineering standards. Notably, the reduction in processing time, from one month to around 12 h, represents a major advancement in the efficiency of civil engineering inspections.

In recent years, Convolutional Neural Networks (CNNs) have been widely successful for numerous computer vision tasks using 2D color and 3D point data, such as object detection and classification, and 3D pose estimation. Despite these successes, it comes with a price --- CNNs require a vast amount of labeled training data to be successful. Many training datasets are now available, however, they are mostly limited to general objects in the public domain such as people, cars, buildings, fruit, and more. Thus, not all fields have sufficient training data. Even if an adequate number of data samples can be built into a dataset, manually collecting and labeling the training data remains a laborious task. Several approaches to reduce or eradicate the need for training data have been proposed by researchers. The two most popular approaches are training with synthetic data and training on generalized object features. Training with synthetic data refers to the use of computer graphics to generate training data. While this approach has demonstrated success, it comes with its own challenges. In contrast, training a CNN on generalized object features identifies specific local 3D features or control points on objects. Local 3D features and their descriptors have proven in the past to be universally applicable and effective for general object detection, and thus, this method allows for a generalized CNN. A generalized CNN refers to a network architecture that, after initial training, can be specialized for a new task with a minimum amount or no new training data. This dissertation investigates an encoder-decoder architecture and determines the capabilities of it to generalize the CNN architecture for object detection tasks in point cloud data. A CNN is trained to recognize 3D features and demonstrates, without retraining, its successful applicability for point cloud object detection using disparate data. Additionally, the geometric consistency of the descriptors is evaluated to obtain further insight. The results demonstrate that the proposed 3D feature descriptors, and their increased geometric consistency contribute to the increased encoder-decoder architecture performance. In summary, this dissertation contributes a state-of-the-art, generalized, feature descriptor-based CNN architecture that can be transferred to different objects without retraining. Furthermore, it provides insight explaining the increased performance of the CNN compared to the state-of-the-art.

Effectively analysis of remote-sensing images is very important in many practical applications, such as urban planning, geospatial object detection, military monitoring, vegetation mapping and precision agriculture. Recently, convolutional neural network based deep learning algorithm has achieved a series of breakthrough research results in the fields of objective detection, image semantic segmentation and image classification, etc. Their powerful feature learning capabilities have attracted more attention and have important research value. In this article, firstly we have summarized the basic structure and several classical convolutional neural network architectures. Secondly, the recent research problems on convolutional neural network are discussed. Later, we summarized the latest research results in convolutional neural network based remote sensing fields. Finally, the conclusion has made on the basis of current issue on convolutional neural networks and the future development direction.

Most popular image classification methods mainly focus on classification ability rather than recognizing new things. However, human lay emphasis on cognition first and then classification, which is closely related to human memory system. Though many memory models have been proposed, they are studied in word list whereas the reports about natural images are still limited. This paper proposes a memory model for image recognition and classification based on convolutional neural network and Bayesian decision. First the image feature is extracted by convolutional neural network and stored in binary form. Then the representation, storage and retrieval processes of visual images are modeled. The test image feature vector is matched in parallel to the studied image vectors, and the likelihood values are calculated. Finally, the odd that the test image belongs to a new class is computed based on all likelihood values. If the odd value is greater than a certain threshold, the test image is regarded as new; otherwise, the Bayesian decision rule for image classification is performed. Experimental results on Caltech-101 and Caltech-256 datasets show that the proposed method can perform well in image recognition and classification tasks. And the hit probability of the method is higher than two typical methods, SRC and ELM, at present while the false alarm rate is far lower than them.

Abstract. In recent years, there has been a significant improvement in the detection, identification and classification of objects and images using Convolutional Neural Networks. To study the potential of the Convolutional Neural Network, in this paper three approaches are investigated to train classifiers based on Convolutional Neural Networks. These approaches allow Convolutional Neural Networks to be trained on datasets containing only a few hundred training samples, which results in a successful classification. Two of these approaches are based on the concept of transfer learning. In the first approach features, created by a pretrained Convolutional Neural Network, are used for a classification using a support vector machine. In the second approach a pretrained Convolutional Neural Network gets fine-tuned on a different data set. The third approach includes the design and training for flat Convolutional Neural Networks from the scratch. The evaluation of the proposed approaches is based on a data set provided by the IEEE Geoscience and Remote Sensing Society (GRSS) which contains RGB and LiDAR data of an urban area. In this work it is shown that these Convolutional Neural Networks lead to classification results with high accuracy both on RGB and LiDAR data. Features which are derived by RGB data transferred into LiDAR data by transfer learning lead to better results in classification in contrast to RGB data. Using a neural network which contains fewer layers than common neural networks leads to the best classification results. In this framework, it can furthermore be shown that the practical application of LiDAR images results in a better data basis for classification of vehicles than the use of RGB images.

Self-attention based models such as vision transformers (ViTs) have emerged as a very competitive architecture alternative to convolutional neural networks (CNNs) in computer vision. Despite increasingly stronger variants with ever higher recognition accuracies, due to the quadratic complexity of self-attention, existing ViTs are typically demanding in computation and model size. Although several successful design choices (e.g., the convolutions and hierarchical multi-stage structure) of prior CNNs have been reintroduced into recent ViTs, they are still not sufficient to meet the limited resource requirements of mobile devices. This motivates a very recent attempt to develop light ViTs based on the state-of-the-art MobileNet-v2, but still leaves a performance gap behind. In this work, pushing further along this under-studied direction we introduce EdgeViTs, a new family of light-weight ViTs that, for the first time, enable attention based vision models to compete with the best light-weight CNNs in the tradeoff between accuracy and on-device efficiency. This is realized by introducing a highly cost-effective local-global-local (LGL) information exchange bottleneck based on optimal integration of self-attention and convolutions. For device-dedicated evaluation, rather than relying on inaccurate proxies like the number of FLOPs or parameters, we adopt a practical approach of focusing directly on on-device latency and, for the first time, energy efficiency. Extensive experiments on image classification, object detection and semantic segmentation validate high efficiency of our EdgeViTs when compared to the state-of-the-art efficient CNNs and ViTs in terms of accuracy-efficiency tradeoff on mobile hardware. Specifically, we show that our models are Pareto-optimal when both accuracy-latency and accuracy-energy tradeoffs are considered, achieving strict dominance over other ViTs in almost all cases and competing with the most efficient CNNs. Code is available at https://github.com/saic-fi/edgevit .

Molecular image-based convolutional neural network for the prediction of ADMET properties

— In today's era, the challenge of object detection within laboratory spaces is paramount. The need for automated monitoring systems supporting object detection within laboratory settings, particularly for service robots, is crucial for scientific research and safety. Utilizing convolutional artificial neural networks has shown promising capabilities in object recognition. This paper proposes a novel method for detecting and localizing specific objects within laboratory environments using deep learning techniques. Employing a lightweight primary feature extractor allows for the identification of crucial object features without imposing excessive computational demands. Additionally, a depthwise separable convolution module is introduced, designed to capture essential components within multi-level convolutional patches specific to laboratory conditions. To address the challenges encountered within laboratory spaces, a new dataset is proposed, encompassing issues such as lighting variations, blurriness, occlusions, and complex backgrounds. Evaluation results indicate that the proposed model surpasses other lightweight object detection models in laboratory settings, offering high accuracy and efficiency. This model can be efficiently implemented on low-tier devices, ensuring real-time object detection and processing within laboratory environments. Keywords — object detection, service robot, convolutional neural networks, You Only Look Once version 5. Abstrak — Di era sekarang ini, tantangan deteksi objek di dalam ruang laboratorium menjadi sangat penting. Kebutuhan akan sistem pemantauan otomatis yang mendukung deteksi objek di dalam laboratorium, khususnya untuk robot servis, sangat penting untuk penelitian ilmiah dan keselamatan. Memanfaatkan jaringan saraf tiruan convolutional telah menunjukkan kemampuan yang menjanjikan dalam pengenalan objek. Makalah ini mengusulkan metode baru untuk mendeteksi dan melokalisasi objek tertentu di dalam lingkungan laboratorium menggunakan teknik pembelajaran mendalam. Menggunakan ekstraktor fitur utama yang ringan memungkinkan identifikasi fitur objek yang penting tanpa membebani komputasi yang berlebihan. Selain itu, modul konvolusi yang dapat dipisahkan secara mendalam juga diperkenalkan, yang dirancang untuk menangkap komponen penting dalam patch konvolusi multi-level yang spesifik untuk kondisi laboratorium. Untuk mengatasi tantangan yang dihadapi di dalam ruang laboratorium, dataset baru diusulkan, yang mencakup masalah seperti variasi pencahayaan, kekaburan, oklusi, dan latar belakang yang kompleks. Hasil evaluasi menunjukkan bahwa model yang diusulkan melampaui model pendeteksian objek ringan lainnya di lingkungan laboratorium, menawarkan akurasi dan efisiensi yang tinggi. Model ini dapat diimplementasikan secara efisien pada perangkat tingkat rendah, memastikan pendeteksian dan pemrosesan objek secara real-time di lingkungan laboratorium. Kata kunci — deteksi objek, robot layanan, jaringan saraf konvolusi, Anda Hanya Melihat Sekali versi 5.

With the wider adoption of edge computing services, intelligent edge devices, and high-speed V2X communication, compute-intensive tasks for autonomous vehicles, such as object detection using camera, LiDAR, and/or radar data, can be partially offloaded to road-side edge servers. However, data privacy becomes a major concern for vehicular edge computing, as sensitive sensor data from vehicles can be observed and used by edge servers. We aim to address the privacy problem by protecting both vehicles’ sensor data and the detection results. In this paper, we present vehicle–edge cooperative deep-learning networks with privacy protection for object-detection tasks, named vePOD for short. In vePOD, we leverage the additive secret sharing theory to develop secure functions for every layer in an object-detection convolutional neural network (CNN). A vehicle’s sensor data is split and encrypted into multiple secret shares, each of which is processed on an edge server by going through the secure layers of a detection network. The detection results can only be obtained by combining the partial results from the participating edge servers. We have developed proof-of-concept detection networks with secure layers: vePOD Faster R-CNN (two-stage detection) and vePOD YOLO (single-stage detection). Experimental results on public datasets show that vePOD does not degrade the accuracy of object detection and, most importantly, it protects data privacy for vehicles. The execution of a vePOD object-detection network with secure layers is orders of magnitude faster than the existing approaches for data privacy. To the best of our knowledge, this is the first work that targets privacy protection in object-detection tasks with vehicle–edge cooperative computing.