A real-time image recognition system using a global directional-edge-feature extraction VLSI processor

In this paper, a global feature extraction very large scale integration (VLSI) architecture for real-time object recognition is presented. To minimize the latency between capturing images and final recognition, the image sensor and feature extraction circuits are integrated on the same chip. The digital pixel sensor (DPS) configuration has been used because of its intrinsic compatibility with digital processing circuits. A block-readout architecture developed for DPS has been adopted for the massively parallel processing of image data. As a result, the directional edge filtering at each pixel site for local feature extraction is performed in a line-parallel manner. To eliminate trivial local features, the rank-order filter algorithm has been adapted for the processing and implemented in an efficient global feature extraction circuitry that can retain any given number of relatively more significant features in an image as essential features. This is accomplished in only 11 cycles. A proof-of-concept chip was designed in a 0.18 µm five-metal complementary metal–oxide–semiconductor (CMOS) technology, and the function of this VLSI processor was verified by circuit simulation.

Introduction: This paper presents a novel approach for classifying different types of steel rods using a combination of pre-trained convolutional neural networks (CNNs) and fuzzy logic systems. We utilize VGG16, ResNet50, and InceptionV3 models to extract features from grayscale images of steel rods, which are then classified using Support Vector Machines (SVM). Furthermore, we incorporate a fuzzy logic system to improve the final classification accuracy by combining the individual accuracies of each CNN model. Experimental results demonstrate the effectiveness of our approach in achieving high classification accuracy compared to traditional methods. Objectives: This research presents a novel multi-model approach for steel rod classification utilizing three distinct fuzzy logic systems: Fuzzy Recurrent Neural Network (RNN), Fuzzy Convolutional Neural Network (CNN), and Fuzzy Neural Network. The proposed system addresses the critical challenge of accurate defect detection and classification in steel manufacturing processes. By combining these complementary approaches, we achieve enhanced feature extraction capabilities and improved classification accuracy. Experimental results demonstrate that our multi-model system achieves a classification accuracy of 97.8%, surpassing single-model approaches by an average of 8.2%. The system successfully identifies surface defects, structural anomalies, and dimensional variations with high precision, making it suitable for real-time industrial applications. Methods: This paper introduces a novel approach for the feature extraction and classification of steel rods by integrating multiple models and utilizing fuzzy logic systems. The methodology leverages pre-trained convolutional neural networks (CNNs) for extracting comprehensive feature representations from steel rod images. These extracted features are then combined and integrated using fuzzy logic to enhance classification accuracy and reliability. By addressing the limitations of traditional manual inspection methods and single-model approaches, this integrated multi-model system offers a robust solution for real-time industrial implementation. The proposed system aims to improve the efficiency and effectiveness of quality control processes in steel rod manufacturing, ensuring consistent and accurate classification even under varying conditions and complexities. This innovative approach paves the way for more advanced and adaptable automated inspection systems in the industry. Results: The paper aims to develop an integrated multi-model system that combines three distinct fuzzy logic approaches to enhance feature extraction capabilities through complementary methodologies. By improving the classification accuracy and reliability, the system addresses the limitations of existing methods. The goal is to create a robust and adaptable system that is suitable for real-time industrial implementation for steel rod analysis and classification. This comprehensive approach leverages the strengths of multiple models and fuzzy logic techniques, ensuring accurate and efficient classification processes, even under complex and varying conditions. The proposed multi model system is designed to meet the stringent quality control requirements of industrial applications, ultimately improving productivity and product quality through reliable automated inspection and classification for steel rod with accuracy of 97.80%, precision of 97.30% and recall 97.50%. Conclusions: This paper presents a novel approach for the classification of steel rods using a combination of pre-trained CNN models and fuzzy logic systems. The experimental results demonstrate the effectiveness of our method in achieving high classification accuracy. This research demonstrates the effectiveness of a multi-model fuzzy logic approach for steel rod classification. The system achieves significant improvements over single-model approaches while maintaining real-time processing capabilities. Our approach provides a foundation for further research and development in automated industrial image classification.

There are two fundamental aspects in the design of the real-time systems: meeting real-time constraints and reliability. These two aspects are in conflict and are usually researched separately. To provide for fault-tolerance, there is the need for a greater execution time; but on the other hand, real-time systems require tasks to be completed by their deadlines. There is a clear and urgent need for mechanisms that are able to integrate the real-time and the reliability requirements. So, it is necessary to address the real-time and fault-tolerance requirements together during the design of the system. This would make the development of reliable real-time systems simple and efficient. Our approach is based on the idea that if the system is adaptable then reliability is easier to achieve. We have developed a mechanism, known as real-time recoverable action (RTR-action), which integrates the real-time and the reliability requirements.

Digital Image-based Elasto-Tomography (DIET) is an emerging technology for non-invasive breast cancer screening. This technology actuates breast tissue and measures the surface motion using digital imaging technology. The internal distribution of stiffness is then reconstructed using Boundary Element or Finite Element Methods (FEM or BEM). However, obtaining accurate imaging at high frequency and high resolution in terms of numbers of pixels is challenging if enough accuracy is to be obtained in the motion sensing to deliver a useful result. The overall focus of such mechatronic and digitally centred systems is on providing a low-cost, radiation dose-free and portable screening system capable of screening numerous patients per day — in direct contrast to current low throughput, non-portable and high cost x-ray and MRI based approaches. Thus, DIET technology relies on obtaining high resolution images of a breasts surface under high frequency actuation, typically in the range of 50–100Hz. Off-the-shelf digital cameras and imaging elements are unable to capture images directly at these speeds. A method is presented for obtaining the required high speed image capture at a resolution of 1280×1024 pixels and actuation frequency of 100Hz. The prototype apparatus presented uses two imaging sensors in combination with frame grabbers and a dSpace™ control system, to produce an automated image capture system. The system integrates a precision controlled strobe lighting system to selectively capture sinusoids at different points in the sinusoidal cycle of response. The final working system produced images that enabled effective 3D motion tracking of the surface of a silicon phantom actuated at 100Hz. The surface of the phantom was strobed at pre-selected phases from 0 to 360 degrees, and an image was captured for each phase. The times at which image capture occurred were calculated for a phase lag increment of 10 degrees resulting in an image effectively every 0.00028s for the actuator cycle of 0.01s. The comparison of the actual trigger times and pre-selected ideal trigger times gave a mean absolute error of 1.4%, thus demonstrating the accuracy of the final system. Final validation is performed using this system to track motion in a silicon gel phantom. The motion is tracked accurately using a novel Euclidean Invariant signature method. Both cameras delivered similar results with over 90% of points tracked to within 1–2%. This level of accuracy confirms the ability to effectively accurately reconstruct the stiffness as validated in other related studies.

We transform an LCD into a display that supports both 2D multi-touch and unencumbered 3D gestures. Our BiDirectional (BiDi) screen, capable of both image capture and display, is inspired by emerging LCDs that use embedded optical sensors to detect multiple points of contact. Our key contribution is to exploit the spatial light modulation capability of LCDs to allow lensless imaging without interfering with display functionality. We switch between a display mode showing traditional graphics and a capture mode in which the backlight is disabled and the LCD displays a pinhole array or an equivalent tiled-broadband code. A large-format image sensor is placed slightly behind the liquid crystal layer. Together, the image sensor and LCD form a mask-based light field camera, capturing an array of images equivalent to that produced by a camera array spanning the display surface. The recovered multi-view orthographic imagery is used to passively estimate the depth of scene points. Two motivating applications are described: a hybrid touch plus gesture interaction and a light-gun mode for interacting with external light-emitting widgets. We show a working prototype that simulates the image sensor with a camera and diffuser, allowing interaction up to 50 cm in front of a modified 20.1 inch LCD.

ABSTRACTLung cancer causes a high mortality rate in the world than any other cancers. That can be minimised if the symptoms and cancer cells have been detected early. One of the techniques used to detect lung cancer is by computed tomography (CT) scan. CT scan images have been used in this study to identify one of the lesion characteristics named ground glass opacity (GGO). It has been used to determine the level of malignancy of the lesion. There were three phases in identifying GGO: image cropping, feature extraction using grey level co-occurrence matrices (GLCM) and classification using Naïve Bayes Classifier. In order to improve the classification results, the most significant feature was sought by feature selection using gain ratio evaluation. Based on the results obtained, the most significant features could be identified by using feature selection method used in this research. The accuracy rate increased from 83.33% to 91.67%, the sensitivity from 82.35% to 94.11% and the specificity from 84.21% to 89.47%.

The human visual system’s real-time focus-tunable imaging capability has inspired curved imaging system development. However, curved image sensors mimicking the human retina typically lack tunable curvature to match the curved Petzval surface throughout the focus-tunable range. Here, we propose a focus-tunable real-time curved imaging system based on a tunable-curvature perovskite curved image sensor. Using parylene C film for waterproofing and encapsulation, the ultrathin perovskite curved image sensor with the hierarchical mesh design is demonstrated, which conforms to spherical surfaces of different radii while maintaining mechanical stability during deformation. The image sensor exhibits low detection limit (10 nanowatts per square centimeter) and long-term stable dynamic photoresponse. Combining the image sensor with a tunable lens achieves clear images at different distances via simultaneously adjusting both sensor and lens radii. Furthermore, through integrating image sensor with a readout matrix and wireless data collection component, the real-time imaging system is demonstrated with real-time focus-tunable imaging capabilities.

In this paper, a novel study is proposed for the feature extraction of high volumes of remote sensing images by using a simulated annealing feature extraction (SAFE) approach. For hyperspectral imagery, complete modular eigenspace (CME) has been developed by clustering highly correlated hyperspectral bands into a smaller subset of band modular based on greedy algorithm. Instead of greedy paradigm as adopted in CME approach, this paper introduces a simulated annealing (SA) approach for hyperspectral imagery. It presents a framework which consists of three algorithms, referred to as SAFE, CME and the feature scale uniformity transformation (FSUT). SAFE selects the sets of non-correlated hyperspectral bands based on SA algorithm while utilizing the inherent separability of different classes in hyperspectral images to reduce dimensionality and further to effectively generate a unique CME feature. The proposed SA features avoids the bias problems of transforming the information into linear combinations of bands as does the traditional principal components analysis and provides a fast procedure to simultaneously select the most significant features according to a scheme of SA. The experimental results show that the SAFE approach is effective and can be used as an alternative to the existing feature extraction algorithms.

The usage of the Internet of Things (IoT) conception in the industrial sector along with applications is referred to as the Industrial Internet of Things (IIoT). Various applications have been subsumed in the IIoT. Nevertheless, cybercriminals mostly target these systems. Thus, here, a novel methodology of Cyber Attack Detection (CAD) system has been proposed in IIoT to overcome the aforementioned issue. UNSW-NB2015 and DS2OS are the two IIoT datasets utilized in this work. Initially, in both datasets, the missing values are replaced; subsequently, the feature extraction is performed. Next, by utilizing Poisson Distribution-based Naked Mole Rat Optimization Algorithm (PD-NMROA), the significant features are selected as of both datasets. After that, by employing MaHalanobis distance-based K-Means (MaH-KMeans) algorithm, the features extracted as of the datasets are normalized along with clustered. Eventually, to classify the data, the clustered features are inputted to the TanSwish - Restricted Boltzmann Dense Machines (TS-RBDMs). The experiential outcomes displayed that the proposed methodology obtained higher efficacy in contrast to the prevailing systems.

The development of Multilayer Perceptron (MLP) models for networked learning systems heavily relies on the specific application case study and the accurate parameterization aligned with the chosen computer vision feature extraction models. This study proposes an MLP model for identifying potassium deficiency in cocoa plants. The feature extraction methodology employs object feature extraction that commonly used in computer vision, including Local Binary Pattern (LBP), Gray Level Co-Occurrence Matrix (GLCM), and Hue Saturation Value (HSV) models. These computer vision techniques aid in analyzing leaf characteristics classified into two categories: normal conditions and leaves identified with potassium deficiency. The dataset used in this research comprises two conditions: with a white background and without any specific background. The study evaluates various feature extraction techniques based on MLP parameters, incorporating network learning rates and optimizing solvers. Employing the ROC analysis method throughout the data collection, algorithm development, validation, and analysis phases reveals that the most effective classification performance, reaching up to 93.33% accuracy on the background dataset and 90.00% on the non-background dataset, is achieved using HSV-based color feature extraction with MLP parameters set at an initial learning rate of 10-3 and employing the Adam optimization solver. These outcomes underscore the suitability of HSV color feature extraction for identifying potassium deficiency in cocoa plant leaves. However, optimizing parameters remains crucial to maximize its application in real-time identification systems. Future research should refine these parameters to enhance the model's robustness and efficacy across broader agricultural contexts.

Abstract. This paper explains the critical importance of modelling and eliminating the effect of motion of an Unmanned Aerial Vehicle (UAV) as a result of air turbulence and unstable flight conditions on a camera sensor and the image geometry. A new method for improving the geometrical distortions caused by this motion is introduced. We have developed a hardware and software tool to minimize and model the geometric distortion of the image from commercial off-the-shelf (COTS) cameras which are commonly being used in aerial mapping UAVs. Due to the rolling shutter mechanism of the most common SLR cameras, sideway and forward motions of the UAV during image capture will have a strong effect on the image geometry and final product accuracies. As the amount of this random distortion varies from one photo to the next, a unique and robust camera calibration model cannot be established for interior orientation and image processing using photogrammetric methods, even by self-calibration. To achieve the highest possible accuracy, we also consider temperature effects on the camera calibration parameters. In this paper we show the results, accuracies and benefits of using this method compared with a typical UAV mapping system. To the best of our knowledge this is the first time that this method has been implemented in a UAV mapping system.

This paper proposes a novel beehive-shaped reflector for application to light-emitting diode (LED) transceivers for illumination and bi-directional visible light communication (VLC). By using a diffuse propagation model extended to line-of-sight and direct signals, the distribution of illuminance and the path loss of the transceiver are investigated to evaluate the performance of the beehive-shaped reflector. To verify bi-directional communication, a VLC-based image capture system, comprising a complementary metal-oxide semiconductor (CMOS) image sensor and video processor unit, is demonstrated. Real-time images captured by the CMOS camera are successfully transmitted to the monitoring system via a free-space channel at a rate of 115.2 kbps.

An artificial compound-eye imaging system has been developed consisting of one planar array of microlenses positioned on a spacing structure and coupled to a commercial CMOS optoelectronic detector array of different pitch, providing different viewing directions for the individual optical channels. Each microlens corresponds to one channel, which can be related to one or more pixels due to the different fill factors of the microlens array and the image sensor. Also alignment problems resulting from the matching of the microlens focal spots and the pixels during the assembly and the possible residual rotation between the artificial compound-eye objective and the pixel matrix are considered. We have written a program to automatically select the illuminated pixels of the sensor which correspond to each channel in order to form the final image. This calibration method is based on intensity criterions besides the geometric disposition of the microlens array. An image capture program that uses only the channels selected by the calibration is also presented. This program additionally implements image post-processing methods adapted to the microoptical compound-eye sensor. They are applied to the captured images in real time and allow increasing the contrast of the captured images. One of the methods used is the Wiener filter that is computed by taking into account an approximation of the multichannel imaging process of microoptical compound-eye sensors. Experimental results are presented, which show a noticeable increase in the frequency response when the Wiener filter is used, partially compensating the characteristic low spatial resolution of the artificial compound eyes.

Computational Photography explores the design space at the interaction of optics, sensing, and computation. I'll review my favorite research projects within this area, most of them by other people, and a few that I was involved with. These projects involve modifications to the camera lens, aperture, shutter, light source or image sensor. By designing algorithms appropriate for each camera modification, novel image capture or manipulation capabilities can be achieved.

Implementation of a real-time image visualization system on a reconfigurable chip (FPGA) is proposed. The system utilizes an innovative stereoscopic image capture, processing and visualization technique. Implementation is done as a two stage process. In the first stage, the stereo pair is captured using two image sensors. The captured images are then synchronized and sent to the second stage for fusion. A controller module is developed, designed, and placed on the FPGA for this purpose. The second stage is used for reconstruction and visualization of the 3D image. An innovative technique employing dual-processor architecture on the same single FPGA is developed for this purpose. The whole system is placed on a single PCB resulting in a fast processing time and the ability to view 3D video in real-time. The system is simulated, implemented, and tested on real images. Results show that this system is a low cost solution for efficient 3D video visualization using a single chip.