Digital-analog hybrid matrix multiplication processor for optical neural networks

Object detection is a fundamental problem in computer vision, with applications spanning self-driving cars, surveillance systems, medical imaging, robotics, and smart cities. Among the myriad of algorithms developed for this task, the You Only Look Once (YOLO) family stands out for its ability to perform real-time and accurate object detection. This article provides a comprehensive analysis of the YOLO algorithm series, from YOLOv1 to YOLOv8, evaluating them across key performance metrics, including precision, recall, mean Average Precision (mAP), frames per second (FPS), and overall effectiveness. Unlike traditional two-stage detectors such as R-CNN, YOLO formulates object detection as a single regression problem: a single pass over the image simultaneously predicts bounding boxes and class probabilities. This end-to-end design enables YOLO to achieve high speed while maintaining a competitive accuracy-efficiency trade-off. We examine architectural innovations across YOLO versions, including batch normalization, anchor boxes, residual blocks, feature pyramid networks, and attention mechanisms, and discuss their impact on performance. Lightweight models (e.g., YOLOv5-Nano, YOLOv8-Small) are explored with a focus on their suitability for mobile and embedded systems, highlighting YOLO’s adaptability to resource-constrained environments. Challenges such as small object detection, occlusion, and domain-specific tuning are also addressed. This article serves as a practical guide for researchers, developers, and practitioners aiming to leverage YOLO for real-world object detection tasks.

Abstract- With the rapid growth of urban populations and the increasing number of vehicles on the road, traffic congestion has become a significant problem in metropolitan areas. Conventional traffic control systems, which rely on pre-set signal timers, often fail to address real-time traffic conditions effectively, leading to inefficient traffic flow, increased fuel consumption, and environmental degradation. The paper “Smart Traffic Control System Based on Traffic Density Using YOLO” introduces a novel approach to address this issue by integrating artificial intelligence and computer vision into traffic management. The proposed system utilizes the YOLO (You Only Look Once) object detection algorithm to detect and classify vehicles in real-time from live video feeds captured at intersections. YOLO, known for its speed and accuracy in object detection tasks, calculates the vehicle density in each direction. Based on the detected density, the system dynamically adjusts the green light duration for each lane, ensuring that the direction with higher traffic receives longer green light periods. This method aims to optimize signal switching and minimize overall vehicle wait time. The paper outlines the complete architecture of the system, which includes modules for video input processing, vehicle detection using YOLO, traffic density computation, and adaptive signal timing control. By leveraging deep learning, the system enhances decision-making capabilities without relying on costly hardware like inductive loops or infrared sensors. This review evaluates the effectiveness and innovation of the proposed system, comparing it with traditional and sensor-based models. It highlights the advantages of using YOLO, such as real-time processing, high accuracy, and scalability. Additionally, the review discusses the system’s limitations, such as dependency on video clarity and environmental conditions, and proposes areas for future enhancement, including integration with cloud computing and edge AI for broader deployment. Keywords- Smart Traffic Control System, YOLO Object Detection, Traffic Density Estimation, Real-Time Traffic Monitoring, Adaptive Traffic Signal Control, Vehicle Classification ( Cars, Bikes, Buses, Trucks, Rikshaws), Intelligent Transportation System.

In this work, we present our intelligent SSD, SeeSSD , an energy-efficient computational SSD for a real-time object detection system. SeeSSD embeds an FPGA-based CNN processing engine and the firmware that performs the convolutional operation on the target image. SeeSSD processes the image data at the storage before sending it to the host. This reduces the amount of data transferred to the host and lowers the data movement overhead, thus reducing transfer time and saving power. By using our SeeSSD system and YOLO_Embed, an object detection neural network model, we are able to outperform the fastest YOLO model for an embedded controller, YOLO-Lite, in terms of performance, accuracy, and energy efficiency. YOLO (You Only Look Once) models are a series of one-stage object detection neural models that have become very popular due to their fast speed and high accuracy. The contribution of this work includes designing and implementing our SeeSSD system with a lightweight object detection model, YOLO_Embed, for reducing the data movement overhead, performing real-time inference, and lowering the overall power consumption. We implemented the entire software stack associated with the SeeSSD system; on-device CNN acceleration engine implemented on FPGA, object identification interface for SeeSSD using YOLO_Embed, and embedded software layer in SeeSSD for on-device convolutional processing. We calculated our YOLO_Embed model’s accuracy on object detection dataset benchmarks such as PASCAL VOC 2012, which came out to be 38.1% mAP (mean Accuracy Precision). Our system was able to perform inference in 0.21 seconds while reducing the power consumption by approximately 1.2 × and 1.4 × for CPU-Only and CPU+GPU systems, respectively. We were also able to reduce the data movement overhead by 24 × for a single target image.

Object detection from UAV thermal infrared images and videos using YOLO models

The emergence of nanophotonic devices has enabled to design light-speed on-chip optical circuits with extremely low latency. This paper proposes an optical implementation of scalable Deep Neural Networks (DNNs) enabling light-speed inference. The key issue in optical neural networks is the scalability limited by area, power and the number of available wavelengths. Due to the scalability, it is thus difficult to design an all-optical hardware accelerator for a large-scale DNN. To solve this problem, this paper firstly proposes an optical Vector Matrix Multiplier (VMM) structure operating with a low latency. The multipliers in a VMM are highly parallelized based on the Wavelength Division Multiplexing (WDM) technique, which reduces the area overhead without sacrificing the ultra-high speed nature. This paper then proposes the electrical digital interfaces for storing and handling intermediate VMM data without sacrificing the ultra-high speed nature, which enables to reuse the VMM multiple times with a low latency.

Object detection from deep learning has performed better in many applications, and yet real images usually depict challenges such as noise blurring or jitter rotation. Such problems generally degrade the accuracy and the efficiency of object detection systems dramatically. The purpose of this work is to identify objects satisfactorily by applying a method called You Only Look Once (YOLO), which tackles numerous problems while it provides plenty of advantages over other solutions.Unlike other algorithms that are known as Convolutional Neural Networks (CNN)or Fast-Convolutional Neural Networks, process images partially by focusing on specific areas. YOLO is different from that; it looks at an entire image at once while predicting bounding boxes and class probabilities for objects within these boxes by making a single forward pass of the network. This holistic approach makes YOLO significantly faster and much more efficient than other algorithms that process parts of the image separately.For this project, we have used the YOLO algorithm for detecting all types of objects in a real-time scenario. Moreover, an Android Application has been developed incorporating YOLO-based object detection for enhancing user experience. This application does not only detect objects in images or live video feeds but also provides voice feedback to the user. Such an enhancement would make the system much more accessible for individuals with visual impairments because they could receive immediate auditory information about the objects detected in their environment.

Object detection has become the cornerstone of various real-world applications, ranging from autonomous driving where it helps detect road objects to medical imaging for early disease prediction and gesture recognition systems. Due to their ability to operate continuously and cost-effectively, computer vision models are increasingly being employed in surveillance and monitoring tasks. Unlike humans, who can intuitively recognize and understand objects within images, machines require advanced algorithms to mimic such capabilities. A key challenge in computer vision lies in recognizing and tracking objects in real-time with accuracy and reliability. Human visual systems perform these tasks naturally and swiftly, enabling complex activities like driving. For computers to match such performance, efficient object detection models along with additional hardware such as sensors are essential. This study presents a comparative analysis of two popular deep learning-based object detection algorithms You Only Look Once (YOLO) and Faster Region-based Convolution Neural Network (Faster RCNN). Both models are evaluated based on their detection accuracy, speed, and performance in real-time scenarios. The paper aims to highlight the strengths and limitations of each model, offering insights into their suitability for different applications. The findings suggest that while both algorithms have their merits, the choice between them depends on the specific requirements of a task, such as the trade-off between detection speed and precision. Comparative study shows that Faster RCNN outperforms its accuracy compared with YOLO.

This paper provides a comprehensive study of the security of YOLO (You Only Look Once) model series for object detection, emphasizing their evolution, technical innovations, and performance across the COCO dataset. The robustness of YOLO models under adversarial attacks and image corruption, offering insights into their resilience and adaptability, is analyzed in depth. As real-time object detection plays an increasingly vital role in applications such as autonomous driving, security, and surveillance, this review aims to clarify the strengths and limitations of each YOLO iteration, serving as a valuable resource for researchers and practitioners aiming to optimize model selection and deployment in dynamic, real-world environments. The results reveal that YOLOX models, particularly their large variants, exhibit superior robustness compared to other YOLO versions, maintaining higher accuracy under challenging conditions. Our findings serve as a valuable resource for researchers and practitioners aiming to optimize YOLO models for dynamic and adversarial real-world environments while guiding future research toward developing more resilient object detection systems.

The objective of this study is to examine two powerful components: the YOLO (You Only Look Once) framework for object recognition on edge devices and the MobileNet architecture. With the rising demand for real-time, efficient, and accurate object detection in constrained environments, these models offer distinct advantages suited to different applications. YOLO is renowned for its high accuracy and speed, making it ideal for complex detection tasks, whereas MobileNet is optimized for efficiency, enabling deployment on devices with limited computational resources. The paper discusses the evolution of these models, their architectural differences, and their respective strengths and limitations when implemented on edge devices. Furthermore, we explore the feasibility of integrating these frameworks into a hybrid model that leverages the lightweight feature extraction of MobileNet and the robust detection capabilities of YOLO. Such a hybrid approach could significantly enhance real-time object detection performance across a wide range of hardware platforms.  Researchers and practitioners seeking to deploy advanced object detection models in real-world applications, particularly where computational efficiency and accuracy are paramount may find this review as a valuable resource.

Object detection is a critical component in intelligent surveillance, disaster response, and autonomous dronebased monitoring systems. In recent years, the You Only Look Once (YOLO) family of algorithms has transformed real-time object detection by achieving an effective balance between detection speed and accuracy. This review paper presents a comprehensive analysis of YOLO models from YOLOv1 to YOLOv8, emphasizing their architectural evolution, backbone development, and feature extraction strategies. Each version’s key innovations are discussed in detail—from YOLOv1’s unified detection framework and YOLOv2’s introduction of anchor boxes to YOLOv8’s advanced anchor-free detection mechanism. The comparative evaluation focuses on model accuracy (mAP), inference speed (FPS), and deployment feasibility for embedded and UAV-based systems. Findings from existing literature and experimental insights reveal that successive YOLO versions have progressively enhanced small object detection, robustness to scale variation, and computational efficiency. Overall, the study concludes that YOLOv8 currently achieves the most optimal trade-off between precision, speed, and resource utilization, establishing it as a leading framework for real-time aerial human detection and a promising foundation for future advancements in autonomous surveillance and rescue applications.

One popular of object detection model for object detection is You Only Look Once (YOLO) with humans are among the most often utilized for detection objects. Despite the various of human datasets, just a few research that compared the datasets performance against various versions of the YOLO algorithm. This study compares the performance of YOLOv10, YOLOv11, and YOLOv12 on eight different datasets, such as CrowdHuman, CityPersons, Wider Person, Mall Dataset, INRIA, Microsoft Common Object (MS COCO), PASCAL VOC, and MOT17. Precision, recall, mAP@50, and mAP@50-95 are used to measure the YOLO model version's performance on each dataset. The results indicate that each datasets have different perfomance on each version of YOLO, so the performance on model depends on the variation of the dataset. The best results on the MOT17 dataset are obtained by YOLOv12, with 0.909 in precision, 0.775 in recall, 0.88 in mAP@50, and 0.695 in mAP@50-95. On the City Person dataset. However, YOLOv11 performs best result, with 0.782 in precision, 0.529 in recall, 0.694 in mAP@50, and 0.476 in mAP@50-95. Therefore, choosing a YOLO version that is appropriate for the dataset's complexity is essential to creating the best detection model Therefore, selecting the appropriate YOLO version according to the dataset complexity is crucial to obtain the most optimal detection model.

Artificial intelligence has prevailed in all trades and professions due to the assistance of big data resources, advanced algorithms, and high-performance electronic hardware. However, conventional computing hardware is inefficient at implementing complex tasks, in large part because the memory and processor in its computing architecture are separated, performing insufficiently in computing speed and energy consumption. In recent years, optical neural networks (ONNs) have made a range of research progress in optical computing due to advantages such as sub-nanosecond latency, low heat dissipation, and high parallelism. ONNs are in prospect to provide support regarding computing speed and energy consumption for the further development of artificial intelligence with a novel computing paradigm. Herein, we first introduce the design method and principle of ONNs based on various optical elements. Then, we successively review the non-integrated ONNs consisting of volume optical components and the integrated ONNs composed of on-chip components. Finally, we summarize and discuss the computational density, nonlinearity, scalability, and practical applications of ONNs, and comment on the challenges and perspectives of the ONNs in the future development trends.

This research explores the feasibility of You Only Look Once (YOLO), a deep learning object detection algorithm, for lane and object detection in challenging Indian road environments. Traditional methods struggle with faded lane markings, dense and diverse traffic, and unpredictable scenarios. YOLO's speed and accuracy make it suitable for real-time ADAS applications. The methodology leverages Roboflow, a platform for computer vision tasks, to explore data acquisition, model selection, training, and evaluation. This research aims to contribute to developing safer and more reliable ADAS systems for Indian roads.

The growing computational demands of artificial intelligence motivate the development of optical neural networks (ONNs), while their practical deployment remains constrained by optimization complexity and limited adaptability. We propose a mutual learning knowledge distillation framework for optical diffractive neural networks (ODNNs), synergizing photonic computing with adaptive training strategies. Our architecture enables bidirectional knowledge transfer between parameter-efficient optical student networks and electrical ResNet teachers, leveraging both labeled data and probabilistic guidance. The validation on Fashion-MNIST and Kuzushiji-MNIST demonstrates classification accuracy improvements of 5.9% and 13.88%, respectively, over conventional ODNNs training. In addition, we further developed the photonic to photonic mutual learning computational paradigm, in which an optical network with 0.98 × 106 parameters can bootstrap compact photonic classifiers with 0.19 × 106 parameters, achieving an accuracy of 79.5% on MNIST and yielding an absolute improvement of 9% relative to the noncooperative baseline. Even though the all-optical mutual learning paradigm still has a gap of about 19% with the state-of-the-art ODNNs, the number of parameters of the guided student networks is smaller than the state-of-the-art networks by several or even dozens of times. These results establish a paradigm for overcoming fundamental limitations in ONN optimization, paving the way for a new era in which ONNs can solve large-scale practical problems through photonic–electronic and photonic–photonic co-design.

Event Abstract Back to Event Logarithmic Hybrid Optical Neural Network-type systems: towards digital-optical cognitive models of robust object recognition Ioannis Kypraios1* 1 Oxford University, Centre for Innovation & Enterprise, APEM Computing Lab, United Kingdom Abstract - By combining the complex logarithmic r-theta mapping of a space-variant imaging sensor [1, 2] with the hybrid digital-optical neural network filter together with a window unit multiple objects of the same class or of different classes can robustly be recognized. The resulted logarithmic r-theta mapping for hybrid digital-optical neural network system or briefly referred to as logarithmic hybrid optical neural network system is shown to exhibit with a single pass over the input data simultaneously in-plane rotation, out-of-plane rotation, scale, log r-θ map translation and shift invariance, and good clutter tolerance by recognizing correctly the different objects within the cluttered scenes. Here, we study the biologically-inspired knowledge learning and knowledge representation [3] achieved by the logarithmic hybrid optical neural network-type of object recognition systems (see Fig. 1 and Fig. 2). We investigate the effects that altering the knowledge representation can have on the problem’s learned knowledge and the problem solving process, in overall. Further, we study the logarithmic unconstrained-, logarithmic constrained-, and logarithmic modified-hybrid optical neural network systems architectures’ designs [4, 5]. We show how the logarithmic unconstrained-hybrid optical neural network object recognition system applies an unconstrained representation of the problem’s knowledge to maximize the search of solutions, how the logarithmic constrained-hybrid optical neural network object recognition system uses a constrained representation of the problem’s knowledge to guide the search towards certain solutions over others in the multidimensional search space, and how the logarithmic modified-hybrid optical neural network object recognition system uses memory-like masks to recall certain solutions over others in the multidimensional search space. Fig. 1. (a) Simplified human retina and visual cortex model used for description purposes only; (b) A digital-optical computational model for the cognitive interaction between the retina and the human visual cortex with the general logarithmic hybrid optical neural network architercture. Fig. 2. Biologically-inspired knowledge learning and knowledge representation with the logarithmic hybrid optical neural network-type of object recognition systems. Figure 1 Figure 2 References [1] C. F. R. Weiman, “Video compression via log-polar mapping”, Real-Time Image Processing II, SPIE Symposium on OE/Aerospace Sensing, Vol. 1295, 266-277 (1990) [2] C. G. Ho et al., “Sensor geometry and sampling methods for space-variant image processing”, Pattern Analysis and Applications, Vol. 5, 369-384 (2002) [3] I. Lee, and B. Portier, "An empirical study of knowledge representation and learning within conceptual spaces for intelligent agents", 6th IEEE/ACIS International Conference Computer and Information Science, 463-468, Melbourne, Australia (2007) [4] I. Kypraios et al., “Fully invariant complex Logarithmic r- map for the Hybrid Optical Neural Network filter for object recognition within cluttered scenes”, 50th Anniversary International Symposium ELMAR 2008, IEEE Region 8/IEEE Croatia Section/EURASIP, Zadar-Croatia, Vol. 1, 141-146 (2008) [5] I. Kypraios et al., “Logarithmic r-θ mapping for hybrid optical neural network filter for object recogntion within cluttered scenes”, Recent Advances in Multimedia Signal Processing and Communications, SCI231, Springer-Verlag Berlin Heidelberg, 91-120 (2009) Keywords: digital-optical, cognitive, object recognition, Robust and Adaptive Systems, Knowledge representation, learning and memory Conference: Neuroinformatics 2013, Stockholm, Sweden, 27 Aug - 29 Aug, 2013. Presentation Type: Poster Topic: Neuromorphic engineering Citation: Kypraios I (2013). Logarithmic Hybrid Optical Neural Network-type systems: towards digital-optical cognitive models of robust object recognition. Front. Neuroinform. Conference Abstract: Neuroinformatics 2013. doi: 10.3389/conf.fninf.2013.09.00066 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 09 Apr 2013; Published Online: 11 Jul 2013. * Correspondence: Dr. Ioannis Kypraios, Oxford University, Centre for Innovation & Enterprise, APEM Computing Lab, Begbroke, Oxfordshire, OX5 1PF, United Kingdom, ioanniskyp@yahoo.com Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Ioannis Kypraios Google Ioannis Kypraios Google Scholar Ioannis Kypraios PubMed Ioannis Kypraios Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.