Review for "An artificial visual perception system based on ZnO threshold switching neurons with integrated rate and time-to-first-spike coding"

The visual perception system is the most important system for human learning since it receives over 80% of the learning information from the outside world. With the exponential growth of artificial intelligence technology, there is a pressing need for high-energy and area-efficiency visual perception systems capable of processing efficiently the received natural information. Currently, memristors with their elaborate dynamics, excellent scalability, and information (e.g., visual, pressure, sound, etc.) perception ability exhibit tremendous potential for the application of visual perception. Here, we propose a fully memristor-based artificial visual perception nervous system (AVPNS) which consists of a quantum-dot-based photoelectric memristor and a nanosheet-based threshold-switching (TS) memristor. We use a photoelectric and a TS memristor to implement the synapse and leaky integrate-and-fire (LIF) neuron functions, respectively. With the proposed AVPNS we successfully demonstrate the biological image perception, integration and fire, as well as the biosensitization process. Furthermore, the self-regulation process of a speed meeting control system in driverless automobiles can be accurately and conceptually emulated by this system. Our work shows that the functions of the biological visual nervous system may be systematically emulated by a memristor-based hardware system, thus expanding the spectrum of memristor applications in artificial intelligence.

Simulation of biological visual perception has gained considerable attention. In this paper, an optoelectrical In2O3 transistor array with a negative photoconductivity behavior is designed using a side-gate structure and a screen-printed ion-gel as the gate insulator. This paper is the first to observe a negative photoconductivity in electrolyte-gated oxide devices. Furthermore, an artificial visual perception system capable of self-adapting to environmental lightness is mimicked using the proposed device array. The transistor device array shows a self-adaptive behavior of light under different levels of light intensity, successfully demonstrating the visual adaption with an adjustable threshold range to the external environment. This study provides a new way to create an environmentally adaptive artificial visual perception system and has far-reaching significance for the future of neuromorphic electronics.

Developing an artificial visual sensory system requires optoelectronic materials and devices that can mimic the behavior of biological synapses. Organic/polymeric semiconductors have emerged as promising candidates for optoelectronic synapses due to their tunable optoelectronic properties, mechanic flexibility, and biological compatibility. In this review, we discuss the recent progress in organic optoelectronic synaptic materials and devices, including their design principles, working mechanisms, and applications. We also highlight the challenges and opportunities in this field and provide insights into potential applications of these materials and devices in next‐generation artificial visual systems. By leveraging the advances in organic optoelectronic materials and devices, we can envision its future development in artificial intelligence.

This paper presents a methodology we have developed and used to implement an artificial binocular vision system capable of emulating the vergence of eye movements. This methodology involves using weightless neural networks (WNNs) as building blocks of artificial vision systems. Using the proposed methodology, we have designed several architectures of WNN-based artificial vision systems, in which images captured by virtual cameras are used for controlling the position of the ‘foveae’ of these cameras (high-resolution region of the images captured). Our best architecture is able to control the foveae vergence movements with average error of only 3.58 image pixels, which is equivalent to an angular error of approximately 0.629°.

The ultimate goal of Computer Vision has been, for more than half of century, to create an artificial vision system that could imitate the human vision. The artificial vision system should have all the capabilities of the human vision system but must not carry the same flaws. Robotics and Automation are just two examples of research areas that use artificial vision systems as the main sensorial element. In these areas, the use of color-coded objects is very common since it relieves the burden of information processing while being an unobtrusive restraint of the environment. We present a novel computer vision library called UAVision that provides support for different video sensors technologies and all the necessary software for implementing an artificial vision system for the detection of color-coded objects. The experimental results that we present, both for the scenario of robotic soccer games and for traffic sign detection, show that our library can work at more than 50fps with images of 1 megapixel.

Previous research suggests that direct electrical stimulation of the visual cortex with an array of electrodes may be a viable means to provide a limited restoration of a lost visual sense to the profoundly blind. More recent work indicates that this goal may only be achieved by passing currents into the cortex via arrays of electrodes which penetrate into the visual cortex. An electrode array possessing a unique three dimensional architecture is described which could form the cornerstone of an 'artificial vision system'. A system for inserting such complex structures into cortical tissues is also described. Finally, estimates are made regarding the extent of visually guided task performance which may be expected from the limited, pixelized visual sense which could be produced with an artificial vision system.© (1991) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

Comparing biological and artificial vision systems: Network measures of functional connectivity

Near-infrared (NIR) synaptic devices integrate NIR optical sensitivity and synaptic plasticity, emulating the basic biomimetic function of the human visual system and showing great potential in NIR artificial vision systems. However, the lack of semiconductor materials with appropriate band gaps for NIR photodetection and effective strategies for fabricating devices with synaptic behaviors limit the further development of NIR synaptic devices. Here, a two-terminal NIR synaptic device consisting of the In2Se3/MoS2 heterojunction has been constructed, and it exhibits fundamental synaptic functions. The reduced band gap and potential barrier of In2Se3/MoS2 heterojunctions are essential for NIR synaptic plasticity. In addition, the NIR synaptic properties of In2Se3/MoS2 heterojunctions under strain have been studied systematically. The ΔEPSC of the In2Se3/MoS2 synaptic device can be improved from 38.4% under no strain to 49.0% under a 0.54% strain with a 1060 nm illumination for 1 s at 100 mV. Furthermore, the artificial NIR vision system consisting of a 10 × 10 In2Se3/MoS2 device array has been fabricated, exhibiting image sensing, learning, and storage functions under NIR illumination. This research provides new ideas for the design of flexible NIR synaptic devices based on 2D materials and presents many opportunities in artificial intelligence and NIR vision systems.

Some fundamental visual features have been found to be fully extracted before reaching the cerebral cortex. We focus on direction-selective ganglion cells (DSGCs), which exist at the terminal end of the retinal pathway, at the forefront of the visual system. By utilizing a layered pathway composed of various relevant cells in the early stage of the retina, DSGCs can extract multiple motion directions occurring in the visual field. However, despite a considerable amount of comprehensive research (from cells to structures), a definitive conclusion explaining the specific details of the underlying mechanisms has not been reached. In this paper, leveraging some important conclusions from neuroscience research, we propose a complete quantified model for the retinal motion direction selection pathway and elucidate the global motion direction information acquisition mechanism from DSGCs to the cortex using a simple spiking neural mechanism. This mechanism is referred to as the artificial visual system (AVS). We conduct extensive testing, including one million sets of two-dimensional eight-directional binary object motion instances with 10 different object sizes and random object shapes. We also evaluate AVS’s noise resistance and generalization performance by introducing random static and dynamic noises. Furthermore, to thoroughly validate AVS’s efficiency, we compare its performance with two state-of-the-art deep learning algorithms (LeNet-5 and EfficientNetB0) in all tests. The experimental results demonstrate that due to its highly biomimetic design and characteristics, AVS exhibits outstanding performance in motion direction detection. Additionally, AVS possesses biomimetic computing advantages in terms of hardware implementation, learning difficulty, and parameter quantity.

A self-powered artificial retina perception system for image preprocessing based on photovoltaic devices and memristive arrays

Conductive metal-organic frameworks (MOFs) have performed well in the fields of energy and catalysis, among which two-dimensional (2D) and three-dimensional (3D) MOFs are well-known. Here, we have synthesized a one-dimensional (1D) conductive metal-organic framework (MOF) in which hexacoordinated 1,5-Diamino-4,8-dihydroxy-9,10-anthraceneedione (DDA) ligands are connected by double Cu ions, resulting in nanoribbon layers with 1D π-d conjugated nanoribbon plane and out-of-plane π-π stacking, which facilitates charge transport along two dimensions. The DDA-Cu as a highly conductive n-type MOF has high crystalline quality with a conductivity of ~ 9.4 S·m−1, which is at least two orders of magnitude higher than that of conventional 1D MOFs. Its electrical band gap (Eg) and exciton binding energy (Eb) are approximately 0.49 eV and 0.3 eV, respectively. When utilized as electrode material in a supercapacitor, the DDA-Cu exhibits good charge storage capacity and cycle stability. Meanwhile, as thse active semiconductor layer, it successfully simulates the artificial visual perception system with excellent bending resistance and air stability as a MOF-based flexible optoelectronic synaptic case. The controllable preparation of high-quality 1D DDA-Cu MOF may enable new architectural designs and various applications in the future.

In this study, we designed a biomimetic artificial visual system (AVS) inspired by biological visual system that can process RGB images. Our approach begins by mimicking the photoreceptor cone cells to simulate the initial input processing followed by a learnable dendritic neuron model to replicate ganglion cells that integrate outputs from bipolar and horizontal cell simulations. To handle multi-channel integration, we utilize a nonlearnable dendritic neuron model to simulate the lateral geniculate nucleus (LGN), which consolidates outputs across color channels, an essential function in biological multi-channel processing. Cross-validation experiments show that AVS demonstrates strong generalization across varied object-background configurations, achieving accuracy where traditional models like EfN-B0, ResNet50, and ConvNeXt typically fall short. Additionally, our results across different training-to-testing data ratios reveal that AVS maintains over 96% test accuracy even with limited training data, underscoring its robustness in low-data scenarios. This demonstrates the practical advantage of the AVS model in applications where large-scale annotated datasets are unavailable or expensive to curate. This AVS model not only advances biologically inspired multi-channel processing but also provides a practical framework for efficient, integrated visual processing in computational models.

Time-effective detection and recognition of objects of interest in images is still a matter of intensive research in computer vision community because the artificial vision systems fail to outperform the detection results by a human being. The detection problem is complicated when objects of interest have low contrast and various sizes or orientations and can be located on noisy and inhomogeneous background with occlusion occurrences. In many practical applications, the real-time implementation of object detection algorithms in such conditions is a matter of great concern. The results of numerous neurological and psychophysical investigation of human visual system (HVS) indicate that the human vision can successfully cope with these complex situations because of using the visual attention mechanism associated with a model-based image analysis.1,2 The goal of the investigation presented here was not the simulation of human visual perception but the incorporation of its advantageous features into computer vision algorithms. Besides many remarkable properties of HVS like the mentioned model-based visual attention, the HVS has also some disadvantages while detecting and identifying objects. For instance, it is prone to illusions that should be not automatically copied onto an artificial vision system.3

The artificial vision system (AVS) uses image analysis methods that can interpret images and identify nutritional deficiency symptoms in plant, even in the early stages of development. The objective of this study was to propose methods of image processing using analysis by texture to identify the deficiency of calcium (Ca) in maize (Zea mays L.) plants grown in nutrient solution. Plants were grown in nutrient solution in a greenhouse. Calcium doses were 0.0; 1.7; 3.3 and 5.0 mM of Ca, with four replications. Plant and leaf images were sampled at three main stages of maize development: V4 (plants with four leaves fully developed), V6 (plants with six leaves fully developed) and V8 (plants with eight leaves fully developed). Sampled material was split into (i) index leaf (IL) of the growing stage (V4 = leaf 4, V6 = leaf 6, and V8 = leaf 8), and (ii) new leaf (OL), both to image capture and chemical analysis. Such leaves were scanned, processed by the AVS and chemically analyzed. The texture methods used by the AVS to extract deficiency characteristics in the leaf images were: Volumetric Fractal Dimension (VFD), Gabor Wavelet Energy (GWE) and VFD with canonical analysis (VFDCA). The amount of Ca in the solution resulted in variation in the concentration of Ca in NL and IL, allowing the observation of typical symptoms of Ca deficiency. The AVS method was able to identify all Ca levels in leaves, being the GWE the best indicator using color images, scoring 80% of rights in images of the middle section of new leaves in V4.

Retina-inspired in-sensor broadband image preprocessing for accurate recognition via the flexophototronic effect