Comparing biological and artificial vision systems: Network measures of functional connectivity

Near-infrared (NIR) synaptic devices integrate NIR optical sensitivity and synaptic plasticity, emulating the basic biomimetic function of the human visual system and showing great potential in NIR artificial vision systems. However, the lack of semiconductor materials with appropriate band gaps for NIR photodetection and effective strategies for fabricating devices with synaptic behaviors limit the further development of NIR synaptic devices. Here, a two-terminal NIR synaptic device consisting of the In2Se3/MoS2 heterojunction has been constructed, and it exhibits fundamental synaptic functions. The reduced band gap and potential barrier of In2Se3/MoS2 heterojunctions are essential for NIR synaptic plasticity. In addition, the NIR synaptic properties of In2Se3/MoS2 heterojunctions under strain have been studied systematically. The ΔEPSC of the In2Se3/MoS2 synaptic device can be improved from 38.4% under no strain to 49.0% under a 0.54% strain with a 1060 nm illumination for 1 s at 100 mV. Furthermore, the artificial NIR vision system consisting of a 10 × 10 In2Se3/MoS2 device array has been fabricated, exhibiting image sensing, learning, and storage functions under NIR illumination. This research provides new ideas for the design of flexible NIR synaptic devices based on 2D materials and presents many opportunities in artificial intelligence and NIR vision systems.

This paper presents a methodology we have developed and used to implement an artificial binocular vision system capable of emulating the vergence of eye movements. This methodology involves using weightless neural networks (WNNs) as building blocks of artificial vision systems. Using the proposed methodology, we have designed several architectures of WNN-based artificial vision systems, in which images captured by virtual cameras are used for controlling the position of the ‘foveae’ of these cameras (high-resolution region of the images captured). Our best architecture is able to control the foveae vergence movements with average error of only 3.58 image pixels, which is equivalent to an angular error of approximately 0.629°.

In this study, we designed a biomimetic artificial visual system (AVS) inspired by biological visual system that can process RGB images. Our approach begins by mimicking the photoreceptor cone cells to simulate the initial input processing followed by a learnable dendritic neuron model to replicate ganglion cells that integrate outputs from bipolar and horizontal cell simulations. To handle multi-channel integration, we utilize a nonlearnable dendritic neuron model to simulate the lateral geniculate nucleus (LGN), which consolidates outputs across color channels, an essential function in biological multi-channel processing. Cross-validation experiments show that AVS demonstrates strong generalization across varied object-background configurations, achieving accuracy where traditional models like EfN-B0, ResNet50, and ConvNeXt typically fall short. Additionally, our results across different training-to-testing data ratios reveal that AVS maintains over 96% test accuracy even with limited training data, underscoring its robustness in low-data scenarios. This demonstrates the practical advantage of the AVS model in applications where large-scale annotated datasets are unavailable or expensive to curate. This AVS model not only advances biologically inspired multi-channel processing but also provides a practical framework for efficient, integrated visual processing in computational models.

Developing an artificial visual sensory system requires optoelectronic materials and devices that can mimic the behavior of biological synapses. Organic/polymeric semiconductors have emerged as promising candidates for optoelectronic synapses due to their tunable optoelectronic properties, mechanic flexibility, and biological compatibility. In this review, we discuss the recent progress in organic optoelectronic synaptic materials and devices, including their design principles, working mechanisms, and applications. We also highlight the challenges and opportunities in this field and provide insights into potential applications of these materials and devices in next‐generation artificial visual systems. By leveraging the advances in organic optoelectronic materials and devices, we can envision its future development in artificial intelligence.

The visual perception system is the most important system for human learning since it receives over 80% of the learning information from the outside world. With the exponential growth of artificial intelligence technology, there is a pressing need for high-energy and area-efficiency visual perception systems capable of processing efficiently the received natural information. Currently, memristors with their elaborate dynamics, excellent scalability, and information (e.g., visual, pressure, sound, etc.) perception ability exhibit tremendous potential for the application of visual perception. Here, we propose a fully memristor-based artificial visual perception nervous system (AVPNS) which consists of a quantum-dot-based photoelectric memristor and a nanosheet-based threshold-switching (TS) memristor. We use a photoelectric and a TS memristor to implement the synapse and leaky integrate-and-fire (LIF) neuron functions, respectively. With the proposed AVPNS we successfully demonstrate the biological image perception, integration and fire, as well as the biosensitization process. Furthermore, the self-regulation process of a speed meeting control system in driverless automobiles can be accurately and conceptually emulated by this system. Our work shows that the functions of the biological visual nervous system may be systematically emulated by a memristor-based hardware system, thus expanding the spectrum of memristor applications in artificial intelligence.

The human entorhinal-hippocampal circuitry is characterized by an information-specific functional organization where two routes, that are preferentially connected to the parahippocampal cortex or the perirhinal and retrosplenial cortices, divide the entorhinal cortex as well as hippocampal subiculum and CA1 subregions.

Previous research suggests that direct electrical stimulation of the visual cortex with an array of electrodes may be a viable means to provide a limited restoration of a lost visual sense to the profoundly blind. More recent work indicates that this goal may only be achieved by passing currents into the cortex via arrays of electrodes which penetrate into the visual cortex. An electrode array possessing a unique three dimensional architecture is described which could form the cornerstone of an 'artificial vision system'. A system for inserting such complex structures into cortical tissues is also described. Finally, estimates are made regarding the extent of visually guided task performance which may be expected from the limited, pixelized visual sense which could be produced with an artificial vision system.© (1991) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

The discovery of the fifth gravitational wave, GW170817, and its electromagnetic counterpart, resulting from the merger of neutron stars by the LIGO and Virgo teams, marked a major milestone in astronomy. It was the first time that gravitational waves and light from the same cosmic event were observed simultaneously. The LIGO detectors in the United States recorded the signal for 100 s, longer than in previous detections. The merging of neutron stars emits both gravitational and electromagnetic waves across all frequencies—from radio to gamma rays. However, pinpointing the exact source remains difficult, requiring rapid sky scanning to locate it. To address this challenge, the Gravitational-Wave Optical Transient Observer (GOTO) project was established. It is specifically designed to detect optical light from transient events associated with gravitational waves, enabling faster follow-up observations and a deeper study of these short-lived astronomical phenomena, which appear and disappear quickly in the universe. In astrophysics, it has become more important to find astronomical transient events like supernovae, gamma-ray bursts, and stellar flares because they are linked to extreme cosmic processes. However, finding these short-lived events in huge sky survey datasets, like those from the GOTO project, is very hard for traditional analysis methods. This study suggests a deep learning methodology employing Convolutional Neural Networks (CNNs) to enhance transient classification. CNNs are based on how biological vision systems work and how they are structured. They mimic how animal brains hierarchically process visual information, making it possible to automatically find complex spatial patterns in astronomical images. Transfer learning and fine-tuning on pretrained ImageNet models are utilized to emulate adaptive learning observed in biological organisms, enabling swift adaptation to new tasks with minimal data. Data augmentation methods like rotation, flipping, and noise injection mimic changes in the environment to improve model generalization. Dropout and different batch sizes are used to stop overfitting, which is similar to how biological systems use redundancy and noise tolerance. Ensemble learning strategies, such as Soft Voting and Weighted Voting, draw inspiration from collective intelligence in biological systems, integrating multiple CNN models to enhance decision-making robustness. Our findings indicate that this bio-inspired framework substantially improves the precision and dependability of transient detection, providing a scalable solution for real-time applications in extensive sky surveys such as GOTO.

Statistical and Deterministic Regularities: Utilization of Motion and Grouping in Biological and Artificial Visual Systems

Developmental malformations (dysgenesis) of the corpus callosum lead to neurological conditions with a broad range of clinical presentations. Investigating the altered brain connectivity patterns is crucial to understanding both adaptive and maladaptive neuroplasticity in corpus callosum dysgenesis patients. Here, we acquired structural diffusion-weighted and resting-state functional MRI data from a cohort of 11 corpus callosum dysgenesis patients (five with agenesis and six with hypoplasia) and compared their structural and functional connectivity patterns to healthy subjects selected from the Human Connectome Project. We found that these patients have fewer structural inter- and intra-hemispheric brain connections relative to healthy controls. Interestingly, the patients with callosal agenesis have a scant number of inter-hemispheric connections but manage to maintain the full integrity of functional connectivity between the same cortical regions as the healthy subjects. On the other hand, the hypoplasic group presented abnormal structural and functional connectivity patterns relative to healthy controls while maintaining the same total amount of functional connections. These results demonstrate that acallosal patients can compensate for having fewer structural brain connections and present functional adaptation. However, hypoplasics present atypical structural connections to different brain regions, leading to entirely new and abnormal functional brain connectivity patterns.

The ultimate goal of Computer Vision has been, for more than half of century, to create an artificial vision system that could imitate the human vision. The artificial vision system should have all the capabilities of the human vision system but must not carry the same flaws. Robotics and Automation are just two examples of research areas that use artificial vision systems as the main sensorial element. In these areas, the use of color-coded objects is very common since it relieves the burden of information processing while being an unobtrusive restraint of the environment. We present a novel computer vision library called UAVision that provides support for different video sensors technologies and all the necessary software for implementing an artificial vision system for the detection of color-coded objects. The experimental results that we present, both for the scenario of robotic soccer games and for traffic sign detection, show that our library can work at more than 50fps with images of 1 megapixel.

Some fundamental visual features have been found to be fully extracted before reaching the cerebral cortex. We focus on direction-selective ganglion cells (DSGCs), which exist at the terminal end of the retinal pathway, at the forefront of the visual system. By utilizing a layered pathway composed of various relevant cells in the early stage of the retina, DSGCs can extract multiple motion directions occurring in the visual field. However, despite a considerable amount of comprehensive research (from cells to structures), a definitive conclusion explaining the specific details of the underlying mechanisms has not been reached. In this paper, leveraging some important conclusions from neuroscience research, we propose a complete quantified model for the retinal motion direction selection pathway and elucidate the global motion direction information acquisition mechanism from DSGCs to the cortex using a simple spiking neural mechanism. This mechanism is referred to as the artificial visual system (AVS). We conduct extensive testing, including one million sets of two-dimensional eight-directional binary object motion instances with 10 different object sizes and random object shapes. We also evaluate AVS’s noise resistance and generalization performance by introducing random static and dynamic noises. Furthermore, to thoroughly validate AVS’s efficiency, we compare its performance with two state-of-the-art deep learning algorithms (LeNet-5 and EfficientNetB0) in all tests. The experimental results demonstrate that due to its highly biomimetic design and characteristics, AVS exhibits outstanding performance in motion direction detection. Additionally, AVS possesses biomimetic computing advantages in terms of hardware implementation, learning difficulty, and parameter quantity.

Impaired functional default mode network in patients with mild neurological Wilson’s disease

The artificial vision system (AVS) uses image analysis methods that can interpret images and identify nutritional deficiency symptoms in plant, even in the early stages of development. The objective of this study was to propose methods of image processing using analysis by texture to identify the deficiency of calcium (Ca) in maize (Zea mays L.) plants grown in nutrient solution. Plants were grown in nutrient solution in a greenhouse. Calcium doses were 0.0; 1.7; 3.3 and 5.0 mM of Ca, with four replications. Plant and leaf images were sampled at three main stages of maize development: V4 (plants with four leaves fully developed), V6 (plants with six leaves fully developed) and V8 (plants with eight leaves fully developed). Sampled material was split into (i) index leaf (IL) of the growing stage (V4 = leaf 4, V6 = leaf 6, and V8 = leaf 8), and (ii) new leaf (OL), both to image capture and chemical analysis. Such leaves were scanned, processed by the AVS and chemically analyzed. The texture methods used by the AVS to extract deficiency characteristics in the leaf images were: Volumetric Fractal Dimension (VFD), Gabor Wavelet Energy (GWE) and VFD with canonical analysis (VFDCA). The amount of Ca in the solution resulted in variation in the concentration of Ca in NL and IL, allowing the observation of typical symptoms of Ca deficiency. The AVS method was able to identify all Ca levels in leaves, being the GWE the best indicator using color images, scoring 80% of rights in images of the middle section of new leaves in V4.

Structural re-parameterization is a raising field, which aims at improving the performance of convolutional neural networks (CNNs) through training an over-parameterization model and transferring it into a compact inference model. However, the performance improvements of prior structural re-parameterization works often come at the cost of heavy extra training resources, which increases carbon emissions and limits the potential applications on large-scale industrial tasks. To this end, first, we conduct experiments with a series of blocks composed of multiple identical branches to investigate the mechanism behind the structural re-parameterization, and then provide an interpretation. Moreover, motivated by the studies of effective receptive fields in the biological visual systems and neural networks, we propose a novel compact block named circular mask block (CMB). Given a neural network, we replace the regular convolutional layer with CMB to construct a training architecture, which can be trained to gain an accuracy boost with No extra training parameters and limited extra training FLOPs. After training, the training architecture can be transformed into the original architecture for inference. Extensive experiments are performed on CIFAR-10 and ImageNet to evaluate the effectiveness of our method. For example, we improve 0.85% top-1 accuracy of ResNet-50 on ImageNet without extra training parameters and only 11.32M extra training FLOPs, which saves 434x training FLOPs compared with prior works.