Focused-Region segmentation for light field images based on PCNN

Learning from EPI-Volume-Stack for Light Field image angular super-resolution

Light field images can record multiple information about the light rays in a scene and provide multiple views from a single image, offering a new data source for 3D reconstruction. However, ensuring the quality of light field images themselves is challenging, and distorted image inputs may lead to poor reconstruction results. Accurate light field image quality assessment can pre-judge the quality of light field images used as input for 3D reconstruction, providing a reference for the reconstruction results before the reconstruction work, significantly improving the efficiency of 3D reconstruction based on light field images. In this paper, we propose an Adaptive Vision Transformer-based light-field image-quality assessment model (AViT-LFIQA). The model adopts a multi-view sub-aperture image sequence input method, greatly reducing the number of input images while retaining as much information as possible from the original light field image, alleviating the training pressure on the neural network. Furthermore, we design an adaptive learnable attention layer based on ViT, which addresses the lack of inductive bias in ViT by using adaptive diagonal masking and a learnable temperature coefficient strategy, making the model more suitable for training on small datasets of light field images. Experimental results demonstrate that the proposed model is effective for various types of distortions and shows superior performance in light-field image-quality assessment.

Image segmentation is a very important aspect in the field of image processing. According to certain principles, it will be divided into a number of small regional disjoint. Small region is a sense sets of common attributes of the connectivity pixels. The so-called connectivity which refers to the fact that any two points in the pool existing a connectivity path between all belong to the set. Pulse coupled neural networks was proposed based on the brain simultaneously pulsed release phenomenon of the visual cortex of the cats, monkeys, and other animals, the similar input data can be outbreak synchronized through interrelated neurons. The digital image can be processed by pulse coupled neural networks, the pixels of space based approach and brightness can be made up to a group. Pulse coupled neural networks has a good nature in image processing applications, which can effectively for the various aspects of image processing, and it can be used in the image processing system. Experimental results show that the method can achieve better image segmentation and have a common applicability.

In this paper, a review of parameter optimization methods of pulse-coupled neural networks (PCNNs) is presented. Considering that PCNN has been used in image processing for many years, the aim of this paper was to provide an overview of the work that has been done and to serve as a useful reference for those who are looking for PCNN parameter optimization methods and those who are researching PCNN applications for a specific field. This paper first briefly reviews the PCNN model, including the standard PCNN and several variants of PCNN. Then, we emphasize the optimization methods for PCNN’s parameters, describing three types of parameter optimization methods in detail. Next, the paper summarizes the applications of the optimized models of PCNN with adaptive parameters in image segmentation, image fusion, image denoising and edge detection.

As light field images continue to increase in use and application, it becomes necessary to adapt existing image processing methods to this unique form of photography. In this paper we explore methods for applying neural style transfer to light field images. Feed-forward style transfer networks provide fast, high-quality results for monocular images, but no such networks exist for full light field images. Because of the size of these images, current light field data sets are small and are insufficient for training purely feed-forward style-transfer networks from scratch. Thus, it is necessary to adapt existing monocular style transfer networks in a way that allows for the stylization of each view of the light field while maintaining visual consistencies between views. To do this, we first generate disparity maps for each view given a single depth image for the light field. Then in a fashion similar to neural stylization of stereo images, we use disparity maps to enforce a consistency loss between views and to warp feature maps during the feed forward stylization. Unlike previous work, however, light fields have too many views to train a purely feed-forward network that can stylize the entire light field with angular consistency. Instead, the proposed method uses an iterative optimization for each view of a single light field image that backpropagates the consistency loss through the network. Thus, the network architecture allows for the incorporation of pre-trained fast monocular stylization network while avoiding the need for a large light field training set.

Image analysis for airborne reconnaissance and missile applications

The light field image, also known as the plenoptic image, contains the information about not only the intensity of light in a scene but also the direction of the light rays in space. Since the light field image contains very rich photometric and geometric information, it will have very widespread application in the future. For example, immersive content capture for virtual and mixed reality presentation or depth from light field for auto driving applications. To be more specific, a light field image can be enhanced with physical models for an autonomous decision making process, which is also an important task of the Dynamic Data Driven Applications Systems (DDDAS) [11]. Besides, the rich geometry and photometric information contained in the light field can be updated with real-time measurements, which is a focus of DDDAS such as smart city related image and video processing tasks. However, to make the light field images easier to be utilized, one of the most important tasks is to compress the light field images efficiently so they can be easily distributed over the current communication infrastructure.

Light Field, a computational photography technique has various exciting features like refocusing and novel view synthesis. Previous works in light field photography have focused their work on creating light field images having a limited field of view. This limitation of light field images motivates the research for a acquisition system which can capture light field images with larger field of view. Recent works have been done to acquire light fields using a cylindrical arrangement of cameras which resolve the conventional complexity issue of acquisition of light field and subsequently merging them by essentially combining the capture and merging processes into a single exercise. We use this cylindrical system to generate a novel True Zoomed Light field panorama from the captured light field, which gives a wide 360? field of view similar to traditional panoramas along with the zoom applied to the whole panorama. The captured panorama can also be manipulated in various ways as is possible with other light field recordings. Also various zooming options which can be easily found in traditional camera, either tends to change the relative shape of the objects in image or introduces pix-elation effect. We use true zoom so as to improvise the zooming effect. We generate novel “True Zoomed Light Field Panorama” using our system. This type of panorama gives user a feeling of being closer to or farther from the whole environment at the same time.

Perceptual encryption has received widespread attention as a technology for protecting multimedia visual information. In multimedia applications, perceptual encryption only protects a portion of the content without previewing all the information. At present, perceptual encryption is mostly oriented to conventional plain images while few methods aim at visual security measures for perceptually encrypted images. Existing solutions usually adopt well-known quality assessment metrics to measure the visual quality of encrypted images. However, they often exhibit undesired behavior on perceptually encrypted images with low quality. As a typical representation of three-dimensional scenes, light field images record the intensity and direction of light during propagation which is distinct from 2-D images. In this paper, we construct a perceptually encrypted light field image database (PE-LFID) for quality assessment based on 14 reference light field plaintext images. For each scene, we employ four encryption methods, each of which has six levels. Additionally, a novel visual security evaluation method based on PE-LFID is proposed by taking into account the local and global features of the light field images. First, we use a multi-threshold edge detection method to obtain the edge similarity in the spatial domain of the light field image. Afterwards, the epipolar plane image (EPI) generated from the angular domain of the light field image is used to calculate the gradient magnitude similarity, which is expressed as a global feature. Furthermore, the final quality prediction score is calculated by adaptively weighting between local and global features. We conduct extensive experiments on the proposed PE-LFID to assess the performance of classical and state-of-the-art IQA models. The experimental results demonstrate the effectiveness of the proposed method for visual security evaluation of perceptually encrypted light field images, as well as the scalability of PE-LFID.

It needs set parameters on image segmentation based on PCNN (Pulse Coupled Neural Network) now. This paper points out the new method for medical image segmentation based on improved PCNN and Tsallis entropy. The new methods can automatically segment the medical images without selecting the PCNN parameters. It gets the best results with combining with the Tsallis entropy. The new method is very useful for PCNN application in the medical images segmentation.

Visual object tracking is a fundamental research topic in computer vision. In this paper, we proposed a novel hybrid tracking method based on Pulse Coupled Neural Network (PCNN) and Multiple Instance Learning (MIL). Most modern trackers may be inaccurate when the training samples are imprecise which causes drift. To resolve these problems, MIL method is introduced into the tracking task, which can alleviate drift to some extent. However, the MIL tracker may detect the positive sample that is less important. PCNN is different from traditional artificial neural networks, which can be applied in many image processing fields, such as image segmentation. So, the PCNN was employed as sample detector which can know the most important sample when training the classifier. Then, a more robust and much faster tracker is proposed to approximately maximize the bag likelihood function. Empirical results on a large set of sequences demonstrate the superior performance of the proposed approach in robustness, stability and efficiency to state-of-the-art methods in the literature.

This paper discusses some of the advantages and disadvantages of pulse-coupled neural networks (PCNNs) for performing image segmentation in the realm of medical diagnostics. PCNNs were tested with magnetic resonance imagery (MRI) of the brain and abdominal region and nuclear scintigraphic ventilation/perfusion imagery of the lungs (V/Q scans). Our preliminary results show that PCNNs do well at contrast enhancement. They also do well at image segmentation when each segment is approximately uniform in intensity. However, there are limits to what PCNNs can do. For example, when intensity significantly varies across a single segment, that segment does not properly separate from other objects. Another difficulty is finding the optimum parameter values so that a uniform response is achieved over a set of imagery. Also, a set of parameters that properly segments objects in one image is sometimes unsuccessful on a similar image.

Pulse-coupled neural network (PCNN), which simulates the synchronous oscillation phenomenon in the visual cortex of small mammals, has become a useful model for image processing. In the model, several parameters were usually required to properly set for adjusting the behavior of neurons. However, undesired behavior may occur owing to inappropriate parameters setting. To alleviate this problem, we propose to simplify some parameters of PCNN, and apply it into image segmentation. First, exponential delay factors are abandoned for adjusting the neuron input, and the neural input is then associated with image information as well as pulse output. In addition, neural threshold inherent in PCNN is simplified as an adaptive threshold related to image properties, allowing our model to easily alter the behavior of neurons. Particularly, the characteristic of synchronous pulse is thereby kept by introducing a fuzzy clustering method, instead of linking coefficient for grouping pixels with similarity and spatial proximity through iterative computation. Experimental results on synthetic and real infrared images show that the proposed model has high performance of segmentation. Furthermore, our model has better adaptability for segmenting real-world images when compared with several existing PCNN-based methods and some classic segmentation methods.

Learning an epipolar shift compensation for light field image super-resolution

It is the key step of automatic apple picking and sorting to segment of the apple image in natural environment using image information processing technology. In order to improve the intellectualization of apple picking process and effectively extract the target of image segmentation, a novel image segmentation method of Huaniu apple based on pulse coupled neural network (PCNN) and watershed algorithm is proposed. Firstly, the PCNN model is improved and optimized on the basis of image RGB to L*A*B space conversion. Then, the improved PCNN combined with watershed algorithm was used to segment the apple image on considering the image texture and color feature information. The theoretical analysis and experimental results show that the novel method is effectively able to segment target image, and can preserve the image details and the edges as well. The proposed algorithm has better subjective vision effect and objective quality than the traditional watershed and other related algorithms.