Regularization approach to scene-based nonuniformity correction

Nonuniform response and lower dynamic range are two prevalent issues in the infrared focal plane array (IRFPA) imaging system. Adjusting the integration time is an efficient way to improve the dynamic range of IRFPA imaging system. However, in scene-based nonuniformity correction (SBNUC), with the change of integration time, the correction parameters are no longer applied and it needs to re-converge to get new ones. Therefore, a novel algorithm, combining one-point calibration (based on adaptive adjusting integration time) with neural network (NN) correction, is presented in this paper. The method firstly acquires images based on the blackbody source at different integration time and then adopts a linear regression technique to estimate them as the one-point calibration parameter. Furthermore, it performs the nonuniformity compensation using a neural network approach. In addition, the proposed method only needs to store two regression coefficients. In this way, the algorithm reduces the nonuniformity caused by the change of integration time, as well as denoises the fixed pattern noise (FPN). Experiments on a real infrared video sequence show that the proposed algorithm has improved the convergence rate and the quality of the corrected image, also has the advantage of adaptive adjusting integration time.

In this paper, a novel scene-based nonuniformity correction (SBNUC) method using statistics between adjacent detectors for infrared focal-plane array (IRFPA) is proposed. Differential operation is designed as a decorrelation process to minimize the scene information remained into the nonuniformity parameters updating. Then, a global transfer matrix is established to construct the nonuniformity compensation matrix for the entire image. The adjacent differential statistics are demonstrated to be more effective and sufficient to estimate the nonuniformity when temporal variation is lacking. In addition, the proposed method exhibits the more superior capability of converging and does better in correcting all spatial components of nonuniformity. Application in simulated imagery and real infrared image shows the proposed method's excellent performance in convergence and elimination of ghosting artifacts.

The explosive growth of online short videos has brought great challenges to the efficient management of video content classification, retrieval, and recommendation. Video features for video management can be extracted from video image frames by various algorithms, and they have been proven to be effective in the video classification of sensor systems. However, frame-by-frame processing of video image frames not only requires huge computing power, but also classification algorithms based on a single modality of video features cannot meet the accuracy requirements in specific scenarios. In response to these concerns, we introduce a short video categorization architecture centered around cross-modal fusion in visual sensor systems which jointly utilizes video features and text features to classify short videos, avoiding processing a large number of image frames during classification. Firstly, the image space is extended to three-dimensional space–time by a self-attention mechanism, and a series of patches are extracted from a single image frame. Each patch is linearly mapped into the embedding layer of the Timesformer network and augmented with positional information to extract video features. Second, the text features of subtitles are extracted through the bidirectional encoder representation from the Transformers (BERT) pre-training model. Finally, cross-modal fusion is performed based on the extracted video and text features, resulting in improved accuracy for short video classification tasks. The outcomes of our experiments showcase a substantial superiority of our introduced classification framework compared to alternative baseline video classification methodologies. This framework can be applied in sensor systems for potential video classification.

In this paper, an improved scene-based nonuniformity correction (NC) algorithm for infrared focal plane arrays (IRFPAs) using multiframe registration and iteration method is proposed. This method estimates the global translation and iterates between several adjacent frames. Then mean square error between any two properly registered images is minimized to obtain nonuniformity correction parameters. The detailed method includes three main steps: First, we assume that brightness along the motion trajectory is constant, and a linear detector response and model the nonuniformity of each detector with a gain and a bias. Second, several adjacent frames are used to compute relative motion of any two adjacent frames. Here we use the Fourier shift theorem, their relative translation can be obtained by calculating their normalized cross-power spectrum. We choose K adjacent frames, so the total number of iteration is K*(K-1)/2. Then the mean square error function is defined as the corresponding difference between the two adjacent corrected frames, and it is minimized making use of the least mean square algorithm. The use of correlation of adjacent frames sufficiently, together with iteration strategy between them, can get fast and reliable fixed-pattern noise reduction with low few ghosting artifacts. We define the algorithm and present a number of experimental results to demonstrate the efficacy of the proposed method in comparison to several previously published methods. The performance of the proposed method is thoroughly evaluated with clean infrared image sequences with synthetic nonuniformity and real infrared imagery.

Abstract. Real-time change detection and analysis of natural disasters is of great importance to emergency response and disaster rescue. Recently, a number of video satellites that can record the whole process of natural disasters have been launched. These satellites capture high resolution video image sequences and provide researchers with a large number of image frames, which allows for the implementation of a rapid disaster procedure change detection approach based on deep learning. In this paper, pixel change in image sequences is estimated by optical flow based on FlowNet 2.0 for quick change detection in natural disasters. Experiments are carried out by using image frames from Digital Globe WorldView in Indonesia Earthquake took place on Sept. 28, 2018. In order to test the efficiency of FlowNet 2.0 on natural disaster dataset, 7 state-of-the-art optical flow estimation methods are compared. The experimental results show that FlowNet 2.0 is not only robust to large displacements but small displacements in natural disaster dataset. Two evaluation indicators: Root Mean Square Error (RMSE) and Mean Value are used to record the accuracy. For estimation error of RMSE, FlowNet 2.0 achieves 0.30 and 0.11 pixels in horizontal and vertical direction, respectively. The error in horizontal error is similar to other algorithms but the value in vertical direction is significantly lower than them. And the Mean Value are 1.50 and 0.09 pixels in horizontal and vertical direction, which are most close to the ground truth comparing to other algorithms. Combining the superiority of computing time, the paper proves that only the approach based on FlowNet 2.0 is able to achieve real-time change detection with higher accuracy in the case of natural disasters.

The non-uniform response of infrared focal plane array (IRFPA) detectors has a serious impact on image quality. First, the image is not clear visually, there are serious fixed mode noise, and second, the dynamic range, which reduces the resolution of infrared image. In view of the shortcomings of the existing non-uniformity correction schemes and the actual engineering requirements, this paper proposes a scheme combining registration-based correction algorithm with micro-motion system, which calculates the relative displacement between two adjacent images by projection estimation. At this time, the scene motion is judged. When the motion is inadequate, the scene motion is controlled to make the controller operate according to a certain trajectory, so that the scene conditions can meet the requirements of the correction algorithm as far as possible, solve the practical application requirements, make the image quality achieve better results, and perform the performance of the algorithm. The test results show that the scheme can effectively correct the non-uniformity and obtain better correction effect.

Analyses of infrared focal plane array figure of merit and its impact on sensor system trades

Calibration-based NUC Method in Real-time Based on IRFPA

Obtaining videos for surveillance purpose or to use them for future predictions is a challenging task as a video has a large number of image frames displayed in a sequence, and modeling every frame is not possible, so various methods are used for building an intelligent vision system, which is used for obtaining videos and also in video anomaly detection. This paper provides an overview of research directions for different types of anomalies and also tells about different techniques in machine learning for managing the problem of anomaly detection in videos and images using computer vision. Computer vision is used to make computers capable of extracting information from digital images or videos. It trains computers to interpret and understand the visual world. When machines detect errors, abnormal or unnatural behavior of datasets, it is called anomaly detection. In this paper, anomaly detection techniques and anomaly detection in datasets using computer vision are classified accordingly.KeywordsAnomaly detectionOutliersComputer visionArtificial intelligenceFeature extractionClassification

In scene-based nonuniformity correction (NUC) methods for infrared focal plane array cameras, the statistical approaches have been well studied because of their lower computational complexity. However, when the assumptions imposed by statistical algorithms are violated, their performance is poor. Moreover, many of these techniques, like the global constant statistics method, usually need tens of thousands of image frames to obtain a good NUC result. In this paper, we introduce a new statistical NUC method called the multiscale constant statistics (MSCS). The MSCS statically considers that the spatial scale of the temporal constant distribution expands over time. Under the assumption that the nonuniformity is distributed in a higher spatial frequency domain, the spatial range for gain and offset estimates gradually expands to guarantee fast compensation for nonuniformity. Furthermore, an exponential window and a tolerance interval for the acquired data are introduced to capture the drift in nonuniformity and eliminate the ghosting artifacts. The strength of the proposed method lies in its simplicity, low computational complexity, and its good trade-off between convergence rate and correction precision. The NUC ability of the proposed method is demonstrated by using infrared video sequences with both synthetic and real nonuniformity.

The nonuniformity of infrared focal plane array (IRFPA) seriously affects the image quality and the measurement accuracy of the infrared imaging system. The two-point nonuniformity correction (NUC) method based on blackbody calibration performs a less than very satisfactory result to correct the sky scene with gradient gray distribution. Aiming at this problem, a new NUC method based on gradient scene calibration is proposed. First, the correction models are deduced in theory with the reference of the blackbody and the gradient scene. Then the pure sky images at the two different pitch angles are used as two reference temperature points to obtain the gain and offset coefficients by two-point correction. Next, two correction coefficients are filtered to remove residual noise. Finally, an outfield experiment with the proof-of-concept camera is performed. Experimental results show that the signal-to-noise ratio (SNR) of the point target image corrected by the proposed method is increased from 5.56 to 9.84 compared with the blackbody calibration-based NUC. And it has the preference of low complexity and easy implementation in the real-time system.

In this work, we introduce a temporal-spatial approach for infrared focal plane array (IRFPA) stripe nonuniformity correction in infrared images that generates visually appealing results. We posit that the nonuniformity appears as a striped structure in the spatial domain and that the pixel values change slowly in the temporal domain. Based on this, we formulate our correction method in two steps. In the first step, weighted guided image filtering with our adaptive weight is utilized to predict the stripe nonuniformity using a single frame. In the second step, the temporal profile of each pixel can be formed using a few frames of successive nonuniformity images. Further, we present a temporal nonlinear diffusion equation to remove scene residuals from the temporal profile of nonuniformity images in order to estimate a more accurate value of the stripe nonuniformity. The results of extensive experiments demonstrate that the proposed nonuniformity correction algorithm substantially outperforms many state-of-the-art approaches, including both traditional and deep convolution-neural-network-based methods, on four popular infrared videos. In addition, the proposed method only requires a fraction (less than ten) of the video frames.

In this paper a novel nonuniformity correction method that compensates for the fixed-pattern noise (FPN) in infrared focal-plane array (IRFPA) sensors is developed. The proposed NUC method compensates for the additive component of the FPN statistically processing the read-out signal using a noise-cancellation system. The main assumption of the method is that a source of noise correlated to the additive noise of the IRFPA is available to the system. Under this assumption, a finite impulse response (FIR) filter is designed to synthesize an estimate of the additive noise. Moreover, exploiting the fact that the assumed source of noise is constant in time, we derive a simple expression to calculate the estimate of the additive noise. Finally, the estimate is subtracted to the raw IR imagery to obtain the corrected version of the images. The performance of the proposed system and its ability to compensate for the FPN are tested with infrared images corrupted by both real and simulated nonuniformity.

This paper proposes an energy function method that can evaluate power system dynamic behaviors, especially on whether synchronization can be achieved or not after line outages by quasi-dynamic simulation instead of time consuming dynamic simulations. To judge the synchronization of generation, potential energy and energy dissipation of the system are considered in the proposed energy function. By using the energy function, whether the equilibrium before line outages stays inside the basin of attraction of the equilibrium after line outages can be predicted. The proposed method is tested on the IEEE 39-bus system by dynamic simulations. It is shown that the proposed method can correctly predict the synchronization condition of generators.

Active contours or snakes are widely used for segmentation and tracking. We propose a new active contour model, which converges reliably even when the initialization is far from the object of interest. The proposed segmentation technique uses an external energy function where the energy slowly decreases in the vicinity of an edge. This new energy function is calculated using an efficient dual scan line algorithm. The proposed energy function is tested on computational speed, its effect on the convergence speed of the active contour and the segmentation result. The proposed method gets similar segmentation results as the gradient vector flow active contours, but the energy function needs much less time to calculate.