Automatic target detection and tracking in forward-looking infrared image sequences using morphological connected operators

This paper presents a technique for automatic airborne target recognition and tracking in forward-looking infrared (FLIR) images with a complex background. An image splitting and merging method is applied for detecting target signals. The presence of a complex background due to clouds and sun glint generates clutter in the image with the resulting possibility of false alarms. A Bayesian classifier trained using the NMI (normalized moment of inertia) feature is proposed for efficient clutter rejection. After classification, target candidates are entered into a tracking filter. As an efficient and robust multi-target tracking filter in cluttered environments, the JDC-JIHPDAF is proposed. Experimental results using a wide range of real FLIR images ensure reliable classification and automatic target recognition performance.

Robust real-time ship detection and tracking for visual images automatically has become one of the crucial requirements for numerous situations. In order to improve the performance of automatic ship target detection and tracking system, a novel method based on mean shift is proposed for automatic detection and tracking of ship in the corrected video sequences. Firstly, video frames are corrected based on real-time attitude of the imaging equipment in order to decrease the drift of the ship target from frame to frame. Secondly, sea horizon is extracted based on Ostu algorithm and Hough transform. Thirdly, the ship location is detected based on the detection of grayscale peak. Finally, the ship is automatically tracked by using the mean shift algorithm. The result shows that this method has not only improved the robustness of the system against the shaking while tracking the ships, but also achieved a high success rate of tracking ships automatically in corrected video sequences.

(U) Future successful ballistic missile booster intercepts will require advanced automatic target detection, tracking, classification and identification (ADTCI) image processing techniques. Two such techniques are presented in this classified SECRET paper using the synthetic scene generator model (SSGM) in combination with the advanced systems (AVS) image processing package. Two challenging multispectral cases are treated: (1) missile hardbody occultation by the missile exhaust plume, and (2) variable plume/hardbody system (PHS) gradient intensities generated by missile tumbling due to exiting the sensible atmosphere. The target detection, tracking and edge extraction methods selected for this study include morphological, open-close operations within decision- level fusion for the obscuration case and pixel-level fusion for variable edge intensities. Other investigators have approached this issue on similar image processing techniques. The multispectral (2.69 - 2.95 micrometer SWIR; 4.17 - 4.2, 4.35 - 4.50 micrometer MWIR; and 8.0 - 12.0 micrometer LWIR) target/background imagery includes SWIRM/MWIR boost phase track (with occlusion problem) and LWIR aimpoint selection (with tumbling problem). The two classified missile systems are: (1) a depressed-angle submarine launched ballistic missile (SLBM) and (2) a medium range ballistic missile (MRBM). The results indicate that for 6 degrees of freedom (6 DOF) hardbodies, ATDCI geometrical pattern reference libraries should be optimized to accommodate the extreme variable gradient geometries for tumbling midcourse targets. For boost- phase missile hardbody occultation by missile exhaust plumes, segmentation and feature extraction should be implemented in each bandpass before processing to the ATDCI classifier. This study demonstrates that although the plume/hardbody system edges were extracted, the geometry of the target edge often deviated from symmetry.

Target detection and tracking is a key technology in intelligent monitoring system. However, it is still difficult to design a robust, accurate and real-time target detection and tracking algorithm. Taking target Tracking as the research focus, this paper focuses on one of the current mainstream algorithms, namely, TLD (tracking-learning-detection) target Tracking algorithm. Based on the theory of TLD target tracking and corner detection, a new target tracking method was proposed to solve the existing problems. A classifier is built online to track and learn the target in real time, and a target contour detector is added in the system to complement the random forest detector to solve the problem of tracking failure. Experimental results show that the proposed algorithm can effectively overcome the influence of fast change or depth occlusion, and improve the tracking efficiency of moving targets.

Automatic detection and tracking of maritime targets in imagery can greatly increase situation awareness on naval vessels. Various methods for detection and tracking have been proposed so far, both for reasoning as well as for learning approaches. Learning approaches have the promise to outperform reasoning approaches. They typically detect targets in a single frame, followed by a tracking step in order to follow targets over time. However, such approaches are sub-optimal for detection of small or distant objects, because these are hard to distinguish in single frames. We propose a new spatiotemporal learning approach that detects targets directly from a series of frames. This new method is based on a deep learning segmentation model and is now applied to temporal input data. This way, targets are detected based not only on appearance in a single frame, but also on their movement over time. Detection hereby becomes more similar to how it is performed by the human eye: by focusing on structures that move differently compared to their surroundings. The performance of the proposed method is compared to both ground-Truth detections and detections of a contrast-based detector that detects targets per frame. We investigate the performance on a variety of infrared video datasets, recorded with static and moving cameras, different types of targets, and different scenes. We show that spatiotemporal detection overall obtains similar to slightly better performance on detection of small objects compared to the state-of-The-Art frame-wise detection method, while generalizing better with fewer adjustable parameters, and better clutter reduction. © 2019 SPIE.

This paper proposes a method to detect objects of arbitrary poses and sizes from a complex forward looking infrared (FLIR) image scene exploiting image correlation technique along with the preprocessing of the scene using a class of morphological operators. This presented automatic target recognition (ATR) algorithm consists of two steps. In the first step, the image is preprocessed, by employing morphological reconstruction operators, to remove the background as well as clutter and to intensify the presence of both low or high contrast targets. This step also involves in finding the possible candidate target regions or region of interests (ROIs) and passing those ROIs to the second step for classification. The second step exploits template-matching technique such as minimax distance transform correlation filter (MDTCF) to identify the true target from the false alarms in the pre-selected ROIs after classification. The MDTCF minimizes the average squared distance from the filtered true-class training images to a filtered reference image while maximizing the mean squared distance of the filtered false-class training images to this filtered reference image. This approach increases the separation between the false-class correlation outputs and the true-class correlation outputs. Classification is performed using the squared distance of a filtered test image to the chosen filtered reference image. The proposed technique has been tested with real life FLIR image sequences supplied by the Army Missile Command (AMCOM). Experimental results, obtained with these real FLIR image sequences, illustrating a wide variety of target and clutter variability, demonstrate the effectiveness and robustness of the proposed method.

Over the last two decades, researchers investigated various approaches for detection and classification of targets in forward looking infrared (FLIR) imagery using correlation based techniques. In this paper, a novel technique is proposed to recognize and track single as well as multiple identical and/or dissimilar targets in real life FLIR sequences using a combination of extended maximum average correlation height (EMACH) and polynomial distance classifier correlation filter (PDCCF). The EMACH filters are used for the detection stage and PDCCF filter is used for the classification stage for improving the detection and discrimination capability. The EMACH and PDCCF filters are trained a priori using target images with expected size and orientation variations. In the first step, the input scene is correlated with all the detection filters (one for each desired or expected target class) and the resulting correlation outputs are combined. The regions of interest (ROI) are selected from the input scene based on the regions with higher correlation peak values in the combined correlation output. In the second step, PDCCF filter is applied to these ROIs to identify target types and reject clutters/backgrounds based on a distance measure and a threshold. Moving target detection and tracking is accomplished by applying this technique independently to all incoming image frames. Independent tracking of target(s) from one frame to the other allows the system to handle complicated situations such as a target disappearing in few frames and then reappearing in later frames. This method has been found to yield robust performance for challenging FLIR imagery in terms of faster and accurate detection and classification as well as tracking of the targets.

This paper deals with the problem that how to use the distance information of millimeter wave (MMW) radar in the target-tracking of forward looking infrared (FLIR) image sequences. Many conventional image based trackers use timing updating to update template, but how to confirm an appropriate updating period is a difficult problem. However, the presence of noise and image magnification as the camera approaches the target may cause a tracked template to drift. In a MMW/FLIR dual-mode seeking guidance system, MMW radar can provide the precise distance between the missile and the target. The distance information can be used to direct how to update template. Firstly, a distance updating strategy is confirmed after image blowup's influence on pertinence is analyzed. Secondly, magnified template is used to direct template updating after images are magnified by distance information. Lastly, an adaptive template updating algorithm using distance information is presented. Results acquired for real FLIR images obtained from an aircraft are presented. Experiments results show that the higher tracking accuracy and the stronger stability are obtained in the algorithm proposed than those in other methods.

Automatic target tracking in forward-looking infrared (FLIR) imagery is a challenging research area in computer vision. This task could be even more critical when real-time requirements have to be taken into account. In this context, techniques exploiting the target intensity profile generated by an intensity variation function (IVF) proved to be capable of providing significant results. However, one of their main limitations is represented by the associated computational cost. In this paper, an alternative approach based on genetic algorithms (GAs) is proposed. GAs are search methods based on evolutionary computations, which exploit operators inspired by genetic variation and natural selection rules. They have been proven to be theoretically and empirically robust in complex space searches by their founder, J. H. Holland. Contrary to most optimization techniques, whose goal is to improve performances toward the optimum, GAs aim at finding near-optimal solutions by performing parallel searches in the solution space. In this paper, an optimized target search strategy relying on GAs and exploiting an evolutionary approach for the computation of the IVF is presented. The proposed methodology was validated on several data sets, and it was compared against the original IVF implementation by Bal and Alam. Experimental results showed that the proposed approach is capable of significantly improving performances by dramatically reducing algorithm processing time.

In recent years, in addition to classical machine vision algorithms, AI-based techniques such as image segmentation and object detection and recognition have been utilised in the transport sector for various purposes. These include vehicle-on-board applications for driver assistance or autonomous “self-driving”, such as object detection for collision avoidance and traffic monitoring and management applications. One application in the field of aviation is “remote tower”. “Tower” air traffic controllers work by observing the traffic at airports directly from a control tower building. Remote tower uses video cameras to replace the “out-of-the-window” tower view, enabling the physical tower to be eliminated and allowing controllers to provide services from a remote location. Digitization of the image stream also gives the opportunity to develop functions to support the controller tasks and improve situational awareness.This paper describes the development of a system to support remote tower operators in airport traffic control. The system features pan-tilt-zoom (PTZ) cameras with an automatic tracking function for aircraft flying in the vicinity or moving on the surface of the airport. The function operates in two modes: “sensor” tracking based information from an aircraft surveillance sensor, and “optical” tracking based on recognising and tracking aircraft or ground vehicles from PTZ camera images.This paper discusses the challenges and solutions concerning the automatic tracking function of the PTZ camera utilising image recognition.Challenges in Automatic Object Tracking Using Image Recognition:This research has developed functionality for recognising target such as aircraft and vehicles at airports and their surroundings in video images using YOLO. YOLO incorporates both target extraction and recognition processes within images, having the advantage of enabling rapid processing up to target recognition.On the other hand, several challenges arise when using YOLO recognition results for target tracking. One such challenge is the occurrence of “switching” during target tracking. Target switching refers to instances, when multiple aircraft are present in the image, tracking shifts to an object with higher recognition accuracy, or where tracking mistakenly follows another target if the originally recognised and tracked target was lost by an obstacle. As our first trial, to suppress this “switch”, we attempted a method whereby the velocity vectors of each object are calculated from the image recognition results, and the difference between the velocity vectors of the target object and other recognised objects is used as a discrimination parameter.This method determined how closely “current movement” matched “past movement”. These were estimated as a cubic function between “current movement” and “past movement”, using information on the “last position of the past movement”, the “first position of the current movement”, the “velocity of the past movement”, and the “velocity of the current movement”. Movements were judged to be closer together when the coefficients for the cubic “change in acceleration” and the quadratic “acceleration” were smaller. However, as a result, when aircraft crossed at slow speeds, fluctuations in the image recognition segmentation frames interfered, preventing the detection of velocity vectors. Furthermore, in cases where detection was unstable due to factors such as aircraft becoming obscured, resulting in gaps in the data, the coefficients became smoother. This led to instances where other targets were misidentified.Object identification based on posture detection:This paper investigated whether continuous tracking of target aircraft could be achieved even in crossing situations by learning aircraft orientation using YOLO v8 and identifying the aircraft's orientation. To recognise aircraft orientation, the ‘Posing’ function was utilised for ‘Pose’ learning. Annotation data comprising six points—nose, fuselage, right wing, left wing, tail, and vertical tail fin—was created and used for training.In the case study, we could determine the aircraft's orientation through image recognition. By utilising this orientation recognition result as a determination parameter for tracking, we confirmed that continuous tracking is achievable even in situations where automatic target tracking would previously be impossible using only velocity vectors.

A new background estimation and suppression algorithm was presented. In the algorithm, targets and observing noises were considered as mixed interferences of the image background. With this situation, image background was estimated adaptively and then background suppression was done in order to improve the signal-to-noise ratio (SNR) of targets. In this algorithm, firstly, a Zernike-facet model of image background was built up. Secondly, the total least squares (TLS) method was used to solve parameters of the model. Finally, background estimation and suppression were done using the model and its parameters. Simulations and several experiments demonstrating the effectiveness of this proposed algorithm were reported. And results show that this algorithm can be effective to estimate background in mixed noise environment and can preserve detail information of targets and improve SNR of targets. As a result, detecting probability and false probability will be improved in next process for automatic target detection and tracking.

A sequential dual-energy subtraction technique for image-guided radiation therapy (IGRT) was developed. Here, we report on a computerized method for creating sequential soft-tissue images and the accuracy of tracking targets on the images obtained, in comparison to conventional fluoroscopic images. Two sets of sequential chest images during respiration of a normal subject were obtained with X-rays of different energy separately with a flat-panel detector (FPD). Sequential soft-tissue images were created from the two sets of sequential images consisting of real-time images and reference template images, respectively. The creation of sequential soft-tissue images consisted of three steps: one-to-one image correspondence of the two sequential images, image registration, and image subtraction in each frame. Motion tracking of lung vessels was then performed by the template-matching technique. For evaluation of the accuracy of motion tracking on the sequential soft-tissue images, the results were compared with those on the original sequential images. Sequential soft-tissue images provided more accurate tracking than the original images (P < 0.01). There was no significant error throughout all frames in the soft-tissue images, whereas the rib shadow introduced a tracking error in the original images. The maximum errors were 4.1 +/- 0.3 mm in the sequential soft-tissue images and 28.1 +/- 20.0 mm in the original images. In conclusion, sequential soft-tissue images were helpful for tracking of a target affected by respiratory motion. Dual-energy subtraction has the potential to improve the accuracy of IGRT without implanted markers.

Many automatic target recognition, detection, and identification problems usually suffer from lack of adequate resolution of the data, especially among infrared imaging systems. A number of super-resolution reconstruction algorithms have been proposed. The challenge is how to recapture additional high-frequency information from adjacent frames in an image sequence that contains slightly different, but unique, information. In addition, real-world infrared sequence images are noisy and low contrast, and low spatial resolution. Since broad-banded noise mainly affects high-frequency information to be recaptured, the challenge is how to avoid smoothing out the high-frequency data by the regularization are not smoothed out. This paper presents a new super-resolution reconstruction approach based on wavelet domain for super-resolution image reconstruction of infrared IR sequences Minimizing the regulation cost function in wavelet domain forms a multi-scale high-resolution estimate. The effects of noise are incorporated into the iterative process in the proposed method. The estimation errors in high- and low- frequency bands are processed separately to solve the problem of variable correlations of observed images and slow convergence. The proposed approach was tested on the infrared aerial image sequences provided by Defense Research Establishment in Valcartier. Experiment results show that a significant increase in the spatial resolution can be achieved by the proposed approach while the noise is smoothed out.

Automatic target tracking in clutter initiates and updates both true tracks and false tracks. True tracks follow targets, and false tracks do not. False track discrimination is the procedure which confirms (vast majority of) true tracks, and terminates (vast majority of) false tracks. False track discrimination requires a measure of track quality to distinguish between true tracks and false tracks in a statistical sense. This paper compares two powerful track quality measures; the track score, as used in multi hypothesis tracking (MHT) and the probability of target existence, as used in integrated track splitting (ITS) filter. Both theoretical and simulation comparisons are presented in a single target tracking situation

In this paper, a new real-time and intelligent target tracking system basing on the pan/tilt controlled-stereo camera embedded on an UGV (unmanned ground vehicle) is proposed. In the proposed system, firstly the face area of a moving person is detected from a sequence of left images captured by a stereo camera using a threshold value of YCbCr color model and then, the stereo camera is kept to track the moving target in real-time by controlling the pan/tilt system. Secondly a depth map of the target face is extracted by using a relationship between the geometry of a stereo camera system and the disparity map of a rectified stereo image. Finally, from this extracted depth map, a distance from the camera system to the target and the coordinates of the target are calculated and these target coordinates are used for extracting 3D information of the target to plan paths and construct the coordinates maps. From some experiments using 1,280 frames of the input stereo image pairs, it is analyzed that displacement on the center position of the face after tracking is kept to be very low values of 0.6% for 1,280 frames on average. Also, the error ratio between the measured and computed distance values of the target from the camera system is kept to be very low value of 0.5% on average. In addition, the proposed system has achieved a speed of 0.04 sec/frame for target detection and 0.06 sec/frame for target tracking. These experimental results suggest a possibility of implementing a practical stereo camera-based automatic target tracking system in the UGV having a high degree of accuracy and a very fast response time.© (2004) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.