Structural displacement estimation by fusing vision camera and accelerometer using hybrid computer vision algorithm and adaptive multi-rate Kalman filter

Infrastructure systems are tied closely to the function of human society in many aspects. Effective approaches to maintaining infrastructure are critical for public safety. The assessment of structural conditions, diagnosis of structural damage, and infrastructure safety protection may be achieved by monitoring and analyzing structural responses. Many approaches have been pursued to monitor different types of structural responses; displacement monitoring is one of the commonly used approaches. Various types of sensors have been developed to monitor structural displacement with a range of cost, ease-of-use, and effectiveness. Radar systems provide a family of high-accuracy displacement measurements. Specifically, continuous-wave (CW) radar is a promising technology for structural applications. CW radar transmits the microwave to the target continuously and captures the signal reflected. By analyzing the difference between the transmitted and reflected microwaves, the radar can measure the relative displacement between the target and the radar. Some specific types of CW radars have been developed to monitor structural displacement and have demonstrated promising performance. Radar-based structural monitoring applications are presented and specific types of CW radars developed for structural monitoring are introduced. The operating mechanism for conducting displacement measurement with CW radars is presented. The hardware designs of the CW radars and the signal processing from radar signals to final displacement information are described. The structural displacement monitoring using the CW radar is quantified in both laboratory and field validation experiments. Challenges associated with applying the CW radar to monitoring structural displacement are summarized, and future work to improve the performance of CW radar is suggested.

Structural dynamic displacement is an important indicator of the health condition of long-span bridges. Monitoring of bridge dynamic displacement has been a critical task within the context of structural health monitoring (SHM). Traditionally, the structural displacement measurement methods are contact, wired and time-consuming. With the great progress of image processing technique and vision robot science, the visionbased monitoring systems have been developed rapidly and received a huge amount of attentions in the field of SHM. However, in the practical applications, the environmental factors will affect the operational performance of the vision-based system. In this study, a series of comparative experiments are conducted to explore the robustness of vision-based structural dynamic displacement monitoring under varying illumination conditions by use of various types of vision targets (quick response (QR) code and LED lamp). The experimental results indicate that the robustness of the vision-based system is weakened under poor illumination conditions when the QR code is used to be the vision target in comparison with the vision target of LED lamp.

This article introduces an augmented adaptive unscented Kalman filter (KF). The proposed novel technique is suitable to simultaneously estimate both the diagonal process noise covariance matrix and the unknown inputs, thus combining previously reported KF estimators for unknown inputs (dual or joint KF) and for covariance matrices (adaptive KF). A selective scaling method is also introduced to improve the convergence property of the suggested KF. The development of the novel KF is also motivated by a specific estimation problem related to crane systems. Cranes represent a special class of weight handling equipment as they are underactuated and described by nonlinear dynamics such that the load present oscillatory behavior. In addition to the increasing need for their automation in various industrial fields, these features also make them a benchmark system in control engineering with numerous control laws reported in the literature for sway elimination and trajectory tracking. A common issue to realize most of the advanced control laws on real, eventually industrial size cranes is the necessity to know the sway angle and frictions on the configuration variables. It is shown in simulation and also with real experiments on a reduced size overhead crane system that the suggested KF is suitable to estimate both the sway angles and the frictions.

The needle puncture technology is one of the simplest minimally invasive medical procedures. Due to the asymmetry of the needle tip, the lateral force exerted by the tissue causes the tip to deflect and reaches the target position. It is important to maintain the needle tip in a desired plane in 2D control. In the process of puncture, the position is measured and the posture of the needle tip needs to be estimated in real time. Three estimation algorithms for estimating the states of the needle tip are employed in this paper. The first algorithm is adaptive Kalman filter (AKF), which is applied to the statistical properties of the noise is not completely known. The second is unscented Kalman filter (UKF). The last one is the combination of AKF and UKF, called adaptive unscented Kalman filter (AUKF) which combines the advantages of both. It should be noted that AKF is based on the feedback linearization model, but UKF and AUKF are directly based on the nonlinear model. On the base of analysis of three estimation algorithms, we study the control method. In this work, we estimate the states via the three methods and analyze the results. The simulation results of the estimation algorithms and the control method illustrate the differences between the three algorithms.

Contact force and torque sensing for serial manipulator based on an adaptive Kalman filter with variable time period

To decrease the cost of instrumentation for the strain and displacement monitoring method that uses sensors as well as considers the structural health monitoring challenges in sensor installation, it is necessary to develop a machine vision-based monitoring method. For this method, the most important step is the accurate extraction of the image feature. In this article, the edge detection operator based on multi-scale structure elements and the compound mathematical morphological operator is proposed to provide improved image feature extraction. The proposed method can not only achieve an improved filtering effect and anti-noise ability but can also detect the edge more accurately. Furthermore, the required image features (vertex of a square calibration board and centroid of a circular target) can be accurately extracted using the extracted image edge information. For validation, the monitoring tests for the structural local mean strain and in-plane displacement were designed accordingly. Through analysis of the error between the measured and calculated values of the structural strain and displacement, the feasibility and effectiveness of the proposed edge detection operator are verified.

Over the past decade, studying animal behaviour with the help of computer vision has become more popular. Replacing human observers by computer vision lowers the cost of data collection and therefore allows to collect more extensive datasets. However, the majority of available computer vision algorithms to study animal behaviour is highly tailored towards a single research objective, limiting possibilities for data reuse. In this perspective, pose-estimation in combination with animal tracking offers opportunities to yield a higher level representation capturing both the spatial and temporal component of animal behaviour. Such a higher level representation allows to answer a wide variety of research questions simultaneously, without the need to develop repeatedly tailored computer vision algorithms. In this paper, we first cope with several weaknesses of current pose-estimation algorithms and thereafter introduce KeySORT (Keypoint Simple and Online Realtime Tracking). KeySORT deploys an adaptive Kalman filter to construct tracklets in a bounding-box free manner, significantly improving the temporal consistency of detected keypoints. In this paper, we focus on pose estimation in cattle, but our methodology can easily be generalised to any other animal species. Our algorithm is able to detect up to 80% of the ground truth keypoints with high accuracy, with only a limited drop in performance when daylight recordings are compared to nightvision recordings. Moreover, by using KeySORT to construct skeletons, the temporal consistency of generated keypoint coordinates was largely improved, offering opportunities with regard to automated behaviour monitoring of animals. • KeySORT allows to largely improve the consistency of keypoint detection. • Robustness of multi-animal pose estimation improves by greedy keypoint association. • Adaptive Kalman filters can handle simultaneously both stable and dynamic systems.

ABSTRACTThe Grévy's zebra, an endangered species native to Kenya and southern Ethiopia, has been the target of sustained conservation efforts in recent years. Accurately monitoring Grévy's zebra populations is essential for ecologists to evaluate ongoing conservation initiatives. Recently, in both 2016 and 2018, a full census of the Grévy's zebra population was enabled by the Great Grévy's Rally (GGR), a citizen science event that combines teams of volunteers to capture data with computer vision algorithms that help experts estimate the number of individuals in the population. A complementary, scalable, cost‐effective and long‐term Grévy's population monitoring approach involves deploying a network of camera traps, which we have done at the Mpala Research Centre in Laikipia County, Kenya. In both scenarios, a substantial majority of the images of zebras are not usable for individual identification due to ‘in‐the‐wild’ imaging conditions—occlusions from vegetation or other animals, oblique views, low image quality and animals that appear in the far background and are thus too small to identify. Camera trap images, without an intelligent human photographer to select the framing and focus on the animals of interest, are of even poorer quality, with high rates of occlusion and high spatiotemporal similarity within image bursts. We employ an image filtering pipeline incorporating animal detection, species identification, viewpoint estimation, quality evaluation and temporal subsampling to compensate for these factors and obtain individual crops from camera trap and GGR images of suitable quality for re‐ID. We then employ the local clusterings and their alternatives (LCA) algorithm, a hybrid computer vision and graph clustering method for animal re‐ID, on the resulting high‐quality crops. Our method processed images taken during GGR‐16 and GGR‐18 in Meru County, Kenya, into 4142 highly comparable annotations, requiring only 120 contrastive same‐vs‐different‐individual decisions from a human reviewer to produce a population estimate of 349 individuals (within 4.6 of the ground truth count in Meru County). Our method also efficiently processed 8.9M unlabelled camera trap images from 70 camera traps at Mpala over 2 years into 685 encounters of 173 unique individuals, requiring only 331 contrastive decisions from a human reviewer.

The random noise is an important factor that affects the precision of the MEMS gyroscope. Based on the time-series analysis method, the AR model of the MEMS gyro drift signal is established. Then the adaptive Kalman filter is used to filter the drift signal. Comparison the original signal and the signal filtered by the adaptive Kalman filter, we found that the adaptive Kalman filter has the good filtering effect in the processing the zero drift signal of the MEMS gyro.

In this work, the problem of battery state of charge estimation is investigated using a model based approach. An experimentally validated model of a battery developed by AllCell Technologies, specific for light electric vehicles (electric scooter or bicycles) is used. Two state of charge estimation algorithms are developed: an extended Kalman filter and an adaptive extended Kalman filter. The adaptive version of Kalman filter is designed in order to adaptively set a proper value of the model noise covariance, using the information coming from the on-line innovation analysis. A comparison between the two approaches is conducted that shows that the adaptive Kalman filter can deal with the problem of incorrect value of the model noise covariance matrix producing lower estimation error.

Noises such as thermal noise, background noise or burst noise can reduce the reliability and confidence of measurement devices. In this work, a recursive and adaptive Kalman filter is proposed to detect and process burst noise or outliers and thermal noise, which are popular in electrical and electronic devices. The Kalman filter and neural network are used to preprocess data of three detectors of a nondispersive thermopile device, which is used to detect and quantify Fusarium spores. The detectors are broadband (1 µm to 20 µm), (6.09 ± 0.06 µm) and (9.49 ± 0.44 µm) thermopiles. Additionally, an artificial neural network (NN) is applied to process background noise effects. The adaptive and cognitive Kalman Filter helps to improve the training time of the neural network and the absolute error of the thermopile data. Without applying the Kalman filter for thermopile, it took 12 min 09 s to train the NN and reach the absolute error of 2.7453 × 104 (n. u.). With the Kalman filter, it took 46 s to train the NN to reach the absolute error of 1.4374 × 104 (n. u.) for thermopile. Similarly, to the (9.49 ± 0.44 µm) thermopile, the training improved from 9 min 13 s to 1 min and the absolute error of 2.3999 × 105 (n. u.) to the absolute error of 1.76485 × 105 (n. u.) respectively. The three-thermopile system has proven that it can improve the reliability in detection of Fusarium spores by adding the broadband thermopile. The method developed in this work can be employed for devices that encounter similar noise problems.

Transfer alignment is an initialization of the missile INS prior to launch using data provided by master INS or GPS/INS system in aircraft. Generally, Kalman filter is applied to estimate misalignment attitude. Unfortunately, the Kalman filter provides optimal estimates only if the system model is accurate and filter noise is white and accurately obtained. Considering the complicated vibrated air environment, these demands can not be satisfied all the time. Therefore, after establishing a 12-state velocity matching transfer alignment model, an improved adaptive method is derived on the basis of the study on traditional adaptive Kalman filter such as Sage-Husa and intense adaptive filter. The improved adaptive filter is aimed at modifying the model and noise matrix with the process of filtering according to the practical measurement data. Numerical simulations show that with this improved adaptive filter under a "s" turn maneuver, transfer alignment Kalman filter keeps good working condition under large range of P, Q and R, and reaches good performance on estimating precise and convergent speed.

Designing for a convergent adaptive Kalman filter (AKF) which can be used in time-varying stochastic linear systems is considered. A necessary condition which can guarantee the designed AKF is convergent is proposed and proved. This necessary condition can be satisfied by choosing appropriate gains. In fact, choosing the gains is just a procedure of designing a convergent AKF. The AKF designed by the above procedure is convergent and only correlative with coefficient matrices of the system. So, it adapts to deal with time-varying stochastic linear systems. At last, the findings in this paper are validated by some simulations.

Magnetohydrodynamic (MHD) activity in fusion devices is typically analyzed by examining time-frequency spectrograms obtained from various diagnostics. MHD modes often co-exist with various types of noise and complex patterns generated by other events like pellet injection or active diagnostics. Traditionally, identifying MHD modes has been a manual task, making it labor-intensive. To overcome this issue, this study proposes the use of computer vision (CV) algorithms for noise removal and automatic feature extraction. First, the automatic detection of straight-line patterns is achieved by applying the Hough transform. Then, the discrete wavelet transform is proposed to break down spectrograms into sub-images of different scales, removing broadband noise and pellet injection signatures. The multiscale decomposition is subsequently extended to multiple directions using either 2D Fourier transforms or curvelets, achieving a high signal-to-noise ratio in spectrograms and eliminating undesired frequency sweeps of toroidal Alfvén eigenmodes antenna. Once MHD activity is successfully enhanced, a pipeline of algorithms for ridge detection, thresholding and labeling perform a segmentation of the image, automatically labeling individual modes. This study demonstrates the effectiveness of CV algorithms for the identification of MHD modes. The use of such algorithms may potentially help in the analysis process and the creation of large databases of modes.

Given the complex traffic factors and strong mobility of a transit vehicles, it is difficult to predict its travel time with the conventional prediction models. The emphasis of this research effort is to improve the adaptability of the Kalman filter model with regard to transit bus travel time prediction. In order to resolve the filter divergence and calculated divergence in the prediction process, an adaptive fading Kalman filter algorithm is put forward in this paper. Compared with the conventional Kalman filter prediction model, a "forgotten factor" is added to the adaptive fading Kalman filter algorithm to restrain the influence of the old data on the filter. Furthermore, it can correct the predictive value and the model parameters with the latest measured value, hence higher adaptability and predictive accuracy. Finally, the GPS data of transit vehicles in the city of Yichun is used in the paper to verify the effectiveness of the new model. The results reveal that the adaptive fading Kalman filter algorithm is more effective than the conventional one. It performs better in terms of convergence and dynamic adaptability, implying a brighter application prospect.