Traffic Camera Calibration via Vehicle Vanishing Point Detection

For images taken in man-made scenes, vanishing points and focal length of camera play important roles in scene understanding. In this paper, we present a novel method to quickly, accurately and simultaneously estimate three orthogonal vanishing points (TOVPs) and focal length from single images. Our method is based on the following important observations: If we establish a polar coordinate system on the image plane whose origin is at the image center, angle coordinates of vanishing points can be robustly estimated by seeking peaks in a histogram. From the detected angle coordinates, altitudes of a triangle formed by TOVPs are determined. Novel constraints on both vanishing points and focal length could be obtained from the three altitudes. By using the constraints, radial coordinates of TOVPs and focal length can be estimated simultaneously. Our method decomposes a 2D Hough parameter space into two cascaded 1D Hough parameter spaces, which makes our method much faster and more robust than previous methods without losing accuracy. Enormous experiments on real images have been done to test feasibility and correctness of our method.

This paper proposes an effective and practical method for self-calibration of traffic surveillance cameras. Based on analyzing multiple moving vehicles across multiple frames, the Canny edge detector and Hough transform are first adopted to obtain orthogonal horizontal vanishing points pairs, from which corresponding vertical vanishing points are derived. Next, mean shift clustering and Laplace linear regression are employed to deal with noise and outliers during estimation of vanishing points. To overcome the unreliable estimation issues of orthogonal vanishing points pairs, we further utilize the projective line segments obtained from 3-D vehicle model to create more reliable pairs and iteratively improve the calibration results. Finally, the estimation of distribution algorithm (EDA) is also applied to relax the assumptions made on camera parameters and the moving trajectories of vehicles during the iterations. Experimental results on different datasets prove the feasibility of our proposed scheme.

Spherical objects and vanish points are often used for camera calibration. An occluding contour of a sphere is projected to a conic in the perspective image, and using a moving active camera, the trajectory of a vanishing point in the perspective images is also a conic when the camera is rotated about a fixed 3D axis whereas the translation of the camera is arbitrary. In fact, the problems of camera calibration using conics from spheres or vanishing points can be described by same mathematic representations. Two linear approaches to the problems are proposed in this paper: one based on the geometric interpretation of the relation between image conics and the image of the absolute conic, and the other using the special structure of the problems in algebra. Only three such conics are needed for the two linear approaches, and the minimum number for previous nonlinear optimization methods is also three. All five intrinsic parameters are recovered linearly without making assumptions, such as, zero-skew or unitary aspect ratio which are often used in previous methods. The two linear algorithms have been tested in extensive experiments with respect to noise sensitivity and also made comparisons with recent calibration techniques.

Estimating a vanishing point (VP) is a core problem for understanding three-dimensional scenes and autonomous navigation. Existing methods are essential to estimating VPs in indoor and urban environments. However, doing so in diverse, unstructured, changing, and unexpected field environments remains a considerable challenge. Traditional methods of estimating structural VP have some shortcomings as they rely heavily on feature-intensive computation, making them less reliable due to a lack of adequate structures in a field environment due to disorganized disturbances. Inspired by the oblique effect, neurons prefer to respond to horizontal and vertical stimuli more than to diagonal, which can help estimate VPs. This study proposes a methodology to estimate VPs from a monocular camera for a field environment. Local orientation features are assigned to clusters inspired by the oblique effect. By extracting end points of different clusters, virtual local orientation features are reshaped. Based on geometric inferences of orientation, a VP is approximately estimated using optimal estimation and self-selectability. No prior training is needed, and camera calibration and internal parameters are not required. This approach is robust to changes in color and illumination using geometric inference, making it a perfect fit for field environments. Experimental results demonstrated that the method can successfully estimate VPs. This study presents a groundbreaking approach to evaluating VPs using a monocular camera. Inspired by the oblique effect, our method relies on explainable geometric inferences instead of prior training, resulting in a highly robust model that can handle changes in color and illumination. Our proposed approach significantly advances scene understanding and navigation, making it an ideal solution for field environments.

Road detection is important in computer vision for autonomous driving, pedestrian detection and other applications. Visible light (VL) camera is often used for daytime road detection, and infrared camera is often used for night road detection. Vanishing point (VP) detection is useful for inferring road region. Many VP detection methods have been proposed and applied successfully in VL road image. However, there is no special VP detection method for infrared road image. In this paper, we propose a VP detection approach for infrared road detection. The novelty of our approach relies on the rational assumption that the regions are very similar along the direction of the true VP. This assumption is involved in finding effective VP voters by using a non-local similarity manner, and these VP voters estimate the VP together. Quantitative and qualitative experiments show the effectiveness and efficiency of the proposed method.KeywordsVanishing point detectionInfrared Road detectionNon-local similarity

The purpose of this study is to develop an indoor guidance method based on augmented reality (AR) that superimposes annotations on an indoor environment (e.g., wall surfaces) and works on devices with limited computing power such as smartphones. Such a method requires a simple algorithm for the object registration. For the purpose, this paper proposes a two-dimensional (2D) object registration method based on perspective distortion correction using vanishing points in the scene and a stable vanishing point detection method. Current vanishing point detection methods have the problem of slight drift of the vanishing point in the videos despite maintaining a fixed camera position. The proposed method has solved the problem by employing a weighted particle filtering method. Recent approaches often employ deep neural networks (DNNs) for the registration and the stable vanishing point detection, however DNNs require rich resources for computing such as Graphics Processing Unit (GPU). Since the proposed method requires at most two vanishing points for the registration, the method can be said a lightweight method. This paper shows that the proposed weighted particle filtering method suppresses the vanishing point drift better than the conventional moving average method, and that it enables registration of  objects based on the method.

We propose a novel approach that integrates underparameterized RANSAC (UPRANSAC) with Hough Transform to detect vanishing points (VPs) from un-calibrated monocular images. In our algorithm, the UPRANSAC chooses one hypothetical inlier in a sample set to find a portion of the VP’s degrees of freedom, which is followed by a highly reliable brute-force voting scheme (1-D Hough Transform) to find the VP’s remaining degrees of freedom along the extension line of the hypothetical inlier. Our approach is able to sequentially find a series of VPs by repeatedly removing inliers of any detected VPs from minimal sample sets until the stop criterion is reached. Compared to traditional RANSAC that selects 2 edges as a hypothetical inlier pair to fit a model of VP hypothesis and requires hitting a pair of inliners, the UPRANSAC has a higher likelihood to hit one inliner and is more reliable in VP detection. Meanwhile, the tremendously scaled-down voting space with the requirement of only 1 parameter for processing significantly increased the performance efficiency of Hough Transform in our scheme. Testing results with well-known benchmark datasets show that the detection accuracies of our approach were higher or on par with the SOTA while running in deeply real-time zone.

Camera calibration is an important issue in computer vision. In this paper, we propose an improved camera auto-calibration algorithm from sequential images based on VP (vanishing point) and EKF (extended Kalman filter) to determine camera intrinsic parameters. This is the first vanishing point-based auto-calibration algorithm, which only uses a sequence of monocular images as input without any other information. According to geometry constraints of projective projection, we compute the vanishing points in three orthogonal directions by observing an object moving in one direction from an image sequence. Afterwards intrinsic parameters can be calculated. The extended Kalman filter is used to track the feature points in the image sequence rapidly and accurately. Compared with existing methods based on vanishing point, our approach simplifies calibration process, gets rid of calibration objects and manual intervention, avoids correspondences between 2D image and 3D world features and reduces errors to a large extent. Simulations and real image experiments validate the proposed approach and indicate that it is accurate and robust to noise. As a result, it could be applied to almost all real scenes like on-orbit camera calibration, autonomous vehicle navigation, space vehicle rendezvous and docking.

Regarding the autonomous of robot navigation, vanishing point (VP) plays an important role in visual robot applications such as iterative estimation of rotation angle for automatic control as well as scene understanding. Autonomous navigation systems must be able to recognize feature descriptors. Consequently, this navigating ability can help the system to identify roads, corridors, and stairs; ensuring autonomous navigation along the environments mentioned before the vanishing point detection is proposed. In this paper, the authors propose solutions for finding the vanishing point in real time based density-based spatial clustering of applications with noise (DBSCAN). First, we proposed to extract the longest segments of lines from the edge frame. Second, the set of intersection points for each pair of line segments are extracted by computing Lagrange coefficients. Finally, by using DBSCAN the VP is estimated. Preliminary results are performed and tested on a group of consecutive frames undertaken at Nam-gu, Ulsan, South Korea to prove its effectiveness.

We present a method for the automatic detection of vanishing points in urban scenes based on finding point alignments in a dual space, where converging lines in the image are mapped to aligned points. To compute this mapping the recently introduced PClines transformation is used. A robust point alignment detector is run to detect clusters of aligned points in the dual space. Finally, a post-processing step discriminates relevant from spurious vanishing point detections with two options: using a simple hypothesis of three orthogonal vanishing points (Manhattan-world) or the hypothesis that one vertical and multiple horizontal vanishing points exist. Qualitative and quantitative experimental results are shown. On two public standard datasets, the method achieves state-of-the-art performances. Finally, an optional procedure for accelerating the method is presented.

Projective geometry is an effective tool for shape transform of images in computer vision. It contains too many degrees of freedom, and the feature points must be chosen semi-automatically. In order to solve these problems, an improved algorithm of viewing transform with fixed viewpoint based on vanishing point is proposed. First, the relationship model of viewing angle and vanishing point is created. Then, the new method chooses the feature points automatically by detecting vanishing points. Finally, a shape transform based on panning and tilting is made. The test results of two types of images and comparison with other methods demonstrate the applicability and effectiveness of this algorithm.

Road segmentation plays an important role in many computer vision applications, either for in-vehicle perception or traffic surveillance. In camera-equipped vehicles, road detection methods are being developed for advanced driver assistance, lane departure, and aerial incident detection, just to cite a few. In traffic surveillance, segmenting road information brings special benefits: to automatically wrap regions of traffic analysis (consequently, speeding up flow analysis in videos), to help with the detection of driving violations (to improve contextual information in videos of traffic), and so forth. Methods and techniques can be used interchangeably for both types of application. Particularly, we are interested in segmenting road regions from the remaining of an image, aiming to support traffic flow analysis tasks. In our proposed method, road segmentation relies on a superpixel detection based on a novel edge density estimation method; in each superpixel, priors are extracted from features of gray-amount, texture homogeneity, traffic motion and horizon line. A feature vector with all those priors feeds a support vector machine classifier, which ultimately takes the superpixel-wise decision of being a road or not. A dataset of challenging scenes was gathered from traffic video surveillance cameras, in our city, to demonstrate the effectiveness of the method.

In this paper we propose a novel method for vanishing points detection based on convolutional neural network (CNN) approach and fast Hough transform algorithm. We show how to determine fast Hough transform neural network layer and how to use it in order to increase usability of the neural network approach to the vanishing point detection task. Our algorithm includes CNN with consequence of convolutional and fast Hough transform layers. We are building estimator for distribution of possible vanishing points in the image. This distribution can be used to find candidates of vanishing point. We provide experimental results from tests of suggested method using images collected from videos of road trips. Our approach shows stable result on test images with different projective distortions and noise. Described approach can be effectively implemented for mobile GPU and CPU.

For a mobile robot, it is necessary to have the ability to move accurately and autonomously in an known and/or unknown environment. The vanishing points existing in indoor environment are a key factor for the success of robot localization and navigation. To successfully detect these vanishing points, this paper proposes a novel approach of utilizing the distribution of orientation information from omnidirectional vision. The local orientation data are first computed by proper edge detection and then characterized by the local dominant orientation signature descriptors. These descriptors are further filtered based on the strength of its own orientation as well as its neighboring orientations. The final orientation field is constructed by using the interpolation of radial basis functions. The proposed approach of vanishing point detection has been extensively tested in various indoor environments such as narrow pathway library and multi-path corridor. Experimental results demonstrate good performance compared with existing approach.

In this paper we present a novel camera self-calibration technique to automatically recover intrinsic and extrinsic parameters of a static surveillance camera by observing a traffic scene. The scene must consist of one or more pedestrians and a zebra-crossing. We first extract a horizontal vanishing point and a vanishing line from a zebra-crossing. The observation of pedestrians allows calculating a so called vertical line of mass. All lines of mass are parallel in 3D space and therefore the vertical vanishing point can be estimated. The second horizontal vanishing point can be calculated by introducing the triangle spanned by three orthogonal vanishing points. All three vanishing points are then taken to gather the intrinsic parameters. The extrinsic parameters are calculated after the determination of the camera's height from the distance between two zebra-crossing edges. By combining static and dynamic calibration objects, the method gets robust against outliers. This robustness in combination with the practicability is shown in our experiments, which are carried out by using synthetic and real data of different application scenarios.