Lutetia surface reconstruction and uncertainty analysis

In this paper, a linear calibration method is proposed for a paracatadioptric camera using the images of two spheres. Two spheres are selected in space, and the two groups of their projection circles on the unit viewing sphere are made to intersect at four points. The quadrilateral consisting of four points is a rectangle, so a group of orthogonal directions can be determined in space to obtain a group of orthogonal vanishing points in the paracatadioptric image plane. Because of the relationship between orthogonal vanishing points and intrinsic camera parameters, the intrinsic parameters of a paracatadioptric camera can be linearly solved by at least five views satisfying the above conditions. First, one estimates the sphere images and their antipodal sphere images. Second, by solving the intersection of the images of two spheres and the intersection of the images of their antipodal spheres, a group of orthogonal vanishing points can be obtained in the image plane. Finally, by taking the relationship between the orthogonal vanishing points and the intrinsic camera parameters as constraint conditions, the intrinsic parameters of the paracatadioptric camera can be obtained. Simulation results and real image data demonstrate the effectiveness of our new algorithms.

In this paper, we propose a method of extracting feature descriptors from discrete spherical images using convolutional neural networks (CNNs). First, a captured full-view image is mapped to a discrete spherical image. Second, the features-from-accelerated-segment test algorithm is used to extract feature points in the discrete spherical image. Finally, an unsupervised CNN is used to obtain the descriptors of patches around each feature point. In the experiments, we compare these descriptors' performance to the closest existing state-of-the-art feature descriptors of discrete spherical images, spherical oriented FAST and rotated BRIEF (SPHORB), for image pairs having different camera rotation, noise levels, and general motions. The experimental results demonstrate that our proposed CNN-based discrete spherical image feature descriptors clearly outperform SPHORB both in accuracy and robustness.

Palmprint recognition has recently attracted broad attention due to its rich discriminative features, contactless collection manner and less invasive. However, most existing methods focus on within-illumination palmprint recognition, which requires the similar illumination of query samples acquisition as the gallery samples, significantly limiting its practical applications in the open environment. In this paper, we propose a cross-illumination palmprint recognition method by jointly learning the unified binary feature descriptors of multiple illumination palmprint images. Given two different illuminations of palmprint images, we first calculate the direction-based ordinal measure vectors (DOMVs) to sample the important palmprint direction features. Then, we jointly learn a unified feature mapping that project the two-illumination DOMVs into binary feature codes. To better exploit the palm-invariant features of multi-illumination samples, we make the binary feature codes as similar as possible by minimizing the feature distance between the two illumination samples of the same palm. Moreover, we maximize the variances of all binary feature codes among the training samples for each illumination, such that the discriminative power can be enhanced in an unsupervised manner. Finally, we convert the binary feature codes of a palmprint image into a block-wise histogram feature descriptor for cross-illumination palmprint recognition. Experimental results on three cross-illumination palmprint datasets show that our proposed method achieves competitive cross-illumination palmprint recognition performance in comparison with the state-of-the-art palmprint feature descriptors.

Context-aware obstacle detection for navigation by visually impaired

In this paper, a three stage hierarchical image retrieval scheme using a color, texture and shape visual contents (or descriptors) is proposed, since single visual content is not produce an adequate retrieval results effectively. This scheme has reduced the searching space during the image retrieval process at a certain extent due to the hierarchical mode. In initial stage, the shape feature descriptor has been computed by simple fusion of histograms of gradients and invariant moments of segmented image planes. The shape based retrieval process has reduced the search space by discarding the non-relevant images from the universal dataset (or original dataset) effectively and kept the retrieved images into the intermediate dataset. In the second stage, the texture feature descriptors have been computed from the intermediate sub-image dataset by applying the adaptive tetrolet transform on image plane of preprocessed HSV color image. This transform provides the multi-resolution images with finer details by employing the tetrominoes and the proper arrangement of tetrominoes contributes the effective local geometry of image plane. The gray level co-occurrence matrix based texture feature descriptor is obtained by computing second order statistical parameters from each decomposed sub-image. At this stage, the most of the irrelevant images are discarded by retrieving the images from intermediate dataset but still some undesired images are left, those will be handled at the last stage. At this stage, fused color information is captured by applying the color autocorrelogram on both the non-uniform quantized color components of the preprocessed HSV color image. Finally, the color feature descriptor produces the desired retrieval results by discarding the irrelevant images from the texture based sub-image dataset. The proposed scheme has also low computational overhead due to the use of three descriptors at different stages separately. The retrieved results show the better accuracy as compared to the other related visual contents based image retrieval schemes.

Calibration of the transformation between two planes, the image plane and the 2D platform, is important for applications where 2D metric information on the image plane is used to compute the 3D information in a world coordinate system. Efforts have been taken in computer vision literature to perform a full-scale camera calibration, including intrinsic and extrinsic parameters. For some applications, calibration of intrinsic parameters is not required as often as that of the extrinsic parameters. This paper addresses the problem of calibrating a camera's extrinsic parameters with respect to a 2D platform to which a camera is observing. A rational approximation method is proposed. Simulation and experimental results show that the proposed rational approximation method achieves comparable accuracy with the well-known homography-based approach. This work is motivated by our mobile sensor network project. Other relevant applications include vision-based metrology.

In this paper, a novel visual servoing technique for a 5-dof mobile manipulator with an eye-to-hand camera configuration is introduced. The proposed technique can be categorized as an image based (or 2D) visual servoing using a fixed camera in conjunction with a conic mirror (aka, a Catadioptric camera system) providing panoramic vision. Two fictitious landmarks mounted on robot's end-effector along with their mirror reflections, when viewed by the camera, provide enough information for 3D reasoning based on the four points viewed on the image plane. Instead of directly using the image features associated with these four points, five new image features are chosen to make the image Jacobian rank efficient. A dual estimation/control strategy based on Extended Kalman Filter (EKF) is utilized to (1) estimate camera's intrinsic and extrinsic parameters, and (2) track the coordinates of the landmarks and their reflections on the image plane. The relationship between the translational and rotational velocity of a frame attached to the robot's end-effector and the rate of change of the proposed image features are fully formulated. The robustness of the proposed technique in translational and rotational servoing of a 5-dof holonomic mobile manipulator is illustrated through computer simulations.

Spacecraft pose estimation is an essential input to the Guidance, Navigation and Control process. Pose estimation based on monocular camera images does not require modifications to the target vehicle and has fewer resource requirements than LIDAR systems. We provide a comprehensive investigation and novel contributions in foreground recognition and extraction, image feature generation, and pose estimation. We evaluated $12$ image feature detector and descriptor performances and proposed a new biologically inspired image descriptor. We also assessed the bag of visual words codebook technique for object localisation and evaluated linear, non-linear and non-parametric classifiers. We tested the ResNet and Inception-ResNet convolutional neural networks on target localisation. We developed compact semantic segmentation autoencoders based on AlexNet, U-Net and VGG. We made several new contributions in the image saliency generation. First, we developed a novel principal component analysis based formula for graph manifold ranking Optimal Affinity Matrix inversion which reduces computation time and stabilises the ranking inversion process. We developed a novel weighted gradient orientation histogram feature for monochromatic image superpixel identification and provided three enhanced versions of the graph manifold ranking tested on $32,536$ images. Our technique out-performs the state-of-the-art saliency method in precision and our fastest method is $12{\times}$ faster than the original graph manifold ranking technique. We introduce an innovative false-coloured high-frequency salient feature image to enhance foreground and background pixel histogram distinction. We propose a novel space background classification scheme using pixel statistics to detect Earth passage. We evaluated appearance based pose matching using principal components analysis, SoftPOSIT and $e$P$n$P for pose estimation. We propose a novel homography transform projection method that simplifies the perspective-$n$-point correspondence. We introduce improvements to the SoftPOSIT initiation to reduce the effects of local minimum trapping using centroid matching. We developed region-based pose estimation using level-set segmentation and pixel statistics. Our tests show the region-based method out-performs the appearance-based and point-based methods in speed, precision and stability.

In this paper, we present an approach to the driver head motion tracking problem using Bayesian learning and inter-frame optical flow clustering. Two cameras (one operating in visual band and the other in the infra-red range) are mounted on vehicle dashboard to determine the driver head motion by examining the temporal variation of video-histogram differences (VHD). Progresively increasing VHD indicates that the driver head moves from its steady position. An optical flow map is generated as a measure that would lead the tracker in the direction of head motion in synthetically generated volumetric view geometry involving a sequence of motion frames. Experiments show that the proposed system can track the driver head and identify the motion as left or right in two consecutive video frames parallel to the image plane of the camera or forward and backward direction from one frame to next. This simplifies the problem of tackling head movement with arbitrary motion in space via considering two consecutive frames at a time and marking the motion from the current frame to the next in sequence. Since the head position is marked in each frame with its optical flow track, the arbitrary motion is viewed as a classification problem of belonging to left-right, right-left, backward-forward, and forward-backward in the inter-frame duration via the Bayesian learning process of the Gaussian density.

A new image feature descriptor for content based image retrieval using scale invariant feature transform and local derivative pattern

Cascade of descriptors to detect and track objects across any network of cameras

The paper presents an evaluation of a range of contemporary image feature detectors and descriptors for the mobile robot visual simultaneous localization and mapping. The analysis of the impact of the detector and descriptor choice on the accuracy of robot trajectory reconstruction was performed using precise ground truth data. Moreover, as processing time is an important, the average computation times for each of the detector-descriptor pairs are also included.

Image feature detector and descriptor algorithms have made a big advance in almost every area of computer vision applications including object localisation, object tracking, mobile robot mapping, watermarking, panorama stitching and 3D surface reconstruction by assisting the detection and description of feature points in a set of given images. In this paper, we evaluate the performance of four robust feature detection algorithms namely SIFT, SURF, BRIEF and ORB on multi-view 3D surface reconstruction of microscopic samples obtained by a scanning electron microscope (SEM), a widely used equipment in biological and materials sciences for determining the surface attributes of micro objects. To this end, we first develop an optimised multi-view framework for SEM extrinsic calibration and its 3D surface reconstruction. We design a Differential Evolutionary-based algorithm to solve the problem in a global optimisation platform. Several qualitative and quantitative comparisons such as reliability on SEM extrinsic calibration and validity on 3D visualisation performed on real microscopic objects as well as a synthetic model. The present evaluation is expected to provide better insights and consideration to determine which algorithm is well deserved for multi-view 3D SEM surface reconstruction.

In this paper a theoretical investigation of the Encircled Energy of an annular pupil (with different obscuration ratio ε = 0.25, 0.5, 0.75) for diffraction limited and defocus (optical system suffers from focal shift) optical system. The numerical evaluation was carried out using pupil function technique in Cartesian coordinates which has the advantage of introducing any kind and amount of aberration in the optical system. Another quality criterion for measuring the efficiency of optical systems is the Gradient determination. The maximum gradient value is near the center of the Encircled Energy curve in the image plane and will be decreases when the wave front aberration increases in the image plane. The presented results shows beater information when the optical system has a large amount of focal shift, and shows a clear reduction in the Encircled Energy values with increasing obscuration ratio This indicates that the central spot in the central ring group contains only a fraction of the total energy, so the whole set of central rings considered as the Effective Image Element.

Proper membrane localization of ion channels is essential for the function of neuronal cells. Particularly, the computational ability of dendrites depends on the localization of different ion channels in specific subcompartments. However, the molecular mechanisms that control ion channel localization in distinct dendritic subcompartments are largely unknown. Here, we developed a quantitative live cell imaging method to analyze protein sorting and post-Golgi vesicular trafficking. We focused on two dendritic voltage-gated potassium channels that exhibit distinct localizations: Kv2.1 in proximal dendrites and Kv4.2 in distal dendrites. Our results show that Kv2.1 and Kv4.2 channels are sorted into two distinct populations of vesicles at the Golgi apparatus. The targeting of Kv2.1 and Kv4.2 vesicles occurred by distinct mechanisms as evidenced by their requirement for specific peptide motifs, cytoskeletal elements, and motor proteins. By live cell and super-resolution imaging, we identified a novel trafficking machinery important for the localization of Kv2.1 channels. Particularly, we identified non-muscle myosin II as an important factor in Kv2.1 trafficking. These findings reveal that the sorting of ion channels at the Golgi apparatus and their subsequent trafficking by unique molecular mechanisms are crucial for their specific localizations within dendrites.