Fringe Projection Profilometry Based on Saturated Fringe Restoration in High Dynamic Range Scenes

Visual simultaneous localization and mapping (SLAM) algorithms in dynamic scenes can incorrectly add moving feature points to the camera pose calculation, which leads to low accuracy and poor robustness of pose estimation. In this paper, we propose a visual SLAM algorithm based on object detection and static probability update strategy for dynamic scenes, named YKP-SLAM. Firstly, we use the YOLOv5 target detection algorithm and the improved K-means clustering algorithm to segment the image into static regions, suspicious static regions, and dynamic regions. Secondly, the static probability of feature points in each region is initialized and used as weights to solve for the initial camera pose. Then, we use the motion constraints and epipolar constraints to update the static probability of the feature points to solve the final pose of the camera. Finally, it is tested on the TUM RGB-D dataset. The results show that the YKP-SLAM algorithm proposed in this paper can effectively improve the pose estimation accuracy. Compared with the ORBSLAM2 algorithm, the absolute pose estimation accuracy is improved by 56.07% and 96.45% in low dynamic scenes and high dynamic scenes, respectively, and the best results are almost obtained compared with other advanced dynamic SLAM algorithms.

Recording video in high dynamic range (HDR) scenes usually suffers from the trouble of getting poor contrast videos. In this paper, an integrated signal processing system, which remaps the image to enhance video visibility, is presented. The system extracts pixel local contrasts and nonlinearly stretches the dynamic range of pixel contrast values through histogram equalization. Undesired artifacts in the contrast map are then smoothed by a similar filter, which averages the pixel luminance values only when colors of neighboring pixels are similar to the central one. The luminance value for a pixel is then replaced by the geometric mean of its original value and smoothed contrast value. Finally, the photographic tone reproduction algorithm normalizes pixel values such that the average of log luminance values in the entire video frame is around 18% gray tone. The proposed approach shows better performance in both tone and color reproduction compared with other popular approaches.

In modern orthodontics treatment, mix dentition patients who have narrow maxillary dental arch width are treated by lateral expansion treatment of dental arch. Maxillary lateral expansion treatment can increase the width diameter of the dental arch and palatal width diameter. However, changes in the morphology of the palatal basal region due to expansion treatment have not been evaluated. First, we carried out an investigation of long-term growth changes in palatal morphology in the general Japanese population. Additionally, in the patients who were treated by expansion, we compared the effects of expansion with different types of expansion devices. Longitudinal records（34 study casts, 6 to 12 years of age）were classified according to growth stage（T1: early mixed dentition, T2: middle mixed dentition, and T3: late mixed dentition or permanent dentition）. A line drawn perpendicularly from the reference plane to the palatal basal region was divided into 10 sections, and the palatal width was measured in each region. Palatal widths were set at deep area（DA）, middle area（MA）, and shallow area（SA）. Comparison of the palatal width（three regions）between Rapid maxillary expansion （RME N=15）and Slow maxillary expansion（SME N=15）were evaluated same methods as above. DA and the Palatal depth were significantly smaller at T1 than T2 and at T2 than T3, respectively（both p <0.05）. The SA changed significantly less from T2 to T3 than did the DA（p <0.01）. Although, in DA, expansion amount was significant difference between RME and SME, there were not significant difference in SA. The growth amount of palatal width was different between the DA and the SA. RME was expanded greater in DA than SME.

Fusing several differently exposed images is particular useful for taking pictures in high dynamic range scenes. However, the scene changes resulted from moving objects and vibrations caused by photographers must be compensated adaptively in practical camera applications. In this paper, we propose a complete image fusion system aiming at extending the dynamic range of a picture by fusing two or more differently exposed images. The proposed global and local stabilization algorithms efficiently remedy the vibration problems and get a quite stable image fusion result

We describe a portable unattended persistent surveillance system that corrects for harsh illumination conditions, where bright sun light creates mixed contrast effects, i.e., heavy shadows and washouts. These effects result in high dynamic range scenes, where illuminance can vary from few luxes to a 6 figure value. When using regular monitors and cameras, such wide span of illuminations can only be visualized if the actual range of values is compressed, leading to the creation of saturated and/or dark noisy areas and a loss of information in these areas. Images containing extreme mixed contrast cannot be fully enhanced from a single exposure, simply because all information is not present in the original data. The active intervention in the acquisition process is required. A software package, capable of integrating multiple types of COTS and custom cameras, ranging from Unmanned Aerial Systems (UAS) data links to digital single-lens reflex cameras (DSLR), is described. Hardware and software are integrated via a novel smart data acquisition algorithm, which communicates to the camera the parameters that would maximize information content in the final processed scene. A fusion mechanism is then applied to the smartly acquired data, resulting in an enhanced scene where information in both dark and bright areas is revealed. Multi-threading and parallel processing are exploited to produce automatic real time full motion corrected video. A novel enhancement algorithm was also devised to process data from legacy and non-controllable cameras. The software accepts and processes pre-recorded sequences and stills, enhances visible, night vision, and Infrared data, and successfully applies to night time and dark scenes. Various user options are available, integrating custom functionalities of the application into intuitive and easy to use graphical interfaces. The ensuing increase in visibility in surveillance video and intelligence imagery will expand the performance and timely decision making of the human analyst, as well as that of unmanned systems performing automatic data exploitation, such as target detection and identification.

3D reconstruction of high-dynamic-range (HDR) scenes using a minimal number of images is a critical challenge for fringe projection profilometry. Deep learning (DL) has demonstrated significant potential, but previous approaches have never considered a balance between accuracy and speed, resulting in a slow inference latency for networks. A lightweight U-shaped transformer-CNN architecture is proposed that integrates dual attention mechanisms (channel and pixel attention) which enable the network to focus more attention on HDR regions. In addition, an efficient transformer (ET) layer establishes a longer dependence that enables compensation for the HDR regions by utilizing their surrounding pixels’ phase information. To address the data scarcity challenge, the network is first pre-trained on a synthetic dataset and subsequently fine-tuned using a real-world dataset that contains 720 HDR scenes. Experiments have proven that the proposed network achieves a phase error as low as 5.32 × 10 − 4 rad with it only containing 0.55 million parameters and a real-time processing speed of 16.56 frames per second (FPS) on an RTX 4060 GPU. Compared with other networks, the proposed network provides an effective phase recovery performance with faster inference speed for HDR scene reconstruction, which is crucial for actual industry applications.

Purpose:The aim of this research was to calculate the fluence, dose equivalent (DE), and kerma of thermal, epithermal and fast photoneutrons separately, within ICRU soft-tissue-equivalent phantom in the radiotherapy treatment room, using MCNPX Monte Carlo code.Materials and Methods:For this purpose, 18 MV Varian Linac 2100 C/D machine was simulated and desired quantities were calculated on the central axis and transverse directions at different depths.Results:Maximum fluence, DE and kerma of total photoneutrons on central axis of the phantom were 43.8 n.cm-2.Gy-1, 0.26, and 3.62 mGy.Gy-1, at depths 2, 0.1, 0.1 cm, respectively. At any depth, average of fluence, DE and kerma in the outer area of the field were less than the inner area and in general were about 72%, 52%, and 45%, respectively.Conclusion:According to this research, within the phantom; variation of fluence, DE and kerma in transverse direction were mild, and along the central axis at shallow area were sharp. DE of fast photoneutrons at shallow and deep areas were one order of magnitude greater than thermal photoneutrons.

Single-shot fringe projection profilometry (FPP) is essential for retrieving the absolute depth information of the objects in high-speed dynamic scenes. High-precision 3D reconstruction using only one single pattern has become the ultimate goal in FPP. The frequency-multiplexing (FM) method is a promising strategy for realizing single-shot absolute 3D measurement by compounding multi-frequency fringe information for phase unwrapping. In order to solve the problem of serious spectrum aliasing caused by multiplexing schemes that cannot be removed by traditional spectrum analysis algorithms, we apply deep learning to frequency multiplexing composite fringe projection and propose a composite fringe projection deep learning profilometry (CDLP). By combining physical model and data-driven approaches, we demonstrate that the model generated by training an improved deep convolutional neural network can directly perform high-precision and unambiguous phase retrieval on a single-shot spatial frequency multiplexing composite fringe image. Experiments on both static and dynamic scenes demonstrate that our method can retrieve robust and unambiguous phases information while avoiding spectrum aliasing and reconstruct high-quality absolute 3D surfaces of objects only by projecting a single composite fringe image.

SUMMARYCrayfish in headwater streams are subjected to predation from two sources: (a) fish; and (b) terrestrial predators including wading birds and mammals. Field observations of the mortality of tethered crayfish of two size classes were used to examined how predation risks imposed by the two types of predators varied with water depth and crayfish size. We also examined the depth distribution of large and small crayfish in stream pools with and without predatory fish.Predatory fish, mostly green sunfish,Lepomis cyctuellus, and creek chub,Semotilus atromaculatus, rapidly consumed tethered small crayfish (body length = 25–30 mm) in deep areas (0.4–0.7 m), but not in shallow areas (0.05–0.15 m) of stream pools. Tethered large crayfish (body length = 50–75 mm) suffered little mortality from fish at either depth. Terrestrial predators such as racoons,Procyou lotor, and herons,ButoridesandArdea, caused little crayfish mortality in deep areas, but rapidly consumed both small and large crayfish exposed in shallow areas.A comparison of habitat use by crayfish in pools with and without fish suggests that small, but not large, crayfish shift their distributions to shallow water in the presence of fish. In pools without fish, the distribution of small crayfish was independent of depth, whereas large crayfish occupied deep water.

Event-based cameras are bioinspired vision sensors whose pixels work independently from each other and respond asynchronously to brightness changes, with microsecond resolution. Their advantages make it possible to tackle challenging scenarios in robotics, such as high-speed and high dynamic range scenes. We present a solution to the problem of visual odometry from the data acquired by a stereo event-based camera rig. Our system follows a parallel tracking-and-mapping approach, where novel solutions to each subproblem (three-dimensional (3-D) reconstruction and camera pose estimation) are developed with two objectives in mind: being principled and efficient, for real-time operation with commodity hardware. To this end, we seek to maximize the spatio-temporal consistency of stereo event-based data while using a simple and efficient representation. Specifically, the mapping module builds a semidense 3-D map of the scene by fusing depth estimates from multiple viewpoints (obtained by spatio-temporal consistency) in a probabilistic fashion. The tracking module recovers the pose of the stereo rig by solving a registration problem that naturally arises due to the chosen map and event data representation. Experiments on publicly available datasets and on our own recordings demonstrate the versatility of the proposed method in natural scenes with general 6-DoF motion. The system successfully leverages the advantages of event-based cameras to perform visual odometry in challenging illumination conditions, such as low-light and high dynamic range, while running in real-time on a standard CPU. We release the software and dataset under an open source license to foster research in the emerging topic of event-based simultaneous localization and mapping.

The dependence of an object's colour on the illuminant chromaticity makes it difficult to use colour as a reliable cue in machine vision applications, particularly in naturally illuminated high dynamic range scenes. To solve this problem the outputs from four logarithmic sensors with different spectral responses can be used to obtain a two dimensional description of an object's chromaticity that is independent of the illuminant. The spectral responses of these four sensors have been optimised. A simple test of colour separability then suggests that using the data from these sensors it is possible to match the ability of the human visual system to separate similar colours. A comparison of the performance of the proposed system when the reflectance and illuminantion data are both changed suggests that readily available data (Munsell reflectance spectra and CIE standard daylight spectra) can be used to design a generic system to separate colours that are described as matching each other. However, for applications that require discrimination between very well matched colours it may be necessary to use an application specific system designed using data relevant to the application.

Event cameras are bio-inspired vision sensors that output pixel-level brightness changes instead of standard intensity frames. They offer significant advantages over standard cameras, namely a very high dynamic range, no motion blur, and a latency in the order of microseconds. We propose a novel, accurate tightly-coupled visual-inertial odom- etry pipeline for such cameras that leverages their outstanding properties to estimate the camera ego-motion in challenging conditions, such as high-speed motion or high dynamic range scenes. The method tracks a set of features (extracted on the image plane) through time. To achieve that, we consider events in overlapping spatio-temporal windows and align them using the current camera motion and scene structure, yielding motion-compensated event frames. We then combine these feature tracks in a keyframe- based, visual-inertial odometry algorithm based on nonlinear optimization to estimate the camera’s 6-DOF pose, velocity, and IMU biases. The proposed method is evaluated quantitatively on the public Event Camera Dataset [19] and significantly outperforms the state-of-the-art [28], while being computationally much more efficient: our pipeline can run much faster than real-time on a laptop and even on a smartphone processor. Fur- thermore, we demonstrate qualitatively the accuracy and robustness of our pipeline on a large-scale dataset, and an extremely high-speed dataset recorded by spinning an event camera on a leash at 850 deg/s.

Recently we proposed a wavelet-based dynamic range compression algorithm to improve the visual quality of digital images captured in the high dynamic range scenes with non-uniform lighting conditions. The fast image enhancement algorithm which provides dynamic range compression preserving the local contrast and tonal rendition is a very good candidate in aerial imagery applications such as image interpretation for defense and security tasks. This algorithm can further be applied to video streaming for aviation safety. In this paper the latest version of the proposed algorithm which is able to enhance aerial images so that the enhanced images are better then direct human observation, is presented. The results obtained by applying the algorithm to numerous aerial images show strong robustness and high image quality.

When we take a photograph through glass windows or doors, the foreground scene is reflected in the captured image. The reflected components overlap with the background scene and make object recognition and identification more difficult. This paper proposes a novel reflection removal method using multiple polarized images taken with different exposure times. To achieve a high accuracy reflection removal in high dynamic range scenes, in which photographed images have under-/over-exposed pixels, we introduce a minimization problem of weighted nonnegative matrix factorization (WNMF) with total variation regularization. To solve this minimization problem, we also introduce an alternating optimization scheme with the alternating direction method of multipliers (AO-ADMM). The advantages of the proposed method over some conventional methods are demonstrated in experiments of reflection removal using real-world images.

Real-time motion-induced error reduction for phase-shifting profilometry with projection points tracking method