INTELLIGENT 3D CRACK RECONSTRUCTION USING CLOSE RANGE PHOTOGRAMMETRY IMAGERY

Automated fiber placement (AFP) is an advanced manufacturing technology for large‐scale carbon fiber reinforced plastics (CFRP) composite parts. During AFP processes, unidirectional CFRP prepreg tapes are laid up on the mold to become laminates by a robotic arm under heating and pressure conditions. However, the efficient monitoring of lay‐up defects during AFP, especially the out‐of‐plane defects including wrinkles, bridging, gaps, and overlaps, needs more advanced solutions, as manual inspection of the lay‐up defects still accounts for a large proportion of real manufacturing processes at present. To tackle this issue, an innovative method to realize efficient and accurate detection of AFP lay‐up defects was developed in this work. First, a bespoke AFP platform containing a 5‐degree‐of‐freedom moving platform, air cylinder, compaction roller, heating gun, and structural‐light camera was constructed for AFP experiments. A defect detection algorithm based on PointNet++, a convolutional neural network (CNN) for 3D point cloud segmentation, was then created. PointNet++ was trained by the dataset containing 3D point clouds of intentionally laid‐up defects, which were created on the AFP platform and captured by the structural‐light camera. Compared to the existing AFP defect inspection methods, such as laser profilometer and thermography, this method has the advantages of low cost and convenience to operate. The completed method was experimentally validated to be capable of precisely tracking the locations of out‐of‐plane AFP defects in less than 1 second. The detection accuracy of the method can achieve above 72% in Intersection over Union (IoU) for the out‐of‐plane defects. Highlights A lay‐up defect detection method for AFP by structural light scanning. 3D point cloud segmentation by PointNet++ was used for defect segmentation. A bespoke AFP platform was constructed for AFP experiments. The method can track the locations of AFP defects in less than 1 second. The method has the advantages of low cost and convenience to operate.

Industrial computed tomography (CT) crack segmentation is a key technology in industrial CT image processing. Unfortunately, the interference of artifact and noise in CT image often bring great trouble to the crack segmentation. In order to improve the segmentation accuracy of cracks in CT images, we propose to develop and test a new crack segmentation algorithm based on linear feature enhancement by analyzing the features of cracks in CT images. Firstly, the total variational model is used to denoise the input image. Next, a Frangi multiscale filter is used to extract linear structures in the image, and then the extracted linear structures are used to enhance the contrast of the image. Finally, the cracks in the image are detected and segmented by Otsu algorithm. By comparing with the manual segmentation results, the average intersection-over-union (IOU) reaches 86.10% and the average F1 score reaches 92.44%, which verifies the effectiveness and correctness of the algorithm developed in this study. Overall, experiments demonstrate that the new algorithm improves the accuracy of crack segmentation and it is effective applying to industry CT images.

Unmanned Aircraft Systems (UASs) can collect high resolution and high quality images for local mapping. Before the UAS images can be used for accurate mapping tasks in local areas, the precise position and orientation of the UAS images should first be determined. Direct georeferencing by POS (Position and Orientation System), a combination of GPS (Global Positioning System) and IMU (Inertial Measurement Unit), is the best choice; however, most commercial UASs cannot carry highly accurate IMUs because of the limited payload. Therefore, this study will discuss the accuracy of indirect georeferencing for UAS images. One approach for indirect georeferencing is general aerial triangulation (AT) by using well-distributed ground control points (GCPs). The other one is GPS-supported AT with GPS observations as airborne controls. In this paper, the camera is calibrated by the field method, and the accuracy of these two approaches for indirect georeferencing is presented. Based on 20 horizontal check points and 29 vertical check points, this study shows the stereoscopic viewing accuracy of general AT for UAS images, collected by Canon EOS 5D Mark II camera with 24 mm F/1.4L II USM lens at a flying height of 550 m, is about 0.26 m (ca. 1.73 pixels) in planimetry and 0.27 m (ca. 1.80 pixels) in height. GPS-supported AT produced the stereoscopic viewing accuracy about 0.44 m (ca. 2.93 pixels) in planimetry and 0.55 m (ca. 3.67 pixels) in height. The test results show that the accuracy of these two indirect georeferencing approaches of fixed-wing UAS images can be used for updating local 1/5,000 topographic maps in Taiwan.

In the present study, an integrated framework for automatic detection, segmentation, and measurement of road surface cracks is proposed. First, road images are captured, and crack regions are detected based on the fifth version of the You Only Look Once (YOLOv5) algorithm; then, a modified Residual Unity Networking (Res-UNet) algorithm is proposed for accurate segmentation at the pixel level within the crack regions; finally, a novel crack surface feature quantification algorithm is developed to determine the pixels of crack in width and length, respectively. In addition, a road crack dataset containing complex environmental noise is produced. Different shooting distances, angles, and lighting conditions are considered. Validated through the same dataset and compared with You Only Look at CoefficienTs ++ (YOLACT++) and DeepLabv3+, the proposed method shows higher accuracy for crack segmentation under complex backgrounds. Specifically, the crack damage detection based on the YOLOv5 method achieves a mean average precision of 91%; the modified Res-UNet achieves 87% intersection over union (IoU) when segmenting crack pixels, 6.7% higher than the original Res-UNet; and the developed crack surface feature algorithm has an accuracy of 95% in identifying the crack length and a root mean square error of 2.1 pixels in identifying the crack width, with the accuracy being 3% higher in length measurement than that of the traditional method.

The study of historical maps has gained significant importance in recent years due to their ability to shed light on past geographical and topographic landscapes since they serve as crucial sources for understanding past conditions. They not only provide insights into the geography and topography during the time they were created but also allow for studying long-term changes over time. In this study, an oversized historical map of the Bailiwick of Neuland from the years 1780 to 1790 was discovered in the archives of the municipal building office in Winsen an der Luhe, Germany. The map, measuring approximately 5 m × 2 m, was digitised by photogrammetric methods and subsequently georeferenced. The process involved photographing the map with two different cameras and two UAV systems at the Geodetic Laboratory of HafenCity University Hamburg. This allowed to generate a high-resolution orthophoto from each data set. The resulting orthophoto achieved a pixel size of 0.2 mm, ensuring a detailed representation of the map. To ensure accuracy, the best photo block was scaled in a bundle block adjustment using ground control points with an accuracy of 1 mm and scale bars with an accuracy of 0.1 mm. Georeferencing of the historical map was conducted using current digital orthophotos of Lower Saxony with a resolution of 20 cm. A third-degree polynomial transformation was applied during georeferencing, resulting in mean residuals of 2.5 m at the ground control points. This process ensured that the historical map was accurately aligned with the current digital orthophotos, allowing for precise spatial referencing.

Rock bolts are crucial components in the subterranean support systems in underground mines that provide adequate structural reinforcement to the rock mass to prevent unforeseen hazards like rockfalls. This makes frequent assessments of such bolts critical for maintaining rock mass stability and minimising risks in underground mining operations. Where manual surveying of rock bolts is challenging due to the low-light conditions in the underground mines and the time-intensive nature of the process, automated detection of rock bolts serves as a plausible solution. To that end, this study focuses on the automatic identification of rock bolts within medium- to large-scale 3D point clouds obtained from underground mines using mobile laser scanners. Existing techniques for automated rock bolt identification primarily rely on feature engineering and traditional machine learning approaches. However, such techniques lack robustness as these point clouds present several challenges due to data noise, varying environments, and complex surrounding structures. Moreover, the target rock bolts are extremely small objects within large-scale point clouds and are often partially obscured due to the application of reinforcement shotcrete. Addressing these challenges, this paper proposes an approach termed DeepBolt, which employs a novel two-stage deep learning architecture specifically designed for handling severe class imbalance for the automatic and efficient identification of rock bolts in complex 3D point clouds. The proposed method surpasses state-of-the-art semantic segmentation models by up to 42.5% in Intersection over Union (IoU) for rock bolt points. Additionally, it outperforms existing rock bolt identification techniques, achieving a 96.41% precision and 96.96% recall in classifying rock bolts, demonstrating its robustness and effectiveness in complex underground environments.

With the appearance of low-cost and high-resolution consumer-grade digital cameras, a practical three-dimensional (3D) measurement system using a consumer-grade digital camera is greatly anticipated in various fields. In these circumstances, the authors have been concentrating on developing a practical 3D measurement system that includes photogrammetric software called the Image Based Integrated Measurement (IBIM) system. The IBIM system device consists of full/half-mirrors, a consumer-grade digital camera, and a laser distance meter. The most remarkable advantage of the system is its ability to calculate exterior orientation parameters, interior orientation parameters, and pseudo ground control points (GCPs) without using scale bars or the GCPs in the object field. The system has the ability to calibrate multiple cameras of different resolutions using a camera-variant parameter set. However, there remains one issue that needs to be resolved before this system can be effectively used, namely, improvement of the system which does not depend on the IBIM system device. With this motive, a practical photogrammetry method using a consumer-grade digital cameras and a hand-held laser distance meter is proposed. To test our proposed method, the bundle of distances from the center camera position to the feature points in the object field were measured individually at archaeological sites in Greece. In order to evaluate the possibility and practicability of the proposed photogrammetry method, this paper describes and evaluates the camera calibration techniques using images from multiple cameras of different resolutions and a bundle of distances.

In this paper we present the further evaluation of DLR’s modular airborne camera system MACS-Micro for small unmanned aerial vehicle (UAV). The main focus is on standardized calibration procedures and on photogrammetric workflows. The current prototype consists of an industrial grade frame imaging camera with 12 megapixel resolutions and a compact GNSS/IMU solution which are operated by an embedded computing unit (CPU). The camera was calibrated once pre-flight and several times post-flight over a period of 5 month using a three dimensional test field. The verification of the radiometric quality of the acquired images has been done under controlled static conditions and kinematic conditions testing different demosaicing methods. The validation of MACS-Micro is done by comparing a traditional photogrammetric evaluation with the workflows of Agisoft Photoscan and Pix4D Mapper. The analyses are based on an aerial survey of an urban environment using precise ground control points and acquired GNSS observations. Aerial triangulations with different configuratrions of ground control points (GCP’s) had been calculated, comparing the results of using a camera self-calibration and introducing fixed interior orientation parameters for Agisoft and Pix4D. The results are promising concerning the metric characteristics of the used camera and achieved accuracies in this test case. Further aspects have to be evaluated by further expanded test scenarios.

Currently UAV Photogrammetry is characterized a largely automated and efficient data processing. Depicting from the low altitude more often gains on the meaning in the uses of applications as: cities mapping, corridor mapping, road and pipeline inspections or mapping of large areas e.g. forests. Additionally, high-resolution video image (HD and bigger) is more often use for depicting from the low altitude from one side it lets deliver a lot of details and characteristics of ground surfaces features, and from the other side is presenting new challenges in the data processing. Therefore, determination of elements of external orientation plays a substantial role the detail of Digital Terrain Models and artefact-free ortophoto generation. Parallel a research on the quality of acquired images from UAV and above the quality of products e.g. orthophotos are conducted. Despite so fast development UAV photogrammetry still exists the necessity of accomplishment Automatic Aerial Triangulation (AAT) on the basis of the observations GPS/INS and via ground control points. During low altitude photogrammetric flight, the approximate elements of external orientation registered by UAV are burdened with the influence of some shift/bias errors. In this article, methods of determination shift/bias error are presented. In the process of the digital aerial triangulation two solutions are applied. In the first method shift/bias error was determined together with the drift/bias error, elements of external orientation and coordinates of ground control points. In the second method shift/bias error was determined together with the elements of external orientation, coordinates of ground control points and drift/bias error equals 0. When two methods were compared the difference for shift/bias error is more than ±0.01 m for all terrain coordinates XYZ.

Photographic field control measurement is divided into the measurement of the basic control points of the regional network and the simultaneous measurement of the image control points and checkpoints. The main purpose is to connect the aerial camera coordinates with the ground control points, and then measure the plane according to the mathematical model. The accuracy of the point and elevation points, which in turn reflects the accuracy of the DOM digital product. Image control data directly reflects the accuracy of digital products. Therefore, it is necessary to perform joint control of image points. This paper mainly studies the key technologies of field control point layout and joint measurement, aerial triangulation and digital product generation.

Damage due to age or accumulated damage from hazards on existing structures poses a worldwide problem. In order to evaluate the current status of aging, deteriorating and damaged structures, it is vital to accurately assess the present conditions. It is possible to capture the in situ condition of structures by using laser scanners that create dense 3D point clouds. However, extracting meaningful information from point clouds automatically is challenging, and the current state-of-the-art requires a certain amount of user-interaction. Previously in our research, widely accepted point cloud processing steps were used to divide laser scanner data into meaningful object clusters, on which several damage detection methods were applied. Our current focus is on combining useful information extracted from laser scanner data with color information since novel laser scanners are capable of recording color data mapped on 3D point clouds. The color data provides additional information in the 4 th dimension, which makes it possible to detect damage types such as cracks, corrosion, and related surface defects that are generally difficult to detect by using only laser scanner data, and also helps to track volumetric changes on structures such as spalling. Even though crack and corrosion detection from images with varying resolution are extensively researched topics, damage detection by using laser scanners with and without images is a new research area that holds many opportunities for enhancing current practice in structural health monitoring. The aim of our research is to benefit from the best features of both laser scans and images, and combine them to create an automatic and effective damage detection method, which will complement the need for skilled labor during visual inspections of damaged structures and allow documenting related information automatically. This work is of interest to researchers who are focused on structural health monitoring and interested in the use of new laser scanners and damage detection algorithms for structures, as well as practitioners, who will benefit exploring new methods for improving their understanding of laser scanning technology and its capabilities for robust damage detection and documentation.

A hybrid attention deep learning network for refined segmentation of cracks from shield tunnel lining images

Uncrewed aircraft systems (UASs) and structure-from-motion/multi-view stereo (SfM/MVS) photogrammetry are efficient methods for mapping terrain at local geographic scales. Traditionally, indirect georeferencing using ground control points (GCPs) is used to georeference the UAS image locations before further processing in SfM software. However, this is a tedious practice and unsuitable for surveying remote or inaccessible areas. Direct georeferencing is a plausible alternative that requires no GCPs. It relies on global navigation satellite system (GNSS) technology to georeference the UAS image locations. This research combined field experiments and simulation to investigate GNSS-based post-processed kinematic (PPK) as a means to eliminate or reduce reliance on GCPs for shoreline mapping and charting. The study also conducted a brief comparison of real-time network (RTN) and precise point positioning (PPP) performances for the same purpose. Ancillary experiments evaluated the effects of PPK base station distance and GNSS sample rate on the accuracy of derived 3D point clouds and digital elevation models (DEMs). Vertical root mean square errors (RMSEz), scaled to the 95% confidence interval using an assumption of normally-distributed errors, were desired to be within 0.5 m to satisfy National Oceanic and Atmospheric Administration (NOAA) requirements for nautical charting. Simulations used a Monte Carlo approach and empirical tests to examine the influence of GNSS performance on the quality of derived 3D point clouds. RTN and PPK results consistently yielded RMSEz values within 10 cm, thus satisfying NOAA requirements for nautical charting. PPP did not meet the accuracy requirements but showed promising results that prompt further investigation. PPK experiments using higher GNSS sample rates did not always provide the best accuracies. GNSS performance and model accuracies were enhanced when using base stations located within 30 km of the survey site. Results without using GCPs observed a direct relationship between point cloud accuracy and GNSS performance, with R2 values reaching up to 0.97.

In the domain of road inspection and structural health monitoring, precise crack identification and segmentation are essential for structural safety and disaster prediction. Traditional image processing technologies encounter difficulties in detecting cracks due to their morphological diversity and complex background noise. This results in low detection accuracy and poor generalization. To overcome these challenges, this paper introduces MixSegNet, a novel deep learning model that enhances crack recognition and segmentation by integrating multi-scale features and deep feature learning. MixSegNet integrates convolutional neural networks (CNNs) and transformer architectures to enhance the detection of small cracks through the extraction and fusion of fine-grained features. Comparative evaluations against mainstream models, including LRASPP, U-Net, Deeplabv3, Swin-UNet, AttuNet, and FCN, demonstrate that MixSegNet achieves superior performance on open-source datasets. Specifically, the model achieved a precision of 95.2%, a recall of 88.2%, an F1 score of 91.5%, and a mean intersection over union (mIoU) of 84.8%, thereby demonstrating its effectiveness and reliability for crack segmentation tasks.

Slope instability in landslide prone area is a costly problem, which can lead to travel disruption, property damages, and injury or loss of life. For many years, slope monitoring activities are largely carried out by conventional survey instruments such as Total Station, levelling and Global Positioning System (GPS). However, these techniques is a time consuming, hazardous, and costly process, and it can be difficult to ensure that problems are recorded and handled in a consistent manner. Terrestrial laser scanners (TLS) find rapidly growing interest in remote sensing and photogrammetry field as efficient tools for fast and reliable three-dimensional (3D) point cloud data acquisition. The TLS technology is based on the reflectorless and contactless acquisition of a point cloud of the topography using the time-of-flight distance measurement of an infrared laser pulse. This paper describes the capabilities of TLS in slope mapping studies. The main objective of this study is to evaluate the slope mapping pattern at different resolution. This study has been conducted at Cameron Highland, Malaysia, which approximately located at latitude 4d26'36" and longitude 101d23'4". Several data acquisitions on selected cut slopes were performed using the Topcon GLS1500 laser scanner. Ground control points were established using Real Time Kinematic GPS to provide a local coordinate system on laser scanning data. Resolution of a laser scanner determines a density of point cloud during data acquisition. In this experiment, there is a different number of resolution has been set during data acquisition in order to evaluate the result of slope mapping pattern. It was found that, resolution gives an effect in slope mapping pattern.