Automated deep learning system for power line inspection image analysis and processing: Architecture and design issues

Abstract: The paper presents sharing several of experiences and practices on smart robotic application for overhead transmission line maintenance and inspection. The robotic puller passes over barriers below the power line, such as the road with traffic, power distribution lines, river, or vegetation making tasks achieved conveniently, safely, and rapidly without impact on nearby communities.The target of the research is to create a mobile robot prototype for inspection of overhead power lines. The inspection robot shall crawl along the ground wire and transpose autonomously across installed equipment on the ground wire, such as vibration dampers, suspension clamps, compression dead ends, etc. In addition, the inspection robot is able to take photos and videos during a transmission line inspection . Using the robot, transmission line inspection’s labor cost can be reduced, and the new method helps improve patrol and inspection efficiency, comparing to the conventional manpower method. Trough utilization of the new maintenance and inspection robots, utilities can minimiz transmission line operation & maintenance budget.

Objective: The aim of the article is to present the state of the art on threats to UAVs posed by sudden changes of weather conditions, involving in particular storms and hail. Identifying such threats is likely to facilitate the development of hardware and software solutions aimed at improving the safety of the UAV missions. Introduction: Severe weather phenomena strongly affect the safety of unmanned aerial vehicle (UAV) missions. The article describes in detail the implementation concept of atmospheric discharge and precipitation images using wireless Internet connections for an exemplary mobile application for multi-rotor flight control. The concept was developed to reduce risks associated with flights in storm and precipitation areas, and to maximise the performance of electric multi-rotor UAVs with a maximum take-off mass of no more than 25 kg. This is dealt with in the introductory section and in three substantive sections. UAV capabilities are discussed in the first section, while the second one analyses specific weather conditions which merit particular attention when constructing new UAVs, and the third one outlines some of the solutions employed in UAV steering systems control apps. A number of concluding remarks are presented in the end. Conclusions: The analysis of usefulness has proven electric UAVs suitability for professional missions. However, these vehicles display a number of limitations, one of the most important being their high sensitivity to weather conditions. With no hermetically sealed hatch, on-board equipment is exposed to penetrating damp caused by various types of precipitation, including drizzle, rain and snow. These can cause a short-circuit and equipment damage, while hailstones can result in UAV mechanical damage. Severe turbulences and strong ascending or descending air currents, which are likely to occur during low-altitude flights, pose a threat of UAV's collision with overground objects. Using more resistant composite materials in the construction of hermetic UAVs could increase the application range of these vehicles. The wide range of possible uses of UAVs encourages further investigations aimed at ensuring their best possible working conditions.

A power line segmentation model in aerial images based on an efficient multibranch concatenation network

The Unmanned Aerial Vehicle (UAV) has been around for a long time but has been widely used recently by humans. Their acceptance of various communications-based applications is expected to improve coverage, compared to traditional ground-based solutions. In this paper, the Deep-learning and Image Processing Process framework is expected to provide solutions to the various problems already identified when UAVs are used for communication purposes. UAVs are used in disaster relief because of their accessibility even in inaccessible places. In this paper, we propose research into Deep learning and Image Processing strategies for UAVs. In deep learning is a form of machine learning that teaches computers to do what comes naturally to people: learn by example and get a lot of attention recently and for a good reason. It achieves previously impossible results. Image processing is the process of performing a specific task on an image, finding an enhanced image or extracting useful information from it. So our paper has the idea of using in depth face recognition and photo processing a digital photo taken by the UAV to identify victims of rescue, overcoming back to the latest UAV technology some of which include blurry images, unable to identify the victim when there are too many objects and much more. The solution includes a variety of features that allow for the distribution of images. It includes features and presentation of image detection and demonstrates the effectiveness of drone use in damage applications.

Agricultural pests and diseases affect grain yield and quality. In addition to the use of chemical agents, the biological control of diseases and insect pests has attracted more and more attention due to an urgent need for environmental protection. However, traditional operational methods do not integrate well with biological control technology. With the advantages of simple operation, low operating cost, high operating efficiency and wide application range, use of multirotor unmanned aerial vehicles (UAVs) in biological control is of great significance. An intelligent delivery system comprising an insect natural enemy (Trichogramma) capsule based on an M45 electric multirotor UAV was built to release Trichogramma ostriniae in corn fields under full autonomous mode. During operation, the M45 UAV can fly accurately in accordance with operational requirements; average deviation between the actual and planned routes was 0.24 m, the average flight speed was 9.18 m s-1 and the average altitude was 5.08 m. The effective coverage rate of natural enemies was 99.33% and the coverage redundancy was 36.75%. After delivery, an investigation into the field control effect showed an average control effect of 83.70%, which can result in effective pest control. A natural enemy carrier intelligent delivery system based on an M45 electric multirotor UAV was found to have good flight performance. In addition, natural enemies were evenly distributed, giving even coverage of the target fields. The system had a good control effect. © 2021 Society of Chemical Industry.

During the past decade, deep learning is one of the essential breakthroughs made in artificial intelligence. In particular, it has achieved great success in image processing. Correspondingly, various applications related to image processing are also promoting the rapid development of deep learning in all aspects of network structure, layer designing, and training tricks. However, the deeper structure makes the back-propagation algorithm more difficult. At the same time, the scale of training images without labels is also rapidly increasing, and class imbalance severely affects the performance of deep learning, these urgently require more novelty deep models and new parallel computing system to more effectively interpret the content of the image and form a suitable analysis mechanism. In this context, this survey provides four deep learning model series, which includes CNN series, GAN series, ELM-RVFL series, and other series, for comprehensive understanding towards the analytical techniques of image processing field, clarify the most important advancements and shed some light on future studies. By further studying the relationship between deep learning and image processing tasks, which can not only help us understand the reasons for the success of deep learning but also inspires new deep models and training methods. More importantly, this survey aims to improve or arouse other researchers to catch a glimpse of the state-of-the-art deep learning methods in the field of image processing and facilitate the applications of these deep learning technologies in their research tasks. Besides, we discuss the open issues and the promising directions of future research in image processing using the new generation of deep learning.

A three-dimensional (3D) laser scanning system is the main part of reverse engineering. It is a comprehensive technological system, including numerical control (NC), a laser, computer-aided design/manufacturing (CAD/CAM), precision mechanisms, computing image sampling, and image processing. Generally, it consists of a motion control system, an image sampling system, and an image processing system, etc. This paper focuses on introducing control principles, hardware constitutions, and software algorithms of the servo motion system, and analyzes and presents the hardware constitutions, software algorithms, and experimental results of the image sampling and processing system in great detail. Finally, the authors come to the conclusion that the errors all occur within the range ±0.1 mm.

Electric Unmanned Aerial Vehicles (UAVs) experience problems and risks associated with battery aging and abuse effects. Therefore, a Battery Health Management (BHM) system is necessary to make the battery a safe, reliable, and cost-efficient solution. BHM systems are essential to ensure that the mission goal(s) can be achieved and to aid in online decision-making activities such as fault mitigation and mission replanning. To accomplish these tasks, we have adopted a model-based prognostics architecture for battery-powered UAVs where a battery model is used as the basis of two sequential task, (i) the State of Charge (SOC) estimation, and (ii) the End of Discharge (EOD) prediction. Small-size aircraft usually have weight, size and cost constraints. Therefore, there is a need to accurately (i) estimate the SOC, and (ii) predict the EOD time of Li-Po batteries in small-size UAVs that can operate in constrained environments. This work proposes a modification to an electrochemistry-based battery model that allows reducing computational resources without losing accuracy in prognostic results. The resulting hybrid battery model is validated and applied to prognostic of the EOD time in discharge cycles of a Li-Po battery of a small size quadcopter that performs delivery missions. Prediction results using the proposed hybrid battery model are shown to be very accurate while its estimation and prediction processing times are significantly lower than processing times using the electrochemistry-based battery model.

In recent years, unmanned aerial vehicles (UAV) have been increasingly used in power line inspections. Birds often nest on transmission line towers, which threatens safe power line operation. The existing research on bird’s nest inspection using UAVs mainly stays at the level of image postprocessing detection, which has poor real-time performance and cannot obtain timely bird’s nest detection results. Considering the above shortcomings, we designed a power inspection UAV system based on deep learning technology for autonomous flight, positioning and photography, real-time bird nest detection, and result export. In this research, 2000 bird’s nest images in the actual power inspection environment were shot and collected to create the dataset. The parameter optimization and test comparison for bird’s nest detection are based on the three target detection models of YOLOv3, YOLOv5-s, and YOLOX-s. A YOLOv5-s bird’s nest detection model optimized for bird’s nest real-time detection is proposed, and it is deployed to the onboard computer for real-time detection and verification during flight. The DJI M300 RTK UAV was used to conduct a test flight in a natural power inspection environment. The test results show that the mAP of the UAV system designed in this paper for bird’s nest detection is 92.1%, and the real-time detection frame rate is 33.9 FPS. Compared with the previous research results, this paper proposes a new practice of using drones for bird’s nest detection, dramatically improving the real-time accuracy of bird’s nest detection. The UAV system can efficiently complete the task of bird’s nest detection in the process of electric power inspection, which can significantly reduce manpower consumption in the power inspection process.

An image processing system that incorporated some retinal properties was investigated for the processing of two dimensional images. The system was required to carry out basic image processing tasks such as edge detection. A new filtering technique was deduced from the physiological findings on the distribution of the receptive fields of the retinal ganglion cells. This filtering technique was then incorporated in the design of an image processing system in, which the spatial resolution increased linearly towards the centre of, the image. The design, was based on a discrete polar distribution of processing areas on an inhomogeneous triangular sampling grid. This resulted in a highly localized processing system which simplified the development of the higher image processing tasks such asboundary following. The retinal image processing system was simulated on a VAX 11/750. The computational cost of conducting operations such as edge detection, boundary detection and boundary following, using the designed system, was evaluated and compared with that of the conventional image processing system.

Power systems currently face several challenges in order to address the growing need for sustainable energy worldwide. In this context, nowadays several tasks either directly or indirectly related to power systems can be automated to guarantee their continuous operation. The surveillance and inspection of transmission lines represents a necessary task to meet the energy demands by the commercial and industrial customers. Since power-lines can be thought as hostile environments for humans it is thus necessary to develop new techniques to improve this process, to reduce costs, to improve response times and more important, to avoid operational risks. Although there has been an incipient development of robotic applications in several areas of power systems, it is still a technology under research and development, since it is highly dependent on the nature of the task to be performed and no unified framework of applications exists. This article aims to present the preliminary system for tracking transmission lines based on detection of wires through artificial vision. The system is mounted on a robotic arm, simulating an unmanned aerial vehicle (UAV) which uses the transmission lines detection to position itself. The same system will be used later to inspect the state of the transmission line with the aim of reporting such state and to offer corrective guidelines against damages or problems in the power system if necessary.

In this article, a novel lithium ion batteries thermal design to manage ultrahigh thermal shock of a commercial small four‐rotor electric unmanned aerial vehicle (UAV) is developed and optimized. The cooling relies on internal heat pipe and convective cooling of UAV body, so that the heat generated by the battery can reach exterior cooling through a dedicated thermal path. The thermal management system is passive and requires no additional energy consumption. Finite element simulation is established to model the system and advanced thermal materials, and the system is optimized by response surface method in a dual objectives approach: minimizing maximum battery temperature and weight of the UAV. Simulation results show that the new passive cooling system can keep the temperature of the lithium ion battery within a reasonable range. The effects of convective heat transfer coefficient, discharge rate, and fuselage thermal conductivity were studied. After optimizing the system, the highest temperature of battery discharging at 3C rate can be kept no more than 36°C in the cost of 32.18 g weight increment.

and medicine is one of the most important study tools, as investigations of many current questions in a wide range of scientific directions, including physiology, use images as the initial data form. The first stage in the establishment of the Image Processing Center at the Pavlov Institute of Physiology of the Russian Academy of Sciences in 1996 has been discussed previously [2]. The center was developed as part of Project No. 225 (April 13, 2001) under the auspices of the Federal “State Support for the Integration of Higher Education and Basic Science” program (3.2, “Creation of collective industrial-academic centers with state-of-the-art equipment”). The center became the “Collective Biological and Medical Image Processing Center.” The main user was the Institute of Physiology, while others included St. Petersburg State University and St. Petersburg State Medical University. Financial support for the center under the heading “Development and introduction of information systems for biological image processing and analysis” was continued in 2002 in the framework of the “Mechanisms of adaptive behavior: intracellular signal transduction and gene expression” program of the Presidium of the Russian Academy of Sciences. Expansion of the computerized resources of the center provided biological and medical science research institutions and companies with the ability to process images, as well as facilities for training students and involving them in investigations in the framework of the integration of academic and industrial science. Scientific research programs involving students, post-graduate scientists, and post-doctoral scientists were developed using state-of-the-art computerized image processing methods. The main aim of the present article is to inform colleagues working in different areas of physiology and obtaining experimental data as images of the new capabilities of the center, particularly specialized information systems (SIS) which have been developed and to invite them to take part in collaborative studies. New video-based computer systems are continually being created at the center for processing images in different laboratories of the institute, and new specialized SIS have been developed. These support image processing, formation of databases in subject areas, and the export of results into statistical analysis programs. The facilities at the center support the processing of static and dynamic images of any physiological and biological nature. They provide for the analysis of images reflecting the structural-functional organization of neural networks in the brain, the structural-functional organization of autonomic processes, the neurogenic mechanisms of activity of the internal organs, and the genetic mechanisms of organisms. Images reflecting the mechanisms of sensory and cognitive processes are analyzed, as are those related to the psychophysical image recognition patterns, the mechanisms of neuroendocrine regulation, and the formation of pathological CNS states. Our approach in creating SIS for image processing consists of using the physiological principles of the organiNeuroscience and Behavioral Physiology, Vol. 41, No. 7, September, 2011

Power transmission network is the most critical part of a power system due to its connectivity with generation and distribution stations. Though it is a riskier employment to carry out the routine inspections of the transmission lines manually, the task of inspection is imperative to the continuous operation of the power system. However, the new trend of transmission line inspection is based on extracted details of the lines by means of Remotely Operated Vehicles (ROVs) traversing through them. This paper proposes a method being tested by a prototype for traversing alone the transmission conductor, inspecting the line through real time video streaming, detecting faults and pinpointing them through Geo Tagging. Automated transmission line inspection and fault detection is proposed to carry out through image processing and sensory data acquisition. Radio Frequency (RF) technology is the main communication mechanism between the operator and the ROV. This technology will expand the remotely operating distance of the ROV. Furthermore, a mechanism was developed to enable the robot to cross over from one span to another in the transmission network which include suspension type insulators.

Since insulators play an essential part in transmission lines, insulator defect detection could be an important assignments for intelligent inspection of high-voltage transmission lines. In this paper, an improved YOLOv5 algorithm for insulator defect detection task for aerial images with various backgrounds is proposed. We use collected aerial images of insulators with one or more defects in different scenarios, perform data augmentation of exposure and noise on the images to expand the sample (expanded to 2125 images), and establish a dataset by combining aerial images of normal working insulators. By adjusting the CSP(Cross-Stage-partial-connections) and CBL(Convalution-BatchNorm-Leaky_relu) modules in the YOLOv5 model to change the depth and width of the model, change model parameters, and build five different scale YOLOv5 models to further meet the real-time task. ResNet and DarkNet are used for the transfer learning of the YOLOv5 model, and various optimization methods are used in the Backbone structure, Neck structure and output of the model, then the established data set is trained and tested on each YOLOv5 model. Among them, the YOLOv5n model has the fastest detection speed, which can reach 10ms, and the precision also reaches 95%. The YOLOv5x model has the highest precision, reaching 97%, and the detection speed is 21ms. These models are all able to satisfy the accuracy and real-time mission in the process of aerial photography and analysis among which YOLOv5n can achieve lightweight tasks while being efficient enough.