6D Object Localization in Car-Assembly Industrial Environment

In the actual industrial production environment, the surface defects of products are subtle, and the number of different types of defect data samples is also quite small. Most deep learning models rely on a large number of training samples and parameters to achieve high-precision defect detection. At the same time, the edge computing layer in the actual industrial environment may also encounter transmission delays and insufficient resources. Training a proper model for a specific type of surface defect while simultaneously satisfying the real-time accuracy of defect detection is still a challenging task. To effectively deal with the above challenges, we propose an edge-cloud computing defect detection model based on the intrinsic mean feature detector in the Lie Group space. The modules in the model adopt a symmetrical structure, which can extract related features more effectively. Different from existing models, this model utilizes the Lie Group space intrinsic mean feature as a metric to characterize the essential attributes of different types of surface defects. In addition, we propose an intrinsic mean attention mechanism in the Lie Group manifold space that is easy to implement at the edge service layer without increasing the number of model parameters, thereby enhancing the detection performance of tiny surface defects. Extensive experiments on three publicly available and challenging datasets reveal the superiority of our model in terms of detection accuracy, real-time detection, number of parameters, and computational performance. In addition, our proposed model also shows competitiveness and advantages compared with state-of-the-art models.

The next generation of service robots is to offer services relying heavily on visually guided manipulation, besides navigation, such services as errand, logistics, appliance, home keeping, etc. For the successful introduction of these services, it is critical to establish the consumer-level dependability in 3D object recognition and pose estimation in a natural setting where a large variation of environment, e.g., perspective, texture, form factor, illumination, occlusion, etc., is common. To address this problem, we propose an approach of the two-layered particle filter to the dependability in 3D object recognition and pose estimation. In the upper layer, a set of object pose candidates is identified and maintained in the search space as a set of super-particles, each of which is assigned a probability of the true pose and evolved in time along with the accumulation of further evidences. To define the object pose candidates, first, we acquire initially weak evidences quickly and interpret them in terms of possible object poses in space. These interpretations serve as the region of interest for detailed investigation by which the pose probabilities are computed for individual interpretations based on the likelihood and unlikelihood of various features available in the corresponding regions of interest. During the process of probability computation, we select the object pose candidates to be used as super-particles in the upper layer. In the lower level, the pose uncertainties associated with the individual candidates are represented as particles that are subject to the propagation in time. Finally, the experimental results support the strength of the proposed approach in the real environment in terms of its dependability in 3D object recognition and pose estimation.

With the advancement of intelligent manufacturing, robots are increasingly involved in industrial production, for example, replacing human beings to identify and manipulate objects. However, it is still a challenging task for object recognition and pose estimation in industrial environment, especially under large environmental noise, mutual occlusions and high-speed requirements. In this paper, we propose a new method for 6D object pose estimation, Super Key 4-Points Congruent Sets (SK-4PCS), which is a fast global method combined with local features. The proposed method includes three steps. Firstly, the key points in 3D point clouds are extracted by local features, which are invariant with respect to rotation and translation. Secondly, the smart indexing scheme is used to realize the query of 4-points congruent sets globally. Finally, the coarse pose obtained quickly is further refined by the Iteration Closest Point algorithm (ICP) [1]. The proposed method is also compared with existing methods by the evaluation indicator Visible Surface Discrepancy (VSD) on the LINEMOD dataset [2]. The experimental results show that the refined SK-4PCS results in significant speedup and satisfactory accuracy in 6D object pose estimation.

With the continuous development of computer technology and deep learning, object detection technology has received great attention in recent years. Especially on the problem of 3D object detection in AR field, a variety of novel algorithms are in bloom. However, these algorithms are either low precision or slow speed. Therefore, we propose an object detection and pose estimation method for AR Application, which is a model for 6D pose estimation of 3D object based on 2D object detection. In our model, we firstly input the 2D projection image of the 3D model into the improved RetinaNet model, and detect the coordinates of the vertex and center point of the object on the image. Then the coordinates on the image and the coordinates on the CAD are matched by point matching. Then, the EPnP improved by Perspective-n-Point (PnP) algorithm is used to complete the pose estimation of 3D object. Finally, we test the performance of our algorithm. The results show that our accuracy has reached the average level of the mainstream algorithms.KeywordsPose estimationObject detectionDeep learningConvolutional neural network

Enabling robots to automatically locate and pick up randomly placed and oriented objects from a bin is an important challenge in factory automation, replacing tedious and heavy manual labor. A system should be able to recognize and locate objects with a predefined shape and estimate the position with the precision necessary for a gripping robot to pick it up. We describe a system that consists of a structured light instrument for capturing 3D data and a robust approach for object location and pose estimation. The method does not depend on segmentation of range images, but instead searches through pairs of 2D manifolds to localize candidates for object match. This leads to an algorithm that is not very sensitive to scene complexity or the number of objects in the scene. Furthermore, the strategy for candidate search is easily reconfigurable to arbitrary objects. Experiments reported in this paper show the utility of the method on a general random bin picking problem, in this paper exemplified by localization of car parts with random position and orientation. Full pose estimation is done in less than 380 ms per image. We believe that the method is applicable for a wide range of industrial automation problems where precise localization of 3D objects in a scene is needed.

3D object pose estimation by using a depth sensor is one of the important tasks in activities by robots. To reduce the pose ambiguity of an estimated object pose, several methods for multiple viewpoint pose estimation have been proposed. However, these methods need to select the viewpoints carefully to obtain better results. If the pose of the target object is ambiguous from the current observation, we could not decide where we should move the sensor to set as the next viewpoint. In this paper, we propose a best next viewpoint recommendation method by minimizing the pose ambiguity of the object by making use of the current pose estimation result as a latent variable. We evaluated viewpoints recommended by the proposed method and confirmed that it helps us to gain better pose estimation results than several comparative methods on a synthetic dataset.

A multi-module generative adversarial network augmented with adaptive decoupling strategy for intelligent fault diagnosis of machines with small sample

Open-set domain adaptive fault diagnosis based on supervised contrastive learning and a complementary weighted dual adversarial network

The treatment of tannery wastewaters is a complex task due to the complexity of the waste: a mixture of several pollutants, both anionic and cationic as well as organic macromolecules which are very hard to treat for disposal all together. Geopolymers are a class of inorganic binders obtained by alkali activation of aluminosilicate powders at room temperature. Such activation process leads to a cement like matrix that drastically decreases mobility of several components via entrapment. This process taking place in the matrix can be hypothesized to be the in-situ formation of zeolite structures. In this work we use a metakaolin based geopolymer to tackle the problem directly in an actual industrial environment. To obtain a geopolymer, the metakaolin was mixed with 10 wt% of wastewater added with sodium hydroxide and sodium silicate as activating solutions. This process allowed a rapid consolidation at room temperature, the average compressive strength was between 14 and 43 MPa. Leaching tests performed at different aging times confirm a high immobilization efficiency close to 100%. In particular, only the 0.008 and 2.31% of Chromium and Chlorides respectively are released in the leaching test after 7 months of aging.

Surface Mount Technology (SMT) pin solder joint defects are hard to detect since the joints are smaller and denser and have high similarity between defect samples and standard samples in solder joint images. We propose an improved YOLOv5 defect detection algorithm embedding Cascade Shuffle Space to Depth (CSSD), Coordinate Attention (CA) mechanism module, and K-means++ algorithm. The proposed improved Yolov5 significantly impacts the loss and model parameter reduction and higher positioning precision of the defect location on the disk. The optimum anchor box produces better clustering and stability. Compared with the original YOLOv5 under the same test conditions, the method in this paper improves the precision by 12.2%, recall by 9.4%, mAP by 9.0%, and model parameters reduced by 1.3M. In conclusion, the experimental results show that the algorithm proposed in this paper has a better detection effect and a smaller parameter scale. It also can better meet the defect detection and model deployment in the actual industrial production environment.

Up to now, the machine learning and deep learning algorithms discussed are usually designed to work alone. In the actual industrial operating environment, the working conditions are complex and changeable, so the model once trained in the source domain is used in the target domain, the diagnostic accuracy cannot be guaranteed, and it takes a lot of time to train the model. In view of the above two problems, this paper proposes a bearing fault diagnosis method based on the Transfer Learning Convolutional Neural Network (CRNN). Experimental results show that compared with other existing methods, the CRNN model proposed in this paper reduces the training time of the model while ensuring high accuracy, and the generalization ability of the model is improved, which has a good application value.

This paper describes a matrix-based discrete event control approach for an automated material handling system operating in a warehouse. The proposed approach addresses both low-level hardware synchronization/sequencing, and high-level operational control of rail-guided vehicles by means of an integrated framework based on binary matrices. The paper illustrates how to obtain the model of the entire warehouse by assembling the discrete event models of the atomic components. Once defined, the matrix-based dynamic model can be used for both controller synthesis and detailed discrete-event simulation of the system. In particular, a summary of simulation results obtained for an actual industrial warehouse environment is provided to illustrate the potential of the integrated matrix-based approach on large-scale, challenging automation problems

Six degrees-of-freedom (6D) object pose estimation plays an important role in pattern recognition of fields such as robotics and augmented reality. However, there are issues with low accuracy and real-time performance of 6D object pose estimation in complex scenes. To address these challenges, in this article, RFF-PoseNet (a 6D object pose estimation network based on robust feature fusion) is proposed for complex scenes. Firstly, a more lightweight Ghost module is used to replace the convolutional blocks in the feature extraction network. Then, a pyramid pooling module is added to the semantic label branch of PoseCNN to fuse the features of different pooling layers and enhance the network’s ability to capture information about objects in complex scenes and the correlations between contextual information. Finally, a pose regression and optimization module is utilized to further improve object pose estimation in complex scenes. Simulation experiments conducted on the YCB-Video and Occlusion LineMOD datasets show that the RFF-PoseNet algorithm can strengthen the correlation of features between different levels and the recognition ability of unclear targets, thereby achieving excellent accuracy and real-time performance, as well as strong robustness.

This work focuses on the problem of 6D pose estimation for novel objects when a reference 3D model or posed reference images are not available. While existing methods can estimate the precise 6D pose of objects, they heavily rely on curated CAD models or reference images, the preparation of which is a time-consuming and labor-intensive process. Moreover, in real-world scenarios, 3D models or reference images may not be available in advance and instant robot reaction is desired. In this work, we propose a novel framework named HIPPo, which eliminates the need for curated CAD models and reference images by harnessing image-to-3D priors from Diffusion Models, enabling model-free zero-shot 6D pose estimation. Specifically, we construct HIPPo Dreamer, a rapid image-to-mesh model built on a multiview Diffusion Model and a 3D reconstruction foundation model. Our HIPPo Dreamer can generate a 3D mesh of any unseen objects from a single glance in just a few seconds. Then, as more observations are acquired, we propose to continuously refine the diffusion prior mesh model by joint optimization of object geometry and appearance. This is achieved by a measurement-guided scheme that gradually replaces the plausible diffusion priors with more reliable online observations. Consequently, HIPPo can instantly estimate and track the 6D pose of a novel object and maintain a complete mesh for immediate robotic applications. Thorough experiments on various benchmarks show that HIPPo outperforms state-of-the-art methods in 6D object pose estimation when prior reference images are limited.

The need for efficient automation methods has prompted the rapid development in the field of Robotics. The development of intelligent robots leads to the ability of them becoming an operator highly efficient and able to adapt to a wide range of problems. Still, despite of the several robotic solutions available, the majority of current industrial robots do not use the Robotic Operative System (ROS) and have limitations in terms of autonomously correct errors during their tasks. An important aspect to consider when developing a new robot is the selection of the methodology for recognition of the objects to be handled. In this paper, it will be presented an approach with enough flexibility to be potentially applicable to different scenarios of object recognition and handling normally found in industrial environment. The main aim is for this application to be applied to pick and place routines in robotics. Moreover it will be presented one particular case study that used the proposed approach, the European Robotics Challenges (EUROC) — a challenge aiming to develop a robot for shop floor logistics and manipulation. Our proposed approach is based on the three tiers paradigm: 1-recognition/sensing tier, 2-effector tier and 3-the control tier and was built using the ROS framework. Since ROS is becoming one of the most useful tools in robotics nowadays, the possibility of using a methodology able to be expressed in ROS allows for the development of a standard approach to pick and place operations. Another advantage of our proposed pick and place approach is the ability to have a robot safely and efficiently inserted in an unknown environment. This is possible due to the insertion of an adaptive control tier in our methodology. The proposed approach can be valuable in the field of robotics and can be potentially applied in multiple tasks and it has already allowed us to advance to the next stage of the already mention challenge.