Evaluation of Weakly Supervised Learning Paradigms on Automatic Visual Inspection

In the task of action recognition, object and scene can provide rich source of contextual information for analyzing human actions, as human actions often occur under particular scene settings with certain related objects. Therefore, we try to utilize the contextual object and scene for improving the performance of action recognition. Specifically, a latent structural SVM is introduced to build the co-occurrence relationship among action, object and scene, in which the object class label and scene class label are treated as latent variables. Using this framework, we can simultaneously predict action class labels, object class labels as well as scene class labels. Moreover, we use a mid-level discriminative feature to separately describe the information of action, object and scene. The feature is actually a set of decision values from the pre-learned classifiers of each class, measuring the likelihood that the input video belongs to the corresponding class. In this paper, we use SVM as action and scene pre-learned classifiers, and use deformable part-based object detector as the object pre-learned classifier, so that object location can be obtained as a by-product. Experimental results on UCF Sports, YouTube and UCF50 datasets demonstrate the effectiveness of the proposed approach.

Newer knowledge of nutrition

The pixel-wise Markov image models introduced in the previous chapters have a limited ability in dealing with texture images. In order to describe large scale behavior in texture images, it is necessary to increase the size of the neighborhood. However, this dramatically increases the number of parameters and makes the computation for the parameter estimation quite difficult. Considering that the scales of the texture elements can be diverse, one possible way of dealing with various scales of texture elements is to express image data hierarchically using a multiscale transform. Laplacian pyramids [27] and wavelet multiresolution representations [124] are techniques that can decompose an image into multiple scales. Successive filtering and decimation operations in multiscale transforms provide a pyramid structure for image data. As a result, we have a sequence of image data for various scales and frequency bands. Specifically, coarse image components with smaller image resolution are located at the higher level and fine image components capturing detail can be found at the lower levels of the image pyramid. This multiscale image structure can provide useful information for the analysis and representation of the image data, especially for texture images. Processing successively from coarse to fine levels of the pyramid, image features at different scales and frequency bands are captured and class labels are assigned accordingly to large regions with low frequency and then refined to small image blocks and eventually to pixels with higher frequency information. This multiscale processing also allows us to save computations. That is, coarsening the image data for a multiscale image representation may smooth out local minima and result in faster convergence with a reduced sensitivity to an initial class label. Then, the optimal class labels at the current level of the pyramid to the next finer level can be obtained by searching in the vicinity of the class labels obtained at the previous coarser level. Since there exist strong inter scale and intra scale dependencies among vertically and horizontally connected neighboring data in the image pyramid, the multiscale images can be described more efficiently via Markov modeling. Adopting Markov models for the multiscale image data, the following issues arise: (i) how to describe various inter scale and intra scale interactions in terms of the Markov models, (ii) how to formulate the optimization problem for the class labels in the image pyramid, (iii) how to obtain the optimal class labels defined in (ii). These issues are treated in this chapter.

Virtualisation of Product Development may be interpreted in two ways: The virtualisation of the “Factory” or the virtualisation of the product development process. In order to virtualise product development, general development environments should be developed. In this talk, I will present some contributions to such environments: TrollCreek – a Case Based Reasoning (CBR) environment for utilization of knowledge and experience. XPLANO a Rule Based system for Automatic Process Planning. TrollEye – an Automatic Visual Inspection (AVI) environment using Machine Vision.

In a conventional supervised learning paradigm, each data instance is associated with one single class label. Multilabel learning differs in the way that data instances may belong to multiple concepts simultaneously, which naturally appear in a variety of high impact domains, ranging from bioinformatics and information retrieval to multimedia analysis. It targets leveraging the multiple label information of data instances to build a predictive learning model that can classify unlabeled instances into one or multiple predefined target classes. In multilabel learning, even though each instance is associated with a rich set of class labels, the label information could be noisy and incomplete as the labeling process is both time consuming and labor expensive, leading to potential missing annotations or even erroneous annotations. The existence of noisy and missing labels could negatively affect the performance of underlying learning algorithms. More often than not, multilabeled data often has noisy, irrelevant, and redundant features of high dimensionality. The existence of these uninformative features may also deteriorate the predictive power of the learning model due to the curse of dimensionality. Feature selection, as an effective dimensionality reduction technique, has shown to be powerful in preparing high-dimensional data for numerous data mining and machine-learning tasks. However, a vast majority of existing multilabel feature selection algorithms either boil down to solving multiple single-labeled feature selection problems or directly make use of the imperfect labels to guide the selection of representative features. As a result, they may not be able to obtain discriminative features shared across multiple labels. In this article, to bridge the gap between a rich source of multilabel information and its blemish in practical usage, we propose a novel noise-resilient multilabel informed feature selection framework (MIFS) by exploiting the correlations among different labels. In particular, to reduce the negative effects of imperfect label information in obtaining label correlations, we decompose the multilabel information of data instances into a low-dimensional space and then employ the reduced label representation to guide the feature selection phase via a joint sparse regression framework. Empirical studies on both synthetic and real-world datasets demonstrate the effectiveness and efficiency of the proposed MIFS framework.

PurposeHeritage is the latent part of a sustainable built environment. Conservation and preservation of heritage is one of the United Nations' (UN) sustainable development goals. Many social and natural factors seriously threaten heritage structures by deteriorating and damaging the original. Therefore, regular visual inspection of heritage structures is necessary for their conservation and preservation. Conventional inspection practice relies on manual inspection, which takes more time and human resources. The inspection system seeks an innovative approach that should be cheaper, faster, safer and less prone to human error than manual inspection. Therefore, this study aims to develop an automatic system of visual inspection for the built heritage.Design/methodology/approachThe artificial intelligence-based automatic defect detection system is developed using the faster R-CNN (faster region-based convolutional neural network) model of object detection to build an automatic visual inspection system. From the English and Dutch cemeteries of Surat (India), images of heritage structures were captured by digital camera to prepare the image data set. This image data set was used for training, validation and testing to develop the automatic defect detection model. While validating this model, its optimum detection accuracy is recorded as 91.58% to detect three types of defects: “spalling,” “exposed bricks” and “cracks.”FindingsThis study develops the model of automatic web-based visual inspection systems for the heritage structures using the faster R-CNN. Then it demonstrates detection of defects of spalling, exposed bricks and cracks existing in the heritage structures. Comparison of conventional (manual) and developed automatic inspection systems reveals that the developed automatic system requires less time and staff. Therefore, the routine inspection can be faster, cheaper, safer and more accurate than the conventional inspection method.Practical implicationsThe study presented here can improve inspecting the built heritages by reducing inspection time and cost, eliminating chances of human errors and accidents and having accurate and consistent information. This study attempts to ensure the sustainability of the built heritage.Originality/valueFor ensuring the sustainability of built heritage, this study presents the artificial intelligence-based methodology for the development of an automatic visual inspection system. The automatic web-based visual inspection system for the built heritage has not been reported in previous studies so far.

This paper presents a comprehensive study of the application of artificial intelligence technologies to address applied challenges in modern mechanical engineering, with a focus on the development of a prototype automatic visual inspection system. Based on an analysis of current scientific and industry literature, an analytical review of key areas of AI implementation in mechanical engineering was conducted. It was established that AI has evolved from an automation tool to a key component of intelligent manufacturing systems, capable of solving cognitive problems and enabling the transition to predictive and adaptive control models. A systematic classification of mechanical engineering problems solved using AI was developed and presented, structured by stages of the product lifecycle (PLM). This clearly demonstrates the broad impact of this technology — from generative design and predictive maintenance to logistics optimization and service analytics. A detailed analysis of existing commercial solutions and successful implementation cases was conducted. It was revealed that predictive maintenance (PdM) and automatic visual inspection (AVI) systems based on computer vision are the most mature and cost-effective. A comparison of platforms from leading vendors (Siemens, Autodesk, Dassault Systèmes) showed a trend toward deep integration of AI into unified PLM ecosystems, creating digital, continuous data flows.

Automatic visual inspection has become an important application of pattern recognition, as it supports the human in this demanding and often dangerous work. Nevertheless, often missing abnormal or defective samples prohibit a supervised learning of defect models. For this reason, techniques known as one-class classification and novelty- or unusual event detection have arisen in the past years. This paper presents a new strategy to employ Hidden Markov models for defect localization in wire ropes. It is shown, that the Viterbi scores can be used as indicator for unusual subsequences. This prevents a partition of the signal into sufficient small signal windows at cost of the temporal context. Our results outperform recent time-invariant one-class classification approaches and depict a great advance for an automatic visual inspection of wire ropes.

Dimensionality reduction of image features using the canonical contextual correlation projection

画像処理を用いた研磨作業における外観検査の自動化 Neural Networkを用いた良否判定

Multilayered Perceptrons as Classifiers for Automatic Inspection

In most mass-production manufacturing facilities, an attempt is often made to achieve 100% quality assurance of all parts, subassemblies, and finished products. One of the most difficult tasks in this process is that of inspecting for visual appearance —an inspection that seeks to identify both functional and cosmetic defects. Undoubtedly, the automation of visual inspection will increase productivity and improve product quality.

Training with noisy class labels impairs neural networks’ generalization performance. In this context, mixup is a popular regularization technique to improve training robustness by making memorizing false class labels more difficult. However, mixup neglects that multiple annotators, e.g., crowdworkers, typically provide class labels. Therefore, we propose an extension of mixup, which handles multiple class labels per instance while considering which class label originates from which annotator. Integrated into our multi-annotator classification framework annot-mix, it performs superiorly to eleven (mostly state-of-the-art) approaches in an evaluation study with eleven datasets comprising noisy class labels from either human or simulated annotators. Our code is publicly available through our GitHub repository at https://github.com/ies-research/multi-annotator-machine-learning/tree/annot-mix.

A common approach for image classification is based on image feature extraction and supervised discriminative learning. For medical image classification problems where discriminative image features are spatially distributed around certain anatomical structures, localizing the region of interest (ROI) essential for the classification task is a key to success. To address this problem, we develop a multi-atlas label fusion technique for automatic ROI detection. Given a set of training images with class labels, our method infers voxel-wise scores for each image showing how discriminative each voxel is for categorizing the image. We applied our method in a 2D cardiac CT body part classification application and show the effectiveness of the detected ROIs.

In this paper, we propose a Phase Fourier Reconstruction (PFR) approach for anomaly detection on metal surfaces using salient irregularities. To get salient irregularity with images captured from an automatic visual inspection (AVI) system using different lighting settings, we first trained a classifier for image selection as only dark images are utilized for anomaly detection. By doing so, surface details, part design, and boundaries between foreground/background become indistinct, but anomaly regions are highlighted because of diffuse reflection caused by rough surfaces. Then PFR is applied so that regular patterns and homogeneous regions are further de-emphasized, and simultaneously, anomaly areas are distinct and located. Different from existing phase-based methods which require substantial texture information, our PFR works on both textual and non-textual images. Unlike existing template matching methods which require prior knowledge of defect-free patterns, our PFR is an unsupervised approach which detects anomalies using a single image. Experimental results on anomaly detection clearly demonstrate the effectiveness of the proposed method which outperforms several well-designed methods [8, 12, 15, 16, 18, 19] with a running time of less than 0.01 seconds per image.