Research on image segmentation based on unsupervised learning methods in computer vision

We apply computer vision and machine learning methods to analyze two datasets of microstructural images. A transfer learning pipeline utilizes the fully connected layer of a pre-trained convolutional neural network as the image representation. An unsupervised learning method uses the image representations to discover visually distinct clusters of images within two datasets. A minimally supervised clustering approach classifies micrographs into visually similar groups. This approach successfully classifies images both in a dataset of surface defects in steel, where the image classes are visually distinct and in a dataset of fracture surfaces that humans have difficulty classifying. We find that the unsupervised, transfer learning method gives results comparable to fully supervised, custom-built approaches.

Benchmark datasets used for testing computer vision (CV) methods often contain little variation in illumination. The methods that perform well on these datasets have been observed to fail under challenging illumination conditions encountered in the real world, in particular, when the dynamic range of a scene is high. The authors present a new dataset for evaluating CV methods in challenging illumination conditions such as low light, high dynamic range, and glare. The main feature of the dataset is that each scene has been captured in all the adversarial illuminations. Moreover, each scene includes an additional reference condition with uniform illumination, which can be used to automatically generate labels for the tested CV methods. We demonstrate the usefulness of the dataset in a preliminary study by evaluating the performance of popular face detection, optical flow, and object detection methods under adversarial illumination conditions. We further assess whether the performance of these applications can be improved if a different transfer function is used.

Optical accuracy is a primary driver of parabolic trough concentrating solar power (CSP) plant performance, but can be damaged by wind loads, gravity, error during installation, and regular plant operation. Collecting and analyzing optical measurements over an entire operating parabolic trough plant is difficult, given the large scale of typical installations. Distant Observer, a software tool developed at the National Renewable Energy Laboratory, uses images of the absorber tube reflected in the collector mirror to measure both surface slope in the parabolic mirror and offset of the absorber tube from the ideal focal point. This technology has been adapted for fast data collection using low-cost commercial drones, but until recently still required substantial human labor to process large amounts of data. A new method leveraging advanced deep learning and computer vision tools can drastically reduce the time required to process images. This new method addresses the primary analysis bottleneck, identifying featureless, reflective mirror corner points to a high degree of accuracy. Recent work has shown promising results using computer vision methods. The combined deep learning and computer vision approach presented here proved highly effective and has the potential to further automate data collection and analysis, making the tool more robust. The method presented in this paper automatically identified 74.3% of mirror corners within 2 pixels of their manually marked counterparts and 91.9% within 3 pixels. This level of accuracy is sufficient for practical Distant Observer analysis within a target uncertainty. A commercial drone collected video of over 100 parabolic trough modules at an operating CSP plant to demonstrate the deep learning and computer vision method’s usefulness in processing large amounts of data. These troughs were successfully analyzed using Distant Observer, paired with the new deep learning and computer vision algorithm, and can provide plant operators and trough designers with valuable insight about plant performance, operating strategies, and plant-wide optical error trends.

Conventional unsupervised image segmentation methods use color and geometric information and apply clustering algorithms over pixels. They preserve object boundaries well but often suffer from over-segmentation due to noise and artifacts in the images. In this paper, we contribute on a preprocessing step for image smoothing, which alleviates the burden of conventional unsupervised image segmentation and enhance their performance. Our approach relies on a convolutional autoencoder (CAE) with the total variation loss (TVL) for unsupervised learning. We show that, after our CAE-TVL preprocessing step, the over-segmentation effect is significantly reduced using the same unsupervised image segmentation methods. We evaluate our approach using the BSDS500 image segmentation benchmark dataset and show the performance enhancement introduced by our approach in terms of both increased segmentation accuracy and reduced computation time. We examine the robustness of the trained CAE and show that it is directly applicable to other natural scene images.

Research in surgical robotics and automation has made remarkable advancements in recent years thanks to new methods in computer vision, control, and deep learning. Autonomous end-effector manipulation is a challenging task in surgical robotics, and cutting with scissor tools is largely unexplored. A concurrent work explored path and trajectory generation for cutting deformable materials using the da Vinci Research Kit (dVRK) [1]. However, an efficient and realistic simulation is necessary for methods such as reinforcement learning (RL) or learned trajectory planning. Our previous work built a simulation for the dVRK in Unity for training RL algorithms on rigid body tasks [2]. To our knowledge, there is no dVRK simulation available that includes the cutting of deformable materials. This paper introduces a cutting simulation of a deformable mesh, which can represent a tissue layer, built onto our Unity dVRK simulation.

Rapid-Motion-Track: Markerless tracking of fast human motion with deep learning

Background: Making an automatic diagnosis based on virtual slides and whole slide imaging or even determining whether a case belongs to a single class, representing a specific disease, is a big challenge. In this work we focus on WHO Classification of Tumours of the Central Nervous System. We try to design a method which allows to automatically distinguish virtual slides which contain histopathologic patterns characteristic of glioblastoma – pseudopalisading necrosis and discriminate cases with neurinoma (schwannoma), which contain similar structures – palisading (Verocay bodies).Methods: Our method is based on computer vision approaches like structural analysis and shape descriptors. We start with image segmentation in a virtual slide, find specific patterns and use a set of features which can describe pseudopalisading necrosis and distinguish it from palisades. Type of structures found in a slide decides about its classification.Results: Described method is tested on a set of 49 virtual slides, captured using robotic microscope. Results show that 82% of glioblastoma cases and 90% of neurinoma cases were correctly identified by the proposed algorithm.Conclusion: Our method is a promising approach to automatic detection of nervous system tumors using virtual slides.

In this paper, an unsupervised segmentation method using clustering is presented for color images. We propose to use a neural network based approach to automatic feature selection to achieve adaptive segmentation of color images. With a self-organizing feature map (SOFM), multiple color features can be analyzed, and the useful feature sequence (feature vector) can then be determined. The encoded feature vector is used in the final segmentation using fuzzy clustering. The proposed method has been applied in segmenting different types of color images, and the experimental results show that it outperforms the classical clustering method. Our study shows that the feature encoding approach offers great promise in automating and optimizing the segmentation of color images.

The implementation of routine breast cancer screening and better treatment strategies made possible to offer to the majority of women the option of breast conservation instead of a mastectomy. The most important aim of breast cancer conservative treatment (BCCT) is to try to optimize aesthetic outcome and implicitly, quality of life (QoL) without jeopardizing local cancer control and overall survival. As a consequence of the impact aesthetic outcome has on QoL, there has been an effort to try to define an optimal tool capable of performing this type of evaluation. Starting from the classical subjective aesthetic evaluation of BCCT (either by the patient herself or by a group of clinicians through questionnaires) to an objective aesthetic evaluation (where machine learning and computer vision methods are employed), leads to less variability and increasing reproducibility of results. Currently, there are some offline software applications available such as BATⒸ and BCCT.core, which perform the assessment based on asymmetry measurements that are computed based on semi-automatically annotated keypoints. In the literature, one can find algorithms that attempt to do the completely automatic keypoint annotation with reasonable success. However, these algorithms are very time-consuming. As the course of research goes more and more into the development of web software applications, these time-consuming tasks are not desirable. In this work, we propose a novel approach to the keypoints detection task treating the problem in part as image segmentation. This novel approach can improve both execution-time and results.

Lately unmanned aerial vehicles (UAV) became an unseparated part of modern life. These machines gained their particular popularity in the field of amateur or professional video production and small cargo deliveries. But they are not only limited by civilian lines of activity. Nowadays they are also used in military conflicts either as scout or as a deadly precise weapon. Due to UAV’s easy market access, these drones can be used by the hands of terrorists groups. It’s not a rocket science to attach an explosive device to UAV, guide it to strategic area and detonate it from far away. This is one of the main danger sources for such places like nuclear power plants. A downward trend also led to new type of thread – swarm attacks, when group of small UAVs strikes their victim. A timely detection of such attacks is not a trivial task. This question is being studied by many researches from different fields. The most promising methods comes from radiolocation and computer vision areas. Next in the paper, modern algorithms of detection, classification and tracking of UAVs by using optical flow will be reviewed. Usually, the common approach for this task comes from classic computer vision methods or, highly popular today, deep learning methods.

The automation of insect pest control activities implies the use of classifiers to monitor the temporal and spatial evolution of the population using computer vision algorithms. In this regard, the popularisation of supervised learning methods represents a breakthrough in this field. However, their claimed effectiveness is reduced regarding working in real-life conditions. In addition, the efficiency of the proposed models is usually measured in terms of their accuracy, without considering the actual context of the sensing platforms deployed at the edge, where image processing must occur. Hence, energy consumption is a key factor in embedded devices powered by renewable energy sources such as solar panels, particularly in energy harvesting platforms, which are increasingly popular in smart farming applications. In this work, we perform a two-fold performance analysis (accuracy and energy efficiency) of three commonly used methods in computer vision (e.g., HOG+SVM, LeNet-5 CNN, and PCA+Random Forest) for object classification, targeting the detection of the olive fly in chromatic traps. The training and testing of the models were carried out using pictures captured in various realistic conditions to obtain more reliable results. We conducted an exhaustive exploration of the solution space for each evaluated method, assessing the impact of the input dataset and configuration parameters on the learning process outcomes. To determine their suitability for deployment on edge embedded systems, we implemented a prototype on a Raspberry Pi 4 and measured the processing time, memory usage, and power consumption. The results show that the PCA-Random Forest method achieves the highest accuracy of 99%, with significantly lower processing time (approximately 6 and 48 times faster) and power consumption (approximately 10 and 44 times lower) compared with its competitors (LeNet-5-based CNN and HOG+SVM).

Breast lesion segmentation in magnetic resonance (MR) images is one of the most important parts of clinical diagnostic tools. Pixel classification methods have been frequently used in image segmentation with two supervised and unsupervised approaches up to now. Supervised segmentation methods lead to high accuracy, but they need a large amount of labeled data, which is hard, expensive, and slow to be obtained. On the other hand, unsupervised segmentation methods need no prior knowledge and lead to low performance. However, semi-supervised learning which uses not only a few labeled data, but also a large amount of unlabeled data promises higher accuracy with less effort. In this paper, we propose a new interactive semi-supervised approach to segmentation of suspicious lesions in breast MRI. Using a suitable classifier in this approach has an important role in its performance; in this paper, we present a semi-supervised algorithm improved self-training (IMPST) which is an improved version of self-training method and increase segmentation accuracy. Experimental results show that performance of segmentation in this approach is higher than supervised and unsupervised methods such as K nearest neighbors, Bayesian, Support Vector Machine, and Fuzzy c-Means.

Разработано программное обеспечение ImgOpinion, которое позволяет выполнять оптико-структурный анализ цифрового изображения объекта методами компьютерного зрения. Оно предназначено для использования в экспертных лабораториях в составе аппаратно-программного комплекса компьютерного зрения MS-Unit, который включает специализированный осветитель “Фотобокс 3138” с цифровой регистрирующей камерой. Выходные данные строятся в виде цифрового паспорта огнестрельного повреждения, в котором приводятся криминалистически значимые характеристики объекта исследования, а также выявленных на нем следов выстрела. The ImgOpinion software, capable to perform the optical and structural analysis of a digital image of the object by computer vision methods, was developed. It can be employed in expert laboratories as a part of the computer vision hardware-software complex MS-Unit together with specialized illuminator “Photobox 3138” with a digital recording camera. The output data is constructed in the form of a digital passport of the gunshot residue, which contains forensically significant characteristics of the object under investigation, as well as a gunshot residue detected on it. Purpose. The research has solved the problem of creating a specialised hardware and software complex for forensic examination, the use of which partially automates the processes of identification and quantitative characterisation of gunshot traces and products. Methodology. Non-destructive research methods are implemented in computer vision systems, which include two main stages: a digital image acquisition and its mathematical processing. The hardware and software complex of a computer vision usually consists of an illuminator, a digital recording video camera and a specialised program for processing the raw data. Findings. ImgOpinion software has been developed, which allows performing the optical-structural analyses of digital images using the computer vision methods and is intended for use in expert laboratories as part of the MS-Unit computer vision hardware and software complex, which includes a specialized illuminator “Photobox 3138” with a digital recording camera. Autonomous and portable device has a mobile design and provides white light with colour temperature 5000 K (CIE D50) and colour rendering index CRI = 97+. Six independently switched LED monochrome illuminators with narrow spectral bands from 365 to 880 nm (UV to IR spectral bands) provide the uniform illumination without extraneous stray light on the working field (light intensity drop below 2 % at the edges of the working field). Value. The ImgOpinion output is constructed in the form of a digital passport of a gunshot injury, which contains forensically relevant characteristics of the object of investigation. The digital passport is an adaptation of the forensically significant information on the firearm traces for the integration into specialized databases for further development of the automation of expert evaluation processes, conducting the automated trace comparison and the determination of incident circumstances by it.

Today, computer vision has shown a variety of roles in our lives, making people’s life more convenient. Also, a variety of artificial intelligence models and algorithms have emerged for computer vision. Image matching is an important technique in the field of computer vision to find the similarities between two images or multiple images with the help of matching algorithms to achieve scientific and accurate processing of images. Therefore, summarizing the effective approximate solution to this problem as well as the future research is the main part for current research. The paper firstly describes the basic elements of the image matching technique. Then, some representative traditional image matching algorithms proposed in the field of computer vision research in recent years are summarized and reviewed. Finally, the future research directions and research ideas of image matching are discussed, providing reliable guidance and reference for subsequent research.

Automated computer vision methods and tools offer new ways of analysing audio-visual material in the realm of the Digital Humanities (DH). While there are some promising results where these tools can be applied, there are basic challenges, such as algorithmic bias and the lack of sufficient transparency, one needs to carefully use these tools in a productive and responsible way. When it comes to the socio-technical understanding of computer vision tools and methods, a major unit of sociological analysis, attentiveness, and access for configuration (for both computer vision scientists and DH scholars) is what computer science calls “ground truth”. What is specified in the ground truth is the template or rule to follow, e.g. what an object looks like. This article aims at providing scholars in the DH with knowledge about how automated tools for image analysis work and how they are constructed. Based on these insights, the paper introduces an approach called “active learning” that can help to configure these tools in ways that fit the specific requirements and research questions of the DH in a more adaptive and user-centered way. We argue that both objectives need to be addressed, as this is, by all means, necessary for a successful implementation of computer vision tools in the DH and related fields.