Real Time Vision Based Method for Finger Counting Through Shape Analysis with Convex Hull and PCA Techniques

This paper presents a real time method to estimate the number of fingers observed in a video. The method tracks the fingertips and exploits the shape of the hand contour to determine the number of fingers observed in a sequence of images. The first step of the proposed method is to detect the hand observed in the input image by segmentation into foreground and background areas using skin colour detection method. The foreground corresponds to the area representing the hand to be tracked. Due to the problem of the lighting variation, HSL colour space was used to represent the colour. The second step consists of computing the hand contour. Then a convex Hull and convexity defects are calculated to detect the fingertips. Principal components analysis (PCA) [13] method is applied on the convex hull to deal with the cases in which only one finger is observed in the image or when the hand is closed. The proposed method could be used to produce different Human Computer Interaction systems (HCI). Experimental results obtained from real images demonstrate the potential of the method.

Hand is an important part of human in communicating with other persons and interacting with objects in the world. In this thesis, we will explore hand analysis, which includes hand detection, hand posture and hand shape analysis.
\n
\nFirst I will focus on hand detection. We start by exploring pixel-level hand detection. It is a challenging problem because the appearance of hands can vary greatly in images. We use a random forest framework in order to predict hand mask and hand part labels in an efficient and robust manner. Our approach exploits the inherent contextual information from structured hand labelling for pixel-level hand detection and hand part labelling. Having obtained a pixel-level hand probability map, we present an end-to-end system to detect hands in a bounding box fashion. Our model can not only detect and localize the hands but also provide detailed information within the bounding box by using a multi-task convolutional neural network. Experimental results show that hand detection can be improved by utilizing more information such as hand shape and hand landmark points than only using bounding boxes.
\n
\nIn the second part of my thesis, we will concentrate on two specific problems based on the good hand detection results. The first problem is to recognize the hand postures efficiently. We present a flexible method for hand posture recognition by fusing information from color and depth images using kernel descriptors. Existing methods usually focus on designing intuitive features for color and depth images. On the contrary, our method extracts common patch-level features, and combines them by means of kernel descriptors. Linear SVM is used to predict the class label efficiently.
\nThe second problem is related to hand shape. We present an interesting application of shape analysis to train a hand shape recognizer automatically from simple sketches, such as a “stick-figure” of a hand shape. We introduce the Hand Boltzmann Machine, a generative model built upon unsupervised learning, to represent the hand shape space of a binary image, and formulate the user provided sketches as an initial guidance for sampling to generate realistic hand shape samples. A hand shape recognizer is trained from these samples and used to classify the hand shapes intuitively.

In the past decade, people's interest in computer vision has grown. Stable multiplication rate boost with powerful computing power every 13 months, face detection and recognition has changed from esoteric concept to popular One of the better and successful applications of computer vision research and image analysis and image analysis Algorithm-based understanding. In our work, we will be detecting contours (shapes) of various geometrical figures in the sample given binary mask extracted from HSV (Hue Saturation Value) range using Python 3.6, Open-Source Computer Vision Library (OpenCV 3x) and Numpy. The very fundamental functions are used for processing the frames, that involves getting live video feed, loading them frame-by-frame, and detecting various shape of red colors inside the live feed frames and segment them with previously saved image with the exact same location. In this paper we discuss three primary problems of computer vision: Finding contours, detection of specified color and segment with another image. We tried state-of-the-art algorithms for these three tasks. HSV color extraction is used to get the mask of desire colored object. In order to get high quality image segmentations results, we applied Morphological operations to refine the mask such as, Opening which removes unnecessary details which in our case are the white region on the boundary of the cloth that is not required, Erosion, Dilatation operation and Gaussian Blurring to smoothen the result. We link this contour detector with Convex Hull, this strategy performs better than our existing image segmentation algorithms on measures of both boundary and segmentation quality. Finally, we replaced pixel value of red color region detected by HSV mask with the pixel value of our background image with the help of bitwise operation to get the augmented result of invisible cloak.

Estimating the correspondence between the images using optical flow is the key component for image fusion, however, computing optical flow between a pair of facial images including backgrounds is challenging due to large differences in illumination, texture, color and background in the images. To improve optical flow results for image fusion, we propose a novel flow estimation method, wavelet flow, which can handle both the face and background in the input images. The key idea is that instead of computing flow directly between the input image pair, we estimate the image flow by incorporating multi‐scale image transfer and optical flow guided wavelet fusion. Multi‐scale image transfer helps to preserve the background and lighting detail of input, while optical flow guided wavelet fusion produces a series of intermediate images for further fusion quality optimizing. Our approach can significantly improve the performance of the optical flow algorithm and provide more natural fusion results for both faces and backgrounds in the images. We evaluate our method on a variety of datasets to show its high outperformance.

Deep learning has become the main way of the object detection task for autonomous vehicles. Meanwhile, this method typically requires vast amounts of training data to reach their full potential. However, collecting the data from real world and labeling manually is an expensive, time-consuming and error-prone process. Synthetic image has the potential to replace real image for training neural networks, because image creation and labeling annotations are free in this way. For the network trained by synthetic images, the reality gap between real and synthetic images is the main obstacle to use it in the real world. And most previous works are only devoted to generate synthetic images with a good performance on model training, but lack of analysis of the domain gap that affects the performance. This work designs a method of generating the real and synthetic images to analyze and reduce the reality gap between synthetic and real images for car detection. Firstly, this work put one single car with no-background in a random background image to generate real single car and synthetic single car images. In order to further reduce the domain gap in content level, this method keeps the car distribution in synthetic images is similar with the distribution of car in real world. For the purpose of reducing the domain gap in appearance level, the parameters of camera model are same as the camera parameters of image collecting cars and the image is rendered by using the PBRT(Physically Based Ray Tracing) when we generated the synthetic images. Secondly, by training the neural network of instance segmentation with different datasets, the across validation result proves that the reality gap between synthetic and real images is no more than the domain gap between real images. Thirdly, the training results of datasets with different samples diversity show that the diversity of the samples yields better generalization between different datasets for car detection which can effectively reduce the domain gap.

Current visualization techniques for mapping polarization data to a color coordinates defined by the Hue, Saturation, Value (HSV) color representation are analyzed in the context of perceptual uniformity. Since HSV is not designed to be perceptually uniform, the extent of non-uniformity should be evaluated by using robust color difference formulae and by comparison to the state-of-the-art uniform color space CAM02-UCS. For mapping just angle of polarization with HSV hue, the results show clear non-uniformity and implications for how this can misrepresent the data. UCS can be used to create alternative mapping techniques that are perceptually uniform. Implementing variation in lightness may increase shape discrimination within the scene. Future work will be dedicated to measuring performance of both current and proposed methods using psychophysical analysis.

Multiple images taken from different imaging devices are required to be combined together to obtain a single fused image that provides an enhanced representation of an observed scene. The images obtained from imaging devices having different modalities usually provide complementary information. For instance, visible and infrared (IR) images provide complementary information of an object or a scene. Visible images provide the outline of an object or a scene whereas infrared images provide information of some special effects such as hidden objects or persons. Further, color image processing is also gaining importance in most of the real-life applications as color images provide better visual information and are highly suitable for human visual perception. Thus, fusion of visible and infrared images in HSV color space is proposed in the paper. Visual evaluations and objective comparisons on different datasets of visible and infrared images show that the proposed algorithm can effectively combine the information contained in input images and therefore can be successfully utilized in surveillance, military and many other real-life applications.

본 논문은 조명의 변화가 심한 영상에서 손 형태를 안정적으로 인지하는 기법에 관한 것이다. 제안한 방법은 HSI 색상공간에서 색상(Hue) 및 색상 기울기(Hue-Gradient)를 기반으로 정의된 배경모델을 구축하고, 실시간으로 입력되는 영상과의 배경차분(background subtraction)기법을 이용하여 배경과 손을 구분한다. 추출된 손의 영역으로부터 18가지의 특징요소를 추출하고 이를 기반으로 다중클래스 SVM(Support Vector Machine) 학습 기법을 사용하여 손의 형태를 인지한다. 제안 기법은 색상 기울기를 배경 차분에 적용함으로써, 조명 환경이 배경 모델의 조명과 다르게 급격한 변화가 이루어졌을 때도 안정적으로 손의 윤곽정보를 추출할 수 있도록 하였다. 또한, 실시간 처리를 저해하는 복잡한 손의 특성정보 대신, OBB의 크기에 대하여 정규화된 두 개의 고유값과 객체 기반 바운딩 박스(OBB)를 구성하는 16개 세부 영역에서의 손 윤곽픽셀의 개수를 손의 특성정보로 사용하였다. 본 논문에서는 급격한 조명 변화 상황에서 기존 RGB 색상요소를 기반으로 하는 배경차분법과 색상을 기반으로 하는 배경차분법, 본 논문에서 제안하는 색상 기울기 기반 배경 차분법의 결과를 비교함으로써 제안 기법의 안정성을 입증하였다. 6명의 실험대상자의 1부터 9까지의 수지화 2700개의 영상으로부터 손 특성 정보를 추출하고 이에 대하여 훈련을 통한 학습 모델을 생성하였다. 학습모델을 기반으로 실험자 6인의 손 형태 1620개의 데이터에 대하여 인지 실험을 실시하여 92.6%에 이르는 손 형태 인식 성공률을 얻었다. In this paper, we present a robust hand recognition approach to sudden illumination changes. The proposed approach constructs a background model with respect to hue and hue gradient in HSI color space and extracts a foreground hand region from an input image using the background subtraction method. Eighteen features are defined for a hand pose and multi-class SVM(Support Vector Machine) approach is applied to learn and classify hand poses based on eighteen features. The proposed approach robustly extracts the contour of a hand with variations in illumination by applying the hue gradient into the background subtraction. A hand pose is defined by two Eigen values which are normalized by the size of OBB(Object-Oriented Bounding Box), and sixteen feature values which represent the number of hand contour points included in each subrange of OBB. We compared the RGB-based background subtraction, hue-based background subtraction and the proposed approach with sudden illumination changes and proved the robustness of the proposed approach. In the experiment, we built a hand pose training model from 2,700 sample hand images of six subjects which represent nine numerical numbers from one to nine. Our implementation result shows 92.6% of successful recognition rate for 1,620 hand images with various lighting condition using the training model.

A siamese-based verification system for open-set architecture attribution of synthetic images

Biometrics is a standalone scientific discipline which enjoys more and more attention of many researchers. The provision of the general security plays a key role in many modern branches. In the presented paper a person identification task is solved using the shape of a human hand, and also the hand contour classification algorithm based on an evolutionary estimator is also. The proposed methodology provides the comparison of the identified person with a set of model contours. The examination of the proposed method was performed with use of a database which contains 940 images of the scanned hands from 94 persons, including 10 images from every person. Totally 88360 combinations of the input images. The proposed evolutionary estimator uses an EPSDE algorithm, which is derived from a differential evolution algorithm which was proposed at the end of the 90’s. The model of the hand contour of every person is represented by only one image, which has movable finger contours in the classification process regarding the knuckle positions of the hand. Thanks to that, it is not necessary to use the pegs to hold the individual fingers in correct positions. The hand can be both placed on a support desk or can be freely hung in the air. All results obtained at classification time with use of the presented evolutionary estimator provide accuracy of approximately 98%.

In order to eliminate the complex background of pedestrian images in the re-identification task, this paper introduces grab-cut and co-segmentation algorithms to eliminate the complex background for the different pedestrian datasets. Specifically, we use grab-cut to handle small person dataset. For the bigger dataset, the designed co-segmentation algorithm which based on HSV color space is applied. Experimental results show that the proposed methods have a good effect.

Smart CCTV (Closed-Circuit Television) technology has increasingly been developed in the last few years to judge the situation and notify the administrator or take immediate action for security and surveillance reasons. Currently the methods to detect object motion typically include the Frame Difference Method (FDM) which can detect moving objects and the Background Subtraction Method (BSM) which is able to detect motionless objects. Those results can be obtained only if there were some background images ready in advance. The Adaptive Background Subtraction Method (ABSM) also could not recognize an object very well if there are rapid scene changes or an object does not move relatively for a long time. To resolve such a problem, in this research, a filmed image has been divided into the regular sized blocks and then, only the necessary parts of the previous frame image are updated in real-time and a background image was generated so that it is insensible to surrounding environment changes such as object motion, noise or light variations. We proposed a novel moving object detection method which showed high performance with regard to the MSE (Mean Squared Error) and the accuracy of detecting the moving object contours compared to other existing methods. We also evaluated quantitatively the detectability for a moving object region by quickly creating a background image even if it is difficult to shoot a background image or we do not have the baseline image prepared in advance. The proposed method could be used for cases that any background image does not exist or hard to be generated. It is also good for observation of many places at the same time with only a single CCTV system since it is especially robust to abrupt scene changes.

A real-time hand detection and tracking system is developed in this work in which the gestures are recognized by counting the number of fingers. More than fifteen kinds of hand gestures are recognized in this system and these gestures are then mapped with the keyboard keys. For background Subtraction we used Grayscale color space followed by. contour of hand, which is developed using Canny’s Algorithm to which we applied Chan’s algorithm to obtain the convex hull. From the defect points of Convex Hull, we obtained the center point of the hand. Using this, we counted the fingers and thereby recognizing the gestures. We performed several trials to validate the feasibility and applicability of the proposed system, and confirmed the accuracy of 93.20%.

The main steps of image analysis are image capturing, image storage (compression), correcting imaging defects (e.g. non-uniform illumination, electronic-noise, glare effect), image enhancement, segmentation of objects in the image and image measurements. Digitisation is made by a camera. The most modern types include a frame-grabber, converting the analog-to-digital signal into digital (numerical) information. The numerical information consists of the grey values describing the brightness of every point within the image, named a pixel. The information is stored in bits. Eight bits are summarised in one byte. Therefore, grey values can have a value between 0 and 256 (2(8)). The human eye seems to be quite content with a display of 5-bit images (corresponding to 64 different grey values). In a digitised image, the pixel grey values can vary within regions that are uniform in the original scene: the image is noisy. The noise is mainly manifested in the background of the image. For an optimal discrimination between different objects or features in an image, uniformity of illumination in the whole image is required. These defects can be minimised by shading correction [subtraction of a background (white) image from the original image, pixel per pixel, or division of the original image by the background image]. The brightness of an image represented by its grey values can be analysed for every single pixel or for a group of pixels. The most frequently used pixel-based image descriptors are optical density, integrated optical density, the histogram of the grey values, mean grey value and entropy. The distribution of the grey values existing within an image is one of the most important characteristics of the image. However, the histogram gives no information about the texture of the image. The simplest way to improve the contrast of an image is to expand the brightness scale by spreading the histogram out to the full available range. Rules for transforming the grey value histogram of an existing image (input image) into a new grey value histogram (output image) are most quickly handled by a look-up table (LUT). The histogram of an image can be influenced by gain, offset and gamma of the camera. Gain defines the voltage range, offset defines the reference voltage and gamma the slope of the regression line between the light intensity and the voltage of the camera. A very important descriptor of neighbourhood relations in an image is the co-occurrence matrix. The distance between the pixels (original pixel and its neighbouring pixel) can influence the various parameters calculated from the co-occurrence matrix. The main goals of image enhancement are elimination of surface roughness in an image (smoothing), correction of defects (e.g. noise), extraction of edges, identification of points, strengthening texture elements and improving contrast. In enhancement, two types of operations can be distinguished: pixel-based (point operations) and neighbourhood-based (matrix operations). The most important pixel-based operations are linear stretching of grey values, application of pre-stored LUTs and histogram equalisation. The neighbourhood-based operations work with so-called filters. These are organising elements with an original or initial point in their centre. Filters can be used to accentuate or to suppress specific structures within the image. Filters can work either in the spatial or in the frequency domain. The method used for analysing alterations of grey value intensities in the frequency domain is the Hartley transform. Filter operations in the spatial domain can be based on averaging or ranking the grey values occurring in the organising element. The most important filters, which are usually applied, are the Gaussian filter and the Laplace filter (both averaging filters), and the median filter, the top hat filter and the range operator (all ranking filters). Segmentation of objects is traditionally based on threshold grey values. (AB

Hybrid leader corona virus herd optimizer with multilevel thresholding techniques for foreground and background image segmentation