Emotion Recognition from Facial Expression Using Machine Vision Approach

The objective of this study was to observe the potential of machine vision (MV) approach for the classification of eight citrus varieties. The leaf images of eight citrus varieties that were grapefruit, Moussami, Malta, Lemon, Kinow, Local lemon, Fuetrells, and Malta Shakri. These were acquired by a digital camera in an open environment without any complex laboratory setup. The acquired digital images dataset was transformed into the multi-feature dataset that was the combination of binary, histogram, texture, spectral, rotational, scalability and translational (RST) invariant features. For each citrus leaf image, total 57 multi-features were acquired on every non-overlapping region of interest (ROI), i.e. (32x32), (64x64), (128x128), and (256x256). Furthermore, the optimized 15 features using the supervised correlation-based feature selection (CFS) technique were acquired. The optimized multi-features dataset to different MV classifiers namely Multilayer Perceptron (MLP), Random Forest (RF), J48 and Naïve Bayes using10-fold cross-validation method were plugged-in. The results produced by MLP presented an average overall accuracy of 98.14% on ROIs (256x256) outperforming the other classifiers. The classification accuracy values by MLP on the eight citrus leaf varieties, namely; Grapefruit, Moussami, Malta, Lemon, Kinow, Local lemon, Fuetrells, and Malta Shakri were observed 98%, 98.75%, 99.25%, 97.5%, 97%, 95.87%, 95.5%, and 99.37% respectively.

PurposeThe purpose of this paper is to describe innovative machine vision methods that have been employed for the capture and analysis of 3D skin textures; and the resulting potential for assisting with identification of suspicious lesions in the detection of skin cancer.Design/methodology/approachA machine vision approach has been employed for analysis of 3D skin textures. This involves an innovative application of photometric stereo for the capture of the textures, and a range of methods for analysing and quantifying them, including statistical methods and neural networks.Findings3D skin texture has been identified as a useful indicator of skin cancer. It can be used to improve realism of virtual skin reconstructions in tele‐dermatology. 3D texture features can also be combined with 2D features to obtain a more robust classifier for improving diagnostic accuracy, thereby assisting with the long‐term goal of implementing computer‐aided diagnostics for skin cancer.Originality/valueThe device developed for capturing 3D skin textures is known as the “Skin Analyser”, and as far as the authors know it is unique in the world in being able to recover 3D textures from pigmented lesions in vivo. There currently exist numerous methods for analysing lesions, including manual inspection (using established heuristics commonly known as ABCD rules), dermoscopy and SIAoscopy. The ability to capture and analyse 3D lesion textures complements these existing techniques and forms a valuable additional indicator for assisting with the early detection of dangerous skin cancers such as melanoma.

Reliable and accurate measurement of particle size and subsequent analysis of particle size distribution (PSD) is central to characterization of particulate minerals. Traditionally PSD of particulate materials was determined using standard mechanical sieving, but it was shown to be deficient in accurate particle size measurements, hence PSD analysis. In this paper a highly inexpensive machine vision based approach was proposed as an alternative to mechanical sieving. The test mineral used was ball-milled celestite. This machine vision approach used a document scanner as imaging device and a user-coded java ImageJ plugin performed the image processing and automated the PSD analysis. The acquired color images were preprocessed to binary images and the particles analyzed after grouping them based on their distinct length. Volumes of all particles were evaluated assuming a prolate spheroid geometry from measured lengths and widths. A new approach of using sum of volumes as weighting factor (Svolume) was utilized for particle grouping in ASABE Standard PSD analysis. The plugin also evaluated several significant dimensional (16) and distributional (27) parameters that characterize the PSD of samples. Standard mechanical sieving of samples was performed for experimental verification and results compared with machine vision method. The cumulative undersize PSD followed a log-normal distribution, and the plot against particle size exhibited a linear trend. Shapes of log-normal plot of cumulative undersize PSD were similar between both methods; however, the mechanical sieving curves lagged by 0.2-0.5 mm. The deviation was attributed to the falling-through effect of longer particles through sieve openings.

Currently available driver assistance systems (i) warn the driver based on vehicle state sensors (e.g., door open, outside temperature near or below the freezing point), (ii) offer route guidance information (navigation systems based on GPS and digital road maps), or—in some critical situations—(iii) even actively influence vehicle handling under carefully delimited conditions (anti-blocking-system, electronic-stability-program). This contribution reports about investigations to combine passive GPS- and map-based route guidance with model-based machine vision in order to automatically assess or even execute driving maneuvers in inner-city traffic situations. Information provided by todays route guidance systems is treated as a generic description of lane structure. The schematic description of lane structures extractable from commercially available standard digital maps is automatically instantiated by a machine vision approach which interprets video image sequences recorded by cameras from within a driving vehicle. The resulting model of the lane structure in front of the vehicle is subsequently exploited in order to control vehicle maneuvers in real-time as a proof of principal system competences. The machine-vision-based execution of driving maneuvers can at any time be overridden by the driver.

Functional mapping of dynamic happy and fearful facial expressions in young adults with familial risk for psychosis — Oulu Brain and Mind Study

A machine vision approach to on-line estimation of run-of-mine ore composition on conveyor belts

Cumulative trauma disorders (CTDs) are a leading cause of disability which generates huge direct and indirect costs in the construction industry. Existing efforts identify the risk of CTDs through survey or observatory studies which are subjective and expensive. This paper attempts to automatically identify the risk of CTDs on construction sites with the aid of a video surveillance system. Specific emphasis is placed on low back disorders (LBDs) which is one of the major work-related CTDs. To this end, a risk identification method is proposed based on location changes of region of interests in the resulting video streams at minimum supervision. It is used to detect repetitive trunk flexion-extension motions, awkward trunk postures, and assess risk of job-related LBDs. Preliminary experimental analysis on lab datasets demonstrates the promise of the proposed method.

Facial expression is one of the most powerful and direct mediums embedded in human beings to communicate with other individuals’ feelings and abilities. In recent years, many surveys have been carried on facial expression analysis. With developments in machine vision and artificial intelligence, facial expression recognition is considered a key technique of the developments in computer interaction of mankind and is applied in the natural interaction between human and computer, machine vision and psycho- medical therapy. In this paper, we have developed a new method to recognize facial expressions based on discovering differences of facial expressions, and consequently appointed a unique pattern to each single expression.by analyzing the image by means of a neighboring window on it, this recognition system is locally estimated. The features are extracted as binary local features; and according to changes in points of windows, facial points get a directional motion per each facial expression. Using pointy motion of all facial expressions and stablishing a ranking system, we delete additional motion points that decrease and increase, respectively, the ranking size and strenghth. Classification is provided according to the nearest neighbor. In the conclusion of the paper, the results obtained from the experiments on tatal data of Cohn-Kanade demonstrate that our proposed algorithm, compared to previous methods (hierarchical algorithm combined with several features and morphological methods as well as geometrical algorithms), has a better performance and higher reliability.

Visual scanning of the eye region of human faces predicts emotion recognition performance in Huntington's disease.

Patients with amygdala lesions were found to be impaired in recognizing the fear emotion both from face and from music. In patients with Parkinson's disease (PD), impairment in recognition of emotions from facial expressions was reported for disgust, fear, sadness and anger, but no studies had yet investigated this population for the recognition of emotions from both face and music. The ability to recognize basic universal emotions (fear, happiness and sadness) from both face and music was investigated in 24 medicated patients with PD and 24 healthy controls. The patient group was tested for language (verbal fluency tasks), memory (digit and spatial span), executive functions (Similarities and Picture Completion subtests of the WAIS III, Brixton and Stroop tests), visual attention (Bells test), and fulfilled self-assessment tests for anxiety and depression. Results showed that the PD group was significantly impaired for recognition of both fear and sadness emotions from facial expressions, whereas their performance in recognition of emotions from musical excerpts was not different from that of the control group. The scores of fear and sadness recognition from faces were neither correlated to scores in tests for executive and cognitive functions, nor to scores in self-assessment scales. We attributed the observed dissociation to the modality (visual vs. auditory) of presentation and to the ecological value of the musical stimuli that we used. We discuss the relevance of our findings for the care of patients with PD.

In the past, roughness values measured directly on machined surfaces were used to develop mathematical models that are used in predicting surface roughness in turning. This approach is slow and tedious because of the large number of workpieces required to obtain the roughness data. In this study, 2-D images of cutting tools were used to generate simulated workpieces from which surface roughness and dimensional deviation data were determined. Compared to existing vision-based methods that use features extracted from a real workpiece to represent roughness parameters, in the proposed method, only simulated profiles of the workpiece are needed to obtain the roughness data. The average surface roughness Ra, as well as dimensional deviation data extracted from the simulated profiles for various feed rates, depths of cut, and cutting speeds were used as the output of response surface methodology (RSM) models. The predictions of the models were verified experimentally using data obtained from measurements made on the real workpieces using conventional methods, i.e., surface roughness tester and a micrometer, and good correlation between the two methods was observed.

The paper presents a machine-vision-based system for automated inspection of standard spur gears. Image processing algorithms are used for the measurement of important gear dimensions such as radii of addendum circle, dedendum circle and pitch circle, module, number of teeth, pressure angle, tooth thickness, circular pitch, radial run-out and tooth alignment error. Deviations from theoretical values according to gear standards are computed and a decision is made regarding acceptance/rejection. The performance of machine vision inspection system is evaluated in terms of its accuracy and precision. Accuracy is based on deviation of machine vision values from those obtained using traditional metrology instruments and gear standards. Precision is measured using partial gauge R&R study. The results obtained for gear images taken by different operators using different imaging devices are repeatable, reproducible and in good agreement with the true values. The results indicate that the machine vision approach is accurate and precise.

A system is presented which coordinates a parallel-jaw gripper and a vision system with the aim of choosing grasping points taking into account stability conditions. The choice of grasping points is done by a new heuristic approach that tries to manage all kind of planar parts with or without internal centre of gravity, possibly containing holes. It needs only two moment-based features, centroid and the direction of the main axis, easily obtained from a Freeman chain-code representation of the boundary. The results are exclusively visual parameters, expressed in visual coordinates (pixels). These algorithms have been implemented in a 4D0F robot arm with one camera fixed over the hand. Visual feedback was used in the control system to improve precision up to the limits of the available visual system.

Humans have traditionally found it simple to identify emotions from facial expressions, but it is far more difficult for a computer system to do the same. The social signal processing subfield of emotion recognition from facial expression is used in a wide range of contexts, particularly for human-computer interaction. Automatic emotion recognition has been the subject of numerous studies, most of which use a machine learning methodology. The recognition of simple emotions like anger, happiness, contempt, fear, sadness, and surprise, however, continues to be a difficult topic in computer vision. Deep learning has recently drawn increased attention as a solution to a variety of practical issues, including emotion recognition. In this study, we improved the convolutional neural network technique to identify 7 fundamental emotions and evaluated several preprocessing techniques to demonstrate how they affected the CNN performance. This research focuses on improving facial features and expressions based on emotional recognition. By identifying or recognising facial expressions that elicit human responses, it is possible for computers to make more accurate predictions about a person's mental state and to provide more tailored responses. As a result, we examine how a deep learning technique that employs a convolutional neural network might improve the detection of emotions based on facial features (CNN). Multiple facial expressions are included in our dataset, which consists of about 32,298 photos for testing and training. The preprocessing system aids in removing noise from the input image, and the pretraining phase aids in revealing face detection after noise removal, including feature extraction. As a result, the existing paper generates the classification of multiple facial reactions like the seven emotions of the facial acting coding system (FACS) without using the optimization technique, but our proposed paper reveals the same seven emotions of the facial acting coding system.

IntroductionRecent developments in the field of social cognition have led to a renewed interest in basic and social emotion recognition in early stages of Alzheimer's Disease (AD) and FrontoTemporal Dementia (FTD). Despite the growing attention to this issue, only few studies have attempted to investigate emotion recognition using both visual and vocal stimuli. In addition, recent studies have presented conflicting findings regarding the extent of impairment in patients in the early stages of these diseases. The present study aims to investigate emotion understanding (both basic and social emotions), using different tasks with visual and auditory stimuli, to identify supramodal deficits in AD and FTD to provide a reliable tool to better outline their behavioral and emotional profile and useful instruments for their management.MethodsEighteen patients with AD and 15 patients with FTD were included in the study. Healthy control (HCs) subjects were recruited to obtain normative data for basic emotion recognition tests and social emotion recognition tasks. To evaluate basic emotion recognition, the Facial Emotion Recognition Battery (FERB) and the Emotional Prosody Recognition Battery (EPRB) were administered. To evaluate social emotion recognition, the Faux Pas (FP), Reading the Mind in the Eyes (RME), and Reading the Mind in the Voice (RMV) tests were employed.ResultsFTD patients performed significantly worse than HCs in most of the subtests of the basic emotion recognition batteries, where, instead, AD patients were significantly impaired only when required to match emotional facial expression in different individuals (subtask of the FERB). Moreover, FTD patients scored significantly lower in RME and RMV tests compared both to AD patients and to HCs. In addition, ADs were selectively impaired in RMV as respect to HCs.DiscussionFTD patients showed deficits in emotion recognition, affecting both basic and social emotions, whether conveyed through facial expressions or prosody. This result may explain the well-known social behavioral difficulties observed in FTD patients from the early stages of the disease. The fewer and specific deficits in AD patients with comparable MMSE scores may be attributed to the mild degree of impairment, as these deficits may appear later in the progression of AD.