Towards an Automatic Determination of Grasping Points through a Machine Vision Approach

In this study emotion-based face expression recognition framework has been proposed using a machine vision (MV) approach. The face emotion dataset has been collected local survey in Bahawalpur city dataset divide into 3 classes happy, sad, and angry. A total of 600 images of size (256 x 256) were transformed into a gray level format and employed a median filter for noise removal. Three non-overlapping regions of interest (ROIs) of size (50 x 50) have been taken and analyze 1800 (600 x 3) ROIs on the overall dataset. Total 45 Statistical features named as texture, histogram, and binary features were extracted. Select optimize features using the correlation-based feature section technique. The optimized dataset employed of MV classifiers namely random forest (RF), logistic (Lg), and J48 are obtained very promising accuracy 96.33%, 95.67%, and 95.33% respectively.

Vitreo-retinal surgery encompasses the surgical procedures performed on the vitreous humor and the retina. A procedure typically consists of the removal of the vitreous humor, the peeling of a membrane and/or the repair of a retinal detachment. Operations are performed with needle shaped instruments which enter the eye through surgeon made scleral openings. An instrument is moved by hand in four degrees of freedom (three rotations and one translation) through this opening. Two rotations (? and ? ) are for a lateral instrument tip movement. The other two DoFs (z and ?) are the translation and rotation along the instrument axis. Actuation of for example a forceps can be considered as a fifth DoF. Characteristically, the manipulation of delicate, micrometer range thick intraocular tissue is required. Today, eye surgery is performed with a maximum of two instruments simultaneously. The surgeon relies on visual feedback only, since instrument forces are below the human detection limit. A microscope provides the visual feedback. It forces the surgeon to work in a static and non ergonomic body posture. Although the surgeon’s proficiency improves throughout his career, hand tremor may become a problem around his mid-fifties. Robotically assisted surgery with a master-slave system enhances dexterity. The slave with instrument manipulators is placed over the eye. The surgeon controls the instrument manipulators via haptic interfaces at the master. The master and slave are connected by electronic hardware and control software. Implementation of tremor filtering in the control software and downscaling of the hand motion allow prolongation of the surgeon’s career. Furthermore, it becomes possible to do tasks like intraocular cannulation which can not be done by manually performed surgery. This thesis focusses on the master console. Eye surgery procedures are observed in the operating room of different hospitals to gain insight in the requirements for the master. The master console as designed has an adjustable frame, a 3D display and two haptic interfaces with a coarse adjustment arm each. The console is mounted at the head of the operating table and is combined with the slave. It is compact, easy to place and allows the surgeon to have a direct view on and a physical contact with the patient. Furthermore, it fits in today’s manual surgery arrangement. Each haptic interface has the same five degrees of freedom as the instrument inside the eye. Through these interfaces, the surgeon can feel the augmented instrument forces. Downscaling of the hand motion results in a more accurate instrument movement compared to manually performed surgery. Together with the visual feedback, it is like the surgeon grasps the instrument near the tip inside the eye. The similarity between hand motion and motion of the instrument tip as seen on the display results in an intuitive manipulation. Pre-adjustment of the interface is done via the coarse adjustment arm. Mode switching enables to control three or more instruments manipulators with only two interfaces. Two one degree of freedom master-slave systems with force feedback are built to derive the requirements for the haptic interface. Hardware in the loop testing provides valuable insights and shows the possibility of force feedback without the use of force sensors. Two five DoF haptic interfaces are realized for bimanual operation. Each DoF has a position encoder and a force feedback motor. A correct representation of the upscaled instrument forces is only possible if the disturbance forces are low. Actuators are therefore mounted to the fixed world or in the neighborhood of the pivoting point for a low contribution to the inertia. The use of direct drive for ' and and low geared, backdriveable transmissions for the other three DoFs gives a minimum of friction. Disturbance forces are further minimized by a proper cable layout and actuator-amplifier combinations without torque ripple. The similarity in DoFs between vitreo-retinal eye surgery and minimally invasive surgery (MIS) enables the system to be used for MIS as well. Experiments in combination with a slave robot for laparoscopic and thoracoscopic surgery show that an instrument can be manipulated in a comfortable and intuitive way. User experience of surgeons and others is utilized to improve the haptic interface further. A parallel instead of a serial actuation concept for the ' and DoFs reduces the inertia, eliminates the flexible cable connection between frame and motor and allows that the heat of the motor is transferred directly to the frame. A newly designed z-?? module combines the actuation and suspension of the hand held part of the interface and has a three times larger z range than in the first design of the haptic interface.

The two visual systems hypothesis in neuroscience suggests that pointing without visual feedback may be less affected by spatial visual illusions than cognitive interactions such as judged target location. Our study examined predictions of this theory for target localization on a large-screen display. We contrasted pointing interactions under varying levels of visual feedback with location judgments of targets that were surrounded by an offset frame. As predicted by the theory, the frame led to systematic errors in verbal report of target location but not in pointing without visual feedback for some participants. We also found that pointing with visual feedback produced a similar level of error as location judgments, while temporally lagged visual feedback appeared to reduce these errors somewhat. This suggests that pointing without visual feedback may be a useful interaction technique in situations described by the two visual systems literature, especially with large-screen displays and immersive environments.

The objective of this study was to observe the potential of machine vision (MV) approach for the classification of eight citrus varieties. The leaf images of eight citrus varieties that were grapefruit, Moussami, Malta, Lemon, Kinow, Local lemon, Fuetrells, and Malta Shakri. These were acquired by a digital camera in an open environment without any complex laboratory setup. The acquired digital images dataset was transformed into the multi-feature dataset that was the combination of binary, histogram, texture, spectral, rotational, scalability and translational (RST) invariant features. For each citrus leaf image, total 57 multi-features were acquired on every non-overlapping region of interest (ROI), i.e. (32x32), (64x64), (128x128), and (256x256). Furthermore, the optimized 15 features using the supervised correlation-based feature selection (CFS) technique were acquired. The optimized multi-features dataset to different MV classifiers namely Multilayer Perceptron (MLP), Random Forest (RF), J48 and Naïve Bayes using10-fold cross-validation method were plugged-in. The results produced by MLP presented an average overall accuracy of 98.14% on ROIs (256x256) outperforming the other classifiers. The classification accuracy values by MLP on the eight citrus leaf varieties, namely; Grapefruit, Moussami, Malta, Lemon, Kinow, Local lemon, Fuetrells, and Malta Shakri were observed 98%, 98.75%, 99.25%, 97.5%, 97%, 95.87%, 95.5%, and 99.37% respectively.

Typing in Japanese is a difficult process for novice and intermediate learners of Japanese dueto the writing system of the Japanese language and its comparatively involved input method on akeyboard. Considering that spell checkers, which enable the user to check and correct their ownerrors and select the correct kanji word, are designed for native speakers, the learners of Japaneseas a foreign language (JFL) may not recognize their spelling errors and are thus unable to selfcorrect using this built-in tool.The present study addresses this problem and conducts an experiment to evaluate theeffectiveness of a visual feedback tool by its error recognition rate on the learners spelling errorswhen typing in Japanese. The participants were 46 beginner level JFL learners in a third semesterJapanese course, and the majority consist of native speakers of Chinese or English. Theparticipants participated in two experimental sessions. In both sessions, participants were audiorecorded while reading aloud a list of words in Japanese for pronunciation analysis and screenrecorded while typing the same list of Japanese words. These recordings are used to analyze thecharacteristics of error patterns in both pronunciation and typing. During the typing sessions, visualfeedback is provided to the participants via a customized dictionary tool when participants makea spelling error.The results show that regardless of the native language, the learners have difficulty on certainwords that include long vowels or double consonants. The recorded error patterns align with thefindings of previous studies (Hatasa, 2001; Nakazawa, 2003; Tsuchiya, 2000), and the visualfeedback showed an average error recognition rate of 76% of the participants’ spelling errors. Theparticipants also assessed the dictionary tool in terms of usability, and their responses indicate thatsuch tools are very useful during typing. The researcher concludes that using a visual feedbackdictionary tool is effective in recognizing the spelling errors of the learners when typing, and itincreases the learner’s awareness of spelling accuracy.

The task of robotic grasping brings together several challenges. Among them, we focus on the calculus of where gripper plates should be placed over an object’s surface in order to grasp it. To do this, we have developed a method based on visual information. The main goal is to geometrically analyse a single 3D point cloud, where the object is present, to find the best pair of contact points so a gripper can perform a stable grasp of the object. Our proposal is to find these points near a perpendicular cutting plane to the object’s main axis through its centroid. We have found that this method shows promising experimental results fast and accurate enough to be used on real service robots.

We introduce a novel posture guidance office chair and evaluate the effectiveness of vibrotactile and visual feedback methods for guiding seated postures. For visually dominant office work such as typing on the computer, it is possible that delivering posture feedback visually may overload the visual sense while haptic feedback may be a viable alternative. We performed two experiments to compare vibrotactile and visual feedback--posture compliance and dual-task cognitive workload assessment. In the first experiment, our results showed no statistically significant difference in effectiveness between using vibrotactile and visual feedback to obtain postural compliance to a reference posture. In the second experiment, participants experienced typing performance and response time degradations from both types of feedback. However the differences in performance degradation were not statistically significant between the two feedback methods. We conclude that vibrotactile and visual feedback are similarly effective for guiding quasistatic postures in routine tasks such as seated office work.

A machine vision approach to on-line estimation of run-of-mine ore composition on conveyor belts

Bend gestures offer a rich interaction language for flexible displays due to their large degrees of freedom. However, without an understandable visual feedback system, working with such interaction could be a difficult challenge for users. In this thesis, we have developed three visual feedback systems by using feedforward and feedback mechanisms with the goal of guiding users to correctly perform bend gestures on a handheld flexible display. Additionally, we have conducted an experiment containing two tasks, which, using a flexible prototype, evaluate the usefulness of our visual feedback designs. Our results suggested that users could perform sets of bend gestures faster when the visual feedbacks appear at the exact location of the bend gesture instead of always at a fixed location on the screen. Also, we found that, while feedforward improved users' performance, using feedback had a negative effect. Finally, we propose a set of design guidelines for designing visual feedback systems for bend gestures on flexible devices.

BackgroundThe benefit of visual feedback of the center of pressure (COP) on quiet standing is still debatable. This study aimed to investigate the adaptation effects of visual feedback training using both the COP and center of gravity (COG) during quiet standing.MethodsThirty-four healthy young adults were divided into three groups randomly (COP + COG, COP, and control groups). A force plate was used to calculate the coordinates of the COP in the anteroposterior (COPAP) and mediolateral (COPML) directions. A motion analysis system was used to calculate the coordinates of the center of mass (COM) in both directions (COMAP and COMML). The coordinates of the COG in the AP direction (COGAP) were obtained from the force plate signals. Augmented visual feedback was presented on a screen in the form of fluctuation circles in the vertical direction that moved upward as the COPAP and/or COGAP moved forward and vice versa. The COP + COG group received the real-time COPAP and COGAP feedback simultaneously, whereas the COP group received the real-time COPAP feedback only. The control group received no visual feedback. In the training session, the COP + COG group was required to maintain an even distance between the COPAP and COGAP and reduce the COGAP fluctuation, whereas the COP group was required to reduce the COPAP fluctuation while standing on a foam pad. In test sessions, participants were instructed to keep their standing posture as quiet as possible on the foam pad before (pre-session) and after (post-session) the training sessions.ResultsIn the post-session, the velocity and root mean square of COMAP in the COP + COG group were lower than those in the control group. In addition, the absolute value of the sum of the COP − COM distances in the COP + COG group was lower than that in the COP group. Furthermore, positive correlations were found between the COMAP velocity and COP − COM parameters.ConclusionsThe results suggest that the novel visual feedback training that incorporates the COPAP–COGAP interaction reduces postural sway better than the training using the COPAP alone during quiet standing. That is, even COPAP fluctuation around the COGAP would be effective in reducing the COMAP velocity.

Interactive rendering requires rapid visual feedback. The render cache is a new method for achieving this when using high-quality pixel-oriented renderers such as ray tracing that are usually considered too slow for interactive use. The render cache provides visual feedback at a rate faster than the renderer can generate complete frames, at the cost of producing approximate images during camera and object motion. The method works both by caching previous results and reprojecting them to estimate the current image and by directing the renderer's sampling to more rapidly improve subsequent images. Our implementation demonstrates an interactive application working with both ray tracing and path tracing renderers in situations where they would normally be considered too expensive. Moreover we accomplish this using a software only implementation without the use of 3D graphics hardware.

Objective To explore the effect of ankle strategy stability limit training on balance and gait in recovering stroke patients with hemiplegia. Methods Forty recovering stroke patients were randomized into an intervention group and a control group.The patients in the intervention group were given ankle strategy stability limit training using visual feedback on the static long sets of a Smart Equitest Balance Master (SEBM) machine.Those in the control group practiced routine postural balance training using mirror visual feedback in parallel bars.Both groups of patients practiced balance and posture control for 30 minutes,once daily,6 days a week for two weeks. Both groups were also given routine therapy and other rehabilitation.The patients' balance function was evaluated using the Berg Balance Scale (BBS),and their gait was assessed using the walk across technique (WA). Results There was no significant difference between the two groups with regard to general information,BBS scores or WA results before treatment.After 2 weeks of treatment,BBS scores as well as the step length and pace in the WA improved significantly in both groups,but all improved significantly more in the intervention group.There was no significant difference in width of gait. Conclusion Ankle strategy stability limit training can enhance weight-bearing on stroke patients' affected foot as well as their balance and the symmetry of their steps. Key words: Ankle joint; Stroke; Musculoskeletal equilibrium; Gait; Ankle strategy; Stability limits

PurposeHaptic feedback could improve the efficiency of needle insertion procedures by providing surgeons with enhanced sensing and guiding capabilities. A framework has been developed to provide physicians with enhanced haptic feedback during CT-guided needle insertion procedures in oncology.MethodsThe physicians encountered needle-tissue interaction and guidance forces with visual feedback to accurately reach the tumor. The force feedback to users was enhanced by amplifying several parameters in the feedback model, such as tip forces and radial forces. The study evaluated the effect of multiple haptic and visual feedback algorithms on user performance in efficiently inserting the needle. In this experimental pilot study, 12 participants including three interventional radiologists engaged in five experimental scenarios simulating a needle insertion.ResultsThe results showed that enhanced force feedback for tumor perception reduced tumor targeting error and trajectory deviation, compared to natural force feedback. This was also the case when tumor perception and haptic guidance were both enhanced. Additionally, real-time visual feedback and enhanced force feedback for guidance reduced the duration to finish the task significantly. Participants still preferred real-time visual feedback over enhanced haptic feedback cues.ConclusionsOn average, small tumors (around 3mm in diameter) can be successfully targeted with enhanced haptic feedback in the radial and axial directions. Additionally, critical regions, such as veins within the liver, can be avoided more effectively as users maintain the desired trajectory with greater accuracy.

In the past, roughness values measured directly on machined surfaces were used to develop mathematical models that are used in predicting surface roughness in turning. This approach is slow and tedious because of the large number of workpieces required to obtain the roughness data. In this study, 2-D images of cutting tools were used to generate simulated workpieces from which surface roughness and dimensional deviation data were determined. Compared to existing vision-based methods that use features extracted from a real workpiece to represent roughness parameters, in the proposed method, only simulated profiles of the workpiece are needed to obtain the roughness data. The average surface roughness Ra, as well as dimensional deviation data extracted from the simulated profiles for various feed rates, depths of cut, and cutting speeds were used as the output of response surface methodology (RSM) models. The predictions of the models were verified experimentally using data obtained from measurements made on the real workpieces using conventional methods, i.e., surface roughness tester and a micrometer, and good correlation between the two methods was observed.

PurposeThe purpose of this paper is to describe innovative machine vision methods that have been employed for the capture and analysis of 3D skin textures; and the resulting potential for assisting with identification of suspicious lesions in the detection of skin cancer.Design/methodology/approachA machine vision approach has been employed for analysis of 3D skin textures. This involves an innovative application of photometric stereo for the capture of the textures, and a range of methods for analysing and quantifying them, including statistical methods and neural networks.Findings3D skin texture has been identified as a useful indicator of skin cancer. It can be used to improve realism of virtual skin reconstructions in tele‐dermatology. 3D texture features can also be combined with 2D features to obtain a more robust classifier for improving diagnostic accuracy, thereby assisting with the long‐term goal of implementing computer‐aided diagnostics for skin cancer.Originality/valueThe device developed for capturing 3D skin textures is known as the “Skin Analyser”, and as far as the authors know it is unique in the world in being able to recover 3D textures from pigmented lesions in vivo. There currently exist numerous methods for analysing lesions, including manual inspection (using established heuristics commonly known as ABCD rules), dermoscopy and SIAoscopy. The ability to capture and analyse 3D lesion textures complements these existing techniques and forms a valuable additional indicator for assisting with the early detection of dangerous skin cancers such as melanoma.