On discovering relevant features for tongue colored image analysis

Background Research on intelligent tongue diagnosis is a main direction in the modernization of tongue diagnosis technology. Identification of tongue shape and texture features is a difficult task for tongue diagnosis in traditional Chinese medicine (TCM). This study aimed to explore the application of deep learning techniques in tongue image analyses. Methods A total of 8676 tongue images were annotated by clinical experts, into seven categories, including the fissured tongue, tooth-marked tongue, stasis tongue, spotted tongue, greasy coating, peeled coating, and rotten coating. Based on the labeled tongue images, the deep learning model faster region-based convolutional neural networks (Faster R-CNN) was utilized to classify tongue images. Four performance indices, i.e., accuracy, recall, precision, and F1-score, were selected to evaluate the model. Also, we applied it to analyze tongue image features of 3601 medical checkup participants in order to explore gender and age factors and the correlations among tongue features in diseases through complex networks. Results The average accuracy, recall, precision, and F1-score of our model achieved 90.67%, 91.25%, 99.28%, and 95.00%, respectively. Over the tongue images from the medical checkup population, the model Faster R-CNN detected 41.49% fissured tongue images, 37.16% tooth-marked tongue images, 29.66% greasy coating images, 18.66% spotted tongue images, 9.97% stasis tongue images, 3.97% peeled coating images, and 1.22% rotten coating images. There were significant differences in the incidence of the fissured tongue, tooth-marked tongue, spotted tongue, and greasy coating among age and gender. Complex networks revealed that fissured tongue and tooth-marked were closely related to hypertension, dyslipidemia, overweight and nonalcoholic fatty liver disease (NAFLD), and a greasy coating tongue was associated with hypertension and overweight. Conclusion The model Faster R-CNN shows good performance in the tongue image classification. And we have preliminarily revealed the relationship between tongue features and gender, age, and metabolic diseases in a medical checkup population.

According to the theory of traditional Chinese medicine (TCM), the tongue's color, texture and shape can reflect a person's physical health. Therefore, the tongue image analysis and matching are playing more and more important role in the tongue image's computer auxiliary diagnosis in TCM. Currently, the majority of the existing tongue image analysis methods are based on the single image feature (such as color, texture and shape), it is difficult to describe completely the characteristics of the tongue. This paper proposes a multi-feature fusion method (MFF) to improve the recognition rate for tongue image matching and tongue diagnosis in TCM. The optimization objective of this method is to minimize the number of the discordant pairs (inversions) between the predicted rank and the target rank, and then the proposed method learns a group of rational parameters on training data set to fuse multiple tongue image features. The experimental results show that the MFF method has very promising performance on tongue images matching and tongue diagnosis.

Tongue, the primary taste organ in the mouth, can reflect the whole body's health conditions based on the Traditional Chinese Medical (TCM) theories. Watching the tongue is one of the most common, essential and reliable methods for the TCM doctor to make diagnoses. In this thesis, a new health system is introduced based on tongue image analysis. The technologies adopted in this system ranged from tongue image processing algorithms to machine learning applications. The tongue image algorithms used in this work include image segmentation, tongue recognition and tongue image classification. Image segmentation was used to get rid of other unrelated parts, such as lip, face and neck, while keeping the tongue only. Then two recognition methods were applied to check whether the segmented result is a tongue or not. For different tongue patterns, the Support Vector Machine is applied to train a machine learning model and make predictions to classify the tongue into different labeled groups. An app named 'iTongue' is designed to monitor the body status by taking and processing tongue images in smart phones. The app provides a user-friendly, fast and powerful health tool based on TCM theories. The whole system is implemented in a webbased environment. An advanced portal was developed to connect the users and the TCM doctors. The users will not only obtain the analysis label of tongue images, but also get some life style recommendations based on the tongue image analysis. This portal helps the user understand more about his body status and guide him to adopt a more suitable diet and improve exercise.

Near-infrared (NIR) synaptic devices integrate NIR optical sensitivity and synaptic plasticity, emulating the basic biomimetic function of the human visual system and showing great potential in NIR artificial vision systems. However, the lack of semiconductor materials with appropriate band gaps for NIR photodetection and effective strategies for fabricating devices with synaptic behaviors limit the further development of NIR synaptic devices. Here, a two-terminal NIR synaptic device consisting of the In2Se3/MoS2 heterojunction has been constructed, and it exhibits fundamental synaptic functions. The reduced band gap and potential barrier of In2Se3/MoS2 heterojunctions are essential for NIR synaptic plasticity. In addition, the NIR synaptic properties of In2Se3/MoS2 heterojunctions under strain have been studied systematically. The ΔEPSC of the In2Se3/MoS2 synaptic device can be improved from 38.4% under no strain to 49.0% under a 0.54% strain with a 1060 nm illumination for 1 s at 100 mV. Furthermore, the artificial NIR vision system consisting of a 10 × 10 In2Se3/MoS2 device array has been fabricated, exhibiting image sensing, learning, and storage functions under NIR illumination. This research provides new ideas for the design of flexible NIR synaptic devices based on 2D materials and presents many opportunities in artificial intelligence and NIR vision systems.

Tongue image analysis has been an active study in medical imaging. Existing tongue image processing approaches deal with the issue of image alignment in oversimplified ways. These approaches mainly extract patches or simple regions on pre-defined positions, which are severely sensitive to tongue deformations. In this paper, we present a conformal mapping method for tongue image alignment, the principle of which is to determine the interior mapping based on the boundary mapping so that it is robust to the deformations. The conformal alignment consists of two stages: the mapping on the boundary is firstly established via the Fourier descriptor before the mapping is extended onto the interior region via Cauchy's integral and finite-difference method. Average tongues and eigen-tongues are constructed based on the conformal alignment for feature extraction. Experiments show that the proposed alignment is robust against tongue deformations and can be employed to correct existing rigid partition methods. Numerical evaluations on time efficiency and accuracy also show that our method is considerably fast and very accurate, compared with several baseline methods in this field. For the task of disease detection, the features based on the aligned images outperform some state-of-the-art features. The results reveal that the proposed method provides an efficient and accurate tool for deformable medical image alignment and disease diagnosis. A MatLab script of the proposed algorithm is available on https://codeocean.com/capsule/4382908/tree/v1.

This paper presents a methodology we have developed and used to implement an artificial binocular vision system capable of emulating the vergence of eye movements. This methodology involves using weightless neural networks (WNNs) as building blocks of artificial vision systems. Using the proposed methodology, we have designed several architectures of WNN-based artificial vision systems, in which images captured by virtual cameras are used for controlling the position of the ‘foveae’ of these cameras (high-resolution region of the images captured). Our best architecture is able to control the foveae vergence movements with average error of only 3.58 image pixels, which is equivalent to an angular error of approximately 0.629°.

BackgroundThere is a growing trend in the use of mobile health (mHealth) technologies in traditional Chinese medicine (TCM) and telemedicine, especially during the coronavirus disease (COVID-19) outbreak. Tongue diagnosis is an important component of TCM, but also plays a role in Western medicine, for example in dermatology. However, the procedure of obtaining tongue images has not been standardized and the reliability of tongue diagnosis by smartphone tongue images has yet to be evaluated.ObjectiveThe first objective of this study was to develop an operating classification scheme for tongue coating diagnosis. The second and main objective of this study was to determine the intra-rater and inter-rater reliability of tongue coating diagnosis using the operating classification scheme.MethodsAn operating classification scheme for tongue coating was developed using a stepwise approach and a quasi-Delphi method. First, tongue images (n=2023) were analyzed by 2 groups of assessors to develop the operating classification scheme for tongue coating diagnosis. Based on clinicians’ (n=17) own interpretations as well as their use of the operating classification scheme, the results of tongue diagnosis on a representative tongue image set (n=24) were compared. After gathering consensus for the operating classification scheme, the clinicians were instructed to use the scheme to assess tongue features of their patients under direct visual inspection. At the same time, the clinicians took tongue images of the patients with smartphones and assessed tongue features observed in the smartphone image using the same classification scheme. The intra-rater agreements of these two assessments were calculated to determine which features of tongue coating were better retained by the image. Using the finalized operating classification scheme, clinicians in the study group assessed representative tongue images (n=24) that they had taken, and the intra-rater and inter-rater reliability of their assessments was evaluated.ResultsIntra-rater agreement between direct subject inspection and tongue image inspection was good to very good (Cohen κ range 0.69-1.0). Additionally, when comparing the assessment of tongue images on different days, intra-rater reliability was good to very good (κ range 0.7-1.0), except for the color of the tongue body (κ=0.22) and slippery tongue fur (κ=0.1). Inter-rater reliability was moderate for tongue coating (Gwet AC2 range 0.49-0.55), and fair for color and other features of the tongue body (Gwet AC2=0.34).ConclusionsTaken together, our study has shown that tongue images collected via smartphone contain some reliable features, including tongue coating, that can be used in mHealth analysis. Our findings thus support the use of smartphones in telemedicine for detecting changes in tongue coating.

Developing an artificial visual sensory system requires optoelectronic materials and devices that can mimic the behavior of biological synapses. Organic/polymeric semiconductors have emerged as promising candidates for optoelectronic synapses due to their tunable optoelectronic properties, mechanic flexibility, and biological compatibility. In this review, we discuss the recent progress in organic optoelectronic synaptic materials and devices, including their design principles, working mechanisms, and applications. We also highlight the challenges and opportunities in this field and provide insights into potential applications of these materials and devices in next‐generation artificial visual systems. By leveraging the advances in organic optoelectronic materials and devices, we can envision its future development in artificial intelligence.

Traditional Chinese Medicine (TCM) has a long history and has been recognized as a popular alternative medicine in western countries. Tongue diagnosis is a significant procedure in computer-aided TCM, where tongue image analysis plays a dominant role. In this paper, we proposed a fully automatic tongue detection and tongue segmentation framework, which is an essential step in computer-aided tongue image analysis. Comparing with other existing methods, our method is fully automatic without any need of adjusting parameters for different images and do not need any initialization.

Comparing biological and artificial vision systems: Network measures of functional connectivity

The pursuit of groundbreaking health care innovations has led to the convergence of artificial intelligence (AI) and traditional Chinese medicine (TCM), thus marking a new frontier that demonstrates the promise of combining the advantages of ancient healing practices with cutting-edge advancements in modern technology. TCM, which is a holistic medical system with >2000 years of empirical support, uses unique diagnostic methods such as inspection, auscultation and olfaction, inquiry, and palpation. AI is the simulation of human intelligence processes by machines, especially via computer systems. TCM is experience oriented, holistic, and subjective, and its combination with AI has beneficial effects, which presumably arises from the perspectives of diagnostic accuracy, treatment efficacy, and prognostic veracity. The role of AI in TCM is highlighted by its use in diagnostics, with machine learning enhancing the precision of treatment through complex pattern recognition. This is exemplified by the greater accuracy of TCM syndrome differentiation via tongue images that are analyzed by AI. However, integrating AI into TCM also presents multifaceted challenges, such as data quality and ethical issues; thus, a unified strategy, such as the use of standardized data sets, is required to improve AI understanding and application of TCM principles. The evolution of TCM through the integration of AI is a key factor for elucidating new horizons in health care. As research continues to evolve, it is imperative that technologists and TCM practitioners collaborate to drive innovative solutions that push the boundaries of medical science and honor the profound legacy of TCM. We can chart a future course wherein AI-augmented TCM practices contribute to more systematic, effective, and accessible health care systems for all individuals.

Tongue imaging in traditional Chinese medicine is one of the important indicators for diagnosis, and its research has become a hot topic in the field of traditional Chinese medicine. In recent years, with the continuous development of computer technology, more and more research has applied computer technology to the analysis and diagnosis of tongue image classification in traditional Chinese medicine, which is of great significance for the diagnosis and treatment of traditional Chinese medicine. The research on the classification of tongue images is also of great importance. This article reviews the current status and existing problems of tongue image research.

Physical examination, which refers to various organs, depends on a lot of medical apparatus and instruments. The diagnostic result according to the examination reports is objective and reliable, but people cannot get the result instantly. As an important diagnostic method in traditional Chinese medicine (TCM), tongue diagnosis is straightforward, which depends on a sophisticated Chinese medicine doctor to examine the tongue of a patient. However, due to its qualitative, subjective and experience-based nature, traditional tongue diagnosis has a very limited application in clinical treatment. In this paper, we bridge the gap between tongue samples and their physical examination reports. With tongue images and diagnostic results according to the physical examination reports taken into account, the Health Identification System (HIS) we developed explores the relations between the tongue and health condition of a patient. Given a new tongue image, HIS automatically suggests a diagnostic result as tongue diagnosis does. Experimental results derived from a group of 1426 patients as well as other 747 volunteers show that SVM with linear kernel can obtain better performance with less computational cost than SVMs with other kernels. Ultimately, we embed SVM with linear kernel into HIS to learn a model for the tongue diagnosis.

This paper reviews the current applications, challenges, and future development trends of intelligent diagnostic devices in Traditional Chinese Medicine (TCM). With continuous advancements in artificial intelligence (AI), machine learning (ML), and data analytics, intelligent diagnostic devices are gradually being integrated into TCM diagnostics, covering various aspects such as pulse diagnosis, tongue diagnosis, and facial diagnosis. These advanced technologies not only enhance the objectivity and accuracy of TCM diagnosis but also suggest promising trends for personalized treatment development. This paper examines recent progress in intelligent devices for tongue image analysis, pulse diagnosis, and facial image analysis, highlighting existing challenges related to data quality, interdisciplinary collaboration, and ethical issues. Furthermore, the paper envisions future trends in intelligent diagnostic devices, emphasizing the importance of miniaturization, portability, telemedicine, and standardization. We expect intelligent technology to accelerate the modernization and globalization of TCM. Finally, the paper calls for future research to focus on the deep integration of TCM and modern technology, promoting the standardization of intelligent diagnostic devices and improving diagnostic reliability to foster innovative developments in TCM diagnostics.

The ultimate goal of Computer Vision has been, for more than half of century, to create an artificial vision system that could imitate the human vision. The artificial vision system should have all the capabilities of the human vision system but must not carry the same flaws. Robotics and Automation are just two examples of research areas that use artificial vision systems as the main sensorial element. In these areas, the use of color-coded objects is very common since it relieves the burden of information processing while being an unobtrusive restraint of the environment. We present a novel computer vision library called UAVision that provides support for different video sensors technologies and all the necessary software for implementing an artificial vision system for the detection of color-coded objects. The experimental results that we present, both for the scenario of robotic soccer games and for traffic sign detection, show that our library can work at more than 50fps with images of 1 megapixel.