Feature-based behavior coding for efficient exploratory analysis using pose estimation

The newly developed computer vision pose estimation technique in artificial intelligence (AI) is an emerging technology with potential advantages, such as high efficiency and contactless detection, for improving competitive advantage in the sports industry. The related literature is currently lacking an integrated and comprehensive discussion about the applications and limitations of using the pose estimation technique. The purpose of this study was to apply AI pose estimation techniques, and to discuss the concepts, possible applications, and limitations of these techniques in table tennis. This study implemented the OpenPose pose algorithm in a real-world video of a table tennis game. The research results show that the pose estimation algorithm performs well in estimating table tennis players’ poses from the video in a graphics processing unit (GPU)-accelerated environment. This study proposes an innovative two-stage AI pose estimation method for effectively addressing the current difficulties in applying AI to table tennis players’ pose estimation. Finally, this study provides several recommendations, benefits, and various perspectives (training vs. tactics) of table tennis and pose estimation limitations for the sports industry.

Learning motivated by electrical stimulation of the brain.

Studying animal behaviour allows us to understand how different species and individuals navigate their physical and social worlds. Video coding of behaviour is considered a gold standard: allowing researchers to extract rich nuanced behavioural datasets, validate their reliability, and for research to be replicated. However, in practice, videos are only useful if data can be efficiently extracted. Manually locating relevant footage in 10,000 s of hours is extremely time-consuming, as is the manual coding of animal behaviour, which requires extensive training to achieve reliability. Machine learning approaches are used to automate the recognition of patterns within data, considerably reducing the time taken to extract data and improving reliability. However, tracking visual information to recognise nuanced behaviour is a challenging problem and, to date, the tracking and pose-estimation tools used to detect behaviour are typically applied where the visual environment is highly controlled. Animal behaviour researchers are interested in applying these tools to the study of wild animals, but it is not clear to what extent doing so is currently possible, or which tools are most suited to particular problems. To address this gap in knowledge, we describe the new tools available in this rapidly evolving landscape, suggest guidance for tool selection, provide a worked demonstration of the use of machine learning to track movement in video data of wild apes, and make our base models available for use. We use a pose-estimation tool, DeepLabCut, to demonstrate successful training of two pilot models of an extremely challenging pose estimate and tracking problem: multi-animal wild forest-living chimpanzees and bonobos across behavioural contexts from hand-held video footage. With DeepWild we show that, without requiring specific expertise in machine learning, pose estimation and movement tracking of free-living wild primates in visually complex environments is an attainable goal for behavioural researchers.

The "Children ADHD Disease Detection Using Pose Estimation Techniques" project seeks to transform the early diagnosis of Attention Deficit Hyperactivity Disorder (ADHD) in children using sophisticated pose estimation algorithms. The novel method scrutinizes children's movement patterns and postures to detect possible symptoms of ADHD, providing a non-invasive and objective alternative to conventional assessment methods. ADHD, a neurodevelopmental condition that affects school performance and social function, stands to gain enormously from early detection and treatment. The project improves the validity and responsiveness of ADHD diagnoses through the use of PoseNet for pose estimation. The system records and analyzes the movement of the body during particular activities that are designed to trigger typical ADHD patterns of behavior. The project uses sensors, cameras, or depth sensors to record detailed movement data such as joint locations, angles, and time information. This information is used as input for a trained Support Vector Machine (SVM) model, which makes predictions about movement patterns of ADHD versus normal development. While this technology is promising, it is meant to augment and not replace thorough clinical evaluation, since ADHD is a complex disorder that needs to be approached in a holistic manner. Ethical issues, specifically around data privacy and the possibility of misuse, also need to be considered. This project is a major step forward in child mental health assessment. By bringing high-tech capabilities to the imperative of early and correct detection of ADHD, it can potentially enhance diagnostic precision, facilitate earlier intervention, and ultimately raise the quality of life in children with ADHD. Extensive testing and verification will be imperative to establish reliability, making this technique an important resource in ADHD assessment and care.

There is a critical need to grow and strengthen the pipeline of physician scientists who have expertise in sociomedical and behavioral research and are dedicated to addressing the nation's challenges posed by Alzheimer's disease and related dementias (ADRD). In 2021 The Indiana ADRD Medical Scientist Training Program (IADRD MSTP) was designed to meet this need and is built on the infrastructure of a robust portfolio of ADRD research, graduate training programs in medical neurosciences and sociology, and our existing MD-PhD program at Indiana University School of Medicine. The Aims of the IADRD MSTP are: 1) To recruit and train a competitive pool of diverse students who have an interest and commitment to social and behavioral research and patient care focused on ADRD; 2) To engage MD-PhD students early in mentored sociomedical and behavioral research that integrates IUs systems-based medical training curriculum with our cutting edge ADRD research that reinforces commitment and minimizes attrition of physician-scientists ADRD; and 3) To graduate students with dual MD-PhD degrees with strong methodological training in social and behavioral science and experts in ADRD who will be successful independent investigators at the best academic medical centers nationwide. The program includes rigorous didactic training in social, behavioral, and clinical research methods, with flexibility to allow students to focus their effort on one methodological area of interest; early initiation of ADRD research experiences with multidisciplinary teams of mentors and advisors; and the provision of educational experiences that enhance students' abilities to become independent researchers.

Accurate objective, automated limb lameness detection and pose estimation play an important role for animal well-being and precision livestock farming. We present a wearable sensor-based limb lameness detection and pose estimation for horse walk and trot locomotion. The gait event and lameness detection are first built on a recurrent neural network (RNN) with long short-term memory (LSTM) cells. Its outcomes are used in the limb pose estimation. A learned low-dimensional motion manifold is parameterized by a phase variable with a Gaussian process dynamic model. We compare the RNN-LSTM-based lameness detection method with a feature-based multi-layer classifier (MLC) and a multi-class classifier (MCC) that are built on support vector machine/K-nearest-neighbors and deep convolutional neural network methods, respectively. Experimental results show that using only accelerometer measurements, the RNN-LSTM-based approach achieves 95% lameness detection accuracy and also outperforms the feature-based MLC or MCC in terms of several assessment criteria. The pose estimation scheme can predict the 24 limb joint angles in the sagittal plane with average errors less than 5 and 10 degs under normal and induced lameness conditions, respectively. The presented work demonstrate the successful use of machine learning techniques for high performance lameness detection and pose estimation in equine science. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —Automation technologies are increasingly used for precision agriculture but few have focused on monitoring individual animals in open field for precision livestock farming. Limb lameness detection and pose estimation in open field is labor-intensive, unsafe for farmers, and inefficient. The presented machine learning-enabled, wearable inertial sensor-based design provides an effective and efficient approach for horse limb lameness detection and pose estimation applications. We present an RNN-LSTM for lameness detection and an integrated manifold learning model is used to predict the horse limb joint angles in walk and trot gaits under normal and induced lameness conditions. We also present a systematic analysis and experiments to demonstrate the impacts of the wearable sensor locations and signal information on lameness detection and pose estimation performance. Several other machine learning-based lameness detection methods are also presented and compared. The extensive multi-horse testing results are presented to demonstrate the superior accuracy and higher performance than other types of machine learning methods. One attractive feature of the proposed design lies in its high performance and fast computational capability for potential real-time applications in open field.

Reconstructing “behavioural accounting”

An automatic 3D model reconstruction technique is presented to acquire complete 3D models of real objects. The technique is based on novel approaches to pose estimation and integration. Two different poses of an object are used because a single pose often hides some surfaces from a range sensor. The presence of hidden surfaces makes the 3D model reconstructed from any single pose a partial model. Two such partial 3D models are reconstructed for two different poses of the object using a multi-view 3D modeling technique. The two partial 3D models are then registered. Coarse registration is facilitated by a novel pose estimation technique between two models. The pose is estimated by matching a stable tangent plane (STP) of each pose model with the base tangent plane (BTP) which is invariant for a vision system. The partial models are then integrated to a complete 3D model based on voxel classification defined in multi-view integration. Texture mapping is done to obtain a photo-realistic reconstruction of the object.

As the largest generation in U.S. history — the population born in the two decades immediately following World War II — enters the age of risk for cognitive impairment, growing numbers of people will experience dementia (including Alzheimer's disease and related dementias). By one estimate, nearly 14 million people in the United States will be living with dementia by 2060. Like other hardships, the experience of living with dementia can bring unexpected moments of intimacy, growth, and compassion, but these diseases also affect people's capacity to work and carry out other activities and alter their relationships with loved ones, friends, and coworkers. Those who live with and care for individuals experiencing these diseases face challenges that include physical and emotional stress, difficult changes and losses in their relationships with life partners, loss of income, and interrupted connections to other activities and friends. From a societal perspective, these diseases place substantial demands on communities and on the institutions and government entities that support people living with dementia and their families, including the health care system, the providers of direct care, and others.Nevertheless, research in the social and behavioral sciences points to possibilities for preventing or slowing the development of dementia and for substantially reducing its social and economic impacts. At the request of the National Institute on Aging of the U.S. Department of Health and Human Services, Reducing the Impact of Dementia in America assesses the contributions of research in the social and behavioral sciences and identifies a research agenda for the coming decade. This report offers a blueprint for the next decade of behavioral and social science research to reduce the negative impact of dementia for America's diverse population. Reducing the Impact of Dementia in America calls for research that addresses the causes and solutions for disparities in both developing dementia and receiving adequate treatment and support. It calls for research that sets goals meaningful not just for scientists but for people living with dementia and those who support them as well.By 2030, an estimated 8.5 million Americans will have Alzheimer's disease and many more will have other forms of dementia. Through identifying priorities social and behavioral science research and recommending ways in which they can be pursued in a coordinated fashion, Reducing the Impact of Dementia in America will help produce research that improves the lives of all those affected by dementia.

Behavioral science and machine learning have rapidly progressed in recent years. As there is growing interest among behavioral scholars to leverage machine learning, we present strategies for how these methods that can be of value to behavioral scientists using examples centered on behavioral research.

In 1971, the U.S. Senate Subcommittee on Constitutional Rights began a three-year study to investigate the federal funding of all research involving behavior modification. During this period, operant programs of behavior change, particularly those implemented in closed institutions, were subjected to specific scrutiny. In this article, I outline a number of scientific and social factors that led to this investigation and discuss the study itself. I show how behavioral scientists, both individually and through their professional organizations, responded to this public scrutiny by (1) self-consciously altering their terminology and techniques; (2) considering the need to more effectively police their professional turf; and (3) confronting issues of ethics and values in their work. Finally, I link this episode to the formation of the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research, whose recommendations resulted in changes to the ethical regulation of federally funded human subjects research that persist to the present day.

Increasing blood donor recruitment and retention is of key importance to transfusion services. Research within the social and behavioral science traditions has adopted separate but complementary approaches to addressing these issues. This article aims to review both of these types of literature, examine theoretical developments, identify commonalities, and offer a means to integrate these within a single intervention approach. The social and behavioral science literature on blood donor recruitment and retention focusing on theory, interventions, and integration is reviewed. The role of emotional regulation (anticipated anxiety and vasovagal reactions) is central to both the behavioral and the social science approaches to enhancing donor motivation, yet although intentions are the best predictor of donor behavior, interventions targeting enactment of intentions have not been used to increase donation. Implementation intentions (that is, if-then plans formed in advance of acting) provide a useful technique to integrate findings from social and behavioral sciences to increase donor recruitment and retention. After reviewing the literature, implementation intention formation is proposed as a technique to integrate the key findings and theories from the behavioral and social science literature on blood donor recruitment and retention.

During the health crisis, construction companies are under tremendous pressure to adopt innovative techniques for employee management to meet the health and safety demands and reinstate productivity, starting from managing attendance and monitoring performance. The construction industry can implement more equipment's and robots to address worker safety while maintaining a safe distance, thus reducing the labor force. This intern results in man-machine interaction conflicts and accidents due to the failure of machines or components that needs addressing. Most workplace accidents in South Korea occur in the construction industry, and 60% of them are due to falls from heights. The aging population in South Korea makes the construction industry more prone to mental fatigue, resulting in such accidents. Thus, this study aims to integrate systems of Health and Safety with Performance using pose estimation techniques. Using deep learning techniques, IIOT (Industrial Internet of Things), ICT with wireless access points, and wearable devices such as smart bands, shoes, and helmets aids to monitor workers' Performance, Health, and safety in indoor conditions or after structural frame completion. Combining previous warning alert systems with pose estimation and monitoring will alert workers on unsafe and uncomfortable poses and improve productivity significantly. A future version of this system could use LoRa to extend to outdoor environments.

Outdoor AR applications on mobile devices need accurate estimates for the pose of the device. In this paper, we develop SplatPose, a novel pose estimation technique that uses a data-driven 3D modeling technique called Gaussian Splatting. SplatPose uses a trained Gaussian Splatting model to render an image at an estimated device location, then matches features with the camera image to estimate pose. % Because this matching can be fast, SplatPose can, in theory, estimate pose entirely on a mobile device, while existing approaches cannot. To this end, SplatPose trains Gaussian Splatting models to be robust to appearance changes, thereby improving accuracy. It also incorporates a novel fast renderer to improve rendering speed. Using an AR pose estimation benchmark dataset, we show that SplatPose outperforms the state-of-the-art in terms of accuracy, and is up to an order of magnitude faster on a mobile device.

The estimation of player positions is key for performance analysis in sport. In this paper, we focus on image-based, single-angle, player position estimation in padel. Unlike tennis, the primary camera view in professional padel videos follows a de facto standard, consisting of a high-angle shot at about 7.6 m above the court floor. This camera angle reduces the occlusion impact of the mesh that stands over the glass walls, and offers a convenient view for judging the depth of the ball and the player positions and poses. We evaluate and compare the accuracy of state-of-the-art computer vision methods on a large set of images from both amateur videos and publicly available videos from the major international padel circuit. The methods we analyze include object detection, image segmentation and pose estimation techniques, all of them based on deep convolutional neural networks. We report accuracy and average precision with respect to manually-annotated video frames. The best results are obtained by top-down pose estimation methods, which offer a detection rate of 99.8% and a RMSE below 5 and 12 cm for horizontal/vertical court-space coordinates (deviations from predicted and ground-truth player positions). These results demonstrate the suitability of pose estimation methods based on deep convolutional neural networks for estimating player positions from single-angle padel videos. Immediate applications of this work include the player and team analysis of the large collection of publicly available videos from international circuits, as well as an inexpensive method to get player positional data in amateur padel clubs.