Human Motion Video Generation: A Survey.

We introduce BoltzGen, an all-atom generative model for designing proteins and peptides across all modalities to bind a wide range of biomolecular targets. BoltzGen builds strong structural reasoning capabilities about target-binder interactions into its generative design process. This is achieved by unifying design and structure prediction, resulting in a single model that also reaches state-of-the-art folding performance. BoltzGen’s generation process can be controlled with a flexible design specification language over covalent bonds, structure constraints, binding sites, and more. We experimentally validate these capabilities in a total of eight diverse wetlab design campaigns with functional and affinity readouts across 26 targets. The experiments span binder modalities from nanobodies to disulfide-bonded peptides and include targets ranging from disordered proteins to small molecules. For instance, we test 15 nanobody and protein binder designs against each of nine novel targets with low similarity to any protein with a known bound structure. For both binder modalities, this yields nanomolar binders for 66% of targets. We release model weights, data, and both inference and training code at: https://github.com/HannesStark/boltzgen.

The Deepfake phenomenon has become very popular nowadays thanks to the possibility to create incredibly realistic images using deep learning tools, based mainly on ad-hoc Generative Adversarial Networks (GAN). In this work we focus on the analysis of Deepfakes of human faces with the objective of creating a new detection method able to detect a forensics trace hidden in images: a sort of fingerprint left in the image generation process. The proposed technique, by means of an Expectation Maximization (EM) algorithm, extracts a set of local features specifically addressed to model the underlying convolutional generative process. Ad-hoc validation has been employed through experimental tests with naive classifiers on five different architectures (GDWCT, STARGAN, ATTGAN, STYLEGAN, STYLEGAN2) against the CELEBA dataset as ground-truth for non-fakes. Results demonstrated the effectiveness of the technique in distinguishing the different architectures and the corresponding generation process.

Sign language is a language that uses a combination of hand gestures and lip movements for people with hearing impairment to communicate. In Indonesia there are two sign language systems used, Sign System for Indonesian Language known as SIBI (Sistem Isyarat Bahasa Indonesia) recognizes as the official sign language system by the Indonesian Government. This research is focused on the generation process of skeleton sequence; in which represent a SIBI hand gesture excluding the finger joints. The hand skeleton that will be generated from the generation process is limited to root-word gestures only. Some researchers were using a Restricted Boltzmann Machine model and its variant known as Deep Belief Networks (DBN) to solve the sequence modelling problems. One of DBN variants is Sigmoid Belief Network (SBN). An SBN is a Bayesian network that models a binary visible vector. Deep Temporal Sigmoid Belief Network (DTSBN) is a sequence of SBNs (with deep architecture) arranged in such way that at any given time step has a fully generative process capability, where data are readily generated from the model using ancestral sampling. Since, DTSBN performance is quite novel for this particular case, we decided to implement the DTSBN model using the SIBI dataset from the previous research to construct generated hand-skeleton gestures which represent SIBI's root-word gestures. Based on the success of the experimental DTSBN model that has been successfully generated new skeleton sequences, which represent a SIBI hand gesture. Some of the inputs to the model include cartesian coordinates from shoulder joints, elbow joints, and wrist joints and the newly generated data are proven have no significant difference with the actual data set.

ECG signal generation based on conditional generative models

The generative process underlies many information retrieval models, notably statistical language models. Yet these models only examine one (current) version of the document, effectively ignoring the actual document generation process. We posit that a considerable amount of information is encoded in the document authoring process, and this information is complementary to the word occurrence statistics upon which most modern retrieval models are based. We propose a new term weighting model, Revision History Analysis (RHA), which uses the revision history of a document (e.g., the edit history of a page in Wikipedia) to redefine term frequency - a key indicator of document topic/relevance for many retrieval models and text processing tasks. We then apply RHA to document ranking by extending two state-of-the-art text retrieval models, namely, BM25 and the generative statistical language model (LM). To the best of our knowledge, our paper is the first attempt to directly incorporate document authoring history into retrieval models. Empirical results show that RHA provides consistent improvements for state-of-the-art retrieval models, using standard retrieval tasks and benchmarks.

Motion primitives play an important role in motion planning for autonomous vehicles, as they effectively address the sampling challenges inherent in nonholonomic motion planning. Employing motion primitives (MPs) is a widely accepted approach in nonholonomic motion planning based on sampling. This study specifically addresses the problem of learning from human-driving data to create human-like trajectories from predefined start-to-end states, which then serve as MP within the sampling-based nonholonomic motion planning framework. In this paper, we propose a deep learning-based method for generating MP that capture human-driving trajectory data features. By processing human-driving trajectory data, we create a Motion Primitive dataset that uniformly covers typical urban driving scenarios. Based on this dataset, a vehicle model long short-term memory neural network model is constructed to learn the features of the human-driving trajectory data. Finally, a framework for the generation of MP for practical applications is given based on this neural network. Our experiments, which focus on the dataset, the MMP generation network, and the generation process, demonstrate that our method significantly improves the training efficacy of the MP generation network. Additionally, the MP generated by our method exhibit higher accuracy compared to traditional methods.

This edited book is a comprehensive compilation of the latest developments and trends in the fields of commerce and technology. The book brings together the work of leading experts, researchers, and professionals who are shaping the future of these critical industries. The aim of the book is to provide a comprehensive understanding of the most recent trends and to offer insightful perspectives on the challenges and opportunities that lie ahead. The book covers a range of topics including e-commerce, digital marketing, data analytics, and artificial intelligence, among others. The chapters delve into the impact of technology on the commerce sector and explore innovative solutions to help businesses stay ahead of the competition. The authors also provide practical recommendations and case studies to help readers understand how they can apply these trends in their own organizations. In addition to discussing the latest trends, the book also provides a historical perspective on the development of commerce and technology and the challenges faced by businesses and policymakers. The authors offer a critical examination of the current landscape and provide insights into how businesses can overcome the challenges posed by technology and remain relevant in a rapidly changing world. The book is an invaluable resource for students, researchers, professionals, and policymakers. It provides a comprehensive understanding of the recent trends in commerce and technology and is an essential tool for anyone who wants to stay ahead of the curve. We invite you to join us on this journey and be part of shaping the future of commerce and technology. Keywords: Commerce, Technology, Trends, Development, Challenges, Opportunities.

The article raises the issue of using the historical experience of planning initiated by the establishment of the State Planning Committee in Soviet Russia in 1921. Of course, literal borrowing of the Soviet directive planning model is out of the question. However, it has also been proved in actual practice that planning methods can be successfully applied in countries with market economies. These methods are especially efficient for ensuring accelerated development and deep structural transformations in the economy. Therefore, turnabout to using planning experience is not a step back to the past, but on the contrary, a step forward to ensuring deep transformations of the modern economy. Today’s economic development has been facing deep contradictions, first and foremost of which is the obvious gap between opportunities created by the latest technologies and increasing risks and threats to the development of human civilization. The traditional market model, especially in its neoliberal version, does not open up opportunities for resolving these contradictions and often even exacerbates them. Thus, it is necessary to transition to a different socio-economic model that is based on the latest trends in technologies and allows using them to overcome the crisis of modern civilization. The author considers such a model in the framework of the noonomy concept developed by him, i.e., a non-economic way of meeting people’s needs. In this regard, planning methods act as one of the necessary tools for the transition to such a model.

The rise of Large Language Models (LLMs) has impacted research in robotics and automation. While progress has been made in integrating LLMs into general robotics tasks, a noticeable void persists in their adoption in more specific domains such as surgery, where critical factors such as reasoning, explainability, and safety are paramount. Achieving autonomy in robotic surgery, which entails the ability to reason and adapt to changes in the environment, remains a significant challenge. In this work, we propose a multi-modal LLM integration in robot-assisted surgery for autonomous blood suction. The reasoning and prioritization are delegated to the higher-level task-planning LLM, and the motion planning and execution are handled by the lower-level deep reinforcement learning model, creating a distributed agency between the two components. As surgical operations are highly dynamic and may encounter unforeseen circumstances, blood clots and active bleeding were introduced to influence decision-making. Results showed that using a multi-modal LLM as a higher-level reasoning unit can account for these surgical complexities to achieve a level of reasoning previously unattainable in robot-assisted surgeries. These findings demonstrate the potential of multi-modal LLMs to significantly enhance contextual understanding and decision-making in robotic-assisted surgeries, marking a step toward autonomous surgical systems.

Large Language Models (LLMs) increasingly shape how people access, produce, and reason with information. These models do not merely generate language, they mirror the data, discourse, and cognitive patterns on which they are trained. As a result, they often reflect and amplify existing social and cognitive biases, and even stereotypical representations, influencing what information is surfaced, how perspectives are represented, and which voices are privileged or silenced. Understanding these reflections requires going beyond technical bias detection toward examining how bias emerges in LLM outputs, how users perceive and react to it, and how design choices can reinforce or mitigate biased interactions. REFLECT offers a human-centered exploration of bias in LLMs, connecting perspectives from computer science, human–computer interaction, and cognitive psychology. REFLECT examines multiple ways in which bias manifests in LLMs (from selection effects to acquiescence and stereotypical associations) and discusses what these manifestations reveal about the interaction between human data, model training, and generative processes, also exploring interaction and design strategies that can help make such reflections visible and open to critical interpretation. Designed as a half-day session, REFLECT provides a concise yet reflective introduction to understanding bias as an emergent property of LLMs. By the end, participants will be equipped with conceptual and practical tools to identify and interpret how LLMs reflect biases, fostering more transparent, accountable, and trustworthy human–AI interactions.

In this paper, we propose a novel approach to convert given speech audio to a photo-realistic speaking video of a specific person, where the output video has synchronized, realistic, and expressive rich body dynamics. We achieve this by first generating 3D skeleton movements from the audio sequence using a recurrent neural network (RNN), and then synthesizing the output video via a conditional generative adversarial network (GAN). To make the skeleton movement realistic and expressive, we embed the knowledge of an articulated 3D human skeleton and a learned dictionary of personal speech iconic gestures into the generation process in both learning and testing pipelines. The former prevents the generation of unreasonable body distortion, while the later helps our model quickly learn meaningful body movement through a few recorded videos. To produce photo-realistic and high-resolution video with motion details, we propose to insert part attention mechanisms in the conditional GAN, where each detailed part, e.g. head and hand, is automatically zoomed in to have their own discriminators. To validate our approach, we collect a dataset with 20 high-quality videos from 1 male and 1 female model reading various documents under different topics. Compared with previous SoTA pipelines handling similar tasks, our approach achieves better results by a user study.

This paper proposes an approach to automate chemistry experiments using robots by translating natural language instructions into robot-executable plans, using large language models together with task and motion planning. Adding natural language interfaces to autonomous chemistry experiment systems lowers the barrier to using complicated robotics systems and increases utility for non-expert users, but translating natural language experiment descriptions from users into low-level robotics languages is nontrivial. Furthermore, while recent advances have used large language models to generate task plans, reliably executing those plans in the real world by an embodied agent remains challenging. To enable autonomous chemistry experiments and alleviate the workload of chemists, robots must interpret natural language commands, perceive the workspace, autonomously plan multi-step actions and motions, consider safety precautions, and interact with various laboratory equipment. Our approach, CLAIRify, combines automatic iterative prompting with program verification to ensure syntactically valid programs in a data-scarce domain-specific language that incorporates environmental constraints. The generated plan is executed through solving a constrained task and motion planning problem using PDDLStream solvers to prevent spillages of liquids as well as collisions in chemistry labs. We demonstrate the effectiveness of our approach in planning chemistry experiments, with plans successfully executed on a real robot using a repertoire of robot skills and lab tools. Specifically, we showcase the utility of our framework in pouring skills for various materials and two fundamental chemical experiments for materials synthesis: solubility and recrystallization. Further details about CLAIRify can be found at https://ac-rad.github.io/clairify/.

In this paper, we tackle the well-known problem of dataset construction from the point of its generation using generative adversarial networks (GAN). As semantic information of the dataset should have a proper alignment with images, controlling the image generation process of GAN comes to the first position. Considering this, we focus on conditioning the generative process by solely utilizing conditional information to achieve reliable control over the image generation. Unlike the existing works that consider the input (noise or image) in conjunction with conditions, our work considers transforming the input directly to the conditional space by utilizing the given conditions only. By doing so, we reveal the relations between conditions to determine their distinct and reliable feature space without the impact of input information. To fully leverage the conditional information, we propose a novel architectural framework (i.e., conditional transformation) that aims to learn features only from a set of conditions for guiding a generative model by transforming the input to the generator. Such an approach enables controlling the generator by setting its inputs according to the specific conditions necessary for semantically correct image generation. Given that the framework operates at the initial stage of generation, it can be plugged into any existing generative models and trained in an end-to-end manner together with the generator. Extensive experiments on various tasks, such as novel image synthesis and image-to-image translation, demonstrate that the conditional transformation of inputs facilitates solid control over the image generation process and thus shows its applicability for use in dataset construction.

A unified framework of community hiding using symmetric nonnegative matrix factorization

Statistical learning algorithms strongly rely on an oversimplified assumption for optimal performance, that is, source (training) and target (testing) data are independent and identically distributed. Variation in human tissue, physician labeling and physical imaging parameters (PIPs) in the generative process, yield medical image datasets with statistics that render this central assumption false. When deploying models, new examples are often out of distribution with respect to training data, thus, training robust dependable and predictive models is still a challenge in medical imaging with significant accuracy drops common for deployed models. This statistical variation between training and testing data is referred to as domain shift (DS).To the best of our knowledge we provide the first empirical evidence that variation in PIPs between test and train medical image datasets is a significant driver of DS and model generalization error is correlated with this variance. We show significant covariate shift occurs due to a selection bias in sampling from a small area of PIP space for both inter and intra-hospital regimes. In order to show this, we control for population shift, prevalence shift, data selection biases and annotation biases to investigate the sole effect of the physical generation process on model generalization for a proxy task of age group estimation on a combined 44k image mammogram dataset collected from five hospitals.We hypothesize that training data should be sampled evenly from PIP space to produce the most robust models and hope this study provides motivation to retain medical image generation metadata that is almost always discarded or redacted in open source datasets. This metadata measured with standard international units can provide a universal regularizing anchor between distributions generated across the world for all current and future imaging modalities.