Toyota Industries (United States)

We introduce PAPRLE (plug-and-play robotic limb environment), a modular ecosystem that enables flexible placement and control of robotic limbs. With PAPRLE a user can change the arrangement of the robotic limbs and control them using a variety of input devices, including puppeteers, gaming controllers, and virtual reality (VR) devices. This versatility supports a wide range of teleoperation scenarios and promotes adaptability to different task requirements. We also introduce a pluggable puppeteer device that can be easily mounted and adapted to match the target robot configurations. PAPRLE supports bilateral teleoperation through these puppeteer devices, agnostic to the type or configuration of the follower robot. The modular design of PAPRLE facilitates novel spatial arrangements of the limbs and enables scalable data collection, thereby advancing research in embodied artificial intelligence (AI) and learning-based control. We validate PAPRLE in various real-world settings, demonstrating its versatility across diverse combinations of leader devices and follower robots. The system will be released as open source, including both hardware and software components, to support broader adoption and extension.

Collaborative robots must quickly adapt to their partner's intent and preferences to proactively identify helpful actions. This is especially true in situated settings where human partners can continually teach robots new high-level behaviors, visual concepts, and physical skills (e.g., through demonstration), growing the robot's capabilities as the human-robot pair work together to accomplish diverse tasks. In this work, we argue that robots should be able to <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">infer their partner's goals</i> from early interactions and use this information to <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">proactively plan behaviors</i> ahead of explicit instructions from the user. Building from the strong commonsense priors and steerability of large language models, we introduce ProVox (“Proactive Voice”), a novel framework that enables robots to efficiently personalize and adapt to individual collaborators. We design a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">meta-prompting protocol</i> that empowers users to communicate their distinct preferences, intent, and expected robot behaviors ahead of starting a physical interaction. ProVox then uses the personalized prompt to condition a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">proactive language model task planner</i> that anticipates a user's intent from the current interaction context and robot capabilities to suggest helpful actions; in doing so, we alleviate user burden, minimizing the amount of time partners spend explicitly instructing and supervising the robot. We evaluate ProVox through user studies grounded in household manipulation tasks (e.g., assembling lunch bags) that measure the efficiency of the collaboration, as well as features such as perceived helpfulness, ease of use, and reliability. Our analysis suggests that both meta-prompting and proactivity are critical, resulting in 38.7% faster task completion times and 31.9% less user burden relative to non-active baselines.

We present BioSpark, a system for analogical innovation designed to act as a creativity partner in reducing the cognitive effort in finding, mapping, and creatively adapting diverse inspirations. While prior approaches have focused on initial stages of finding inspirations, BioSpark uses LLMs embedded in a familiar, visual, Pinterest-like interface to go beyond inspiration to supporting users in identifying the key solution mechanisms, transferring them to the problem domain, considering tradeoffs, and elaborating on details and characteristics. To accomplish this BioSpark introduces several novel contributions, including a tree-of-life enabled approach for generating relevant and diverse inspirations, as well as AI-powered cards including 'Sparks' for analogical transfer; 'Trade-offs' for considering pros and cons; and 'Q&A' for deeper elaboration. We evaluated BioSpark through workshops with professional designers and a controlled user study, finding that using BioSpark led to a greater number of generated ideas; those ideas being rated higher in creative quality; and more diversity in terms of biological inspirations used than a control condition. Our results suggest new avenues for creativity support tools embedding AI in familiar interaction paradigms for designer workflows.

We demonstrate a framework for real-time active learning in state-changing experimental systems, integrating path-dependent acquisition functions with a fully autonomous, cloud-controlled experimental platform to accelerate materials discovery.

With the rise of stochastic generative models in robot policy learning, end-to-end visuomotor policies are increasingly successful at solving complex tasks by learning from human demonstrations. Nevertheless, since real-world evaluation costs afford users only a small number of policy rollouts, it remains a challenge to accurately gauge the performance of such policies. This is exacerbated by distribution shifts causing unpredictable changes in performance during deployment. To rigorously evaluate behavior cloning policies, we present a framework that provides a tight lower-bound on robot performance in an arbitrary environment, using a minimal number of experimental policy rollouts. Notably, by applying the standard stochastic ordering to robot performance distributions, we provide a worst-case bound on the entire distribution of performance (via bounds on the cumulative distribution function) for a given task. We build upon established statistical results to ensure that the bounds hold with a user-specified confidence level and tightness, and are constructed from as few policy rollouts as possible. In experiments we evaluate policies for visuomotor manipulation in both simulation and hardware. Specifically, we (i) empirically validate the guarantees of the bounds in simulated manipulation settings, (ii) find the degree to which a learned policy deployed on hardware generalizes to new real-world environments, and (iii) rigorously compare two policies tested in out-of-distribution settings. Our experimental data, code, and implementation of confidence bounds are open-source. 1

Abstract This study introduces a generative imputation model leveraging graph attention networks and tabular diffusion models for completing missing parametric data in engineering designs. This model functions as an AI design co-pilot, providing multiple design options for incomplete designs, which we demonstrate using the bicycle design CAD dataset. Through comparative evaluations, we demonstrate that our model significantly outperforms existing classical methods, such as MissForest, hotDeck, PPCA, and tabular generative method TabCSDI in both the accuracy and diversity of imputation options. Generative modeling also enables a broader exploration of design possibilities, thereby enhancing design decision-making by allowing engineers to explore a variety of design completions. The graph model combines GNNs with the structural information contained in assembly graphs, enabling the model to understand and predict the complex interdependencies between different design parameters. The graph model helps accurately capture and impute complex parametric interdependencies from an assembly graph, which is key for design problems. By learning from an existing dataset of designs, the imputation capability allows the model to act as an intelligent assistant that autocompletes CAD designs based on user-defined partial parametric design, effectively bridging the gap between ideation and realization. The proposed work provides a pathway to not only facilitate informed design decisions but also promote creative exploration in design.

A novel multiscale simulation scheme for textile composites is proposed, which accurately predicts their mechanical properties. A key aspect of our method is the numerical construction of their representative volume elements by considering the fiber yarn deformations during weaving and molding of textile composites. We found that fiber yarn deformations during weaving and molding can be precisely computed using a novel method for modeling dry yarn mechanical behaviors with beam and solid elements. We validated our proposed method by comparing its mechanical property predictions with the experimental results obtained from tensile, compression, and bending tests of carbon fiber composites. The validation demonstrated that our method can accurately and robustly predict the mechanical properties of a composite by considering fiber yarn deformations during weaving and molding.

Escalating global CO2 emissions highlights the immediate need for scalable sustainable practices. Corporate and policy roles aside, there’s a need for carbon-neutrality-based systems and practices to bridge the disconnect between actions and the perceived impact on the environment. This one-day workshop focuses on individuals and communities, advocating for human-centered tools to bridge this awareness-action gap. While Human-AI Interaction (HAI), cognitive science theories, and social computing tools have shown promise in various domains, their potential remains largely unexplored in the context of sustainability. This workshop aims to delve into these avenues for crafting tractable systems for effective, contextually relevant interventions and driving sustainable behaviors. By engaging multidisciplinary researchers, we aim to intertwine local insights with behavioral theory and technology, fostering intrinsic carbon literacy and a sustainability ethos, ensuring lasting and scalable impacts.