Honda (Germany)

Abstract Significant multi-agent advances addressing the challenge of learning policies for acting in ad hoc teamwork have been made. In ad hoc teamwork, a team of agents must cooperate effectively without prior coordination or communication. Many existing approaches, however, struggle to perform well in open environments where the setting can change significantly during deployment. This paper presents a new reinforcement learning approach to tackle collaboration in open environments controlling one agent with a changing number of distinct other agents, each with an individual task. The approach uses policy blending based on an online goal inference module and a collection of learned policies modeling the individual interaction impact between the agent and populations of partners with different tasks. Blending is done using the estimated goals of others and a posterior-based action blending with entropy adjustment and regularization. Our approach addresses issues of existing policy blending mechanisms, such as handling conflicting modes in action distributions leading to oscillation and instability and adapting to uncertain states dynamically. In experiments in two collaborative open environments based on Overcooked and Level-based Foraging, our approach outperforms a baseline learner, trained with the joint reward of all agents, across changes to both agents and tasks. Ablation studies further highlight the importance of our posterior-based blending mechanism to achieve high rewards as well as the provided goal weighting. The proposed approach provides an important step towards the application of reinforcement learning to AI assistance beyond strictly closed worlds and towards more realistic scenarios.

Advanced Aerial Mobility encompasses many outstanding applications that promise to revolutionize modern logistics and pave the way for various public services and industry uses. However, throughout its history, the development of such systems has been impeded by the complexity of legal restrictions and physical constraints. While airspaces are often tightly shaped by various legal requirements, Unmanned Aerial Vehicles (UAVs) must simultaneously consider, among others, energy demands, signal quality, and noise pollution. In this work, we address this challenge by presenting a novel architecture that integrates methods of Probabilistic Mission Design (ProMis) [ 1 , 2 ] and Many-Objective Optimization [ 3 ] for UAV routing. Hereby, our framework facilitates compliance with legal requirements under uncertainty while producing effective paths that minimize various physical costs a UAV needs to consider when traversing human-inhabited spaces. To this end, we combine hybrid probabilistic first-order logic for spatial reasoning with mixed deterministic-stochastic route optimization, incorporating physical objectives such as energy consumption and radio interference with a logical, probabilistic model of legal requirements. We demonstrate the versatility and advantages of our system in a large-scale empirical evaluation over real-world, crowd-sourced data from a map extract from the city of Paris, France, showing how a network of effective and compliant paths can be formed.

In interactions between automated vehicles (AVs) and crossing pedestrians, modeling implicit vehicle communication is crucial. In this work, we present a combined prediction and planning approach that allows to consider the influence of the planned vehicle behavior on a pedestrian and predict a pedestrian's reaction. We plan the behavior by solving two consecutive optimal control problems (OCPs) analytically, using variational calculus. We perform a validation step that assesses whether the planned vehicle behavior is adequate to trigger a certain pedestrian reaction, which accounts for the closed-loop characteristics of prediction and planning influencing each other. In this step, we model the influence of the planned vehicle behavior on the pedestrian using a probabilistic behavior acceptance model that returns an estimate for the crossing probability. The probabilistic modeling of the pedestrian reaction facilitates considering the pedestrian's costs, thereby improving cooperative behavior planning. We demonstrate the performance of the proposed approach in simulated vehicle-pedestrian interactions with varying initial settings and highlight the decision making capabilities of the planning approach.

This paper studies the Dynamic Electric Dial-A-Ride Problem (DEDARP), which is a combinatorial optimisation problem that has applications in real-world ridesharing services with electric vehicles. In addition to the challenges from classical scheduling and route planning, we consider here the extra challenge of making real-time dispatching decisions in dynamic environments with new requests arriving over time and selecting proper times for the vehicles to recharge. To solve DEDARP effectively, we propose a Genetic Programming Hyper-Heuristic (GPHH) that evolves heuristics/policies to dispatch vehicles in real time. We have developed a simulation process that generates a solution for any given instance by two policies, one for vehicle allocation and the other for request allocation, and design fitness evaluations based on the simulation. Moreover, we propose a multi-tree GP to evolve these two policies simultaneously, which makes use of advanced terminals to comprehensively represent the state. Experimental results on a wide range of instances show that GPHH can evolve effective policies that make significantly better real-time dispatching decisions than human-designed policies based on prior knowledge.

There seems to be a solid object inside.""What other toppings do you usually like on your pizza?" Figure 1: We imbued a robot with curious behaviors.The figure shows two examples.Left: The robot shakes a container to check whether there is an object inside.Right: The robot asks for the person's preferences.

This paper addresses the problem of human-based driver support. Nowadays, driver support systems help users to operate safely in many driving situations. Nevertheless, these systems do not fully use the rich information that is available from sensing the human driver. In this paper, we therefore present a human-based risk model that uses driver information for improved driver support. In contrast to state of the art, our proposed risk model combines a) the current driver perception based on driver errors, such as the driver overlooking another vehicle (i.e., notice error), and b) driver personalization, such as the driver being defensive or confident. In extensive simulations of multiple interactive driving scenarios, we show that our novel human-based risk model achieves earlier warning times and reduced warning errors compared to a baseline risk model.

We consider the electric autonomous dial-a-ride problem (E-ADARP), a challenging extension of the dial-a-ride problem with the goal of finding minimum cost routes serving given transportation requests with a fleet of electric and autonomous vehicles (EAVs). Special emphasis lies on the minimization of user excess ride time under consideration of the charging requirements of the EAVs while constraints regarding, for example, user ride times and time windows have to be satisfied. We propose a novel large neighborhood search (LNS) approach for the E-ADARP employing the concept of battery-restricted fragments for route representation and efficient cost computations. For the charging of the EAVs, the scheduling, and route evaluation, we introduce two approaches where one deals with these challenges separately and one provides a combined approach. The first approach uses dedicated LNS operators and a forward labeling algorithm whereas the latter employs a novel route evaluation procedure for inserting charging stops on-the-fly as needed. The performance of our LNS-based algorithms is evaluated on common benchmark instances and results show that especially the approach with the on-the-fly insertion almost consistently outperforms former state-of-the-art techniques, finding new best-known solutions for many instances.

Rapid advancements in text-to-3D shape synthesis using generative AI models have achieved remarkable general-ization performance, generating novel designs based on com-binations of different concepts. However, designs synthesized through automated optimization often result in ill-defined shapes, which render such designs impractical for engineering design applications. We present the first end-to-end prompt evolution design optimization (PREDO) framework contextualized in a vehicle design scenario that leverages a vision-language model to penalize impractical car designs synthesized by a generative model. The backbone of our framework is an evolutionary strategy coupled with an optimization objective function that comprises a physics-based solver and a vision-language model for practical or functional guidance in the generated car designs. In the prompt evolution search, the optimizer iteratively generates a population of text prompts, which embed user specifications on the aerodynamic performance and visual preferences of the 3D car designs. Then, in addition to the computational fluid dynamics simulations, the pre-trained vision-language model is used to penalize impractical designs and, thus, foster the evolutionary algorithm to seek more viable designs. Our investigations on a car design optimization problem show a wide spread of potential car designs generated at the early phase of the search, which indicates a good diversity of designs in the initial populations, and an increase of over 20 % in the probability of generating practical designs compared to a baseline framework without using a vision-language model. Visual inspection of the designs against the performance results demonstrates prompt evolution as a very promising paradigm for finding novel designs with good optimization performance while providing ease of use in specifying design specifications and preferences via a natural language interface.

Recent advances in Large Language Models (LLM) open up opportunities for users to interact with domain specific knowledge and execute (semi-)professional tasks in a dialog fashion. Without profound knowledge in data science and programming languages, basic statistics and further detailed analyses can be conducted intuitively through natural language prompts. Accessing common data science methods, LLMs can assist users in visualizing, interpreting, and analyzing large and complex data sets. In this paper, we study the potential of ChatGPT from the perspective of different user expertise levels to process and interpret existing data sets which originate from engineering optimization tasks with varying complexity—– a proportional-integral-controller configuration, an aerodynamic design optimization, and an energy management task—–and use it to cluster the data into meaningful groups. Furthermore, we provide the LLM with a recently developed concept identification metric and assess its capability to use the metric for an allocation of the samples into technically reasonable concepts. On one hand, the formulation of conceptual machine learning pipelines, as well as using well-established methods, such as typical clustering algorithms, are conducted seamlessly by ChatGPT for the given tasks. On the other hand, it is observed that automatically constructing a concept identification algorithm from its mathematical formulation using ChatGPT still requires supervision and support from a domain expert.