Multi-Modal Sketch-based Behavior Tree Synthesis
This program is tentative and subject to change.
Behavior trees (BTs) are widely adopted in the field of agent control, particularly in robotics, due to their modularity and reactivity. However, constructing a BT that meets the desired expectations is time-consuming and challenging, especially for non-expert users. This paper presents BtBot, a multi-modal sketch-based behavior tree synthesis technique. Given a natural language task description and a set of positive and negative examples, BtBot automatically generates a BT program that aligns with the natural language description and meets the requirements of the examples. Inside BtBot, an LLM is employed to understand the task’s natural language description and generate a sketch of the task execution. Then, BtBot searches the sketch to synthesize a candidate BT program consistent with the user-provided positive and negative examples. When the sketch is proven impossible to generate the target BT, BtBot provides a multi-step repairing method that modifies the sketch’s control nodes and structure to search for the target BT. We have implemented BtBot in a prototype and evaluated it on a benchmark of 70 tasks across multiple scenarios. The experimental results indicate that BtBot outperforms the existing BT synthesis techniques in effectiveness and efficiency. Besides, an user study is conducted to demonstrate BtBot’s usefulness.
This program is tentative and subject to change.
Sat 18 OctDisplayed time zone: Perth change
13:45 - 15:30 | |||
13:45 15mTalk | Tunneling Through the Hill: Multi-Way Intersection for Version-Space Algebras in Program Synthesis OOPSLA Guanlin Chen Peking University, Ruyi Ji Peking University, Shuhao Zhang Peking University, Yingfei Xiong Peking University | ||
14:00 15mTalk | Language-Parametric Reference Synthesis OOPSLA Daniel A. A. Pelsmaeker Delft University of Technology, Netherlands, Aron Zwaan Delft University of Technology, Casper Bach University of Southern Denmark, Arjan J. Mooij Zürich University of Applied Sciences | ||
14:15 15mTalk | Multi-Modal Sketch-based Behavior Tree Synthesis OOPSLA Wenmeng Zhang College of Computer Science and Technology, National University of Defense Technology, Changsha, China, Zhenbang Chen College of Computer, National University of Defense Technology, Weijiang Hong National University of Defense Technology, Changsha, China | ||
14:30 15mTalk | Synthesizing DSLs for Few-Shot Learning OOPSLA Paul Krogmeier University of Illinois at Urbana-Champaign, P. Madhusudan University of Illinois at Urbana-Champaign | ||
14:45 15mTalk | Synthesizing Implication Lemmas for Interactive Theorem Proving OOPSLA Ana Brendel University of California Los Angeles, Aishwarya Sivaraman Meta, Todd Millstein University of California at Los Angeles | ||
15:00 15mTalk | Synthesizing Sound and Precise Abstract Transformers for Nonlinear Hyperbolic PDE Solvers OOPSLA Jacob Laurel Georgia Institute of Technology, Ignacio Laguna Lawrence Livermore National Laboratory, Jan Hueckelheim Argonne National Laboratory | ||
15:15 15mTalk | UTFix: Change Aware Unit Test Repairing using LLM OOPSLA Shanto Rahman The University of Texas at Austin, Sachit Kuhar Amazon Web Services, Berk Cirisci Amazon Web Services, Pranav Garg AWS, Shiqi Wang AWS AI Labs, Xiaofei Ma AWS AI Labs, Anoop Deoras AWS AI Labs, Baishakhi Ray Columbia University |