MIT Researchers Train LLMs to Tackle Complex Planning Scenarios with Innovative Framework
Large language models (LLMs) have demonstrated impressive capabilities in various domains. However, they often struggle when directly applied to complex planning problems, such as optimizing a supply chain with multiple variables and constraints. Researchers at MIT have developed a novel framework that guides LLMs to break down these problems in a human-like manner, enabling them to leverage powerful software tools for automated solutions.
The new framework allows users to describe planning problems in natural language, without the need for task-specific examples or specialized training. The LLM then encodes the user’s prompt into a format that can be processed by an optimization solver, a software designed to efficiently address challenging planning scenarios. This innovative approach simplifies complex tasks like minimizing warehouse robot travel distance or scheduling airline crews.
The LLM meticulously checks its work during the formulation process, ensuring accurate translation of the plan for the solver. When an error is detected, the LLM attempts to rectify the specific issue rather than abandoning the entire process. In tests involving nine complex challenges, the framework achieved an impressive 85% success rate, significantly outperforming the best baseline, which only reached 39%.
Yilun Hao, a graduate student at MIT’s Laboratory for Information and Decision Systems (LIDS) and lead author of the research paper, explains, “Our research introduces a framework that essentially acts as a smart assistant for planning problems. It can figure out the best plan that meets all the needs you have, even if the rules are complicated or unusual.” The research will be presented at the International Conference on Learning Representations.
The team’s approach, named LLM-Based Formalized Programming (LLMFP), uses natural language to describe the problem, background information, and the desired outcome. The LLM then identifies key decision variables and constraints, encoding this information into a mathematical formulation suitable for the optimization solver.
The self-assessment module is a key component of the framework. It allows the LLM to identify and correct errors in the problem formulation, as well as incorporate implicit constraints that may have been initially overlooked. For example, the system can recognize that a coffee shop cannot ship a negative amount of roasted beans, a constraint that might not be explicitly stated.
Chuchu Fan, an associate professor of aeronautics and astronautics and LIDS principal investigator, notes, “Plus, an LLM can adapt to the preferences of the user. If the model realizes a particular user does not like to change the time or budget of their travel plans, it can suggest changing things that fit the user’s needs.”
The researchers aim to further enhance LLMFP by enabling it to process images as input, which would facilitate the solution of tasks that are difficult to describe fully in natural language.
