MIT Researchers Teach LLMs to Conquer Complex Planning Challenges
Researchers at MIT have developed a novel framework that empowers large language models (LLMs) to effectively tackle intricate planning problems. This innovative approach guides LLMs to deconstruct problems in a manner akin to human reasoning, subsequently leveraging powerful software tools for automated resolution.
The framework, detailed in a paper by MIT’s Laboratory for Information and Decision Systems (LIDS), enables users to describe complex scenarios in natural language, eliminating the need for task-specific training or prompting. The LLM then translates the user’s input into a format decipherable by an optimization solver, designed for efficiently addressing demanding planning challenges.
Consider the example of a coffee company optimizing its supply chain. The company must manage bean sourcing from multiple suppliers, roasting at different facilities into various coffee types, and shipping to numerous retail locations, all while accounting for varying capacities, costs, and a projected demand increase. Traditionally, LLMs have struggled to independently devise optimal plans for such problems.
The MIT framework, however, facilitates this process by prompting the LLM to meticulously analyze the problem, identify key constraints and decision variables, and formulate a mathematical representation suitable for the optimization solver. The LLM also incorporates a self-checking mechanism at each intermediate step, proactively identifying and rectifying errors in the formulation.
In tests spanning nine complex challenges, including optimizing warehouse robot routes, the framework achieved an impressive 85% success rate, significantly outperforming the best baseline, which only reached 39%. This versatile framework holds promise for application across various multi-step planning tasks, such as airline crew scheduling and factory machine time management.
According to Yilun Hao, the lead author and a graduate student at MIT LIDS, “Our research introduces a framework that essentially acts as a smart assistant for planning problems. It can figure out the best plan that meets all the needs you have, even if the rules are complicated or unusual.”
The research team, including Yang Zhang from the MIT-IBM Watson AI Lab and Chuchu Fan, an associate professor at MIT, will present their findings at the International Conference on Learning Representations.
Fan emphasizes the framework’s accessibility, stating, “We thought that LLMs could allow nonexperts to use these solving algorithms… Could we teach an LLM to do the same thing?” The developed framework, named LLM-Based Formalized Programming (LLMFP), requires only a natural language description of the problem and the desired goal. It leverages the LLM to reason about the problem and translate it into a mathematical formulation solvable by an attached optimization solver.
The self-assessment module allows the LLM to incorporate implicit constraints. For example, the LLM would learn that a coffee shop cannot ship a negative amount of roasted beans.
The researchers aim to extend LLMFP’s capabilities by incorporating image input, further enhancing its ability to solve problems that are challenging to fully describe with natural language.
