Could LLMs Help Design Our Next Medicines and Materials?

The discovery of new medicines and materials hinges on identifying molecules with specific properties, a process that’s historically been slow, expensive, and computationally intensive. However, a new approach from MIT and the MIT-IBM Watson AI Lab suggests that Large Language Models (LLMs) like ChatGPT could revolutionize this field.

The primary challenge has been enabling LLMs to understand and reason about molecular structures in the same way they process language. The MIT team has addressed this by enhancing an LLM with graph-based models, specifically designed for predicting and generating molecular structures.

Their method uses a base LLM to interpret natural language queries about desired molecular properties. The system then intelligently switches between the base LLM and graph-based AI modules to design the molecule, explain the rationale behind the design, and even generate a step-by-step synthesis plan. This involves seamlessly integrating text, graph, and synthesis step generation into a common vocabulary that the LLM can understand and utilize.

The results have been promising. This multimodal technique significantly outperformed existing LLM-based approaches, producing molecules that more accurately matched user specifications and possessed valid synthesis plans. The success rate jumped from 5 percent to 35 percent.

Notably, this new system also outperformed larger, purely text-based LLMs, highlighting the importance of multimodality in molecular design. “This could hopefully be an end-to-end solution where, from start to finish, we would automate the entire process of designing and making a molecule. If an LLM could just give you the answer in a few seconds, it would be a huge time-saver for pharmaceutical companies,” says Michael Sun, an MIT graduate student and co-author of the paper on this technique.

The research team, including lead author Gang Liu from the University of Notre Dame, MIT Professor Wojciech Matusik, Notre Dame Associate Professor Meng Jiang, and senior author Jie Chen from the MIT-IBM Watson AI Lab, will present their findings at the International Conference on Learning Representations.

LLMs, while powerful in natural language processing, aren’t inherently designed to understand the intricacies of chemistry. This limitation hinders their effectiveness in inverse molecular design – the process of identifying molecular structures with specific functions or properties. LLMs convert text into tokens and predict subsequent words, but molecules, with their non-sequential graph structures of atoms and bonds, pose a unique challenge.

Graph-based AI models, on the other hand, excel at representing atoms and bonds as interconnected nodes and edges. However, they often require complex inputs, lack natural language understanding, and yield results that can be difficult to interpret. The MIT researchers’ unified framework, Llamole, combines the strengths of both approaches.

Llamole (large language model for molecular discovery) uses a base LLM as a gatekeeper to interpret user queries in plain language, such as a request for a molecule that can penetrate the blood-brain barrier and inhibit HIV, given a specific molecular weight and bond characteristics. As the LLM predicts text, it intelligently switches between different graph modules. One module uses a graph diffusion model to generate molecular structures based on the input requirements. Another module uses a graph neural network to encode the generated structure back into tokens for the LLM to consume. A final module predicts the reaction steps needed to synthesize the molecule from basic building blocks.

The researchers introduced a new type of trigger token to signal the LLM when to activate each module. A “design” trigger activates the structure sketching module, while a “retro” trigger activates the retrosynthetic planning module.

The output includes an image of the molecular structure, a textual description, and a detailed, step-by-step synthesis plan, outlining the necessary chemical reactions.

In experiments, Llamole outperformed numerous standard and fine-tuned LLMs, as well as a state-of-the-art domain-specific method. It also improved the retrosynthetic planning success rate by generating higher-quality molecules with simpler structures and lower-cost building blocks.

To train and evaluate Llamole, the researchers created two new datasets, augmenting hundreds of thousands of patented molecules with AI-generated natural language descriptions and customized description templates. While the current version of Llamole is limited to designing molecules considering 10 numerical properties, future work will focus on generalizing the system to incorporate any molecular property and improving the graph modules to further enhance retrosynthesis success rates.

The researchers envision expanding this approach beyond molecules to handle other types of graph-based data, such as interconnected sensors in a power grid or transactions in a financial market. “Llamole demonstrates the feasibility of using large language models as an interface to complex data beyond textual description, and we anticipate them to be a foundation that interacts with other AI algorithms to solve any graph problems,” says Chen.