Themis AI: Teaching AI Models What They Don’t Know to Enhance Reliability

As artificial intelligence systems like ChatGPT become increasingly prevalent, their ability to provide seemingly plausible answers to any question is both impressive and concerning. A significant challenge lies in their tendency to conceal gaps in their knowledge or areas of uncertainty, which can lead to critical issues, especially in high-stakes applications such as drug development, information synthesis, and autonomous driving.

Themis AI, an MIT spinout, is tackling this problem head-on with its Capsa platform. Capsa is designed to quantify model uncertainty and correct outputs, ensuring greater reliability. This innovative platform can be integrated with any machine-learning model to swiftly detect and rectify unreliable outputs by enabling AI models to identify patterns indicative of ambiguity, incompleteness, or bias in their data processing.

“The idea is to take a model, wrap it in Capsa, identify the uncertainties and failure modes of the model, and then enhance the model,” explains Themis AI co-founder Daniela Rus, who also serves as the director of MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL). “We’re excited about offering a solution that can improve models and offer guarantees that the model is working correctly.”

Founded in 2021 by Rus, Alexander Amini, and Elaheh Ahmadi, Themis AI has already made significant strides across various industries. The company has assisted telecom companies in network planning and automation, aided oil and gas companies in interpreting seismic imagery using AI, and contributed to research on developing more reliable and trustworthy chatbots.

Amini emphasizes the company’s mission: “We want to enable AI in the highest-stakes applications of every industry. We’ve all seen examples of AI hallucinating or making mistakes. As AI is deployed more broadly, those mistakes could lead to devastating consequences. Themis makes it possible that any AI can forecast and predict its own failures before they happen.”

Rus’s lab has dedicated years to researching model uncertainty, notably receiving funding from Toyota in 2018 to assess the reliability of machine learning-based autonomous driving solutions. This research underscores the critical importance of understanding model reliability in safety-critical contexts.

Further research by Rus, Amini, and their collaborators led to the creation of an algorithm capable of detecting and eliminating racial and gender bias in facial recognition systems. By identifying unrepresentative parts of the training data and generating new, balanced data samples, the algorithm effectively reweights the model’s training data to mitigate bias. A similar approach was later applied to help pharmaceutical companies leverage AI models for predicting drug candidate properties, showcasing the broad applicability of their techniques.

Today, Themis AI is collaborating with numerous enterprises across diverse sectors, many of whom are developing large language models (LLMs). Capsa enables these models to quantify their own uncertainty for each output, leading to more reliable and trustworthy results.

Stewart Jamieson, Themis AI’s head of technology, notes, “Many companies are interested in using LLMs that are based on their data, but they’re concerned about reliability. We help LLMs self-report their confidence and uncertainty, which enables more reliable question answering and flagging unreliable outputs.”

Additionally, Themis AI is engaged in discussions with semiconductor companies to build AI solutions on chips that can function outside of cloud environments, potentially merging the efficiency of edge computing with high-quality results. Pharmaceutical companies are also leveraging Capsa to enhance AI models used in identifying drug candidates and predicting clinical trial performance.

Themis AI remains committed to pushing the boundaries of AI technology. The company is actively exploring Capsa’s potential to improve accuracy in chain-of-thought reasoning, a technique where LLMs explain the steps they take to arrive at an answer.

Rus sees Themis AI as a vital opportunity to ensure her MIT research has a tangible impact on the world. “My students and I have become increasingly passionate about going the extra step to make our work relevant for the world,” she says. “AI has tremendous potential to transform industries, but AI also raises concerns. What excites me is the opportunity to help develop technical solutions that address these challenges and also build trust and understanding between people and the technologies that are becoming part of their daily lives.”