AI Restores Damaged Paintings with a Digitally-Generated Mask in Hours

Art restoration, a meticulous process demanding both skill and patience, has long been a practice where conservators painstakingly identify and repair damaged areas of paintings, precisely matching colors to seamlessly fill in each imperfection. A single painting often contains thousands of such areas, making the restoration process extend from weeks to even decades.

Digital restoration tools have emerged in recent years, offering virtual representations of restored artworks. These tools utilize computer vision, image recognition, and color matching techniques to create a digitally restored version of a painting at a faster pace. However, translating these digital restorations onto original artworks has remained a challenge – until now.

Alex Kachkine, a mechanical engineering graduate student at MIT, introduces a groundbreaking method in a paper published in Nature. This innovative approach physically applies a digital restoration directly onto an original painting through a thin polymer film mask that is aligned and adhered to the artwork. This mask can be easily removed, and its digital file stored for future reference by conservators, providing a clear record of the restoration changes.

“Because there’s a digital record of what mask was used, in 100 years, the next time someone is working with this, they’ll have an extremely clear understanding of what was done to the painting,” Kachkine explains. “And that’s never really been possible in conservation before.”

In a demonstration, Kachkine applied the method to a severely damaged 15th-century oil painting. The AI-driven process automatically identified 5,612 regions requiring repair and filled them using 57,314 different colors. This entire process took only 3.5 hours, significantly faster than traditional methods, estimated to be about 66 times quicker.

Ethical considerations are vital in any restoration project, Kachkine notes. The representation of an artist’s original style and intent must be carefully considered. Therefore, the application of this new method should always involve conservators with expertise in the painting’s history and origins.

“There is a lot of damaged art in storage that might never be seen,” Kachkine says. “Hopefully with this new method, there’s a chance we’ll see more art, which I would be delighted by.”

Kachkine’s restoration process began as a side project, inspired by visits to numerous art galleries. He observed that many artworks were stored away due to damage, requiring extensive restoration efforts.

Digital tools can accelerate the restoration process. AI algorithms quickly analyze vast amounts of visual data, learning connections to generate digitally restored versions closely resembling an artist’s style or time period. However, these digital restorations are typically displayed virtually or printed as stand-alone works, without direct application to original art.

Kachkine’s method involves cleaning the painting, scanning it to identify damaged regions, and using AI algorithms to create a virtual representation of the painting in its original state. Software then maps the regions needing infilling and the exact colors required. This map translates into a two-layer mask printed onto thin polymer films. The first layer is printed in color, while the second is printed in white to fully reproduce the color spectrum.

Using high-fidelity commercial inkjets, Kachkine prints the mask’s layers and carefully aligns them onto the original painting, adhering them with varnish. The printed films can be easily dissolved, revealing the original work if needed, and the digital file of the mask serves as a detailed restoration record.

The restoration was able to fill in thousands of losses in just a few hours.

The new method can be significantly faster than traditional, hand-painted approaches. Kachkine stresses that conservators should be involved throughout the process to ensure the final work aligns with the artist’s style and intent.

“It will take a lot of deliberation about the ethical challenges involved at every stage in this process to see how can this be applied in a way that’s most consistent with conservation principles,” he says. “We’re setting up a framework for developing further methods. As others work on this, we’ll end up with methods that are more precise.”