Merging AI and underwater photography to reveal hidden ocean worlds
The mysterious depths of our oceans hold untold wonders, yet many remain unseen and vulnerable to rapid environmental shifts. In a groundbreaking initiative, scientists and artists are now harnessing the power of artificial intelligence to illuminate these hidden marine worlds, offering an unprecedented look at biodiversity facing a changing climate.
At the forefront of this innovation is LOBSTgER — Learning Oceanic Bioecological Systems Through Generative Representations — a pioneering research project developing at MIT Sea Grant. This ambitious endeavor seamlessly merges advanced artificial intelligence with meticulous underwater photography to document and visually share the delicate ocean life of the Northeastern United States’ Gulf of Maine. This region, known for its rich biodiversity, is warming at an alarming rate, faster than 99 percent of the world’s oceans, with consequences still unfolding for its vibrant ecosystem, home to whales, sharks, jellyfish, and hundreds of other species.
Co-led by acclaimed underwater photographer and visiting artist at MIT Sea Grant, Keith Ellenbogen, and MIT mechanical engineering PhD student, Andreas Mentzelopoulos, LOBSTgER is redefining scientific storytelling. Just as the 19th-century camera revolutionized our ability to capture the natural world, generative AI opens a new frontier in visual communication. It challenges traditional notions of authenticity while expanding our capacity to convey complex scientific and artistic perspectives.
A cornerstone of the LOBSTgER project is its dedication to data integrity. The generative models are exclusively trained on a carefully curated library of Ellenbogen’s original underwater photographs. Each image is a testament to artistic intent, technical precision, accurate species identification, and clear geographic context. This meticulous approach ensures that the AI-generated imagery maintains both visual integrity and crucial ecological relevance. Furthermore, Mentzelopoulos has developed custom code for LOBSTgER’s models, safeguarding the process and outputs from potential biases often found in external data or models, ensuring that the AI builds authentically upon real photography to deepen the public’s connection to the natural world.
Operating at the vibrant intersection of art, science, and technology, LOBSTgER draws from the visual language of photography, the rigorous observation of marine science, and the computational prowess of generative AI. This interdisciplinary approach, a hallmark of MIT’s innovative spirit, is not just creating new ways to visualize ocean life but also reimagining how environmental stories can be powerfully told. It serves as both a cutting-edge research tool and a bold creative experiment.
Underwater photography in New England’s coastal waters presents formidable challenges, including limited visibility, swirling sediment, bubbles, and the unpredictable movements of marine life. Ellenbogen, through his extensive project “Space to Sea: Visualizing New England’s Ocean Wilderness,” has spent years navigating these obstacles to build a comprehensive visual record of the region’s biodiversity. This vast dataset forms the robust foundation for training LOBSTgER’s generative AI models, capturing diverse angles, lighting conditions, and animal behaviors, resulting in an archive that is both artistically compelling and biologically accurate.
LOBSTgER’s custom diffusion models are engineered to not only replicate the biodiversity documented by Ellenbogen but also to internalize and reproduce his unique artistic style. By learning from thousands of real underwater images, the models assimilate intricate details like natural lighting gradients, species-specific coloration, and even the atmospheric texture created by suspended particles and refracted sunlight. The resulting imagery is not just visually accurate; it evokes an immersive and moving experience.
These advanced models offer dual capabilities: they can unconditionally generate new, synthetic, yet scientifically accurate images, and they can conditionally enhance real photographs through image-to-image generation. This allows Ellenbogen to recover lost detail in turbid water, adjust lighting to highlight key subjects, or even simulate scenes that would be nearly impossible to capture in the field. This hybrid methodology is designed to streamline the curation process, empowering storytellers to construct richer, more coherent visual narratives of life beneath the surface. The team believes this approach holds significant promise for other underwater photographers and image editors facing similar environmental constraints.
In a key series, Ellenbogen free-dived in coastal waters to capture high-resolution images of lion’s mane jellyfish, blue sharks, American lobsters, and ocean sunfish (Mola mola). He notes that “Getting a high-quality dataset is not easy. It requires multiple dives, missed opportunities, and unpredictable conditions. But these challenges are part of what makes underwater documentation both difficult and rewarding.” Mentzelopoulos, complementing this field work, has developed original code to train LOBSTgER’s family of latent diffusion models, a complex process requiring hundreds of hours of computation and meticulous hyperparameter tuning.
The project embodies a synergistic parallel process: Ellenbogen’s field documentation through photography captures rare and fleeting encounters with marine animals, while Mentzelopoulos’s lab work translates these moments into machine-learning contexts that can extend and reinterpret the visual language of the ocean. “The goal isn’t to replace photography,” Mentzelopoulos clarifies. “It’s to build on and complement it — making the invisible visible, and helping people see environmental complexity in a way that resonates both emotionally and intellectually. Our models aim to capture not just biological realism, but the emotional charge that can drive real-world engagement and action.”
LOBSTgER heralds a hybrid future that seamlessly merges direct observation with technological interpretation. The team’s long-term objective is to develop a comprehensive model capable of visualizing a broad spectrum of species found in the Gulf of Maine, with aspirations to extend these innovative methods to marine ecosystems worldwide. The researchers assert that photography and generative AI exist on a continuum, not in conflict. Photography captures the ‘what is’ – the real textures, lights, and animal behaviors – while AI expands this vision beyond the seen, towards what can be understood, inferred, or imagined based on scientific data and artistic insight. Together, they form a powerful framework for science communication through compelling imagery.
In a region experiencing rapid ecological change, the act of visualization transcends mere documentation; it becomes a vital tool for fostering awareness, engagement, and ultimately, conservation. LOBSTgER is still in its early stages, and the team is eager to share further discoveries, images, and insights as this transformative project continues to evolve.
