Merging design and computer science in creative ways

In an era defined by rapid technological advancement, visionary researchers are continually finding novel applications for emerging tools. Among them is Alexander Htet Kyaw, an MIT MAD Fellow and a graduate student concurrently pursuing master’s degrees in architectural studies in computation and in electrical engineering and computer science.

Kyaw’s work stands at the forefront of human-AI interaction, creatively combining artificial intelligence (AI), augmented reality (AR), and robotics with gesture, speech, and object recognition. His innovations hold the potential to redefine how we engage with our built environments, transform retail experiences, facilitate complex structural design, and enable the creation of physical objects.

One of his notable achievements is Curator AI, a pioneering platform for which Kyaw and his MIT graduate student partners—Richa Gupta (architecture), Bradley Bunch, Nidhish Sagar, and Michael Won (all from EECS)—secured first prize, including $26,000 in OpenAI products and cash, at the MIT AI Conference’s AI Build: Generative Voice AI Solutions hackathon. Curator AI revolutionizes online furniture shopping by offering context-aware product recommendations. The platform leverages AR to accurately measure room dimensions and identify existing furniture. Users can then articulate their furnishing preferences via natural language, prompting a vision-language AI model to search for and display suitable options that align with both their verbal cues and the room’s visual characteristics. “Shoppers can choose from the suggested options, visualize products in AR, and use natural language to ask for modifications to the search, making the furniture selection process more intuitive, efficient, and personalized,” Kyaw explains. He highlights that Curator AI addresses the common challenge of not knowing where to begin when furnishing a space, providing intelligent, contextual recommendations. While initially developed for furniture, the underlying technology holds broad applicability across various markets.

Another impactful project from Kyaw is Estimate, developed with graduate student teammates Guillaume Allegre, May Khine, and Anna Mathy during the MIT Sloan Product Tech Conference’s March 2024 hackathon. Focused on aiding small businesses, Estimate emerged from collaboration with a Cambridge-based painting company. This innovative tool uses AR and object-recognition AI to precisely measure rooms and generate detailed cost estimates for renovations or paint jobs. It further employs generative AI to visualize post-renovation room appearances and automates invoice generation upon project completion. The team’s efforts earned them the hackathon’s top prize of $5,000.

Kyaw is set to deliver a TedX talk at his alma mater, Cornell University, in April, where he will delve deeper into Curator AI, Estimate, and other projects that harness AI, AR, and robotics for design and fabrication.

Among these initiatives is Unlog, a system that connects AR with gesture recognition to map the dimensions of building components by interpreting fingertip touch in physical space or even in the air. This technology underpinned the creation of Unlog, a striking art sculpture on the Cornell campus constructed from ash logs. Unlog embodies the potential for direct construction from whole logs, bypassing traditional lumber mill processes and promoting sustainable material use. A research paper detailing this work, titled “Gestural Recognition for Feedback-Based Mixed Reality Fabrication a Case Study of the UnLog Tower,” was co-authored by Kyaw, Leslie Lok, Lawson Spencer, and Sasa Zivkovic and published in the Proceedings of the 5th International Conference on Computational Design and Robotic Fabrication in January 2024.

Further demonstrating his ingenuity, Kyaw developed a system that integrates physics simulation, gesture recognition, and AR for designing active bending structures from bamboo poles. This allows users to manipulate digital bamboo modules in AR, with physics simulations illustrating how the bamboo bends and where to attach poles to ensure structural stability. This research was featured in the Proceedings of the 41st Education and Research in Computer Aided Architectural Design in Europe in August 2023, under the title “Active Bending in Physics-Based Mixed Reality: The Design and Fabrication of a Reconfigurable Modular Bamboo System.”

Last year, Kyaw presented a similar concept—deployable bamboo modules for shelters—to MITdesignX, an MIT MAD program supporting promising startups. He subsequently founded BendShelters, a venture dedicated to building prefabricated, modular bamboo shelters and community spaces for refugees and displaced persons in Myanmar, his home country. “Where I grew up, in Myanmar, I’ve seen a lot of day-to-day effects of climate change and extreme poverty,” Kyaw shares. “There’s a huge refugee crisis in the country, and I want to think about how I can contribute back to my community.” BendShelters has garnered recognition from MIT Sandbox, the PKG Social Innovation Challenge, and the Amazon Robotics’ Prize for Social Good.

At MIT, Kyaw is currently collaborating with Professor Neil Gershenfeld, director of the Center for Bits and Atoms, and PhD student Miana Smith. Their joint effort focuses on creating accessible, on-demand, and sustainable object fabrication workflows using speech recognition, 3D generative AI, and robotic arms. Kyaw holds bachelor’s degrees in architecture and computer science from Cornell University. His innovative spirit was further recognized last year with an SJA Fellowship from the Steve Jobs Archive, providing funding for projects at the nexus of technology and the arts.

“I enjoy exploring different kinds of technologies to design and make things,” Kyaw states, articulating his core motivation. “Being part of MAD has made me think about how all my work connects, and helped clarify my intentions. My research vision is to design and develop systems and products that enable natural interactions between humans, machines, and the world around us.” Kyaw’s relentless pursuit of merging design sensibilities with advanced computational methods promises a future where technology seamlessly integrates into and enhances our daily lives and physical environments.