Samsung backs a video AI startup that can analyze thousands of hours of footage

While many AI tools can currently summarize video content, a significant challenge arises when asking models to analyze multiple videos or footage spanning thousands of hours. This limitation has notably impacted industries such as security, which needs to scrub through extensive surveillance footage, and marketing, which seeks to analyze numerous video campaigns and product shoots.

Addressing this critical gap, Memories.ai, an innovative AI startup, has developed a platform capable of processing up to 10 million hours of video. The company aims to provide a sophisticated contextual layer for large video datasets, complete with searchable indexing, intelligent tagging, segmentation, and aggregation, offering a comprehensive solution for businesses with extensive video analysis needs.

The startup was co-founded by Dr. Shawn Shen, a former research scientist at Meta’s Reality Labs, and Enmin (Ben) Zhou, previously a machine learning engineer at Meta. Dr. Shen highlighted the current limitations of leading AI models, stating, “All top AI companies, such as Google, OpenAI, and Meta, are focused on producing end-to-end models. Those capabilities are good, but these models often have limitations around understanding video context beyond one or two hours.”

Shen further elaborated on their inspiration, noting, “But when humans use visual memory, we sift through a large context of data. We were inspired by this and wanted to build a solution to understand video across many hours better.”

In a significant show of investor confidence, Memories.ai recently secured $8 million in a seed funding round. The round was led by Susa Ventures, with notable participation from Samsung Next, Fusion Fund, Crane Ventures, Seedcamp, and Creator Ventures. Initially targeting $4 million, the funding round was oversubscribed due to strong investor interest, doubling their initial goal.

Misha Gordon-Rowe, a partner at Susa Ventures, underscored their investment thesis: “Shen is a highly technical founder, and he is obsessed with pushing boundaries of video understanding and intelligence. Memories.ai can unlock a lot of first-party visual intelligence data with its solution. We felt that there was a gap in the market for long context visual intelligence, which attracted us to invest in the company.”

Samsung Next’s investment was driven by a slightly different vision, foreseeing the broader utility of Memories.ai’s solution for consumers. Sam Campbell, a partner at Samsung Next, explained, “One thing we liked about Memories.ai is that it could do a lot of on-device computing. That means you don’t necessarily need to store video data in the cloud. This can unlock better security applications for people who are apprehensive of putting security cameras in their house because of privacy concerns.”

Memories.ai employs its proprietary tech stack and models for video analysis. Its process involves an initial phase of noise removal, followed by a compression layer that retains only essential data. This optimized data then enters an indexing layer, enabling natural-language search queries, segmentation, and tagging. Finally, an aggregation layer summarizes the indexed data, facilitating comprehensive report generation.

Currently, Memories.ai primarily serves marketing and security companies. Marketing firms utilize the platform to identify brand trends on social media and guide video content creation. For security companies, the tool assists in analyzing surveillance footage to detect and reason through patterns indicative of potentially dangerous actions.

Looking ahead, the startup plans to streamline content syncing by enabling clients to create shared drives, allowing for easier uploads of video libraries for analysis. Shen envisions a future where users can ask complex queries like, “Tell me all about people I interviewed in the last week.” He also foresees the technology’s application in training humanoid robots for intricate tasks and enhancing self-driving cars’ navigational memory.

With a current team of 15 employees, Memories.ai plans to allocate the newly acquired funds towards augmenting its team and enhancing its search capabilities. The company operates in a competitive landscape, contending with startups like mem0 and Letta, which are developing AI memory layers with limited video support, as well as established players such as TwelveLabs and Google, which are actively advancing AI video understanding.

Despite the competition, Shen remains confident in Memories.ai’s distinctive advantage, emphasizing that their solution is more horizontal, allowing for seamless integration with various existing video models.