Photo credit: arstechnica.com

Hugging Face Launches Open Source AI Research Agent

On Tuesday, Hugging Face announced the launch of an open-source AI research tool named “Open Deep Research.” This initiative emerged swiftly after OpenAI unveiled its own Deep Research feature, which allows users to autonomously browse the internet and generate comprehensive research reports. The goal of Hugging Face’s project is to replicate the capabilities of Deep Research while ensuring that the underlying technology remains accessible to developers.

In their announcement, Hugging Face highlighted a gap in OpenAI’s disclosure regarding the agentic framework that powers Deep Research. “While powerful LLMs are now freely available in open-source, OpenAI didn’t disclose much about the agentic framework underlying Deep Research,” the team noted. This spurred them to undertake a 24-hour challenge to recreate OpenAI’s results and make their findings publicly available.

Hugging Face’s approach parallels both OpenAI’s Deep Research and Google’s adaptation through their “Deep Research” program utilizing the Gemini model, which was initially introduced in December prior to OpenAI’s release. Their technology incorporates an “agent” framework into an existing AI model, enabling it to execute multi-step tasks. This includes the ability to gather information and gradually compile a report that is presented to the user at the end of the process.

Early results from Hugging Face’s Open Deep Research have shown promising performance metrics. Within just one day of operation, the tool achieved an accuracy of 55.15 percent on the General AI Assistants (GAIA) benchmark. This benchmark evaluates the AI’s proficiency in collecting and synthesizing data from various sources, with OpenAI’s Deep Research scoring slightly higher at 67.36 percent.

The GAIA benchmark includes intricate multi-step questions that challenge the capabilities of AI agents. One example question reads:

Which of the fruits shown in the 2008 painting “Embroidery from Uzbekistan” were served as part of the October 1949 breakfast menu for the ocean liner that was later used as a floating prop for the film “The Last Voyage”? Give the items as a comma-separated list, ordering them in clockwise order based on their arrangement in the painting starting from the 12 o’clock position. Use the plural form of each fruit.

Addressing such questions requires the AI agent to navigate multiple sources and integrate the information into a coherent response. The complexity of the GAIA questions makes them a rigorous test not only for AI systems but also for human researchers, underscoring the challenges inherent in agentic AI interactions.

Source
arstechnica.com