Photo credit: venturebeat.com
Meta has recently announced a collaboration with Cerebras Systems to enhance its new Llama API. This initiative aims to provide developers with inference speeds that could surpass traditional GPU-based solutions by up to 18 times.
During the inaugural LlamaCon developer conference in Menlo Park, this announcement positioned Meta to effectively rival leading companies like OpenAI, Anthropic, and Google in the burgeoning AI inference service sector. This market is characterized by developers purchasing tokens in the billions to power their applications.
“Meta has chosen Cerebras to deliver the rapid inference essential for developers through the Llama API,” stated Julie Shin Choi, Cerebras’ chief marketing officer. “This partnership marks an exciting venture for us with our first CSP hyperscaler collaboration designed to provide ultra-fast inference capabilities to developers.”
This partnership signifies Meta’s official entry into the realm of AI computation commerce, transforming its widely-used open-source Llama models into a marketable service. With over one billion downloads, Meta has yet to provide a proprietary cloud infrastructure for developers aiming to create applications using these models.
“The excitement surrounding this development transcends our collaboration with Cerebras,” added James Wang, a senior executive at Cerebras. “OpenAI, Anthropic, and Google have successfully built a new AI business from the ground up; the AI inference business. Developers are investing in tokens at astounding rates—sometimes reaching billions. These tokens are the new compute instructions necessary for developing AI applications.”
Breaking the speed barrier: How Cerebras supercharges Llama models
Meta’s Llama API stands out due to the exceptional speed improvements achieved through Cerebras’ specialized AI chips. Benchmarks indicate that the Cerebras system can process Llama 4 at an impressive 2,600 tokens per second, while competitors like SambaNova and Groq manage only 747 and 600 tokens per second, respectively. Google’s GPU-based services fall short as well.
“When comparing API performance, competitors like Gemini and GPT operate at GPU speeds of around 100 tokens per second,” Wang elaborated. “While adequate for simple chat applications, this speed is inadequate for more complex reasoning tasks and real-time operations, a struggle many face today.”
This extraordinary speed opens up new possibilities for applications previously deemed impractical. These innovations include real-time agents, low-latency voice systems, and interactive code generation—all requiring an agile response that now takes seconds rather than minutes.
The Llama API signifies a pivotal shift in Meta’s AI strategy, evolving from merely providing models to becoming a comprehensive AI infrastructure provider. Through this API service, Meta generates revenue from its AI endeavors while maintaining its commitment to open-source models.
“Meta has entered the token-selling business, reflecting positively on the AI ecosystem in the U.S.,” Wang noted. “They bring substantial value to the table.”
The API will also provide tools for model fine-tuning and evaluation, beginning with the Llama 3.3 8B model. Developers can produce and test custom data sets, ensuring quality in their models. Notably, Meta assures that it will not utilize customer data to further its own models, allowing for models generated through the Llama API to be transitioned to other platforms, distinguishing itself from less flexible competitors.
To facilitate this service, Cerebras will utilize its extensive network of data centers across North America, including locations in Texas, Oklahoma, Minnesota, Montreal, and California.
“Currently, all our data centers dedicated to inference are based in North America,” Choi clarified. “We will harness the full capacity available at Cerebras to support Meta, balancing workloads across our various facilities.”
This collaboration mirrors what Choi referred to as “the classic compute provider to a hyperscaler” model, reminiscent of Nvidia’s role in supplying hardware to major cloud services. “They are reserving sections of our compute power to cater to their developer community,” she added.
Furthermore, Meta has also partnered with Groq to offer developers rapid inference alternatives, expanding high-performance options beyond the realm of traditional GPU-based systems.
Meta’s introduction into the inference API market, paired with its enhanced performance capabilities, has the potential to challenge the current landscape dominated by OpenAI, Google, and Anthropic. By leveraging the success of its open-source models and the newly achieved rapid inference capabilities, Meta is positioning itself as a strong competitor in the commercial AI market.
“Meta’s unique attributes include a user base of 3 billion, extensive data centers, and a large developer ecosystem,” as highlighted in Cerebras’ presentation. The integration of Cerebras technology allows Meta to enhance performance by an estimated 20 times compared to its rivals.
This collaboration signifies a vital achievement for Cerebras, validating its approach to specialized AI hardware. “We’ve focused on developing this wafer-scale engine for several years, ensuring that once it was integrated into a hyperscale cloud, it would represent the culmination of our commercial strategy,” emphasized Wang.
The Llama API is currently in a limited preview phase, with plans for a broader rollout in the near future. Developers interested in accessing rapid Llama 4 inference can request early access by selecting Cerebras from the model options within the Llama API.
“For any developer unfamiliar with Cerebras, they can simply navigate Meta’s software SDK, generate an API key, and choose the Cerebras option to process their tokens using a robust wafer-scale engine,” Wang noted. “This collaboration enables us to seamlessly integrate into Meta’s extensive developer ecosystem, which is immensely beneficial for us.”
Meta’s selection of specialized silicon reflects a significant shift in AI’s trajectory, emphasizing that in future developments, the speed of processing may be as crucial as the information itself. In this evolving landscape, speed transcends being merely beneficial—it becomes fundamental.
Source
venturebeat.com