Photo credit: venturebeat.com
Stay informed with the latest developments in AI and beyond through our daily and weekly newsletters. Sign up for exclusive insights and industry-leading coverage.
Enterprise retrieval augmented generation (RAG) continues to play a crucial role in the rapidly expanding field of agentic AI. In light of the ongoing fascination with AI agents, Cohere has unveiled its latest embeddings model, which brings enhancements such as longer context windows and increased multimodality.
The newly released Embed 4 builds upon the advancements made in Embed 3, now offering augmented features for handling unstructured data. With a context window capable of accommodating 128,000 tokens, organizations can effectively produce embeddings for extensive documents, approximately equivalent to 200 pages.
“Current embedding models struggle to inherently comprehend complex multimodal business datasets, prompting companies to create cumbersome data pre-processing systems that yield only marginal improvements in accuracy,” Cohere remarked in a blog post. “Embed 4 addresses these challenges, empowering enterprises and their workforce to efficiently extract insights hidden within vast amounts of unsearchable data.”
Furthermore, enterprises have the flexibility to deploy Embed 4 on secure virtual private clouds or within their on-premise technology frameworks for enhanced data protection.
Organizations can utilize embeddings to convert their documents and other data into numerical forms suitable for RAG applications. By referencing these embeddings, AI agents can provide responses to user prompts effectively.
Expertise in Regulated Industries
Cohere asserts that Embed 4 is particularly effective in tightly regulated sectors such as finance, healthcare, and manufacturing. The company highlights that its models are designed with the security requirements of these industries in mind and possess a comprehensive understanding of business operations.
The training of Embed 4 emphasizes resilience against the typical imperfections found in real-world data, ensuring that it remains accurate even when dealing with issues like typographical errors and formatting inconsistencies. Cohere noted that the model excels in processing scanned documents and handwritten texts, common in legal documents, insurance paperwork, and expense receipts. This capacity minimizes the necessity for complicated data preparation processes, thereby saving businesses considerable time and costs.
Embed 4 can be applied in various contexts including investor presentations, due diligence documentation, clinical trial summaries, repair manuals, and product specifications. Additionally, it maintains support for over 100 languages, consistent with its predecessor.
Agora, a Cohere customer, has successfully utilized Embed 4 for its AI search engine, discovering that the model significantly enhances the relevancy of product results.
“E-commerce data presents unique challenges due to its complexity, which encompasses both images and intricate text descriptions. The ability to consolidate our products into a single embedding not only quickens our search capability but also boosts the efficiency of our internal tools,” stated Param Jaggi, Founder of Agora, in a blog post.
Applications for AI Agents
Cohere champions the potential of models like Embed 4 in enhancing agentic applications, claiming it could serve as “the optimal search engine” for AI agents and assistants within enterprises.
Additionally, Embed 4 generates compressed data embeddings to reduce storage expenses significantly.
The combination of embeddings and RAG searches allows agents to reference specific documents, enhancing their ability to fulfill task-related queries accurately. Many in the industry view this approach as a way to deliver more reliable results, minimizing the risk of inaccuracies or fabricated responses.
Notably, other companies competing in this space include Qodo with its Qodo-Embed-1-1.5B model and Voyage AI, which was recently acquired by the database provider MongoDB.
Source
venturebeat.com