Photo credit: arstechnica.com
In discussions surrounding the integration of source citation capabilities in AI models, Willison emphasizes the importance of verifying accuracy through proper citation. However, he also highlights the challenges inherent in creating effective systems for this purpose. The introduction of Citations seems to represent progress toward enhancing the reliability of AI outputs by incorporating retrieval-augmented generation (RAG) capabilities directly into the model.
This feature isn’t entirely new; as noted by Anthropic’s Alex Albert in a post on X, the Claude model has been designed with the ability to cite sources from its inception. With the launch of Citations, developers can now leverage this capability more easily by utilizing a new parameter— “citations: {enabled:true}”—when sending documents through the API.
Initial Feedback from Users
Anthropic has made Citations available for its Claude 3.5 Sonnet and Claude 3.5 Haiku models through both the Anthropic API and Google Cloud’s Vertex AI platform, where they are reportedly starting to see practical applications.
For instance, Thomson Reuters, which employs Claude for its legal AI reference platform known as CoCounsel, has expressed enthusiasm about utilizing Citations. They believe it will significantly reduce the risk of misinformation, commonly referred to as “hallucination,” while enhancing the trustworthiness of AI-generated responses.
Similarly, Endex, a financial technology firm, shared with Anthropic that the introduction of Citations completely eliminated incorrect sourcing in their outputs, which had been a frequent issue, previously affecting 10 percent of their responses. This improvement also translated into a 20 percent increase in the number of references provided in responses, as reported by CEO Tarun Amasa.
While these initial reports are promising, experts caution that depending on large language models (LLMs) for accurate citation and referencing still poses some risks. As this technology continues to evolve, further comprehensive studies will be necessary to establish its reliability and robustness in various fields.
In terms of pricing, Anthropic will maintain its standard token-based model for users. Notably, any quoted material included in the AI’s responses will not be counted towards the output token costs. For example, referencing a 100-page document would be priced at approximately $0.30 with the Claude 3.5 Sonnet model or $0.08 with Claude 3.5 Haiku, based on the standard API pricing structure provided by Anthropic.
Source
arstechnica.com