Photo credit: venturebeat.com
The review platform Yelp has long been a go-to resource for diners and consumers, leveraging technology to enhance user experiences for many years. Despite its early forays into machine learning, the company faced challenges in effectively integrating contemporary large language models into its offerings.
Yelp discovered that many of its users, particularly those who engaged with the app infrequently, struggled to utilize its AI features, including an AI-driven assistant designed to streamline searches.
“Building something visually appealing is straightforward, but creating a tool that is both attractive and genuinely useful is a significant challenge,” Craig Saldanha, Yelp’s chief product officer, articulated in a discussion with VentureBeat.
The rollout of Yelp Assistant, launched in April 2024 to a wider audience, highlighted these difficulties, as the engagement metrics for the AI tools saw an unexpected decline.
“We were initially encouraged by positive feedback from a beta test group; however, once the service became available to all users, we saw a drop in performance,” Saldanha noted. “Understanding the underlying reasons for this drop was a complex process.”
The issue arose from the realization that casual users, who might seek a new tradesperson or service, were not prepared for direct interaction with an AI.
Evolution of AI Features
Many users associate Yelp primarily with discovering restaurant reviews and viewing menu images. Personally, I rely on Yelp to assess new dining options and hear whether others share my views on a dish. It’s also useful for checking essential amenities at places like coffee shops where I might work for the day.
Saldanha reflects on Yelp’s long-term investment in AI, stating, “Our focus began around 2013-2014 with the intention of creating our own models for better query understanding.” This foundational understanding proved crucial for refining user search intents.
As technology progressed, Yelp tailored its investments toward identifying popular dishes through user-submitted photos and enhancing connections to services, thereby improving navigation within its platform.
The Yelp Assistant empowers users to find suitable professionals for their needs. Users can engage via chat, providing tasks in either pre-defined prompts or through typed messages. The assistant follows up with questions to clarify the task before crafting a message to potential service providers.
While Yelp encourages professional users to engage directly with clients, Saldanha acknowledged that larger companies often employ call centers to manage responses initiated by the AI assistant.
In addition to the assistant, Yelp unveiled Review Insights and Highlights that harness user sentiment analysis. By employing large language models (LLMs), the platform distills user reviews into sentiment scores, allowing for better communication of user experiences. A detailed prompt using GPT-4o is utilized to create datasets for this purpose, with additional fine-tuning from a GPT-4o-mini model.
The highlights feature extracts pertinent details from reviews through similar LLM prompts, although it operates primarily on GPT-4, supplemented by enhancements from GPT-3.5 Turbo. Yelp anticipates further updates incorporating GPT-4o and o1 moving forward.
Following a trend among firms enhancing their review features, Yelp joins others utilizing LLMs to refine search functionalities based on customer feedback. This mirrors efforts by companies such as Amazon with its Rufus AI assistant, designed to facilitate item recommendations.
Performance and Model Selection
For new functionalities, including its assistant, Yelp incorporated OpenAI’s GPT-4o alongside other models, remaining adaptable to different LLMs that may enhance user experience. Saldanha emphasized that Yelp’s extensive dataset is critical to their assistant’s effectiveness, thereby avoiding reliance on a single model.
“We draw from models provided by OpenAI, Anthropic, and others via AWS Bedrock,” he elaborated.
The company has established a framework to evaluate the effectiveness of various models based on criteria such as accuracy, relevance, and user safety. “The premium models tend to yield the best outcomes,” Saldanha noted, mentioning that the firm conducts preliminary trials with each model prior to assessing factors like cost and response speed.
User Education Initiatives
Yelp has made a dedicated effort to instruct both casual and dedicated users on how to leverage these new AI capabilities effectively. Saldanha highlighted the importance of creating a response tone that resonates with users, emphasizing that the AI should exhibit a human-like demeanor—avoiding responses that are either too rapid or too slow and balancing encouragement with professionalism.
“We invested considerable energy into ensuring users felt at ease, particularly with initial interactions. It took us several months to achieve an optimal response tone, which subsequently led to a noticeable uptick in user engagement,” Saldanha remarked.
This effort included training the Yelp Assistant to choose the right language and exude positivity. Following this extensive adjustment, the company is now observing improved engagement with its AI functionalities.
Source
venturebeat.com