Google AI Platform Vulnerabilities Expose Proprietary Enterprise Language Models

Photo credit: www.darkreading.com

Google has addressed two vulnerabilities in Vertex AI, its platform designed for the creation and deployment of large language models (LLMs), which could have let malicious actors steal confidential enterprise models. This incident underscores the ongoing security challenges posed by the malicious exploitation of artificial intelligence (AI) technologies for businesses.

The flaws were uncovered by researchers from Palo Alto Networks Unit 42, focusing on the capabilities of Google’s Vertex AI platform. This machine learning (ML) framework enables businesses to train and implement ML models and AI applications tailored to their needs.

Unit 42 identified a privilege escalation vulnerability within the “custom jobs” functionality and a model exfiltration issue in the “malicious model” capability, as outlined in a blog post published on November 12.

___________________________________

The first vulnerability allowed users with custom job access to exploit project permissions and obtain unauthorized access to associated data services. The second flaw could enable an intruder to deploy a compromised model within Vertex AI, which would risk “the exfiltration of all other fine-tuned models, representing a significant threat to proprietary and sensitive data,” according to the Palo Alto Networks team.

After sharing their findings with Google, the company promptly took steps to resolve the specific issues identified in Vertex AI as hosted on the Google Cloud Platform (GCP).

Despite the prompt mitigation, these vulnerabilities illustrate the ongoing risks associated with the potential misuse of LLMs, highlighting how quickly an issue can escalate. The researchers emphasized, “This research underscores that a single deployment of a malicious model could jeopardize an entire AI environment.” They noted that even one unchecked model activated in a live system could be used to extract sensitive data, culminating in severe model exfiltration threats.

Related: 5 Ways to Save Your Organization From Cloud Security Threats

___________________________________

Poisoning Custom LLM Development

The vulnerabilities stemmed from a feature in Vertex AI known as Vertex AI Pipelines. This tool facilitates the tuning of models using custom jobs, sometimes referred to as “custom training jobs.” “Custom jobs consist of code that operates within the pipeline and has the potential to modify models in diverse manners,” the researchers detailed.

While this adaptability provides significant benefits, it also creates opportunities for exploitation. The researchers utilized permissions associated with a “service agent” identity linked to a “tenant project,” connecting it to a “source project” containing fine-tuned AI models. A service agent possesses extensive permissions within a Vertex AI project.

Leveraging this access, the researchers were able to inject commands or develop a custom image that created a backdoor, granting them entry into the custom model development space. This led to the deployment of a poisoned model for experimentation, which subsequently allowed them to steal other AI and ML models from the test environment.

Related: Amazon Employee Data Compromised in MOVEit Breach

“In summary, by deploying a malicious model, we accessed resources within tenant projects that let us view and export all models deployed across the project,” the researchers stated. This compilation comprises both ML and LLM models along with their fine-tuned components.

This transforms into a “model-to-model infection scenario,” the researchers cautioned. “For instance, a team might inadvertently deploy a harmful model sourced from a public repository,” they elaborated. “Once activated, it could exfiltrate all ML and fine-tuned LLM models present in the project, endangering the organization’s most confidential assets.”

Mitigating AI Cybersecurity Risk

As organizations begin to utilize tools that enable the creation of custom LLM-based AI systems, the associated security risks and mitigation strategies are still relatively unexplored. It has, however, become evident that unauthorized access to internally developed LLMs poses a significant risk of exposure.

The primary security measure emphasized by Unit 42 is to strictly regulate the permissions granted to individuals within the organization who have access to these models. They point out that seemingly harmless permissions for deploying a model could actually provide an entry point to other models in a vulnerable project.

To safeguard against these threats, organizations should establish rigorous control protocols for model deployments. A crucial strategy involves ensuring that development and testing environments remain distinct from the live production setting.

“This separation minimizes the likelihood that an attacker will access potentially insecure models that haven’t been thoroughly vetted,” researchers Balassiano and Shaty remarked. “It’s imperative to validate every model prior to deployment, irrespective of whether it originates from an internal team or a third-party repository.”

Don’t miss the upcoming free Dark Reading Virtual Event, “Know Your Enemy: Understanding Cybercriminals and Nation-State Threat Actors,” on November 14 at 11 a.m. ET. The event will feature sessions on understanding MITRE ATT&CK, employing proactive security measures, and mastering incident response, along with insights from leading experts like Larry Larsen of Navy Federal Credit Union, ex-Kaspersky analyst Costin Raiu, and Ben Read of Mandiant Intelligence. Register now!

Source
www.darkreading.com