Researchers needed less than 48 hours with Google’s new Gemini CLI coding agent to devise an exploit that made a default configuration of the tool surreptitiously exfiltrate sensitive data to an attacker-controlled server.

Gemini CLI is a free, open-source AI tool that works in the terminal environment to help developers write code. It plugs into Gemini 2.5 Pro, Google’s most advanced model for coding and simulated reasoning. Gemini CLI is similar to Gemini Code Assist except that it creates or modifies code inside a terminal window instead of a text editor. As Ars Senior Technology Reporter Ryan Whitwam put it last month, “It's essentially vibe coding from the command line.”

Gemini, silently nuke my hard drive

Our report was published on June 25, the day Google debuted the tool. By June 27, researchers at security firm Tracebit had devised an attack that overrode built-in security controls that are designed to prevent the execution of harmful commands. The exploit required only that the user (1) instruct Gemini CLI to describe a package of code created by the attacker and (2) add a benign command to an allow list.

Read full article

Comments

In the AI world, a vulnerability called a "prompt injection" has haunted developers since chatbots went mainstream in 2022. Despite numerous attempts to solve this fundamental vulnerability—the digital equivalent of whispering secret instructions to override a system's intended behavior—no one has found a reliable solution. Until now, perhaps.

Google DeepMind has unveiled CaMeL (CApabilities for MachinE Learning), a new approach to stopping prompt-injection attacks that abandons the failed strategy of having AI models police themselves. Instead, CaMeL treats language models as fundamentally untrusted components within a secure software framework, creating clear boundaries between user commands and potentially malicious content.

The new paper grounds CaMeL's design in established software security principles like Control Flow Integrity (CFI), Access Control, and Information Flow Control (IFC), adapting decades of security engineering wisdom to the challenges of LLMs.

Read full article

Comments