Hardly a day goes by lately without news of AI agents autonomously hacking websites or AI tools being used by cybercriminals and scammers. But a recently patched vulnerability in the macOS version of OpenAI’s ChatGPT underscores the potential value to attackers of compromising AI software itself as these apps proliferate more and more.

The bug could have been exploited to essentially take over ChatGPT on a victim’s computer, giving an attacker access to all the chat logs and other data stored by the app, as well as interconnections like browser sessions. Discovered by researchers at the Objective-See Foundation, the vulnerability illustrates the deep system access and trust that AI platforms are afforded in order to work—and the target that this puts on their backs.

“Agents need a lot of access to do their job,” says Objective-See Foundation software analyst and longtime macOS researcher Patrick Wardle. “They are like the building manager who has access to the keys to all the rooms. So if they can be corrupted or subverted, that’s super problematic. It can mean that unprivileged code could then potentially have access to all the things.”

OpenAI publicly acknowledged the security flaw and fix in its system change log on September 25. “We continue to evolve our security practices, but recognize a need to move faster,” OpenAI spokesperson Shane Bauer told WIRED in a statement.

The ChatGPT macOS app includes multiple components that communicate with each other securely by checking for digital signatures. The idea is to confirm with these validity checks that both processes are OpenAI components and not outside, potentially malicious software making a request. And the system design goes so far as to require these signature checks at three layers of remove from the request, to ensure that malicious software isn’t somehow directing an OpenAI component to be a proxy and make a seemingly trusted request.

Objective-See Foundation researchers found, though, that there is a trusted component, a script interpreter, that would accept an untrusted script (or list of commands to run) and could then be manipulated to deliver this script into the main ChatGPT process. “They also check the parent and grandparent of that process, but the malicious script just spawns the script interpreter three times and then makes the request so it will satisfy the requirements,” Wardle says. The flaw could only be exploited by an attacker who already had malware installed on a target machine.

The vulnerability was “insanely trivial” to exploit, he adds, and his proof of concept only required about a dozen lines of code. In addition to accessing ChatGPT chat logs, the vulnerability could also be used to get ChatGPT to run commands for the attacker, such as accessing a browser or other sensitive applications, with the requests appearing as legitimate instructions issued by the OpenAI software.