OpenAI Thwarts Cyber Ops Leveraging ChatGPT for Malware and Recon

OpenAI has dismantled over 20 cyber operations this year, exposing activities where threat actors exploited AI models to enhance malware capabilities, conduct reconnaissance, and even evade detection. While these disruptions involved various cyber operations, the most notorious were from threat actors like the China-based SweetSpecter, the Iranian-affiliated CyberAv3ngers, and the Iranian malware development group STORM-0817.

OpenAI’s latest report highlights that, although AI is a powerful tool for defending against cyber threats, it is increasingly being leveraged by attackers in the intermediate stages of their operations. These stages typically occur after acquiring basic tools (internet access, email accounts) but before the execution of final products like malware or fake content. OpenAI's defensive capabilities were crucial in catching these activities, with AI models allowing quick detection of abnormal patterns.

Major cyber-threats identified

One of the key findings is the use of AI by SweetSpecter, a China-linked group active since 2023. This actor was detected using OpenAI's models for vulnerability research, scripting support, and reconnaissance, all while targeting OpenAI employees in spear-phishing attacks. The spear-phishing emails aimed to deploy a malicious LNK file disguised as a ChatGPT-related error message. If opened, the malware known as SugarGh0st RAT would take control of the target machine, enabling actions like command execution and data exfiltration. OpenAI's internal security teams blocked the emails before they could cause damage, emphasizing the role of real-time defense.

Another significant case involved CyberAv3ngers, an Iranian Islamic Revolutionary Guard Corps (IRGC)-affiliated group notorious for targeting industrial control systems (ICS). This group attempted to exploit programmable logic controllers (PLCs) in critical infrastructures like water systems and energy plants, as seen in attacks on water services in Pennsylvania and Ireland in late 2023. Much of their activity involved querying OpenAI’s models for help in writing scripts, debugging code, and refining exploits, yet these efforts didn’t lead to major new capabilities.

AI in malware development

OpenAI also disrupted STORM-0817, a malware development operation from Iran. This group used AI models to debug malware, develop Android surveillance tools, and scrape social media platforms. Their Android malware could collect sensitive information, including contacts, call logs, device IMEI numbers, and even screenshots from compromised devices. Moreover, STORM-0817 sought assistance from AI to develop its malware’s server-side infrastructure and scrape Instagram for targeted data.

Despite these attacks, OpenAI stresses that its models have not given malicious actors groundbreaking new abilities. Instead, the models provided incremental enhancements, making the attackers more efficient but not necessarily more capable than with publicly available tools. This trend suggests that AI’s role in cyberattacks, while concerning, does not yet represent a quantum leap in the attackers' sophistication.

Misinformation and influence campaigns

The report also touches on misinformation campaigns, presenting details about how OpenAI disrupted several networks that used its models to generate content for political influence, including election-related posts in the U.S., Rwanda, and India. However, these operations failed to gain substantial traction, demonstrating the limited reach of AI-generated misinformation in these cases.