In April this year, a team of researchers from the University of Illinois Urbana-Champaign released a paper showing how they had been able to use an LLM (Large Language Model), GPT-4 in particular,  to "autonomously exploit one-day vulnerabilities in real-world systems." 

One-day vulnerabilities are security issues that are known about, but not yet patched.  When a vulnerability is discovered, it is given a number and put on the CVE (Common Vulnerabilities and Exposure) list, which also includes a description and severity level. 

The researchers showed that, when fed a CVE description, "GPT-4 is capable of exploiting 87% of these vulnerabilities compared to 0% for every other model we test (GPT-3.5, open-source LLMs) and open-source vulnerability scanners (ZAP and Metasploit)."  "Fortunately," they added. "our GPT-4 agent requires the CVE description for high performance: without the description, GPT-4 can exploit only 7% of the vulnerabilities. Our findings raise questions around the widespread deployment of highly capable LLM agents."

Teamwork makes the Dream work

Only two months later, the team released another paper.  Building on their previous research, they were able to harness teams of LLMs to successfully exploit real-world zero-day vulnerabilities.

Zero-Day vulnerabilities are security flaws that are not yet known about by the creators of the affected software or hardware (or are very freshly discovered) and not yet patched. Obviously, it's hard to defend a weakness you know nothing about, so threat actors are constantly on the lookout for them.

This time, the researchers used a new technique they call HPTSA (Hierarchical Planning and Task-Specific Agents) to organize a team of LLMs the same way you might organize a project team - with a Planner, a Manager, and a team of specialized Task-Specific Agents. The Planner identifies potential weaknesses and comes up with a plan of attack. The Manager then decides which Agents are best suited for the tasks, deploying and directing their work.

This model was tested on a set of vulnerabilities that the researchers knew about - but the LLMs were not given that information, mimicking a zero-day scenario.  The LLM team was able to successfully exploit over 50% of the zero-day vulnerabilities tested.

A whole new ballgame for Cybersecurity

Now that is is proven that threat actors can potentially use AI to autonomously hack websites, the defenders will need to keep pace. Luckily, the same method can be used to perform penetration testing, to test systems and spot zero-day vulnerabilities - and patch them before they are found by others.  It's easy to imagine that HPTSA will have a huge impact on not only cybersecurity, but in expanding the use of LLMs in unforeseen directions, for good or bad.

As the researchers themselves concluded:

It is unclear whether AI agents will aid cybersecurity offense or defense more and we hope that future work addresses this question.  Beyond the immediate impact of our work, we hope that our work inspires frontier LLM providers to think carefully about their deployments.

Sources:

Richard Fang, Rohan Bindu, Akul Gupta, and Daniel Kang. LLM Agents can Autonomously
Exploit One-Day Vulnerabilities. arXiv preprint arXiv:2404.08144, 2024. https://arxiv.org/abs/2404.08144

Richard Fang, Rohan Bindu, Akul Gupta, and Daniel Kang. Teams of LLM agents can Exploit Zero-Day Vulnerabilities.

arXiv preprint arXiv:2406.01637, 2024. https://arxiv.org/abs/2406.01637