You’d be hard-pressed to find a cybersecurity provider that doesn't tout its AI-based defenses as a panacea. But AI defenses are by no means a silver bullet – and shouldn’t be treated as one.
AI Data Poisoning: A Hidden Threat on the Horizon
AI data poisoning is not a theoretical concept, but a tangible threat. The newly updated OWASP for LLMs lists Training Data Poisoning as a Top 10 vulnerability (LMM03). Model creators do not have full control of the data input into an LLM. Cybercriminals exploit this fact by using poisoned data to bypass AI-based security defenses, leading them astray and teaching them to make the wrong decisions. This deliberate manipulation becomes a silent accomplice, providing an open door to exploit unsuspecting systems.
Imagine you've got this super-smart AI model tasked with spotting anomalies in your system. What if someone were to sneak in data during its training or refinement, deliberately teaching it to ignore real threats?
For the attacker, it's all about disguising the data to look legitimate. In some cases, bad actors will use real data and only tweak a few numbers – anything to fool the AI into thinking it's legit. In essence, data is used to teach AI to make bad decisions.
Fake It 'Til You Break It: Bypassing AI Security Using Harvested Data
In a more sophisticated twist, sometimes the data used to trick AI actually is the real deal. In this scenario, adversaries harvest real data (typically stolen) and replay it to bypass AI models. One way this works is by using harvested digital fingerprints along with recorded mouse movements and gestures that are hard-coded and randomized to develop an automated script. While the data is technically real, it is inauthentic because it wasn’t originally generated by the person using it.
