Key Takeaways
- New generative AI technologies, like chatbots, bring unique security risks such as prompt injection and data leaks.
- Exploiting these AI weaknesses has already led to significant intellectual property concerns and potential financial impacts.
- Large language models (LLMs) can act like “black boxes,” making it difficult to secure specific data within them.
- Defending against AI threats involves both traditional cybersecurity measures and using specialized AI systems to monitor user interactions.
- Smaller, more cost-effective AI models are being explored as security watchdogs for larger, more expensive LLMs.
Generative artificial intelligence is a rapidly evolving field, and with it come new security headaches that can catch businesses unprepared.
Chatbots built on large language models (LLMs) are susceptible to fresh types of attacks. These include “prompt injections,” where tricky prompts manipulate the AI’s behavior, and “data exfiltration,” which involves massive numbers of queries aimed at uncovering sensitive information.
These attacks leverage the complex and sometimes unpredictable nature of LLMs, and they’ve already caused real financial concern.
Chuck Herrin, a security expert at F5, highlighted a major incident involving alleged intellectual property theft from OpenAI. He noted an accusation that a company named DeepSeek used OpenAI’s ChatGPT to train its own model, a technique called “distillation.”
According to Business Insider, Herrin mentioned that news of this alleged IP theft significantly impacted the stock market, illustrating the high stakes involved.
It’s widely known that exploiting AI vulnerabilities is feasible. While models don’t usually “memorize” training data perfectly, repeatedly querying them can allow attackers to mimic their behavior or glean insights into the data they were trained on.
Herrin explained to Business Insider that securing the AI means securing the application programming interface (API) and the entire system around it. Without proper safeguards, APIs become open doors for exploitation.
Adding to the difficulty, LLMs operate like “black boxes.” We know they learn patterns from data, but it’s hard to pinpoint exactly how they arrive at specific answers. This makes it nearly impossible to restrict access to certain information within the model like you could with a traditional database.
Sanjay Kalra from Zscaler compared this to databases where data can be deleted. He told Business Insider that with AI chatbots, easily removing specific information isn’t currently an option.
So, how are companies tackling these AI security issues? The solution involves a mix of old and new approaches.
Fundamental cybersecurity practices remain crucial. Herrin emphasized the importance of basics like controlling who can access the AI (authentication and authorization).
Kalra echoed this, stressing the need for access controls and logging user activity. For example, you might want an AI tool available only to engineers, not marketing or sales.
Interestingly, the other key part of the solution is using more AI.
Because LLMs are hard to secure directly due to their black-box nature, cybersecurity firms are training specialized AI “watchdogs.” These security AIs sit between the user and the main LLM, scanning prompts and responses for suspicious activity like data extraction attempts or efforts to bypass safety rules.
“It takes a good-guy AI to fight a bad-guy AI,” Herrin said, describing it as an ongoing arms race. His company, F5, offers services using purpose-built LLMs to detect these attacks.
However, this AI-on-AI approach isn’t without challenges, especially cost. Using powerful models like GPT-4 for security can be prohibitively expensive.
“The insurance can’t be more expensive than the car,” Kalra noted, suggesting that using large models to protect other large models isn’t practical for most. Instead, the industry is looking towards smaller language models (SLMs).
SLMs require less computing power to train and run. Kalra mentioned that Zscaler is using and developing smaller, specialized AI models for security tasks, making the approach more feasible.
As AI advances, companies find themselves needing the technology itself to defend against its own vulnerabilities. A layered strategy, combining solid cybersecurity fundamentals with vigilant AI security models, is emerging as the way forward.
