Steve Wilson, founder of the OWASP GenAI Security Project, deeply understands the challenges of keeping AI agents secure. He also has a front row seat to how organizations perceive and respond to these challenges.
Right now, he says, teams often fixate on the threat of large breaches and AI hacking tools that are much more powerful than internal security systems. “It’s fun to talk about the super scary things, so we get distracted by them,” Wilson says. As a result, this often pulls teams away from building a proper defense against more realistic threats.
Wilson has been mapping the most effective agentic security approaches and helping educate developers and CISOs on best practices. Many of these AI security practices are outlined in his book, “The Developer’s Playbook for Large Language Model Security.” He also works closely with LLMs and agentic models as the Chief AI and Product Officer at Exabeam.
In this Q&A, Wilson describes the mistakes CISOs are making with agentic security, how teams can build resilience with the agents they have now, and how a famous Bruce Lee quote offers poignant advice for a changing security landscape.
Let’s start with solving the tractable problem, which is that you’ve got a bunch of stuff on your network that you haven’t cataloged, categorized, monitored, or tested.
Steve Wilson, co-founder of the OWASP GenAI Security ProjectIt’s funny that you ask. If you’re doing martial arts and you’re sparring, there are two failure modes.
The first is what happens to everybody when they’re a beginner. You’re like, “oh my god, I’m standing across the room from some guy who’s going to kick my butt and I don’t know what to do.” Everybody freezes their first time, and in Taekwondo that guy is going to kick you in the head and you’re going to get knocked down.
The other failure mode is the wild man who just rushes in. The thinking is that the best defense is a good offense, so you rush in. But really, the best defense is being cognizant of where you are and what the other guys are doing, and being flexible, and using the energy around you to control your own destiny.
That’s Bruce Lee’s famous quote — be water. I’m going to let you roll over me, and I’m going to be calm and cool and collected, and I’m going to run my game plan, and when I do that you can’t beat me. It’s about being super flexible, super in control of yourself.
People keep looking at these new problems like a deer in the headlights and go, well, how am I going to solve it when everyone’s running around with a Claude Mythos-class thing that can defeat all my internal controls? Well, first, how about you put some internal controls in place, so it at least has to work at it.
That’s what I tell people: let’s start with common sense. Don’t worry about solving next year’s problem or worrying that everybody is going to have an agent that’s smarter than you. Let’s start with solving the tractable problem, which is that you’ve got a bunch of stuff on your network that you haven’t cataloged, categorized, monitored, or tested. Then when you’re on solid ground, you can say, “okay, I’ve now done the basics, now let’s talk about the exotic problems.”
I’ve scientifically studied this, and I can tell you how many developers are using coding agents to do their work. It’s 100%. I don’t even say it rounds off anymore. There’s nobody who’s writing all their stuff by hand. It’s just a matter of degree.
So CISOs have had to realize they’re late to the party. They don’t even get to vote. As a CISO I can’t say “no, stop.” The developers don’t know how to do it the old way anymore. We’ve moved on.
Now the question becomes, what do I want the model to be?
As a CISO I have to sit down and have a big grown-up discussion with the engineering leadership. I don’t want to be arguing with developers. I want to sit down with the head of development and the head of product and say: all right, guys, I know this is doing something for you. I’m not here to take away your toys. But we all understand the way we’re doing this is not safe — and if you don’t understand that, let me enumerate the risks that we are taking as a team. Let’s talk about what we want that model to be.
I actually fully believe there’s a better model, where we do try to teach these coding agents to be more team members rather than finger extensions.
That entails a few steps. For starters, let’s make sure that Claude is not using your credentials. We’re going to give you an easy way to get something that it’s going to use, so I can at least track the difference between it and you, even if it’s all on your laptop. And then I can start to do things where I have different role-based access control (RBAC) rules for the agent, and different capabilities and monitoring, and a bunch of things start to unlock.
Fundamentally there are some basic security best practices that got thrown out the window because the agent snuck in the back door. Or they didn’t get thrown out the window — we just haven’t put them in place, and everybody’s frozen.
If I’m going to bring a new piece of software onto the network, am I going to vet that software? I do with every other kind of software in the world. So I should do that. I should know where it came from. What’s the provenance? I should scan it for vulnerabilities.
It’s one of the reasons I built a scanner to run on your agent, because there weren’t good scanners to run on agents. But it’s not that hard right now. I give you a free scanner, 15 minutes later you’ll have an idea what that agent does and doesn’t do.
CISOs need to ask: Am I monitoring my agents? Am I baselining them? Am I taking action when they go off the rails?
There’s actually a huge amount of productivity to be had with relatively modest, real agentic workflows. These can use all the tools that I can use. Even routine things — I had to put together a slide deck for the board and it was going to take me 12 hours and I didn’t have 12 hours. I did it in 20 minutes, because the agent had all the tools and it could go pull the research. I gave it six documents and said, I need a 10-slide presentation, I need graphics, I need it in this template, go. On the software side, I can create a new product in 30 days that would have taken 18 months before.
There are huge productivity unlocks based on today’s technology. The problem is, as William Gibson used to say, the future is already here, it’s just not evenly distributed.
Irrespective of cybersecurity, get a handle on some real impactful use cases in your business where agentic technology could help you. Then you can decide: does it require a general-purpose agent, or do I have a problem in my distribution network for my widgets and I need a distribution network agent that I custom build, that is special to my business? Those are different answers to different problems.
Pick your problem. Then you’ve got your problem set, you know what you’re going to try to accomplish, you have a specification for it. And then coming back to cybersecurity, I say: All right, for this thing — not for an abstract problem, but for this thing — what’s my security strategy? Some of it could be regular old application security (AppSec), some of it could be RBAC, some of it could be monitoring. Maybe it’s all of those things.
Now if the board asks me, “hey, you just rolled out this agent that we’re going to give access to our warehouse supply chain, how are you securing that? You’ve got a five-slide deck.” Well, I’m pre-scanning the agent for problems, I’ve defined this, I have continuous monitoring in place. That can go a really long way.
Inside Exabeam, we’ve been shipping special-purpose, tightly scoped AI agents as part of our product for 18 months now, and they’re super useful. We just put a bunch of forethought into what we are willing to let them do and not do that we can tightly secure. What we’ve been doing the last 18 months is gradually expanding that sphere: Okay, I thought I could protect these use cases, I’ve tried it for a while, maybe I found some things I needed to harden up, great, now I expand it.
But I don’t try to jump to the end case. I don’t tell everybody I’m going to build an agent that’s going to run my security operations center and turn it loose and see what happens.
The way my book’s organized, the first few chapters are very foundational — let’s read some case studies of where some stuff went wrong — and then you build up from there. The second-to-last chapter is basically a capstone chapter. It’s not, “how do you fight an individual flaw or an individual vulnerability?” It’s “how do I build a process to build secure AI?” The book came out in 2024, but I’m actually super pleased with the process it details, because I think it survived pretty well.
The process I created is called RAISE. There are six different things I tell people you should do. Limit the domain of your agent — what data does it have access to? All the data in the world, all my private data? If so, that’s a sign of trouble. Are you treating it with zero trust, does it have its own identity? Are you managing its supply chain? Have you adversarially tested it — have you red teamed it, as we say these days? And are you continuously monitoring it?
For the first year this was abstract. It was a checklist. There’s a great checklist in the book — are you doing these things, check, check, check. Now I can just run it and get scored one to five. And you notice there’s nothing that really scores well. Even the most robust agents out there fall down, because it’s like, well, what’s your job? Doing anything. What data access do you have? Everything in the world. Okay, well, you’re going to get into trouble with that.
Most of the sample code out there that people use to create an agent, it’s pretty terrible from a cybersecurity perspective. But it’s not a mystery why. It doesn’t pass on any of these fundamental categories.
We talk about AI that’s 10 times as smart as we are that can hack anything in the world, and what am I going to do about that, and it’s easy to just get frozen. But when you break it down — hey, I go grab a tool, I can scan this agent in 15 minutes, I can get these vulnerabilities, I can get my maturity score. This is where you’re doing well, this is where you’re not. It gives you something to build off of and get better.
It doesn’t have to be mysterious. It doesn’t have to be rocket science.
If you break the problem down and use the fundamentals that we would do for anything and start doing them, you’ll make fast forward progress.
This Q&A is condensed and lightly edited from a phone interview with Steve Wilson. RAISE is introduced in Wilson’s The Developer’s Playbook for Large Language Model Security (O’Reilly, 2024). The agent scanner he references is Praxen, released under Apache 2.0. Wilson’s views are his own and do not neccessarily represent SecureW2’s position.
SecureW2 is leader in modern cloud PKI and certificate-based agentic AI security. Through its Signal blog, SecureW2 works with real human technology journalists to interview a range of top industry subject matter experts and thought leaders. Subscribe to get them in your inbox.
