The Wild, Wild World of Exploits With Caitlin Condon

What were the big takeaways for you from this? I'm always interested from the really deep research side of things. What kind of stands out to you? Yeah. Well, first of all, you know, what we were trying to do with this report in large part was take a look at how new vulnerabilities are, are operationalized and used in the wild or not as the case may be. And so a lot of our focus was on, you know, CVE 2025 identifiers to sort of, you know, see that that leading edge and how quickly are are threat actors picking these up, How quickly are researchers picking them up, which is, you know, a different, but related question. And so that's, that's sort of the, the background focus. I think there, there were a few things that I was surprised by. And I've been writing reports like this as, as you know, for about the past, you know, 6:00-ish, seven years. And I've never had access before

to this level of data on vulnerabilities. Phone this, and this isn't meant to be, you know, pitchy or whatever. Phone Check has so much data on vulnerabilities and exploits and all of this. And so kind of for the first time, I feel like I have an actual, you know, sense of ground truth when folks are asking me, you know, how many vulnerabilities are actually being exploited? How quickly are they being exploited? You know, what precedes exploitation, what follows all of that.

So that was, that was cool for me as as an analyst and the leader of, of various research teams. I, I think the top thing that, you know, of course drew attention because everybody loves a list. We came out with, we all love lists. It's just how we work. We came out with a list of 2025 routinely targeted vulnerabilities. And that has, you know, a full public methodology behind it. You know, we explained in detail like we were looking at multiple dimensions of threat activity. So we aren't just looking at public exploits or, you know, is this exploited to the wild?

We're also looking at number of thread actors, a number of rants for families that are, are being used with those vulnerabilities. We're looking at, you know, severity and, and all of that across bot Nets, across all of these different dimensions. You know, we sort of pulled out, can we get a, a comprehensive, you know, truly multi dimensional list of vulnerabilities that really drove the most impact from 2025. And I was surprised that the answer was yes. Not only can we do that, it was really clear with that data.

There were 50 bugs, you know, a couple contenders for like the very last slots, but really this was very simple to do with all of those data points, 50 bugs that drove most of the impact in 2025. And, you know, we were sort of trying to answer the question of the Cves that researchers care about. Are these the same ones that ransport groups care about? Are these the same ones that thread actors care about, these the same ones that bought us care about, so on and so forth. And as you would probably expect, you know, for the top like 10:15, there's a lot of overlap. It's sort of like all dimensions, and then you start in react to shells #1 like it wasn't meant to be a hierarchical list, because you can sort by various things, but react to shell wins on all of the things. In a landslide like yeah.

Which like is insane right 'cause this data was compiled December 31st, you know, first couple days of January react to shell came out what December 3rd right it? Was not old. Right in in the last month of the year, it surpassed everything else and Microsoft SharePoint tool shell that you know, 2 chains of of four volts total. That was sort of the the second ranked set of vulnerabilities. So anyway, you know, this was an interesting list for me and we published that with all of that data. Another thing that you know when I first started it at Volmcheck about and I'm I didn't introduce myself. Hi, I'm Caitlin.

I'm the Vice President of Research at Volchek. That's my job. I blew it. I'm sorry, Caitlin. You're good. When I started at Volchek, you know, I learned first day, I didn't know this before the exploit development team. And we, we've got a big one. You know, they're writing lots of exploits or signature surgery, many, many things. They are actually validating public proof of concept exploit code they're accepting or rejecting. And I got in and was like, that is nuts.

Are are you out of your mind? Like this is so much code. And within, you know, probably a few weeks I went, oh, oh, this is necessary. Like this has got to happen if you want any type of clean data because as we see in the report the team curated, I think it was almost 20,000 exploits in which is insane. 2025, right, right, right.

And of those, you know, 20,000 ish, more than 14,000 were targeting new like 2025 vulnerabilities. So the pace of, you know, theoretical POC development and and PO CS being needed across, you know, different get forges and code sharing platforms on the Internet incredibly high. Those hit more than 10,000 2020 25 CV ES. And so basically of the, you know, 40,000 ish 2025 CV ES that were disclosed last year, about a little more than 1/4 had public exploit code. You are using that as your signal for Oh my gosh, this thing now needs to get prioritized, which would be a better prioritization schema than trying to use, you know, CVSS, which we all know does not work for that, not work for that.

So much of that code ends up being, you know, non functional AI generated swap, you know, especially for the emerging threat type vulnerabilities. That is really no longer the right signal to to throw your team into an emergency triage analysis patching mode. That is a massive shift for, I think of a lot of the industry and a lot of the folks who rely on this industry that I don't. I don't really think people are realizing yet. No, I don't think they are at all. And I think as you said, that that is a huge shift. And it's a, it's a different mindset because if you are, you know, if you work on a security team in an enterprise Oregon government agency or something, and you're just looking at public signals like is it in Metasploit is, you know, is somebody said there's a public proof of concept on Twitter. And I went and looked at it and it looks like the right thing. And you spin up these fire

drills every like 6 * a week. You can't do that. There's no way. And as you said, a lot of that stuff is shit. It's broken. It's not it's non functional and it's not. I mean, it's not scare tagging. People are are doing work, but whatever that they can't, you know, enterprises can't just like rely on those public things without any context or like filtering it down to be like, does this actually matter to us?

Like in it's such a hard question these days. It is and they're convincingly bad. You know, I, I feel like we go in this, it's it's the classic cat and mouse game. You know, we, we go in the cycle of like, OK, we know that if this thing has this hallmarks, you know, huge chunks of obfuscated code, the very clear like AI generated, you know, read me's and and explanations, things like that, the emojis, you know, there's certain hallmarks. We know that if something has those hallmarks, like, great, that's AI, maybe it'll work, maybe it isn't. We're still gonna, you know, look at the code and see, does this seem legit?

But you know, we know, OK, there's a fairly high likelihood that this is not gonna work, and we're gonna have to classify it as AI slop. But you know, folks start to to get cleverer and in general, like these are it really does take like an exploit dev team in a lot of these cases to spin up an actual target to, you know, try to make the POC work across several different configurations to try to modify it a little to see, OK, maybe, you know, it's my config slightly different from what this researcher was working on.

Like it takes that level of work sometimes to really say, no, this doesn't work. This is non functional. You know, this is AI slab. It's convincing is the point. So, you know, I'll see, you know, in various like trust groups and other sort of private forums, like I'll see super capable and smart IR teams and P certs and these folks like, you know, calling out these what turn out to be fake POC's as signal. And not only that, you know, we saw during React to Shell people start writing detections on the early POC's that come out and turn out not to be legit.

And those detections are not going to protect you from anything. And we've got sort of this whole other ecosystem of, you know, AI slop hitting honeypots and honey, you know, honeypot people saying everything is on fire, detections being written to detect those. Guess what? None of that has anything to do with the real world. Right. That's really tough to figure that out. Right. And you end up with like this false sense of security if you're writing detections based on some broken POC or, you know, AI generated crap and you think, OK, we got, we got that in there.

We're good on that one. Let's move on to the next one. And as you said, if that doesn't work in like being unprotected and knowing it is better than thinking you're protected and being unprotected, you know? And we actually see, you know, we call out in the report, we still see some of those detections floating around today from reputable sources for react to shell violence, for instance. It's like this ain't gonna do anything for you. But it gets propagated into the ecosystem. It gets propagated into all the AI summaries. You know, Google picks this stuff up and acts like it's authoritatively correct.

You know, there's a whole dead Internet theory. But I think we're, we're seeing this play out very quickly and escalate very quickly in security in a way that, you know, maybe we and folks who are paying attention to exploit development and those like very, very, you know, first minute signals, maybe we understand it. But a lot of the people downstream who rely on on tooling and who rely on decisions from these type of firms, they, they don't get it. And, you know, things move slowly.

Right. And that's what those customers are paying large dollar figures for, right, is some sort of like, you know, built in threat intelligence, whether it's human or machine generated to say like, OK, you're still at risk for this or there's exploitation going on or whatever it is. And you know, you mentioned the trust group thing. I see that all the time too. And like the ones that I'm in, very smart, reputable people that we know will will say like, OK, I've seen a good exploit for this. It's happening. And then like 45 minutes later be like, sorry, my bad. Not not what I thought it was.

But you know, that's a limited blast radius. They might have told like 40 people the wrong information instead of like 40,000 or 40 million on the public Internet. Sure. But you know, the, the public Internet is public. Plenty of folks will just go to GitHub and type in ACV or whatever and say, oh, look, you know, 10 Pocs, like, great, let's go start running these or write detection on detections on these or whatever it might be. You can't really stop people from, from doing that. And so I, I think, you know, we're, we have the, the start of industry awareness now of just how much of a problem this is, but I think it's still pretty early and that hasn't really percolated to the degree it probably needs to.

Yeah, I think you're right about that. And I part of it I think is driven, and you would know better than I would, is driven by the proliferation of threat intelligence teams over the last, I don't know, six or eight years. Like, it's a good thing that we have all these smart people working on these problems, but they're also in a race against each other all the time to get blog posts and podcasts and detections and signals out to their customer base. Yeah.

To be like, we were fastest, we were this, we were that. And you know, it's like at some point it's a, a race to 0. Yep, exactly right. So that was that was really interesting to me. You know, we saw this 17% increase year over year in public POC code hitting same year CBE, so hitting the new vulnerabilities. A lot of that is being driven by AI. I don't, you know, I don't know what the year over year is going to look like at the end of this year, but I would guess it's going to be even worse. Yeah.

For sure. It's got to be right, yeah. And, and meanwhile, you know, of again, same year CBE, so CBE 2025, there were I think like 250 exploits that were weaponized. So you know, if you're looking for actual signal that's one exploitation in the wild is 1, I think about four 450 CV 2025 bugs that were exploited in the wild by the end of the year. So, you know, the, the types of signals that are truly high fidelity, I think have changed and, and those numbers are still very, very small.

But if you don't know any better, if you don't have the right sort of tooling and and mindset in place, it's really difficult to narrow down, you know, what should I be paying attention to out of these thousands and thousands of signals? Yeah, it's, it can be so overwhelming if you just look at, you know, looking at the the widest possible lens, just the number of CV ES that are allocated every year is in the 10s of thousands, right. And then if you filter that down a little bit to ones that say matter to my environment, you know, products that we have and then filter that down to ones that are known exploited, you still could end up with dozens of them for, you know, say a large enterprise. There are known exploited bugs that affect your environment. So it's you have to have those high fidelity signals that you're talking about, Caitlin, to to understand what actually matters in how to go allocate your resources to fix those things and, you know, put your people in the right places. Right.

And you know, as I think you and I may have talked about before, so much of security and organizations perceptions of security is it's really still more of a, a compliance type function. Oh yeah, I, I think I did the math and there's well over at this point like 300,000 vulnerabilities, maybe a percentage or a percentage and a half of those have ever been exploited any given year. Like we saw of the there were 48,000 ish CV ES issued for any year in 2025. Forty thousand of those were 2025 CV ES, which makes sense. You know, 1% of those 40,000 were actually exploited in the wild by the end of the year.

Is are those the 1% that most organizations are spending their time on? I would argue probably not, probably not. You know, essentially what that means is that and like, sure, there's, you know, there's still older vulnerabilities that get exploited. Sure, we'll be filling in Kev's as the year goes on and as as future years goes on. But ultimately what we're saying is like you're wasting most of your time prioritizing things that are are not exploited and will never be exploited. Yeah. And it's really tough with that compliance part of it because for if you talk to folks in the security world as I'm sure you have that that have compliance is some part of their responsibility that takes up so much of their time. And is such a, like, they spend so much time thinking about like the products that they have to buy to be in compliance with XY or Z Reg depending on their industry or all that kind of stuff.

And like that. I I don't know where it came from originally, but the, the axiom that, you know, defenders think in checklists and attackers think in graphs or, you know, I'm gonna forget. His last name? That was John Lambert. Is that that sounds right? That sounds the John Lambert from Microsoft. Microsoft. Yeah. Yes, I think that's right. And it's, you know, it might be oversimplified, but it's also true. Like if you're an attacker, you're, you're looking at things in this, this kind of like practical way of like, I don't care what products you actually have or what, I'm just going to find a way around these barriers and, and look in and in the compliance world, you're just like, well, do I have this thing sitting here that you said I had to have on this checklist?

Right in my spreadsheet. Yeah, right. Yeah, I and I, I think so we, we did do a few deep dives on specific threat actors in this report. You know, we also looked at threat actor activity globally. We take up a pretty global lens at phone check. And, you know, I, I think that we, we did see. That particularly like named, you know, I'm not talking about ransomware per SE, but named state sponsored or, you know, suspected state aligned threat actors.

Sure, they're using, you know, fresh Co days and they have novel, you know, novel O days and techniques and all of that, but they're also super opportunistic. They're going to use whatever is available on them. They don't care if it has ACV. They don't care if it's all like, you know, they're going to look at every single blog on the Internet for the thing that they're interested in. They're going to try, you know, public with POC's for, oh, you know, this network edge device vendor from 2021 and 2022 and 2024 and 2025 and they're going to operationalize in O Dai and they're going to find non vulnerability paths to escalate and to do all the things they're trying to do.

So, you know, exactly to your point, that's not a constraint, you know, a list of repos and CBE is not a constraint for threat actors and that's still how we're operating. So. And if you, you know, if you go and look at say Sis's Cav or you guys have your own Cav, if you go and just pick a four month chunk of it or whatever time period you want to, you'll see vulnerabilities added to that Cav that are years old. Because as you said, some attacker just realized like, oh shit, I, I can exploit this edge device from the CVE from 2019 that they never fixed, they never upgraded. So I don't need a zero day.

I've got this thing from 7 years ago. And then you'll see that pop up in the sisic cab and you're like, where did that come from? You know, why, why is that there? And it's because just what I described, you know, some state sponsored actor ransomware group, you know, discovered this. Not. Discovered the bug Or like yeah. I'm going to use a genuinely novel O day, but then once I'm in the system like, oh look, privileged escalations in Windows from 2012. It's still not attached. I will just avail myself exactly or Linux.

You know, I'm not trying to pick on on Windows, but sure I will avail myself of that another you know, you brought up ransomware. Another interesting thing that we saw in this in our data for this report was that, you know, there's a lot of overlap to to the opportunism point between what researchers are looking at and what threat actors use. And a lot of that is like, you know, if something is publicly available, they're going to give it a shot. Like state sponsored threat actors are going to take any possible Ave. There is much less overlap between what researchers use and what ransomware groups use. And the reason for that appears to be ransomware groups are doing a really good job of keeping those exploits proprietary.

You know, they're targeting software that is not easily available. The, the software producers are, are very antagonistic to any type of public research. They think that's good for them. Well, guess what? You know, we've got a few of these cases where there's no, you know, public or commercial exploits. Obviously there is an exploit because ransomware groups are using it. But but like 5-6 different ransomware families have availed themselves at that CBE. Now the software producers that stuff is targeting might say, well this is good because, you know, lower skilled threat actors can't use it because it's not public.

OK, but you've got six different answers, workers using it and nobody understands how the vulnerability or the attack works. I would argue that is not good. It is not. Folks in the right, you know, I'm not a corporate lawyer, but my point of view is that that's bad in the end. I can think of some software vendors who might fit that description. But in general, you know, ransomware groups tend to be opportunistic too. Like they definitely have targets in mind for sure. But in some cases they'll take, you know, some Windows exploit or Linux exploit or, you know, edge device exploit and just go see where it fits in, you know, and see where they can make some money off of it.

But the old kind of received wisdom that these ransomware groups aren't smart enough to do their own vulnerability research. I does not apply anymore, I don't think I mean. I, I don't know if I even have an opinion on that. I mean, it kind of almost doesn't matter sometimes operationally because if they can buy them and or, you know, if these groups are clearly they're sharing among each other or there's some type of, you know, cross pollination. An access broker is selling to more than one doesn't have restrictive agreements in some way. Like I don't know, you know, clearly that's happening and that that muddies ransomware CBE attribution and ransomware attribution in general.

It's just such a morass. Yeah. No, thank you. Yeah, yeah, yeah. It was really tough walking through, you know, OK, like this ransomware family was deployed. We don't really know if it was this group with high confidence, but we know, you know, what the extension was and we know what the ransom note said or whatever the the indicators might be. You know, when you add initial access brokers into that mix and they're, they're selling exploits to various folks and they're, they're, you know, they're getting access and selling it months later like attribution is. It's real messy.

It's like the entire Claire Danes bulletin board, you know, of strings of like. Right, right, right. I. Trying to connect the dots. You know, we, we do have some stats in there and like, oh, total ransomware CVE usage was down, but we didn't really focus on, on any of that because the attribution is so muddy that the, the total number of CV ES sort of almost doesn't matter. It's just interesting seeing like 1/3 of CVE 2025 vulnerabilities that were used in ransomware attacks have no public or commercial exploits whatsoever.

That's a lot. It is and it's a little concerning that, you know, that's a good indication that those guys are doing original exploit dev or are buying it, you know, or, or someone is who's feeding the rest of that ecosystem. You know, it could be 4 guys in, you know, in Siberia doing it, but whatever, it's getting out into the ecosystem, one where and and it's not getting to you right, your team. And you know, we actually in our data because we're trying to capture everything, you know, we don't want to narrow down users fields of view because of structural decisions that that we've made wherever possible.

You know, we're still capturing as a ransomware CBE. Like even if the actual intrusion we're attributing to was conducted by, you know, I don't know, pick a, pick a Helix kitten or whatever, like an insane name, even if the intrusion was conducted. Bye. I killed Dennis. Bye by that actor, you know if it ended in ransomware deployment and we know what that looked like it was you know this particular family, whatever we're still capturing and attributing that.

And so, you know, I, I think what we see a little bit in this data is like there's overlap between, you know, state sponsored threat actors having a hold of this ransomware and deploying it in, in certain scenarios. And so yes, like that definitely would point to original exploit dev because we know those shops have the resources and motivations to do it. Yeah. And I know that like I wanted to ask you this, I there's a lot of a lot of times in our industry, depending on the source that you're reading, you will see vulnerability and exploit used interchangeably. Those are two different things.

Yes, they are. And I'm sure this drives you and your teams completely bananas. And I know that you hire specifically exploit devs, offensive exploit devs who can write exploits for a given vulnerability or set of vulnerabilities, right for the people. Can you explain the difference between like what is an exploit developer do and what I I mean? I could do this on the surface level, but vulnerability researcher? Exploit Dev. I, I mean, I, I do generally interpret vulnerability researcher to mean exploit dev.

I'm sure you know, not everybody does. But to me, if you're a vulnerability researcher, you are doing original work, you know, you're not just operationalizing Pocs or whatever other people have put together. You are actually, you know, getting the software and patch dipping and reversing and trying to identify where is the exploitable condition. Yeah. You know, we generally, we're, we're trying to work off of CV ES because that's, you know, how the industry and most of the world understands it. But product actors don't necessarily care about CV ES. They just want to know where is there a, you know, a flaw that can theoretically be exploited? And they're developing the capability to literally go and exploit that.

And so that's, that's what my teams, you know, the whole time I've, I've been in these types of roles have been doing. I, the thing that that drives me, it drives me a little bit crazy, but I totally understand where it comes from. My current boss, our CTO and Jake Baines said to me years ago, and I I hope that he still thinks this since I'm saying a lot of recording, he thinks that zero day is actually an adjective and not a noun because there are so many people who interpret 0 day to mean unpatched vulnerability for this zero day exploit. I definitely interpret 0 day to mean zero day exploit. I'm never going to use zero day for a vulnerability, but it's not been passed or mitigated or, you know, whatever it is. We're on the same page there, OK. Cool, you know, I understand where where some of that confusion comes from.

So anyway, point is, I'm not sure I'm totally answering your question. Vulnerability thing that can theoretically be exploited exploit the thing that does the, you know, in, in typical exploit dev parlance, if you have, you know, an RC exploit for something, usually it's going to give you a shell. That was, you know, sort of the that was our I, I hate to bring up Metasploit on a, a Volt check call, but that was sort of our, our thing on Metasploit is, you know, if we're calling this a full on exploit, usually what we're we're getting back is a shell.

It's it's weaponized and so we are developing weaponized exploits today at Von check. You know, there are a few other shops we're developing weaponized exploits. Most of the exploit dev, most of the vulnerability research that goes on in this ecosystem is not going that far, which is fine. There's nothing wrong with that. Most of it is only taking it to like, you know, can I trigger this at some level? Public POC, almost 99% I think of the the exploits that were released publicly, you know, that actually weaponized professional exploit developers proving that they can go all the way, get a shell, you know, dump creds, dump a file.

So whatever it might be, that is a very, very small percentage of the security research that goes on today. Yeah, I think and because the the pool of the people that can do that type of work reliably is pretty finite like it's. It is. And there's a lot of, there's a lot of sort of speculation and I don't know if it's optimism or pessimism or what around, you know, AI is gonna break down all the barriers. Is. It AB I you're still gonna need a a driver who knows what they're doing. Yeah, this isn't Waymo.

You need like somebody doing the actual work, like really hard head down, like the shit that you think of hackers doing like, you know, 9 Red Bulls deep on a on a Thursday night. Like that's the stuff you're talking about. Like, OK, we know there's ACV in this Avanti thing. Somebody just theoretically just, yeah, somebody go, you know, build me and exploit for it. But in order to do that most of the time, I mean all of the time really, you need the software. Like I I think the the unspoken or at least unfocused upon truth about exploit dev is that 50% of it, if not 51 is sys admin work. These guys will spend, you know, more time trying to procure software and often times like you don't need just one piece of software if you're trying to validate and develop an exploit for a known CBE.

Like there's a right answer. You're not a a threat actor who's just going for whatever capability you can. You can find and weaponize opportunistically like there's a correct answer and a lot of incorrect answers. You need the vulnerable version, you need the patch version, you need to diff those. You need to, you know, setting up those targets as well is a huge amount of work. You know, you look at some of the more complex software, which is a lot of it these days, that can take a week plus. It can take two weeks plus. I once had a team member who spent literally, and I mean, this was an incredibly skilled person. I think it was a week and a half setting up a single Oracle piece of software.

That sounds right. So, you know, and, and that they have to read docs to do that, they have to mess with configurations. Like these are the things that you're not going to get, you know, throwing AI at here's a vulnerability description and like an open source repo actually, you know, doing the work to set up and mess with and understand how enterprise software is, you know, configured and written and documented and advised is a a ton of work. And and that's what makes these people actually understand this is how attackers would operate against this. Amen.

Yeah, It's not just throwing some AI tool against a web app, you know, like a commodity thing that's just sitting there that you can attack whenever you want. It's the process you just described that's like, yeah, 51% grunt work of like building the thing that you're then going to try and knock down and probably spending a lot of, you know, actual dollars on that as well. Not just time, but yeah, money and effort. And like as you said, like the number of people who can actually do that work is small.

So their time is valuable. And if they're spending a week and a half setting up a freaking Oracle database, it's not. It's not ideal. Yep, exactly. And I I would argue the number of people who are incentivized to do that type of work is decreasing because. In the AI on the legit, Oh yeah, yes, there's no question about it. If you go and read any of those vulnerability reports I mentioned earlier, they came out in the last like same month. Every one of them has a section about AI in one way or another and how it's affecting vulnerability discovery, exploitation, all that kind of stuff. As well as like the number of vulnerabilities that AI coding tools are introducing into software, which like is a whole other can of worms.

Like that's not. Yes. That's. I I won't touch that one because there's so much there. But yeah, totally agreed. Yeah, on the going back to the the topic we were talking about before with like sort of the signals and like the high fidelity signals, like what bugs matter and what exploits actually work? What are the sources that you guys ingest as like, I'm sure it's a large number of like, you know, stuff you can't mention or things that you don't want to.

But like, where do you look to be like, OK, this is reliable information. If it's not something that you guys are building on your own, that there isn't a workable exploit out there. Yeah. And well, you know, first of all the the ultimate signal in the end is. Has this been used in the wild? Yeah, in. Some type of intrusion or or attack. So that's one of the main things that we are collecting and keying off of in terms of, you know, platforms and sources being monitored.

It sounds awful to say all of them, but you know, we're, we're looking at Git forages, you know, GitLab, GitHub, Giti, you know, we're not just US or sort of Western world focused. We're also, we're taking a global lens and so we're also looking at like CNNVT and you know, Giti and the the Chinese platforms and and databases. You know, we're looking at there's other code sharing platforms that aren't like public hit forges. There's blogs, there's, you know, all sorts of looking at social media.

Like there's all sorts of different sources for this stuff. Of course, we also have sort of, you know, private relationships with government and industry and all of that. A thing that I found surprising is that, you know, even though a lot of these exploits in the end live on public git forges CNASCVE numbering authorities are actually the ones surfacing like the vast majority of public exploits in the the vulnerability information that they are submitting to the CVE program when they issue those CV ES. Really, I didn't I did not expect that exactly. And so good job, good job everyone, please keep doing this.

That is important signal. You know, that doesn't mean that those are all necessarily valid. You know, maybe some of them get enriched after the fact, but you know, ideally like that should be a high ish fidelity signal. If those Cnas are, you know, really successfully demonstrating the vulnerabilities against their their own software and saying like, Yep, we confirm this is a real thing, you know, we're going to fix it. We're going to issue ACBEI may be being a little bit optimistic there because I.

How dare you? I know they're, you know, we, we report vulnerabilities for researchers to, to software suppliers as well. So like we have insight into this process. Those suppliers are getting so, so many reports, a lot of high generated and you know, right. And so it's like, I, I hope that there's actual validation and confirmation going on here. I'm going to assume for right now that you're probably not going to willingly issue ACVE and, you know, spend effort on developing a a fix for your own software if you don't think that attack works, but that might be a little optimistic. I wonder how this works for let you know, take Microsoft for example, on the extreme like huge amount of resources gets a massive number of bug reports every month. I would guess that a large percentage of those are trash or you know, just not quite what

you know, not actual bugs or not they're not going to fix or something like that. So they're triaging a bunch of those. And obviously Microsoft has their own internal exploit dev team that can validate these things, you know, and other, you know, Cisco and Google and places like that too, too. But if you're a smaller software vendor that's still ACNA, like, how are they going about that? I mean, are they relying on external resources to to validate the exploits or like I'm often wondered like how that actually works? I don't have a full answer from what I can see. You know, a lot of those folks of course are underwater and we're, you know, the, the volume of what they're receiving is, is burning them out.

You know, we are involved in a number of CBE program working groups, like I run the, the researcher working group for the, the CBE program. And there's lots of, you know, researcher open source bug bounty type Cnas in this community. And I think we are, you know, hearing from a lot of these folks, like in, in different configurations, they're all seeing this type of problem, you know, do they, whether you're a, a supplier who's receiving reports, whether you're a, a code sharing platform who's trying to make sure that the quality of the code shared on that platform is, you know, good.

Do you start refusing to accept, you know, reports or submissions from certain users? Like, do you start basically using AI as a gate to to say like, OK, AI identifies this is AI generated And like, we're just going to say no, which is, you know, maybe that's 80% correct. Maybe it's 20% incorrect. Maybe 1% of that 20% really matters. It's like the choices that a lot of these people are having to make are not, they're much more sweeping choices than they would prefer to have to make.

But the scale of the problem is so big that they're having to, you know, use a hatchet instead of a surgical knife, and that's not what they would prefer to be doing. No, that's, that's very true. And I think, you know, it's such a huge sea change from even say 10 years ago when, you know, between the two of us, we probably could have named like 90% of the offensive security researchers on the planet, or at least like now. I mean, there's not just, you know, exponentially more humans doing it, there's, you know, 10 to the billionth number of AI models doing this work to some of which is crap, but some of which is legit.

Like I've seen, you know, offensive security researchers who have big names being like, look, these things are finding bugs that I would not have found like just, you know, in places that maybe I wouldn't have thought to look or using code pass that didn't make sense to me or things like that, which is, you know, very cool. But also, I think, you know, we're so early in this in this like, I hate the word journey. We're so early in this like development.

It's hard to tell where this is all going with like AI vulnerability research. But it's just, I'm sure it's going to go to places that we haven't even thought of. Yeah, I and I'd agree with that. Like I'm not saying AI vulnerable research isn't legit. I I think it is. I'm just saying that, you know, the sort of low skill. I'll just point AI at, you know, an open source code base or a vulnerability description like that ain't going to get you very far. But we see all the time, you know, there's brilliant people who are using it in ways that are very effective. It may not be for, you know, totally generating an actual exploit because there's a lot of manual work that still needs to go into that.

But if you're analyzing a target and it's helping you, you, you know, enumerate something or helping you breakdown, you know what that how that piece of software works a little bit. It's helping you build out much, much faster and understanding of what can be exploited, You know, how the product works, building that scaffolding for future exploit dev, even if it's not totally solving the problem for what you're working on right now. And I, I think, you know, skilled exploit developers, whether they're threat actors or, you know, in the commercial sector here, public sector, whatever, they are using it to great effect to, to speed up processes. So I agree we're we're going to see things that I don't, I can't predict.

Yeah. Oh, oh, I think that's absolutely true. I think just even if you had asked me like a year ago what, how much of this we would have seen I, I would have been wildly wrong. Like it's, you know, same. Same. It it's gotten so pro prolific, it's not the right word, but commonplace in just the last 12 months that you it's it's crazy to think about the acceleration is you know, it's just going to compound. I think. I mean, it's, you know, as we're recording this RSA is like 2 1/2 weeks away. And like, I didn't like my inbox is just AI pitches, like everything is, you know, every single thing. Is it?

So yeah, I'm, I'm dreading, but also kind of fascinated to go just like walk around the the Moscone for like 20 minutes and just see what's absolute bullshit. And like, what kind of has some air of like legitimacy to it? It's going to be hard to tell. I don't know. What's the, you know, proof is in the pudding? What's the actual saying there? The proof of the pudding is in the eating or whatever. Show me the shell.

Right. Yes, yeah. Don't just put AI on the product that's been around for 15 years. Right, right, right. Yeah. All right. Well, Caitlin, thank you so much for your time. I love this. This is super fun. Always great to talk to you. Thanks, Dennis. Anytime everybody go read their report I will link it into the show notes so you guys can read it. Also on vonecheck.com. There we go. Look at that, look at the branding. Great job. I'm on it. Talk to you soon, bye.