We Need to Talk About KEV With Tod Beardsley

So I'm doing experiments with like OCR ING a an original text to count sentences and I get errors when the LLM goes in there. This is interesting, like because I'm using something of an antique translation. It's a Constance Garnett translation from 1930. OK, it is all right. The way says it's the best translation, so. I stand with Hemingway. Let's go. But yeah, it is a ridiculous experiment. You can subscribe today at war in peak dot ES. So just war in pieces but a dot before the ESI? Love it.

This is completely insane and I I love everything about it. Yeah. Automation Git. I'm like, I'm backing up everything I'm get I'm publishing through my normal podcast host, but I want to get to a point where I can like go up to someone in a bar and say, please read this sentence out loud. Go and then one click, record, snip clip, post to the podcast, publish. Like that's that's my goal. I don't know how to do it yet, but I've gotten a lot faster.

This is so great. There's I think like 98% of the population. I started War and Peace, never finished it. Nobody, you know. No, lots of people finished. That it's, yeah, I'm sure somebody. Greatest books in literature like on Earth so. I know I've read a lot of books. With it, Dennis, you've never been reading it now. It's even better. Oh my God this reminds me of like there's this podcast called One Heat Minute where these two Australian guys do an episode on one minute of the movie Heat every. Which?

Is like a two hour and 30 minute movie, 2 hour and 40 minute movie. It it started like years ago, like I'm pretty sure they finished, but they at some point got Michael Mann on there to talk about it. And it was just these two dudes that were like, we love this movie. Let's see how this works. This was a fun genre. Like at the very start of like the 20 tens, I want to say, Yeah, I want to say the first, certainly in my opinion, the best in the genre is Star Wars Minute, 2 minutes, 5 minute recap of Star Wars.

And it's really, and they do, they end up getting like people who work on Star Wars, like tech FX people and all that, just like guesting in. And so those episodes tend to be like 20 or 30 minutes long. That's what these two guys did too. Like their episodes? Are like and. It's, I mean, I'd love somebody that's that. Mine is more ridiculous. I think yours is more ridiculous. Yeah, I'm going to, I'm going to go with you on that. Like I, I am all for people that are like fully obsessed with something to that degree where they're just like, I love this so much. I'm going to see if there's other psychos out there like me that love it this much and see if I can get them to subscribe. It's incredible.

I'd love it. What a. What a. Journey. This is on Pieces. We have a companion Patreon podcast called Warren Pages. It's a page by page recap of Warren Pieces. Wait, is this real? Is that? Real. It's real. This is the best. I love it. Absolutely amazing. We're we're already like 100 times longer than your average. Episode right now so I'm I'm shocked if I get any listeners for more in pieces to stick around this long.

Amazing. Oh. Let's talk about what we're here for. I listen, man, I'll talk about this forever, you know, But yeah, we should talk about this report you did, which came out I think last week as we're recording this on. It's basically like, I don't know how you like think about it or, or like describe it, but I sort of looked at it. It's called Kevology and it's kind of like a look at Sis's known exploited vulnerabilities catalog, but not just like, hey, let's break it down all these bones and put them in the categories. It's like looks at it from all these different perspectives and you know, where you know, when exploits come in, the just kind of this holistic picture of it.

I know you, you worked at CISA, you were, you know, in charge of this essentially. What motivated you to to sort of dive deep into it? Maybe now I know since you'd like to dive deep into shit, but I do what what? What motivated you to like, really dig into this? So I would say the inciting event of of kevology colon and analysis of exploit scores and timelines on this is a Kev was we had, I want to say it was like a customer. It wasn't even a customer request.

It was a customer wish of I work at run zero. We do exposure management, asset management, vulnerability stuff, you know, all that. And they had this wish and the wish was, boy, it'd be cool if I could just like deal with all the outstanding Kevs, you know, just with with your product. And we said, yes, that would be cool. And I used to work on the Kev. I kind of know what goes into the Kev. I was a section chief and like you say, essentially in charge of the Kev. It takes like a billion people to make this thing, but sure.

Yeah. But I was, I was on point for it. So. And I'm like, OK. And I'm like, this is going to be trickier than you expect, though. And then I wrote a 20,000 word paper on why, why this is a hard ask. Getting 100% Kev coverage today is I, I would pause. It is impossible with certainly with one, one solution. Like if you buy one piece of security software, right? If if it is sold to you as like, oh, push this button and you can find all your Kev exposures, they're lying to you because that does not seem to fit nicely into one SKU. You know what I mean?

Have covers traditional IT vulnerabilities, operational technology vulnerabilities, cloud stuff that is probably already fixed by the time you get around to it, but still useful for log analysis and all that mobile stuff, which is you know the the boogeyman of every IT operator everywhere because almost all mobile is bring your own device you don't actually have control over it all of that right and so so that's like 5 categories of software right there right yeah and then you have locals and remotes and privilege escalation and all this other stuff going on in the cab.

One of the misconceptions, I'd say, of the CAB is like it is a list of, of the worst, of the worst vulnerabilities that have affected federal government systems that are used for initial access, right? Like, this is, this is often how people think of the cat, right? And that's wrong in like 4 different ways. Yeah. So very much so. Yeah. So, yeah, so that was like the inciting event. This is basically an elaborate excuse to my boss of why we can't do this, but also no one can.

And so, but we want, but we still want to like address the problem right, of like, well, if you if, if it's impossible or just like way too expensive to to say like, yes, we have 100% Kev coverage. Like what can we take from the Kev that is like a useful vulnerability signal? What can we use it as an intelligence source? Can we pick out the things that are on the Kev that are actually relevant to me in my network, right. And so Chemology posits a number of ways, right, that you could do something like that. Like if you really like CBSS scores, we talk about like, oh, well, maybe you want just like the criticality of CBSS, but that's a little bit of a lie.

Maybe you want just like these particular CBSS attributes. And so select through for that and then you get a nice little list of like all my remotes, non authenticated, no user interaction, full shell access bugs, right? Like that's like the straight shot RCE, which is usually what people are thinking of when they're thinking of software vulnerabilities. Almost always. And so like we can, if we can like 0 in on just those as a priority mechanism, right? Like everything on the CAB should be patched for sure. And if you're a federal civilian executive branch agency, you have to patch all of them.

Sorry, that is impossible, but but you still must do it. And so they don't really have a lot of choice, but they can have choices on prioritization sometimes, right. So you want to like be able to sort it and like maybe build other lists out of the cab because it's really big. I say really big, but it's like 1500 and change. I think it's 1528 today vulnerabilities, but that's out of a universe of like over hundreds of thousands, right? An. Unknowable number, yes.

CDs have been published. Almost all of them will never see exploitation action, right. And but these ones definitely did. And so like, that makes them interesting. And so of this very interesting set, what's even more interesting, right, that that's kind of what you know, the kevology approach, right? I I joke that the next paper I read on this is going to be kevonomy as we evolve the theology into a proper onomy. Yeah, like astrology to astronomy, but.

Maybe you could do a podcast where you're just reading the description of each vulnerability like. For this is not a terrible idea. I don't know man. No, wait, that is a terrible idea. I mean, I stick to world literature. That's not dare you. Impenetrable giants of world literature. Yeah, you mentioned that this this basically came about as like a customer request and seeing if this would work. And, and sometimes it does seem like there's there's just as much value in proving that you can't do something as there is in.

Showing and like, and I don't like to take a can't laying down, you know, like and and here's the thing, like what I want to do is arm people like one of the goals of this paper is to arm the the beleaguered IT security operator, the analyst. And I want them to be able to tell their boss of like, why I can't tell you one way or another about this Apple iOS bug, right? Because it's BYOD, it's probably fixed, right? Because like iOS tends, it tends to be hard to avoid an iOS fix. It does, yeah.

After a while and we see that like looking with all this like sorting stuff that we do on it now, and this is all released by the way, we can talk about it in a second. But with all the sorting stuff we do, we can see that like Apple iOS bugs in particular tend to skew way to the left side of an EPS graph. So like EPS is exploit prediction scoring system. It's a percentage. It's a percentage chance of if this vulnerability is going to be exploited in the next 30 days.

And Apple iOS bugs tend to score real low on that because the exploitation that happens in iOS tends to be extremely targeted usually to a journalist, usually in a part of the world that's not very cool to be a journalist in. And that's it. And then it gets burnt and then like Apple notices it, you know, a citizen lab notices it and then it gets fixed, right? And so the chance of that getting exploited again understandably very low. And so you can take that to your boss.

You can say, like, look like I understand this is a big deal bug. It's like straight shot RCE through like a text message or something. But don't worry about it. It was, you know, two months ago. It was used in in in a campaign probably. And EPSS is telling me like this is not going to pop up for us. Right, Yeah, the the Apple example I think is a really good one. I was talking to somebody else about this the other day. I think when the the latest Apple release came out about. OK, yes, there was a thing in there that was exploited, you know, in iOS. We don't know exactly when, but we know that Google tag was the one that, you know, noticed it.

So that tells you that's exactly what you just described. It's probably. Certainly espionage or government. Back against like 4 people or something like that. Some very small subset of journalists are no. Of. Course. As a journalist, yeah, not in favor of that shit, but you know. Like, you should know about it and be aware of it, and then every once in a while you're going to see some Apple thing that isn't quite yet patched, right?

But like almost always when you see a CVE for an Apple bug, which is an important distinction actually, CVE for an Apple bug almost always means that the fix has already been rolled out and in most cases already applied. OK, Yeah, this is another thing I wanted to ask you about. There's a section in the report where you talk about the time delta between when CV ES are issued and when they enter the Kev. And like entering the Kev is here. Sometimes the CVE is before it, sometimes it's after it. Which is a weird thing to think about really. Rarely is it after, but it does happen. It happens which like.

Can you explain how and why that happens? I can think about why it happens with Apple. But this was a point of contention when I was in SZA at the federal U.S. Federal government. So one of the barriers to entry to Kev is that for a vulnerability to be listed on the scissor Kev, it must have a CVE ID associated with it. Must have, there must be a CVE ID assigned to it, which is a little bit of a tricky word because vendors like Microsoft, Google to Apple, pretty much every like major soft software vendor today, right?

Technology vendor today is their own CBE numbering authority. So they can reserve CB ES all day long, right? And So what we will typically see is most cases the CBE was reserved and published like weeks to months ago, right? And then eventually Kev catches up with the evidence and all that, right? Yeah, but every once in a while, Sizzle will get a communication from the vendor of the software saying like, hey, we're gonna publish this thing.

There's active exploitation right now. You should probably put this on the Kev. And then we would say, like, that sounds bad. Do you have evidence of it? And they'll say, yes, we do. And then they send it over and we look at it and we're like, looks like an exploit to me. Yeah, we'll list it on the Kev. I'll say we they now list it on the Kev. You some, we try to get that CDE published, like we'll push them to get the CDE published and all that, but if they don't, we'll still list it on the Kev because we don't want to slow down patching. But then the CDE itself might get published 2 days later, three days later or something like that. And so you sometimes will be in a zone where like I have a CDEIDA description of the bug on the Kev. Kev's always have descriptions, so even without the CDE you'll get like a little bit of warning on that. Those are rare though.

Most of the time it is something that's already something that's already identified, something that already has patch, right you you always have to have a patch or reasonable mitigation to get on the Kev. Otherwise it's just a list of bugs to look at, there's nothing to do. We've got lots of those lists we don't need, yeah. Don't need another one Has a CVID, has a reasonable mitigation, has exploitation evidence and that's like non consensual exploitation. So like, yeah, so like, red teaming is consensual, like ultimately.

Right. And so bizarre hits are a little bit of a Gray area, but like, generally speaking, the attack should have been successful. But that's always like a weird case that takes a lot of work to actually verify. Yeah. And then 4th is has to be relevant to federal interests, right? Which doesn't just mean the computers that, like federal government runs. Sure, it could be State, local, tribal territorial governments are a big. Source. Or the defense industrial base or something like that. Defense is a big deal.

Critical infrastructure is a big deal. So like, water and Power Distribution are like the big ones for us, for SZA and international allies is a big deal. So like, OK, if someone's burning through Canada, like, we want to know about it. Yeah. Because maybe we're next, you know? So that's why. Right? And so. So, yeah. So like every once in a while you'll see Cdes published, they're always reserved before they go on the cab and in the worst case is like, oh, there's this crazy O day. It's with a vendor that isn't its own CVE authority. No problem.

SZA is also a CVE numbering authority, so they can pop one out real fast in those cases. You mentioned in that they. Get real nerdy about this. So I'm gonna no. Listen, I love this shit for whatever reason. I enjoy it too and I will talk about it all day. I've had long conversations with other people about like the CBE system for no real reason, like just. I don't even know why but. Should join one of our working groups. Steve Christie tried to get me to do that at some point. I don't know man.

Steve, you mentioned that one of the things that CISA or, you know, even other people like, like you guys might ask for is the evidence of exploitation. Like sometimes that that's easy. Like you could have victims publicly saying we were exploited or something like that. But in the case of, you know, let's pick like an an O day that is not publicly known yet and somebody is saying we've seen this be exploitable. Like what qualifies as evidence? Like what are you looking for so.

So for starters, if you have evidence of exploitation of a CVE and it hits all those things I talked about, please get in touch with the people who write the Kev and that is kev@mail.sizza.dhs.gov. It's a lot of machines is. Someone actually monitoring that anymore, Todd? Yeah, no, no, Yeah, humans read that. OK, good. I know. Personally read that. Excellent. That's good to know. Yes, and they they will pay attention to that. So if you have evidence like start a communication there like or just like drop AP cap on them and say nothing, it would be funny.

But so things like P caps are really good, like for things that are in the clear that aren't encrypted, things like system logs showing, you know, especially for things like privilege escalation, you know, EDR logs are really good for that kind of thing. OK, You know, firewall, you know, smart firewall, IPS kind of stuff. Intrusion prevention, system logs. A lot of times when I was doing the evidence work, it would be we'd get a collection, right? We'd have like, oh, we have this like anomaly detection thing that logged a thing over on this network device. And then we can see by the time stamps, because good for you, you're using NTP.

Amazing. We can see by the time stamps that then this machine started acting real funny and has a bunch of logs about it, right. And so like, that's the kind and that's the fun part, right? Is like being the Poirot of, of exploiting. Yeah. You know, mystery solving is super fun. Like it's a very fun job. And then you can kind of like try and correlate that with maybe other. Things you make a case, right, right. And a lot of times you're making a case of it's this CVE and not that CVE cuz that is important. There might be a patch set is released from somebody that addresses 5 CVS.

Solarwind's web helpdesk, just as we're recording this, just did one four or five days ago. Yeah, very recently. And there were five CV ES in there. But you will notice only one of them is listed on the cab because it's just that one that that had the exploitation evidence. So that's, that is often important, especially when people are trying to prioritize things because CV ES are lists of vulnerabilities, not lists of patches. And so even if like, and if that patch is like no good, you know, like or there's an evasion for it or something like that, you have to know like, well, which one am I evade? Which one am I now patching again and all that, all that kind of stuff.

The patches that we list that they list, God Dang it that just that sizzle lists have gone through some like assurances that like, yes, this actually patches the thing. If a patch smells funny when whoever's investigating it, like they'll, they'll talk to the vendor, They'll say like, I don't think you're actually patching the thing. You're patching just like 1 vector. And sometimes that'll get like a new patch in there. But I'm, I'm very confident that every patch listed on, on the scissor cab is, is good, right? Because otherwise we're sending, they're sending, you know, hundreds and hundreds of like federal IT workers out to do a thing on a timeline.

It's not going to work. Then they got to do it again. I mean, like, that is a bad use of taxpayer money. It sure is, yeah, We've got enough of those. We don't need another one. Which is ultimately why Kev is so selective, right? Like Kev will get some knocks from other people in cybersecurity and saying like, Kev is so slow and like we knew about this thing, you know, days and days and days before it says, why are you so pokey on the on this? And it's like, and the reason for it is that Kev prioritizes accuracy more than speed in those cases. Totally fair.

And it's and it's fair. And if it's like a really hot vulnerability that everybody, everybody knows about because everybody reads the cipher or bleeping computer or something, yeah, they're going to be on it anyway. They don't really need to wait for the Kev to tell them to do the thing, hopefully, right? Kev is the Kev is definitely not your first line of defense. It is much closer to your life. I would hope so, yeah. I mean, that reminds me a little bit of the thing that you always hear from Microsoft and other large vendors of like, why aren't you getting this patch out faster? And you know, the common answer is we have 10,000 versions of this in 400 languages and we have to test all of those so it doesn't break stuff on your network. And Microsoft's really good about that too, because they really, they legitimately do like no one thinks about, you know, the Microsoft point of sale system that also runs, you know, Windows.

Me or whatever it is, Windows me literally. Yes, yeah, like when that thing, but like when they're doing an ME patch, which I guess they probably wouldn't anymore until unless they need to. So like, OK, so like Windows 10 IoT build for example, right. That is a thing that's in the world and it's actually arguably a better desktop operating system in my opinion. But the IoT build is a very stripped down version of Windows. But because of this, they have to test, right? Like you always have to test and so and there's every version of Windows has like 20 or 30 flavours and then you have all the language stuff too. That sometimes matters, especially for things like font rendering bugs.

Which there are plenty of. Yeah, Yeah, exactly. And those are the, you know, those are the things that the mitigating factors where you're like, we really need this patch. It's being exploited in the wild. But you know, the the people on the that are building the patch are like, OK, we know that, but we don't wanna break a bunch of other shit. When we give you this patch, then you're gonna yell at us for that. Cuz then you'll have a bad time and you might not patch again. I remember you and I are gentlemen of a certain vintage.

Of a certain age. I remember when Microsoft was issuing hotfixes and the hotfixes opened with like, don't you'll apply this unless you're actually experiencing this problem. Absolutely. Security fixes. It was hilarious because it's like, oh, don't apply this until you've already been owned. It was like. Well, I'm. Going to go ahead and just apply it now. Right. Or I'm going to throw this server in the ocean because, you know, it's probably. That is no longer the case. We, we've at least I, you know, we often talk about like nothing ever changes and everybody has bad habits and they've had these bad habits.

I think we have made a, I think we have pivoted very slowly over the last 30 years from like let's be extremely careful with the vendor supplied patches to like let's trust the vendor supply patches unless I get other other information, right. Yeah. And Chrome really was the one that did it. Google Chrome with with the auto patching of the browser was insanity at the time. Oh my God, People could. You lost their minds.

Yeah. Oh how dare you patch my browser And meanwhile IE 6 is chugging along. It's like I don't need any patches, I'm good. Like oh did you want a patch? Here have a 12 step process. Yeah, you can't watch, but I wouldn't recommend it. Yeah, that's so true the the Chrome scenes now. Where like it is newsworthy when a patch, a security fix actually breaks something on like a major piece of software, right? Like that is, that is its own story. It used to be a regular occurrence like it used to.

To be constant. Yes, honest to God. And everyone got burned like 20 times on that particular stove. But I think we've gotten to a point where we're like, OK, I'm pretty sure the stove is cooled off now. The Chrome like auto patching was a huge, huge deal. It's hard to remember now, like, and especially for folks that weren't around at the time, it was a massive uproar in, I mean, I guess it was before that, but Microsoft saying, hey, we're going to push patches to you instead of you having to come and get them from us. People lost their damn minds or how dare you. Like I have those things for breaking. It is a delicate House of Cards.

This is barely hanging on by a thread and you're going to just shove software at it? Come on. Let's not do that. Yeah, but now we're just like, why isn't my TV or my dishwasher patched like every two days? Like, right, let's go. Like I I know there's a tax. House like home, like Soho, you know, small office Home Office routers are now more or less auto patching, I would say. But it's like, I don't know, it's like 7030 I think between the vendors and are you one of the lucky ones like that?

Just you might have to. And those are devices that you literally were were unable to patch for a very long time. Like not only was it technically hard, but a lot of times the vendors wouldn't even issue patches or. No, they would just like they wouldn't like you bus button and you're like well if you want this fixed, buy the next one. It's £200. Yeah, the planned obsolescence. Yeah, it's a beautiful thing. Welcome to IoT.

No, Dennis, IoT never dies. No, no, it does. Here's. The thing there's hardly any moving parts in most IoT, so like they will last for. Decades. And do they do? We're seeing it now just like, yeah, light bulbs and like, little devices everywhere that like you wish would die but aren't. Yeah, you got your low energy LEDs. Those those suckers are going to burn for 20 years. They'll be fine. Yeah. Oh God, I'm.

Sorry for depressing. Giving me Jesus. This is one of the conceits I think of like classic cyberpunk literature of like, oh cool, I'm going to get a cool cyber arm and I'll have like super strength in that arm and I can change my fingers. How are you dealing with hardware upgrades? Go back into surgery like. Yeah, exactly how are they going to replace that for you like when the new model comes out? How do you update the firmware? Can someone else update the firmware in the midst of combat?

Yeah, there's probably a good movie there somewhere. It's probably, or a bad movie more likely. Was there anything that like, as you dug into this, really like surprised you, honestly. Obviously you're like you're dug into this to begin with, but as you were writing this, was there any anything that like like ha, I didn't, I kind of thought it would be the other way or I wasn't expecting that kind of result. I would say it was so it was surprising to me. It's just like all the ways you could play around with time with all this stuff that was like the you already brought it up like my favorite graph in this whole thing. It's entitled CDE and exploit publish dates versus Kev additions.

Yeah, there's also like two other things in there and it's a super fun graph. I'm gonna try to describe it. So I won't try to describe it, but like it creates this really like pleasing pattern, right? Of I have AI have a solid line of Kevs, like I get Kev updates, you know, 2 to 6 * a week every week since, you know, Kev started in 2021. And that's like the baseline, right? This is how this whole graph is, is built out. And then you can see like these like little waterfalls of like metasplate modules that were written, you know, 10 years ago, all kind of tracking along. And then and then it gets added to the Kev several years later. But then as time goes on, you'll see like Metasploit modules tend to cluster up a little bit closer to the line. The nuclei templates used to be published well after the Kev was published, but now those are falling down to like being published same day of.

And the reason for that is the project discovery people run run a little bug bounty for for exploits against certain kinds of vulnerabilities and Kev listed as a kind of vulnerability and so. And money is a good motivator. So there we. Go they are now. Sure, I have my own thoughts about how bug bunnies work and in the age of AI slop, but generally speaking, the nuclei stuff does tend to be pretty good.

There's a good community there writing like good good templates and not relying on just like automatic Yolo. But those lot but like it's really funny. It's fun to see like as Kev moves along and and expands out to more technologies, how the the public exploit development starts to coalesce around it, How CV ES are all over the place for a cab, which is another funny thing. Like you'll see things listen on Kev. Like, it would not surprise me tomorrow to see like a new 2008 vulnerability. Like, listen on, Kev, because old exploits work. Yeah, you will find you will find it in the wild. And so then when SZA learns about the exploitation, gets a chance to inspect the evidence, see that the patch actually works.

Like, it will get out to the cab. And that's just, and that's just kind of like they don't really care. It doesn't matter if it if it was patched 10 years ago or this week. No it doesn't. And you see those like fairly regularly if you pay attention to the Kev addition like RSS features around. Yeah, pretty routinely. Yeah, 08 is actually a good example. Like you could probably find several Microsoft like 2008 bugs that have been added to the Kev in the last six months per year every time. I see one surprising, I would say like the other thing that really that jumped out at me.

It was a surprise when you bucket all the Kev vulnerabilities into just like a series on EPSS scores. So EPSS is exploit prediction scoring system chance that you're gonna get exploited in the next 30 days. It describes a really fun inverse bell curve. So like 90 to 100% over well over represented, right? Probably gonna get popped again with this somewhere in the world, right? But then and then it falls off pretty sharply towards the middle and then it shoots up again at the end where it's like the zero to 10% chance.

Also pretty, pretty common to add those to the Kev and that's fun. Like I, I and so like it tells me that Kev actually does have pretty good visibility into very targeted attacks because and that's the only way you could do this, right? That's the only that's the only explanation I can come up with. That's the only one I can think of too, and. Oh, so you're getting intelligence on exploitation that nobody sees other than you?

This is what I wanted to ask you about that specifically. I'm glad you brought that up. Is I think that from the outside, the public perception might be in that, you know, I'll just say for myself, you might think like, OK, I hope that the federal government has some insights into this stuff that maybe we don't have and have intelligence signals that we don't have about exploitation because they run a massive signals intelligence operation and a bunch of other things.

The NSA is part of DoD, not part of federal civilian executive branch, right? Right. They're friends. Yes, they know each other. Yeah. And you tend the, the federal government would tend to get information from, you know, people that might have been victims of targeted attacks or they're seeing the same Intel signals from, say, Citizen Lab or EFF or whoever it happens to. Be tag or Yeah, yeah. Tag. So yeah, like those in some, it's not every case, but the sources of intelligence on this just run the gamut, right? Classified to unclassified TLP rainbow concerned researchers that see something weird on the Internet, right? Yeah, it's everywhere.

And so this is one of the reasons why, by the way, Kev doesn't really do a lot on attribution. Another thing, Kev gets dinged at all the time. It's like, well, if we attributed everything, I see where, you know, it's good to give credit and it's good to like tout your public private partnerships and all that. If we started attributing everything, it'll get real obvious real fast of which ones come from classified Intel. Those ones will not get attributed. So don't super want to put like a big target on that.

Yeah. Because like every once in a while, well, I guess it's, I don't know, you see it every once in a while in the Microsoft advisories. Like they'll thank the NSA like explicitly sometimes, you know, and NSA puts out, you know, public advisories every once in a while too. And you're like, yeah, OK, that's something to pay attention to. Like if they're being that. NSA is making a big deal out of it. I would say you should pay attention to it. Yeah, it's a good investment. Going out of their way to like, talk to Microsoft, then, yes, right. Yeah, in a publicly attributable way. You know, that is the signal of like, we would very much like Americans to fix this thing right now. Thank you.

Yeah. We have a good reason we're not selling wolf tickets like we're, you know, we know what we're talking about here. Yeah. It reminds me a little bit of, you know, in a, in a different way. You worked on Metasploit for a long time. The the sort of default bar for like shit's about to go crazy is this was just added to Metasploit. Like, you know, say 15 years ago. That's the way that we would look at things from the outside and be like, is there a Metasploit module for this? Yes, OK.

It lowers the part of entry right? Like. Exactly that was the argument. Right. And it creates an incentive, a little bit punitively creates an incentive to please, for real, patch this thing because like, literally everybody knows how to exploit it now. You know, everything's easy once you do it once. Yes, it's like putting together IKEA furniture. Like God, I could have done that in 20 there. Are some metisplay modules that are a little bit finicky and you got to fiddle with them a little bit. And like sometimes you've got like payload restrictions and all this other stuff that goes into like sure, advanced metisplate usage.

But like the rule of thumb, this was a a coinage from your friend of mine, Josh Korman. I am the cavalry. This is HD Moore's law of you must be this tall and the type is number of metasplate modules. Yeah, to ride the Internet, you must be this tall. And so I would say metasplate is a pretty good signal that the cat's out of the bag on this thing. I talk about nuclei a lot. We use nuclei templates in product at run zero and we contribute back. We're like not just open source leeches. We do contribute back.

But like a nuclei is is arguably much easier to write in, especially for like web app stuff, which Medsplate is not great at. What Medsplate is good at like stack based buffer overflows, right? Like that's Medsplate bread and butter. Nuclei I'd say is arguably better at, well, arguably easier to write exploits for for like web applications, right? Because it's very HTTP first. Whereas Medsplate was like kind of more telnet first. Well, it is from like 2003 or whatever. Yeah, metasploit's still great.

And they're still doing good, good work over there. And they've got like usually 2 to 400 contributors doing the thing and metasploit's still like a cool thing to list on your resume, so. Yeah, I would. I would say so, yeah. It is funny that it's crazy to think about that, that that framework is well over 20 years old at this point. And it was like. I worked on it when it was Pearl. You don't have to date yourself that hard time. I remember when it came out and it was like it was controversial. It was.

Like there were like 20 exploits all in one. It was amazing. When people lost their shit, they were like, what are you doing putting this out in the world? Like, you know, for, I don't know, people that don't know the history, go read the history. But like, it was extremely controversial. Like you know it. The the problem Metasploit and Nuclei solve is like I would like to test the patch or I would like to test my detection techniques with real with like live ammo, right? And also I'd like to do that in a safe a way that I can and so I don't need to be pulling, you know, rando. Please compile this C++ thing from pack and Reddit. Or right.

Whereas in those like I'll get a shell but also this guy will get a shell. That has definitely happened. Everybody gets a shell. Everybody gets a shell. Why not? And so like one of the things we contribute to nuclei here at run 0 pretty routinely is like things that maybe, you know, they're they sometimes nuclei templates don't clean up after themselves when they're doing exploitation, which can leave like stuff like stuff like just littered on the target that you don't really want stuff that will like definitely like an EDR in two weeks when they finally update their signatures. Yeah.

So we'll we we're usually on the janitorial side of nuclei templates. I love that description. That's great. Before I let you go, tell me a little bit about the Kev Collider. Yes, I'm glad you asked. So I read this whole paper with like four or five different like ways to slice and dice Kev issues. And then I thought, well, that's cool. But this paper will get out of date real fast. So we put together this web app.

That is all it is, is a view of the Kev and we keep tracking, right? Like we add whenever there's new Kevs, we updated and everything, and we track the Cavs in a way that like exposes all of the cavology that you just learned about when you read this paper. And so now you try and so you can and you can pick the things that are interesting to you in that moment. So like, let's say you want to just like focus on denial service bugs. I don't know, you're weird that way and you just want to do that. So you go through and you pick the CVSS like, you know, availability high, you know, is the, is the impact and just pick that and then deselect the other two for extra fun, right? And so now you're left with a list of like, I don't know, 10 or something that are on the cab. But now you know it, right?

And you can just like, and it loads in your browser. I really like the tech in it cuz it's all very static. It loads everything in your browser all at once. So like it feels very snappy and all the filtering is expressed as GET parameters. And so all you have to do is like share this link with your friend and they will see exactly the same view that you see. There's no saving, there's no session state, there's no like login or then certainly no like pay wall or anything like that. Yeah, it's cool. Like a test bench of like, hey, I was thinking like, what if I took all the EPSS that is like in the 90th percentile.

So like very likely to get exploited according to EPSS, but also don't have a network component, right? Like show me those. Like that's a cool thing to look at, right? Because that would tell you like your high value privilege escalation bugs essentially is what that would probably end up being. And then you can start looking for patterns, right? And especially if you start looking at time, you look at like the deadlines that says release because like usually they release with like a three-week deadline. Sometimes they release with a one day deadline.

Yeah. I want to take a look at those. Interesting ones where you're like, oh. Yeah, you can mix and match like all of the all of these facets and we expose like, I don't know, 15 or 20 of them. So you can easily like configure yourself into a place where like no hits. So then do that and then back off a little bit, a little bit, a little bit. Yeah. Start finding like the things that, you know, pass your filter for priority. Again, you definitely want to patch everything that's on the cab. You want to make sure that you have something there, right. But for prioritization.

And it gets you into a place where you can start to see like when there's a new vulnerability, like, oh, let's see, like how it shakes up against my existing filter that I've saved as a bookmark. Does it pop there? Like, that's cool. I like that a lot. Yeah, it's very cool. I was I I think the I probably spent more time playing with the Collider than I did actually reading like the. Oh yeah. Sorry about that, but no. No, no, no, no there. I read the paper too. The. Collider's lots of fun, and it's called a Collider, so it's.

Called a Collider we're smashing. Exactly. Vulnerabilities together and see what falls out. Yeah, and it's got like a cool robot logo, so go check it out. It's got a cool robot. It's got a, it's got Doctor Kev, I think is the character name. He's a mad scientist who is a part machine, I think. I'd love it. This is great. This is so much fun Todd. Thanks so much for your time man. Go check out Todd's various podcasts that last like 6 seconds. If you need if you need a quick six second break, one pieces is for you. But I would be remiss if I did not say the words runzero.com. Try try Runzero.

Go, go, go. All right. Thanks so much, Todd. It was great to see you man. Thank you.