The Future of Vulnerability Management With Jeremiah Grossman and Robert "RSnake" Hansen

It's, it's unfortunate because they don't really know who I am at all. And so I, I kind of walk in and they're like, oh, here's some random exec. I can, you know, just kind of bowl over with technical jargon. I'm like, no, this doesn't. Work it's, you know, we should probably be more polite to them, but at the same time they sent that person to represent the company. We're we're meant to like, try to figure out how this stuff works. I mean, isn't that what we're there for? It's like it's kind of their fault not knowing. It's exactly what you're there for and honestly, like you said it, it is their fault for sending those folks. Like if they're not prepared, that's not your fault. That's the the vendor's fault, man. Like you, you can't just send the sacrificial lambs out there, especially to Black Hat I.

Mean we're not, we're not trying to make them look dumb. We're like, how does this work? Right. Well, I there's AI to try very hard. Yeah, I know. Well, that's the hard thing. I mean, I want to, I want to get into what you guys are doing now, but like everything is so cyclical in this industry. I mean in life in general, obviously, but like this industry especially, we've all been at this a very long time. So we've seen like four or five different cycles of almost every conceivable product category and, you know, iteration of these things where it's like this but now with Wi-Fi or this, but now for IoT or this, but now with AI, you're just like, OK, you forgot. Cloud, but yeah, that's great.

But. We're still going through that. But yeah, I mean, everything just kind of goes in these like 7 to 8 year cycles and you can kind of, you know, see it coming on the horizon, you know? So tell me about what you guys are doing now. I mean, I, I know a decent amount about it, but I, you guys haven't been super public with it so far. So Root Evidence is your newest venture. You guys do a lot of other things together and separately. But Jeremiah, tell me where this idea came from and what what you guys were hoping to sort of achieve with this. Sure.

You know, I think I want to tell the story a little, a little different because it's your podcast and I think you'll appreciate it mostly like how Robert and myself work together and with our team. What we love cybersecurity, This is what we do. We've been doing cybersecurity longer than we haven't. And what we love more than anything is finding the biggest, most important problems in the space and developing tech to solve it. That's what we love doing and and what most people don't see behind the scenes. When Robert and I talk together, we fail a lot. We try to find problems.

We try to design solutions and it's and it's and it fails and we try to fail as much as possible until we find something that vaguely works. So before we talk root evidence, we'll just talk bit discovery. So you can see how our thought process goes. We knew attack surface management works. There's our focus was a big problem for for years and we try to build tools to solve it. Deceptively hard problem. But what what really happened in about 2017 is we are, we were talking with lots of incident responders as we normally do for actuarial data and they were telling us about 1/3 to 1/2 of the incidents they're responding to had something to do with the previously unknown asset.

Everybody will classify it as they didn't patch, that's true. But they would have patched if they knew the asset existed. And so, you know, so after failing for like 6-7 years trying to build good ASM tech, you know, on an offhanded comment, I go, Robert, you know, the only way to solve this problem is if we have a copy of the Internet. And Robert being the guy that he is, he takes me seriously and goes spends a year and a half of his life and comes back and goes, I have a copy of the Internet. We can do, we can do ASM now. I was like, I don't know what that means, but can you show me? So, so I'll let Robert take it from there. What what that actually looked like before we get through in evidence.

I don't know what that means. Well, what it looked like. Was probably $100,000 a year and a divorce but my I. Remember this part of it. My basement was a de facto data center and it was actually so loud that you actually couldn't go down in my basement without ear pro on because it was just the noise. Fans were just crazy. But, but, but basically what we did is we figured out that you can theoretically grab all of the metadata for every single asset on the Internet. I mean, it's, it's possible and it was even possible to do in such a way you could actually kind of fit it on one very, very beefy for The Time Machine, but you could do it. And so then, OK, let's say I have that data. What can I do?

Well, I can, if I'm smart about how I index it, I can actually query it. And that, that is the part that took me a year and a half was making it so it was queryable. It, it, it, it's not so much the data collection that was hard. That actually was not that hard. The really difficult part was how do you do this and make it very fast, like ideally sub second fast. And so, yeah, that's that was where I spent a huge amount of that time. So so then, you know, we get the prototypes going, we raise some money, we bring the team together because there's a four person, five person team that we bring together to do these start-ups. We form a company and we hard charge it for for three years and then tenable acquires us, you know, like clockwork. And you know, that was a good a good outcome, a good experience. Our tech goes to a big company and it can get in front of 100,000 companies like and then, you know, there wasn't an ASM space and then there was like, we helped pioneer that. That's really cool.

So then we get over to Tenable and we have a vulnerability management background, but from Absec. So now we're in this vulnerability management world from CVE land and, and we talked with a lot of customers a lot, big and small and everybody has the same complaints. Everybody's buried in vulnerabilities far more than you can fix it now or anytime soon. Then they all have to prioritize. But then the prioritization problem, no one likes the current standard models out there.

Everybody customizes because that everything's broken. Then when they figure out what they want to fix, they don't know how to justify the spend of patching relative to the risk reduction. And the problems go on and on and on. So we're like, OK, and, and then, then the breaches are happening. So as we do, we go to the our actuarial contacts, we talk with a lot of incident responders both inside and outside of cyber insurance. And here's what's wild. What they tell us is that 30 to 50% of the breaches and financial losses, the claims have something to do with a remotely exploitable CVE vulnerability.

So if we can fix that part of vulnerability management, fix the right volumes at the right time, we can make a huge impact. So we have to start doing some research. This level of new thinking and disruption, we had to do a start up for it. We couldn't, we couldn't do it at at Tenable. There's a they're doing different things, let's just put it that way. So when we left, we spent two straight years literally meeting with 100 companies 'cause we do our homework, 'cause it's in this space, it's very easy to build a cool tech for a problem that doesn't exist. Oh.

Yeah. So lots of those, yeah. Yeah, RSA floor is littered with them. There's dead bodies of those technologies, yeah. And, and, and for us, there's one thing we do and one thing we don't do. When we talk with customers, one thing we do is we try to become the world's foremost authority in the problem. We want them to tell us their problems over and over again until we get the problem statement down. What is the problem? Then we can on our side because it's our job to design the solution that we bring back to them and go, what do you think?

You know, if we built this, what do you think? So two straight years until we've really locked it down and the answer for us really goes like this. And you have to understand one thing and everybody will know this. There's over 300,000 Cdes now cataloged, growing at right around 40,000 a year. And it's everybody knows with AI and stuff like that, more, more AI means more software. More software means more vulnerabilities. Sure, we're already buried, but what no one's looking at is what's actually getting exploited. If you look at the data that we have that we're able to collect, not not just from like the CSI Kevs and the Volm check Kevs, when we talk to the actual incident responders, what is getting used, It's really less than 1 to 2% of Cdes in total that are getting used for a financial loss. Now one step further than that to really understand the problem is that that number 1% has been the same this year as last year as the year before.

That indicates that we are getting it really wrong as an industry as far as what vulnerabilities were patching when. Because if you're patching the right bones at the right time, the adversary would be forced to innovate, use new vulnerabilities and not just scale. So we know for a fact we're getting it wrong. So what we decided to do was 1 is download a copy of the Internet pre normal. Behavior. Yeah, yeah, this. And I'm speaking for Robert. Like I had to do this stuff like no, I, I, I come up with stupid ideas and I go so Robert, we have to do it a little bit differently this time.

He goes, what? I go, well, I want you to to create a tax surface maps ahead of time for every company on the Internet. So, so, so can, you know, work with the team and go solve that. OK. So and and then when it comes to vulnerabilities, focus on just the ones that the vulnerabilities that reduce down to financial loss, which are in the hundreds. Then you look at the catalyst and then it's everything after that because and then you which ones you fix when has to do with your patch capacity. If you do that, it's a whole new world. You don't get hacked through a remotely remote vulnerability if you do that.

Yeah, I think, I think this is, it's worth saying as well, one of the things we did in our homework assignment was we downloaded everything we could in the vulnerability management space that's public. So like all of this CDE lists and NVD and the Chinese data sets and anybody who has vulnerabilities, right. And we try to like marry up like, OK, well, this group says you just need to fix these voles, these group, you know, these catalysts say you need to fix these and so on. And we try to marry up all this stuff. And then, you know, you know, the metasploit lists and what has exploits and what doesn't have exploits associated with it, you know, actual exploit code associated with it. And the weirdest thing, Dennis, none of them agreed. They didn't agree.

There was no. Overlap really between? You know, not really, not really. There's like some of the Kevs had the, you know, some Kevs are bigger than the other Kevs and they kind of consume each other. So there's some overlap there. But for the most part, no, there really, really wasn't. And a lot of the, the prior knowledge, the people like institutional knowledge, like if there is exploit code for vulnerability, it is more likely to be exploited is actually not true. That actually does not seem to correlate really at all. Or you know, if something's on the metasploit list, it will be likely to financial loss. Kinda, not really.

I mean a little bit, but I mean it's better than a flip of a coin, but not by much. And. Sorry to interrupt, that's really interesting to me because I'd love to hear your theories on this as an offensive minded researcher, Robert. Like those are, those are always like 2 markers that you know, people like me would look at like is this in Metasploit? Is there some other version of public exploit code available? If so, then that should move up the priority list. Do you like?

Robert should talk more about this, but I, yeah, because I was going to ask Robert the same question and go into it like there's red teams out there that write exploit code so they can use it. That's not the adversary. So Rob, you got to go into the stuff like what we learnt on, like why they don't overlap. Yeah, that that's exactly the point. Like there's other lists out there too, like if you and they kind of drive things like EPSS, for instance, they come from like Fortinet as an example. And so Fortinet has all this data flowing in of like these active exploits being used in the wild.

And people are like, well, look, there's evidence of exploitation. Well, there's evidence of exploitation that has possibly some correlation with a bad guy. But more likely than not, it's just random like bug bounty guys and random offensive researchers and this whole, this whole ecosystem of us probing the Internet, like guys like me hitting the entire, hitting every IP on the Internet or something. And so that is not a particularly good indicator of actual financial loss though it turns out. And so I think we've been kind of over engineering our industry to really be focused on each other instead of the adversary. I think that's really where this comes from. That's a really interesting point that I think, you know, I've heard other people make that in various different ways over the the course of time.

Like I think the most recent example I can think of was that that React bug from December, you know, that turned out to be a pretty a relatively big deal. And one of the everybody was sort of freaking out because there was exploit code out there and all that. And there was a bunch of attack volume. But I think it was grey noise that pointed out like, look, everything that we're seeing is red teams, like almost exclusively red teams, red teams and other like known non bad actors, you know, which is kind of an interesting thing. It's people trying to figure out if they're vulnerable and, you know, bug bounding guys or people like you, Robert. Right.

So what's what? I'm sorry. Go ahead, Robert. Well. I was just going to say that I think if you if you kind of blur your eyes a little bit, yes, it might turn into something bad. I'm not, I'm not saying that it won't turn into something bad. And I think that's that's a large part of what people want to know about this. It's like, OK, yeah, but what's going to happen next week? So a bunch of security researchers are talking about it right now. This is this is where OSN gets really confusing because you're like, well, adversaries are talking about it. Yeah. But does that mean they're using it? You know, like there's there's a very important distinction. And it really comes down to how, how easy is it for them to use to get the most out of whatever they're trying to do. And, and it's sort of like it's, it's exploit code availability from their perspective, which is different than the researchers. It's like utility from their perspective, which again is different than the researchers, etcetera, right. So it's like you really have to put yourself in their shoes. And it turns out that it's just not that many volumes are in that perfect sweet.

Spot. And and if you, if you look at it, the React one that you just mentioned right there, if there's one this month or last month or whatever, how many are are those every month? Yeah, it's not that many. If you look at like the Mandiant list, just an existing example, their their list of, I forget the term initial access vector vulnerabilities or something like that. They only have 12 on their list. That's it from 2000. 12. Yeah. And and that's the one thing that seems to kind of overlap with the data that we think is actually good, which is the DFIR guys.

So we think actually Mandiant's right. We think that there really is only, you know, dozens or whatever, maybe 50 Max per year kind of thing that come out. But importantly, most of them are not 0 days. They're not the ones that everyone's freaking out about. They're the ones that have been around for like 4 or five years and people have been kind of sitting on them and finally someone figures out how to weaponize it exactly the way they need to, and then it's off to the races. So it's not 0 days. In fact, 0 days are almost never the thing that is actually used in the wild.

So the, so you know, the premise is, is that you have to scan everything that you own, let's say Internet facing every day. So there's two vectors there that are really, there's actually 31 is the right vulnerabilities, testing the right vulnerabilities on everything that you have, which is a very hard problem. And then for a lot of companies doing it every day, because a lot of times the scanning tech doesn't scale and even if it does, it's cost prohibitive. So those are the factors that go into the next generation of solutions, which we're designing.

And if you get all that right, we're laser focused on not necessarily reducing breaches, but reducing financial loss. Those are two separate things. Everybody else can chase patch statistics. We don't want companies to lose money. That's where we're fundamentally different. I wanted to talk about a little bit about that zero day, non zero day, you know, existing bug sort of dichotomy there with you guys. Because, you know, people like me, idiots on the Internet love to talk about 0 days and how cool they are and like how dangerous they are and all that kind of stuff.

But like if you've been at this for more than 1/2 an hour, you realize that they're not used all that often. Like you said, Robert, you know, and when they are, that's like, that's kind of something, you know, you notice because it's rare, you know that it does not happen all the time. I've had a million discussions with you about this over the years of like, look, I don't need that zero day. I know who their web host is. I know they're running this version of their hosting software. I can get in with this bug from four years ago.

Or I know who their support, you know, their help desk company is. I can call them and probably get in like I don't. I don't need some crazy exploit game. We're seeing exploits being used in the wild that are back from 2008. Yeah. It's crazy. And so clearly the bad guys don't care about age. They only care about utility. Why do I care if it's came out today or last week or a year ago? The only thing I really care about is is population density and utility and those things that affect them, right?

Like they don't, they don't care if they can break into one thing. I mean, maybe that's kind of interesting, but they need to rinse and repeat. They need to do it over and over again. They're never just going to be like, all right, we did one. I guess we can retire now. It's not going to happen. So in our industry, we, we hold on high the style points, the art of the hack, all fine and good for our for ourselves, you know, to break the bounty, to break the challenge. That's not how they do it. Using a zero day means cost. They have an ROI model. If you'd used a zero day, that means they had to invest in finding the exploit, tooling it up and deploying it. They don't want to take that on.

They're they're in the business for illicit gain. Why would they want to do that if they don't have to? Right. If somebody else has already done that work for them or part of that work for them, and they can just piggyback on that and, you know, achieve their goal, then yeah, why mess? With the rest of it. And by all the statistics, it's working. It's still working despite a 25 year old vulnerability management industry. 1% of volumes is causing 30 to 50% of breaches. This, this should scare us.

You know, This is why we're here. How in your discussions like you know, before launch and after launch and all that kind of stuff with the customers, you mentioned all the homework you guys have done, How much do the customers understand what you just explained to Jeremiah that like the massive population of bugs really isn't their problem, It's this small subset of them now? Honestly, we layout the problem for them. They go yes, absolutely. And they get the premise instantly because if you really look at it, they're already doing it. That's why they all customize their prioritization models. No one uses basic CVSS or anything from a vendor. They're modifying it by value of the asset. Is it is the vulnerability, does it use authentication or not? Things like that.

They're already reducing the volume down to the best that they can. So they get our model instantly where where we use them for because, you know, we've all agreed what the problem is the general concept of the solution. So we work with our design partners every few weeks we may make a little progress. We go like this and they go no like that. We make a little progress and because basically they drive our product road map on implementation, you know our concepts are sound. Now how do we make it the most usable and actionable we can inside the organization? More than just the concept.

Now the hard part begins. Yeah. I wondered, you guys have both been Csos or in that sort of seat, how much of this reliance on these big vulnerability management tools and all that kind of stuff is kind of driven by like, well, I bought this thing so we have to use it like we've, we've got a, you know, we invested X hundreds of thousands of dollars in this. I think, I think it's not that. I think it's compliance. I think that's the reason they they have a compliance mandate. They maybe don't even really understand why they're doing it, but they have to do it.

It's been, it's been foisted upon them by somebody. And so they're like, OK, well, what's the minimum we can do? And we've actually heard a lot of scary stories like #1 the cost of the VM program exceeds the cost of breach. That's terrifying #2 we've heard people literally turning off the signatures. So it's running, but literally giving you no, no data. Because that is actually a better ROI than running it with it turned on and people think you're crazy. What are you talking about?

It's like, actually they're not crazy because the the expenditure of time of them wasting on a bunch of vones that don't matter turns out to be not a good expenditure of time. So they're actually saving money by not working on any vones at all, despite the losses, even including the losses, which is crazy. We're, we're hearing just unbelievable things right and left from these customers. You, you kind of have to be there to believe it. And it's kind of shocking to hear for the first time if you're not in the room. But honestly, I, I think everyone's operating with pretty good sense of the fact that they have to do this cheaper. They, they're not, they're rational people. These are not crazy individuals.

They they know that they're spending 99 point something percent of their time on vulnerabilities that don't matter. And so how do you reduce that? How do you get rid of it? And we're coming with the, we're the only company out there that's coming with actual real data, not just hey, you know, put your finger in the wind. Hey I hope these vones are the ones that matter kind of thing like. What what Robert's getting to is something very core. That is, everybody is currently missing the the the customers, the companies out there get it intuitively. No one has given it a name.

What they need for their program when management asks is a defensible strategy. What did they fix, when and why? Because eventually Vadnais will happen, a breach will happen. How could this happen? They have to have a defensible strategy for what they were doing because, and that enables what our industry is supposed to do trust. So in a vulnerability management context right now, when a breach happens due to remotely exploitable vulnerability and the board asked the CISA or the Vaughn management team, how did this happen? The answer almost universally is we patch a lot of vulnerabilities.

We hadn't got to that one yet. We missed that asset. We didn't know it existed or we're overwhelmed by vulnerabilities. We just that that's the answer. It's an unfortunate answer, but it's the the only one they have where what we want to do is to is be able to, the answer is we have fit. We, you know, if, if a breach occurred to a vulnerability, we have fixed every vulnerability that has led to breach and financial loss that we're aware of. The one that hit us was brand new. We were the first.

That's a much more defensible position. Yeah. That's a lot easier conversation to have with the board than we just didn't get to that one or you know, I don't know, it was over the holidays and that shit happens like. And then at that point, then you might be able to have an argument that the adversary was sophisticated then and only then. Right. Yeah. Just because they they hit you does not mean they're sophisticated. Yeah.

But if they're if, if they're using a brand new volume, maybe just published or a zero day, OK, there was some, there was some investment there to hit you or and maybe a few others first. But if we can get to that point and get the feedback loop going, we know of a company, you know, a customer or a company or the incident responders tell us brand new vulnerability is now causing loss. Loop it through the whole ecosystem fast. And so that's where we come in. We need a very fast, scalable performance, you know, system to scan everybody for those one or two things like right now.

Yeah. I think this is a good time to kind of dovetail slightly back into the fund. Not that we're going to dig into that too much, but one of the things the carriers gave us was sort of a laundry list of things that matter and don't matter and went out to breakfast. Jared and I have a really good relationship with these guys. So we can just have like lunch or breakfast or whatever these guys. And one of them told me he's like, Hey, I, you know, we've got enough years of actual royal data. We know which things work and don't work. And I'm like, would you, would you give that list to me? And he's like, yes, so he did.

And one of the shocking things that Jared and I found, you know, through talking with this one guy and some others is that things that you think are on there and should be on there or whatever are definitely not on there. And I'm not saying they're not on there because they're thinly deployed and they don't have enough data. They're they're not on there. And these are well known big chunks of our industry. Like there is nothing on Web App SEC at all on that list. There's no firewalls on that list. There's very meaningfully large things that do not move the needle from a cyber insurance perspective at all. Why do you think that is?

I mean, you would know better than I would, but is it? We've got our theories. Yeah, OK, so I'll give the I'll give the firewall one and Jr. can do the web one. How about that? Well, I think the firewall one is because firewalls are how people break in now. I mean, if you've got an Avanti or Fortinet firewall, that's how they get in. So it turns out if I've got one, that's just another initial axis vector. So it doesn't really change the calculus from their perspective. Like I really hope that Palo Alto firewall isn't exploitable because if it is, you know, this whole thing goes down.

So it turns out it, it is not a good indicator of what you're going to get breached or not. Go ahead. Sure. So, so it's one of the concepts that we, we wrote about and we, we, we presented on called the epoch theory cybersecurity, that there's different eras of time, what adversaries do, how they increase their sophistication. So if we go like way back when to like the turn of the century, literally there, there were no network firewalls in the world. So you just go straight to a corporate network and you break in. That caused everybody to install perimeter firewalls and block off all ports except 80, because 443 wasn't a thing yet. And so all the attacks were forced to be pushed to the ABSEC layer in the 2008 era.

We knew about the volumes in 2001, but the adversary didn't care because the firewalls weren't deployed then. But by 2008, firewalls were everywhere, so you had to attack the web layer and then 443 came online and all of a sudden between 2018 and 2016, it was an ABSEC world. Adversaries went after it. You know, we were there at the time ABSEC got more secure and it pushed everybody to the authentication layers, you know, you know, anything that didn't have two FA phishing scams, you know, credential stuffing and things like that work are kind of where we are work are today.

But now, you know, now these, these VPN concentrators and network firewalls and those central points, they are just web apps too. So they get they get targeted. So that's where the current thinking is. Every time we do things right in cybersecurity, we have to expect the adversary to shift. If they are not shifting, we are not doing something right. That's a great point. Yeah. I mean, the firewall thing, like we all laughed about that, but it's entirely true like that their initial access vectors. I mean, if you want to talk about like a library of existing vulnerabilities, just go and look at those big, you know, edge device vendors and you have your pick. It's The Cheesecake Factory menu. You can, you know, you got whatever you want. And these are interesting things will come. Like Robert said, we, we speak with the carriers and the, and their incident responders weekly, if not daily. And you know there's depending on the, the, you know, the the carrier or the responder,

they'll say if you have this device, we are not covering you. Period. Like if you're driving a Ford Pinto, we are not giving you car insurance. And we're going to see that more and more. And they're going to be right. Like anybody who has this stuff, there's a claim because from the carriers, this is like the most wild thing because we come from cybersecurity. We talk to the insurance guys, but we're cybersecurity. The the carriers are not interested in predicting breaches or stopping breaches. They want to predict claims.

That's different. Yeah, right. And they want to probably predict like groups of claims that are clustered around some specific costs. Yeah, they don't, they don't care about one off singular claims. I mean there, there are some exceptions, but it's very rare. They're worried about the bulk of them like if they can solve 80% of it and then that's just 80% more revenue for them. Great. You know, so yeah, you're exactly right. They're really focused on these big mass of vulnerabilities, which is why they have all this trend line data.

And until very recently they didn't have enough of it to start making decisions. But now they finally have enough. They've been collecting it for long enough. And so like that's, you know, so like in the business, you know, the business quote UN quote has the same prerogative as the carriers. If if a breach doesn't cause a loss, does anybody care? Yeah, almost nobody. I mean, it's yeah, they they really don't. So we as an industry, we, we haven't had the actuarial data everything. If you look at our entire industry, it was our best guess of how we were getting broken into, you know, by who, how, why it was a best guess, no actuarial data.

So our whole industry said we're trying to protect all things at all times. And it was just, it's just unnecessary. But in our defense, we had no other way to go about it. Now we're actually getting have lists and actuarial data. So now we can shift the spend the focus to the areas are most likely to cause the adversary to force to shift. So that's what our company's about, what our fund thesis is about is forcing those shifts. Yeah, that's, it's really interesting because for most of like our careers, certainly there wasn't any like single source of truth you could go to to get that data.

I mean, mostly what we had was either anecdotal data, you know, from like people you would talk to at a conference or something like the DBIR, which would have a group of, you know, some subset of data from incident responses that that, you know, that Verizon went to or maybe Mandiant or somebody like that, anybody that publishes something like that. But you know, that's, you know, just by nature biased because it's their own customer set and you only get a little window into what's going on so. The funny thing though, I mean while I agree with everything you just said, and if you look at the Veris database, I think there's only like 31 Cdes total in the database. 31 there's. Something, there's something crazy. Yeah, it's super, super small.

That's probably closer to the right answer than the full CDE list. You know, it's like if I had to pick one, that one's way closer to being right because it's, it's real data that actually led to actual claims. And some DFIR experts spending countless hours digging through logs like that's that's real data. We spend a lot of time on fake data or not fake in the true sense of fake, but fake, like something about the log is telling us something that's not quite what we think we're reading. And we just take it face value.

We're like, OK, well, that's what it said. It's like, no, no, no. You really have to think through this quite a bit to know you're not. You're not looking at garbage data, basically. Yeah. And I mean, with a lot of those reports, you can sort of make them say what you want them to say. You know, you can look at any given data set and be like, well, we're a healthcare company, financial services getting crushed by this problem. We're not really getting hit by that. So we're cool, we're fine. We don't really need to worry about that. But then you know. For the moment. Yeah, exactly.

Yeah. Next year you could be on the front cover of that report. So that you said that means we need a feedback loop, a very fast feedback loop in this scenario when the first healthcare provider gets hacked. OK, cool, let's protect the rest. But does that? I mean, I know that that happens like anecdotally and peers talk to each other and that kind of thing, but it's not really formalized at least in the in as far as I know. I mean, you guys would move there. That's that's why we're forming those relationships who, who has the data?

We're getting that together because no one's done it before the into the responders. They do great work, but they don't look up either. They actually, it's been very interesting to us. They don't get the value of their own data that they have actuarial data. They're doing job to job. They get paid, you know, they go here's budget for 100 hours, try to figure out who who did it and how like that's their job. And then when it's done, they move on. They don't really think or have time to communicate that back as a set of data back to proactive industry. They don't do that, not because they don't want to, just no one ever asked them before.

Right. And weirder, some of the things we found out in the back end are really interesting. A lot of the software literally doesn't even proactively like try to get you to enter the initial access vector. And so even if you wanted it, there's really no way to to elicit it out of the DFIR expert who's working the case. So unless they somehow come up with their own list, you know the software is not going to do that for them. So this is for us, this is where the bleeding edge is and where where we improve our again, our theory is, is that if we use actuarial data in a proactive context, we can make a lot of ground and it, it seems very simple.

We think it is. It's it's hard, but it's a simple concept and it's time. Is it in the discussions you guys are having with customers or potential customers? You mentioned, Jared, that that like as soon as you explain it, like something kind of clicks and they're like, yeah, we get it. Is there any common pushback or not problems or like feedback that you've gotten from folks that you're like, oh, OK, yeah. No, because we're not trying to necessarily supplant the what they're already doing with the other guys. That's not our intention.

Our intention is when we say this vulnerability is important, this one you fix, we want to reduce it down to dollars and cents. Like for instance, it's going to cost them time and money to fix. Well, how much time and money to fix? We want to help them do the calculation. This vulnerability has led to loss if exploited. It's an estimated loss for you based upon the data we have of this amount. Now let's calculate and estimate how much it's going to cost you to patch. If it's going to cost you, let's say 100,000 in time and money to patch and it's going to retire $5 million at risk, you do it. If on the other hand, for whatever reason these volts happen, if it's going to cost you 5 million to fix and only reduce by a million, maybe you don't. Yeah, it's a fundamentally different conversation you have and they like it.

So there's no push back on it. It's just how do we do that and how should that look? Now, I, I mean, obviously there's a lot of sort of secret sauce or whatever that goes into what you just described, Jerry, but can you give, give us this kind of a, a quick primer on how those calculations are done? Obviously, you're using this actuarial data that you guys are pulling in, but say if I'm a financial services company, how are you doing that risk benefit calculation for them?

Here's one example. It's one report that people can look at like the the IBM cost of a data breach report. They, they put that out there and it says on the broad average for mid tier companies, when you get breached, it's a $4.4 million loss on average if memory serves. Then you can parse the data down on industry, size of company and things like that, that if there's a substantial breach, what do you expect to lose? So those numbers are published out there. They're not perfect. Perfectly accurate. They're not meant to be, but we have similar data that we can augment internally that we get from our data sources to go for this company at your size, at your revenue, this type of home equals an estimated on this critical asset because it's revenue generate whatever should equal about this, it's not perfectly accurate, but I'll be directionally correct. You know at the then how much does it cost you to fix? Well, we can tell you how many places you're vulnerable to it. Is it one asset or is it 100,000 of them? And they have to tell us, you

know, on their side, how many buttons do they have to push to fix it? And they, we can give them the, the formulas, but they have to plug in the variables, you know, to, to estimate it. And then you just put those together. So that's how we would approach the concept. But those are the concepts. But now visually, you know, so we want to get the security team to be able to report dollars and cents to the business. We need this amount of time and money to retire this amount of dollar risk. What do you want to do guys and make it out like that? No more red, orange and green.

No more 9.2. This is this is this is elementary school cybersecurity. No, every other space does not do color coding. No, yeah, the stop light color coding and the, you know, CVSS and everything like that, that just, you know, is in every vulnerability advisory. That should sales going back to the CFO. So we're gonna go after the green accounts, right? Yeah, no. Like all that stuff.

Like it comes without context. Like, OK, this is a, you know, named critical RCE, which we know is in something that's widely deployed that could be very bad. But if you have that and somebody gets into it and they can't go anywhere else from that, it's not that bad for you. Like, you know if the if it's a dead end for the adversary, in that case you don't give a shit. You know, but what's really what's really interesting is we've talked about over the course of the conversation, the number of vulnerabilities that matter at any point in time is not that high.

So if you fix those, your real vulnerability volume that's concerning is way smaller than people actually think it is, That we don't have a vulner, we don't have a patch problem. That's not what we have. We have an analytics problem. Yeah, we, we've always had that and it's, I don't, you know, there's lots of our friends that have tried to, you know, fix that or solve that problem in one way or another over the years and have in various ways. But it's not like it's not a completely fixable problem, I don't think.

We, we think, you know what I'm blanking on the name Robert, which they call the fair model. You know, there's, there's fair and other financial models like that. I we think one of the challenges that they had was one is hard to implement. Everybody knows that, but it was they were designed devoid of actuarial data. That was what that was the world then, that's not the world now. We have actuarial data now. I was gonna say, is that be that data didn't really exist then, right? Like, right. Yeah. OK. About 10 years ago, you know, when they came out or something like that.

And also that's changed. And also the tech resources, the idea of scanning everything all the time, it was prohibitively expensive to scale at that level. We have cool new tools now, cloud environments and all sorts of ways that we can drive the cost down. So the tech wasn't available to make the right solution. Now there is. Right. Yeah. And I wanted to go back to something that we were talking about earlier with this sort of like, you know, 0 days as opposed to like this long history of existing bugs. And, you know, a lot of the exploitation that we see, you know, let's just say everyday, a lot of it is opportunistic stuff.

You know, there's attackers, you know, cybercrime groups, probably some AP TS that are just like, let's see what we can do with this Cisco bug today or let's see what we can do with this, you know, Fortinet bug today. And you know, if you don't have those, you're not that worried about it. But let's say you're a high value target for an APT, you know, if the Chinese are coming after you, if you're a financial services company or government agency, you know, and then you're worried about those rare bugs. But the conversations that I've always had with defenders in those organizations, they're well aware of that. And they're like, look, if those teams have us in their sights, they're probably getting in like we, we can do the best we can do.

But they're professionals. They're literally, this is their job. They wake up and do every day. So, you know, like the old if the Mossad wants you, you're going to get Mossadded. You know, and, and also VM doesn't go. VM has never solved the zero day problem. I mean, in fact, I don't know any company that's even really very close to getting to the point where they would be able to solve it. At least it can't. It can't. How do you stand for a vulnerability that you don't know exists? Yeah. So unless you're unless you have something like Expo for networking or something, this the idea that you're going to be doing that and especially in a production in a safe manner is basically zero. You're not going to you're not going to do that type of analysis.

So you really need other kinds of tooling, which goes back to what I was saying before, like some of these insurers and reinsurers, what they're telling us works and doesn't work. A lot of what works to solve this problem is more post detonation reducing dwell time type technologies. OK, because you have to assume the breach happens. It does. It will happen and that should be OK as long as you catch them quickly. If you let them root around your network and do a bunch of stuff for a couple weeks or months, you get what you pay for.

So yeah, I, I think it's you can't, you can't boil the ocean with 1 technology. You do need a layered defense, but not in the, the layered like like network defense type layering, you know, the same thing 20 times in a row. It's more like different technologies solve different problems. And and So what Robert's getting at here is something very important to her, you know, because we make investments as well. Like this is our thesis. Like we want to reduce financial loss, not necessarily breaches. So we have evidence that kind of is more on the proactive side, but we have to assume, you know, remote breach, the adversary lands at a point and then can you detect them through a Canary or something like that relatively quickly? So, you know, you know, we, we look for those investment opportunities.

Then if you get infected with something and you know, it beacons out to AC2 server that's detectable, you know, those sorts of things. And so you can find those types of solutions that reduce financial loss. And those are the solutions that we look either to build or to invest in and, and help out. Oh, and, and real quick and you know, we, our, our fund already had its first exit wire speed. Wire speed was a AI enabled MDR not for not for detection of the event, but for response to the event.

You know, so you see something weird, use AI to respond and ask some questions and do the, the human part of it. And it was if the company was acquired, it did a great job and it reduced the cost of MDR. That's why we were there and it it worked. Yeah. Another thing I think you were kind of almost alluding to, Dennis, was, you know, nation state actors, they they tend to act differently than people who are purely financially motive motivated and in many different ways.

They will spend a lot more time on a single target. They will burn odes. You know, they, they really are doing this nice low and slow thing like they, they look and act very, very differently. And that's more where your Kev lists start becoming useful with the existing Kev lists that are out there like C sic Kev and E sic Kev, etcetera. And furthermore, again, you still want the defense in depth thing, you know, so they will break in. They're they're gonna get past that because they know the lists just like we know the lists. And so you're gonna, you're gonna end up having to update that list regularly.

But, but really you should, that's not how you solve that problem. You have this layered defense thing and that's how you solve that problem. The other thing, the maybe last thing about this that I wanted to mention is that asset management or asset discovery problem that you mentioned way back at the beginning, Jer, where, you know, I think you said 1/3 to 1/2 of all breaches come from, you know, come through some asset that people didn't really know that they had. I know that's not really the problem you guys are trying to solve per SE, but is that even a solvable problem?

I mean, there's a bunch of tools out there that sort of do that, but or do that but kind. Of kind of and and we have a lot of experience building them. So I know you do, Yeah, but it's still. A problem, obviously. Right. I mean, the, the the problem still very much exists. You have to scan everything. So you have to have a find a way at least Internet facing to find everything. We know 'cause we built them. But the state-of-the-art right now on an ASM that's public are tools to help you create an attack surface map.

It's now possible to create one with a certain level of quality, and that's fine for today. But what we're looking for is how do you automate the entirety of that process and have an attack surface map ready to go for everyone on the Internet. That's what we're building. That's the next evolution of that. And explain why Jared, because I think that may not be obvious. Because you need speed. Well, one is no one has time, the time or money to curate the stuff because new assets are coming and going, ports are open and going patching, you know, software updates are happening.

So it's hard to keep up. But you you mean on on the on the carrier side, Robert? Yep. So the the the carriers are they need solutions to scan hundreds of thousands of companies in sub one minute speeds the whole environment so. Try to build an inventory for 100,000 or 100. You know 180,000 and 200,000 companies. You can't.

Who's gonna do the work to create 100,000 attack surface maps? Yeah. Yes, you have a tool to automate it, but you need a person to do the work. You know, it's very similar to the evolution of white hat security where we had our threat Research Center as a symbiotic kind of kind of thing. And then AI is now taking over for that. That's where we're now in attack surface mapping. And then you get speed and scale and cost efficiency is what we're going for. OK, yeah. And, you know, at the beginning of this, Jeremiah, you explained that you, I told Robert you need like a copy of the Internet to, you know, figure this problem out and arsenic went and did it.

I think people might think that's like exaggeration, but I remember arsenic, you called me one day and we're like, hey, I'm going to put you on video. I'm going to show you this thing that I'm working on. You know, this is years and years ago. I don't even know how long ago. Like I was like, oh, that's cool. What are you doing exactly? Like I just like making a copy of the Internet. Like what part of it? No, like the public part. Like all of it. And and this confuses people because they're like, but that's like like Google or whatever. I'm like, no, no, no, no, because Google was that's they're the crawlable web that's very small compared to the actual size of the Internet.

Yeah, it's a. It's a real big data problem. So without like going down this entire rabbit hole that took you years and all that, how exactly did that work? Like what were the broad strokes of how you went and did that? Actually, if you don't mind, Robert, if you don't mind, most everybody probably listening to this podcast is familiar with Show Dan and the data set that Show Dan has. So they are at least familiar with someone that knows a lot about the Internet. So let's use that as a launching point, because that's where you start.

Yeah, yeah, the the major problem I didn't like. I've never. And by the way, nothing against the guys who run Showdown. I think what they do is amazing. So, you know, props to those guys actually. But my problem with it is I'm a web guy and the web requires you to send Http://headers if you want to get anything other than the virtual host, the default host, right? And so right there show Dan's a non starter for me. I, I just cannot use it for basically any web stuff or, or a huge chunk of the web stuff. Because if you think, you know, let's say there's millions of websites behind Cloudflare alone, they just disappear because you, you're not sending host headers at them. So you have to send host headers, which means you have to think about the Internet in ADNS centric way. And so right, right there show Dan's out the door. And then there's guys out there who do the crawling web. They they just want to crawl the web. You know, they, they have to build their own crawlers or use common crawl or whatever.

And that's, that's wonderful. But now you're missing all the routers and switches and IP telephones and all that stuff that isn't crawlable. And then there's a whole bunch of guys who have netflow versions of this and I'm like, well, that's great, except for there's a whole bunch of sites you never go to. So you're never going to see this thing that's just hanging out there forever. And so like everyone kind of had it sort of right. And then there's the manual guys. But the problem they have is they don't update fast enough. And this is one of those things that needs to update all the time. And so it's really a massive hybrid of tons and tons of ideas and concepts and tools.

And there's no who is data in it. Right. Yeah, exactly. So the the real problem became correlation getting them to to getting is 1 asset like another one. That's that's what took me all that time to build. It wasn't the downloading. That's that's actually kind of easy comparatively. Anyway, the hard part is knowing the two things belong together. And so that that's where I spent almost all my time. And. It's interesting. So it's like a kind of a social graph in some ways of like. Kind of.

Yeah, OK. And it's a 7 billion or so Internet facing assets, 7 billion. I was going to ask the number 7 billion. So how much does that grow every day? Do you have a any estimate? We, I don't know if we knew the growth rate for when we are doing bit discovery. We saw what we called the drift rate, how much does the attack surface map change month to month and it was about a change would be an asset, you know ADNS entry going up and down any moment in time that was a change, a poor opening closing could be a change or the software version change.

So all those contributes to shadow IT and we are noticing about a 1 to 2% drift monthly on the average attack surface map. Yeah, last time we ran bit discoveries around 4 billion. So what you know when we started the company, so 4.4 I think billion. So it's grown pretty substantially in the last, you know, eight years or something. That's crazy. What an what an absolutely wild project that was. I mean.

We learned. Way bigger. This one's way harder. Yeah, we learned more about the Internet than anybody ever. We saw things like imagine like you have a copy of the Internet, You could research the infrastructure of adversaries. You see websites that show natural disasters before they're announced to movies. Before they're announced, you see the craziest stuff. Yeah, we were actually thinking about building a stock prediction system on. Top just about to say like I can. Change some.

It just turns out that it's not positively correlated. It only correlates when it's a negative. So if they're going to lose money, it correlates well. You can't you. Can't predict the upswing? There's stuff you could do with that too. We could see strange stuff, like we could tell if somebody's expanding in a data center in a region or moving data centers in a region. We could see like all kinds of weird shifts. We didn't know what any of it meant, only what it was happening. That's so cool. But if I saw when you saw MailChimp leave Rackspace to go to Google Cloud, Rackspace missed Earnix, so maybe that's useful.

Very cool, yeah. Weren't you building this around the same time that HD was sort of building? God, the power grid in Austin must have been. Having. Some. Seizures around that time. I think he used the cloud. I had mine in the basement. It's it's it's, it's not a grow lab. It's a copy of the Internet that. Doesn't stop the. FBI from being interested. Believe it or not, arsenic is downloading Internet. I would like to know what's going on. Hello, we're familiar with your work. Sir, you have some questions.

You have a data center in your basement. Jeremiah's got like $1,000,000 in cars behind him so. I don't know. Yeah, All right, boys. Well, anything else that we forgot to cover? I could talk to you guys all day, but I know that you you guys actually have things to do like go fix the Internet and. And break it. Yeah, and break it, that's your job. We actually, you know, when we launched the company, one thing we were very humbled and appreciative of us is all the support we got from the industry. Everybody knew vulnerability management was a problem. Some of the smartest people on the planet have gone after this problem.

It's very, very hard. So when we raised our hand to take it on, we got a lot of well wishes and praise, like people want that this problem solved. And we just really appreciated the support. Every time we send an e-mail, make a call and go, hey, we need a little help here, everybody jumps in to help. And this is not something that we're going to solve just with our, with our team and our funding dollars. This is going to take an army of super smart people, purpose built. And, and it's, it's, it's really nice to know that, you know, we, Robert and I have been in this space 25 years and we've built reputations, you know, from our contributions where people are willing to help us. So we just want to say thanks to everybody. Yeah, I would add one thing.

So Jared did the security optimist thing. I don't know if you saw. Yeah, Kylie said that to me. It's great. Yeah. I think it is great, but one thing along the optimist route and back to the data for a second, we did a bunch of analysis and it looks like there are not that many adversarial groups on the Internet that are for profit. Now, there might be a lot of nation states out there, but for the for profits, there are not that many. We think that there's somewhere in the hundreds, I don't know, you know, somewhere between 1 and 900, I guess, You know, somewhere in there that is a small enough group we actually could win. We could win.

This, this is this is a weird thing. I've never felt this way in my entire life up until this company. At this time, I actually think it's possible we might actually win. Well, OK, so this is a totally different discussion, but it's something that I've been thinking about too. Like I like you guys. I'm an optimist and I've always thought and I, from discussions with you guys and other people, like a lot of threat and tell researchers, let alone law enforcement, Like they know generally who these bad guys are. They know who they're associated with. And in a lot of cases, they know physically where they are. But most of the time they can't really get to them. They might be in North Korea or Russia or other places that aren't our friends.

So winning has to have a different. It doesn't mean that we're going to get those guys and throw them in a courtroom. It means that we're doing what we can do on our side to stop them from achieving their goals, because that's the way. It's economics. If we make it too expensive for them to profit, they will do a different crime. Yeah, exactly. There's other ways to get yachts and fancy. Cars. There's a reason why we don't have bank robbery anymore. Right, yes, yeah, which is I was actually going to text you about this arsenic.

I I'm positive you saw this. There was a thing that I think I saw it yesterday about this data center that just disappeared off the Internet, I think early, late of. The holidays, I've not seen that. Oh, also the B, GB GP1, you're talking about the BGP routes from Venezuela where they changed the routes, is that what you're talking? About No, it's something else. It turned out like a guy noticed it because he was hired. He was doing some external monitoring of this data Center for a customer or whatever and the whole thing just went offline. This is a rabbit hole, but turns out it was a robbery crew that did a power search to fry the all the monitoring equipment and alarms in this data center because they were going after precious metals depository that was in a underground bunker. Like some real heist shit.

But you're right. Yeah, we don't have bank robberies because it's just way too hard. Go do cyber crimes on the Internet, it's a lot easier. And you know, we get a lot of pessimism in our space, But Robert, Robert and I, from what we've learned, we think we can make this way more expensive that make the make the crime go way, way down. We really think we can. It's not going to be easy, but we we have line of sight now. Is there, I mean this is a could be a much longer discussion. Is there one or two things that you think are the real blockers for that like? Compliance.

Well, I'm sorry. Compliance. It's, it's I, I, I, you know, everybody wants to believe it. Maybe it was true at one time that it's, it sets the floor. It doesn't. It's wasting resources. The compliance models were designed in a world devoid of, devoid of actuarial data. We've moved on. We, we need to redo compliance for outcomes, not controls. The adversary shifts. The compliance standards change every three years. The adversaries move every 18 months. This cannot possibly work, so we have to be able to shift the dollar resources and focus around and anything that prevents that is a problem, compliance being one. That's a great point.

Yeah, that's not a way that I thought about it, but it makes a ton of sense. Yeah, almost. It sets the ceiling instead of the floor because I I run out of resources because I'm stuck dealing with so much compliance instead of focusing on the areas I really need to focus on. So we're, we're, we're going to start conversations, you know, we'll put the call out to start having conversations with those that set the standards and go like, and we know that they want to do a good job. We know they want to do the right thing and make these things work. I mean, it's why they're there. They just never had access to the right data before. So we want to work with them.

So at some point we'll make the call and go like, let's reimagine this stuff. Yeah. I mean, a lot of those standards and, you know, especially government regulations were written decades ago, before, you know, modern attacks existed. They were, yes. Yeah. So there will be there will be some resistance to it. The resistance will come from cyber security, unfortunately, because people make a lot of money on doing boilerplate reports. We're going to have to fight it. It is what it is. But that fortunately our the answers for what works are there if they're there. Yeah.

All right. Well, that's probably a great place to end it. Guys. Thank you so much for so much time. I really appreciate it. This is great. So always great to see you, Dennis. Yeah, you too guys. Take care. Hopefully I'll see you in person soon.