How a Radical Reimagining of Cybersecurity Metrics Turned Pushback into Customer Success

Sure. Um, So not like a technically complex, but, but, uh, broader than that, right? Yeah, OK, now we got power. Um, I think for predictive prioritization at. Um We came in, I came into a product that was, uh, kind of started as a research project and it had been, um, in process for some time, uh, with the remit of like get it out the door, ship it because this is gonna be a key part of our cyber exposure strategy and the cornerstone for the company and blah blah, blah. It's gonna be a very important thing. Um, My first approach was to say we are trying to ship a new metric for security engineers to base their prioritization on. Um, we're gonna want to ship this as as a beta, uh, or an early release or some name to say like, hey, work this into your workflow, find a way. Um, start thinking about it. Uh, and so then 6 months later, we'll be able to show a bunch of data that says, ah, looking back on this stuff we've been publishing, if you had followed our advice, like here would have been the outcomes and here's some examples of, of users or customers who like had these great outcomes. Um, I got a lot of pushback on that, uh, because internally, like marketing and sales had selected a launch date and started telling people about it, uh, and they needed to launch. Uh, so my, my fallback was let's do a bunch of user interviews and try to understand what we can so we can make changes before launch if need be. Uh, we made those changes. Um, that was great. I thought that was a win, uh, in terms of kind of delivering something that was useful to our customers. What we discovered after launching was that Every single one of our customers had governance and processes and SLAs built into their cybersecurity stance. So they had processes or governance that said, You know, for critical vulnerabilities as defined by XYZ, as defined by the industry standard, we must address within 30 days. So if we came to them and said, actually, half of those you don't need to address because they're not that important, and these other ones over here, you do need to address, they're more important. They would, they would say this is super helpful, they would say thank you, and then they would go do the thing that was built into their bonus structure. Um, it was, uh, something we should have understood before shipping, um. Maybe it was something we missed, but, but like that was, that was a big, uh, whoops for us, I think. We didn't understand the dynamics in the market and, and that a little bit better dealing with. Yeah, just explain that a little bit better. So, um, Cybersecurity team at Visa or JPMorgan or any company, um, they get vulnerabilities that that are detected or published. Um, they have all built processes, some built around like PCI compliance or some other thing where they have an internal SLA that says for every critical vulnerability we'll address it within 15 days or within, and then every high vulnerability we'll address within 30 days or 60 days. Our pitch and our value prop was, you don't have the ability to address all of these things. You're, you're dealing with too many. Um, let us prioritize them for you and give you a better list to work off of. That was the product. Um, What we didn't understand was that somebody in it's a director or an executive had tied their bonus, um, and their, their compensation to, I am going to address this SLA with like 80% accuracy or or like like I'm going to address 80% of these. My team will do this. That is the thing that my bonus is tied to. So when we came in with this new product meant to make their lives easier by giving them the same security with less work. Nobody used it. They were all interested, um, but like somebody in the chain had a bonus and like actual compensation tied to this old way of like this previous governance that they had built using the tools that they had at the time. Um, and so that customer interaction was something that we hadn't understood prior to shipping. Um, I think like the longer term solution was to start to build trust and work at the executive level to start to get that governance realigned around our standard and around our metric and our product that we were shipping, to say, hey, you need to address all of the critical vulnerabilities within 15 days. And all the high vulnerabilities within whatever.

I think we would have reset expectations internally, right? So, in our external relationships, we would have done more focus at the executive level, more focus at the, you know, hey, this is not just a new product that you drop into your workflow. This requires changing your workflow. Um, and we hadn't done any of that, right? Internally, we could have set expectations differently that like users are gonna start to ingest and use this, but it's not going to affect their processes like like their actual outcomes for six months or a year until they change some of their internal process and governance. Um. Instead, we had like a a like a a sizzle in the pan where everything was amazing and it's all everybody was talking about for the first like 2 or 3 months. We shipped our fast follow, we started getting it into more of our products, um, and then it kind of fizzled out and people weren't talking about it anymore because uh they were getting this frustration of people wanting to use it but not being able to and then like they moved on to the next shiny thing. Um, so we would have had to, I'd say change our, our external communication strategy, uh, around who we were talking to and what story we were telling, as well as change our internal expectations and timeline for how long we thought it would be before users started to see value.

I would say I, I'll, I'll, I'll split it into two, like externally, our biggest obstacle was existing processes and, and kind of organizations being slow to move. Um I, I think that was something we didn't anticipate super well. Um, internally, uh, I think the biggest obstacle was an organization that wasn't focused on user value. Um, When I was suggesting we should do a beta, we should understand like how people are going to use this before we tell them to rely on it for production. Um, I was perhaps being too conservative, um, which is, which is a fair critique, uh, but We didn't have a world where, uh, at where marketing and sales and engineering leadership were all focused on, uh, value to our customers. Ah, many were focused on, can I sell it? Is it exciting? Um, are we reaching the launch date that we published six months ago that we told users that it would hit? Um, Those things are important, but I think that a, a, you know, stronger culture would have been focused on value to customers with the kind of assumption that the other stuff follows.