Why This Sr Product Manager Embraces Team Input Over Data in Critical Decisions

Honestly, I don't know that I have a good example of a technical decision that I had to make, um, because I typically try to work with my team to let them make technical decisions. Um, I'll make decisions around priorities or around, um, you know, goals or non-goals, things that are not important to us, relative costs. I'll even be the one to help build the framework for how we're going to make a decision. Um, you know, weighing this priority against that priority, uh, but for technical decisions, I try not to be the, the, the person who is making the decision. Uh, so I may not have a good example for you there.

I typically avoid making technical decisions, right? So, um, how do we want to architect this, ah, what language or platform or tool do we want to use? Um, as far as design decisions or, or like broader decisions, I typically will make those with the team. Um, I don't want to tell somebody else how to do their job, and, you know, typically I don't like it when they tell me how to do my job.

So, I think, um, Uh, I actually like to to call back to to something that Bezos talks about for this, uh, which is a type 1 and type 2 decision. Um, the short version being that like, and I might have these backwards, but, but it doesn't really matter, like one of them, type one, let's call it, is a decision where, uh, if you're wrong, you go back and you change it and like the world doesn't end. Um, type 2 is a decision that like the future of the business relies on it and Uh, it may be really difficult to go back and change later. Um, and for if a decision is going to be a type one decision, uh, I try not to stress about it. You get the data that you have, you make a decision that seems reasonable, um, and you go with it, uh, cause you can always change it later. Um, so within that broader framework, Um, I'm trying to think of an example like at Tenable, ah, we were doing some work on building out new models, uh, to talk about risk for our customers. Um, and we had a decision to make about like Uh, without getting too deep into the weeds, we had this like overall risk number and risk model, and we had different things that could contribute to it. So for example, if you're not scanning your assets, uh, it's the equivalent of like walking into a dark room without turning on the lights, without using a flashlight. Um, you don't know what the risk is there. Uh, and so we had kind of a school of thought of like not scanning should lead to a lower or a higher risk score. There's more risk in not understanding your risk. The alternative was, we can use that to build a confidence score. You have very low risk, but we have very low confidence in that, um. At the end of the day, uh, I felt one way. I felt that we should, uh, integrate the two scores because it is risky not to look at, at, you know, not to scan, not to evaluate your assets and not to understand your risk deeply. And, and we were in the business of selling a risk evaluation to our customers. Um, they, they bought a scanning tool, but what they were really looking for was a, an evaluation of risk, of cyber risk. Um, So, there was some disagreement, uh, internally. We didn't have like strong data. Um, we talked to a few customers and, and kind of didn't get a strong signal, um, and I made the decision to put it into, uh, the product. Um, there were probably a dozen similar decisions where we, we got into a disagreement and I let somebody else win because it didn't seem that important to me. Uh, but this felt like one that was gonna have, um, ramifications for a while, uh, in the product, and it felt like the right thing to do, coming back to our values of like, what is our goal. Our goal is to help users understand their risk, and the more numbers that we give them, um, the worse that's gonna be. So, bringing that all together to understand your cyber risk and your exposure, um, conveniently named the cyber exposure score, uh. I, I felt like was the, the right call. Um, it didn't have any real technical implications, uh, as far as like how to build things so much as like where to combine the numbers in our pipelines. So that might be why I didn't think of that right away.

Yeah. Um, we looked at a couple, uh, honestly, it, it was, it was pretty binary actually. Um, but we looked at not including the confidence score, right? Our, our understanding of, of how well you're scanning or understanding your risk. We looked at in incorporating that or providing that confidence score as a separate metric that we would need to define and, and display and like report on and report over time and all that. Um. And then we looked at integrating it into a portion of your overall exposure score or your, or your overall risk score that we provided as a number. Um. Uh, as you said, like I, I kind of leaned and pushed us towards that, that last one, but we, those were the options that we seem to have in front of us. We didn't, uh, kind of brainstorming any alternatives outside of that.

Um, So, I think I, I, I want to push back on your use of the word outcome. Um, it, it's the correct use of the word outcome. But when I, when I think of outcomes, right? I think of like outcomes that users would pay for, right? That's, that's valuable outcomes. So like we made a decision in the product, but I think the way to measure success would be like, was it impactful to our users. Um, either in terms of revenue or like in terms of them being able to reduce their cyber risk, um.

I think it was the likelihood of leading to uh users interpreting and understanding the data. Right, when you're, when you're dealing with data products, um, You can provide all the data in the world, uh, and it is worthless if people don't understand it and interpret it and like take action based on it. So I was basing my opinion on or or my stance on on my opinion that integrating them would be the thing that would get people to pay attention, um, and that they wouldn't pay attention and they wouldn't, you know, change anything uh if we didn't do it that way.

I'm not, um, we can't go back and run the experiment. Uh, we can't, you know, test it one way or the other anymore. Um, I am It, it, it's a lot of things that combined, but it was our experience with the dashboard and seeing that like users are overwhelmed by too many numbers and not understanding which ones are good and which ones are bad and which should be high and which should be low, and what does this mean and how do I take action. Um, it was kind of a broader experience in UX uh over the years and knowing that like, The fewer things I can put in front of a user, uh, the more they're going to focus on it. Um, it was the combination of our, our marketing efforts and sales efforts that were all focused around the cyber exposure score. So if we kept telling you like this is the score you need to worry about and you need to, to think about, then integrating the important pieces into that, um, at the CISO level or at the executive level, they're going to see one score, uh, and if it goes down, They're gonna ask why did it go down, and then somebody else is gonna need to do that investigation and understand how to fix it. Um, surfacing all the detail at the top felt like the wrong way to get a CISO or an executive to, um, you know, to, to, to like take action, uh, and our, our outcome that we cared about was CISO is taking action, um, based on, you know, not being as secure as they'd like to be.