The Surprising Connection Between Customer Feedback and High-Performance Metrics in Product Management

OK, um, it's something I should watch out for, right?

OK. So just to clarify, um, Uh, that we have a high performance product or, uh, is that in some way tied to the goal that we would set for the team or what's that? Is that in some ways tied to the goal that we would set in the quarter?

Right, right, um. Would you like an example? I can give an example if you like. Um, So, um, I'd like to tell you about a time when um Um, basically Um, again, coming back to FDR, which is Freedom Debt relief, um, I was imagining the custom acquisition funnel, and on the tail end of that, uh, on the, at the end of the funnel, uh, there's a 300 to 400 person sales team that would call those customers and try and see if they were matched with the program and enroll them into the program. So my main KPIs. Uh, were the, the funnel conversion rate and the close rate of the sales team, which I co-owned with the, with the head of sales. Um So Um, what I had, um, A few months into my role here, I had observed that there were Um, we were getting some negative reviews from customers on the BBB site. And um I'd also like heard it anecdotally, but then someone um on my team raised that, you know, we're actually getting these negative reviews. So I dug deeper, actually hopped on a couple of calls with those customers, but obviously with, with consent and uh try to understand like what they were trying to get from the program. And, basically, uh, what that led me to is that these customers felt that they were being harassed by phone calls, um, because we had like, we were calling them 3 times on day 1, twice on day 2, and so on and so forth. Um, that led me to, um, Basically building a product for SMS and setting goals and KPIs for that, which basically impacted the um the close rate for that was that was a KPI that I was being measured for and how and how sales was also being measured. So does that make sense and would you like me to dive into that further?

Yeah, Yeah, so, uh, just, uh, broadly speaking, my framework for success is, um, you know, what is important to the customer. Um, And um So, first of all, it, it comes back to, you know, how do we really define a KPI or key performance indicator that is uh gonna help the customer the most. So, um, that is step number one, really like. Setting up a goal that would help the customer the most and uh once that is done, uh, establishing a baseline of what if the, the currency of the product, if we have numbers on the currency of the product, what that baseline is, and what do we really expect to be reasonable for us to hit in the next quarter or half and set goals accordingly. And the way I do that with my teams uh is by Doing these, uh, really, uh, by looking at historical, like what knowledge we have, like in terms of past experiments we run, um, and also running some really quick experiments to validate or invalidate hypotheses, uh, to get really quick signal in terms of, you know, do we really think that this idea can have legs and so that we can explore it in the next 3 to 6 months. So that's my framework in general and happy to dive into an example with the one I, I gave you, but.

Yeah, so, um, so I'd like to come back to the example I was, I was, I had started on, which is, um, introducing SMS as a contact mechanism. The KPI that we're already being measured on, by the way, was close rate in this case, um, like how many of the customers or enrollment rate for customers if that makes sense. But

That was in place, uh, but, but, um. That was not, uh, with this new project, this new innovation that we added of SMS, uh, as a new contact mechanism. Um, we were able to boost that by, by 20%, which was not on anyone's radar at that time, um, and, but how do we really know that 20% is a real reasonable number? That's the exploration that I did that I'm happy to dive deeper into. Sure, keep going. OK, so, um, Would you like me to dive deeper into like the entire process or just focus on the number just,

Yeah, so we went from 6% close rate to 7.2%. OK,

Yeah, so, um, Yeah, I, I had a um Um, conversation with the head of sales because I had co-owned this, you know, the sales team was, you know, obviously a key stakeholder in this whole process. Um, they had concerns about using SMS because, you know, it would come up as spammy, you know, in the fintech space, it would, you know, there's regulation and all those things, um, and they were concerned about both the sales team abusing this feature and also not adopting it. So it was, um, it was interesting, um. So to really establish, to first of all, alleviate those concerns and also to establish, you know, whether this idea is gonna work, whether this idea has legs, um, I, um, convinced them to start a small pilot, um, and obviously with other stakeholders involved as well, so everyone was in the loop. Um, a small pilot involving 5 sales, 5 salespeople where they would, uh, send SMSs in addition to email and phone, which we already had, uh, so the baseline control group of the rest of the 300 to 400 salespeople was email and phone, but these 5 salespeople had in addition access to SMS, um, with some, some, um, preset scripts that I worked on, help with, with the head of sales, um, to basically help see if we can boost the, the, the. Um, enrollment rate for the clients that those sales people get in contact with and over the next. Uh, 3 weeks to 4 weeks, we were able to get a 25% lift in, in that, uh, for the, that, that, uh, pilot group. Um, and when we extrapolate that and we roll it out to the rest of the group, of the rest of the sales team, it ended up being, um, about a 20% lift overall, and so from 6% to 7.2%. Um. So that's how we validate or or invalidated whether this idea had legs and whether, you know, how much of a goal should be set for the close rate, you know, because it involved an investment of like 9 months to, to 12 months to really build this SMS feature out, so it was a pretty big investment of the team, um, and, uh, but yeah, this is how we quickly got, got a sense of whether, you know, this idea has legs or not.