How a Simple Fitness Test Revolutionized Customer Retention for a Product Manager at Trainer Road

Uh, this is probably my favorite story, um, or my favorite part of working at train road, and the, the thing that I feel proud of, um, So, when I was on, or when I still am on the Team Clif Bar cycling team, um, we had the problem of really trying to figure out a way to assess people's current fitness levels and make sure that we could artificially rank them, um, to, to see who we could bring to a race, to see who's going to kind of perform at, with a, a low cost of entry, um, from them and the, the The kind of the metric that we could use is easily easily digestible and and transferable between people. So when I came to Trainer Road, um, We had, uh, the hardest problem we had at Trainer Road was the churn was pretty monumental for the first time customers. Um, it was like 30 or 40%, um, just the, even though the our NPS was really, really high, our retention and our first month. It was just awful. Um, and so if we could get people into the ecosystem correctly, uh, the values there, but getting them to not churn within the first couple of days or first week or first couple of rides was nearly impossible. Um, And so, one of the things that I did, as soon as I had some access to some data, it was, it was interesting because I, I didn't put these two together until I started digging in a little bit more and, and seeing what the problem was. Um, The way our main pathway into the into the application was an 8 or 20 minute test, um, which has its own set of, uh, it has its own set of difficulties, yes. So it has its own set of difficulties because hypothetically, you actually need to know a value that you're shooting for before you start participating, um, or else the test is, is skewed, um, and the value you're getting at the end isn't actually representative of what you are capable of doing or should be able to do. Um, and so. Um, what I had done with my team is we took, um, I've done some VO2 maxing testing with the step test on the treadmill, um, and it seemed like a pretty easy, um, Thing to recreate, especially with the invention of smart trainers, kind of as soon as we had an ergometer, we could put resistance and scale resistance accordingly. Um, I guess we, we could have done this with, uh, with some old tech, and it would have been slightly, slightly more scaled, but, um, as soon as we had a smart trainer, you could just effectively. Put people through the cardiac test of increasing either the percent grade of the treadmill or increasing the speed or something like that. And so, when, what I've done with the team is created an artificial starting point and an ending, and you go until time to exhaustion. So there's no necessary, um, There's no, no starting point, you know, you don't need to know anything other than start pedaling and stop pedaling when you can't anymore. Um. And so One of the last problems that this kind of took out is you need pretty much zero education to start. You effectively, the onboarding piece is really, really simple. Um, you can, the edges of the curve for the results of this test, even if you put in a really strange value, really high or really low, you still get very close to what your, your end goal would be. Um, and so that was kind of another big, uh, lever that was missing from Trainer Road is the onboarding education piece because we effectively, we needed you to know what you needed to do before we gave you the answer. Um, and so what we, what we ended up doing is, uh, we created the test, uh, ran it internally, um, and then we We created this internal beta group, um, and started having, started running math. Uh, I mean, effectively, we're trying all these different kind of formulas to try to figure out something. I'd only done it with 20 people, and so doing it with internally with 50 people, and then we, we started doing it with 500, um, and we had support agents doing the math on the fly and writing sport tickets back to them because we actually didn't build a system for this because we weren't sure how it was gonna work yet. Um, so it was, you had to take a, you had to take a Do a ride and then um send in a sport ticket, or, and then we automated it a little bit. And as soon as you've taken a test, we would send you the sport ticket, blah, blah, blah. So after 5000 tests, we took the data and started looking at the values that we're seeing. Um, and over the next, I think we did it for 3 or 4 months after that, um, We tweaked the values just slightly and put in some edge case catching, um, because who knows what can happen. Um, and we really don't want a bad experience. And so we actually, one of the overarching problems with train and road is it's notoriously too hard. Um, and so we did just slightly tweak our test, um, so that you're, you fall slightly lower on the spectrum to set you up for success better in the whole ecosystem. Um, and so that was a really Interesting, um, caveat that increased at all, all sorts of other things that having that first entry point actually made the rest of the system better. Um, and then the last problem is that, uh, I had just created this thing, and nobody, nobody would believe that it's better. Um, and so we had to come up with some sort of, we, we, we did the science-backed, I think we had 10,000 people do the test, and we plotted it, we had a bunch of people do the The Old tests and the new tests and then we, we followed up on some metrics of, um, success and failure on a sequential workouts afterwards. So we had a, we had some numbers that said that the that the values we were getting were correct. They were, um, creating a great way to Uh, live in the ecosystem and have success later and throughout for a while. Um, it's less stressful, you don't need to know as much, but we needed people to buy in. So we did these, we had the idea for, um, the live ramp test. Um, and so part of the reason that this also existed is because I used to make my team do it together. Um, because then you have some sort of, uh, you need, you need competition, at least we did, um, and so this was something that people liked the characters that worked at Trainer Road, and we could, they could have a, they could have an underlying belief on who's gonna win or who's gonna lose, and then that showed them that, uh, It is valuable and we're all actually using it. Um, and so, the long and the short of it is, it's the most written workout on train road, um, and it's copied by every other competitor in the space, uh, at the end, which is, um, I guess that's the goal, uh, at some point, um, but it's, it's kind of the one thing that, uh, It, it's a, it, uh, from a, from a number standpoint and metric standpoint, it, it increased, it dropped churn by 50%. Um, it ended up kind of changing the trajectory of our ARR by a pretty mon, uh, I mean, it's not a hockey stick, they don't exist, but it was a good, it was a good change in trajectory, and it changed kind of the way to double our ARR in like 2.5 years instead of 3.5 years, and things like that. That which, which definitely changed the way um we brought people in and then it's a new, I guess there were some engineering challenges and things like that that we had to do too at the same time, but it was, it was the first whole like kind of piece of the holistic system that I got to take out and put in something new and change, change it for the better.

Yeah, so, um, initially we started with, uh, So the, I guess the easiest way to do it is, um, there's a few different buckets to look at, and then we can combine the buckets. So the first bucket is rides in the 1st 30 days, rides in the 1st 2 weeks, rides in the first week, and first ride. Um, and so each of those kind of starts a different, um, Character journey, or, or customer journey. Um, and so depending on what your first ride is, you're gonna have a much different experience if you pulled something off the wall, or if you did a test or something like that, or, um, you have a coach. Uh, and so the first ride actually re-separates everybody out into a new thing, because technically, um, we can only control so much after our first experience. Um, and so, What we ended up doing is pulling, separating out all the buckets out from the first ride. So if the first ride is something like that starts with an A, and so they are actually just looking at the workout list and picking something that starts with an A, that's a different type of customer than someone who's going to do a FTP test versus someone who's going to do a ramp test, versus someone who's going to get a coached workout, or like a custom workout or a free ride. Um. And then, uh, also, we had a different um bucket for anybody who was in a first workout of a training plan. So we have, you know, dozens of training plans. And so we had separate, um, workout ideas for anybody who's on a training plan and, and how that works and what that experience is. So as soon as we had all those buckets, we started picking and choosing the biggest bucket and kind of the highest impact we could have. So one of the laughable ones is we reorganized. For the people who are pulling a workouts, we actually renamed some workouts so that the first few workouts you see in the A's are easy. Um, that was an amazing, that was like the smallest, one of the smallest tweaks I've ever done at work where I renamed some things and it changed.

OK, just change it, just change the name, pick a, pick easier workout. So I cloned some of our easy workouts and renamed them, like a high success. Anybody who didn't churn, if they did this workout first, I renamed those, cloned them, made them just slightly differently, and put those in the 1st 5 workouts. Boom, those guys are taken care of. Um, the training plan people were actually really solid if their FTP was correct. And so what we ended up doing for them, that's the biggest chunk, so the biggest bucket we could do. Um, and what we

up

is

uh, training plan, uh, people on a training plan, so it's like 40 or 50% of our athletes, um, and what we ended up doing was anybody who was on a, uh, who had taken an FTP test, um, we actually Uh, depending on it, if it fell within a value, um, we did some automated support tickets to make sure that they felt like they were being seen and heard and part valued as part of our company, or as, as a customer of the company. And then anybody who wasn't taking an FTP test would, uh, We started prompting them, um. With we built an onboarding, uh, that, that effectively made them either create an create an FTP, gave them some options to bring in an FTP because as soon as that was the, the main fulcrum that as soon as they had that value, then the trajectory started and we could, we could, um, kind of increase their chances of success. So that was the second bucket and then for, um. The last one is, uh, the last big one was the, the fact that, um, a lot of people just did workouts. Willy-nilly, kind of. Um, they were like pelotoners or kind of the, the enthusiasts, I guess is the way to do it. And they, they fill their time with something else. And so what we ended up doing with them is, I guess I created the, I revamped the Uh, some training plans that are, they're for time crunch cyclists is what we call them, but we, we pulled out all of the harder workouts and created kind of a more sustainable, regular training plan. So even if you're doing something once or twice a week, they're all a little easier, and actually you can kind of do that as supplementary exercise to whatever else you're doing, and that actually increased the, the rise per week for that, that bucket. Pretty large amount just because instead of doing things, they were slightly more fun. They usually had some sprints, they were kind of more fast paced. They always had workout techs. We kind of built some features, and even though it's not a lot of people, it's like 10%, um, getting the active rides up for them was actually a big, uh, a big win, because we didn't do anything. We just no extra work, we just replaced kind of what they, what they needed to do. So that was kind of the three main, that's probably 75%, and then Everything else was a lot, a lot smaller, um. And we ended up building other things for them in the long run.

Let's do it. I think I, I think I talk too much, um.

Yes. I know I talk too much. Literally the