Unlocking Revenue: The Surprising Insights Behind Why 90% of Pilot Programs Fail

Sure. Um, the Let me give you an example from sort of two startups ago where When I, when I walked into the role, um, and I was running product marketing at the time. And I started looking at one specific area which was, you know, how much revenue will be making out of the pilot's approach. So we had a pilot's approach, OK, we would call it conference room pilots and we would essentially offer the technology, uh, think of it as an AI assistant on the phone for sales people, uh, we would offer this technology free for a month to, um, our customers. And then at the end of it, the expectation was that, you know, based on the results we get out of uh these, these pilots, we would be able to make a case to the decision maker to then, you know, uh purchase the product. When I first looked at uh the data in Salesforce, um, you know, and I had one more team member only at the at the time, um. We were shocked at the numbers because it basically felt like out of every 50 or so we did, 45 would do the pilot and walk away. So,

They're walking away. So, uh, we would spend all this effort with the sales engineers, uh, you know, the product side, the marketing side, stand these sandboxes up, um, and, uh, you know, and then these people, in fact, the, the worst part about it, Brandon, was that, so, you know, I'm I'm now starting to go through the, the, the solutions, right, uh, and we, we went through this, so I said, well, maybe they're not happy with what's going on in the pilots. You know, maybe the pilot isn't actually delivering enough value.

There wasn't, and a huge part of it was because somewhere in the 6 months prior to me joining, we had cracked General Electric as an anchor customer, and you know, at, as a startup which is hungry, hungry, hungry, and then you suddenly get this one big fat customer

yeah,

yeah,

yeah, and I'm in the first like 3 months of my job and so I'm like, wait, why aren't we talking enough about these. Fail with these unconverted pilots essentially and I remember the CRO saying, listen, you know, you wanna focus on that problem, you should, but like we got a whole bunch of other things. A, we gotta keep G happy. 2, we gotta find the next G, blah blah blah. So anyways, I said, OK, I gotta like look at this because this, this goes back to even what I'm trying to do, which is position the company, reposition the company, kind of figure out where we wanna be with our product marketing. So first I looked at uh the survey data. We had a series of questions which were all around um how much time did you save using this product, you know, are your sales people gonna be more productive? Do you feel like

No, we, these were essentially questions that were given to, uh, the sales rep end user. So sales users and sales managers were, um, uh, sales reps and sales managers were the end users of the product, OK, and we would typically identify, you know, small batches of 10 to 20 people within a company within a certain region or a certain city and offer it to them. So imagine Honeywell having 20 of their reps. In one particular area sort of using the product, our sales rep would actually show up, um, you know, we would show up in some cases do like a, a 2 hour training, and then we would, you know, get them to download the product on their phones and then once they downloaded the product, these questions that I just described to you, we would ask these questions at the end of week one. So it's OK, now that you've used the product for a week, um, you know, how is it helping you? How much time is it saving for you? Is it making you productive, you know, do you feel like it's taking away some of your CRM burden, things of that sort, all extremely tactical. The only pre-qualification, to be honest, coming in was that we had 4 reps at the time, and if the rep was like, hey, I spoke to Brandon, and they would love to do a pilot, and I think there's something in here. They're like, OK, let's do a pilot. So, um, I realized that the, the, the first thing that shocked me was that the, the rating scale, we had a, you know, 5, you know, 7 point rating scale, most of the numbers are 5, 5.5, 6. I was like, wait, these people are actually getting value out of the product. They're saying that they're generally happy with it, but, um, they're walking away from it. So then. I went back and I interviewed some of these users and I tried to understand like what was going on there. In parallel, I started interviewing the account uh execs, right, the ones who were getting these pilots in and started to understand what was going on kind of, um, after the, after the pilot resort and I uncovered a whole bunch of things, right? One thing I uncovered was that, you know, the end users, um, they, they like the time savings they were getting out of it. But, but the number of times they're actually using the product in that first week was very limited. They were like using it for 3 to 4 weeks, so it immediately became clear to me in just this initial interview with these reps that they needed to use the product a little longer, that, you know, they, they had to be. Almost triggered to use the product and in our trainings we weren't training them how to use the product the right way. We would come in, show the app, and our sales engineer would kind of walk away. There wasn't a what I would say kind of this like, you know, very deep engagement built in where you're, you know, or engagement built in during the pilot. We would do the training. They would start the pilot and we would walk out. Um, on the, so, so I'm observing a set of issues, OK, in terms of the experience of the pilot and what could be done to increase the engagement and, you know, get the users to live in the product longer and so on. The other thing I'm, uh, I note, I started learning about as I spoke to the account execs was that, you know, what they would do is Brandon, they would package the results of these surveys, put a slide together and send it to, uh, the business decision maker in these companies like, hey, we just ran this pilot, um, you know, when can we have a meeting. And there was a whole series of missed opportunities of actually making a business value or an ROI case on how much value this product would actually drive if not 15 users, but let's say 1500 or 15,000 users actually use the product. So then with my team as I started collecting this data, I said, OK, now let's start figuring out what we wanna change with the next round of pilots that we do. Um, some of the things that we attempted were I changed the questions around, I changed the duration of the pilot, I changed how we could connect the outcomes of these questions to an actual ROI model which could literally show you sort of the earnings per share impact that you would have if you used it at this scale, you know, across your country, you know, across the country, across all your reps, or let's say across, you know, internationally, and the key thing was. That package that got put together that would go back to the business executives making that business case, I started working on the 1st 3 myself. I remember the breakthrough was Cisco. We were doing a pilot with Cisco at the time and I said, guys, this is the pilot where I would love to test out the new questions. I would love to test out all these new things that I'm experimenting with right now, you know, with you all, um, we won the Cisco, the, the Cisco pilot. Which was, you know, about a month, actually converted to about a 70K deal after and so once we had one win, then I said, OK, now let's start doing this for the pilot going forward. I would say, you know, kind of on numbers by the end of it. And remember there are other things going on in parallel too, right, because in parallel I'm doing the rest of my product marketing job which is changing the positioning of the company, changing the way we even market the pilots, make it almost like an exclusive pilot, and so on. We were able to get to 80% pilot conversions within the next 80%. 80%, uh, you know, we just, we said no to pilots as well, right? So we're doing better at pre-qualification. So every rep didn't just get a pilot because they wanted to do it. They had to satisfy a certain set of criteria. They had to actually build a business case that, you know, I'm not just doing this pilot for free. I'm doing this pilot because there's a 10x bigger opportunity down the line. That's why I can put some sales engineering. I,

Absolutely we, we did it and we, we did that by changing the input variables as well, um, and, uh, you know, just getting better quality customers, you know, prospects kind of into the pilot, um, we started using the pilots that converted to our most successful accounts I think Cisco and you know, uh, you know, some of the other divisions at GE we, uh, you know, ended up, uh, taking this approach. And what this did was that the next time we started a pilot we could show the journey that one of these larger organizations took with us to like get to 5000 users or such and that actually helped us, you know, change the pilot, you know, the last thing I'll kind of leave you with on this pilot experience is it's a, it's a, it's a lot of the small things as well that matter, right? We actually started shipping. iPads and mobile phones, you know, we had a, we purchased 20 of them and we start shipping them to the end users so that they could literally pull it out and start using it and they didn't even have to go through the friction of using their personal phones and so on and so forth. So over time 7 to 10 different iterations just kept chipping away at a lot of these small things and once it got baked as the process, I said, guys, you and the sales engineering team, you guys can run with this right now. I don't need to be in your way, um, and uh. You know, that was, that was a process that actually helped me a lot in the future as well because, uh, you know, it kind of taught me about what kind of um things you need to do to make a pilot process successful so that it doesn't feel like you just have a bunch of vanity, um, you know, pilot customers that aren't paying you a single dollar.

I was uh shooting for 50%, to be honest, right? I said, OK, I, um, if I, in the next 6 months. Because I knew the customers we were bringing in, I knew that these pros, uh, you know, this, uh, entry, this customers coming in into the entry of the pilot, they weren't bad customers, you know, they weren't like podunk no name companies from like somewhere in the world. These were all US companies that are all on Salesforce that you knew that they had some sort of challenges which is why they wanted to experience that. So I said, you know what, I am just going to start with seeing if we can get one out of two of these pilots, right, but. What I didn't factor in was that You know, people were also perceiving us a certain way during that initial period when I walked in, right? So we were a mobile sales productivity suite, um, bunch of words there, right, but basically people would use our pilots, uh, and they, they would, uh, go through the pilot and then they would say, you know, I love your product, but as much as Salesforce One sucks, it's actually not that bad, it's free, you know, it's like a free mobile product, so. So during this time we also went through a major repositioning of the company and so the, you know, the new cohorts that got exposed to this like say 2 months down the line, they were not seeing us as a mobile sales productivity seat. We were giving them an experience saying, well, we are, we are your AI assistant. Think of us as Alexa for your sales reps or Cortana for your sales reps. And so I think that also had a different effect because we were giving them something which was more desirable. Versus like, uh, OK, here's another mobile app that I need to download on my phone. Um, we also threw in an Alexa experience where you could talk to your Alexa and update your Salesforce as part of the pilot so I introduced a new use case into it and I think all those things also probably give me a better result than I anticipated originally.

You know, we had before it became popular and before it, like, Became momentum we did it in the wild west days running in when you or I could walk into an Alexa and ask for Salesforce data and it wouldn't know the difference between our voices, um, but a lot of what we did then with that new positioning is ultimately what ended up helping us get money from the Alexa fund, um, because we were showing those use cases in our pilots and many of those pilots became paying customers as well.