It’s not so complicated: Growth experiments for startups

Chase Mohseni: Hi. I’m Chase Mohseni, I’m the Head of Marketing & Growth at an AI company called Pencil. And I deal with how we’re going to actually get customers into the platform, and how we actually get them to upsell when they’re in the platform. So, essentially, fueling the business from the front side.

Spyros Tsoukalas: Chase, welcome to the GrowthMentor Podcast. I’m excited to have you here. And I’m even more excited to get to that stage with Chase. So please tell us something we didn’t know about growth experiments.

Chase Mohseni: I think there’s two things that people don’t know or do with growth experiments enough. The first thing is doing something that I’ve I’m not coining this. This is the someone else’s coined this before, but essentially a pre mortem on the outcomes. A lot of times what people will do is they’ll come and say, We’re gonna run this experiment, here’s the best case scenario. And how often is an experiment, as successful as the best case scenario? Well, in my experience is like 10% of the time. And so my, my strategy always is, let’s talk about what’s the worst case scenario and see if we can live with that. So that when we’re running that experiment, we can say, Okay, well, we’ve gotten to the worst case scenario, it’s not that bad. How can we retool based on the things that we’ve learned? The second thing that people do in experiments that I find a little frustrating that they should be doing is, they go and look at their graphs, and their and their, and all of the data that they have available to them. And they build the experiments based on that. And so it’s essentially just a quant experiment. Without right sizing, what they’re trying to accomplish through conversations with their customers, and or just stuff that they have in their network, whether it’s reviews, or different than, I don’t know, posts on Reddit, that you can essentially get a sense of what’s going on with your customer, and how that lines up with the experiment. So obviously, the talking to customers is going to be the least scalable, but most impactful way of doing that. So I think there’s there’s kind of those two things which is, which is first they don’t think about the worst case so that they can plan for that and figure out how they’re going to deal with that. And the second one is they don’t talk to customers enough when they’re right sizing the quantitative side of the experiment that they want to run.

Spyros Tsoukalas: You reminded me of my background as an engineer, because back when we were studying computer science, worst case scenarios was a very important part of algorithms and how you analyse what’s going to run in the code. So given those exciting facts that I personally didn’t know, and would you like to elaborate a little more around myths or that like, are related to startups and growth experiments, and maybe other mistakes, important ones that we should mention here?

Chase Mohseni: I think one one myth is that startups have it all figured out. You will go and talk to 100-year old company, and they’ll ask you questions about how should we do this? How should we do that. So I think there’s always whitespace or blue ocean, if you want to call it in every single company that exists. I think the idea is you have to dig and be willing to have conversations and analyse data to figure those things out. So essentially dovetails from my last thing, which is people don’t do both things and synthesise them enough. The biggest myth in my mind is that everyone sees the outside of a company and thinks that they’ve all figured it out. And if you just go inside of any company, whatever size it is, there’s some level of chaos going on. And the best thing you can do is find a way to help them control it in some way. Obviously, like atoms exploding across the universe, from you controlling that, that one portion of the chaos, something else will undoubtedly spring up. But I think that for me is kind of the biggest myth about startups in general. I also think the biggest one, maybe in growth marketing, when I when I talk to people, especially, you know, talking to people on the DTC side is that running ads is growth. This is a myth, right? Growth is about sustainability and, and how you delight your customers from active from purchase through activation, through repeat purchases, purchase through getting them to be a word of mouth, person who delivers word of mouth and new customers to you. So that’s kind of the other myth that I think I would love for to be dispelled more. And so I think this is a little more, a little more built out on the tech side. So people who are in tech, but in DDC, I think it’s not as it’s not as fully understood. And so those are kind of the two myths that I’ve been thinking about a lot recently.

Spyros Tsoukalas: Great thoughts I’m I’m impressed like I wasn’t expecting like an answer that would fascinate me that much. Thank you Chase. So we have mentioned all these mistakes and maybes and things going kind of wrong with startup experiments, etc. So what should they be doing? Like instead? Like, is there a framework? Like some steps that you could describe?

Chase Mohseni: Yeah, so I think the, the number one thing is to first understand yourself, because there’s like business frameworks, but then you have to go action them. And so it’s understanding how you index. So for instance, I come from a creative background originally. And I had to learn everything about data, and I was not good at it for a long time, still not the best, I won’t, I wouldn’t sit here and say, you know, I could mess with a data scientist, they would, they would kick my ass. But what I do understand is you need to be able to balance both your sight, qualitative mind and quantitative mind. And so the thing that I recommend to every single person is to understand your data and see if there are any through lines in there. That makes sense. So whenever you’re running an experiment, or you’re doing an analysis, you’re trying to figure out what things to do next to layer on. And then go talk to say 10 customers and see if there are through lines there. So say each customer is a customer A, likes when a product is delivered in one day. Customer B doesn’t mind if it’s two days. Customer C doesn’t mind if it’s three days, again, terrible example. And then see how your data lines up with that does customer one who has the one day purchase have a higher AOV? Or does customer three have have a higher LTV and what proportion of the new customers 1000 customers over a month, have? Like what proportion of them have chosen those different things, so you can understand what new experiments to run. But if you don’t do one or the other, you will over index on say data that you have or interviews that you have, and not come to a more metered answer or more metered kind of hypothesis, if you will. So my my big kind of call to arms for anyone, whether you’re kind of more data, heavy, or more qualitative and creative, heavy is lean into the thing that makes you uncomfortable. Because there’s a lot of really interesting things. And actually, I find a lot of the people who are uncomfortable doing those things end up getting more rich answers, because they’re going to ask questions that are a little counterintuitive, rather than the person who is used to it. So for instance, a person who is a quant will maybe ask them questions in an interview that a qualitative person would have not asked, because they’re synthesising the information in a different way. And they’re trying to get an answer that maybe someone else wouldn’t have, wouldn’t have asked. Same thing, by the way, on the exact qualitative focused going into the going into the quant side of things. So that’s like my, my big one. And if I could get anyone to do anything, ever, it’s understanding that you want to do like in media res, you know, the middle is going to be the best place that you can be. And understanding that you want to have a good distribution on your team of how people index so that there’s a healthy tension there.

Spyros Tsoukalas: So I think he described how I sometimes, like reflect on how I take, I make some decisions, because I think that based on what I want, based on short term benefit, if I may say, sometimes you prioritise either the one or the other. But you either have like, quality or quantitative or qualitative data. And I think I do it myself in my own life. Anyway, great, great. Great as again, I’m really fascinated, like, honestly. So how can we link all this effort with data and analysis with growth loops? I know you’re particularly good with them. So how do you link them?

Chase Mohseni: So I think, stepping back, you have to understand your business, or at least have a hypothesis around how the business works. And so like, for instance, I’m sure you guys have a way you look at how the growth loops work in GrowthMentor, right. It’s like we onboard the best people in the world, they get people to, to have a great experience and give them advice. The both people so supply side and demand side are excited about this, they write a review, a new person comes in checks out, for instance, my profile, and then the other person will, because they had a great experience, book someone else. And so it essentially keeps spinning because of those magic moments that people are having. And so I think stepping back understanding what your kind of key metrics that will, will make will the again tell you what a magic moment is. So what’s your key activation moment or activation moment, then what’s the key engagement moment and understanding how you get people into those. So if I’m stepping back and saying, I’ll just use Pencil, for instance. We make ads for people, making an ad is very, very challenging. It’s a grind. The magic moment for people is they input all their brand info, which they would do if they were using Canva or they were talking to a designer. That is whether using the cheap the cheapest The lowest the lowest end of the pricing tiers Canva. Or you’re using a designer who’s gonna cost you say 5k a month, you have to put in brand assets, you have to put in copy, you have to give them all of the things that they need. The difference is, with Canva, you’re taking a while to make one unit. And it’s sometimes static, sometimes video, but you’re doing everything pencil, you put everything in, you click a button on what you select, and the magic moment is oh my god, three minutes later, you have a set of four ads and have all the placements, their copy script written everything you’re like, wow. Okay. So it’s saying, okay, that’s the first activation moment. The second one, though is, can we get them to take that ad and export it so that they can go run it on a paid social channel? And so that’s another key moment. And so the idea is like, what do we have to do in between that, to make it exciting for them? And then the last one is, how do we go from that activation moment to re engaging them, so they use the platform again, and so that’s, you know, feeding them, hey, you have more ads waiting for you that have high predictions to win based on the algorithm, that is, hey, you have your ads are winning in your account, you should create more, hey, you know, our team has gone and made you some ads, etc. And so it’s understanding really your customers, and then what they need to do to make the loop spin. And so what we found is, if we get people within that first two weeks to do, you know, two batches, there is a high propensity that they will stick around and have a longer LTV cycle, and also that the word of mouth from them will be much higher. If we see that it’s below that, not that it doesn’t happen. There’s obviously leakage on both sides of that, but essentially, like the, the, with an N of, you know, a couple 1000. And like the my confidence being about 80% on that, usually people go forward and become either marketers for us. Or they they either expand their accounts or keep going with us for a long time. And when it’s under that they usually usually because we have a pay a free plan they will stay on, but they won’t really do much, or they’ll cancel their plan. So essentially, it’s just understanding on a TLDR of this. What is that? What is the magic moment for your customer, and then how can you compress the time cycles between the next magic moment or reengaging. And so my biggest thing is, I don’t want there to be a week in between that, because that’s a long time, because this is not a product that they have. What’s the best way I can put it, like a deep daily engagement with like, use Instagram on a daily, hourly basis, sometimes people are scrolling through that thing. So there’s like a huge deep engagement where it is rooted in your the fabric of your day to day life. This is not that. And so what we want to do is essentially make sure that they understand the value that they’re receiving, so that we can compress the time cycles with which they engage with the platform, that’s how I would get the growth loop spinning is get them in through an ad, make the magic moment happen within that first use case, and then make sure they get through the engagement cycle with in less than a week. So that they get through the second magic moment again, in that first you know, 7 to 10 days so that it keeps spinning and then that leads to net new customers because they will go tell their friends about it. And what we found is our product because it’s cool. I’m doing air quotes. There’s a high word of mouth coefficient there, once we get people to understand the understand the value of that. The other one I’ll just put aside, one is the brand marketing loop. And so because our product is cool, people talk about it, people want to engage with us, essentially, we have built a second product, which is just the brand that runs adjacent to the actual product. And essentially building kind of really trying to build as much strong content as we can. So that when people are in market, there, they have a much, much easier time getting through that magic moment they want to buy and they’ll come in on a paid plan right away. And then they will be because they want to essentially be a part of the Pencil story. Again, air quotes, they’ll activate much faster and stick around because they have been engaging with our content through our newsletter podcast, the video content we do on YouTube and then, you know, engagement in you know, the DTC and b2b communities at large. So I know I went on a bit of a tirade there, but it’s essentially understanding what the magic moments are for your customers. What those mean for activation and then what you need to do to compress the reengagement time cycles.

Spyros Tsoukalas: Got it good. It’s a very interesting thought like how you structured all this, like, in my head, like growth loops have never been described like that, to me at least. So before the last question, because we’re running out of time, like any particular comment around the word of mouth, you already kind of mentioned it, but like any any particularly thing that you would like to mention, around word of mouth.

Chase Mohseni: Yeah, I think virality and word of mouth. People will use it and they’ll, you know, they’ll use Robin Hood as an example. And they’ll say, well, they had a waitlist and it was rock viral because they you know, they got people on there. And then they had a big that a big user base to start. Again, I don’t have the data on this to support. But generally those weightless activation rates are pretty atrocious. And so maybe it had a million people. But how many people activated now they had a million people. So even if they got 10%, that’s a huge amount of users. But to say that the best case scenario is going to be you is a little, I think, a little naive. And so on word of mouth. What you want to do is make make it incumbent upon your users to talk about your brand. And I’m not just talking about your product, but just the magic around whatever your your brand is specifically. And so I think there are two ways you can describe word of mouth, there is the I talked about the brand product. So investing in giving value to your customers outside of your product. And what that does is people start talking about, hey, you should check these guys out. I don’t know how many times I’ve seen as mentioned on Twitter for people that have never used us once. And it is literally because of product engagement, the brand product engagement that they’ve had, where they whether they listen to our podcast, they’ve talked to that on me or someone on our team, or they have just seen our content, etc. The second one is just building it into the product where people understand that they can either, you know, get some benefit through affiliate or something like that, or that they understand that they can get some sort of value out of it. And so what we’re trying to essentially do is say, for every say one customer can we get like point three customers so that when we go and build our when we go build our acquisition kind of funnel, going into the loop that we’re looking for, if we get 100 customers, that’s $100. But we bring in another 33, from word of mouth, does that CAC go down to 67, because now we have 133 for the same cost. And so essentially, it’s understanding first, like what you think is a reasonable number to, to focus on there. The second one is, what are the mechanisms. And so affiliate is the simplest one that you could you could build in that obviously messes with your margin on first purchase, or first month MRR and your payback period. But essentially, if you know what the LTV on your on your customer is and kind of what that runway is, you can then model out what it’s going to be say, oh month two, we feel like we’re in a pretty good place in terms of our gross margins. And what the like overall contribution margin of this customer is to the to the bottom line. But it’s it’s a bit tricky because I can sit here and give you a playbook and it will not matter to your business. Right because your business is unique and how your customers interact with it. The biggest one is there’s two ways you can do it, you can either do a paid incentive, or you can build it into your product. So loom is a perfect example of this right? Every time you send a loom, the loom branding is everywhere. And maybe they send it to someone else on the team. And essentially, it becomes something that is just embedded in the team. And so that’s that is a viral word of mouth product. Slack is a similar one, everyone has used this example. So this is nothing new. But essentially, you sign up for slack, you need two more people to make it make it something and then it becomes viral, you add more teammates and it kind of compounds from there. And then you’ve used it so much the depth of use is so high that you stay on build by programme etc. And so the best way I would I would describe this is how can you get the depth of usage, whatever you’re doing the depth of usage enough that people will talk about you. And so whether that is your brand, whether that is your your product, or whether that is some viral portion of your product that you’ve created. The one thing I’ll call out is people can smell bullshit. And so if you’ve done something where you’re gaming it, you’re like, Oh, I’ll have a like a budget calculator. And that will be viral. It’s like that’s not like that’s valuable on the surface. But there’s no true deeply intrinsic value that a customer will take out of that that will make them want to use their product, they’ll just say check out the spending calculator, and whether like irrespective of your product being attached to that. That’s not a huge value add. So the biggest one in terms of driving word of mouth is make sure you focus on actual value delivery for a customer. And there’s a bunch of different tactics you can use. But that would be my macro thing is understand what your model looks like understand the value that you’re going to provide and understand whether it’s going to be a like a like a money thing, or it’s going to be a product usage thing that drives word of mouth with with customers and the market at large.

Spyros Tsoukalas: Thanks for the expensive uncertainties. Last thing on our side. So any tools you recommend and why you use them or why you love them.

Chase Mohseni: What do I use and love.

Spyros Tsoukalas: Pencil of course.

Chase Mohseni: Yeah Pencil of course. What do we use we use for customer success we use Vitally. Think that’s a very good tool. Obviously, we have our HubSpot, we use Logrocket for watching sessions. We’ve gone to, we use Gong to review our sessions, which I think is great. We use with some of our customers, we always recommend them to use Triple Whale for attribution. I think this is incredibly important in paid media today. Those are some of the tools that we’re using right now.

Spyros Tsoukalas: says thank you very much for your time and effort within GrowthMentor and for this episode, and I hope you enjoyed it as much as I did.

Chase Mohseni: Yeah, I did. Thank you.

Spyros Tsoukalas: Thank you very much.