How Data Science Can Help Optimize Your Business Strategy

Serial Entrepreneur: Secrets Revealed EP96

[00:00:00] 

Uh, we had a little bit of a bumpy start on this show. We’re actually trying to reopen the room. Uh, but bear with us if, uh, if it’s a little bit confusing. Uh, if you’re listening to this in replay or in podcasts, uh, you might not know this, but we actually run a live show here. We’re live on Clubhouse every, every Friday, two o’clock Eastern.

We do the show starts, um, serial Entrepreneur Secrets Revealed, and we have fascinating speakers who, who, who come on stage and share with us their expertise. And you have an opportunity to actually get on stage and ask questions and sometimes share some of your expertise as well on stage here on Clubhouse.

It’s a really fascinating platform. If you’re not already on Clubhouse, it’s something to consider joining. Uh, and we have about 920,000 members [00:01:00] who follow a startup club. and, uh, you’re, if you’re not already doing so, we recommend that you do follow Startup Club. Uh, Mimi, uh, Ostrider writes our blog and she does a really great job.

It’s at, it’s at www.startup.club. And if you do want to find out the speakers that are coming on over the next few weeks, um, please sign up to that mailing list. Cuz what we do is once a week we’ll send out a mailing, li um, a li uh, a mailer, uh, announcing who are the speakers for that week. And we have some incredible speakers coming on.

I know today we have two very good speakers as well. My co-moderator, Michele Van Tilborg, she will be joining us shortly as well. Uh, and, uh, hopefully we will get this kicked off. Edna, welcome. All right. Oh, there’s, she. I can’t even see you. Michele. I’m here. I can’t see you. Yep. Hold a [00:02:00] refresh. I see, I see Edna.

I see you now hold you refresh. I see you. I gotcha, I gotcha. Yes. Gotcha. Awesome. All right. I’m gonna let you take it away from here, Michele. Absolutely. All right. Well, we’re very excited, um, to have the two gentlemen here that are joining us on stage, Manu Rec High, and I hope I’m saying that right. You correct me if I’m not Manu.

Manu is the managing director at Inventus Capital. So we have, you know, a, a VC here in the audience with us. Manu is also, um, gonna tell us how we can contact him, should we have any questions or any ideas, et cetera, that are good for him and his company. And then we have his partner with him, who is Ricardo?

Ricardo is a data scientist and he was the founder of Data Flare. Ricardo also does, um, Teaching college teaching at [00:03:00] universities like Cornell and Berkeley. So we feel very honored to have these two gentlemen here with a very extensive resume. And when they approached us about, you know, what would the members of startup club enjoying hearing, um, we talked a lot about how do entrepreneurs figure out the formulas, how do they figure out how to propel their, the growth of their company?

So they have fully embraced this topic. They have worked with many different companies, funding them, as well as getting them started and successful. So without further ado, we wanna jump in to start the conversation. And they have also prepared, um, a presentation. So I, this is kind of the first time that we’ve done this.

Um, Mimi, are you gonna pin it? The link? Yes, I’ll pin it. Yeah, Mimi is gonna put it up here and pin the link. [00:04:00] So if you, if you care to, um, you can just click on that link and you can follow along this really cool presentation that they’ve put together alongside with their talks. So, I don’t know about the rest of you, but you know, we’re often these days just swimming and Dan, you know, you can’t hardly like look at media any, or LinkedIn or anything business oriented even, you know, the news without hearing about the importance of data today.

Uh, data scientists obviously is a very popular field right now. So these gentlemen are gonna tell. How to wade through tons of data and figure out very quickly what is actionable and what is working and what is not working. So I, I’m very excited to hear this. I am actually a data lover myself. And, um, I’m gonna kick it over to you, Manu.

Why don’t you give us a little bit more introduction about yourself and your company and the role that [00:05:00] data plays in, um, helping the startups grow. Thank you. Uh, well, good afternoon everyone, and, uh, I’m really glad to, uh, be joining this conversation. Um, so, you know, I started my career, um, designing products, uh, as a product manager.

Um, and then over the years, um, as I, um, you know, started to get more experience, eventually I ended up joining a venture. That I was actually an investor in. Um, and for the last 13 years now, I’ve been doing full-time investments, um, through the Venture fund at Ventus Capital, and we do early stage, uh, seed startups in technology and they’re split between Silicon Valley and India.

Um, and, uh, we do both sas, um, enterprise companies as well as consumer startups as well. And, you know, we’d use companies that, you know, leverage data to be able to make decisions and be able to move faster than their peers. Um, so Ricardo and Mike Journey started, uh, back [00:06:00] in 2005 when I was leading, uh, uh, Gmail at, so I was working in Google.

I was leading Gmail Orchid, which is a social networking site and calendar. And, uh, we ended up, uh, collaborating. He was in. And we were trying to figure out why our products were doing so well and, and had as growth and Ricardo, uh, was able to put together, uh, the, uh, the analysis and details that allowed us to figure out and then be leveraging it to then actually approach new markets as well.

So we were doing data science before the word was coined. Uh, I had a scientific background in, in biochemistry, molecular bio, so I was setting up experiments just like I would do a scientific experiment, trying to sort of figure things out and, and asking the right questions, you know, uh, is probably 90% of the battle.

Uh, we’re swimming data these days. Data is very plentiful. Um, but asking the right questions and being able to find those needles in haystack, uh, is sort of the essence of what data science is all about. [00:07:00] So I’ll stop talking and, and I’ll let, uh, Ricardo sort of, uh, add to it.

So, Ricardo, right hand corner, the little mic. There you go. Welcome, Ricardo. Thank you so much for having us. It’s major pleasure to be here. Uh, so yeah, I’m a computer science by training with PhD on artificial intelligence. Uh, I’ve, for the past 15 years, uh, I was a professor. I am a professor, uh, researching AI and machine learning here in Brazil.

And I started recently my own data science company, uh, with the aiming to help startups leverage their data and basically trying to, to bring. Some of the academia expertise to those startups, uh, have established a deep cooperation with inventors. Uh, and with Manu, we go way back. Manu really likes to say that we have been doing data science [00:08:00] before the term existed, but I think this is just gives up our, our ages.

Uh, but we call it like computational data analysis back then. Even then we’re trying to differentiate from statistics and econometrics or so. Uh, but yeah, so this, this is about me. Awesome. So why don’t we just go ahead and dive into it. Um, you know, one of the things I learned, um, also Manu as a product manager at big companies is, as you said, you know, you really have to know the.

before you can do, you know, get proper data and analysis to try to go on your journey. So how do you guys suggest, you know, we’re startups here. Some of us or most of us are spending money, for example, on Google ads, on Facebook ads. How do we really figure out in our marketing programs or in our website flows?

What works? Like what would you [00:09:00] recommend, you know, just top three things to like start down the path of given it’s so overwhelming thinking of all the moving pieces. Yeah. Uh, Michele, that’s a very good question and uh, you know, this is one of the things we really help our startups figure out. Yeah, so initially as you’re building out a product, um, and you don’t have product market fit, the main thing is around doing lots of experiments to figure out what those things are.

So I’ll, I’ll use examples for both consumer and enterprise cause they’re slightly different. Um, but if you take an enter consumer company, um, and you, uh, yeah, your goal is to figure out what that aha moments are like, what are those pieces that allow the users to keep coming back, repeat, get in that recycled mode.

Um, and then, and then as you sort of figured that, uh, atomic use case out, um, then you can actually leverage your marketing spends a lot better. Right. So, uh, a, I actually added a few examples for this, um, on the slides. [00:10:00] Um, so if you look at slide seven, um, and you can do this afterwards, um, um, but I’ll, I’ll walk you through it.

So for example, on Gmail. When I was running this, it took us a while, but we figured out that if you could send three invites to someone within a 45 day period, uh, they had a much, much higher conversion rate. So, as you can imagine, 2005, when we, 2004 when we launched Gmail, um, there’s already, you know, most everybody had four or five, six email addresses already.

You didn’t need a seventh, one or fifth email address. And so we were trying to be a market, uh, trying to penetrate the market when it was already very saturated and understanding what it took to, to grow was very important. So early on we actually focused on figuring out what, what those things were. And, uh, similarly I found that my colleagues at Facebook had figured this out, uh, that, you know, if you had 10 friends in seven days, and so the goal became for the team with the product team and the data.

And the marketing [00:11:00] to actually, as soon as the new user joins in, what can you do to actually get them to 10 in seven days? Cause they had a 90% plus, uh, probability of, of being, uh, 30 day active users after that and 90 days and so on. So, so I would sort of implore our, your teams to startup founders to actually really focus on, on, on finding those pieces.

And data comes in very handy. Um, so being results oriented is very important. Uh, and then having small, nimble teams that you can organize around, um, uh, somebody needs to be the asshole that asked a hard questions, uh, gets people around. Um, and then make sure you define the success metrics before you start those experiments.

So, you know, you may run eight experiments and one of them will work, but you know what the success metric is, so you’ll know when you get there that, that those experiments actually worked. Um, for enterprise companies, uh, you know, it’s slightly different. You, you don’t have the rapid feedback loops. You deal with consumer companies.

Um, but there’s the same thing. Um, You know, making sure you’re testing assumptions, [00:12:00] you know, understanding and asking the right question of data that your, your customers, your enterprise customers are giving you, making sure you have access to those data to be able to then leverage and be able to, to, uh, help all your clients across the board.

If you’re a SaaS company, for example, um, uh, I don’t know, Ricardo, if you wanted to add something else to it. Yeah, sure. Uh, the, what is interesting is precisely how do we make the questions right. There is this phrase, uh, coined by Russell Woff is that says the managers who don’t know how to measure what they want, they settle for wanting what they can measure.

And this is, is absolutely true if we, you have to, to be able to measure what you want. But knowing what you want is the actually tricky part. And this is a part that, uh, it doesn’t, it, it’s not enough for someone to have a data science background and re be really good with the data stacks and doing data analysis.

If the person does not understand the [00:13:00] business, if they are not fully integrated with everyone, uh, in the, in the company, they, they won’t be able to make good questions. And the starting point of ev every data analysis is a good question. Um, and, and let me add to this, as Manu pointed out, that defining good metrics is a very important part.

So this is part of the answers from that questions that we come up with. So we come up with the questions, and now we have to be able to measure. And there are many ways to think about this in terms of what are, what should we be measuring? But it all has to be from data, uh, collected. You have to collect the data first, right?

So for startups especially, this is really hard because they are understaffed. Often they won’t have someone to do that. Uh, but if they don’t track early on what’s going on, then it’s like, Walking on a room inside a room, uh, that is [00:14:00] completely dark. You can kind of feel the walls, the objects, you can, everything that is around you, but you can’t really plan anything if you don’t have the full, uh, uh, vision.

So what data analytics, what data gives us when we collect it correctly, when we make the right questions, is it, it, it lights up the room, right? You can see everything. You can see the trends, and you can plan to, or you can plan your path and really, uh, uh, to, to where you want to, to get to. So, yeah. Yeah.

Ricard Ricardo, or menu. Um, yeah, this is a fascinating topic. Uh, and we have a number of companies in our incubator, you know, e-commerce companies, vacation rental companies, you know, um, it’s just fascinating, but I don’t even know where, like to start. I, I, it sounds like we need to hire these rolls out. , but as an entrepreneur, I guess what we’re trying to figure out here for the entrepreneur [00:15:00] is what are the questions we should ask or who are the type of people we could hire or consultants and you know, yeah, yeah.

You’ve got me thinking, are we doing, doing, you know, are we running our companies correctly? Are we making a lot of mistakes here? We’re not having the right data or information. We might be overpaying for advertising. Um, now we have many, many employees working on these problems, but I’m not certain we have the right people.

Are we talking about programmers or like, two questions there? I guess the first question is, you know, what is it that the startup entrepreneur who doesn’t understand data science, what can he or she do within their startup to sort of begin this?

Yeah, no, I, I, I think, uh, Colin, that’s a very good question. Um, you know, people come to this conclusion, you know, over time. Um, so, uh, at the early stage, as you build in the product, there is no data to have, right? So the key is if you [00:16:00] have, you know, you need to build a product market fit and get there, right?

And as you start engaging and you get data, customers and early customers, you know, you start building a database and, and, and asking the right questions is very important. So an inquisitive mind, uh, and most founders have it, um, you know, uh, if you don’t have it, you know, you, you know, we probably bet on the wrong founder.

So, so, uh, you know, sort of nudging them along, helping them to sort of figure out what data could do. Um, you know, having those conversations in board meetings and, and discussions is, is what allows them to sort of. , but you can’t, you know, we can take the horse to water, but you can’t force ’em. Right. And if the founder’s not convinced that, the CEO’s not convinced, it doesn’t matter who you hire because you know, they don’t care as much.

Right. Um, so, so I think it’s, it’s, you know, everybody’s made differently and, and, uh, you know, uh, so for example, when I was younger and, uh, you know, we, we used to joke about there’s three different types of product managers, and you eventually product managers, because you have [00:17:00] such a large scope of things, you, you end up, you know, a lot of them end up become CEOs as well.

But as, as a product manager, there’s three different types. There’s somebody focused on design, so a design focused product manager or process oriented product manager, or a data oriented one, right? Um, so the, the ones that are more data oriented that already had done this in the past are more likely to, to be inquisitive, but it doesn’t stop anybody else from being as well.

Right. Uh, you’re building a company, you have put everything into it. Um, . So, you know, I think exposing them to conversations like the ones we’re having today and discussing it is, is what triggers them to sort of see the value of it. I, I’m gonna throw out there a few things, um, that I think, and you can tell us techniques, you know, to, to get a good test going.

But one thing I always think about is pricing, right? Like, what is the right price point? And I think all of us as entrepreneurs experience [00:18:00] that at some point you look at the, you know, competitive layout, but then in this web, you know, day, especially if it’s, um, you know, maybe a virtual product, it doesn’t have to be a virtual product.

There are a lot of ways to do price testing and it may be the same. But, um, you know, buy it for one year, whatever, you know, and you get 10% off or buy it month to month. There’s lots of ways to do multi-variate or ab test, right? That you can then dive in, see if you have, um, you know, a statistical significance to help you formulate where your pricing should be, for example.

And Colin, you know, we could talk a little bit about this. Colin and I ran a company.club domains. We did a lot of price testing, um, and we did it very, you know, much in a, um, scientific way. Um, we decided that our [00:19:00] first year, for example, registrations were our levers. So that was the first thing, right? You have to pin up your, what is your assumption, and then you can ask the question.

I think as you’re saying, for example, it could be, um, you know, should we have, um, a yearly. Pricing or should it be monthly, for example. And then you can run these tests to figure out like where you get the highest conversion. That’s the great thing right about the internet, is you can really hone down on it.

And then I think the other big area that is one you know, that some of our companies spend a lot of money on is ads. and I think we all know you can run ads, ab variations, uh, landing pages, et cetera, and get data. But what I always did that I found very helpful is I actually would wake what I called the testing matrix, just like what you said, uh, maybe not as scientific, but we would just say, okay, here’s our [00:20:00] assumption.

Our assumption is we’ll get higher conversion, higher, you know, return on investment if we do monthly only plans leading with those on the landing page. And then we would go from there and, and just test the assumption. And you just keep on going. Once you figure out like, okay, people prefer to save 10% on annual, then you know, it brings to your mind lots of other questions and lots of scenarios to testing and I think you always should be testing all these kind of factors.

Curious about your thoughts, um, Manu. Michele, uh, I think you nailed it. I think one of the, one of the most important experiments or data scientists to figure out your conversion rates, uh, your business model, um, you know, at the end of the day, you know, you’re building a product to be able to, to earn, uh, and, and, you know, the customers have to pay for the product you’re building, right?

So, I’ll give you a perfect example. Ricardo and I worked at a company, you know, where I was. They, they had a product, uh, at Low Labs as a gaming company. Um, you know, gaming, [00:21:00] uh, typically has, you know, one to 3% conversion rates for people, you know, free people, users, to end up paying users. Um, you know, we were competing with Zynga and other large corporations.

Uh, we would number four or five in, in the game. Uh, yeah, we actually, and you know, sort of spent a lot of time understanding the users, figuring out which users were paying more f you know, and then reverse engineering and saying, okay, well if I have that demographic, and I find more people like that. So we’ve continuously experiment and we were doing about three to four experiments a day, uh, trying to figure out, and then changing the, the how, how, um, complicated the game was, how hard the game was for the first two or three days for user, uh, how, you know, how easy you make it.

And then we figured out that there’s an, a curve that users would go into. Uh, if you were very, very strong gamer, you know, you would, if things were too easy, you wouldn’t spend too much time on it because it was too easy, right? And if it’s too hard, you would give up. So there was sort of an angle. Each user had to be customized to be able to [00:22:00] figure out how quickly you would make them go through it.

And so optimize revenue. Long story short, we kept experimenting back and forth and our conversions for from free to paying users went from 1% to about 12 point a half percent. Right. That’s a massive, the industry, the best in the industry were about three to 4%, and we were markedly higher. So we could have, you know, three times less users and spend less on marketing, uh, to be able to get the same bang and buck, right?

And so our return on invested capital or advertis advertising dollars went up dramatically. And so it’s not just about how efficiently you invest in marketing, it’s also how, how efficiently you actually mine your users so that you actually can, you know, uh, get more revenue per user and more users are paying, which is also very strong point about just product market.

So don’t have to be separate decision making, you know, uh, the, the better your product is, the more people will pay, the more people will pay more money. And, and so [00:23:00] you have to be able to have those teams not be siloed, but having, making those discussions intermittently. So, so I completely agree. Um, and, and, and, you know, it’s just like everything else.

You have to experiment, test, and, and, and never be satisfied with what you have, right? And there’s always an optimization you can do to improve and, and get more yield outta it. And that experiment I just told you took me about nine months, eight to nine months to figure out. Uh, we went from one to 13%, but that was a huge improvement in, in our, in our uh, uh, returns.

I, I mean, that’s seriously impressive. One thing that I’m also taking away from this is, you know, as entrepreneurs we also have tend to have very good instincts and gut feelings. And I, and I think a lot of these good tests, man, and Ricardo, start with that entrepreneur instinct. Like, we’re not saying, uh, you know, you’re just looking at pouring through data and trying to figure out, you [00:24:00] know, this little gem.

I, I feel like what you just said explains that, and it’s like the perfect combination of it. I think you probably intuitively know, oh gosh, if we make it too long, so we think our hypothesis too hard, then we’re not gonna get conversion. So you, you really, it sounds like you really worked with a small group and you really went with that intuition to then figure out, okay, are we just making this up in our head or is this actually true?

thoughts. No, absolutely. Uh, I think it has to start with intuition. Uh, and then you kept keep testing your intuitions, right? So in the scientific method, you have a hypothesis and you know, you have to be open to the fact that you could be wrong, right? And being wrong is not a bad thing. It just means that those are that angle or that area of focus you need to do less in interested and then focus more on the other ones.

So information is always useful. Uh, and even if you’re wrong, [00:25:00] it actually helps you figure out where you need to spend your next ounce of energy and dollars, right? So, uh, you know, uh, you know, I always, you know, so, so data science and optimization can only optimize the hill you’re on. Right. It can never help you figure out a new hill as easily.

Right. And that’s where entrepreneurs and backgrounds and intuition comes into play. So, so I always warn people, you know, you know, if this is a very good tool and you depend on it to be able to squeeze every last piece out of it, but it doesn’t help you find the next hill, right? And so if your competitor comes and, and completely changes the way they, they approach a problem and they had, that founder had a very different insight.

Well, you could have the best data science team and won’t help you, right? Um, so, so, so, you know, be careful. Uh, I mean, this is a very powerful tool and, and way to do things, but, you know, uh, it doesn’t replace the entrepreneur’s intuition, right? Uh, that’s, I I think I, I would, you put, but then you.[00:26:00] 

Yeah. And on that note, you know, we’re talking a lot about quantitative analysis, you know, what are your thoughts, um, about, I would say more qualitative, like doing front end surveys and whatnot. Where do you see that in the whole life cycle of the testing where you actually are hearing the voice of the customer, you know, maybe before the product’s even released?

I’m, I’m curious your guys’ thoughts on that. Um, I know in our e-commerce companies, we just, you know, we do a lot of research. We obviously have a lot of instinct, and then we try to do small quantities in, in the ones that are, you know, consumer good products. And then we, our test is actually selling it.

But I’m curious about like, how you’ve used, um, you know, traditional [00:27:00] surveys even in, in your products life. No, absolutely. Uh, Ricardo, you wanna take that first? Yeah. Uh, that is an integral part of the, how we do data analysis nowadays. It’s, it’s really hard to just use numbers because numbers will never tell you the why’s.

So it’s really hard to know why things are happening. And the best you can do really is come up with h what is happening and how it’s happening. Uh, and it, so it surveys and qualitative analysis in general, we can use it before even starting any company just to figure out if it makes sense or not. And there is a way to collect data that is more qualitative.

Uh, surveys usually you can turn into quantitative data. You can count the number of people that will say that would buy some product or so, but I would. as far as saying that even interviewing people and doing focus groups, this is extremely helpful to understand things that you, you would, you wasn’t even paying [00:28:00] attention in terms of putting into a survey, right?

Because if you do a survey, you can only put what you know there and there is a lot of unknowns that can be uncovered by just talking to people. At some point, you need to quantify that and see, okay, is this just, this person is just this group, is this some awkward thing that is happening here or is it broader?

Um, and then you have to integrate those qualitative, uh, uh, analysis during the whole data analysis process. Every time you find something that seems odd, that doesn’t seem like to make sense, you have to go to the user, uh, to the cons, the customer and talk to them and try to figure out what’s going on.

Convert that into surveys to see how. How, how spread it is and then try to quantify it back to data so that you can keep tracking. Excellent. Yeah. And somebody, you know, one research company I was using, this is a while back ago when we did a lot of [00:29:00] surveys. I was at eBay. I remember them even telling me like, be careful when you’re doing surveys.

It’s just one data point. You have to look at many data points. So you know, the entrepreneur, the product manager, whoever it is you have leading this in your company, I, I agree with you. They need to be hyper focused on it and hyper curious. It really is. Uh, I would almost say Colin, . You know, we talk a lot about culture, but it is a top-down thing and it really is something that tends to be embedded in the culture where you’re really of the mindset, where you’re trying to always improve that conversion, that r o roi, that customer satisfaction, whether it be a net promoter score, whatever the key metrics are that drive your business.

Yeah. And I often, often say that business can sometimes be an equation. Yes. Right. Like pod.com we keep, you know, we, of course we have the creativity and we’ve invented all these new lines of products and, [00:30:00] um, dog beds and dog bed rugs and everything. Uh, but after you invent the product, then you gotta go into, you know, there’s seven different ways to sell this product through different methods and how we even scale the product.

Well, we’ll start with a hundred units, then we’ll do a thousand, then we’ll do 10,000. Right. So it’s How do you, and how do you manage that and how does it all. Flaw. Everything’s an equation, essentially. And if you can crack the equation, if you can crack the code, we talk about that on the show, believe it or not.

Manny and Ricardo we’re, we talk about the cracking the code of what it takes to be as serial entrepreneur or to be a successful at your startup. And, uh, today I feel like, well, I’m learning a, I’m really just sitting back and learning today and listening to you two because you’re, you two are absolutely amazing and, uh, I think you are part of the code.

I think you, you, you’re onto something here that, that, uh, maybe we don’t talk about enough. Maybe the, the topic, you know, doesn’t appear to be as exciting when you see it. [00:31:00] But then when you start to think about it going from one to 13, that changes the equation of your company that really makes a differe.

Yeah. Thanks Colin. Um, just to sort of hone it down, since you mentioned you have a lot of e-commerce companies in your, uh, portfolio, um, just to give an example how, the question that Michele asked about qualitative and quantitative, just to sort of, uh, give you an example. So one of the companies we invested in, uh, called Poshmark, um, you know, the founder was a middle-aged Indian man.

Um, not something you would actually bet on for a fashion company, uh, but him and his co-founder. Um, uh, Tracy, um, initially had this very unique insight that, you know, all transaction, all these e-commerce companies were very transactional. Uh, they’re built by men, uh, built for transactions, efficiency search, but that’s not how females shop, right?

Uh, you know, the retail therapy being able [00:32:00] to go in. And so his insight was that, you know, just because they’re built this way doesn’t mean this is how the consumers wanted. So he spent time understanding how female shop, uh, he had insights from a previous company he worked at or ran called Caboodle, where was a precursor to Pinterest.

Um, so, so experience and his background and his. You know, relentless focus that, you know, it’s just, just because men happen to be engineers and product managers and, and CEOs of these large e-commerce companies. This is how they built companies, what they would like to shop, not how the 50 of the world, the other world would actually like to shop.

So anyway, so his insights led him to build, which was sort of, uh, had a social angle to it, right? Um, and, and that insight worked. Uh, he got traction. Um, but, you know, beyond that, you know, in terms of optimizing it now, so I, Ricardo and I worked on it a little bit. Uh, we found out that, um, you know, there was, uh, underlying the data, there was a tertiary data that actually showed that [00:33:00] as, you know, you, you’re e your cohorts declined, but in this case, the cohort would decline and start going back up again.

And I had only seen that, that analysis or that, um, data graph in, in gaming companies, uh, where you truly have a network effect or in social networking sites, uh, I hadn’t seen that in an e-commerce company before. Um, so we kept digging in and found that cohort after cohort actually behaved like this. So underlying, we actually realized that the, that his insights on how he built a product actually was coming true.

But now you have a very massive effect on your cost of acquisition because as people join in and they wanna invite more of their friends, they wanna collaborate. And as, as more people joining the value for the previous users who may have gone dormant, they come back. Cause there’s more people that may have the products they want.

So, and it built that social culture. And I went to one of their user groups in la uh, they had, you know, uh, 400, 500 people, women. And, and the conversation was nothing about, uh, commerce transactions, it [00:34:00] was actually building relationships with each other. So it’s very different than, you know, and that cohort and that, that, that, that engagement really built a strong company.

So it was a combination of his insights, but using data to figure out. How to optimize those relationships, how to build those out. Uh, actually when hand in hand together to, to Ricardo’s point, uh, you can’t have your left side of your brain not understand what the right side of your brain is doing. Yeah.

They, they, they, they have to go step and barrel together. Oh, okay. That’s, that’s, that’s great. Good stuff. All we’re going to, um, simple, uh, we want to jump down to you and, uh, if you have a question or a comment on this topic, uh, thank you for being so patient, uh, simple. Uh, thank you Paul. And, uh, I was listening to, uh, brilliant questions by Michele, and, uh, uh, some answers definitely pay mano Ricardo, uh, not to be too tech properly.

It [00:35:00] might help, uh, all other people in this room. Uh, mano, could you please tell me exactly what was that gem which made the conversion from one person to 13 person from. Uh, the data science angle, uh, you shared, uh, the intuition part and everything is brilliant. Is there any technical part which actually, uh, contributed to this huge change in conversions?

Yeah, absolutely. So, so there’s a few insights, uh, and lots of experiments. Um, so, uh, our insights was that, you know, uh, having, having seen how users were using the product, that if the product was, if the game was too easy for certain cohort of users, they would give up too soon before modernization could actually kick in, right?

You have to last on a game longer enough. The longer you play a game or longer you spend in a product, the more money you will spend on the product, right? And so that, that’s, that’s simple. So our goal was how do you keep each cohort of users longer in our [00:36:00] product for them to be able to, to actually, uh, uh, pay.

Um, so. If you were too easy in the, you know, you start playing the game, it’s like, oh, this is too easy. I’m gonna give up. Go play something else. Cause it’s not challenging enough. Um, then, then you would lose that user and you won’t be able to monetize it. And you’ve already spent money on cost of acquisition already.

Right. And if it’s too hard for certain users for the same game, then they would, uh, give up as well because, you know, it’s, it’s not up to their, their, you know, their capability. So what we had to then, you know, that insight allowed us to sort of figure out, okay, well on the same game, how do I make. Easier or harder for different users.

Uh, so the insights on the data actually went back into product to be able to now to understand which users are coming in. Can I figure out the users within the first day, the first few hours so I can put them in a different track, right? So we had multiple tracks. And so it’s like, well, given what this user has shown in the first few minutes of using the game, and we got better and better over time, as you [00:37:00] can imagine, right?

As as, as we got, you know, more signals and then we can sort of say, Hey, I have a 70% confidence as the user is going to be level three or versus level eight off the bat, I’ll put them into different category. and those experiments back and forth actually allowed us to, to keep improving the revenue per user, um, to, to go up higher.

Right. Um, and then we figured out what those demographics were. So then we did a qualitative research, right? So, um, uh, we had another person, professor, she went back and interviewed those people that were actually paying a lot more money. There were, there were three sigmas, four sigmas off the average. And then they would, uh, she would go interview these people, figure out what their backup, their, their demographic, their background was, and then we would actually lead that information back into our marketing team, um, to be able to acquire more users like that.

So if I knew that these user would pay me, $500 a month versus the average of, you know, x you know, $10 a month, then I’m willing to pay 50 or 60 or $80 [00:38:00] even higher for a cost of acquisition. Cause I know this user will pay, pay me a higher revenue. So, so instead of looking at averages across the board for my acquisition, we were able to segment them into an infinite number of, of, of categories to be able to then go after them.

And so we used to then call it whale hunting. I need to find users that behave like my other top paying users and find more of those users. Um, so did that answer your question? I know there’s a lot of technical details that we went underneath him. Uh, I also hired, uh, economists. Uh, we had, you know, data scientists, we had very multidisciplinary team to be able to think out of the box.

Uh, and what we didn’t have on our team was any marketeers. There were engineers, uh, data scientists, um, uh, economists, um, people that didn’t know what couldn’t be possible. Uh, a typical marketing person would sort of say, well, these things are not doable, so I’m gonna do it a traditional way. Uh, I had a disdain for that.

And so we found ways to make it work, uh, outside of, uh, the traditional norms. [00:39:00] Brilliant. Manu, I mean, thank you. Brilliant. Uh, and thanks Paul for giving me the opportunity. Thank you. Yeah, abs, absolutely. I, I feel like I know data science, it’s, you’re going a lot deeper than, than we normally would, would like, normally the way I would think about a business.

But KPIs are so critical when it comes to setting up your startup, and if you can figure out the right KPIs, just figuring them out can sometimes be very challenging. You know, cost of acquisition, lifetime value of a customer, um, arpu, uh, cost, whatever it is. Like just figuring those things out can be sometimes very tricky.

And if we can figure them out, we can actually, um, begin to. move the dial in many different ways. So can you talk just a little bit about the KPIs, either Ricardo or Manu, that, that, you know, that businesses should be thinking about? I know that’s sort of high level, that’s a bit more high level [00:40:00] than going into like detailed, you know, R O A O A S return on ad spend.

Right. But that kind of stuff. But yeah, I’ll, I’ll, I’ll take the, uh, I’ll do a high level answer there and then I’ll have Ricardo, so take it from here. But, you know, KPIs matter or depend on who’s actually consuming it, right? Um, if I am, um, um, you know, a product manager or a customer success manager, the data that, or the KPIs I need to be able to manage my stuff is going to be different to what the CEO or the VP or somebody else may consume in the, in the startup as well, right?

So to be able to, KPIs are there to make helpful decisions. Um, And, and the person consuming it needs to be able to understand those pieces as well. Right. Um, so, you know, in the gaming company for example, you know, we had, you know, we had dashboards on our TV screens, you know, probably over hundred of them.

Right. Um, it mattered to which. Person was running experiments, which things were important and, [00:41:00] and, and could you move the needle to the main company goal? So you have to have, simplify your KPIs at the company level so that here’s the three or four levers that are most important. Now, you may have sublevels and, and, and sublevels underneath that, and that’s okay, right?

That’s, that’s where your teams are actually looking at three layers up below that one kpi. , but to be able to understand that these, these lower level KPIs or tertiary level KPIs are actually boiling it up to the top level KPIs, and they’re making a difference in what the company needs to move to, right?

And then you’re testing these assumption, so maybe you had the wrong KPI for the first year or two, and then as you learn more, the more data becomes available, and then you realize that, hey, there’s a, there’s a different piece that Ashley is a leading indicator, right? You don’t want your KPI to be a lag indicator, right?

So if you find that something else is Ashley, the, the primary cause, well then, then you look at that more, right? Uh, it may not be, your revenue is usually a, a, a, a lagging, uh, um, indicator, right? Um, customer engagement, you know, if you have a leaky bucket, you know [00:42:00] that’s going to affect your revenue down the road.

So how do I fix those things? You know? So I, I focus on things that either help me with. Uh, you know, uh, engagement, retention, um, you know, if you’re a SaaS company, net revenue retentions a very important metrics to look at. Uh, not just the, the top level revenue because, you know, you know, revenue and as, as nps, uh, which is the qualitative metrics of how well your customers actually like your product is very important because that tells you long term, uh, you know, whether they will recommend, you know, referrals, uh, and other pieces as you build your product and company out as well.

So, and nps, you talking about the net promoter score? Yes. You’re talking Yeah. And that’s a fascinating, that’s a whole other topic, but that’s a fascinating one, isn’t it? And um, it’s really a simple measurement technique. You can Google it on the internet net promoter score, if you’ve never heard of this.

It’s something you should, every company should just take a look at. It’s a very simple survey method, which can give you [00:43:00] a, a very, a number. And they’ve shown that with that net promoter score. That the, um, the comp, but some of the top companies like Apple and Harley Davidson, um, they’ve done extremely well.

And a lot of it’s related to the fact that they have, um, people who are enthusiasts, they’re recommending that company to their friends and family. So yeah. We, we, I know we, we, if you’re new to like, startups today, like we’re, it is, we’re jumping, we’re doing a lot of acronyms and stuff like that, but I just thought I could come in and talk a little bit about Net Promoter Score.

We should probably do a show on just that alone. Invite you back. Yeah. I’m, I’m just gonna jump in real quickly before we go up to Gem. Um. One way that I have often found is good in terms of thinking about KPIs is I look at competitive, similar companies. If they’re publicly traded, I go look at what they’re [00:44:00] saying in their filings, what they’re saying on their quarterly calls.

It can be, you know, a big eye-opener for you. For example, you know, like I said, we sold doc club domains to GoDaddy, so I would look at our competitors quarterly earnings and their calls and what they were saying in press releases. That oftentimes can give you some pretty strong proven hints about, you know, things to think about and experiment in terms of how, you know, to really, you know, drive your company to be more profit.

thank you. But I really wanna get to Jim. Jim has a lot of experience as an entrepreneur. And Jim, we’d love to hear your question or any insight you have for our members and our folks here on the stage. Thank you, Jim. Thank you, Michele. Very interesting conversation. Uh, it’s. above my head in many ways, but that’s fine.

Uh, uh, [00:45:00] early on in the conversation, uh, we’re talking about AB testing. So, uh, I got a couple, uh, thoughts here. If, say a, uh, a startup person has a product, uh, they’re bringing it to market. They’re kind of like in the, uh, initial phase of testing price points. Uh, they’re copy, uh, their images. Uh, the first part is, is that.

do you run multiple testing or experiments at the same time, or do you say do price evaluation first and then go to another, uh, say if it’s the product images and then go to the copy. How would you suggest for a person, uh, bringing a new product to market and it’s already live, say it’s on Amazon or, uh, their Shopify store, what would be a easy way for them to start building a data set and [00:46:00] start making, uh, some informed decisions?

So that’s my question. Does this a very good question and it really hard to answer because there is a lot of nuances on how to run AB tests. Uh, I think with Michele mentioned VIR tests that allows you to test many different variables at once, so you don’t have to run a lot of different ab tests. Uh, , but my personal preference is to run a very few small number of, very small number of ab tests that you can focus and learn from that.

I often find that there is a statistical, uh, um, phenomena that happens that if you run a lot of tests, some of those will be positive, uh, by chance, right? Because there is always a, a, a, there is always the possibility that when you run an AB test, the difference, even though it’s may be statistically significant, uh, it’s not really, [00:47:00] and the more test that you run, the higher your false positive will be.

So, It’s not advisable to run a lot of ab tests simultaneously unless you are, you have a lot of variables that you can work on and can segment a larger market, and that is not the case, uh, at all times. So my personal preferences is run a small number of ab tests, maybe just one single ab test, uh, at a time, and really dig into that and learn everything that you can from that, and then go to the next one.

Uh, and, and I’m not even considering budget thing, uh, budget, uh, the budget that you might have for that because AB test can be expensive to run if you run multiple discounts for many different segments. Uh, that can run, can be quite expensive, uh, but not taken into account then into account. My main worry of learning a lot of ab tests is false positives and that you’ll be learning things that are not true.

Yes, that, that’s what I’ve, uh, noticed Also, uh, [00:48:00] uh, just a follow up question. The length of the test should, uh, there be a certain, uh, timeframe, uh, seven days, 14 days, say if I’m, uh, out there testing, uh, price points, is there any, uh, timeframe that, uh, will give you enough data? Hopefully you have enough sales that you can, uh, take a look at it.

There, there is, uh, so there is this, you can look online in that you can find, uh, uh, equations or, or even an online system that can calculate you the power of a such Cisco test. And that, uh, requires you to have a certain number of data points. So the time that you are going to be running an experiment will depend on how many data points you are going to be generating.

So if you are selling only once a day, then you might have to run that for two weeks, 30 days even. Uh, if you are selling multiple products an hour, then maybe. Running through a day is enough. [00:49:00] Uh, it all depends on what is the sample size that you have and how confident you want to be on the difference that you are observing from the, the, from the difference on, on the m b tests, uh, it’s really hard to, to say, uh, one week is enough.

Uh, but it really, really depends on, on the number of data points they’re generating. And just another follow up. So say if a person does, uh, run it for say, two weeks, uh, should they, uh, abandon that for say, uh, 30 days and then revisit it? Or, uh, what, what’s your feeling on, uh, going back to a AB test, say on pricing, you know, uh, you know, what timeframe, uh, should you have a rest period in between testing?

I’m not sure I have a good opinion on this. You mean you are [00:50:00] writing a Navy test, you didn’t finish it and then you go, you stop that and return after a while? Well, uh, here, here’s example. Say if I, uh, launched a item and let’s say 39 99, but I looked at the, uh, competition. Uh, I always try to be in the top quadrant on pricing.

That’s my goal always. When I, uh, have a product, you know, I will not be in the qu top quadrant price point wise, but you know, , I want to kind of test to seeing if 39 99, uh, is doing well. I’ll, you know, put a couple of variables in there, but I, my ultimate goal is to get to 49 9 and without changing any other variables, meaning images, or copy, uh, and say the original test, uh, I was able to move, move it up to say 42 99.

But [00:51:00] should I just, uh, let it rest there for say, 90 days and come back, gather some more data, some more sales, and then go back and, uh, test, uh, higher price points. I guess that’s a more specific.

I think I, I, I I can take that question. Um, okay. Yeah. So, so, so the way to do, uh, AB experiments that could actually, you know, without messing up what you have already is to actually do it on a smaller cohort, right? So you’re not exposing everybody. Um, so, you know, like if the experiment is negative and, and is bad, you know, I wouldn’t want that to, to sort of ripple through all my user base, right?

Um, so typically, you know, statistically valid answer would be, you know, I get 5,000 people exposed to it roughly, that would gimme, you know, 90% plus confidence level on my AB results. Um, then, you know, I don’t need to expose this to everybody, right? I, I will get [00:52:00] my answers on how effective my AB test is, and then I can roll this onto everybody else, right?

So that’s just good sort of, uh, rollout product, uh, management piece. Um, And, you know, if it was negative and if, if my changes actually caused you negative effect. Well, I don’t want that to ripple through across the board. So I think that that may be a, a, a better process to follow. Okay. I, I appreciate that.

Thank you for your answers. I greatly appreciate your time today. You’re welcome.

All right. Like, I, I feel like I’m learning so much, but I’m also feeling kind of guilty. Colin, I don’t know about you. I’m feeling like I missed a few classes in college. Yeah. Well, you know, I, I have done this kind of thing, but it’s been a long time. I’m just being Yeah. You know, honest here, I, I feel like we’re only scratching the surface.

I, I know. I haven’t even, we haven’t really figured out that there’s probably so much to learn that could make such a huge impact on our [00:53:00] businesses. Yes, there is. On data science, we just, and I think it is a discipline, right? Manu and Ricardo, like it really, we kind of need to set that tone and. Start asking some of the questions.

I, I think, Manu, you said that in the very beginning, somebody has to, I’m sorry, I don’t wanna offend anybody, but somebody kind of has to be a bit of an asshole and just start asking the hard questions and demanding, you know, a, a test to answer those questions and not just believing people’s feelings right.

About things. Yeah. I’m gonna ask, could ai, sorry to help with this too, like, could AI actually help solve some of these problems we have? Absolutely. Great question. No, that’s a very good question. So actually there’s a, there’s a couple companies, uh, if I can plug them in, uh, two companies that I invested in, but Ricardo’s actually being, uh, on the data science part as well.

Um, so for, for step function, uh, this company, uh, step function ai, they actually. Companies figure out what their churn [00:54:00] might be, right? Because churn can come from everything. You know, what, you know, the customer attraction to NPS score, to product releases, to bunch of other things and, and how they’re using the product.

If my engagement changes, what is that a signal that eventually this person may churn out? Um, so understanding that can actually really help your customer success team to focus on, on high value items and yeah, and, and focus. Um, so rather than spending hours and hours and hiring data scientists and, and data scientists like Ricardo, not easy to come by, right?

They’re expensive, they’re hard to come by. The right one may not happen. So can I leverage other SaaS tools like step function to be able to plug in and gimme the insights that I need to be able to make that work, right? So they do this very effectively for, for SaaS companies to be able to get to those answers.

Um, For e-commerce companies. Uh, there’s another company that Ricardo and I work with, uh, you know, called out of the blue. And so, you know, when you look at events that happen for an [00:55:00] e-commerce company, you know, you could have a, a war started in Ukraine, well that could affect your sales or, you know, it was too cold and, and UPS couldn’t.

There’s a whole bunch of things, but those are big, sort of systemic things that can happen, that can change your, your revenue or your sales. But there’s a whole bunch of things that happen day to day, right? Maybe one of your, um, um, pricing was offers or servers were down, something else happens. And how do you keep track of all these events that happened once your product is, you know, live and running?

Um, and, and so this company Step, uh, sorry, outta the blue, manages all these complex. And boils it down to a few KPIs that, that your teams that are on the ground, ac acting can actually take and, and, and work on, right? Um, so knowing that this is a hard problem, how do you simplify? It? Can, can you have, uh, a scalable AI solution that can actually help you get to those answers?

So, so there are solutions, um, uh, you know, I don’t think, you know, it’s easy for, for most people to hire a [00:56:00] 300,000, 400 data scientist. And if you, you know, or you can do it part-time. So you, Ricardo, for example, you know, you, I mean, three or four hours of time a week is actually, you know, um, an average statistician or data scientist could spend two weeks trying to figure the same thing out, right?

Uh, so, so more solutions in AI are coming through, uh, regenerative ai, um, you know, things like chat, gdp, you know, all these things will improve how quickly can get the data, but, As we mentioned at the beginning, uh, having more information is not always helpful. Getting to the right answers is, is and asking the right questions is, is the battle and some of these companies are helping fair?

Well, if all these e-commerce companies have these questions and they keep coming up, maybe if you’re starting an e-commerce company, well you do. You have to rebuild everything from scratch or could you sort of step on giants that are already there. And so R Blues tries to build that level of expertise so that you don’t have to worry about rethinking [00:57:00] everything again.

Did that answer your question?

Yeah, no, that’s great. That’s great. I just, I guess just so much wealth of information here. Yeah. In such a short period of time. We’re out of time. Um, but we’ve got vahid on stage and I really, you know, Vahid, I dunno if you have a question or thought about this whole data process. Uh, appreciate you also jumping on stage there.

and, uh, hey, good morning everybody. No, loved it. This is an amazing room. I came in, my wife is in a deposition, so I have to be quiet. Um, what I wanted to, uh, say to Ricardo is that, you know, you have those outliers and false positives. I contacted, uh, two, three days ago, I contacted one of these SaaS companies that was, that I was getting my photos from and images and vector files, and I asked them, I’ve been a client for four years.

I didn’t renew my subscription. One of my guys wanted to use it again, so I contacted them because the price has gone up 50 bucks from last year [00:58:00] to a hundred and like 50 bucks. So I sent a message, I was like, Hey, do you guys customer support? I said, do you guys have any coupons running for, for, you know, for your product?

And the customer support replied back the next day, no, we’re not running or anything. I said, okay, sure. So I went on the, the, the page and I’m like, okay, I’m gonna sign up for it. I went to sign up and he gave me a 20. . So then I screenshotted that and I sent it to customer support. And I said, just within the last two days, I told three other people that your prices have increased, but your price is actually almost close to what it was last year.

And you could have had three more clients. But I referred them and I told ’em, you were expensive because you didn’t have any coupon going on. So sometimes they’re losing business because there’s a disconnect between the people within the company. Like they don’t even know that their shopping cart is offering a 25% discount, and they could have had couple of [00:59:00] clients paying.

So you’re absolutely right. They gotta see the data. They gotta look at the data. I think one email coming to customer support like that should be taken. Imagine how many people don’t email the company and support it. So I think data, looking at it, and sometimes we’re involved in the company too much so we baby it too much.

Sometimes bringing outside people are looking at the whole thing, see what’s going on. The little things make a big difference. That’s what I wanted to. That, that, that is a great point. Uh, it adds up to, to what we, we were talking about with journalists. Uh, sometimes this happens because of, of the many AB tests that is happening, a lot of experiments that are happening at company and they’re not tracking them properly because it’s way too much for this SP team.

They may ha might have. So one hand does not know what the other hand is doing, and this leads to this kind of, Problems that you, you wouldn’t, you agree that your best client is your old client? I feel like you already got them and everything else. You, they already know what [01:00:00] your product is. They have an account with you, they got the credit card or PayPal or whatever on file.

So I feel like the number one goal should be, you know, trying to revive those or get those, or just make sure that those are raving fans to begin with versus going, spending so much money getting new clients. But it’s another way of thinking about marketing. So I feel like that, and this company in the last 12 months that I possibly have not, uh, you know, been with them, has not sent me a single email that, Hey, by the way, you use your system for four years.

What the heck happened? Why you’re not using it anymore. Right? Nothing. So to me, I’m looking at ’em all like, how many millions of dollars are they like throwing away on Google ads and Facebook ads? But they could have had so much clients from their previous clients and they could have done the retention.

So it’s just something to look at. No, but go ahead. That’s a very good question. So, so let me, let me, let me make an, a slightly, uh, alternate point, uh, for this company. They should have recognized before [01:01:00] you actually churned to actually reach out to you because you know, there’s enough signals that you were gonna churn out, right?

So just to give you some data points so that our company step function, uh, you know, has helped lots of companies. But, you know, just to give you an example with, with Mitel or with AutoD desk. Which is one of their clients, they were able to reduce churn by 50%. So before it happened, before you got dissatisfied and sort of said, this money I don’t need, this is not worth my time or money.

And they could have intervened and sort of said, Hey, maybe if I give you a discount now or can I, can I help you? Or, or, or, what is it about my product that, that is dissatisfying you and, and being proactive before you actually churn out is actually very. So, so yeah, I can go get you back after six months, after you’ve gone, but so much better that two months before you were actually going to churn out.

If I could predict that, then I could spend my energy and, and retaining you better, right? And, and it could have been easy. All they needed to know is that I haven’t logged in because I’m, I’m not personally using this. My team is using these images and, and, and graphics. And [01:02:00] so the minute that they could see that for one month, two, month, three, Nobody logged in.

To me that should have been like a red flag. Why is this person paying for a SaaS company, let’s say 20 bucks a month, right? Or, or 10 bucks a month and they haven’t logged in. Let me find out what’s going. But I feel like sometimes, would you say that maybe they’re hesitant to send that email to see what’s going on?

Because then they might alert you and then you might want to go on your credit card saying, well, you know, they just told me and I’m not using it. Let me just go really cancel now. Right. instead of, you know, a lot of company just hoping that people forget about their subscription and they make a lot of money by, you know, users like me, for example, for 3, 4, 6 months I paid and I didn’t use anything.

So that would be a good split test for me to know that. Is it better to reach out when they don’t use it, or you just let it go because you are providing a service, but technically you’re collecting free money? Yeah, that’s correct. And so companies like function allow you to do [01:03:00] that without having. You know, to, to have somebody like Ricardo, they’re constantly looking at data, right?

So can you proactively do these things? So, so I think companies, you know, this is where, you know, it separates, uh, you know, uh, uh, uh, you know, asking the right question, making sure, hey, if churn is important as part of my long term ltv, and I need to make sure I have the right resources, the right tools in place.

And, and so those companies eventually will do better, right? And, and for every category, there’s multiple players and eventually the ones that actually focus on customers, build relationships, uh, do all the right things will win. And, and then data’s an important part, but not the only part, as, as Ricardo mentioned.

Yeah, I don’t know why it’s getting so much difficult than what it was 10, 15 years ago. Or has it always been like this? I feel like the co-founders and CEOs of companies now go through more pain and suffering and cruel and unusual punishments, , because we started late. Maybe, maybe Colin could give it that input on why is it more difficult these days, man.[01:04:00] 

Uh, I have no idea. . Yeah. I’m sort of with you on this one. Like, I don’t have the answer to that question. I, I, I do, I do like the fact that earlier, I think Ricardo, you brought up this idea of engagement and, and using that as a metric to focus on. Like, so at.club we would track a religiously, um, the engagement level of the domain names cuz when we knew when someone used a domain name, they kept that domain name, they paid for it.

And your renewal rate was almost a hundred percent. I mean, it’s 10 bucks a year. Right. A lot of money. It was all about usage. Right. We even had it, it was one of our core values. Usage is everything. That’s right. Right. So that, you know, it’s just, it’s interesting you brought that up. But we do have to close out the, the show today.

Michele, any last, uh, thought. Yeah, before you, I mean, I’ve immensely enjoyed this and so thank you Manu and Ricardo, just, just real quickly, if somebody wants to get in contact with either one [01:05:00] of you, what is the best way for that to happen? Manu, obviously you work for a VC firm, Ricardo, you work alongside Manu doing this analysis.

What would you want people to do in order to reach out to you man and Ricardo?

Yeah, I mean, uh, people can reach out to us through our website. We have, uh, info inventor scap.com, uh, and uh, one of our team members will definitely respond back. Sorry, could you say that again? Is at the top? Oh, it’s at the top There. Is that inventus inventus dot inventus cap.com because it’s for the podcast listeners as well.

Just wanna make certainly inventus cap.com. Got it. Excellent. Well, I’ve learned so much and you know, you, you’ve in reinvigorated me a little bit. I’m thinking about little things that I need to do for the businesses that we manage, um, you know, just to get better, more, you know, [01:06:00] unbiased input into what the company’s doing.

But if I had to name a couple of things to suggest to people, I’m gonna say, do your net promoter scores. You’ll learn a lot just from the customers. And I think even at the base level, make sure you have Google Analytics installed. . And then, you know, thirdly, I would say I’m gonna work on, um, a high level segmentation.

I’m not gonna make it over complicated, but I’m gonna do a couple of high level segmentations of my customers, because you’re right. Like we should be targeting and spending money on our, you know, buying and better customers. So you’ve reminded me of these things and, um, I feel like I’m putting them on my list to start next week.

Thank you. Yeah, it was a pleasure. Uh, thank you Michele, on calling. All right. Thank you so much. Yeah, absolutely. Appreciate it. Thank you. Thank you very much. Uh, sorry if I may ask another one question, Ricardo, is that time? Is [01:07:00] it okay? Yeah, go ahead. Uh, just thinking about, um, on the healthcare sector, when we talk about, uh, clinical trials and other things, there is one, uh, absolute issue when it comes to patient recruitment.

Patients retention. Uh, if you have any idea who data science can help you, a lot of surveys, but the trouble ratio is around 90 plus person. Um, can AI come into this or data science?

I’m simple. I, I’m sorry, but we could not really hear you very well, and you’re breaking. You were breaking up.

Yeah. All right. Well, I think we’ve come a little bit to the end of our hour here. We actually went over a little bit. It’s great conversation, and thank you everyone for joining. Manu Ricardo, for your [01:08:00] expert advice and for everyone who came to the session and everyone who shared their experience. Have a wonderful weekend and we’ll see you next Friday at 2:00 PM Eastern Time.

Be well.