- The speaker is Sverdolce from Mixpanel, a company that helps measure product usage.
- The speaker asks the audience about the number of users they have.
- The speaker introduces Mixpanel and its capabilities in measuring product usage.
- The speaker discusses the importance of understanding, ease of use, and user retention in product measurement.
- The speaker presents a formula for growth and emphasizes the importance of focusing on a few key metrics.
- The speaker explains the process of optimizing each step in the user journey.
- The speaker concludes by highlighting the simplicity of measuring and fixing product issues.

- The speaker emphasizes the importance of having a good landing page to prevent users from leaving.
- The speaker shares the evolution of their own landing page design and messaging.
- They discuss the importance of measuring whether users understand and find value in the product.
- The speaker suggests using metrics such as sign-ups and engagement with specific features to gauge user interest.
- They also mention the importance of making the first step of using the product easy for users.
- Examples of easy-to-use products like Google and Airbnb are provided.
- The speaker suggests using funnels to measure user progression through the initial user experience.

- The first two steps in a game's funnel often have the greatest drop-off in conversion rate
- Speed is important in user experience, as a complicated process can deter users
- Email and text confirmations have a high drop-off rate and can be a detriment to user retention
- Iterating on the initial user experience is crucial, as it will be an ongoing process
- Allowing users into the product without signing up can lead to better user engagement
- Retention is often ignored by startups, but it is a critical metric for long-term success
- The "shark fin effect" occurs when an app goes viral but lacks retention, leading to a decline in user acquisition
- Tracking new users and daily active users are important metrics for measuring retention
- Monthly active users can be a misleading metric, while daily active users have a strong correlation with churn rate
- Measuring monthly dollar churn is crucial for subscription-based businesses, as it can lead to significant revenue loss over time.

- Retention is crucial for business growth and should be a focus alongside marketing and improving landing pages.
- Monthly churn can lead to flat growth and eventual decline in revenue.
- Talking to customers and gathering feedback is important, even for companies with few users.
- Simplify analytics and focus on one North Star metric that can make a significant impact on the company.
- Don't overwhelm the team with too many metrics, less is more.
- Recommended advanced topics to read about include "The Next Feature Fallacy" by Andrew Chen.

- The speaker suggests asking friends in tangential industries for conversion rate information rather than relying on competitors.
- The speaker discusses the balance between simplicity and informing visitors on a landing page, using examples like Airbnb and Craigslist.
- The importance of having a guiding principle and experimenting to find the optimal approach is emphasized.
- The speaker shares a case study about video tutorials and the importance of running experiments to develop hypotheses and measure results.
- The difference between user retention and dollar churn is explained, with the speaker highlighting the significance of retaining revenue.
- The speaker mentions that the discussed metrics can also be applicable to hardware startups with a software component.
- The speaker addresses a question about targeting different company sizes on landing pages and the changes in Mixpanel's approach over time.

- The speaker discusses how they initially had no idea who their customers were and only later realized that their target market was small businesses and startups.
- They employed a trick of charging customers for their product but giving it to them for free if they said no, in order to gather feedback and see if they would pay for it.
- The speaker explains that they focused on building features that would make a difference and retain existing users, rather than just acquiring new users.
- They mention the importance of gauging product market fit through qualitative and quantitative measures, such as retention and frequent use.
- The speaker advises against complicating things and getting easily distracted as a founder, emphasizing the importance of staying focused on one thing for a significant period of time.

Sverdolce from Mixpanel discusses the importance of measuring product usage and user retention. They emphasize the need to focus on key metrics and optimize each step in the user journey. The speaker highlights the significance of a good landing page, understanding user value, and making the initial product experience easy. They also stress the importance of retention, tracking metrics like daily active users and monthly dollar churn. The speaker advises simplifying analytics, gathering customer feedback, and staying focused on one North Star metric. They share a case study on video tutorials and mention the applicability of these metrics to hardware startups. The speaker also discusses Mixpanel's approach to targeting different company sizes and their focus on building features for existing users. They caution against complicating things as a founder and emphasize the importance of staying focused on one thing for a significant period of time.

Suhail Doshi, founder of YC alumnus Mixpanel and a world-class expert on measurement, details how startups should think about discovering the important facts about how their product is used.

You can find the lecture transcript and slides here: https://www.startupschool.org/videos/39

Learn more at https://www.startupschool.org/

We are very grateful to have Sverdolce from Mixpanel, who co-founded Mixpanel almost 10 years ago now, and is going to talk about how to measure your product, which, as you heard from Gustav, is really the other side of the coin of growth and everything that helps you build something that people truly want and will use.

I thought I would start off by just kind of like, I sometimes try to gate, try to change my presentation a little bit on the fly, depending on the audience.

And so I thought I would just ask a really quick question, just a quick show of hands.

I'm going to define a user as just someone that uses your product, whether it's a B2B company, paying free, doesn't really matter, like they did something in your product.

How many people have like 0 users right now?

And then how many people have 100 or less?

Well, okay, You have greater than 0, between 1 and 100.

And then how many people have more than 10,000?

All right, so we've got, I'd say, majority of people have 0 users right now.

So I'm going to start off with just a quick about me, for those that haven't heard of Mixpanel.

I think the first line of code was written in October 2008, technically.

I was in my parents' house in my bedroom.

And 10 years later, the company has about 300 employees.

And we have about 7,000 paying customers.

And we're starting to close in on around $100 million in annual recurring revenue.

So just to give you a sense of 2008 to now.

If you don't know what Mixpanel is, we make pretty graphs like this.

We help you measure what people are doing inside the product.

So this is like someone inviting a colleague to their app.

We might help you measure a funnel and see the conversion rate of people opening your app to viewing an article.

I know Gustav went over that a little bit today.

And I'm just showing you this just because everything that I talk about today is basically something that you could probably do in Mixpanel or even any other tool.

There are a bunch of tools that are great to measure these things, but I want you to know that if you walked out the door and you wanted to apply any of it, you totally could.

There's like a freemium plan and we cater very much to startups.

Those are all of our early customers, So feel free to do that.

So today what I wanted to dive into was I thought I would take a top-down approach to thinking about analytics and metrics.

Instead of kind of coming at you with charts and visualizations and numbers and things like that, I thought it would be better to think about the problem that you're trying to actually solve and ignore for a moment that you have to do complex analysis and calculations.

It's not hard or you'll just use tools that will make that a lot easier for you.

So that's 1 thing that I thought I would do.

I tried to think pretty hard about like what the heck did I care about when I started a company?

And I thought, well, I think the first thing is just like, do people even understand what I made?

And Mixpel had a really hard time with that in its first 18 months.

We had to change a lot of things, and most people do.

They kind of understand what it is, but do they find it relatively easy to use the product on day 1?

And that's kind of this never-ending mission that you'll end up taking when you start your company and when you ship your product.

And then the last thing is, I kind of tried to pare it down a little bit, but I thought the most important thing would be then next, are people coming back and using my product.

And I think that you'll find that, like, I know Gustav talked about this, and I'm going to be extremely repetitive, because it's 1 of the most important things that early founders make a mistake with.

So from there, I thought that it would also be helpful for you to see it kind of broken down in a formula.

And I think, I like to think about things from first principles, right?

Like what are the things that would cause you to grow?

There's just like people that visit some page or the app, right?

And then there's like people that sign up.

And then of those people, like who did something— How many of those people did something valuable, like watching a video or making a recipe or taking a scooter ride?

And then of the people, how many of those people end up coming back some period of time later?

And then how many people will spread that product, like tell a friend, share it, maybe you even have a sales team, who knows, right?

And these 5 things basically equal growth, and these are your levers.

And the thing that I want to kind of point out and I want to stress is that today I'm going to cover the basics.

But I want to kind of implore on you that the basics are really important.

Even thousand person companies get the basics wrong.

In fact, you'll find that if you don't get the, even these thousand person companies that get it wrong, they tend to overcomplicate the number of things that they should actually measure and track.

And it's really easy to make 25 metrics, conceive of them, align them, divide them all by, give each team 5, and then really overcomplicate something that is actually quite simple to do.

When you overcomplicate it, What happens is that companies, even large companies, have immediate paralysis.

Even mid-sized large companies have this decision-making problem.

I used to have this funny phrase at Mixpanel where I used to say that all data is contestable.

You could see this number go off and to the right, and you go, oh, well, I don't know.

And people will have these arguments no matter what, because causation is really hard to discover.

And so what I find is that just picking like 3 to 5 things on Northstar and really simplifying it will do more good for you.

And I know because not only have I watched thousands of companies make this mistake at all sizes, I have made this mistake.

I had a piece of paper that had like 25 submetrics and it was nearly impossible to actually keep track of it.

And what people end up doing is in reality what people care about is There are metrics that guide the team, and there are metrics that help you monitor and assess if anything is going wrong.

And today we're gonna talk about things that guide the team.

So, with this formula, you should be able to assess anything going wrong in the company, right?

If you have a really confusing landing page, you should be able to figure that out.

If your signup process is too difficult, you should be able to figure that out.

If you have a product that people don't really value, you will figure that out with just that formula.

And you can get more complicated after that.

You could say, OK, landing page views are down all of a sudden.

Let me look at that segmented by country.

Maybe I changed something in a different country.

So you should be able to assess any reason why you're not growing just starting out with that.

That's why I say starting simple will last a very long time.

You have this funnel of people going from visits to eventually retaining and spreading the product.

And on each 1 of these things, you're trying to basically solve 1 of these 3 problems.

Are people coming back because they find it valuable?

And the funny thing about this is even if you're like Airbnb and you're thousands and thousands of people, this process of like optimizing each 1 of these steps is never ending.

No matter what scale you're at, you will always be changing something.

And you probably have competition, so you'll have no choice.

And then once you are able to measure those 5 things and assess what's wrong, you have your levers and you should be able to take action and fix any of those things.

Put a team on it, maybe you're fixing it.

So, we're going to start with the first thing, which is, is my product easy to understand?

In order to illustrate this idea, I decided that it would be somewhat sympathetic because I could show you really awesome landing pages of great companies, or I could just show you what it's like to start out.

And I also wanted to mention that having a really bad landing page, there's this really awesome quote that PG used to have in our batch that I thought I would impart on you, which is that people basically just have their mouse hovering on the Back button.

And they're just waiting to say, I don't know what this is.

And that's immediately how most companies are just losing most of their users from moving on.

And so just remember that your enemy is the back button.

So this is what Mixpanel's first front page looked like in 2009.

That's not what our logo looks like anymore.

This is a design by me, original design by me.

And in all of its glory, If you actually go read the words, there's tons of grammatical mistakes because I was horrible at writing.

And for some reason in my mind, I thought, I know what the tagline for the company should be.

And I think the reason why was because I thought these graphs were like so pretty looking compared to what was out there that I thought like that would be like the value proposition of the company.

And of course this is like totally wrong, like You're all laughing.

I wish I could have made this landing page, presented it to you, and then you'd all laugh at me.

I'd be like, okay, I should change that thing.

And I tried to target small startup, big company.

Application developer, I'm trying to target everybody.

Every mistake that 1 could make on a landing page is right here, right?

Like the sign up button, at least I kind of got that right.

Thank you Adora for teaching me how to optimize buttons that slide.

But that's about maybe all that I got right, right?

Just maybe the roughly the sign up button.

And then, you know, after a lot of work, a lot of hard work, a lot of iteration, a lot of talking to customers, a lot of measurement, you know, eventually a couple years later, it kind of turned into this.

The sign-up button's ginormous and yellow.

And what we figured out, though, after a lot of time and a lot of energy was that we transformed from metrics that'll make you drool to actions speak louder than pages.

And you'll notice it's extremely prominent.

The reason why was because the number 1 question that we would get as a company was, that's cool, but how are you different than Google Analytics?

And so it turned out this tagline for us at the front page lasted the next 5 years of the company.

And because it very much differentiated us as something where we would measure engagement, we did something different.

We were about what people did in your app.

And then from there, we tried to explain what various, We tried to urge people to go down the website and look at different features and see if those things were valuable.

We tried to be benefits-oriented in the copy.

Tried to show customers and case studies who maybe found it valuable.

And we've probably made 4 changes after this.

But I thought I'd present this 1 just because it was the 1 that maybe lasted, like the the copy and the general essence of what we did and said we did, what we said we did, kind of stayed the same, largely, but lots of design changes.

So the real question is, you know, great, so like, how do I measure, how did you measure whether people really understood, whether they understood what you were doing?

And so I think there's like 1 tried and true number.

It's like very simple, it's just, did people even bother to sign up?

Bothering to sign up doesn't mean that they're going to use your product.

Bothering to sign up is like, I don't know, I'm kind of interested, I'll kick the tires.

This seems kind of interesting and cool to me.

And that'll tell you a lot about whether what you're doing is even something valuable for people.

The second thing that you can try, these are just ideas, trying to find the ratio of people that are going from just like hitting your page, right, And then doing any kind of thing after that.

So instead of S-A-K-A, instead of hitting the back button.

Like did they click on segmentation, or funnels, or retention in that case?

I know there are a lot of B2B companies these days.

How many people are making a B2B company?

1 thing that I discovered was that people, a lot of your users, will click on a pricing page link at the top.

It's like second or third most clicked on link if you're a B2B company, I think, generally speaking.

I think this is totally contestable, but I find that people clicking on your pricing page is actually a pretty decent indicator of, okay, cool, you have something kind of valuable.

And so even something like that, I think, can tell you a lot.

And then if you're in consumer, the benefit of being in consumer is that you generally have, you usually will have like hundreds or thousands of users such that you could actually go do an A-B test.

Whereas in B2B, it's a lot harder to achieve that.

And so I really think Gustav is absolutely right that you should try to conduct as many experiments as you can and really A-B test your copywriting and find out what resonates with people.

But these are all things that you can do to basically figure out, is this easy?

So the second 1 is, well, is it easy to get started with my product?

And the thing about this is like a lot of people, the thing about these first 2 things is that if you don't make the first step easy, if it's easy to understand, A lot of people put so much energy, when we're trying to build a company, we try to think about all the cool features and things that we could make that would be really valuable for customers.

The problem with that is if you don't even get step 1 right, you don't even have a chance at any of the subsequent steps.

So when we think about is it easy to get started with my product, forgot about password is kind of a useless feature if you have a very small percentage of people that are basically not even bothering to get started.

In fact, I remember there was a YC company called Zenter, which turned into Google Slides.

And I remember Robbie Walker, during 1 of our Tuesday dinners, said that they didn't even implement the save button for their online PowerPoint, because they knew that none of their users would ever ask, like, hey, can I save this?

They just didn't even bother to do it, because they wanted to see would anyone even use the PowerPoint presentation product.

And when they imparted that on me, I was kind of like, wow, that's crazy.

And for, I don't know, maybe like 18 months of Mixpanel, we didn't even have forgot about password, because we were just so ruthlessly trying to prioritize things that really would really matter to customers.

And it turned out we didn't get very many requests anyway.

So this is really important, And here's an example of easy to use, something you use every single day.

You just go to Google, you type it in, and you immediately get to use the product.

A more complex example is Airbnb, who might have 10 steps or something like that.

We worked with Airbnb for a while, helping them optimize initial user experience.

And even though Airbnb probably has like 11 pages of, you have to go to Airbnb and then you have to fill out this form and then you get taken to the results and then you click on those results and then you click on many different results because you're trying to price compare And you're going through this very complex checkout process.

So there are actually many steps to booking something online.

But Airbnb had to kind of figure out, how do we A, describe what we're doing, what we offer, what's useful for the world, and B, how do we help people get started as quickly and as easily as possible?

And they have this very simple, book unique homes and experiences.

You fill out these very basic things and you hit search and that's it.

Not their entire flow, but it's the first way to get started.

And they even try to kind of impute something like what you might be able to find.

But the thing is is that this seems simple, right?

This seems like something that you could just easily make.

It doesn't take, how hard is that, right?

You could do that with Twitter Bootstrap in no time.

The problem, the thing that's really hard is figuring out that That's the right thing.

That takes like, they probably changed that 50 times before they figured out this is the right current optimal thing at this stage for the company.

And so it takes a lot of work, and you have to grind very hard to get to this point.

So the question is like, okay great, so how do you, how would you measure something like that?

Well I think the really simple answer is a funnel.

How many people go from a landing page to signing up to doing that valuable thing like watching a video, doing in Airbnb's case it would have been just doing a search.

Once you get past that, measuring your funnel for the entire initial user experience.

There's weird things where there's actually situations, in games, games can have 20 steps, and I've seen hundreds of gaming companies' funnels, and even though they might have like 20 or 30 steps, sometimes they actually convert very highly.

They might get like 80, 90% conversion rate through all those 20 steps.

Just the first 2 are actually the ones that have the greatest drop off.

And then the last thing is just actually speed.

This is 1 that I think a lot of people don't actually do.

Which is How fast can someone just get started?

And that will tell you a lot about how complicated your experience is.

Does it take them like 5 minutes to get something figured out?

In Mixpanel's case, we cared a lot about like, we had a complicated flow.

Because not only did you have to sign up, but you had to be able to write code.

And that was the 1 downside, we weren't really sure where people with Google Analytics, you just copy and paste the JavaScript.

With Mixpanel, you had to like actually go and track a line of code.

And so we had this question in our mind, was that easy enough?

And so speed became really important to us.

And I wanted to give you a couple Tips and tricks, things that I have seen that companies do that are generally not great.

You probably don't have a lot of people spamming your service.

You probably don't have a lot of fake actors in the system trying to sign up.

So if you don't have those problems, don't optimize for them until you do.

And I just want you to know that generally, email and text confirmations have an enormous drop off.

Like, I'm talking like, you get of the 100% of the people that come, like only 30% will basically go on and click on those email confirmations, or 40%.

You'll spend a lot of time trying to optimize that.

And then to make matters even worse, you have to hope to God that it doesn't go to spam.

It's really important to iterate on your initial user experience.

You will do this forever, especially as the product changes.

And it becomes really critical to have someone on it, at least 1 responsible person, just thinking about that all the time.

And then the last thing is not every company, not every product is capable of doing this.

But to the extent that you can just let users into the product, it's always better to do that.

See all Y Combinator transcripts on Youtube

Suhail Doshi - How to Measure Your Product