See all Y Combinator transcripts on Youtube

Speaker 1

00:00

Today's guest speaker is Emmett Shear. Emmett is the CEO of Twitch, which was acquired by Amazon, where he now works. And Emmett is going to do a new format of class today. And talk about how to do great user interviews.

Speaker 1

00:13

So this is the talking to users part of starting a startup. Should be really useful. Thank you very much for coming.

Speaker 2

00:18

Thanks, Sam. Context, everyone knows where I'm coming from, from this. We started our, I started my first startup with Justin Kahn right out of college.

Speaker 2

00:32

We started this company called Kiko Calendar. It didn't go so well. It went, it went all right. We, we, we built it.

Speaker 2

00:39

We we sold it, but we sold it on eBay. So that's not necessarily the end you want for your startup. And it was it was a good time. We learned a lot.

Speaker 2

00:48

We learned a lot about programming. We didn't know anything about calendars. Neither of us were users of calendars, nor did we, during the period of time we worked on Kiko, talk to anyone who actually did use a calendar. So that was not optimal.

Speaker 2

01:01

We got the build stuff part of the startup down. We did not get the talk to users part. The second startup we started, we used a very common trick that lets you get away with not talking to users, which is that we were our own consumer. We we had this idea for a television show, Justin TV, a reality show about Justin Conn's life.

Speaker 2

01:20

And we built a whole set of technology and website around the reality show we wanted to run. And so we were the user for that for that product. And that's actually 1 way to cheat and get away with not talking to many other users is, if you're just building something that literally is just for you, you don't need to talk to anyone else, because you know what it is you want and what you need. But that's actually a really limiting way to start a startup.

Speaker 2

01:46

Most startups are not just built for the person who is who is using them. And when you do that, every now and then you get really lucky and you are representative of some huge class of people who all want the exact same thing you do. But very often also that just turns into a side project that doesn't go anywhere. So we kept working on Justin.TV for a while, and we actually achieved a good deal of success, because it turned out that there were people out there who wanted to do the same thing we did, which was broadcast ourselves live on the Internet.

Speaker 2

02:12

But the issue with Justin.TV, the thing that, the thing that sort of kept us from achieving greatness is we hadn't figured out yet how to how to build towards anything beyond that initial TV show. We knew how to, we built a great product actually. If you wanted to run a live 24-7 reality television show about your life, we had the website for you. We had exactly what you needed.

Speaker 2

02:34

But if we wanted to go do more than that, if we wanted to open it up to a broader spectrum of people, a broader spectrum of use cases, we didn't have we didn't have the insight to figure that out because we weren't that user. And so at some point we decided to pivot Justin TV. We decided we needed to go in a new direction. We thought we'd built a lot of valuable technology, but we hadn't identified the use case that would let it get really big.

Speaker 2

02:58

And there were 2 directions that seemed promising. 1 of them was mobile, and 1 of them was gaming. And I led the gaming initiative inside of the company. And what we did with gaming that was very, very different from what we'd ever done before was, we actually went and talked to users.

Speaker 2

03:13

Because while I loved watching gaming video, I was very aware that neither I nor anyone else in the company knew anything about broadcasting video games. And so I was amped about the content, I thought there was a market there, that was sort of the insight that that I had that I think wasn't common at the time, which is how much fun it was to watch video games. Quick show of hands, people know about watching video games on the Internet here? Okay, I'm just going to assume that people listening to this also know about it.

Speaker 2

03:38

If you don't know about watching video games on the Internet, you should go read about that. Because it's sort of important context for the stuff I'm going to talk about. But the main point is I thought that was awesome but I didn't know anything about the side of it that was really important, which is actually acquiring the content to start broadcasting. So we went out and We ran a, actually a very large number of user interviews.

Speaker 2

04:04

We talked to a lot of people and brought that data back. And that formed the core of all the decision making that was for the next 3 years of product features on Twitch. Was sort of, I mean, the insights we got from that. And we continued to talk to users and in fact built an entire part of the company whose job it is basically to talk to our users.

Speaker 2

04:24

Which is a whole division that we just didn't even have at Justin TV. We had no 1 at the company whose job it was to talk to our most important users. So, so that was Twitch. And I'm gonna, I wanna give you guys a little bit of a, a little bit of an insight into with Twitch, what what that, what that meant going to go talk to users.

Speaker 2

04:46

So we determined that the broadcasters were the most important people. And the reason we determined that was when we went and looked into the market, we looked into what, what determined why people watched a certain streamer, or went to a certain website. They would just follow the content, right? You had a, you had a piece of content you loved and the broadcaster would come with you.

Speaker 2

05:08

And that's actually the 1 really important point about user interviews, which is that who you talk to is as important as what questions you ask and what you pull away from it. Because if you go and talk to a set of users, if we'd gone and talked to viewers only, we would have gotten a completely different set of feedback than talking to the broadcasters. And talking to the broadcasters gave us insight into how to build something for them. And that turned out to be strategically correct.

Speaker 2

05:32

I wish I could tell you the recipe for figuring out who the target user is for your product and who your target user should be, but there isn't a recipe. It comes down to think really hard and, and use your, use your judgment to figure out who you're really building this for. So what I want to do is a little bit, something a little bit interactive now. Which is, we're, I'm gonna, I got a bunch of ideas from, from you guys actually, so sort of suggested ideas.

Speaker 2

05:59

And I'm, I'm gonna pick 1 of them. And I want everyone to sort of sit down and do, do step 1 of this process for me right now. Which is, think about, who would you go ask about this? Like, which people?

Speaker 2

06:11

Where would you go to find the people you need to talk to about this in order to in order to learn about what you should build. And so the idea we're going to use is, let me see here, of these ideas. So here, it's a lecture focused note taking app. The idea is, I don't think that the state of the art for note taking is good enough yet.

Speaker 2

06:29

And I want to make a note taking app that improves that experience. Makes taking notes in class better. Or taking notes while listening to a lecture online better. So, you know, maybe it has collaboration features.

Speaker 2

06:42

Maybe it like helps you focus better somehow. It has multimedia enhancements. I don't know, right? All sorts of possible features.

Speaker 2

06:49

But that's the, that's the idea. So, take like, take 120 seconds right now and think about not what you would ask or what the right features for this app is, but who would you talk to? Who should the, who, Who's gonna give you that feedback that's gonna tell you whether this is good or not? I actually mean it right now.

Speaker 2

07:06

Take your laptop out, like type, write some stuff down. Think, think about, like, the, you can, it's, it's good enough to, like, think that in your head. But actually, like, if you actually just write it down and like just come up with the 5 people you'd talk to, the 5 types of people you'd talk to. And who you think the most important 1 was.

Speaker 2

07:24

Like actually do it. Because there's nothing like actually running through a practice of something and trying to do it to actually get it into your head the right way to do it. I'm gratified to hear clicking in the, of keyboards now. If you're following along at home, pause, actually do it, think about who you, who would you talk to?

Speaker 2

07:42

Because that's a, That is the first question for almost any startup that you need to answer is like who is my user and, and where am I gonna find them? All right. That's like way shorter than you'd normally use to think about this problem. It's actually a really tricky problem and like figuring out where to source the people is pretty hard.

Speaker 2

08:28

But we're going to move along anyways in the, in this highly abbreviated version of learning how to build a product and run a user interview. So can can I, can I get 1 volunteer from the audience to come up and tell, tell us what who you would talk to and we'll talk about it? You guys are all pre-selected. Here you go.

Speaker 2

08:49

I don't know how to turn this thing on. Here we go. So who do you talk to? Yeah.

Speaker 3

08:57

I would definitely talk to college students First, obviously, because we sit in a lot of lectures. And specifically, I want to talk to college students studying different subjects to see if maybe, you know, if you're an English major, if that makes a difference versus your studying math or computer science in terms of how you want to take notes in

Speaker 2

09:15

different lectures. And so you, so you're gonna talk to, talk to a bunch of college students. Would you pick any particular subset of college students?

Speaker 2

09:23

Like, sounds like you really, we're gonna talk to all college students, or like a broad array?

Speaker 3

09:27

I wanna talk to college students like, And break down the divisions by like people who study different areas maybe. And then also maybe it would make sense for people who have like different study techniques. Cuz some people take a lot of notes, some people don't take that many notes, but still jot stuff down.

Speaker 2

09:42

Right. So I mean that, that's a really good start. Like that's, that is actually obviously a group of users you want to go talk to, especially if you're targeting something at, you know, at college students as the consumer. And if you're talking to college students as a consumer, the, you're going to get a lot out of students about what their current note taking habits are and, you know, what they would be excited about.

Speaker 2

10:02

1 of the problems of selling things to college students is that college students don't actually spend very much money. It's, it's really hard to get you guys to open your wallets, especially if you want them to pay for a school related thing. I mean, people don't even want to buy textbooks, right? I think you probably, probably all use Chegg or that, you know, borrow it from your friend or whatever.

Speaker 2

10:20

And so, 1 of the things that I think you'd be missing if you go after just the students, right, is you want to figure out Who is the most important person to this, to this app? And if you actually had a note taking app, my guess is for colleges, the people most likely to actually buy a note taking app that you guys would use would be college IT, right? I mean, presumably for most, for the most part, if you want to sell software to students, like the people who have to get bought into that, is usually the school administrators. So that would be not, that would be 1 approach.

Speaker 2

10:50

If you thought, okay, well, you presumably go talk to the college students and you find out they don't actually buy any note taking software right now at all. I mean, likely. It's possible they do. In which case I'm, I'm completely wrong.

Speaker 2

11:01

This is why you actually have to go talk to the users. But you then have to try to maybe try other, other groups, right? So I would talk to college, I would talk to IT administrators as well. I think that's a, that's another area that's really promising.

Speaker 2

11:13

You might look, talk to parents, Right? Who, who, who spends money on their kids' education and is like willing to pull their wallet out? Like the, you know, parents of kids. Parents of kids who are freshmen or going off to college for the first time.

Speaker 2

11:24

You need this app to make your kid productive so that they don't fail out of college. And, and there's actually a lot of groups that are potential, that aren't necessarily the obvious user, but who are critical, critical to your app's success, potentially. And you, when you're, when you're, at the very beginning of a startup like this, when you're like, you have this idea that you think is awesome. You want to have that broadest group you possibly can.

Speaker 2

11:46

You don't just want to talk to 1 type of person and, and, and learn that. You want to get familiar with the space. You want to get familiar with the various kinds of people who could be contributing. All right.

Speaker 2

12:00

So let's let, let's have someone come up and we're gonna pretend we're gonna run this user interview. So we're gonna talk to a college student and try to find out what we should build. You know, what we should get into this note taking app. So, so some, another volunteer please for, for running an interview.

Speaker 2

12:19

Yes. All right, so hello.

Speaker 4

12:23

Hi, I'm Stephanie.

Speaker 2

12:24

Hi Stephanie. Nice to meet you. Welcome, thank you for agreeing to do this user interview with us.

Speaker 2

12:28

So I wanted to hear from you about, you know, what are your note taking habits? Like, how do you take notes today?

Speaker 4

12:35

Sure. So I take notes in a variety of ways. I like to now because of speed and efficiency and just to come back to it later, it's easy for me to just take notes on my laptop. And so a lot of those notes will be primarily text based.

Speaker 4

12:48

But in certain classes, so for example, if I'm taking a history class, most of it will be in text. But if I'm taking a physics class, for example, there are gonna be more complex diagrams, different angles that I have to draw. And so that's a little harder for me to get the.

Speaker 2

13:02

What software do you use for this stuff today?

Speaker 4

13:04

I just do pen and paper for that.

Speaker 2

13:06

You do pen and paper. So you do a combination. You take notes with pen and paper.

Speaker 2

13:08

You take notes with your computer sometimes. Yeah. And when you take notes with, When you take all these notes at the end, do you actually review them? Do you, be honest, do you actually go back and ever actually look at these notes?

Speaker 4

13:20

The pen and paper, not so much. But yes to the software base, because it's more easy to access and it's easier for me to share and collaborate and maybe even merge notes with other classmates and friends.

Speaker 2

13:33

So what do you, what do you stick notes today on your computer?

Speaker 4

13:37

Google Docs and Evernote.

Speaker 2

13:39

Google Docs and Evernote. And tell me more about like why 2 things at the same time.

Speaker 4

13:46

So Evernote is easy if I'm trying to just collect it for myself, I think. And yes, you can share, but I think Google Docs for me is easier to share. And it depends also if, you know, a friend has already created a folder, for example, in Google Docs, and I just have to add to that folder if it's a group project, for example, versus if it's for my personal use, I, I tend to go more towards that.

Speaker 2

14:06

So it sounds like you do a lot of like, note taking collaboration.

Speaker 5

14:08

Yeah.

Speaker 2

14:09

I wish

Speaker 4

14:09

it was integrated.

Speaker 2

14:10

What tell me more about that, like, like, do you take a, do you wind up taking most of the notes, most of the value of the notes out of notes other people take? Or is it mostly your own notes you review at the end of the semester? How does that work?

Speaker 4

14:24

It's mostly mine because I'm pretty picky about the way I like things organized. Like design wise or formatting even color. I'm really particular with and like the font that we use, and that really affects the way I study.

Speaker 4

14:38

So I tend to like it, to like, to like to personalize it, even after I merge.

Speaker 2

14:43

So you, you, you pull in notes from other people, But then you merge them into, into the main. What

Speaker 4

14:47

works for me.

Speaker 2

14:48

Right. Awesome. And if you if you have Evernote notes and you have Google Docs notes and you have pen and paper notes. Once the semester's over, do you ever go back to any of that stuff?

Speaker 2

15:02

Or is it like, or quarter, you guys are in quarters here, right? Once the quarter is over, do you ever go back to any of that stuff? Do

Speaker 4

15:09

you ever? For classes, not so much. But if it's notes that I've taken for talks like these, for example, or if it's interview prep that I'm doing, I tend to go back because it's things that I like to kind of keep fresh in my mind and to help me prep for future things.

Speaker 2

15:23

So that's interesting. Tell me more about that. Like you take notes not just in class.

Speaker 4

15:27

Yeah. So I take notes to also just summarize main points. So if it's like inspirational quotes, for example, from talks that I go to like these. And then like maybe I'm going to an event where I'm actually going to meet someone.

Speaker 4

15:38

And it, it helps to actually to think about and to remember and recall what was shared at the time that, you know, I attended the talk or something.

Speaker 2

15:47

Awesome. All right, well, normally I'd actually dig into a lot more detail. There's a huge amount of like, open questions that are still in my mind after hearing that stuff. Questions about which people do you collaborate with?

Speaker 2

15:57

Questions about whether or not you, like, like what the volume of notes are and like how long of note taking stuff. And just sort of digging into like what the current behavior is. But like in the interest of time and not like keeping everyone here hearing about the intricacies of 1 person's note taking habits forever we're going to move on. But thank you very much Stephanie.

Speaker 2

16:15

I appreciate that. So that's like, that kind of stuff. You notice we're not talking about the actual content of the app at all. Like I'm not, I'm not, I'm not really interested in features.

Speaker 2

16:27

I don't really want to know about what they, the specific feature set in Google Docs or Evernote, I might start digging in a little bit more into which features actually get used. Like, if she's actively collaborating, you know, how does that work? I heard some interesting things about, oh, we have, we use folders. That's interesting to me.

Speaker 2

16:43

But the main thing you're trying to do when you're running these first set of interviews is not necessarily get, like, questions about, like, user flows and, like, optimizing that, or questions about, like, the specifics of, of, of any of that stuff kind of can be distracting. Because users think they know what they want. But like, you, you get the you get the horseless carriage effect where you're, you're you're, you're getting asked for a faster horse instead of trying to design the actual real solutions to the problem. If you start asking people about features.

Speaker 2

17:16

You want to stay as far away from features as possible. Because the things they tell you wind up almost feeling overwhelmingly real. When you have a real user asking you for a feature, it's almost, it's very hard to say no to them. Because here's a real person who really has this problem, and they're saying build me this feature.

Speaker 2

17:31

But as you start to talk to lots of people and really get a sense for what their problems are, you figure out if this is actually a promising area or not. And like, based on what I heard there, it's like starting from that user interview, I'm not necessarily positive there is a problem. Or there's at least, there's a big enough problem that it's worth building a whole new product for. Because I didn't hear a lot of like, things where I'm, where I, where there was a, a big blocker, where there's something really wrong with the way, the way it it was working.

Speaker 2

18:01

And unless I had some big idea I would take that as a, you know, maybe a negative sign. But it doesn't necessarily mean that you can't, you can't move forward and keep talking to more people. Because just because you talk to the first person and you don't get anything out of it doesn't mean there's not going to be a ton more people who actually have a problem. And you, once you've talked to about 678 people you're usually about done.

Speaker 2

18:23

It's unlikely you're going to discover a bunch of new information there. Which is why it's important to talk to different extremes of people, right? Go, go find people who are at different, different points because if you talk to 6 or 7 Stanford college students, you're going to get a very different response than if you talk to 6 or 7 high school students or 6 or 7, 6 or 7 parents. All right, once again, look at the.

Speaker 2

18:49

So based on that though, right? I think the, I think it's possible you could come up with a set of ideas, right? You have this information about how someone takes notes. You've, you've come up with potentially when you came up with this idea you had, you had some ideas as to, when you heard this idea you had some ideas as to like, how you could build something cool.

Speaker 2

19:13

And so, if you're gonna build just 1 feature on top of Google Docs, what would that feature be, right? And that's, for, for, for a new product like this, it might be a good way to like get started thinking about where to go, which is okay, so they're extensively using this thing right now. How can we make that experience just 1 quantum better, something that would be really exciting to this person, to be 1 1 step ahead. And so, I want you to take 2 minutes right now and think about what that feature might be.

Speaker 2

19:44

Actually like, try to, try to come up with what what you might do based on what you heard from from Stephanie that could convince her to switch away from her current collaborative, multi-person, all working together workflow on Google Docs to your new, your new thing that has all the features of Google Docs plus this 1 special thing that's like going to make it that's going to make it more more useful and convince them to stop using the thing they're already using. Awesome. All right. So I'm gonna invite our, our third guest.

Speaker 2

21:04

If you, if you, if you have something up. I, I don't want to put you on the spot if you feel like you, you're not sure, but yeah?

Speaker 6

21:12

So what I, what I, is it on?

Speaker 7

21:14

Yes.

Speaker 6

21:14

Okay. What I thought about was like, the, the reason she uses Evernote is because like, of like sticky note type notes, like, like more thoughts and like details. So I feel like Google Docs has like documents and not like smaller notes. So I feel like a feature that would be like super, like a mobile version of Drive that doesn't like, isn't that clunky and like doesn't make you make real documents could be like really useful.

Speaker 2

21:40

Awesome. So right, that's a, that's a good insight, right? That's exactly, that's 1 of the things you get out of that user interview. And now you have this idea, right?

Speaker 2

21:50

You've gotten this user feedback, you've got this idea. What if we had a Google Docs that had the collaborative aspects and the group aspects of that, but where you, you could pull in more little 1 off notes and it was, it was designed more around note taking. And so the question is now once you have this idea, which I think it's, it's a actually a really reasonable approach, is this enough? Is this something people would actually switch just to have?

Speaker 2

22:17

And the way to validate, there's 2 ways to validate that. 1 is if you're quick at programming, you can literally just go build it and throw it out in the world and see what happens. And that's that's great. And if that, when that works, that's that's an excellent way to approach it.

Speaker 2

22:31

But a lot of the time, that 1 little thing that's just a little bit better might take you 3 months to actually build something that's worthy of actually using. And so, you actually want to go out and validate that idea further before you go ahead and start building it. And so you might take that idea and you might go back go back out. And you know, you can sit down with with diagrams, you can, you can draw what the what it looks like.

Speaker 2

23:00

Draw the workflow and go bring that in front of people. But the 1 thing you really don't want to do is ask them. This is, this is just a sort of a trap I want to warn you against doing is just don't go out and say, if you come up with a feature idea and go out and ask people, are you, you know, I got this idea for a feature, are you excited about it? Because the, the feedback you get from users if you tell them about a feature and ask them is this feature good?

Speaker 2

23:23

Is often, oh yeah, that's great. Like that sounds like such a good idea. But when you actually take that in front of people and you actually build it, You then find out that while they thought it was such a clever idea, no 1 actually like cares to switch to get it. And so, the 1 question you can't ask is, is this feature actually good or not?

Speaker 2

23:42

Yes, Sam? What is

Speaker 1

23:43

the minimum that you can do in your experience to actually get a build in on that question. And asking, you know, between asking and actually building the whole thing.

Speaker 2

23:51

Yeah, so Sam's asking if, what's the, what's the minimum you can actually get away with to validate, given that you can't actually just go and ask them, is this good or not? And it's, it's highly dependent the answer to that is highly dependent on the particular feature. But usually the best thing you can do is really just hack something together, right?

Speaker 2

24:10

It's you find, if your idea is to build something on top of Google Docs, don't for your V1 Go rebuild an awesome Google documents but for note taking application. Find a way to write a browser extension that, that, that stuffs just that little bit of incremental feature in. And, and see if it's actually useful for people. Go like, go, go, find a way to cheat, is what it comes down to.

Speaker 2

24:35

Because if you can't actually put it in front of people, it's really, really hard to to find that out. For bigger things, we're actually trying to get people to spend money. It actually gets a lot easier. So if you're selling it, it's great actually.

Speaker 2

24:48

Sales is this cure all for this problem. Get people to put, give you their credit card, and I guarantee you they're actually interested in the future. It's it's 1 of the most validating things you can do for a product is go out there and actually get them to commit to pay you up front. And the problem is, when you're working on a student note taking app, that's gonna be relatively hard.

Speaker 2

25:05

Because you probably, unless your idea is that you're actually gonna sell it. It's probably something where you're thinking, at least the, the trial version's free. And you're not necessarily gonna learn that much by trying to charge people money. But if you go out there and you can, if you can get people to say, hey, I'm gonna, I'm gonna give you money, the money test is amazing.

Speaker 2

25:21

It really does clarify whether or not they're really excited about it or not, because if you're not $5 excited about it, you're probably not very excited about it. So, the last thing I wanted to do is actually work through with you guys what happened at Twitch. So I, I brought some slides of feedback that I'd like to get put up. That's my, my only slides for the for the thing.

Speaker 2

25:44

And it's, it's, what it is, it's, it's, it's representative excerpts of Twitch feedback. I had a whole like 26 page document full of all the feedback and then I realized that reading that was going to be a little bit tedious and there was no way I'd make it through it in a lecture. So, pretend that like this is, stuff is all representative of like lots of people said this kind of thing out to, to us when we asked them questions. And I've already pre-condensed it for you into the real feedback you got.

Speaker 2

26:10

So, when we were working on Twitch to go launch it, we we went and we talked to a bunch of existing Justin TV broadcasters and ask them about their experience broadcasting, what they liked about broadcasting, why they broadcasted, what they broadcasted, what else was going on in their life. And the interesting thing is when you talk to users of your product, who are, who are detail users of your product, they come back to you with actually very detail. Things about features, because they actually get mired in the feature. And you have to sort of read between the lines, but they ask for first things like, I want to be able to clear the ban list in my chat room.

Speaker 2

26:43

Like this, that was actually a very common request. Because there was a particular issue with how our chat rooms worked. People would ask for the ability to edit the titles of highlights after creating them. And, and it's, it's, this was like, this stuff was really consistent.

Speaker 2

26:55

As you talk to broadcasters, you probably talked to 12, 14, something like that, broadcasters on the Justin TV gaming platform. We got, we got all this feedback and, you know, what else do we have? We have, your, your competitors have all these cool features like polls and scrolling text. I can personalize chat there.

Speaker 2

27:15

And we have some positive feedback. They're like, oh, you guys don't have ads. That's great. I didn't feel the ban trolls.

Speaker 2

27:20

So a bunch of stuff about chat, a bunch of stuff around interactivity with, with interactivity with the with their viewers, and that was all really interesting, right? So this is what the, this is what the Justin TV broadcasters wanted us to build. And this is what they, what, what, where they felt pain using the, using the product. And so, if you thought that what we did was go and address these problems, you would be wrong.

Speaker 2

27:47

Because actually, people who are using your service already and are willing to put up with all these issues, kind of, kind of means that these are probably not actually the biggest problems. Because if you're willing to ignore the fact that you can't edit the ban lists and titles are uneditable and there's no way to get trolls out of your channel. And you're using the service anyways. Maybe those aren't huge problems.

Speaker 2

28:05

And so, that sort of brings up another really important point, which is you have to compare you have to actually compare groups of people and compare the level at which they they argue with each other. So if you go to the next slide Yes, nice. We got competitor broadcaster feedback, which is really interesting. So, this is stuff that you, we heard a lot from people who are using other broadcast platforms.

Speaker 2

28:29

They wanted to be able to switch multiple people onto their channel at the same time. They, they complained about us not having a rev share program where they talked a lot about how they were trying to make a living. They really wanted to make money pursuing this pursuing this gaming broadcasting thing. They talked a lot about video stability.

Speaker 2

28:48

Our service wasn't good in Europe, specifically. But, but just globally, video stability was this huge, huge issue for them. And if you compare and contrast, actually, it was really different. Like The things that people who didn't use our service said about what they cared about was completely different than the things that people who were using the service cared about.

Speaker 2

29:07

And we focused on this stuff because this was the stuff where it was so bad they weren't even willing to use our service because of it. And most of them actually had thought about it, because we were, our user base happened to be a very well educated user base in this, in the area. Who knew about all their options for, for this. And they would, they actually, you know, reaching out to them meant that they, they probably already tried all 4 services and actually had an opinion.

Speaker 2

29:30

It's great when you can get users who are that, that informed and that, that they understand the space that well. And if you go to the, I'm just gonna go to the next slide. Yeah, here we go. The other big thing we did that I thought was really important was we talked to non-broadcasters.

Speaker 2

29:48

So, we went out there and we talked to all the people who weren't using us or competitors. And in many ways, those are the most important people, right? Talking to your competitors, that's a short term win, right? Someone's using a competing, piece of competing software.

Speaker 2

30:00

Unless your, your piece of competing software is something like Google, which is a search engine which everyone uses. Okay, maybe then there are no non-users to convert. But in the case of gaming broadcasting, almost everyone's a non-user, right? The majority, and this is true for most new products.

Speaker 2

30:15

The majority of people you're competing with are non-users. They're people who have never used your service before. And what they say is actually the most important. What they say is, is the thing that blocks you from expanding, from expanding the size of the market with your features, Right?

Speaker 2

30:30

If you, all you do is look at your competitors and yourself, and all you do is talk to your, your, you know, people who use your competitors' products, people who use your products. You can never expand, well not never, but you're not learning the things that help you expand the size of the market. You want to talk to the people who aren't even trying to use 1 of these things yet. Who, who've thought about it maybe, but who aren't, who aren't into it.

Speaker 2

30:48

So what did they say? My computer isn't fast enough. I'm focused on training 12 hours a day for the next tournament. I like making the perfect video and like editing it.

Speaker 2

30:58

And so I just upload things to YouTube. I don't do live streaming. I don't, III have no desire to, to go into that space. Or or this is actually, particularly in Korea, this is a big problem.

Speaker 2

31:09

Once our strategy gets broadcast in a major tournament, we have to start over. We have to like, come up with an entirely new strategy. And so the last thing we ever want to do would be broadcast our practice sessions, are you crazy? That's going to hurt us in the next big tournament.

Speaker 2

31:21

And so this became, this became a big outreach program for us trying to figure out how we can get people over this. We bought people computers. We we worked really closely with gaming broadcast software companies to help the people who made the broadcasting software to make that better. We started building broadcasting into games and into platforms.

Speaker 2

31:38

Like we built broadcasting into the Xbox. We built broadcasting into the PlayStation 4. Because we wanted, needed to overcome this issue that like it was too hard. Broadcasting wasn't wasn't possible.

Speaker 2

31:49

And so you sort of combine these, for us, these are the 3 big groups we looked at for broadcasting. And you combine that feedback and what it tells you is not the features to build, right? Because the features they asked for, things like polls, things like a you know, the, the ability to have child account, like child accounts on your account. We haven't built most of that stuff.

Speaker 2

32:14

But What was important were the, were the, the issues. Like the goals they were trying to accomplish there. People wanted money. People wanted stability and quality.

Speaker 2

32:23

People wanted universal access for viewers all around the world to be able to watch them. And so that became our focus actually. And we dumped almost all of our resources into things that none of, no 1 ever mentioned in an interview. But those are the things that actually address the problem.

Speaker 2

32:39

And the way you could tell that it worked is as we, we would build these things, then we'd go back to this exact same people we interviewed and we'd say, hey, you told us you really cared a lot about making money. Well, we've got you the subscription program that will let you make money. And it, it's, it's astonishing because most people aren't, have never had that experience actually. They've never talked to someone and said it would be really great if your product had feature x.

Speaker 2

33:02

And then, and then like 2 months later or a month later, your product actually has feature x. Or at the very least, a feature that addresses the problem that they brought up. And so it was actually, the, the people we converted first to our product were the people that we talked to about user research. They were the ones who were actually the most impressed, which is kind of fun.

Speaker 2

33:21

But it really worked, because we picked people who were representative, and we picked big broadcasters, small ones, medium ones. And we, we made sure we were addressing their concerns. And that, That was completely different from how we'd approach the problem on Justin.TV. Because in Justin.TV, when we tried to do this, we'd, we'd sat down, we, we trawled through huge amounts of data.

Speaker 2

33:38

Like we, we spent tons of time looking at Google Analytics, looking at Mixpanel, looking at in-house analytics tools. Figuring out how people use the service, looking at where our traffic came from. Completion rates on flows. We spent all this time doing that.

Speaker 2

33:51

And that, that's good. I mean, you can learn things from that. I'm not telling you not to look at your at your data. But it doesn't tell you where you need to go.

Speaker 2

34:01

It doesn't tell you where, what the problems are you need to address. And so we would just sort of invent these ideas in Justin.TV and then 9 times out of 10, without talking to someone, that idea turns out to be bad. And that's actually 1 of the most disappointing things about doing user interviews and user feedback, which is why I think so many people don't do it. Which is that you're going to get negative news about your, your favorite pet feature most of the time.

Speaker 2

34:23

Like you're going to have this great idea, and you're going to talk to a user, and it's going to turn out that that nobody actually wants that. Like no, no one's actually, they're actually completely concerned about completely different things and they don't care about what you thought was important at all. And and that's a little bit sad, but just, just think about how sad you'd be in 4 months when you launch that feature and it turns out no 1 actually wants to use it. So I think that's about it for my, the lecture section of what we're, we're talking about.