See all Lex Fridman transcripts on Youtube

Speaker 1

00:01

What do you think it takes to build a system with human level intelligence? You talked about the AI system in the movie, Her being way out of reach, our current reach. This might be outdated as well, but-

Speaker 2

00:13

It's still way out of reach.

Speaker 1

00:14

It's still way out of reach. What would it take to build horror? Do you think?

Speaker 2

00:20

So I can tell you the first 2 obstacles that we have to clear, but I don't know how many obstacles there are after this. So the image I usually use is that there is a bunch of mountains that we have to climb and we can see the first 1, but we don't know if there are 50 mountains behind it or not. This might be a good metaphor for why AI researchers in the past have been overly optimistic about the result of AI.

Speaker 2

00:45

For example, Newell and Simon wrote the general problem solver. And they called it the general problem solver. And of course, the first thing you realize is that all the problems you want to solve are exponential. And so you can't actually use it for anything useful.

Speaker 2

01:00

But you know.

Speaker 1

01:01

Yeah, so yeah, all you see is the first peak. So, what are the first couple of peaks for her?

Speaker 2

01:06

So the first peak, which is precisely what I'm working on, is self-supervised learning. How do we get machines to learn models of the world by observation, kind of like babies and like young animals? So we've been working with, you know, cognitive scientists.

Speaker 2

01:24

So this Emmanuel Dupou, who is at FAIR in Paris, is, half-time, is also a researcher in French University, and he has this chart that shows at which, how many months of life baby humans can learn different concepts. And you can measure this in various ways. So things like distinguishing animate objects from inanimate objects, you can tell the difference at age 2, 3 months. Whether an object is going to stay stable, is going to fall, about 4 months, you can tell.

Speaker 2

02:04

You know, there are various things like this. And then things like gravity, the fact that objects are not supposed to float in the air but are supposed to fall, you learn this around the age of 8 or 9 months. If you look at a lot of, you know, eight-month-old babies, you give them a bunch of toys on their high chair, first thing they do is they throw them on the ground and they look at them. It's because, you know, they're learning about, actively learning about gravity.

Speaker 1

02:27

Gravity, yeah.

Speaker 2

02:28

Okay. So, they're not trying to annoy you, but they, you know, they need to do the experiment, right? Yeah. So, how do we get machines to learn like babies, mostly by observation with a little bit of interaction and learning those models of the world?

Speaker 2

02:42

Because I think that's really a crucial piece of an intelligent autonomous system. So, if you think about the architecture of an intelligent autonomous system, it needs to have a predictive model of the world. So something that says, here is a world at time T, here is a state of the world at time T plus 1 if I take this action. And it's not a single answer, it can be a

Speaker 1

03:00

bunch of- Yeah, it can be a distribution, yeah.

Speaker 2

03:02

Yeah, Well, but we don't know how to represent distributions in high dimensional continuous spaces. So it's going to be something weaker than that. Okay.

Speaker 2

03:08

But with some, some representation of uncertainty. If you have that, then you can do what optimal control theorists call model predictive control, which means that you can run your model with a hypothesis for a sequence of action and then see the result. Now, what you need, the other thing you need is some sort of objective that you want to optimize. Am I reaching the goal of grabbing this object?

Speaker 2

03:30

Am I minimizing energy? Am I whatever, right? So there is some sort of objectives that you have to minimize. And so in your head, if you have this model, you can figure out the sequence of action that will optimize your objective.

Speaker 2

03:42

That objective is something that ultimately is rooted in your basal ganglia, at least in the human brain, that's what it is. Basal ganglia computes your level of contentment or miscontentment. I don't know if that's a word. Unhappiness.

Speaker 2

03:56

Okay.

Speaker 1

03:56

Yeah. Discontentment. Discontentment.

Speaker 2

03:59

And So your entire behavior is driven towards kind of minimizing that objective, which is maximizing your contentment computed by your basal ganglia. And what you have is an objective function, which is basically a predictor of what your basic idea is going to tell you. So you're not going to put your hand on fire because you know it's going to burn and you're going to get hurt.

Speaker 2

04:24

And you're predicting this because of your model of the world and your predictor of this objective. So if you have those 3 components, 4 components, you have the hardwired contentment objective computer, if you want, calculator, And then you have the 3 components. 1 is the objective predictor, which basically predicts your level of contentment. 1 is the model of the world.

Speaker 2

04:53

And there's a third module I didn't mention, which is the module that will figure out the best course of action to optimize an objective given your model. Okay?

Speaker 1

05:04

Yeah.

Speaker 2

05:05

Cool. It's a policy network or something like that, right? Now, you need those 3 components to act autonomously, intelligently, And you can be stupid in 3 different ways. You can be stupid because your model of the world is wrong.

Speaker 2

05:20

You can be stupid because your objective is not aligned with what you actually want to achieve. Okay? In humans, that would be a psychopath. And then the third way you can be stupid is that you have the right model,

Speaker 1

05:45

you you