See all Lex Fridman transcripts on Youtube

Speaker 1

01:00:01

First, the win itself, I mean, it was so exciting. I mean, so looking back to those last days of 2018, really, that's when the games were played. I'm sure I look back at that moment, I say, oh, my God, I want to be in a project like that. It's like, I already feel the nostalgia of like, yeah, that was huge in terms of the energy and the team effort that went into it.

Speaker 1

01:00:26

And so in that sense, as soon as it happened, I already knew it was kind of, I was losing it a little bit. So it is almost like sad that it happened and oh my god, like, but on the other hand, it also verifies the approach. But to me also, there's so many challenges and interesting aspects of intelligence that even though we can train a neural network to play at the level of the best humans, there's still so many challenges. So for me, it's also like, well, this is really an amazing achievement.

Speaker 1

01:00:57

But I already was also thinking about next steps. I mean, as I said, these Asians play Protoss versus Protoss, but they should be able to play a different race much quicker, right? So that would be an amazing achievement. Some people call this meta reinforcement learning, meta learning, and so on, right?

Speaker 1

01:01:15

So there's so many possibilities after that moment. But the moment itself, it really felt great. We had this bet. So I'm kind of a pessimist in general.

Speaker 1

01:01:27

So I kind of sent an email to the team. I said, OK, let's against TLO first, right? Like, what's going to be the result? And I really thought we would lose like 5-0, right?

Speaker 1

01:01:39

I, we had some calibration made against the 5000 MMR player. TLO was much stronger than that player, even if he played Protoss, which is his off race. But yeah, I was not imagining we would win. So for me, that was just kind of a test run or something.

Speaker 1

01:01:55

And then it really kind of, he was really surprised. And unbelievably, we went to this to this bar to celebrate. And Dave tells me, well, why don't we invite someone who is a thousand MMR stronger in Protoss, like an actual Protoss player, like that it turned up being Mana, right? And we had some drinks and I said, sure, why not?

Speaker 1

01:02:19

But then I thought, well, that's really going to be impossible to beat. I mean, even because it's so much ahead. A thousand MMR is really like 99% probability that Mana would beat TLO as Protoss versus Protoss. Right.

Speaker 1

01:02:32

So we did that. And to me, the second game was much more important, even though a lot of uncertainty kind of disappeared after we beat TLO. I mean, he is a professional player, So that was kind of, oh, but that's really a very nice achievement. But Mana really was at the top.

Speaker 1

01:02:51

And you could see he played much better. But our agents got much better, too. So it's a... And then after the first game, I said, if we take a single game, at least we can say we beat a game.

Speaker 1

01:03:02

I mean, even if we don't beat the series, for me, that was a huge relief. And I mean, I remember the hugging dummies. And I mean, it was it was really like this moment for me will resonate forever as a researcher. And I mean, as a person, and yeah, it's a really like great accomplishment.

Speaker 1

01:03:18

And it was great also to be there with the team in the room. I don't know if you saw the video. So it was really like... I mean, from my

Speaker 2

01:03:25

perspective, the other interesting thing is just like watching Kasparov, watching Mana was also interesting because he is kind of at a loss of words. I mean, whenever you lose, I've done a lot of sports, you sometimes say excuses, you look for reasons, and he couldn't really come up with reasons. So with the off race for Protoss, you could say, well, it felt awkward, it wasn't, but here it was just beaten.

Speaker 2

01:03:55

And it was beautiful to look at a human being being superseded by an AI system. I mean, it's a beautiful moment for researchers. So yeah,

Speaker 1

01:04:04

for sure. It was. It was, I mean, probably the highlight of my career so far because of its uniqueness and coolness.

Speaker 1

01:04:11

And I don't know. I mean, it's obviously, as you said, you can look at paper citations and so on. But this really is like a testament of the whole machine learning approach and using games to advance technology. I mean, it really was everything came together at that moment.

Speaker 1

01:04:28

That's really the summary.

Speaker 2

01:04:29

Also, on the other side, it's a popularization of AI too, because just like traveling to the moon and so on. I mean, this is where a very large community of people that don't really know AI, they get to really interact with it.

Speaker 1

01:04:45

Which is very important. I mean, we must, you know, writing papers helps our peers, researchers to understand what we're doing. But I think AI is becoming mature enough that we must sort of try to explain what it is.

Speaker 1

01:04:58

And perhaps through games is an obvious way, because these games always had built-in AI. So it may be everyone experienced an AI playing a video game, even if they don't know, because there's always some scripted element and some people might even call that AI already, right?

Speaker 2

01:05:14

So what are other applications of the approaches underlying AlphaStar that you see happening? There's a lot of echoes of, you said, transformer of language modeling and so on. Have you already started thinking where the breakthroughs in AlphaStar get expanded to other applications?

Speaker 1

01:05:32

Right. So I thought about a few things for like kind of next month's next years. The main thing I'm thinking about actually is what's next as a kind of a grand challenge, because for me, like we've seen Atari and then there's like the sort of 3 dimensional worlds that we've seen also like pretty good performance from this capture the flag agents that also some people at DeepMind and elsewhere are working on. We've also seen some amazing results on like, for instance, Dota 2, which is also a very complicated game.

Speaker 1

01:06:03

So for me, like the main thing I'm thinking about is what's next in terms of challenge. So as a researcher, I see sort of 2 tensions between research and then applications or areas or domains where you apply them. So on the 1 hand, we've done, thanks to the application of StarCraft is very hard, we developed some techniques, some new research that now we could look at elsewhere. Like, are there other applications where we can apply these?

Speaker 1

01:06:30

And the obvious ones, absolutely, you can think of feeding back to sort of the community we took from, which was mostly sequence modeling or natural language processing. So we've developed and extended things from the transformer and we use pointer networks. We combine LSTM and transformers in interesting ways. So that's perhaps the kind of lowest hanging fruit of feeding back to now a different field of machine learning that's not playing video games.

Speaker 2

01:07:00

Let me go old school and jump to Mr. Alan Turing. So the Turing test, you know, it's a natural language test, a conversational test.

Speaker 2

01:07:11

What's your thought of it as a test for intelligence? Do you think it is a grand challenge that's worthy of undertaking? Maybe if it is, would you reformulate it or phrase it somehow differently?

Speaker 1

01:07:23

Right. So I really love the Turing test because I also like sequences and language understanding. And in fact, some of the early work we did in machine translation, we tried to apply to kind of a neural chatbot, which obviously would never pass the Turing test because it was very limited. But it is a very fascinating idea that you could really have an AI that would be indistinguishable from humans in terms of asking or conversing with it, right?

Speaker 1

01:07:56

So I think the test itself seems very nice and it's kind of well-defined actually, like the passing it or not. I think there's quite a few rules that feel like pretty simple. And, you know, you could really like have, I mean, I think they have these competitions every year.

Speaker 2

01:08:14

Yeah, so the Leibner Prize, but I don't know if you've seen, I don't know if you've seen the kind of bots that emerge from that competition. They're not quite as what you would, so it feels like that there's weaknesses with the way Turing formulated it. It needs to be, that the definition of a genuine, rich, fulfilling human conversation, it needs to be something else.

Speaker 2

01:08:41

Like the Alexa Prize, which I'm not as well familiar with, has tried to define that more, I think, by saying you have to continue keeping a conversation for 30 minutes, something like that. So basically forcing the agent not to just fool but to have an engaging conversation kind of thing. Is that, I mean, is this, have you thought about this problem richly? Like, and if you have in general, how far away are we from, you worked a lot on language, understanding language generation, but the full dialogue, the conversation, you know, just sitting at the bar, having a couple of beers for an hour, that kind of conversation.

Speaker 2

01:09:22

Have you thought about it?

Speaker 1

01:09:23

Yeah, so I think you touched here on the critical point, which is feasibility, right? So there's a great sort of essay by Hamming, which describes sort of grand challenges of physics. And he argues that, well, OK, for instance, teleportation or time travel are great grand challenges of physics, but there's no attacks.

Speaker 1

01:09:46

We really don't know or cannot kind of make any progress. So that's why most physicists and so on, they don't work on these in their PhDs and as part of their careers. So I see the Turing test as in the full Turing test as a bit still too early. Like I am, I think we're, especially with the current trend of deep learning language models, we've seen some amazing examples.

Speaker 1

01:10:11

I think GPT-2 being the most recent 1, which is very impressive. But to understand, to fully solve passing or fooling a human to think that there's a human on the other side. I think we're quite far. So as a result, I don't see myself and I probably would not recommend people doing a PhD on solving the Turing test because it just feels it's kind of too early or too hard of a problem.

Speaker 2

01:10:35

Yeah, but that said, you said the exact same thing about StarCraft about a few years ago. So to Demis, so I appreciate it. You'll probably also be the person who passes the Turing test in 3 years.

Speaker 1

01:10:48

I mean, I think, I think that, yeah.

Speaker 2

01:10:50

So, so we have this on record. This is nice.

Speaker 1

01:10:52

It's true. It's true. I mean, the, the, it's true that progress sometimes is a bit unpredictable.

Speaker 1

01:10:57

I really wouldn't have not, I, even 6 months ago, I would not have predicted the level that we see that these agents can deliver at Grandmaster level. But I have worked on language enough, and basically my concern is not that something could happen, a breakthrough could happen that would bring us to solving or passing the Turing test is that I just think the statistical approach to it, like it's not gonna cut it. So we need a breakthrough, which is great for the community. But given that, I think there's quite a more uncertainty.

Speaker 1

01:11:31

Whereas for StarCraft, I knew what the steps would be to kind of get us there. I think it was clear that using the imitation learning part and then using these battle net for agents were going to be key. And it turned out that this was the case and a little more was needed, but not much more. For Turing test, I just don't know what the plan or execution plan would look like.

Speaker 1

01:11:55

So that's why I myself working on it as a grand challenge is hard. But there are quite a few sub-challenges that are related that you could say, well, I mean, what if you create a great assistant, like Google already has, like the Google Assistant, so can we make it better and can we make it fully neural and so on? That I start to believe maybe we're reaching a point where we should attempt these challenges.

Speaker 2

01:12:20

I like this conversation so much because it echoes very much the Starcraft conversation. It's exactly how you approach Starcraft. Let's break it down into small pieces and solve those, and you end up solving the whole game.

Speaker 2

01:12:31

Great, But that said, you're behind some of the sort of biggest pieces of work in deep learning in the last several years. So you mentioned some limits. What do you think of the current limits of deep learning and how do we overcome those limits?

Speaker 1

01:12:47

So if I had to actually use a single word to define the main challenge in deep learning, it's a challenge that probably has been the challenge for many years and is that of generalization. So what that means is that all that we're doing is fitting functions to data. And when the data we see is not from the same distribution, or even if there are some times that it is very close to the distribution, but because of the way we train it with limited samples, we then get to this stage where we just don't see generalization as much as we can generalize.

Speaker 1

01:13:27

And I think adversarial examples are a clear example of this, But if you study machine learning and literature, and you know, the reason why SVMs came very popular were because they were dealing and they had some guarantees about generalization, which is unseen data or out of distribution, or even within distribution, where you take an image adding a bit of noise, these models fail. So I think really, I don't see a lot of progress on generalization in the strong generalization sense of the word. I think our neural networks, you can always find design examples that will make their outputs arbitrary, which is not good because we humans would never be fooled by these kind of images or manipulation of the image. And if you look at the mathematics, you kind of understand this is a bunch of matrices multiplied together.

Speaker 1

01:14:26

There's probably numerics and instability that you can just find corner cases. So I think that's really the underlying topic many times we see when, even at the grand stage of like Turing test generalization. I mean, if you start passing the Turing test, should it be in English or should it be in any language, right? I mean, as a human, if you ask something in a different language, you actually will go and do some research and try to translate it and so on.

Speaker 1

01:14:57

Should the Turing test include that, right? And it's really a difficult problem and very fascinating and very mysterious actually.

Speaker 2

01:15:05

Yeah, absolutely. But do you think it's, if you were to try to solve it, can you not grow the size of data intelligently in such a way that the distribution of your training set does include the entirety of the testing set?

Speaker 1

01:15:20

I think-

Speaker 2

01:15:20

Is that 1 path? The other path is totally new methodology.

Speaker 1

01:15:23

Right. It's not statistical. So a path that has worked well, and it worked well in StarCraft and in machine translation and in languages, is scaling up the data and the model. And that's kind of been maybe the only single formula that still delivers today in deep learning, right?

Speaker 1

01:15:40

It's that scale, data scale and model scale really do more and more of the things that we thought, oh, there's no way it can generalize to these or there's no way it can generalize to that. But I don't think fundamentally it will be solved with this. And for instance, I'm really liking some style or approach that would not only have neural networks, but it would have programs or some discrete decision-making, because there is where I feel there's a bit more, like, I mean, the example of, the best example, I think, for understanding this is, I also worked a bit on, oh, like, we can learn an algorithm with a neural network, right? So you give it many examples and it's going to sort the input numbers or something like that.

Speaker 1

01:16:24

But really, strong generalization is you give me some numbers or you ask me to create an algorithm that sorts numbers. And instead of creating a neural net, which will be fragile because it's going to go out of range at some point, you're going to give it numbers that are too large, too small and whatnot. If you just create a piece of code that sorts the numbers, then you can prove that that will generalize to absolutely all the possible inputs you could give. So I think that's the problem comes with some exciting prospects.

Speaker 1

01:16:55

I mean, scale is a bit more boring, but it really works. And then maybe programs and discrete abstractions are a bit less developed, but clearly I think they're quite exciting in terms of future for the field.

Speaker 2

01:17:09

Do you draw any insight wisdom from the 80s and expert systems and symbolic systems, symbolic computing? Do you ever go back to those, the reasoning, that kind of logic? Do you think that might make a comeback?

Speaker 2

01:17:23

You'll have to dust off those books?

Speaker 1

01:17:24

Yeah, I actually love actually adding more inductive biases. To me, the problem really is, what are you trying to solve? If what you're trying to solve is so important that try to solve it no matter what, then absolutely use rules, use domain knowledge, and then use a bit of the magic of machine learning to empower, to make the system as the best system that will detect cancer or detect weather patterns, right?

Speaker 1

01:17:56

Or in terms of StarCraft, it also was a very big challenge. So I was definitely happy that if we had to cut a corner here and there, it could have been interesting to do. And in fact, in StarCraft, we start thinking about expert systems because it's a very, you can define, I mean, people actually build StarCraft bots by thinking about those principles, like state machines and rule-based. And then you could think of combining a bit of a rule-based system, but that has also neural networks incorporated to make it generalize a bit better.

Speaker 1

01:18:29

So absolutely, I mean, we should definitely go back to those ideas. And anything that makes the problem simpler, as long as your problem is important, that's okay. And that's research driving a very important problem. And on the other hand, if you wanna really focus on the limits of reinforcement learning, then of course you must try not to look at imitation data or to look for some rules of the domain that would help a lot or even feature engineering.

Speaker 1

01:18:56

So this is a tension that depending on what you do, I think both ways are definitely fine. And I would never not do 1 or the other, if you're, as long as what you're doing is important and needs to be solved, right? Right.

Speaker 2

01:19:11

So there's a bunch of different ideas that you've developed that I really enjoy. But 1 is translating from, image captioning, translating from image to text. Just another beautiful idea, I think, that resonates throughout your work, actually.

Speaker 2

01:19:33

So the underlying nature of reality being language, always, somehow. So what's the connection between images and text, or rather the visual world and the world of language, in your view?

Speaker 1

01:19:46

Right, So I think a piece of research that's been central to, I would say even extending into StarCraft is this idea of sequence to sequence learning, which what we really meant by that is that you can, you can now really input anything to a neural network as the input X, and then the neural network will learn a function F that will take X as an input and produce any output Y. And these X and Ys don't need to be like static or like a fixed vectors or anything like that. It could be really sequences and now beyond like data structures.

Speaker 1

01:20:25

Right. So that paradigm was tested in a very interesting way when we moved from translating French to English, to translating an image to its caption. But the beauty of it is that really, and that's actually how it happened. I ran, I changed the line of code in this thing that was doing machine translation.

Speaker 1

01:20:47

And I came the next day and I saw how it was producing captions that seemed like, oh my God, this is really, really working. And the principle is the same, right? So I think I don't see text, vision, speech, waveforms as something different. As long as you basically learn a function that will vectorize these into, And then after we vectorize it, we can then use, you know, transformers, LSTMs, whatever the flavor of the month of the model is.

Speaker 1

01:21:22

And then as long as we have enough supervised data, really this formula will work and will keep working, I believe, to some extent, model of these generalization issues that I mentioned before.

Speaker 2

01:21:34

So, but the task there is to vectorize, sort of form a representation that's meaningful, I think. And your intuition now, having worked with all this media, is that once you are able to form that representation, you could basically take anything, any sequence. Is there, going back to StarCraft, is there limits on the length?

Speaker 2

01:21:55

So we didn't really touch on the long term aspect. How did you overcome the whole really long term aspect of things here? Is there some tricks?

Speaker 1

01:22:05

So the main trick, so StarCraft, if you look at absolutely every frame, you might think it's quite a long game. So we would have to multiply 22 times 60 seconds per minute, times maybe at least 10 minutes per game on average. So there are quite a few frames.

Speaker 1

01:22:25

But the trick really was to only observe, in fact, which might be seen as a limitation, but it is also a computational advantage. Only observe when you act. And then what the neural network decides is what is the gap going to be until the next action. And if you look at most StarCraft games that we have in the dataset that Blizzard provided, it turns out that most games are actually only, I mean, it is still a long sequence, but it's maybe like a thousand to 1500 actions, which if you start looking at LSTMs, large LSTMs, transformers, it's not that difficult, especially if you have supervised learning.

Speaker 1

01:23:14

If you had to do it with reinforcement learning, the credit assignment problem, what is it in this game that made you win? That would be really difficult. But thankfully, because of imitation learning, we didn't kind of have to deal with these directly. Although if we had to, we tried it.

Speaker 1

01:23:29

And what happened is you just take all your workers and attack with them. And that sort of is kind of obvious in retrospect, because you start trying random actions. 1 of the actions will be a worker that goes to the enemy base. And because it's self-play, it's not going to know how to defend because it basically doesn't know almost anything.

Speaker 1

01:23:46

And eventually what you develop is this, take all workers and attack. Because the credit assignment issue in RL is really, really hard. I do believe we could do better and that's maybe a research challenge for the future. But yeah, even in StarCraft, the sequences are maybe a thousand, which I believe is within the realm of what Transformers can do.

Speaker 1

01:24:10

Yeah, I guess the difference between StarCraft and Go is in Go and chess, stuff starts happening right away. Right. So there's not, yeah, it's pretty easy through self play, not easy, but through self play, it's possible to develop reasonable strategies quickly, as opposed to Starcraft. I mean, in Go, there's only 400 actions, But 1 action is what people would call the God action.

Speaker 1

01:24:34

That would be if you had expanded the whole search tree, that's the best action if you did minimax or whatever algorithm you would do if you had the computational capacity. But in StarCraft, 400 is minuscule. Like in 400, you couldn't even click on the pixels around a unit, right? So I think the problem there is, in terms of action space size is way harder.

Speaker 1

01:25:01

So, and that surge is impossible. So there's quite a few challenges indeed that make this kind of a step up in terms of machine learning. For humans, maybe playing StarCraft seems more intuitive because it looks real. I mean, you know, like the graphics and everything moves smoothly.

Speaker 1

01:25:18

Whereas I don't know how to, I mean, Go is a game that I would really need to study. It feels quite complicated, but for machines, kind of maybe it's the reverse, yes.

Speaker 2

01:25:27

Which shows you the gap actually between deep learning and however the heck our brains work. So you developed a lot of really interesting ideas. It's interesting to just ask, what's your process of developing new ideas?

Speaker 2

01:25:41

Do you like brainstorming with others? Do you like thinking alone? Do you like, like what was it, Ian Goodfellow said he came up with GANs after a few beers. He thinks beers are essential for coming up with new ideas.

Speaker 1

01:25:55

We had beers to decide to play another game of StarCraft after a week. So It's really similar to that story. Actually, I explained this in a DeepMind retreat, and I said, this is the same as the gun story.

Speaker 1

01:26:07

I mean, we were in a bar and we decided, let's play a game next week. And that's what happened.

Speaker 2

01:26:11

I feel like we're giving the wrong message to young undergrads. Yeah, I know. But in general, Do you like brainstorming?

Speaker 2

01:26:18

Do you like thinking alone, working stuff out?

Speaker 1

01:26:20

And so I think I think throughout the years also things changed. Right. So initially I was very fortunate to be with great minds like Jeff Hinton, Jeff Dean, Ilya Sutskever.

Speaker 1

01:26:34

I was really fortunate to join Brain at a very good time. So at that point, ideas, I was just kind of brainstorming with my colleagues and learned a lot. And keep learning is actually something you should never stop doing. Right?

Speaker 1

01:26:48

So learning implies reading papers and also discussing ideas with others. It's very hard at some point to not communicate that being reading a paper from someone or actually discussing. Right. So definitely that communication aspect needs to be there, whether it's written or oral.

Speaker 1

01:27:08

Nowadays, I'm also trying to be a bit more strategic about what research to do. So I was describing a little bit this sort of tension between research for the sake of research, and then you have, on the other hand, applications that can drive the research, right? And honestly, the formula that has worked best for me is just find a hard problem and then try to see how research fits into it, how it doesn't fit into it, and then you must innovate. So I think machine translation drove sequence to sequence.

Speaker 1

01:27:42

Then maybe like learning algorithms that had to like combinatorial algorithms led to pointer networks, StarCraft led to really scaling up imitation learning and the AlphaStar League. So that's been a formula that I personally like, but the other 1 is also valid. And I see it succeed a lot of the times where you just want to investigate model-based RL as a kind of a research topic. And then you must then start to think, well, how are the tests?

Speaker 1

01:28:12

How are you going to test these ideas? You need kind of a minimal environment to try things. You need to read a lot of papers and so on. And that's also very fun to do and something I've also done quite a few times, both at Brain, at DeepMind and obviously as a PhD.

Speaker 1

01:28:28

So I think Besides the ideas and discussions, I think it's important also because you start sort of guiding not only your own goals, but other people's goals to the next breakthrough. So you must really kind of understand this feasibility also, as we were discussing before, right? Whether this domain is ready to be tackled or not, and you don't want to be too early. You obviously don't want to be too late.

Speaker 1

01:28:57

So it's really interesting, this strategic component of research, which I think as a grad student, I just had no idea. I just read papers and discussed ideas. And I think this has been maybe the major change. And I recommend people kind of feed forward to success how it looks like and try to backtrack, other than just kind of looking out, this looks cool, this looks cool.

Speaker 1

01:29:19

And then you do a bit of random work, which sometimes you stumble upon some interesting things. But in general, it's also good to plan a bit.

Speaker 2

01:29:27

Yeah, I like it, especially like your approach of taking a really hard problem, stepping right in, and then being super skeptical about being able to solve the problem. I mean, there's a balance of both, right? There's a silly optimism and a critical sort of skepticism that's good to balance, which is why it's good to have a team of people that balance that.

Speaker 1

01:29:52

You don't do that on your own. You have both mentors that have seen, or you obviously wanna chat and discuss whether it's the right time. I mean, Demis came in 2014 and he said, maybe in a bit we'll do StarCraft and maybe he knew.

Speaker 1

01:30:09

And I'm just following his lead, which is great because he's brilliant, right? So these things are obviously quite important that you want to be surrounded by people who are diverse, they have their knowledge, there's also important to... I mean, I've learned a lot from people who actually have an idea that I might not think it's good, but if I give them the space to try it, I've been proven wrong many, many times as well. So that's great.

Speaker 1

01:30:38

I think it's, your colleagues are more important than yourself, I think. Sure.

Speaker 2

01:30:44

Now, let's real quick talk about another impossible problem, AGI. Right. What do you think it takes to build a system that's human level intelligence?

Speaker 2

01:30:53

We talked a little bit about the Turing test, Starcraft, all of these have echoes of general intelligence. But if you think about just something that you would sit back and say, wow, this is really something that resembles human level intelligence. What do you think it takes to build that?

Speaker 1

01:31:09

So I find that AGI oftentimes is maybe not very well-defined. So what I'm trying to then come up with for myself is what would be a result look like that you would start to believe that you would have agents or neural nets that no longer sort of overfit to a single task, right? But actually kind of learn the skill of learning, so to speak.

Speaker 1

01:31:37

And that actually is a field that I am fascinated by, which is the learning to learn or meta learning, which is about no longer learning about a single domain. So you can think about the learning algorithm itself is general, right? So the same formula we applied for AlphaStar or StarCraft, we can now apply to kind of almost any video game, or you could apply to many other problems and domains. But the algorithm is what's kind of generalizing.

Speaker 1

01:32:06

But the neural network, those weights are useless even to play another race, right? I train a network to play very well at Protoss versus Protoss. I need to throw away those weights. If I want to play now Terran versus Terran, I would need to retrain a network from scratch with the same algorithm.

Speaker 1

01:32:24

That's beautiful, but the network itself will not be useful. So I think if I see an approach that can absorb or start solving new problems without the need to kind of restart the process, I think that to me would be a nice way to define some form of AGI. Again, I don't know the grandiose, like, I mean, should Turing test be solved before AGI? I mean, I don't know.

Speaker 1

01:32:51

I think concretely, I would like to see clearly that meta-learning happen, meaning there is an architecture or a network that as it sees new problem or new data, it solves it. And to make it kind of a benchmark, it should solve it at the same speed that we do solve new problems. When I define you a new object and you have to recognize it, when you start playing a new game, you played all the Atari games, but now you play a new Atari game, well, you're going to be pretty quickly pretty good at the game. So that's perhaps what's the domain and what's the exact benchmark is a bit difficult.

Speaker 1

01:33:27

I think as a community, we might need to do some work to define it. But I think this first step, I could see it happen relatively soon. But then the whole what AGI means and so on, I am a bit more confused about what I think people mean different things.

Speaker 2

01:33:44

There's an emotional psychological level that, like even the Turing test, passing the Turing test is something that we just pass judgment on as human beings what it means to be, you know, as a dog in the AGI system. Like what level, what does it mean? What does it mean?

Speaker 2

01:34:07

But I like the generalization and maybe as a community we converge towards a group of domains that are sufficiently far away That would be really damn impressive if it was able to generalize. So perhaps not as close as Protoss and Zerg, but like Wikipedia. Yeah, that would be a good step. And then a really good step, but then like from StarCraft to Wikipedia and back.

Speaker 2

01:34:30

Yeah. That kind of thing.

Speaker 1

01:34:31

And that feels also quite hard and far, but I think as long as you put the benchmark out, as we discovered, for instance, with ImageNet, then tremendous progress can be had. So I think maybe there's a lack of benchmark, But I'm sure we'll find 1 and the community will then work towards that. And then beyond what AGI might mean or would imply, I really am hopeful to see basically machine learning or AI just scaling up and helping people that might not have the resources to hire an assistant or that they might not even know what the weather is like.

Speaker 1

01:35:13

So I think there's, in terms of the impact, the positive impact of AI, I think that's maybe what we should also not lose focus. The research community building AGI, I mean, that's a real nice goal, but I think the way that DeepMind puts it is, and then use it to solve everything else, right? So I think we

Speaker 2

01:35:32

should paralyze. Yeah, we shouldn't forget about all the positive things that are actually coming out of AI already and are going to be coming out. Right.

Speaker 2

01:35:43

On that note, let me ask, relative to popular perception, do you have any worry about the existential threat of artificial intelligence in the near or far future that some people have?

Speaker 1

01:35:55

I think in the near future, I'm skeptical, so I hope I'm not wrong, but I'm not concerned, but I appreciate efforts, ongoing efforts, and even like a whole research field on AI safety emerging and in conferences and so on. I think that's great. In the long term, I really hope we just can simply have the benefits outweigh the potential dangers.

Speaker 1

01:36:21

I am hopeful for that. But also we must remain vigilant to kind of monitor and assess whether the trade-offs are there and we have enough also lead time to prevent or to redirect our efforts if need be. But I'm quite optimistic about the technology and definitely more fearful of other threats in terms of planetary level at this point. But obviously that's the 1 I kind of have more power on.

Speaker 1

01:36:52

So clearly I do start thinking more and more about this and it's kind of, it's grown in me actually to start reading more about AI safety, which is a field that so far I have not really contributed to, but maybe there's something to be done there as well.

Speaker 2

01:37:07

Well, I think it's really important. You know, I talk about this with a few folks, but it's important to ask you and shove it in your head because you're at the leading edge of actually what people are excited about in AI. I mean, the work with AlphaStar, it's arguably at the very cutting edge of the kind of thing that people are afraid of.

Speaker 2

01:37:27

And so you speaking to that fact and that we're actually quite far away to the kind of thing that people might be afraid of, but it's still worthwhile to think about. And it's also good that you're not as worried and you're also open to thinking about it.

Speaker 1

01:37:45

There's 2 aspects. Me not being worried, but obviously we should prepare for it, For things that could go wrong, misuse of the technologies as with any technologies, right? So I think there's always trade-offs.

Speaker 1

01:38:02

And as a society, we've kind of solved these to some extent in the past. So I'm hoping that by having the researchers and the whole community brainstorm and come up with interesting solutions to the new things that will happen in the future, that we can still also push the research to the avenue that I think is kind of the greatest avenue, which is to understand intelligence, right? How are we doing what we're doing? And, you know, obviously from a scientific standpoint, that is kind of my personal driver of all the time that I spend doing what I'm doing,

Speaker 2

01:38:38

really. Where do you see the deep learning as a field heading? Where do you think the next big breakthrough might be?

Speaker 1

01:38:46

So I think deep learning, I discussed a little of this before, deep learning has to be combined with some form of discretization, program synthesis. I think that's kind of as a research in itself is an interesting topic to expand and start doing more research. And then as kind of what will deep learning enable to do in the future?

Speaker 1

01:39:08

I don't think that's going to be what's going to happen this year, but also this idea of starting not to throw away all the weights, this idea of learning to learn and really having these agents not having to restart their weights. And you can have an agent that is kind of solving or classifying images on ImageNet, but also generating speech if you ask it to generate some speech. And it should really be kind of almost the same network, but it might not be a neural network. It might be a neural network with an optimization algorithm attached to it.

Speaker 1

01:39:45

But I think this idea of generalization to new tasks is something that we first must define good benchmarks. But then I think that's going to be exciting. And I'm not sure how close we are, but I think if you have a very limited domain, I think we can start doing some progress. And much like how we did a lot of programs in computer vision, we should start thinking, I really like a talk that Leon Boutou gave at ICML a few years ago, which is this train-test paradigm should be broken.

Speaker 1

01:40:17

We should stop thinking about a training set and a test set, and these are closed things that are untouchable. I think we should go beyond these. And in meta-learning, we call these the meta-training set and the meta-test set, which is really thinking about, if I know about ImageNet, why would that network not work on MNIST, which is a much simpler problem? But right now it really doesn't.

Speaker 1

01:40:44

But it just feels wrong, right? So I think that's kind of the, there's the, on the application or the benchmark sites, we probably will see quite a few more interest and progress and hopefully people defining new and exciting challenges really.

Speaker 2

01:41:00

Do you have any hope or interest in knowledge graphs within this context? So this is kind of constructing graph. So going back to graphs, well, neural networks and graphs, but I mean a different kind of knowledge graph, sort of like semantic graphs or there's concepts?

Speaker 1

01:41:17

Yeah, so I think the idea of graphs is, so I've been quite interested in sequences first and then more interesting or different data structures like graphs, and I've studied graph neural networks in the last 3 years or so. I found these models just very interesting from like deep learning sides standpoint. But then how, what do we want?

Speaker 1

01:41:44

Why do we want these models and why would we use them? What's the application? What's kind of the killer application of graphs, right? And perhaps if we could extract a knowledge graph from Wikipedia automatically, That would be interesting because then these graphs have this very interesting structure that also is a bit more compatible with this idea of programs and deep learning kind of working together, jumping neighborhoods and so on.