Brian Kernighan is a professor of computer science at Princeton University. He co-authored the C Programming Language with Dennis Ritchie (creator of C) and has written a lot of books on programming, computers, and life including the Practice of Programming, the Go Programming Language, his latest UNIX: A History and a Memoir. He co-created AWK, the text processing language used by Linux folks like myself. He co-designed AMPL, an algebraic modeling language for large-scale optimization.

Support this podcast by supporting our sponsors:
- Eight Sleep: https://eightsleep.com/lex
- Raycon: http://buyraycon.com/lex

EPISODE LINKS:
Brian's website: https://www.cs.princeton.edu/~bwk/
Unix: A History and a Memoir (book): https://amzn.to/3fFJ1yM
Understanding the Digital World (book): https://amzn.to/30ktBJI

PODCAST INFO:
Podcast website:
https://lexfridman.com/podcast
Apple Podcasts:
https://apple.co/2lwqZIr
Spotify:
https://spoti.fi/2nEwCF8
RSS:
https://lexfridman.com/feed/podcast/
Full episodes playlist:
https://www.youtube.com/playlist?list=PLrAXtmErZgOdP_8GztsuKi9nrraNbKKp4
Clips playlist:
https://www.youtube.com/playlist?list=PLrAXtmErZgOeciFP3CBCIEElOJeitOr41

OUTLINE:
0:00 - Introduction
4:24 - UNIX early days
22:09 - Unix philosophy
31:54 - Is programming art or science?
35:18 - AWK
42:03 - Programming setup
46:39 - History of programming languages
52:48 - C programming language
58:44 - Go language
1:01:57 - Learning new programming languages
1:04:57 - Javascript
1:08:16 - Variety of programming languages
1:10:30 - AMPL
1:18:01 - Graph theory
1:22:20 - AI in 1964
1:27:50 - Future of AI
1:29:47 - Moore's law
1:32:54 - Computers in our world
1:40:37 - Life

CONNECT:
- Subscribe to this YouTube channel
- Twitter: https://twitter.com/lexfridman
- LinkedIn: https://www.linkedin.com/in/lexfridman
- Facebook: https://www.facebook.com/LexFridmanPage
- Instagram: https://www.instagram.com/lexfridman
- Medium: https://medium.com/@lexfridman
- Support on Patreon: https://www.patreon.com/lexfridman

And so that's an interesting combination of things.

And so some ways Go captures the good parts of C, it looks sort of like C, it's sometimes characterized as C for the 21st century.

On the surface, it looks very, very much like C.

But at the same time, it has some interesting data structuring capabilities.

And then I think the part that I would say is particularly useful, and again, I'm not a Go expert.

In spite of co-authoring the book, About 90% of the work was done by Alan Donovan, my co-author, who is a Go expert.

But Go provides a very nice model of concurrency.

It's basically the cooperating, communicating sequential processes that Tony Hoare set forth, geez, I don't know, 40 plus years ago.

And Go routines are, to my mind, a very natural way to talk about parallel computation.

And in the few experiments I've done with them, they're easy to write and typically it's gonna work and very efficient as well.

So I think that's 1 place where Go stands out, that that model of parallel computation is very, very easy and nice to work with.

Just to comment on that, do you think C foresaw, or the early Unix days foresaw threads and massively parallel computation?

I mean, maybe it was seen, but not at the level where it was something you had to do anything about.

For a long time, processors got faster and then processors stopped getting faster because of things like power consumption and heat generation.

And so what happened instead was that instead of processors getting faster, there started to be more of them.

And that's where that parallel thread stuff comes in.

So if you can comment on all the other languages, does it break your heart that you'll never get to explore them?

Or how do you feel about the full variety?

It's not break my heart, but I would love to be able to try more of these languages.

The closest I've come is in a class that I often teach in the spring here.

And I often give, I have 1 sort of small example that I will write in as many languages as I possibly can.

I've got it in 20 odd languages at this point.

And that's so I do a minimal experiment with a language just to say, okay, I have this trivial task, which I understand the task, and it takes 15 lines in AWK and not much more in a variety of other languages.

And what pain did I go through to learn how to do it?

And that's a, it's like anic data, right?

So Yeah, but still, it's a little sample because you get to, I think the hardest step of the programming language is probably the first step, right, so there you're taking the first step.

Yeah, and so my experience with some languages is very positive, like Lua, a scripting language I had never used, And I took my little program.

It just takes in lines of text of varying lengths and it puts them out in lines that have no more than 60 characters on each line.

So think of it as just kind of the flow of process in a browser or something.

And in Lua, I downloaded Lua, and in an hour I had it working, never having written Lua in my life, just going with online documentation.

I did the same thing in Scala, which you can think of as a flavor of Java, equally trivial.

It took me several weeks, but it did run like a turtle.

And I did it in Fortran 90 and it painful, but it worked and I tried it in Rust and it took me several days to get it working because the model of memory management was just a little unfamiliar to me.

And the problem I had with Rust, and it's back to what we were just talking about, I couldn't find good, consistent documentation on Rust.

Now this was several years ago, and I'm sure things have stabilized, but at the time, everything in the Rust world seemed to be changing rapidly, and so you would find what looked like a working example, and it wouldn't work with the version of the language that I had.

Rust is a language I would like to get back to, but probably won't.

I think 1 of the issues, you have to have something you want to do.

And if you don't have something that is the right combination, if I want to do it, and yet I have enough disposable time, whatever, to make it worth learning a new language at the same time, it's never gonna happen.

So what do you think about another language of JavaScript that's this, well, let me just sort of comment on what I said.

When I was brought up, sort of JavaScript was seen as probably like the ugliest language possible, and yet it's quite arguably, quite possibly, taking over not just the front end and the back end of the internet, but possibly in the future taking over everything, because they've now learned to make it very efficient.

Yeah, Well, I think you've captured it in a lot of ways.

When it first came out, JavaScript was deemed to be fairly irregular and an ugly language.

And certainly in the academy, if you said you were working on JavaScript, people would ridicule you.

It was just not fit for academics to work on.

And certainly the technology of compiling it is fantastically better than it was.

And so in that sense, it's absolutely a viable solution on backends as well as the frontends.

Used well, I think it's a pretty good language.

I've written a modest amount of it, and I've played with JavaScript translators and things like that.

I'm not a real expert, and it's hard to keep up even there with the new things that come along with it.

So I don't know whether it will ever take over the world, I think not, but it's certainly an important language and worth knowing more about.

There's, maybe to get your comment on something, which JavaScript, and actually most languages, sort of Python, such a big part of the experience of programming with those languages includes libraries.

Sort of using, building on top of the code that other people have built.

I think that's probably different from the experience that we just talked about from Unix and C days when you're building stuff from scratch.

What do you think about this world of essentially leveraging, building up libraries on top of each other and leveraging them?

Yeah, that's a very perceptive kind of question.

1 of the reasons programming was fun in the old days was that you were really building it all yourself.

The number of libraries you had to deal with was quite small.

Maybe it was printf or the standard library or something like that.

And if you want to do something in, you mentioned Python and JavaScript, and those are the 2 fine examples, you have to typically download a boatload of other stuff, and you have no idea what you're getting.

I've been doing some playing with machine learning over the last couple of days, and gee, something doesn't work.

Well, you pip install this, okay, and down comes another gazillion megabytes of something, and you have no idea what it was.

And if it doesn't work, you have no recourse.

There's absolutely no way you could figure out which in these thousand different packages.

And I think it's worse in the NPM environment for JavaScript.

I think there's less discipline, less control there.

And there's aspects of not just not understanding how it works, but there's security issues, there's robustness issues, so you don't wanna run a nuclear power plant using JavaScript, essentially.

So speaking to the variety of languages, do you think that variety is good, or do you hope, think that over time, we should converge towards 123 programming languages?

You mentioned to the Bell Lab days when people could sort of, the community of it.

And the more languages you have, the more you separate the communities.

There's the Ruby community, there's the Python community, there's C++ community.

Do you hope that they'll unite 1 day to just 1 or 2 languages?

I'm not sure that that's right, because I honestly don't think there is 1 language that will suffice for all the programming needs of the world.

Well, arguably, but I think if you look at the distribution of how they are used.

There's something called a dozen languages that probably account for 95% of all programming at this point, and that doesn't seem unreasonable.

And then there's another, well, 2000 languages that are still in use that nobody uses, or at least don't use in any quantity.

But I think new languages are a good idea in many respects because they're often a chance to explore an idea of how a language might help.

I think that's 1 of the positive things about functional languages, for example.

They're a particularly good place where people have explored ideas that at the time didn't seem feasible, but ultimately have wound up as part of mainstream languages as well.

I mean, you just go back as early as recursion and Lisp, and then follow forward functions as first-class citizens and pattern-based languages and, gee, I don't know, closures and just on and on and on.

Lambdas, interesting ideas that showed up first in, let's call it broadly, the functional programming community and then find their way into mainstream languages.

And so I think the languages in the playground themselves are probably not going to be the mainstream, at least for some while, but the ideas that come from there are invaluable.

So let's go to something that when I found out recently, so I've known that you've done a million things, but 1 of the things I wasn't aware of that you had a role in Ample, and before you interrupt me by minimizing your role in it.

Yeah, minimizing functions, right, exactly.

Can I just say that the elegance and abstraction power of AMPL is incredible?

When I first came to it about 10 years ago or so, Can you describe what is the AMPL language?

So AMPL is a language for mathematical programming, technical term, think of it as linear programming.

That is setting up systems of linear equations that are some sort of system of constraints, so that you have a bunch of things that have to be less than this, greater than that, or whatever.

And you're trying to find a set of values for some decision variables that will maximize or minimize some objective function.

So it's a way of solving a particular kind of optimization problem, a very formal sort of optimization problem, but 1 that's exceptionally useful.

And it specifies, so there's objective function constraints and variables that become separate from the data it operates on.

So that kind of separation allows you to put on different hats.

1 put the hat of an optimization person and then put another hat of a data person and dance back and forth.

And also separate the actual solvers, the optimization systems that do the solving.

Then you can have other people come to the table and then build their solvers, whether it's linear or non-linear, convex, non-convex, that kind of stuff.

So, what is the, to you is, maybe you can comment how you got into that world and what is the beautiful or interesting idea to you from the world of optimization?

So I preface it by saying I'm absolutely not an expert on this and most of the important work in AMPL comes from my 2 partners in crime on that, Bob Forerer, who was a professor in the industrial engineering and management science department at Northwestern, and my colleague at Bell Labs, Dave Gay, who was a numerical analyst and optimization person.

Well, let's stay with linear programming.

Yeah, Linear programming is the simplest example of this.

So linear programming as it's taught in school is that you have a big matrix, which is always called A, and you say AX is less than or equal to B.

So B is a set of constraints, X is the decision variables, and A is how the decision variables are combined to set up the various constraints.

So A is a matrix and X and B are vectors.

And then there's an objective function, which is just a sum of a bunch of Xs and some coefficients on them.

And yet that's the thing you want to optimize.

The problem is that in the real world, that matrix A is a very, very, very intricate, very large and very sparse matrix where the various components of the model are distributed among the coefficients in a way that is totally unobvious to anybody.

And so what you need is some way to express the original model, which you and I would write, you know, we'd write mathematics on the board.

The sum of this is greater than the sum of that kind of thing.

So you need a language to write those kinds of constraints.

And Bob Forer for a long time had been interested in modeling languages, languages that made it possible to do this.

There was a modeling language around called GAMS, the General Algebraic Modeling System, but it looked very much like Fortran.

And so Bob spent a sabbatical year at Bell Labs in 1984, and he and—there's only the office across from me, and it's always geography.

He and Dave Gay and I started talking about this kind of thing.

He wanted to design a language that would make it so that you could take these algebraic specifications, you know, summation, sines, oversets, and that you would write on the board and convert them into basically this A matrix.

And then pass that off to a solver, which is an entirely separate thing.

And so we talked about the design of the language.

I don't remember any of the details of this now, but it's kind of an obvious thing.

You're just writing out mathematical expressions in a Fortran-like, or sorry, an algebraic but textual-like language.

And I wrote the first version of this AMPL program, my first C++ program.

We wrote, it was 3,000 lines or something, so it wasn't very big, but it sort of showed the feasibility of it that you could actually do something that was easy for people to specify models and convert it into something that a solver could work with.

At the same time, as you say, the model and the data are separate things.

So 1 model would then work with all kinds of different data in the same way that lots of programs do the same thing but with different data.

So 1 of the really nice things is the specification of the models, human, just kind of like as you say, is human readable.

Like I literally, I remember on stuff I worked, I would send it to colleagues that I'm pretty sure never programmed in their life, just to understand what the optimization problem is.

You said there's a first prototype in C++ to convert that into something that could actually be used by the solver.

It's not too bad because most of the solvers have some mechanism that lets them import a model in a form.

It might be as simple as the matrix itself in just some representation, or if you're doing things that are not linear programming, then there may be some mechanism that lets you provide things like functions to be called or other constraints on the model.

So all Ample does is to generate that kind of thing and then Solver deals with all the hard work.

And then when the Solver comes back with numbers, Ample converts those back into your original form.

So you know how much of each thing you should be buying or making or shipping or whatever.

So we did that in 84 and I haven't had a lot to do with it since except that we wrote a couple of versions of a book on it.

Which is 1 of the greatest books ever written.

I remember liking the typography, so I don't know.

Think of T-Rof as a predecessor to the tech family of things.

It's a formatter that was done at Bell Labs in this same period of the very early 70s that predates tech and things like that by 5 to 10 years.

But it was nevertheless, I'm going by memories, I remember it being beautiful.

Outside of Unix, C, Ogg, Golang, all the things we talked about, all the amazing work you've done, you've also done work in graph theory.

Let me ask this crazy out there question.

If you had to make a bet and I had to force you to make a bet, do you think P equals NP?

The answer is no, although I'm told that somebody asked Jeff Dean if that was the, under what conditions P would equal NP, and he said either P is 0 or N is 1, or vice versa, I've forgotten.

I have no intuition, but I've got a lot of colleagues who've got intuition and their betting is no.

Okay, so what is computational complexity theory?

And do you think these kinds of complexity classes, especially as you've taught in this modern world, are still useful way to understand the hardness of problems?

The last time I touched anything to do with that was before it was invented.

Before, I did this in 1968, and I worked on graph partitioning, which is this question, you've got a graph that is a nodes and edges kind of graph, and the edges have weights, and you just want to divide the nodes into 2 piles of equal size so that the number of edges that goes from 1 side to the other is as small as possible.

Well, as it turns out, I worked with Shen Lin at Bell Labs on this, and we were never able to come up with anything that was guaranteed to give the right answer.

We came up with heuristics that worked pretty darn well, and I peeled off some special cases for my thesis, but it was just hard.

And that was just about the time that Steve Cook was showing that there were classes of problems that appeared to be really hard, of which graph partitioning was 1.

But this, my expertise, such as it was, totally predates that development.

So the heuristic, which now carries the 2 of yours names for the traveling salesman problem and for the graph partitioning, that was, like, how did you, you weren't even thinking in terms of classes.

A heuristic that kind of does the job pretty well.

You're trying to find something that did the job and there was nothing that you would call, let's say a closed form or algorithmic thing that would give you a guaranteed right answer.

I mean, compare graph partitioning to max flow min cut or something like that.

That's the same problem except there's no constraint on the number of nodes on 1 side or the other of the cut.

And that means it's an easy problem, at least as I understand it.

Whereas the constraint that says the 2 have to be constrained in size makes it a hard problem.

Yeah, so Robert Frost has that poem where you have to choose 2 paths.

So why did you, is there another alternate universe in which you pursued the Don Knuth path of algorithm design sort of.

You're infinitely modest, but so you pursued your kind of love of programming.

I mean, when you look back to those, I mean, just looking into that world, does that just seem like a distant world of theoretical computer science?

Then, is it fundamentally different from the world of programming?

I mean, certainly, in all seriousness, I just didn't have the talent for it.

I, when I got here as a grad student, Princeton, and I started to think about research at the end of my, I don't know, first year or something like that.

I worked briefly with John Hopcroft, who was absolutely, you know, you mentioned during award winner, etc., a great guy.

And it became crystal clear I was not cut out for this stuff, period.

And so I moved into things where I was more cut out for it, and that tended to be things like writing programs and then ultimately writing books.

You said that in Toronto as an undergrad you did a senior thesis or literature survey on artificial intelligence.

What was the AI landscape, ideas, dreams at that time?

I think that was 1 of the, well, you've heard of AI winters.

This is whatever the opposite was, AI summer or something.

It was 1 of these things where people thought that, boy, we could do anything with computers, that all these hard problems, we could, computers will solve them.

They will do machine translation, they will play games like chess, they will do, you know, prove theorems in geometry.

There are all kinds of examples like that where people thought, boy, we could really do those sorts of things.

I still have this wonderful collection of papers called Computers and Thought that was published in about that era, and people were very optimistic.

And then of course, it turned out that what people had thought was just a few years down the pike was more than a few years down the pike.

And some parts of that are more or less now sort of under control.

We finally do play games like Go and chess and so on better than people do.

And machine translation is a lot better than it used to be.

But that's 50, close to 60 years of progress, and a lot of evolution in hardware and a tremendous amount more data upon which you can build systems that actually can learn from some of that data.

And the infrastructure to support developers working together, like an open source movement.

The internet, period, is also an empowering.

But what lesson do you draw from that, the opposite of winter, that optimism?

I guess the lesson is that in the short run, it's pretty easy to be too pessimistic or maybe too optimistic, and in the long run, you probably shouldn't be too pessimistic.

It reminds me of this remark from Arthur Clarke, a science fiction author, who says, you know, when some distinguished but elderly person says that something is possible, he's probably right.

And if he says it's impossible, he's almost surely wrong.

But you don't know what the time scale is.

So what are your thoughts on this new summer of AI now in the work with machine learning and neural networks?

You've kind of mentioned that you started to try to explore and look into this world that seems fundamentally different from the world of heuristics and algorithms like search, that it's now purely sort of trying to take huge amounts of data and learn from that data, right?

Most of what I know I've learned from my students, and they're probably disappointed in how little I've learned from them.

But I think it has tremendous potential for certain kinds of things.

I mean, games is 1 where it obviously has had an effect on some of the others as well.

I think there's, and this is speaking from definitely not expertise, I think there are serious problems in certain kinds of machine learning, at least because what they're learning from is the data that we give them.

And if the data we give them has something wrong with it, then what they learn from it is probably wrong too.

And the obvious thing is some kind of bias in the data.

That the data has stuff in it like, I don't know, women aren't as good as men at something, okay?

But if it's in the data because of historical treatment, then that machine learning stuff will propagate that.

The positive part of that is what machine learning does is reveal the bias in the data and puts a mirror to our own society.

And in so doing, helps us remove the bias, you know, helps us work on ourselves.

Yeah, that's an optimistic point of view.

And if it works that way, that would be absolutely great.

And what I don't know is whether it does work that way or whether the AI mechanisms or machine learning mechanisms reinforce and amplify things that have been wrong in the past.

And I don't know, but I think that's a serious thing that we have to be concerned about.

See all Lex Fridman transcripts on Youtube

Brian Kernighan: UNIX, C, AWK, AMPL, and Go Programming | Lex Fridman Podcast #109