See all Lex Fridman transcripts on Youtube

Speaker 1

00:00

The following is a conversation with Jeff Shainlein, a scientist at NIST interested in optoelectronic intelligence. We have a deep technical dive into computing hardware that will make Jim Keller proud. I urge you to hop onto this rollercoaster ride through neuromorphic computing and superconducting electronics and hold on for dear life. Jeff is a great communicator of technical information and so it was truly a pleasure to talk to him about some physics and engineering.

Speaker 1

00:33

To support this podcast, please check out our sponsors in the description. This is the Lex Friedman Podcast, and here is my conversation with Jeff Shainlein. I got a chance to read a fascinating paper you authored called Optoelectronic Intelligence. So maybe we can start by talking about this paper and start with the basic questions.

Speaker 1

00:57

What is optoelectronic intelligence?

Speaker 2

01:00

Yeah, so in that paper, the concept I was trying to describe is sort of an architecture for building brain-inspired computing that leverages light for communication in conjunction with electronic circuits for computation. In that particular paper, a lot of the work we're doing right now in our project at NIST is focused on superconducting electronics for computation. I'll go into why that is, but that might make a little more sense in context if we first describe what that is in contrast to, which is semiconducting electronics.

Speaker 2

01:37

So is it worth taking a couple minutes to describe semiconducting electronics?

Speaker 1

01:42

It might even be worthwhile to step back and talk about electricity and circuits and how circuits work before we talk about superconductivity. Right, okay. How does a computer work, Jeff?

Speaker 2

01:58

Well, I won't go into everything that makes a computer work, but let's talk about the basic building blocks, a transistor. And even more basic than that, a semiconductor material, silicon, say. In silicon, silicon is a semiconductor.

Speaker 2

02:15

And what that means is at low temperature, there are no free charges, no free electrons that can move around. So when you talk about electricity, you're talking about predominantly electrons moving to establish electrical currents and they move under the influence of voltages. So you apply voltages, electrons move around, those can be measured as currents, and you can represent information in that way. So semiconductors are special in the sense that they are really malleable.

Speaker 2

02:46

So if you have a semiconductor material, you can change the number of free electrons that can move around by putting different elements, different atoms in lattice sites. So what is a lattice site? Well, a semiconductor is a crystal, which means all the atoms that comprise the material are at exact locations that are perfectly periodic in space. So if you started any 1 atom and you go along what are called the lattice vectors, you get to another atom and another atom and another atom.

Speaker 2

03:17

And for high quality devices, it's important that it's a perfect crystal with very few defects. But you can intentionally replace a silicon atom with, say, a phosphorus atom. And then you can change the number of free electrons that are in a region of space that has that excess of what are called dopants. So picture a device that has a left terminal and a right terminal, and if you apply a voltage between those 2, you can cause electrical current to flow between them.

Speaker 2

03:47

Now we add a third terminal up on top there, and depending on the voltage between the left and right terminal and that third voltage, you can change that current. So what's commonly done in digital electronic circuits is to leave a fixed voltage from left to right and then change that voltage that's applied at what's called the gate the gate of the transistor. So what you do is you you make it to where there's an excess of electrons on the left, excess of electrons on the right, and very few electrons in the middle. And you do this by changing the concentration of different dopants in the lattice spatially.

Speaker 2

04:24

And then when you apply a voltage to that gate, you can either cause current to flow or turn it off. And so that's sort of your 0 and 1. If you apply voltage, current can flow, that current is representing a digital 1, and from that, from that basic element, you can build up all the complexity of digital electronic circuits that have really had a profound influence on our society.

Speaker 1

04:49

Now you're talking about electrons, can you give a sense of what scale we're talking about when we're talking about in silicon being able to mass manufacture these kinds of gates?

Speaker 2

05:01

Yeah, so scale in a number of different senses. Well, at the scale of the silicon lattice, the distance between 2 atoms there is half a nanometer. So people often like to compare these things to the width of a human hair.

Speaker 2

05:16

I think it's some 6 orders of magnitude smaller

Speaker 1

05:19

than the width of a

Speaker 2

05:19

human hair, I think it's some 6 orders of magnitude smaller than the width of a human hair, something on that order. So remarkably small, we're talking about individual atoms here and electrons are of that length scale when they're in that environment. But there's another sense that scale matters in digital electronics.

Speaker 2

05:34

This is perhaps the more important sense, although they're related. Scale refers to a number of things. It refers to the size of that transistor. So for example, I said you have a left contact, a right contact, and some space between them where the gate electrode sits.

Speaker 2

05:52

That's called the channel width or the channel length. And what has enabled what we think of as Moore's Law or the continued increased performance in silicon microelectronic circuits is the ability to make that size, that feature size ever smaller, ever smaller at a really remarkable pace. I mean, that feature size has decreased consistently every couple of years since the 1960s. And that was what Moore predicted in the 1960s.

Speaker 2

06:27

He thought it would continue for at least 2 more decades and it's been much longer than that. And so that is why we've been able to fit ever more devices, ever more transistors, ever more computational power on essentially the same size of chip. So a user sits back and does essentially nothing. You're running the same computer program, but those devices are getting smaller, so they get faster, they get more energy efficient, and all of our computing performance just continues to improve.

Speaker 2

06:53

And we don't have to think too hard about what we're doing as, say, a software designer or something like that. I absolutely don't mean to say that there's no innovation in software or the user side of things. Of course there is, but from the hardware perspective, we just have been given this gift of continued performance improvement through this scaling that is ever smaller feature sizes with very similar, say, power consumption. That power consumption has not continued to scale in the most recent decades, but nevertheless, we had a really good run there for a while.

Speaker 2

07:31

And now we're down to gates that are 7 nanometers, which is state of the art right now. Maybe Global Foundries is trying to push it even lower than that. I can't keep up with where the predictions are that it's going to end. But 7 nanometer transistor has just a few tens of atoms along the length of the conduction pathway.

Speaker 2

07:52

So a naive semiconductor device physicists would think you can't go much further than that without some kind of revolution in the way we think about the physics of our devices.

Speaker 1

08:03

Is there something to be said about the mass manufacture of these devices?

Speaker 2

08:08

Right, right. So that's another thing. So how have we been able to make those transistors smaller and smaller?

Speaker 2

08:14

Well, companies like Intel, Global Foundries, they invest a lot of money in the lithography. So how are these chips actually made? Well, 1 of the most important steps is this, what's called ion implantation. So you start with sort of a pristine silicon crystal and then using photolithography, which is a technique where you can pattern different shapes using light, you can define which regions of space you're going to implant with different species of ions that are going to change the local electrical properties right there.

Speaker 2

08:49

So by using ever shorter wavelengths of light and different kinds of optical techniques and different kinds of lithographic techniques, things that go far beyond my knowledge base, you can just simply shrink that feature size down. And you say you're at 7 nanometers. Well, the wavelength of light that's being used is over a hundred nanometers. That's already deep in the UV.

Speaker 2

09:10

So how are those minute features patterned? Well, There's an extraordinary amount of innovation that has gone into that, but nevertheless, it stayed very consistent in this ever shrinking feature size. And now the question is, can you make it smaller? And even if you do, do you still continue to get performance improvements?

Speaker 2

09:28

But that's another kind of scaling where these companies have been able to... So, okay, you picture a chip that has a processor on it. Well, that chip is not made as a chip, it's made on a wafer. And using photolithography, you basically print the same pattern on different dyes all across the wafer, multiple layers, tens, probably 100 some layers in a mature foundry process.

Speaker 2

09:55

And you do this on ever bigger wafers too. That's another aspect of scaling that's occurred in the last several decades. So now you have this 300 millimeter wafer, it's like as big as a pizza and it has maybe a thousand processors on it. And then you dice that up using a saw.

Speaker 2

10:08

And now you can sell these things so cheap because the manufacturing process was so streamlined. I think a technology as revolutionary as silicon microelectronics has to have that kind of manufacturing scalability, which I will just emphasize, I believe is enabled by physics. Of course, there's human ingenuity that goes into it, but at least from my side, where I sit, it sure looks like the physics of our universe allows us to produce that. And we've discovered how, more so than we've invented it, although of course we have invented it, humans have invented it, but it's almost as if it was there waiting for us to discover it.

Speaker 1

10:53

You mean the entirety of it, or are you specifically talking about the techniques of photolithography, like the optics involved? I mean the entirety of the scaling down to the 7 nanometers, you're able to have electrons not interfere with each other in such a way that you could still have gates. Like that's enabled to achieve that scale, spatial and temporal, seems to be very special and is enabled by the physics of our world.

Speaker 2

11:21

All of the things you just said. So starting with the silicon material itself, silicon is a unique semiconductor. It has essentially ideal properties for making a specific kind of transistor that's extraordinarily useful.

Speaker 2

11:35

So I mentioned that silicon has, well, when you make a transistor, you have this gate contact that sits on top of the conduction channel. And depending on the voltage you apply there, you pull more carriers into the conduction channel or push them away so it becomes more or less conductive. In order to have that work without just sucking those carriers right into that contact, you need a very thin insulator. And part of scaling has been to gradually decrease the thickness of that gate insulator so that you can use a roughly similar voltage and still have the same current voltage characteristics.

Speaker 2

12:12

So the material that's used to do that, or I should say was initially used to do that was silicon dioxide, which just naturally grows on the silicon surface. So you expose silicon to the atmosphere that we breathe and, well, if you're manufacturing, you're going to purify these gases. But nevertheless, what's called a native oxide will grow there. There are essentially no other materials on the entire periodic table that have as good of a gate insulator as that silicon dioxide.

Speaker 2

12:43

And that has to do with nothing but the physics of the interaction between silicon and oxygen. And if it wasn't that way, transistors could not, they could not perform in nearly the degree of capability that they have. And that has to do with the way that the oxide grows, the reduced density of defects there, its insulation, meaning essentially its energy gaps, you can apply a very large voltage there without having current leak through it. So that's physics right there.

Speaker 2

13:15

There are other things too. Silicon is a semiconductor in an elemental sense. You only need silicon atoms. A lot of other semiconductors, you need 2 different kinds of atoms, like a compound from group 3 and a compound from group 5.

Speaker 2

13:28

That opens you up to lots of defects that can occur where 1 atom's not sitting quite at the lattice site it is and it's switched with another 1 that degrades performance. But then also on the side that you mentioned with the manufacturing, we have access to light sources that can produce these very short wavelengths of light. How does photolithography occur? Well, you actually put this polymer on top of your wafer and you expose it to light, and then you use aqueous chemical processing to dissolve away the regions that were exposed to light and leave the regions that were not.

Speaker 2

14:06

And we are blessed with these polymers that have the right property where they can cause scission events where the polymer splits where a photon hits. Maybe that's not too surprising, but I don't know. It all comes together to have this really complex manufacturable ecosystem where very sophisticated technologies can be devised and it works quite well.

Speaker 1

14:33

And amazingly, like you said, with a wavelength at like 100 nanometers or something like that, you're still able to achieve on this polymer precision of whatever we said, 7 nanometers. I think I've heard like 4 nanometers being talked about or something like that. If we could just pause on this and we'll return to superconductivity, but in this whole journey from a history perspective, what do you think is the most beautiful at the intersection of engineering and physics to you in this whole process that we talked about with silicon and photolithography, things that people were able to achieve in order to push the Moore's law forward.

Speaker 1

15:12

Is it the early days, the invention of the transistor itself? Is it some particular cool little thing that maybe not many people know about? Like, what do you think is the most beautiful in this whole process, journey?

Speaker 2

15:26

The most beautiful is a little difficult to answer. Let me try and sidestep it a little bit and just say, what strikes me about looking at the history of silicon microelectronics is that when quantum mechanics was developed, people quickly began applying it to semiconductors. And it was broadly understood that these are fascinating systems and people cared about them for their basic physics, but also their utility as devices.

Speaker 2

15:55

And then the transistor was invented in the late 40s in a relatively crude experimental setup where you just crammed a metal electrode into the semiconductor and that was ingenious. These people were able to make it work, you know. But so, what I want to get to that really strikes me is that in those early days, there were a number of different semiconductors that were being considered. They had different properties, different strengths, different weaknesses.

Speaker 2

16:24

Most people thought germanium was the way to go. It had some nice properties related to things about how the electrons move inside the lattice. But other people thought that compound semiconductors with group 3 and group 5 also had really, really extraordinary properties that might be conducive to making the best devices. So there were different groups exploring each of these, and that's great, that's how science works.

Speaker 2

16:54

You have to cast a broad net. But then what I find striking is, why is it that silicon won? Because it's not that germanium is a useless material and it's not present in technology or compound semiconductors. They're both doing exciting and important things, slightly more niche applications, whereas Silicon is the semiconductor material for microelectronics, which is the platform for digital computing, which has transformed our world.

Speaker 2

17:22

Why did silicon win? It's because of a remarkable assemblage of qualities that no 1 of them was the clear winner, but it made these sort of compromises between a number of different influences. It had that really excellent gate oxide that allowed us to make MOSFETs, these high-performance transistors, so quickly and cheaply and easily without having to do a lot of materials development. The band gap of silicon is actually, so in a semiconductor, there's an important parameter which is called the band gap, which tells you, if you, there are sort of electrons that fill up to 1 level in the energy diagram.

Speaker 2

18:04

And then there's a gap where electrons aren't allowed to have an energy in a certain range. And then there's another energy level above that. And that difference between the lower sort of filled level and the unoccupied level, that tells you how much voltage you have to apply in order to induce a current to flow. So with germanium, that's about 0.75 electron volts.

Speaker 2

18:27

That means you have to apply 0.75 volts to get a current moving. And it turns out that if you compare that to the thermal excitations that are induced just by the temperature of our environment, that gap's not quite big enough. You start to use it to perform computations, it gets a little hot and you get all these accidental carriers that are excited into the conduction band and it causes errors in your computation. Silicon's band gap is just a little higher, 1.1 electron volts, but you have an exponential dependence on the number of carriers that are present that can induce those errors, it decays exponentially with that voltage.

Speaker 2

19:08

So just that slight extra energy in that band gap really puts it in an ideal position to be operated in the conditions of our ambient environment.

Speaker 1

19:20

It's kind of fascinating that, like you mentioned, errors decrease exponentially with the voltage. So it's funny because this error thing comes up when you start talking about quantum computing. It's kind of amazing that everything we've been talking about, the errors, as we scale down, seems to be extremely low.

Speaker 1

19:42

And like all of our computation is based on the assumption that it's extremely low. So it's

Speaker 2

19:48

not- Well, it's digital computation.

Speaker 1

19:49

Digital, sorry, digital computation. So as opposed to our biological computation in our brain, it's like the assumption is stuff is gonna fail all over the place, and we somehow have to still be robust to that.

Speaker 2

20:01

That's exactly right.

Speaker 1

20:03

So this also, this is gonna be the most controversial part of our conversation where you're gonna make some enemies. So let me ask, because we've been talking about physics and engineering. Which group of people is smarter and more important for this 1.

Speaker 1

20:17

Let me ask the question in a better way. Some of the big innovations, some of the beautiful things that we've been talking about, how much of it is physics? How much of it is engineering? My dad is a physicist and he talks down to all the amazing engineering that we're doing in the artificial intelligence and the computer science and the robotics and all that space.

Speaker 1

20:39

So we argue about this all the time. So what do you think?

Speaker 2

20:42

Who gets more credit? I'm genuinely not trying to just be politically correct here. I don't see how you would have any of what we consider the great accomplishments of society without both.

Speaker 2

20:54

You absolutely need both of those things. Physics tends to play a key role earlier in the development and then engineering, optimization, these things take over. And I mean, the invention of the transistor, or actually even before that, the understanding of semiconductor physics that allowed the invention of the transistor, that's all physics. So if you didn't have that physics, you don't even get to get on the field.

Speaker 2

21:20

But once you have understood and demonstrated that this is in principle possible, more so as engineering. Why we have computers more powerful than old supercomputers in each of our phones, that's all engineering. And I think I would be quite foolish to say that that's not valuable, that's not a great contribution.

Speaker 1

21:46

It's a beautiful dance. Would you put like silicon, the understanding of the material properties in the space of engineering? Like how does that whole process work to understand that it has all these nice properties or even the development of photolithography?

Speaker 1

22:02

Is that basically, would you put that in a category of engineering?

Speaker 2

22:06

No, I would say that it is basic physics, it is applied physics, it's material science, it's x-ray crystallography, it's polymer chemistry, it's everything, I mean.

Speaker 1

22:18

So chemistry even is thrown in there?

Speaker 2

22:20

Absolutely, yes. Yes, absolutely.

Speaker 1

22:22

Just no biology.

Speaker 2

22:24

Okay, we can get to biology.

Speaker 1

22:26

Well, the biology is in the humans that are engineering the system. So it's all integrated deeply. Okay, so let's return, you mentioned this word, superconductivity, so what does that have to do with what we're talking about?

Speaker 2

22:38

Right, okay, so in a semiconductor, as I tried to describe a second ago, you can sort of induce currents by applying voltages and those have sort of typical properties that you would expect from some kind of a conductor. Those electrons, they don't just flow perfectly without dissipation. If an electron collides with an imperfection in the lattice or another electron, it's gonna slow down, it's gonna lose its momentum.

Speaker 2

23:06

So you have to keep applying that voltage in order to keep the current flowing. In a superconductor, something different happens. If you get a current to start flowing, it will continue to flow indefinitely. There's no dissipation.

Speaker 2

23:19

So that's crazy. How does that happen? Well, it happens at low temperature, and this is crucial. It has to be a quite low temperature.

Speaker 2

23:30

And what I'm talking about there, For essentially all of our conversation, I'm going to be talking about conventional superconductors, sometimes called low TC superconductors, low critical temperature superconductors. And so those materials have to be at a temperature around, say, around 4 Kelvin. I mean, their critical temperature might be 10 Kelvin, something like that, but you want to operate them at around 4 Kelvin, 4 degrees above absolute 0. And what happens at that temperature, at very low temperatures in certain materials, is that the noise of atoms moving around the lattice vibrating, electrons colliding with each other, that becomes sufficiently low that the electrons can settle into this very special state.

Speaker 2

24:18

It's sometimes referred to as a macroscopic quantum state because if I had a piece of superconducting material here, let's say niobium is a very typical superconductor. If I had a block of niobium here and we cooled it below its critical temperature, all of the electrons in that superconducting state would be in 1 coherent quantum state. The wave function of that state is described in terms of all of the particles simultaneously, but it extends across macroscopic dimensions, the size of whatever block of that material I have sitting here. And the way this occurs is that, Let's try to be a little bit light on the technical details, but essentially the electrons coordinate with each other.

Speaker 2

25:08

In this macroscopic quantum state, 1 can quickly take the place of the other. You can't tell electrons apart, they're what's known as identical particles. So if this electron runs into a defect that would otherwise cause it to scatter, it can just sort of almost miraculously avoid that defect because it's not really in that location. It's part of a macroscopic quantum state and the entire quantum state was not scattered by that defect.

Speaker 2

25:37

So you can get a current that flows without dissipation and that's called a super current. That's sort of just very much scratching the surface of superconductivity. There's very deep and rich physics there, which is probably not the main subject we need to go into right now. But it turns out that when you have this material, You can do usual things like make wires out of it, so you can get current to flow in a straight line on a chip, but you can also make other devices that perform different kinds of operations.

Speaker 2

26:11

Some of them are kind of logic operations like you'd get in a transistor. The most common or most, I would say, diverse in its utility component is a Josephson junction. It's not analogous to a transistor in the sense that If you apply a voltage here, it changes how much current flows from left to right, but it is analogous in sort of a sense of, it's the go-to component that a circuit engineer is going to use to start to build up more complexity.

Speaker 1

26:44

So these junctions serve as gates?

Speaker 2

26:49

They can serve as gates. I'm not sure how concerned to be with semantics, but let me just briefly say what a Josephson junction is and we can talk about different ways that they can be used. Basically, if you have a superconducting wire and then a small gap of a different material that's not superconducting, an insulator or normal metal, and then another superconducting wire on the other side, that's a Josephson junction.

Speaker 2

27:16

So it's sometimes referred to as a superconducting weak link. So you have this superconducting state on 1 side and on the other side, and the superconducting wave function actually tunnels across that gap. And when you create such a physical entity, it has very unusual

Speaker 1

27:38

current voltage characteristics. Within that gap, like weird stuff happens.

Speaker 2

27:43

Through the entire circuit, So you can imagine, suppose you had a loop set up that had 1 of those weak links in the loop. Current would flow in that loop, independent, even if you hadn't applied a voltage to it, and that's called the Josephson effect. So the fact that there's this phase difference in the quantum wave function from 1 side of the tunneling barrier to the other induces current to flow.

Speaker 1

28:05

So how does he change state?

Speaker 2

28:07

Right, exactly. So how do you change state? Now picture if I have a current bias coming down this line of my circuit and there's a Josephson junction right in the middle of it.

Speaker 2

28:18

And now I make another wire that goes around the Josephson junction. So I have a loop here, a superconducting loop. I can add current to that loop by exceeding the critical current of that Josephson junction. So like any superconducting material, it can carry this super current that I've described, this current that can propagate without dissipation, up to a certain level.

Speaker 2

28:41

And if you try and pass more current than that through the material, it's going to become a resistive material, a normal material. So in the Josephson junction, the same thing happens. I can bias it above its critical current, and then what it's gonna do, it's going to add a quantized amount of current into that loop. And what I mean by quantized is, it's going to come in discrete packets with a well-defined value of current.

Speaker 2

29:11

So in the vernacular of some people working in this community, you would say, you pop a flux on into the loop. So a flux on.

Speaker 1

29:21

You pop a flux on into the loop.

Speaker 2

29:23

Yeah, so a flux on.

Speaker 1

29:24

Sounds like skateboarder talk, I love it. Sorry, go ahead.

Speaker 2

29:28

A flux on is 1 of these quantized sort of amounts of current that you can add to a loop. And this is a cartoon picture, but I think it's sufficient for our purposes.

Speaker 1

29:38

So which, maybe it's useful to say, what is the speed at which these discrete packets of current travel? Because we'll be talking about light a little bit. It seems like the speed is important.

Speaker 2

29:50

The speed is important, that's an excellent question. Sometimes I wonder how you became so astute.

Speaker 1

30:00

Matrix 4 is coming out, so maybe that's related. I'm not sure. I'm dressed for the job.

Speaker 1

30:06

I was trying to get to become an extra on Matrix 4. It didn't work out. Anyway, so what's the speed of these packets?

Speaker 2

30:13

You'll have to find another gig. I know, I'm sorry. So the speed of the pack is actually these fluxons, these sort of pulses of current that are generated by Joseph's injunctions, they can actually propagate very close to the speed of light, maybe something like a third of the speed of light.

Speaker 2

30:31

That's quite fast. So 1 of the reasons why Josephson junctions are appealing is because their signals can propagate quite fast and they can also switch very fast. What I mean by switch is perform that operation that I described where you add current to the loop, that can happen within a few tens of picoseconds. So you can get devices that operate in the hundreds of gigahertz range.

Speaker 2

30:58

And by comparison, most processors in our conventional computers operate closer to the 1 gigahertz range, maybe 3 gigahertz seems to be kind of where those speeds have leveled out.

Speaker 1

31:12

So- The gamers listening to this are getting really excited that overclocked their system to like, what is it, like 4 gigahertz or something? 100 sounds incredible. Can I just, as a tiny tangent, is the physics of this understood well, how to do this stably?

Speaker 2

31:28

Oh yes, the physics is understood well. The physics of Joseph's injunctions is understood well. The technology is understood quite well too.

Speaker 2

31:34

The reasons why it hasn't displaced silicon microelectronics in conventional digital computing, I think are more related to what I was alluding to before about the myriad practical, almost mundane aspects of silicon that make it so useful. You can make a transistor ever smaller and smaller and it will still perform its digital function quite well. The same is not true of a Josephson junction. You really, they don't, they just, it's not the same thing that there's this feature that you can keep making smaller and smaller and it'll keep performing the same operations.

Speaker 2

32:08

This loop I described, any Josephson circuit, well, I want to be careful, I shouldn't say any Josephson circuit, but many Josephson circuits, the way they process information or the way they perform whatever function it is they're trying to do, maybe it's sensing a weak magnetic field, it depends on an interplay between the junction and that loop. And you can't make that loop much smaller. And it's not for practical reasons that have to do with lithography, it's for fundamental physical reasons about the way the magnetic field interacts with that superconducting material. There are physical limits that no matter how good our technology got, those circuits would, I think, would never be able to be scaled down to the densities that silicon microelectronics can.

Speaker 1

32:54

I don't know if we mentioned, is there something interesting about the various superconducting materials involved, or is it all?

Speaker 2

33:00

There's a lot of stuff that's interesting.

Speaker 1

33:02

And it's not silicon?

Speaker 2

33:04

It's not silicon,

Speaker 1

33:05

no. So like it's some materials that also required to be super cold, 4 Kelvin and

Speaker 2

33:11

so on.

Speaker 1

33:12

Yes, yes,

Speaker 2

33:12

yes. So let's dissect a couple of those different things. The super cold part, let me just mention for your gamers out there that are trying to clock it at 4 gigahertz and would love to go to 400.

Speaker 1

33:21

Yeah, what kind of cooling system can achieve 4 Kelvin?

Speaker 2

33:24

4 Kelvin, you need liquid helium. And so liquid helium is expensive, it's inconvenient. You need a cryostat that sits there and the energy consumption of that cryostat is impracticable for, it's not going in your cell phone.

Speaker 2

33:39

So you can picture holding your cell phone like this and then something the size of a keg of beer or something on your back to cool it. Like that makes no sense. So if you're trying to make this in consumer devices, electronics that are ubiquitous across society, superconductors are not in the race for that.

Speaker 1

33:59

For now, but you're saying, so just to frame the conversation, maybe the thing we're focused on is computing systems that serve as servers, like large systems.

Speaker 2

34:10

Yes, large systems. So then you can contrast what's going on in your cell phone with what's going on at 1 of the supercomputers. Colleague Katie Schumann invited us out to Oak Ridge a few years ago, so we got to see Titan and that was when they were building Summit.

Speaker 2

34:26

So these are some high performance supercomputers out in Tennessee, and those are filling entire rooms the size of warehouses. So once you're at that level, okay, there you're already putting a lot of power into cooling. Cooling is part of your engineering task that you have to deal with. So there it's not entirely obvious that cooling to 4 Kelvin is out of the question.

Speaker 2

34:50

It has not happened yet, and I can speak to why that is in the digital domain if you're interested. I think it's not going to happen. I don't think superconductors are going to replace semiconductors for digital computation. There are a lot of reasons for that, but I think ultimately what it comes down to is all things considered, cooling, errors, scaling down to feature sizes, all that stuff, semiconductors work better at the system level.

Speaker 1

35:19

Is there some aspect of just curious about the historical momentum of this? Is there some power to the momentum of an industry that's mass manufacturing using a certain material? Is this like a Titanic shifting?

Speaker 1

35:33

Like what's your sense? When a good idea comes along, how good does that idea need to be for the Titanic to start shifting?

Speaker 2

35:42

That's an excellent question. That's an excellent way to frame it. And you know, I don't know the answer to that.

Speaker 2

35:51

But what I think is, okay, so the history of the superconducting logic goes back to the 70s. IBM made a big push to do superconducting digital computing in the 70s. And they made some choices about their devices and their architectures and things that in hindsight were kind of doomed to fail. And I don't mean any disrespect for the people that did it.

Speaker 2

36:12

It was hard to see at the time. But then Another generation of superconducting logic was introduced, I want to say the 90s, someone named Likorev and Semenov. They proposed an entire family of circuits based on Joseph's injunctions that are doing digital computing based on logic gates and or not, these kinds of things. And they showed how it could go hundreds of times faster than silicon microelectronics.

Speaker 2

36:43

And it's extremely exciting. I wasn't working in the field at that time, but later when I went back and read the literature, I was just like, wow, this is so awesome. And so you might think, well, the reason why it didn't displace silicon is because silicon already had so much momentum at that time. But that was the 90s, silicon kept that momentum because it had the simple way to keep getting better.

Speaker 2

37:06

You just make features smaller and smaller. So, you know, it would have to be, I don't think it would have to be that much better than silicon to displace it. But the problem is it's just not better than silicon. It might be better than silicon in 1 metric, speed of a switching operation or power consumption of a switching operation.

Speaker 2

37:24

But building a digital computer is a lot more than just that elemental operation. It's everything that goes into it, including the manufacturing, including the packaging, including the various materials aspects of things. So the reason why, and even in some of those early papers, I can't remember which 1 it was, Likorev said something along the lines of, you can see how we could build an entire family of digital electronic circuits based on these components. They could go 100 or more times faster than semiconductor logic gates.

Speaker 2

37:59

But I don't think that's the right way to use superconducting electronic circuits. He didn't say what the right way was, but he basically said digital logic trying to steal the show from silicon is probably not what these circuits are most suited to accomplish.

Speaker 1

38:16

So if we can just linger on, you used the word computation. When you talk about computation, how do you think about it? Do you think purely on just the switching, or do you think something a little bit larger scale, a circuit taken together, performing the basic arithmetic operations that are then required to do the kind of computation that makes up a computer?

Speaker 1

38:41

Because when we talk about the speed of computation, is it boiled down to the basic switching or is there some bigger picture that you're thinking about?

Speaker 2

38:49

Well, all right, so maybe we should disambiguate. There are a variety of different kinds of computation. I don't pretend to be an expert in the theory of computation or anything like that.

Speaker 2

39:00

I guess it's important to differentiate though between digital logic, which represents information as a series of bits, binary digits, which you can think of them as zeros and ones or whatever. Usually they correspond to a physical system that has 2 very well separated states, and then other kinds of computation, like we'll get into more the way your brain works, which it is, I think, indisputably processing information. But where the computation begins and ends is not anywhere near as well defined. It doesn't depend on these 2 levels.

Speaker 2

39:39

Here's a 0, here's a 1. There's a lot of gray area that's usually referred to as analog computing. Also in conventional digital computers or digital computers in general, you have a concept of what's called arithmetic depth, which is jargon that basically means how many sequential operations are performed to turn an input into an output. And those kinds of computations in digital systems are highly serial, meaning that data streams, they don't branch off too far to the side.

Speaker 2

40:16

You do, you have to pull some information over there and access memory from here and stuff like that. But by and large, the computation proceeds in a serial manner. It's not that way in the brain. In the brain, you're always drawing information from different places.

Speaker 2

40:31

It's much more network-based computing. Neurons don't wait for their turn. They fire when they're ready to fire. And so it's asynchronous.

Speaker 2

40:39

So 1 of the other things about a digital system is you're performing these operations on a clock. And that's a crucial aspect of it. Get rid of a clock in a digital system, nothing makes sense anymore. The brain has no clock.

Speaker 2

40:51

It builds its own timescales based on its internal activity. So- So you can think

Speaker 1

40:56

of the brain as kind of, I like this, like network computation where it's actually really trivial, simple computers, just a huge number of them and they're networked.

Speaker 2

41:08

I would say it is complex, sophisticated little processors and there's a huge number

Speaker 1

41:14

of them.

Speaker 2

41:14

Neurons are not- No offense,

Speaker 1

41:16

I don't mean to offend you. Sure, no. They're very complicated and beautiful.

Speaker 1

41:19

And yeah, but we often oversimplify them. Yes, they're actually like there's computation happening within a neuron.

Speaker 2

41:25

Right, so I would say to think of a transistor as the building block of a digital computer is accurate. You use a few transistors to make your logic gates. You build up more, you build up processors from logic gates and things like that.

Speaker 2

41:39

So you can think of a transistor as a fundamental building block, or you can think of, as we get into more highly parallelized architectures, you can think of a processor as a fundamental building block. To make the analogy to the neuro side of things, a neuron is not a transistor. A neuron is a processor. It has synapses.

Speaker 2

41:58

Even synapses are not transistors, but they are lower on the information processing hierarchy in a sense. They do a bulk of the computation, but neurons are entire processors in and of themselves that can take in many different kinds of inputs on many different spatial and temporal scales and produce many different kinds of outputs so that they can perform different computations in different contexts.

Speaker 1

42:24

So this is where enters this distinction between computation and communication. So you can think of neuron as performing computation and the inter, the networking, the interconnectivity of neurons is communication between neurons. And you see this with very large server systems.

Speaker 1

42:43

I mentioned offline, I've been talking to Jim Keller whose dream is to build giant computers that, you know, the bottleneck there is often the communication between the different pieces of computing. So in this paper that we mentioned, Optoelectronic Intelligence, you say electrons excel at computation while light is excellent for communication. Maybe you can linger and say, in this context, what do you mean by computation and communication? What are electrons?