The End of Moore’s Law and the Future of Computers: My Long-Read Q&A with Neil Thompson

By James Pethokoukis and Neil Thompson

Moore’s law, which states that the number of transistors on a microchip doubles every two years, has fueled rapid computing gains since the mid-20th century. But will this law last forever? Today’s guest, Neil Thompson, thinks its end is near. I’ve invited Neil on the podcast to explain why Moore’s Law may be coming to an end and what that means for productivity growth and continued innovation.

Neil is an innovation scholar in MIT’s Computer Science and Artificial Intelligence Laboratory, a research scientist at the MIT Initiative on the Digital Economy, and an associate member of the Broad Institute.

What follows is a lightly edited transcript of our conversation, including brief portions that were cut from the original podcast. You can download the episode here, and don’t forget to subscribe to my podcast on iTunes or Stitcher. Tell your friends, leave a review.

Pethokoukis: Let’s start with a basic question, and then maybe a somewhat harder question: What is Moore’s Law, and what does the world look like today if it didn’t exist?

Thompson: So Moore’s Law is an incredibly important trend, which is sort of used to talk about broadly all of the improvement in computing we’ve had in the last five or six decades. The actual origins of it, though, come from basically the miniaturization of computer chips and the elements that are on computer chips. And it was kind of a neat moment where Richard Feynman, the Nobel Prize-winning physicist, gave this speech back in 1959. By the way, this was an after-dinner speech, if you believe it. This was like, you’re at a conference and someone was just saying, “Hey, by the way, why don’t you say a few remarks?”

That’s great.

And in this talk, he says, “Okay, let me tell you about nanotechnology. Because we’re doing all these things in this sort of regular world, and I think we could just keep miniaturizing stuff all the way down to the point where it’s just a few atoms.” And he said that about computers, right?

He said, “We can make them and these parts will be really small, like 10 or a 100 atoms big.” And this was a pretty remarkable thing that he said. It’s a little hard, I think, to get the sense of how big the difference is between our everyday lives and that many atoms. But it’s roughly proportional to, if you were building things the size of the Earth, and you’re like, “You know what? I think we could build that the size of a tennis ball.” And that really laid out a roadmap for us as we improved our technology to say, “We could keep miniaturizing.” And it turned out as we did that, we were able to put more transistors on our chip, which meant we could do more and we could run our chips faster.

And a huge amount of the revolution we’ve had in IT all really comes from this. If we were still using the computers back in those early days, we really could not be doing almost anything that we’re doing today. So it has had an enormous effect on society. And yeah, it has really been very transformational.

I’m old enough that I’m pretty sure I may have watched some sort of film strip or something in school about computers, and they were still pretty big. So this is probably like the ‘70s. It may have been in black and white, so maybe it wasn’t state of the art. But it was classic: giant, monolithic computers with punch cards and things like that. So maybe we’d still be in that world. I don’t know.

I remember those moments too. My mother used to do some of her research with a computer and she’d come back with extra punch cards when they were getting rid of the computer, and we had these things as our notepads.

That’s great. If I did a news search on Moore’s Law and I searched with a phrase like “coming to an end,” I would find many stories over many years about the end of Moore’s Law. Have all those reports of its death been greatly exaggerated, or are we finally there?

Yes, you’re absolutely right. You can go back decades and find people saying, “Well, this is going to be a problem,” or “This is going to be a problem.” And actually, it’s a real credit to the technologists and engineers that they were able to push through that and get past it. But since 2004, what is clear is we’ve lost many of the benefits of Moore’s Law.

Let me make the distinction here, because when Gordon Moore actually made up this law, it was really about the number of transistors you could fit on a chip, which is a very specific, technical thing about what you can do. But what it translated to, generally, was this incredible speed up in the capacity of our chips and how fast we could run them. And so we sort have taken to calling all of those things Moore’s Law. But in practice, actually in 2004–2005, we lost, I think, one of the most important parts of that, which is the speed up on our chips.

So chips at that point were about 3 gigahertz. The chips in your computer today are about 3 gigahertz. And so we’ve really plateaued, whereas before that we were improving it exponentially. So we really have lost a lot of it already. It’s also very clear that we are pretty close to the end, even being able to get more transistors on. And that’s not just me saying that. The people who designed the roadmaps to figure out, “What are the technologies that we need to put together in order to make the next generation of Moore’s Law happen?—those folks have already said, “Whoa, whoa, whoa, that can’t be the path anymore.” So it’s no longer just lone voices in the cold saying it’s going to end. Now, it’s a lot of the community.

Speed and the number of transistors on a chip: I think those are two different things. How are they different?

I think a useful way to think about this is to think about that miniaturization that Feynman had proposed. Every time you shrink those transistors, you can fit more on a chip. And that’s just geometry, right? We know that if we’ve sort of taken up less space, you can fit more on. So that’s really good. And that allows you to have things like more memory, for example, on your chip. And so if you think about cache—that may be something people see when they’re buying their computers—the size of cache has been going up a lot over time. So that’s really good.

But the speed of your computer is also based on how many operations you can do in one second. So you do one thing, and then you do the next thing, and you do the next thing. And one of the things that modern computers are really good at is doing things way, way faster than we can. So if you think about our clock being one second, they can do three billion things in that one second. Three billions sets of operations. It’s enormously fast, but it turns out that also came from the miniaturization that was happening with Moore’s Law.

The limit to running chips faster is always that as you run them faster, they produce more heat. And the problem is if you produce a lot of heat, you’ll eventually melt the chip. So there’s a limit to how fast you can sort of toggle it up to be able to run it faster. And you hit that limit and then you stop. And the nice thing is, as we miniaturized, it turns out we were able to run them faster, right? They produced less heat because they were smaller. And so you could turn up the speed a little bit and you just kept being able to do that over and over again until we hit this limit in 2004 and 2005.

I write a lot about the down shift in productivity growth in the United States and in other countries starting in the ‘70s. And then we had this kind of blip up in the late ‘90s and early 2000s. I think this down shift—and people can debate if productivity is being properly measured—happened at the same time as this amazing improvement in chip performance and the capabilities of computers. So first, do we know to what extent Moore’s Law contributed to productivity growth over the past half century? Do you happen to know if there’s a rough estimate? I’m kind of asking you to do this off the top of your head, so I apologize.

I think that there are sort of two ways to answer that question. There’s the sort of overall IT—what macroeconomists say the effect that IT is having on productivity—and by one estimate, since the 1970s, about a third of all labor productivity improvements have come from improvements in IT. That sort of gives you sort of one version of this.

Right.

But it’s very hard in a lot of ways to measure it, because Moore’s Law has been a pretty stable thing over time, and that tends to be a hard thing for economists to measure. But in some of the work that I did as part of my dissertation work, I can actually show, for example, that in this 2004–2005 era where we lose the speed up, it turns out a whole bunch of firms are hurt by that. Their productivities do not rise as fast as a result of this. And so we definitely can see these effects coming in. And myself and my group, the stuff that we work on, a lot of this is trying to get a much better estimate of this because we think this is actually a pretty crucial question. Because we may have actually been systematically underestimating the effect that IT has had and Moore’s Law has had on the economy.

Given the fact that we may have underestimated it, how concerned should I be going forward? If we’ve had this struggle of measured productivity growth, at least as currently measured, with Moore’s Law, how concerned should I be about productivity growth—which is pretty important for raising living standards going forward—with Moore’s Law at an end?

I am worried about this. I am absolutely worried about this. And the way I think of this is: If you look across society and you think of general productivity improvements, you’re talking 1 or 2 percent per year kind of as how we make things better over time. And at its peak, the improvement that we were making in our chips was 52 percent per year. So it was vastly faster. And that really had sort of spillover effects on everybody else. Everyone else could say, “Let me use computers to do this additional thing and make my part of the economy more productive as well.” And Dale Jorgenson at Harvard has done some nice work splitting that out and seeing how important it is.

And then as we get to the end, the question is, where do we go from here if this engine has been slowing down? And of course, there are some candidates. People talk about artificial intelligence: Are we going to be able to use that? Quantum computing: Are we going to be able to use that? And some of these other things. And the question is, do these things have the legs in the same way that Moore’s Law has had. And I think it’s not at all clear that any of them will be able to sort of take up that mantle in the way that Moore’s Law has done, particularly over so many decades.

Computers start out at the very beginning as a very specific technology developed for use in war—I’m sure you know a lot more about this than I did—whether it’s calculating artillery range or something. Then it became what economists call a general-purpose technology. You’ve written that it’s now transitioning back into a more specific technology where computers and chips become more special purpose. Do I have that right?

Yeah, that’s right. That’s right. So maybe a way to think about this is a Swiss Army knife versus having a whole toolkit full of hammers and screwdrivers and all those kind of things. So you could say, “I’m trying to make this decision, should I invest in a Swiss Army knife or should I buy a screwdriver and all those kind of things?”

I think in my toolbox and I’m sure in yours, we have the full set of things, right? We don’t just have this Swiss Army knife. And the reason that’s true is because, obviously if you specialize, your tool for one particular thing does that thing better. But the remarkable thing about what we got with these sort of general-purpose chips, which is the CPUs, over time was they got better so fast that the choice was more, “Well, do I want to buy a screwdriver today or do I want to buy a new Swiss Army knife four years from now that’s going to be vastly, vastly better?” And so we sort of kept on that path.

But it only works, that trade-off is only worth doing, as long as the Swiss Army knife is getting better fast enough: as long as the CPU is getting better fast enough. And what we’ve seen is a real breakdown in that. That is, as this slows down, we’ve gone from that 52 percent that I had told you before as the rate of growth. By one measure, we’re now down to 3.5 percent per year improvement. And so at that level, you’re much better off saying, “I want to get the screwdriver myself.” And we already see lots and lots of firms doing this. So Google is building their own chips. Amazon is building their own chips. Tesla is building their own chips. Lots of people are going down this road to build a specialized chip that is right for exactly what they want to do, not the one that we’re all on the same platform for.

You mentioned AI earlier. When you talk about AI, what are we talking about? Machine learning, deep learning? And is this world of more specialized chips just fine? If your specialty is deep learning, you have a deep learning chip. I’m probably wildly oversimplifying this. So does that have a negative impact? What do you mean by AI? And then what is the impact of a more specialized chip?

I think you’re so right to point us out. Let’s talk about what we mean by AI, because people mean everything from artificial general intelligence—which obviously has a whole bunch of particular implications—to I would say sort of a catch-all phrase where almost everything from data science these days gets wrapped in this blanket of artificial intelligence. Certainly within the last 10 years, the thing that has really changed remarkably is deep learning in particular.

And for those in your audience who haven’t seen this, this used to be called neural networks. This has existed back since the ‘50s, and it was just that when we didn’t have much computing power, they were kind of small and they didn’t have very many layers, which is sort of one of the ways that people think about them. And as we got more computing power, they got deeper, which is why we call them deep neural networks now. So that’s sort of how we get there. That’s really where the revolution has been in the last 10 years. And at the cutting edge, all of the models that are beating records and stuff are almost all deep learning models.

And does the slowing of Moore’s Law and the moving to more specialized chips have any impact on the progress of deep learning AI?

Yeah. So I mean, what you can see is that when you have something like deep learning where it’s clear that there’s potential here, people have invested in building these specialized chips. And so people may have heard of GPUs, graphics processing units. But people have built even more specialized [chips]. So Google has their own tensor processing unit, TPU.

So these sort of ever-more-specialized things are sort of taking us down this road of becoming more and more efficient at that particular thing. And so the good news about that is that, indeed, you get a big performance gain. So one of the results that Nvidia pointed out not too long ago was that they could get about a 100x improvement from the specialization that they were doing. So that’s on one hand, pretty great.

On the other hand, that’s not at all big compared to the sweep of Moore’s Law, right? And so you were asking, should I not be worried? And I think the answer is you should be worried because specialization gives you that one-time gain—and maybe incremental. Maybe you can build a screwdriver ever a little bit better, but you run out of steam pretty quickly. Whereas Moore’s Law had much, much more legs and many more decades of improvement that it could offer.

Do people think there’s a lot more progress to be made in deep learning? Or do they think it’s being exhausted already, and they’re trying to think of the next thing? Where are we in that revolution?

I think there are people in both camps on that. I think that there are lots of folks that are still excited by all the progress we’re making here, and I think they have some real things they can point to. The one that I’m particularly excited about recently is AlphaFold, which is this ability to model protein folding with deep learning. And that’s a remarkable achievement, right? It’s a problem that we struggled with for a long time. And it has a lot of benefits for how we do medicine in the future. So I think there’s a real promise there. And so they can say that’s great, lots to be done there.

At the same time, we do see us starting to run into limits on how we implement these things. We can run into times where the inherent inefficiency of deep learning—which we can talk about if you’re interested—that inherent inefficiency that comes with using deep learning comes with a very high computational price. And so you see people starting to say, “Well, do I really want to pay that price?” So one example that we saw very closely: We did an MBA case on this where a supermarket had said, “I’m really interested in using deep learning to predict the demand for my products.” And of course, that matters a lot for a supermarket because they’d want to know how much to stock on the shelves and the like.

And it turns out that they did it; they got a real improvement in performance. But for many of the products that they put on their shelves, it was not economically worth it to run that model. The computational cost of just running it overshadowed that. I think we’re seeing both of them and we’re seeing the second group of folks saying, “Well, maybe I’ll use deep learning in some places, but in other places I won’t, or maybe in some places I’ll try to adapt it so that it can become more efficient.”

If we’re at the end of Moore’s Law as we’ve thought about it and now we’re moving to specialized chips, is there a new sort of chip technology that would return us to these massive gains and return us to a general-purpose technology?

None that we know yet. There are candidates.

I’m sorry to hear that.

Yeah, me too. Me too. There are candidates, right? People have proposed architectural changes in the way we do our switches that might make things more efficient. I think there’s some possibility there. Although, again, probably not the decades and decades of Moore’s Law. People have talked about things like optical computing. So right now we have wires and electrons doing our calculations. Maybe we could use light and photons to do it. That seems like that might be interesting, particularly for some kinds of calculations. Then there are other things like quantum computing, which I think many people have the sense of quantum computing is just the next generation of computing.

I actually don’t think that’s right. I think it’s more like a different kind of specializer. So it’s like, we’re going to have our main computers and we’re still going to have them in the future. But then on the side, we’ll say, “Well, for a certain subset of problems, quantum computers can do really well, so we use the quantum computers for those.” So I think that’s what the landscape looks like right now. But as I say, I don’t think there are any of these that really look like they’re going to be the next general-purpose technology, for the moment.

I can’t remember a time when computer chips have been in the news and as much on the national news pages. There are concerns that not enough chips are made in the United States or not the best kind; they’re made in Taiwan, but what if China invades Taiwan? How hard is it to say, “Oh, now we want to make these things in the United States”?

I think some people think you take apart a factory in one place, you move it to the United States. Would that be a very significant change, if companies tried to make more of their chips here? And does that take a number of years? It sounds like it would be hard, harder than I think many politicians think.

It really depends on how cutting edge you want your chips to be. So I was talking before about that miniaturization that’s going on. And so the smaller you get, the harder it gets, the closer you’re getting to moving around individual atoms and things like that. And so if you want to be away from the cutting edge, actually, there are lots of places that can build that. And so that is more broadly available technology.

Neil, I want America to be on the cutting edge. I want us to be on the cutting edge.

Well, yes. I don’t blame you.

I don’t want a chip that runs a toy. I want the best.

The problem with that is building one of those factories these days costs $20 billion or $22 billion. So it’s a big deal. It’s really hard. You need very cutting-edge equipment to do it. The challenge there is that we probably did not worry about this as much in the past because there used to be 25 different companies, all of whom were on the frontier of building chips. And as these factories have become more and more expensive over time, what we’re down to is now basically three different companies that produce these cutting-edge chips. So it is very hard. There are not very many folks that do it at the cutting edge, but it certainly is important that we have good production facilities and that we know that they can be secure. Absolutely.

Neil, thanks for coming on the podcast.

My pleasure. Thanks for having me.

James Pethokoukis is the Dewitt Wallace Fellow at the American Enterprise Institute, where he writes and edits the AEIdeas blog and hosts a weekly podcast, “Political Economy with James Pethokoukis.” Neil Thompson is an innovation scholar in MIT’s Computer Science and Artificial Intelligence Laboratory, a research scientist at the MIT Initiative on the Digital Economy, and an associate member of the Broad Institute.