Carver Mead Says Neuromorphic Efficiency Can Help AI - EE Times Podcast
In this episode of Brains and Machines, we chat with Carver Mead about parallelism, neural net efficiency and risk taking.
Photo Credit: Lance Hayashida (Photographer) and Sonia Chernobieff (Photoshoot Organizer)
In this episode of Brains and Machines, we chat with Carver Mead about parallelism, neural net efficiency and risk taking.
Carver Mead has just finished writing a new paper. At age 90, the renowned pioneer of semiconductor electronics is tackling, with a small group of optics experts, lingering questions rooted in Einstein’s theory of general relativity that concern the effects of gravity on the speed of light.
Neuromorphic engineering is finally getting closer to market reality, propelled by the AI/ML-driven need for low-power, high-performance solutions.
Carver Mead says scientists can either get too much credit or no credit.
"Large-scale adaptive analog systems are more robust to component degradation and failure than are more conventional systems, and they use far less power...For this reason, adaptive analog technology can be expected to utilize the full potential of wafer-scale silicon fabrication."
— Carver Mead, Proceedings of the IEEE (1990)
Originally conceived by engineering guru Carver Mead in the 1980s, neuromorphic computing and its previous incarnation, artificial neural networks, make use of specialized chips that are inspired by the computational functions of the brain.
Information technology, powered by Moore’s Law, provided nearly all the productivity growth of the last 40 years and promises to transform industries, such as health care and education, which desperately need creative disruption. It has powered a wave of globalization that helped bring two billion people out of material poverty and has delivered the gift of knowledge to billions more.
Mead wasn’t sure whether it was he or Waller who coined the term. “It sort of bubbled up in the discussion,” Mead told me, “and then it came out in the article and it stuck.”
"I always had to—especially in the early days—explain that [More's Law] is not a law of physics. This is a law [of] the way that humans are. In order for anything to evolve like our semiconductor technology has evolved, it takes an enormous amount of creative effort by a large number of smart people. They have to believe that effort is going to result in a successful thing or they won’t put the effort in. That belief that it’s possible to do this thing is what causes the thing to happen.
The Moore’s Law thing is really about people’s belief in the future and their willingness to put energy into causing that thing to come about. It’s a marvelous statement about humanity."
— Carver Mead
Mead summed up the energy and enthusiasm of Silicon Valley when he said that, over the first 50 years of Moore’s Law, we have built an amazing platform of computation and networking capability, and we have only scratched the surface of how we can take advantage of it. He views that as our mission for the next 50 years.
By the time Carver Mead coined the term Moore’s Law around 1975, Moore himself had already modified it. Even though Moore never expected his projections to be very precise, it had nearly perfectly predicted semiconductor progress for a decade. However, Moore felt that gains in component density would begin to taper off, and suggested that by 1980 a doubling every two years was a more likely prospect.
"It’s a common theme in technology evolution that what makes a group or company or field successful becomes an impediment to the next generation. … Everyone was richly rewarded for making things run faster and faster with lots of power. Going to multicore chips helped, but now we’re up to eight cores and it doesn’t look like we can go much further. People have to crash into the wall before they pay attention."
— Carver Mead
Carver Mead christened Moore’s Law and helped make it come true. Now he says engineers should experiment with quantum mechanics to advance computing.
[O]ne of the most important and often understated parts of the Mead-Conway legacy is that after more than 30 years and with the design process of integrated circuits going through many transformations, a large fraction of it still remains fresh and relevant to today’s design of information-processing devices.
"Much more work needs to be done to restart that revolution, Mead said, with the goal of explaining in an intuitive way how all matter in the universe relates to and affects all other matter, and how to explore those interrelationships in a way that isn't 'buried in enormous piles of obscure mathematics.'"
— Rik Myslewski, The Register, February 20, 2013
If we want to get that stalled 100-year-old revolution unstuck, Mead said, we've got to ask – and discover – where those constants come from, and not just believe in them as handed down by academics and buried in mountains of math. We need to discover their basis in the interactions and interrelationships of all matter in the universe.
You need to question the assumptions of today's science to "finish the revolution" in physics and keep technology moving forward, Carver Mead told an audience of well over a thousand semiconductor engineers here. Accepted constants such as the speed of light need to be re-examined...
"We have been living with misconceptions and gobbledygook, of thought processes not allowed to go forward...People couldn’t imagine the world being as interesting as it is, partly because some big egos got in the way...Our view of science, that got us this far, is keeping us from going further...There's a bigger conceptual picture we need to integrate into our thinking."
Electronics pioneers, including Carver Mead at the California Institute of Technology in Pasadena, pushed for academics and small companies to gain access to fabrication facilities ('fabs') to build the integrated circuits they needed to test their ideas and train students. At the time, academics were restricted to making simple devices in the lab, doing theoretical work or — as Mead did — having friends at Intel sneak their students' designs into production runs. "You have to have access to a fab to build the real thing," says Mead.
[Carver] Mead started a foundry service for researchers to share the cost of manufacturing prototype semiconductors, a program that inspired the UW's new "OpSIS" silicon photonics foundry service. The service will be used by researchers and companies developing chips with lasers that transmit digital signals with light at phenomenal speeds.
"By the mid-1970s, digital system designers eager to create higher-performance devices were frustrated by having to use off-the-shelf large-scale-integration logic," according to Electronic Design magazine, which inducted Mead and Conway into its Hall of Fame in 2002.
The situation at the time stymied designers' efforts "to make chips sufficiently compact or cost-effective to turn their very large-scale visions into timely realities." But then Conway and Mead introduced their very large-scale integration (VLSI) methods for combining tens of thousands of transistor circuits on a single chip.
Moore’s original article didn’t explicitly state the law in layperson’s language and never referred to it as a law. It was Mead who boiled down Moore’s technical observations about circuit integration and coined the term “Moore’s law.” Since then, Moore’s law has captured the popular imagination to the extent that it’s often misquoted and misapplied to other technologies.
Conventional wisdom descries a black hole between the infinite uncertainty of modern theoretical physics and the can-do spirit of entrepreneurship and engineering. One more reason to ignore conventional wisdom, says Carver Mead, who became a technology legend by working both sides of what often seems an uncrossable divide. A Caltech stalwart – he is the emeritus Gordon and Betty Moore Professor of Engineering and Applied Science – Mead is one of the seminal figures in the story of Silicon Valley, with a résumé stretching back to integrated-circuit pioneer Fairchild Semiconductor and more than 20 startups to his credit.
"I spent my whole career trying to open the world of technology to students. I would love to leave the world with a more intuitive understanding of science and technology. It is wonderful when students get it. We owe our modern civilization to the ones who do get it."
— Carver Mead
"Large Scale adaptive analogue systems are more robust to component degradation and failure than are more conventional systems, and they use far less power. For this reason, adaptive analogue technology can be expected to utilise the full potential of wafer silicone fabrication."
— Carver Mead
[Carver Mead] wondered, why not take advantage of a fundamental property of silicon -- that light of different colors penetrates silicon to different depths? The solution could be as simple as creating separate photodetectors at different depths in the silicon chip.
Carver [Mead] has invented color processing chips that do for color photography and video what Dolby did for sound...Film is toast. Carver Mead killed it.
When we grasp onto some regularity, the temptation is to think we understand it. But the truth is we're still not even close.
The entire photographic process is performed in analog; there is no sacrifice of critical information until the image is converted to digital form for storage. In this operation, resolution can be arbitrarily high. Foveon oversamples its analog pixels in order to create a digital representation that can be manipulated without distortions.
The award is being given to Mead this year "for his many contributions to the field of microelectronics, which have led to a new business model for the industry and enabled a new wave of innovation in information technology."
"Carver Mead has made many diverse and seminal contributions to the computing and semiconductor industries through the development of semiconductors, their design, and specific, archetypal chip designs. Mead's most important contribution enabled the design of digital chips through the concept of lambda scaling, and the first silicon compiler. This technology is the foundation that allows the modern semiconductor industry to design and build complex, VLSI chips."
I am also sure that the single-chip RISC concept would probably not have emerged as quickly or as dramatically as it did were it not for Carver [Mead]'s pioneering work, especially in the area of structured custom design. Making silicon accessible to system designers, leading to a set of abstractions and methodologies that were to influence a generation of EDA technologies, is certainly one of Carver's greatest legacies.
Good Science is a form of channeling, only with science, you also have to go work the equations. It sort of goes back and forth: You get all the inspiration, but then you have to make sure the science works. So I weave back and forth, working both sides, to try to get them to be the same. They're not different, but they're different modes you get into.
"All the people attending [the course] were systems designers rather than chip designers. I could start fresh, with the right conceptual underpinnings, and give a clear picture of the prospects for compilation."
— Carver Mead
The simplest brains can do things we don't know how to do and in ways we don't understand. However, I believe that there is nothing done by the human nervous system that couldn't be done by computers if we totally understood how the nervous system works.
The reason that one must start from the bottom, is that the bottom is where the necessary invariants get built into the internal representation of information. And achieving the proper representation will allow systems to recognize the objects of vision or speech as people do.
Mead goes on to say of coming developments: “We’re not going to need the federal government to come in and bail out all our electronics. We’re going to do just fine, thank you.” Well, I know what you’re thinking, and it’s true. That last remark warmed my heart considerably.
But what does [the] technological revolution mean for the future of the world order? It means that nations will have to grant to their scientists complete freedom of inquiry; to their businessmen and entrepreneurs, freedom to invest, to risk, to create new products and with them new markets; to their entire economies, the freedom to grow and grow, unburdened by heavy taxation and unimpeded by needless regulation.
— President Ronald Reagan
[Carver Mead and Lynn Conway] decided that n-channel MOS was the ideal technology for their methodology and also decided to bypass Boolean logic gates as an intermediate step. Replacing them: simple fieldeffect transistor switches and such devices as stacks, barrel shifters, and functional blocks that are replicated to form larger subsystems.
We are so attached to the idea of the big number-crunching machine for storing information that we don't yet see the real power of the new microelectronics technology.