Singularity Institute

Vernor Vinge explains the concept of the Singularity and speculates about how humanity will react to it.

SI: What is the Singularity?

Vernor Vinge: Lots of people have definitions for the singularity that may differ in various ways. My personal definition for the singularity is that within the relatively near historical future I think humans, using technology, will be able to create or become creatures of superhuman intelligence. And I think the term "singularity" is appropriate for that because, unlike other technological changes, it seems to me pretty evident that this change would be unintelligible to us afterwards, in the same way that our present civilization is unintelligible to a goldfish.

SI: Haven't there been singularities throughout history?

VV: Folks, sometimes after hearing this, will come up and say, "Well, yeah, but there have been singularities before like, say, the printing press." The thing is, you could explain to somebody before Gutenberg what a printing press was and you could actually explain the consequences, even though the consequences might not be believed, and the listener would understand what you're saying. You could not explain a printing press to a goldfish or a flatworm. [chuckles]

In the same way, post-singularity, to have that explained to us now is a qualitatively different exercise than explaining past breakthroughs. That's why I would say, events like the printing press, the invention of fire, the invention of cities, agriculture - all of those are extreme events, but I think the right analogy to the technological singularity is the rise of humankind within the animal kingdom, or perhaps the rise of multicellular life within the Pre-Cambrian era.

SI: Is the Singularity near?

VV: I'd be surprised. I think the singularity is the most likely non-catastrophic event for the near future. That doesn't mean that it's certain. It doesn't mean that terrible things might happen instead. I personally would be surprised if it hadn't happened by 2030.

SI: Is it necessary to program AIs to be friendly?

VV: In my discussions with Eliezer, some of the most interesting discussions are in his notions of safety. We're all familiar with the science-fiction that Asimov wrote about the "Three Laws." A significant part of his career was analyzing the consequences of that. I think one thing that his stories show is the various difficulties with it. Eliezer's notion of a more nebulous kind of loyalty, or niceness, is much more plausible.

It certainly makes sense that, if this is the way we get into human equivalents and superhuman intelligence, then it may very well be that what he's talking about there would be quite important. I think that there are also a number of other ways that we may slide into superhuman intelligence. Some of them are relatively weird, and they have their different flavors.

SI: Should we be alarmed by the Singularity?

VV: Over the years, there's been various anxieties that people have had about the singularity. I think some of this has been attributed to science-fiction -- unfairly, of course! There has been science-fiction that's actually been quite optimistic about singularity-type events. But I think anytime you're contemplating something that can replace the most competitively effective feature that humans have -- that's intelligence -- it's entirely natural that there would be some real uneasiness about this.

I regard the singularity as the most, the nearest analogy in the history of the earth is the rise of humans within the animal kingdom. Anybody who thinks that has some reason to... you think about that, you say, "Brother, there are some things that might not be good for humans." On the other hand, I think it's also true -- and this is a reason for nervousness, but it also points to something larger - and that is, thinking about the possibility of creating or becoming something of superhuman intelligence is actually an example of optimism that is so far-reaching, that it forces one to carefully look at what they wanted.

In other words, humans have been striving to make their lives better for a very long time. We have certain things that are gut instincts about what, in the farthest remove, we want out of life and what we want out of the universe. It is very unsettling to realize that we may be entering an era when questions like, "What is the meaning of life?" are practical engineering questions. There's no doubt that that should be very unsettling. On the other hand, I think it might be healthy if we sit down and look at the things we really want, and look at what they would mean if we could get them and then move forward from there, in terms of trying to make certain things happen or guide things in certain ways.

SI: What signs would you look for that would indicate the Singularity is near?

VV: The notion of the signs of the singularity is something that grew out of a basic piece of methodology, and that is using scenarios instead of using trend projections. I see that the singularity as a whole is a type of scenario, but there are several paths into it that are very different, and perhaps would lead to very different outcomes. Then there are scenarios where the singularity doesn't happen.

For each scenario, for them to be useful scenarios, a person should consider what would be the symptoms you would see going forward, as you moved into the different scenarios. And then, as real time passes, you watch for those. This adjusts probabilities, and in some cases, if it's really effective planning, it may adjust what you're doing, depending on how you rate the possibilities.

So, as I said in that essay at IEEE Spectrum, there are a number of things a person could watch for, both negative symptoms and positive symptoms. Negative symptoms are ones that make you think, "Oh, wow, that decreases the likelihood that we are on the track for any near singularity." An example of a negative symptom is if you began to notice larger and larger software debacles. In fact, that's fun to write about things like that, you could get very large software debacles.

One of the simplest positive ones is just to note weather or not economic motive and the effect of Moore's law is continuing on track. Another thing to watch for is, to me, a fundamental thing that is going on here is that humans are better characterized not as the tool creating animal, but as maybe the only animal that has figured out how to outsource its cognition, how to spread its cognitive abilities into the outside world. We have only been doing that for a little while, like ten thousand years.

Reading and writing is outsourcing of memory. So what we have got here is this process and watch weather it is ongoing. So for instance, in the next 10 years, if you notice more and more substitution of fragments of human cognition into the outside world, if the human responsibility, occupational responsibility, becomes more and more focused on areas of judgment that have not yet been automated, then what you are seeing is rather like a rising tide of this cognitive outsourcing, and that would actually be a very powerful symptom.

SI: Would the Singularity be democratizing?

VV: There are certain paths to the singularity that would be highly democratizing. For instance one of the paths to the singularity is that humanity plus its networks and the data on those networks and the supporting computers that, that ensemble takes on an intellectual role. And actually, if you compare us to 1985, you already see much evidence of that.

I think the presentation tomorrow about Twine, I believe, is going to be talking about that. That's an example where, in effect, the people of the world who are participating, their intellectual power has always in a raw sense outweighed the best and the brightest that were the captains of state and the advisers to the captains of state. With the Internet, these people are finally coming together and talking to one another, and if you participate in the web at all, I think you are quite aware of that. That sort of intellectual power does not mean that governments go away, but it could very well be that it becomes the tail that wags the dog of the world's governments.

In that sense, it is very democratizing. In fact it is that particular scenario I sometimes characterize as being the new populism. Now this is actually only one type of path to the singularity. I suspect that as I said in the essay, there are four or five and they are probably going on concurrently and they will probably mix together. Some of them look very terribly alien. And terms like democratization and politics and such like that applied to some of these paths, in my mind provokes a "Huh?" reaction because it is just so terribly alien.

SI: Is the idea of a Singularity becoming mainstream?

VV: Fifty to sixty years ago it was a very strange idea. You could only find a relatively small number of people who would be talking about it, even among people who are talking about the wonders of future technology. If they talked about things like superhuman intelligence it was generally something that would happen a million years from now when our heads are as big as watermelons and it had grown to an IQ of 400. At that remove it is vaguely comforting in the sense that it shows that all of our struggle eventually lead to something good. Now we are in an era were we are thinking it might happen before it is time for you to retire and that makes people more nervous.

As time goes forward in the twentieth century, in fact I had an essay in '93 where I explicitly said this, the shadow of the singularity casts itself backwards in time and in the early years there were only a few people who talked about it. In the 80's and 90's there were more people who talked about it. Now, in the early 21st century there are a lot of people who talk about it. I suspect that going forward, as long as it doesn't happen, that it will be an idea that becomes a background, assuming that we don't get big physical disasters. If the world blows up, or the suns blows up, or we get a big nuclear war or something like that, that is different. But if technology proceeds, I suspect that until a singularity happens that the notion of the singularity will just become a growing part of the cultural landscape of anybody who is technology literate.

SI: Why support SIAI?

VV: The value of a Singularity Institute and a Singularity Summit in general consideration of the issue is, I believe, the hope that thinking about these things and talking about them, and I am certainly not talking about any sort of tax support government program to do this, but just people in society thinking about it, increases the chances that what we get out of this will be something that is safer than what it would have been otherwise.

SI: Why attend the Singularity Summit?

VV: The personal reason that I would be interested in the Singularity Institute and going to events of the Singularity Institute, is to see what other people have to say about this sort of thing. As part of the general issue of thinking about it both pro and con, by the way by pro and con I mean both skeptical and non-skeptical. I don't mean support or oppose. So basically I figure that if the singularity can be accomplished, it is such an economic wind. It is such an intellectual wind, it is such an artistic wind, it is such a military wind that it will happen. But talking about it, which goes on here at Singularity Institute and other venues is something that I think is interesting, and it does give us as a whole chance to think about these issues and perhaps to help to make them turn out in a good way.