Please email us with your thoughts
You are currently browsing the archives for August, 2007.
August 31st, 2007 –
ZDNET’s fourth podcast interview, this time with Steve Omohundro on “Building Self-Aware Systems.” Steve is president of Self-Aware Systems and advisor to SIAI. He will be one of the featured speakers at the Singularity Summit 2007 on September 8th-9th in San Francisco.
August 29th, 2007 –
ZDNET’s second and third interview with speakers from the Singularity Summit 2007:
“AI, Nanotech, and the Future of the Human Species,” with Steve Jurvetson, Draper Fisher Jurvetson Managing Director
“Pathways to Artificial Intelligence,” with Barney Pell, Powerset CEO
August 27th, 2007 –
August 26th, 2007 –
A special announcement from Powerset Product Manager Mark Johnson:
You probably know Powerset as a company known for its pioneering work in natural language search, but it is also creating a radical new platform for product innovation called Powerlabs. On Saturday night, Powerset will be demoing Powerlabs to participants in the Singularity Summit.
Powerlabs is a community of users who will be able to participate in the development of Powersetâs natural language search product. Within the Powerlabs community, users will be able to interact with demonstrations of Powersetâs technology, generate and refine product ideas, and then see their ideas come to life as Powerset implements them. This is an exciting example of community innovation and youâll be able to see it for yourself (and maybe give Powerset a few ideas about how to make it better!) on Saturday at the Palace of Fine Arts.
August 26th, 2007 –
ZDNET has begun podcasts with speakers from the Singularity Summit 2007, which is happening this September 8th-9th. The first interview, Can ‘Friendly AI’ Save Humans from Irrelevance or Extinction?, is with SIAI Research Fellow Eliezer Yudkowsky. The interviews are being done by Dan Farber, CNET Networks VP of Editorial and ZDNET Editor in Chief.
August 17th, 2007 –
Consider two classes of AIs. One class of AIs manipulates external objects to direct the world towards a goal state, the other doesn’t. AIs with the greatest real-world impact fall into the first category. The objects may be virtual as well as physical, although they’re both ultimately the same thing, as reality is harmonious and unified.
Within the first category, there are AIs with motivations that output the open-ended, indefinite manipulation of external objects, and AIs with motivations that cause the manipulations to stop after a critical threshold of utility maximizing (or satisficing) is performed.
A CEV-AI is an example of the latter category. It extrapolates humanity’s volition, creates an optimizing process that embodies it, then shuts itself down. There’s a technical problem here–how to program it in such a way that it doesn’t attempt to turn the planet into a supercomputer to compute humanity’s volition, disintegrating humanity in the process? Some version of interim Friendliness no doubt, but remember, the CEV-AI’s primary job is to output humanity’s collective will k, not be nice to humanity on a day-to-day basis. I’ll let the Friendly AI theorists try to figure that one out.
But back to the categories, I would think that most possible AIs fall into the first category: the open-ended, indefinite manipulation of external objects. In fact, most intelligences probably do. If a human life were extended to a quadrillion years, those quadrillion years would likely consist of the manipulation of external objects. Same thing if you extended the life of a chimp, or a badger indefinitely. The results might get boring pretty fast (rest, eat, sex, rest, eat, sex), but that manipulation of external objects would keep on going.
Imagine a sim-world, maybe something like that game Spore, that Will Wright thinks will change the world, with an indefinitely-living couple, be it chimp or human, living in it. Eventually their semi-random walk and offspring would encompass the world, and in the case of the human, they might even learn how to convert the world into billions of O’Neill colonies for maximum usefulness. When minds have an open-ended desire to manipulate the external world, in the long run, things never stay the same.
Because AIs would be running on accelerated substrates, the “long run” for them could be a few minutes or hours. An AI with an open-ended desire to manipulate external objects will eventually pattern over anything not to its liking, like a gardener will eventually pluck all the weeds in a garden if he has the time to do so. That’s why it’s damn important to make sure the first AI considers us, with all our flaws and imperfections, to be in alignment with its goals: if not, we’re toast in the long run, and for the AI, the long run ain’t very long at all.
August 15th, 2007 –
The Singularity is generally referred to as when an entity (whether hardware, wetware, both, or neither) comes into existence that is more intelligent than us humans. See for example Michael Anissimov’s The Word “Singularity” Has Lost All Meaning: “Transhuman intelligence is what ‘the Singularity’ has always supposed to mean” (emphasis original), or see Vernor Vinge’s What If the Singularity Does NOT Happen? (a piece I hope to revisit in a future post): “It seems plausible that with technology we can, in the fairly near future, create (or become) creatures who surpass humans in every intellectual and creative dimension. Events beyond this eventâcall it the Technological Singularityâare as unimaginable to us as opera is to a flatworm.”
However, sometimes the Singularity-starting entity is referred to as a power, not as an intelligence. In particular, an “artificial general intelligence” (i.e. a hardware-based entity) is sometimes called a “really powerful optimization process”. See for example, on this blog, a comment by Nick Tarleton (the first comment of AI is not Automatically Friendly), and several mentions on the sl4 archives. I like Eliezer’s definition: “A Really Powerful Optimization Process is an AGI but non-sentient” (link).
Which is the better description, intelligence or power? I can think of two criteria for comparison: Which is the more accurate description and which is the more useful description. (Related: Epistemic vs. prudential beliefs on utilitarian-essays.com’s Why We Should Believe in Free Will.) It seems to me that, on both counts, it is better to refer to this entity as being more powerful than humans, rather than more intelligent.
To decide which is more accurate, we’ll need an even more precise definition of Singularity. (Side note: I often say “a” Singularity as to not rule out the possibility of there being more than one. I’m open to arguments that there can be only one.) Recalling the definition of singularity in mathematics, I’m inclined to define a Singularity as a point that “exhibits extreme behavior”. This seems in the spirit of Vinge’s “Events beyond this event… are as unimaginable to us as opera is to a flatworm”. In this context, it seems that the key feature of the trans-human entity is not that it is intelligent, but that it does stuff. Since the ability to do stuff is power, it would, in this context, be more accurate to say that a Singularity is the product of a trans-human power, not a trans-human intelligence.
Side note: Is it possible for a trans-human intelligence to not also be a trans-human power? Probably. I’d propose that a coherent extrapolated volition (CEV) AGI that destroys itself upon extrapolating the volition that it’s not wanted would fit that description. Is it possible for a trans-human power to not also be a trans-human intelligence? Probably. I’d propose that a large asteroid on collision course with Earth would fit that description. Maybe you could come up with other instances of these.
That a Singularity entity is a trans-human power seems reasonable to me. I doubt many of us here are investing our hours on behalf of some monumental genius who, metaphorically speaking at least, sits around sucking his/her/its thumb. For me, and I reckon I’m not alone on this, the motivation is summed up in the idea of Artificial Intelligence as a Positive and Negative Factor in Global Risk (pdf). Indeed, I’m ultimately more interested in being useful than in being accurate, which brings us to the second comparison.
Which is a more useful description of a Singularity, trans-human intelligence or trans-human power? This could depend on the context of the description. I come up with two main contexts for discussing a Singularity: simple friendly conversation and trying to inspire action. For friendly conversation, it may not much matter. For inspiring action, however, I think it does. From what expertise on framing I’ve picked up from following political blogs, my sense is that “power” is the better motivator than “intelligence”. “Intelligence” probably makes people think of, say, Einstein or Hawking, whereas “power” probably makes people think of, say, Hitler or Churchill. People generally don’t take action to support or oppose eminent theoretical physicists, but they most certainly do take action to support or oppose eminent politicians. If I’m right on this, then when we’re trying to inspire action, we should talk power, not intelligence.
Ultimately, I think the important thing with our language is to get the right points across. We’re (quite fortunately!) not some political machine that must stay rigidly on-message lest we be made to eat our words, so we do have modest flexibility and margin for error in what we say. However, our language does matter. Hopefully this discussion will help us improve it.
August 15th, 2007 –
SIAI Director of Outreach Bruce Klein has been running a simple poll, asking hundreds of people when they think smarter-than-human AI will be developed. Here are the results to date. He has also received a lot of interesting comments. Here’s one of many:
Reply by Robert Bradbury
Bruce, I generally think the question is misphrased.
Over the last week I spent a couple of full days relearning sufficient chess that I could finally beat the Gnuchess program under Linux [1]. I personally tired of playing Backgammon when I wrote a program in 1977 that did a relatively good job defeating me. Scientists recently announced improvements to a checkers program so it can now play a perfect game. Two of the best human poker players in the world recently had a hard time defeating a computer program opponent.
So one answer to the question is that in specific fields AI is already far better than HI (esp. average HI). When it comes to laying out millions of transistors in electronic circuits AI has been better than HI for a decade or more. Computers are now quite adept at speech, OCR, voice recognition, face recognition, database searching, limited composition tasks and driving.
I think most people have fallen into the AGI swamp. Minsky pointed out long ago that the human brain is a complex aggregate of subprograms designed for specific functions. In a growing number of specific areas computer programs and the available hardware they can run on can match or exceed common human capabilities now.
If you want to ask more specific question regarding when will computer processing capacity exceed the human brain in a reasonable footprint the answer is before 2010). Petaflops computers are on order or soon will be with IBM & Sun. A tightly coupled network of a few thousand PS3 has the capacity of a human brain (and does a far better job at protein folding simulations).
Human brain equivalent processing power will be available to the average human in developed countries in the 2010-2020 time frame ($$$ in 2010-2015, $$ in 2015-2020). Human brain equivalent processing power will be available at a lower runtime cost (instructions/watt or instructions/$) in the 2020-2030 time frame. If you want to know when you might have robots that can function in most menial labor jobs, Iâd guess probably around 2025-2040. Its largely driven not by inability to produce the software or hardware but by the fact that the humans are still pretty cheap relative to the investment required to produce such a software+hardware combination.
But if you are asking whether there is this holy grail of AI that can transcend human intelligence I question whether that is feasible. I think one will over time simply have software and hardware combinations that can do more of what humans do faster or and/or at lower cost. Faster â will likely appear to be more intelligent. Lower cost will replace human workers. The big advantage is that once you have sunk the development cost (e.g. cell phone hardware and software) then the additional cost per unit is very cheap [2].
I disagree with some AI proponents in a number of areas. One does not need AI to solve the problems of significantly increased human longevity or nanotechnology. Insights and tools we currently have can deal with those problems. I would even argue that inexpensive AGIs pose significant risks due to their abuse potential [3] You may wish to carefully consider the comments by Robert Sapolsky (last paragraph) in the recent article about Williams Syndrome in the NY Times [4].
1. Gnuchess is still ahead in the match set by dozens, perhaps more than a hundred games but it isnât unbeatable. The only problem is that I was playing it a normal rather than hard difficulty.
2. Cell phones have decided advantages over pony express riders for example.
3. As the Internet-verse is plagued this week by the Storm Worm targeted at subverting vulnerable computers into even larger and more dangerous botnets.
4. The Gregarious Brain, David Dobbs, NY Times, July 8, 2007
August 13th, 2007 –
Numerous readers in the past have posted suggestions for how the universe should be run, in the form of laws, rights, or moral principles. I don’t want to embarrass them, so I won’t post links here, but if you have made such a suggestion then feel free to repost it in the comments below.
We know we’re not (yet) wise enough to write down scientific theories or fashion standards or music styles to be used for the rest of time. These objects are the output of an optimization process; human beings have spent long years studying, thinking, observing, and testing to develop them. If we want future generations to continue this process, it is necessary to communicate the target of the optimizer, not just its output so far.
Wikipedia is not an AI scientist; iTunes is not an AI composer. To build an AI scientist or an AI composer, we need to load in the optimization targets of science and music. Similarly, the 10 Commandments, the Bill of Rights, and Asimov’s Laws are not moralities; they are the output of human moralists. To build an AI moralist, you need to load in the right optimization target; that target is what I’m calling a morality.
Volition extrapolation is our current idea of how to load in optimization targets from existing humans.
August 11th, 2007 –
Why does the sequence 0000000000 seem simpler than the sequence 1010001110? The first sequence has an obvious pattern: it has ten repeated zeros. Thus, we can compactly describe it as “the sequence of ten zeros”. By contrast the second sequence has no such pattern, and its most compact description is probably “the sequence 1010001110″.
This intuitive understanding of simplicity is formalized by Kolmogorov complexity. The Kolmogorov complexity of a sequence is the length of the shortest program that outputs the sequence. Simple sequences tend to have patterns, so they can be generated by short programs. Conversely, if a short program describes a sequence this is itself a pattern. Thus, simple sequences will tend to have low Kolmogorov complexity, complex sequences will tend to have high Kolmogorov complexity.
