The Singularity Institute Blog

I’ve noticed that Singularity discussions seem to be splitting up into three major schools of thought: Accelerating Change, the Event Horizon, and the Intelligence Explosion. Read the rest of this entry »

With the goal of eventually spawning an AGI Roadmap, should we craft an open letter highlighting the potential promise and peril of Artificial General Intelligence?

Thanks to Michael Anissimov for creating the initial draft and also to a number of other SIAI supporters for their contributions, we have a start on an AGI Open Letter.

Please reply below (or email me: bruce@singinst.org) if you think the open letter idea is worthwhile or not and/or if you have ideas for improvement. I plan to tally results and incorporate as many suggestions as possible. Thanks!

Our latest interviews are now online. We put a lot of work into them, and I hope it shows. You can stream online, view in high-res via QuickTime, get the audio, or read the transcript.

This set includes seven of the Singularity Summit 2007 speakers:

* Dr. Peter Norvig, Google Director of Research and co-author of AI: A Modern Approach
* Sam Adams, IBM Distinguished Engineer within IBM’s Research Division
* Jamais Cascio, IEET Fellow and CRN Director of Impacts Analysis
* Dr. Ben Goertzel, SIAI Director of Research and Novamente CEO
* Peter Voss, Adaptive AI CEO
* Dr. James Hughes, IEET Executive Director
* John Smart, Acceleration Studies Foundation President

Please let us know what you think about the interviews. We welcome your feedback.

Thanks to Doug Wolens for filming and editing, Tom McCabe for audio, and Drew Reynolds for transcripts.

There are two related concepts that have a tendency to get tangled up whenever people talk about the Singularity Institute and advanced AI. These are recursive self-improvement and a hard takeoff (which practically no one is aware of or considers feasible) and Friendly AI (which, in the Asimovian sense, millions of people are aware of and assume it will be important one day).

Recursive self-improvement stems from I.J. Good’s “intelligence explosion” idea. That a very smart mind could make itself smarter, and make itself smarter at being smarter, in an explosive self-improvement cycle, presumably culminating in a godlike entity of immense intelligence and capability. I personally consider this idea highly plausible — look at the relatively small hardware differences between humans and chimps, and tell me that if an entity truly understood the mechanics of intelligence it couldn’t design something another increment above humans — but many people don’t. They presume that when an intelligent mind is put on a substrate a million times faster than biological neurons, it will operate at about the same thinking speed as a human being, because of residual dualistic beliefs about the unrelatedness of a mind and the hardware it runs on. Others probably see human beings as the smartest possible entities that ever will exist. This is anthropocentric vanity.

An accelerated, smarter-than-human, self-modifying mind — that is the essence of an intelligence explosion. Now switch over to Friendly AI.

Friendly AI is an effort to make an AI that is helpful, benevolent, and that we don’t regret making. But it means vastly different things depending on how advanced one believes AI can become in a given time. I assume that the majority of people out there envision that future AIs will look and behave similar to Rosie the Robot in the Jetsons. If so, then AI is not really much of a threat, and the research direction of Friendliness looks unnecessary and premature. This is my theory for why most people don’t care about SIAI’s effort.

The majority of people involved with SIAI believe that the first general AI is likely to be creative and smart enough to find ways to massively improve its own hardware and intelligence, quite quickly. One analogy often given is the rise of humans within the planetary ecosystem — from the perspective of millions of years of slow evolution, we were here in a flash, and remade the face of the Earth in barely any time at all. Some watchers of AI believe that AI could bootstrap itself and gain prominence in timescales considered very short by our own standards. For instance, taking over the world in a matter of weeks or days. One might laugh, but from the perspective of biological evolution, which was the most powerful force on Earth for billions of years, human civilization emerged in what might be considered hours or seconds in “evolution time”. Even an AI a little bit smarter than all humans that have ever lived could come up with a new manufacturing technology it could use to give it all the hardware it needs.

From this perspective, Friendliness gets a little more important. If the first general AI ever created has the potential to become the Earth’s top dog, shouldn’t we care about its motivations and moral beliefs?

Yes. But this is where some people say “if it’s so smart, won’t it throw out whatever goals we give it”? But that’s a topic for another post.

Please welcome Jonas Lamis as our Director of Partnerships. In this important role, Jonas will be focused on building collaborative relationships between companies, institutions, foundations, and the Singularity Institute. He will be co-organizing the 2008 Singularity Summit. Jonas has ten years of experience in corporate strategy, business development, and technology marketing with venture-backed enterprise software companies. He is currently Vice President of Alliances for Troux Technologies. He is also the founding editor of Architecture and Governance Magazine and authors the weblog Singularity University. Jonas received his MBA from The University of Texas at Austin, an MS in Systems Engineering and Optimization from The Georgia Institute of Technology, and a BS in Industrial Engineering from Purdue University.

From Jonas:

Ray Kurzweil’s books were the initial driver behind my interest in Singularity enabling technologies. Following last year’s Summit at Stanford, I started a blog on that topic and began contributing to SIAI. Most recently, I have co-launched a “business of robotics” weblog at RobotCentral.com.

I am not a technologist and am definitely challenged to keep up in the more techno-theoretical conversations. But what I hope to bring to the community and SIAI is a business sense that will help bridge awareness and understanding between today’s singularity leaders and the general public and business community. Getting out in front of the coming wave of technology change is critical for the success of both industry and humanity. There is nothing more important than this over the next 10 to 20 years, and I hope to be able to make a difference.

Feel free to send me your ideas. I can be reached at jonas(at)lamis(dot)org.

Sorry for the lack of posting lately. In case you’ve missed any of the Singularity Summit coverage, here’s a recap. We certainly reached a level of awareness rarely seen for these longer-term issues; e.g., the Associated Press story that ran on September 9th was picked up by over 200 outlets, including MSNBC, CNBC, MSN, ABC News, FOX News, Yahoo News, The Boston Globe, The Chicago Tribune, Forbes, The Washington Post, The Sun Herald, The Miami Herald, The Inquirer, etc.

I am concerned about how the coverage gravitated toward negative rather than positive visions. I think it is partially because positive visions, which are seen as clearly desirable by a broad audience, have not been sufficiently presented and disseminated so far. Jamais Cascio also made the astute point that negative outcomes are easier to agree upon; there’s more variance about what positive outcomes would look like. It does seem clear, though, that more inclusive, positive, desirable visions of possible futures with powerful Artificial Intelligence technology are still needed. If you have any thoughts to share on this, I hope you will do so through our community blog.

Associated Press: Techies ponder computers smarter than us

Forbes: Robot chic

Reason: Will Super-Smart Artificial Intelligences Keep Humans Around As Pets?

San Francisco Chronicle: Public meeting will re-examine future of AI

San Jose Mercury News: Venture capitalist: We need to prepare for AI

Wired: Peter Thiel explains how to invest in the Singularity

CNET: Facebook backer Thiel’s investment strategy for singularity

ZDNET: Singularity Summit: Google co-evolving with the web (check out ZDNET’s other articles on the summit as well)

The Inquirer: Computers to become brighter than their operators

Slashdot: Smarter-than-human intelligence and the Singularity Summit

New PDF for the Singularity Summit 2007: The Singularity and Singularity Institute for Artificial Intelligence.