Food for Thought
June 15th, 2007 –
There’s no such thing as a free lunch. But many lunches can be purchased at a reasonable price.
I want to talk to you today about that knee-jerk reaction you get when you hear about SIAI supporters and our crazy notions about solving the world’s problems through artificial intelligence. In particular, we talk about artificial general intelligence (AGI), a system that can think usefully about many different things, just like we can — and eventually much better than we can.
“This is too good to be true,” you say. “Should I file this under perpetual motion machines, or get-rich-quick schemes?”
Rest easy! Perpetual motion is not required.
Most AGI research isn’t really about the machinery. AGI has more to do with software design, and will probably work with computer hardware already existing at the time of its invention. It will be much like your own brain on that score. Your brain cells are basically identical to those of other animals you would not consider very intelligent.
Intelligence, “artificial” or otherwise, cannot break the laws of physics by creating something out of nothing. If it could, we might have gone straight from cave paintings to quantum theory. Thinking, then, is not a wellspring, but a process. This can seem hard to believe after you injure yourself and spontaneously coin new expletives. But intelligence only takes incoming information, does some fancy mixing with earlier information, and spits out something that is, on average, more useful. Sometimes you need new ways to talk about your pain, and your brain is there for you — with earlier source material waiting inside.
Unlike pyramid schemes or Nigerian email scams, thinking also doesn’t have to take away anything from anyone, except for the small amount of energy required to do the processing. The human brain, for instance, can churn out folk songs while running on jelly donuts.
Does that make intelligence an impossible “free lunch” in the economic sense? No. Energy is consumed in the thought process, which in this example came from stuffed pastries. You can put a price on those. But the value added to information when we send it through our brains makes this exchange a no-brainer. Thinking isn’t free, but it’s such a screaming deal that you may never have thought about it as costing anything at all.
If you could think more just by eating more, wouldn’t you? Because our bodies are limited, we have learned how to do many energy-using functions outside of them. We can burn calories to move our legs, and we can also burn gasoline to turn wheels. It so happens that we can go faster on wheels. AGI researchers want to do for the thought process what mechanical engineers have done for the locomotion process.
Of course, there’s more to thinking than speed. Not all minds process information the same way. Minds have different structures, and different prior knowledge. These differences mean your optometrist will look into your eyes differently than your attractive jogging partner. The output you find most useful depends on the occasion.
When something is really important, you look for the mind best able to turn information about your problem into a possible solution. Think about this, though: Every mind you can hit up for advice today is just a human brain, and therefore nearly identical in structure. That 14-ounce bundle of monkey neurons is our best example of general intelligence because it happens to be our only example. I suspect that if we could donate the human brain to a museum of all possible minds, it would end up in a special novelty exhibit where gawkers could laugh at the idea that people ever managed to think with those things.
So, if a pyramid scheme is a get-rich-quick solution to money problems, then working on artificial general intelligence is closer to asking your successful uncle for financial advice. Except the AGI makes your uncle look like a monkey. And, because it’s “general”, it gives you great decorating tips too.
Many people who know about the potential for AGI already have some fun ideas about solving the world’s problems with futuristic technology. These may also trigger your BS detectors. I’ll let other people defend those ideas, and simply advise you to not be fooled into thinking that B must follow from A. Greater intelligence may well come up with solutions that in no way resemble the nano-powered, anime-inspired utopias your purple-haired IT professional may have told you about.
For that matter, human minds may have already come up with the best possible answers to our human challenges. Maybe the idea of improving on anything really is too good to be true. But I, for one, would like the privilege of asking someone fundamentally smarter about it. And I’m willing to shell out jelly donuts to people trying to give me that chance.
Yes, a super-intelligent AGI may sit us down and say, “Sorry, guys. I’ve checked out every solution to every one of your problems, and they all carry higher costs than you would want to pay.”
But if that happens, AGI really will mean a free lunch — at least for you. It’ll be my treat. Are donuts ok?
I would file FAI under get-rich-quick schemes, not free lunches. Free lunches violate the first law of thermodynamics (“In any process, the total energy of the universe remains constant.â link) It’s entirely possible to get rich quick- just place some large leveraged bets in the right direction. It only sounds fishy because of all the charlatans pretending they can guarantee it for you.
When I meet skepticism of AGI’s downside (existential risk), I bring up the early days of nuclear weapons development. (See this for example.) The point is we don’t know and therefore caution must be exercised. Kuipers got that part right, although I’m told he wasn’t too savvy of the tech details when he wrote that.
Regarding the AGI upside (get-rich-quick), the same uncertainty still applies. Maybe compare it to controlled fusion, which wouldn’t solve the same set of problems as FAI (possibly a subset of problems if FAI gets us controlled, energy-productive fusion) but it would still solve many of our problems (possibly all of our energy problems for an astronomically long time). And like controlled, energy-productive fusion, we still don’t know how to build FAI. (See Fusion power for more on that.)
I find that drawing analogies like these helps people get a sense for what we’re looking at here. Since pretty much everyone that I talk to, including non-scientists, are familiar with nuclear weapons and at least loosely familiar with controlled fusion, these particular analogies work well. The analogies can also be starting points for contrasts (such as in protecting nuclear secrets vs AI secrets) in addition to comparisons.
“It only sounds fishy because of all the charlatans pretending they can guarantee it for you.”
A lot of the charlatans really can help you to ‘get rick quick’- with some ridiculously small probability like 1 in 500, so they can boast about it in testimonials. In general, any very large payoff will have a corresponding very large risk, but humans aren’t good at evaluating the average expected payoff from those parameters (just look at the lottery) and so we fall for it.
“these particular analogies work well.”
The danger of using analogies is that people will think of the analogy as an idea for itself rather than a metaphor for another idea. Or they’ll misinterpret the analogy as being a metaphor for something else entirely. A lot of people are quite frankly scared of anything nuclear, and they have succeeded in getting the government to oppose many new nuclear reactors, processing facilities and research programs regardless of merit, which is not what we want for AI.
“A lot of people are quite frankly scared of anything nuclear, and they have succeeded in getting the government to oppose many new nuclear reactors, processing facilities and research programs regardless of merit, which is not what we want for AI.”
It’s not what we want for nuclear either, so the analogy holds again. My favorite example of nuclear paranoia is “MRI” vs “NMR”. It doesn’t get much more clear-cut than that.
If we’re concerned that people will draw the wrong conclusions when we talk about AI, we should probably just not talk about it. But if we’re going to be talking anyways, then it is good to do so effectively. If you can propose a better strategy than these nuclear analogies, I’m all ears.
Meanwhile, this sounds like a good moment for teaching people rationality/bias/etc.
….
“AGI researchers want to do for the thought process what mechanical engineers have done for the locomotion process.”
I think this is a good expression, although to me it leads right down the “Why the future doesn’t need us” path. But perhaps this is a good thing, if that topic is well worth discussing. The bright side of it includes self-deleting CEV’s or thinking for fun (much like we now jog for fun).
“Itâs not what we want for nuclear either, so the analogy holds again.”
It may logically hold, but if you use it on the grounds that it’s understandable to non-scientists, you’re going to get people ignoring you (at best) or denouncing you as cultists (at worst). I can’t think of any technical concepts off the top of my head that can be truly understood solely through analogy.
“I canât think of any technical concepts off the top of my head that can be truly understood solely through analogy.”
Hmm… maybe we’re talking different aspects of AI here, possibly different audience too. I was talking general audience, to give them a general impression of the situation we’re looking at. To this end, I’ve been successful with analogies of this sort. To the extent that I have been explaining the underlying technical concepts, I certainly haven’t done so via analogy to the nuclear sector. Off the top of my head, I’ve said things like “like Skynet except not all done up for the movies”, stressing that Skynet is heavily dramatized and not realistic, and “computers take things very literally” which anyone who’s used a computer has some crude sense for. What would you recommend saying?
[...] my previous post to this blog, I tried to drive home the idea that thinking is a process, and one that will soon be [...]