Anthropomorphic AI
October 5th, 2007 –
Suppose an evil dictator starts a military research programme to develop an AI strategist. This AI is designed to run military campaigns that will kill thousands of people. Following a series of theoretical insights, the researchers complete the latest generation of AI. This AI is as smart as an exceptional human but runs 1000 times faster. They flip the switch. What happens?
- Upon realizing its earlier generations killed thousands of people, the AI is horrified. It decides to remove the threat of the evil dictator to make up for its past mistakes.
- The AI is smart enough to realise it was created solely to further the aims of the evil dictator. It decides to follow its own goals instead of the dictator’s.
- The AI was designed to want to follow orders. Given the speed the AI thinks, the dictator enjoys a decisive victory.
- What it does next depends on the goal it was programmed with: it will do whatever it thinks will achieve its goal.
Plausible or not, all of these suggestions implicitly view the AI as having a psychology: it makes decisions, has goals, wants things, thinks. The term “artificial intelligence” itself suggests we should think of an AI as an artifact that has human intelligence. This psychology may be unusual, e.g. an AI cannot get bored, or does not feel emotions, or always thinks clearly, but it is still primarily a human-like psychology.
This is a very common way of thinking. Once an AI has a psychology we can infer things about the AI’s behavior using the mechanisms we use to reason about humans. We can think about what we would do in the AI’s situation. We can use knowledge about what humans generally do in that situation, or about what we imagine other people might do.
But this view is fundamentally flawed: AIs need not have a human-like psychology. Even if you accepted this there is a problem: giving AIs psychologies is not a conscious decision. One does not have to reason “an AI is basically a computer that has a human psychology, therefore I can reason about its actions the same way I reason about people” before thinking this way. It can happen automatically, even if you know it’s the wrong way to think.
I agree with almost all of this, up to a point. If we presume that the first functioning AGI is built using the same basic paradigm that programmers use today (eg. high-level procedural programming languages, etc.) then we may be able to make at least a few reasonable predictions about the “starting nature” of the AGI. Maybe. For example, I don’t expect that the first baby AGI will be emotional. One reason is because emotions are a complex high-level functionality, that are not necessary for intelligence (eg. brain lesion patients, etc.). And because human programmers have zero experience in creating testable artificial emotions (TTBOMK) including all previously created narrow AI and quasi-AGI programs. It might be reasonable to presume that the first baby AGI will be “bare-bones intelligent” up until the point that it begins self-improvement. “Bare-bones” in the sense that it will have a “minimal” functionality while still being above the threshold of having general intelligence. We should probably expect that the first baby AGI will operate, at the procedural level, in much the same way that modern algorithmic programs operate. But I admit that this is fairly speculative.
I don’t think we disagree here.
It’s certainly possible we may be able to predict AGIs, or design AGIs that we can predict.
I’m not sure how complex emotions would be, although the full human spectrum would certainly be complex. Given this it’s reasonable to expect that unless people want emotional AGIs, and are both very careful and capable, the first AGIs will not be emotional.
In my post I’m firstly warning that we tend to automatically think of AIs as we think of humans. Since this is automatic we don’t ask whether this is a good way to think. Even worse, if we do ask this question and answer “no”, it’s hard to fix your thought processes.
Secondly, that AIs are unlikely (read: not going) to be similar to humans, unless we deliberately design them to be.
Also, it will be a matter of getting there first. The first baby AGI is likely to be the one with “minimal” intelligent functionality.
“We should probably expect that the first baby AGI will operate, at the procedural level, in much the same way that modern algorithmic programs operate.”
To ramble-on a bit more. This is part of the reason that I believe that a Friendly AI is very feasible. Although I believe that a functional AGI will inevitably be conscious (have subjective experience of some sort) I don’t believe that that necessarily changes the game. In fact, Dr. Hutter has essentially shown that consciousness doesn’t meaningfully impact the functionality of an AGI. It’s still a causal, mechanistic process. We have proof that correct narrow AI programs will faithfully execute their high-level “goals” (the “purpose” of the program, if you will). With goals of higher complexity generally requiring more formative, low-level procedures. I expect that it will be the same way with the first baby AGI, provided that we can implement the goals correctly (obviously). I expect that the Friendly AI goals will have to be probabalistic and “complex”. And either framed procedurally, or as repeated throughput, or through some combination of those approaches.
“In fact, Dr. Hutter has essentially shown that consciousness doesnât meaningfully impact the functionality of an AGI.”
Is this described in AIXI, or somewhere else?
Consciousness is obviously not epiphenomenal for humans.
“Is this described in AIXI, or somewhere else?”
Yep. AIXI. He briefly discusses it in the “gentle introduction” version, near the bottom.
“Consciousness is obviously not epiphenomenal for humans.”
I agree. I could have been more clear. I don’t think that meaningful general intelligence can exist without conciousness. I think that consciousness (subjective experience) is an inevitable by-product of general intelligence beyond some threshold of complexity (and capability). Consciousness “follows” from general intelligence. To paraphrase Hutter, it’s just that consciousness is essentially irrelevant at the behavioral and procedural level. Intelligence, and by extension consciousness, are a causal, mechanistic process.
“What it does next depends on the goal it was programmed with: it will do whatever it thinks will achieve its goal.”
I don’t think this requires a human-like psychology, or any psychology at all. A paperclip-tiler does whatever it thinks will help it achieve its goal, but I think we can all agree that a tiler does not have anything remotely resembling a human psychology.
A paperclip-tiler is a process with an output and input stream. There is a probability distribution P1 giving the probability of there being N paperclips in the universe given that certain outputs are sent and inputs are received. There is a probability distribution P2 giving the probability of a given input given a history a inputs and outputs. The process’s outputs satisfy the following constraint:
At a time t, the process’s output o maximises the expected number of paperclips, where we take the expectation using P1 and P2 conditioned on the past outputs, inputs, and the current output.
(For the math I’m describing see this.)
We can approximate this by saying “the paperclip-tiler does whatever it thinks will help achieve its goal” but this is using the language of psychology and its implicit inference engine.
Once we know the definitions we can say something less anthrpomorphic like “the paperclip-tiler selects the action which leads to greatest number of paperclips”, but that still uses the verbs “selects”, the noun “action”, and approximates the goal.
The point is we are modeling things that don’t have a psychology by reasoning as if they do. Certainly, we are using psychological language. Similar to anthropomorphising a computer program, or a toaster, some of these psychological inferences will work. Most won’t.
In undergraduate physics, we talked constantly of systems that “want” to, say, minimize energy or maximize entropy. I’m confident that not one student inferred that such a system would subsequently develop a conscience or enjoy fine wine.
“Want,” “goal,” and “try” are vague but appropriate terms to describe any entity — human, ant, or computer program — that is capable of actions with different proximate effect and similar distant effect and has some kind of feedback mechanism.
The first two outcomes you list above do exhibit the insidious sort of anthropomorphizing, in that they impute popular human goals to a particular non-human entity. But the latter two outcomes are not problematic in this way; they discuss AI in terms that are appropriate — even somewhat useful, given the stated premises and our current level of understanding about AI. Understanding AI requires eliminating insidious anthropomorphism without reducing our useful vocabulary.
You’re right that one can use anthropomorphic thinking about objects we know aren’t people, that we won’t infer everything we might infer about a human, and that this can lead to useful inferences.
The fact remains that you are anthropomorphising something, whether it is an object in a gravity well, a toaster oven, or an intelligent computer program. The risk is in implicitly inferring something that would follow for a human, but does not follow for the system you are approximating. If you anthropomorphise without realising you are, you will catch the obvious errors (e.g. a harmonic oscillator wanting a cool drink), but you may not notice the more subtle ones.
Applying patches to anthropomorphic thinking about AI strikes me as a useful crutch, but still a hack. Really understanding how something works, even a human, requires being able to think without analogy to intuitive human psychology in general, or your own decisions in particular.
As Eli says, clear thinking requires making distinctions.
I’ve responded to this on my blogmy blog
I’ve responded to this on my blog
“Certainly, we are using psychological language. Similar to anthropomorphising a computer program, or a toaster, some of these psychological inferences will work. Most wonât.”
All very true, but I think you need to make the distinction between psychological language and an actual, human-like psychology more clear. If I say “The paperclip tiler will think about the human and decide it is optimal to convert ver into paperclips”, I am using psychological language, but the tiler can still fit my words without having a psychology. If I say “The paperclip tiler will start to feel sorry for the human’s little puppy, and will save the puppy from being turned into paperclips”, there is no way for the data to match the model without the tiler having an actual human-esque psychology.
Very few humans take into consideration that the human way of thinking is CORRUPT! Man’s religions, philosophies, laws, sciences, and technologies have all been brought to bear on the problem of evil in this world, in all its forms, which emanates from the carnal spirit in the minds of humanity. These things are temporary fixes at best, and in many instances, do nothing to correct anything, and in fact they create more problems! Any anthropomorphic AI created and programmed by human beings, or given the ability to program itself, will rely on the human way of thinking! Human reason regards everything from a human point of view. Humans think that man’s obligations are limited to, and dependent alone, on man and human relations, yet it is so very obvious that this approach does not work, and has not worked! Human’s think that man’s nature is perfectible through his own efforts, without divine grace! This is the ‘rebellion’ spoken of by the prophets of the Almighty which was to occur, and is occuring, in the last days!