| Next: | Conclusion | Bookmark | |
| Up: | Levels of Organization in General Intelligence | Monolithic | |
| Prev: | 2.7: The deliberation level |
In the space between the theory of human intelligence and the theory of general AI is the ghostly outline of a theory of minds in general, specialized for humans and AIs. I have not tried to lay out such a theory explicitly, confining myself to discussing those specific similarities and differences of humans and AIs that I feel are worth guessing in advance. The Copernican revolution for cognitive science - humans as a noncentral special case - is not yet ready; two points are not enough to draw a curve, and currently we only have one. Nonetheless, humans are in fact a noncentral special case, and this abstract fact is knowable even if our current theories are anthropocentric.
There is a fundamental rift between evolutionary design and deliberative design. From the perspective of a deliberative intelligence - a human, for instance - evolution is the degenerate case of design-and-test where intelligence equals zero. Mutations are atomic; recombinations are random; changes are made on the genotype's lowest level of organization (flipping genetic bits); the grain size of the component tested is the whole organism; and the goodness metric operates solely through induction on historically encountered cases, without deductive reasoning about which contextual factors may later change1. The evolution of evolvability [Wagner96] improves this picture somewhat. There is a tendency for low-level genetic bits to exert control over high-level complexity, so that changes to those genes can create high-level changes. Blind selection pressures can create self-wiring and self-repairing systems that turn out to be highly evolvable because of their ability to phenotypically adapt to genotypical changes. Nonetheless, the evolution of evolvability is not a substitute for intelligent design. Evolution works, despite local inefficiencies, because evolution exerts vast cumulative design pressure over time.
However, the total amount of design pressure exerted over a given time is limited; there is only a limited amount of selection pressure to be divided up among all the genetic variances selected on in any given generation [Worden95]. One obvious consequence is that evolutionarily recent adaptations will probably be less optimized than those which are evolutionarily ancient. In DGI, the evolutionary phylogeny of intelligence roughly recapitulates its functional ontogeny; it follows that higher levels of organization may contain less total complexity than lower levels, although sometimes higher levels of organization are also more evolvable. Therefore, a subtler consequence is that the lower levels of organization are likely to be less well adapted to evolutionarily recent innovations (such as deliberation) than those higher levels to the lower levels - an effect enhanced by evolution's structure-preserving properties, including the preservation of structure that evolved in the absence of deliberation. Any design possibilities that first opened up with the appearance of Homo sapiens sapiens remain unexploited because Homo sapiens sapiens has only existed for 50,000-100,000 years; this is enough time to select among variances in quantitative tendencies, but not really enough time to construct complex functional adaptation. Since only Homo sapiens sapiens in its most modern form is known to engage in computer programming, this may explain why we do not yet have the capacity to reprogram our own neurons (said with tongue firmly in cheek, but there's still a grain of truth). And evolution is extremely conservative when it comes to wholesale revision of architectures; the homeotic genes controlling the embryonic differentiation of the forebrain, midbrain, and hindbrain have identifiable homologues in the developing head of the Drosophila fly(!) [Holland92].
Evolution never refactors its code. It is far easier for evolution to stumble over a thousand individual optimizations than for evolution to stumble over two simultaneous changes which are together beneficial and separately harmful. The genetic code that specifies the mapping between codons (a codon is three DNA bases) and the 20 amino acids is inefficient; it maps 64 possible codons to 20 amino acids plus the stop code. Why hasn't evolution shifted one of the currently redundant codons to a new amino acid, thus expanding the range of possible proteins? Because for any complex organism, the smallest change to the behavior of DNA - the lowest level of genetic organization - would destroy virtually all higher levels of adaptive complexity, unless the change were accompanied by millions of other simultaneous changes throughout the genome to shift every suddenly-nonstandard codon to one of its former equivalents. Evolution simply cannot handle simultaneous dependencies, unless individual changes can be deployed incrementally, or multiple phenotypical effects occur as the consequence of a single genetic change. For humans, planning coordinated changes is routine; for evolution, impossible. Evolution is hit with an enormous discount rate when exchanging the paper currency of incremental optimization for the hard coin of complex design.
We should expect the human design to incorporate an intimidatingly huge number of simple functional optimizations. But it is also understandable if there are deficits in the higher design. While the higher levels of organization (including deliberation) have emerged from the lower levels and hence are fairly well adapted to them, the lower levels of organization are not as adapted to the existence of deliberate intelligence. Humans were constructed by accretive evolutionary processes, moving from very complex nongeneral intelligence to very complex general intelligence, with deliberation the last layer of icing on the cake.
Can we exchange the hard coin of complex design for the paper currency of low-level optimization? "Optimizing compilers" are an obvious step but a tiny one; program optimization makes programs faster but exerts no design pressure for better functional organization, even for simple functions of the sort easily optimized by evolution. Directed evolution, used on modular subtasks with clearly defined performance metrics, would be a somewhat larger step. But even directed evolution is still the degenerate case of design-and-test where individual steps are unintelligent. We are, by assumption, building an AI. Why use unintelligent design-and-test?
Admittedly, there is a chicken-and-egg limit on relying on an AI's intelligence to help build an AI. Until a stably functioning cognitive supersystem is achieved, only the nondeliberative intelligence exhibited by pieces of the system will be available. Even after the achievement of a functioning supersystem - a heroic feat in itself - the intelligence exhibited by this supersystem will initially be very weak. The weaker an AI's intelligence, the less ability the AI will show in understanding complex holonic systems. The weaker an AI's abilities at holonic design, the smaller the parts of itself that the AI will be able to understand. At whatever time the AI finally becomes smart enough to participate in its own creation, the AI will initially need to concentrate on improving small parts of itself with simple and clear-cut performance metrics supplied by the programmers. This is not a special case of a stupid AI trying to understand itself, but a special case of a stupid AI trying to understand any complex holonic system; when the AI is "young" it is likely to be limited to understanding simple elements of a system, or small organizations of elements, and only where clear-cut goal contexts exist (probably programmer-explained). But even a primitive holonic design capability could cover a human gap; we don't like fiddling around with little things because we get bored, and we lack the ability to trade our massive parallelized power on complex problems for greater serial speed on simple problems. Similarly, it would be unhealthy (would result in AI pathologies) for human programming abilities to play a permanent role in learning or optimizing concept kernels - but at the points where interference seems tempting, it is perfectly acceptable for the AI's deliberative processes to play a role, if the AI has advanced that far.
Human intelligence, created by evolution, is characterized by evolution's design signature. The vast majority of our genetic history took place in the absence of deliberative intelligence; our older cognitive systems are poorly adapted to the possibilities inherent in deliberation. Evolution has applied vast design pressures to us but has done so very unevenly; evolution's design pressures are filtered through an unusual methodology that works far better for hand-massaging code than for refactoring program architectures.
Now imagine a mind built in its own presence by intelligent designers, beginning from primitive and awkward subsystems that nonetheless form a complete supersystem. Imagine a development process in which the elaboration and occasional refactoring of the subsystems can coopt any degree of intelligence, however small, exhibited by the supersystem. The result would be a fundamentally different design signature, and a new approach to Artificial Intelligence which I call seed AI.
A seed AI is an AI designed for self-understanding, self-modification, and recursive self-improvement. This has implications both for the functional architectures needed to achieve primitive intelligence, and for the later development of the AI if and when its holonic self-understanding begins to improve. Seed AI is not a workaround that avoids the challenge of general intelligence by bootstrapping from an unintelligent core; seed AI only begins to yield benefits once there is some degree of available intelligence to be utilized. The later consequences of seed AI (such as true recursive self-improvement) only show up after the AI has achieved significant holonic understanding and general intelligence. The bulk of this paper, Part II, describes the general intelligence that is prerequisite to seed AI; Part III assumes some degree of success in constructing general intelligence and asks what may happen afterward. This may seem like hubris, but there are interesting things to be learned thereby, some of which imply design considerations for earlier architecture.
From the standpoint of computer science it may seem like breathtaking audacity if I dare to predict any advantages for AIs in advance of their construction, given past failures. But from the standpoint of evolutionary psychology, the human mind has surprising flaws to match its surprising strengths. If discussing the potential advantages of "AIs" strikes you as too audacious, then consider what follows, not as discussing the potential advantages of "AIs", but as discussing the potential advantages of minds in general relative to humans. One may then consider separately the audacity involved in claiming that a given AI approach can achieve one of these advantages, or that it can be done in less than fifty years.
Humans definitely possess the following advantages, relative to current AIs:
The potential cognitive advantages of minds-in-general, relative to human minds, probably include:
Fully recursive self-enhancement is a potential advantage of minds-in-general that has no analogue in nature - not just no analogue in human intelligence, but no analogue in any known process. Since the divergence of the hominid family within the primate order, further developments have occurred at an accelerating pace - but this is not because the character of the evolutionary process changed or became "smarter"; successive adaptations for intelligence and language opened up new design possibilities and also tended to increase the selection pressures for intelligence and language. Similarly, the exponentially accelerating increase of cultural knowledge in Homo sapiens sapiens was triggered by an underlying change in the human brain, but has not itself had time to create any significant changes in the human brain. Once Homo sapiens sapiens arose, the subsequent runaway acceleration of cultural knowledge took place with essentially constant brainware. The exponential increase of culture occurs because acquiring new knowledge makes it easier to acquire more knowledge.
The accelerating development of the hominid family and the exponential increase in human culture are both instances of weakly self-improving processes, characterized by an externally constant process (evolution, modern human brains) acting on a complexity pool (hominid genes, cultural knowledge) whose elements interact synergetically. If we divide the process into an improver and a content base, then weakly self-improving processes are characterized by an external improving process with roughly constant characteristic intelligence, and a content base within which positive feedback takes place under the dynamics imposed by the external process.
If a seed AI begins to improve itself, this will mark the beginning of the AI's self-encapsulation. Whatever component the AI improves will no longer be caused entirely by humans; the cause of that component will become, at least in part, the AI. Any improvement to the AI will be an improvement to the cause of a component of the AI. If the AI is improved further - either by the external programmers, or by internal self-enhancement - the AI may have a chance to re-improve that component. That is, any improvement to the AI's global intelligence may indirectly result in the AI improving local components. This secondary enhancement does not necessarily enable the AI to make a further, tertiary round of improvements. If only a few small components have been self-encapsulated, then secondary self-enhancement effects are likely to be small, not on the same order as improvements made by the human programmers.
If computational subsystems give rise to cognitive talents, and cognitive talents plus acquired expertise give rise to domain competencies, then self-improvement is a means by which domain competencies can wrap around and improve computational subsystems, just as the seed AI idiom of coopting deliberative functions into cognition enables improvements in domain competencies to wrap around and improve cognitive talents, and the ordinary idiom of intelligent learning enables domain competencies to wrap around and improve acquired expertise3. The degree to which domain competencies improve underlying processes will depend on the AI's degree of advancement; successively more advanced intelligence is required to improve expertise, cognitive talents, and computational subsystems. The degree to which an improvement in intelligence cascades into further improvements will be determined by how much self-encapsulation has already taken place on different levels of the system.
A seed AI is a strongly self-improving process, characterized by improvements to the content base that exert direct positive feedback on the intelligence of the underlying improving process. The exponential surge of human cultural knowledge was driven by the action of an already-powerful but constant force, human intelligence, upon a synergetic content base of cultural knowledge. Since strong self-improvement in seed AI involves an initially very weak but improving intelligence, it is not possible to conclude from analogies with human cultural progress that strongly recursive self-improvement will obey an exponential lower bound during early stages, nor that it will obey an exponential upper bound during later stages. Strong self-improvement is a mixed blessing in development. During earlier epochs of seed AI, the dual process of programmer improvement and self-improvement probably sums to a process entirely dominated by the human programmers. We cannot rely on exponential bootstrapping from an unintelligent core. However, we may be able to achieve powerful results by bootstrapping from an intelligent core, if and when such a core is achieved. Recursive self-improvement is a consequence of seed AI, not a cheap way to achieve AI.
It is possible that self-improvement will become cognitively significant relatively early in development, but the wraparound of domain competencies to improve expertise, cognition, and subsystems does not imply strong effects from recursive self-improvement. Precision in discussing seed AI trajectories requires distinguishing between epochs for holonic understanding, epochs for programmer-dominated and AI-dominated development, epochs for recursive and nonrecursive self-improvement, and epochs for overall intelligence.
(Readers averse to advance discussion of sophisticated AI may consider these epochs as referring to minds-in-general that possess physical access to their own code and some degree of general intelligence with which to manipulate it; the rationale for distinguishing between epochs may be considered separately from the audacity of suggesting that AI can progress to any given epoch.)
But if no such barrier exists - if it is possible for an artificial system to match DNA and display human-equivalent general intelligence - then it seems very probable that seed AI is achievable as well. It would be the height of biological chauvinism to assert that, while it is possible for humans to build an AI and improve this AI to the point of roughly human-equivalent general intelligence, this same human-equivalent AI can never master the (humanly solved) programming problem of making improvements to the AI's source code.
Furthermore, the above statement misstates the likely interrelation of the epochs. An AI does not need to wait for full human-equivalence to begin improving on the programmer's work. An optimizing compiler can "improve" over human work by expending greater relative effort on the assembly-language level. That is, an optimizing compiler uses the programmatic advantages of greater serial speed and immunity to boredom to apply much greater design pressures to the assembly-language level than a human could exert in equal time. Even an optimizing compiler might fail to match a human at hand-massaging a small chunk of time-critical assembly language. But, at least in today's programming environments, humans no longer hand-massage most code - in part because the task is best left to optimizing compilers, and in part because it's extremely boring and wouldn't yield much benefit relative to making further high-level improvements. A sufficiently advanced AI that takes advantage of massive serialism and freedom from evolutionary misoptimizations may be able to apply massive design pressures to higher holonic levels of the system.
Even at our best, humans are not very good programmers; programming is not a task commonly encountered in the ancestral environment. A human programmer is metaphorically a blind painter - not just a blind painter, but a painter entirely lacking a visual cortex. We create our programs like an artist drawing one pixel at a time, and our programs are fragile as a consequence. If the AI's human programmers can master the essential design pattern of sensory modalities, they can gift the AI with a sensory modality for codelike structures. Such a modality might perceptually interpret: a simplified interpreted language used to tutor basic concepts; any internal procedural languages used by cognitive processes; the programming language in which the AI's code level is written; and finally the native machine code of the AI's hardware. An AI that takes advantage of a codic modality may not need to wait for human-equivalent general intelligence to beat a human in the specific domain competency of programming. Informally, an AI is native to the world of programming, and a human is not.
This leads inevitably to the question of how much programming ability would be exhibited by a seed AI with human-equivalent general intelligence plus a codic modality. Unfortunately, this leads into territory that is generally considered taboo within the field of AI. Some readers may have noted a visible incompleteness in the above list of seed AI epochs; for example, the last stage listed for human-driven and AI-driven improvement is "weak domination" of the improvement process by human programmers (the AI and the programmers make the same kind of improvements, but the programmers make more improvements than the AI). The obvious succeeding epoch is one in which AI-driven development roughly equals human development, and the epoch after that one in which AI-driven development exceeds human-driven development. Similarly, the discussion of epochs for recursive self-improvement stops at the point where AI-driven improvement sometimes opens up new portions of the opportunity landscape, but does not discuss the possibility of open-ended self-improvement: a point beyond which progress can continue in the absence of human programmers, so that by the time the AI uses up all the improvements visible at a given level, that improvement is enough to "climb the next step of the intelligence ladder" and make a new set of improvements visible. The epochs for overall intelligence define tool-level, prehuman, and infrahuman AI, but do not define human-equivalence or transhumanity.
It is interesting to contrast the separate perspectives of modern-day Artificial Intelligence researchers and modern-day evolutionary psychologists with respect to the particular level of intelligence exhibited by Homo sapiens sapiens. Modern-day AI researchers are strongly reluctant to discuss human equivalence, let alone what might lie beyond it, as a result of past claims for "human equivalence" that fell short. Even among those rare AI researchers who are still willing to discuss general cognition, the attitude appears to be: "First we'll achieve general cognition, then we'll talk human-equivalence. As for transhumanity, forget it."
In contrast, modern-day evolutionary theorists are strongly trained against Panglossian or anthropocentric views of evolution, i.e., those in which humanity occupies any special or best place in evolution. Here it is socially unacceptable to suggest that Homo sapiens sapiens represents cognition in an optimal or maximally developed form; in the field of evolutionary psychology, the overhanging past is one of Panglossian optimism. Rather than modeling the primate order and hominid family as evolving toward modern-day humanity, evolutionary psychologists try to model the hominid family as evolving somewhere, which then decided to call itself "humanity". (This view is beautifully explicated in Terrence Deacon's "The Symbolic Species" [Deacon97].) Looking back on the history of the hominid family and the human line, there is no reason to believe that evolution has hit a hard upper limit. Homo sapiens has existed for a short time by comparison with the immediately preceding species, Homo erectus. We look back on our evolutionary history from this vantage point, not because evolution stopped at this point, but because the subspecies Homo sapiens sapiens is the very first elaboration of primate cognition to cross over the minimum line that supports rapid cultural growth and the development of evolutionary psychologists. We observe human-level intelligence in our vicinity, not because human intelligence is optimal or because it represents a developmental limit, but because of the Anthropic Principle; we are the first intelligences smart enough to look around. Should basic design limits on intelligence exist, it would be an astonishing coincidence if they centered on the human level.
Strictly speaking, the attitudes of AI and evolutionary psychology are not irreconcilable. One could hold that achieving general cognition will be extremely hard and that this constitutes the immediate research challenge, while simultaneously holding that once AI is achieved, only ungrounded anthropocentrism would predict that AIs will develop to a human level and then stop. This hybrid position is the actual stance I have tried to maintain throughout this paper - for example, by decoupling discussion of developmental epochs and advantages of minds-in-general from the audacious question of whether AI can achieve a given epoch or advantage.
But it would be silly to pretend that the tremendous difficulty of achieving general cognition licenses us to sweep its enormous consequences under the rug. Despite AI's glacial slowness by comparison with more tractable research areas, Artificial Intelligence is still improving at an enormously faster rate than human intelligence. A human may contain millions or hundreds of millions of times as much processing power as a personal computer circa 2002, but computing power per dollar is (still) doubling every eighteen months, and human brainpower is not.
Many have speculated whether the development of human-equivalent AI, however and whenever it occurs, will be shortly followed by the development of transhuman AI [Moravec88]; [Vinge93]; [Minsky94]; [Kurzweil99]; [Hofstadter00]; [Hawking01]. Once AI exists it can develop in a number of different ways; for an AI to develop to the point of human-equivalence and then remain at the point of human-equivalence for an extended period would require that all liberties be simultaneously blocked4 at exactly the level which happens to be occupied by Homo sapiens sapiens. This is too much coincidence. Again, we observe Homo sapiens sapiens intelligence in our vicinity, not because Homo sapiens sapiens represents a basic limit, but because Homo sapiens sapiens is the very first hominid subspecies to cross the minimum line that permits the development of evolutionary psychologists.
Even if this were not the case - if, for example, we were now looking back on an unusually long period of stagnation for Homo sapiens - it would still be an unlicensed conclusion that the fundamental design bounds which hold for evolution acting on neurons would hold for programmers acting on transistors. Given the different design methods and different hardware, it would again be too much of a coincidence.
This holds doubly true for seed AI. The behavior of a strongly self-improving process (a mind with access to its own source code) is not the same as the behavior of a weakly self-improving process (evolution improving humans, humans improving knowledge). The ladder question for recursive self-improvement - whether climbing one rung yields a vantage point from which enough opportunities are visible that they suffice to reach the next rung - means that effects need not be proportional to causes. The question is not how much of an effect any given improvement has, but rather how much of an effect the improvement plus further triggered improvements and their triggered improvements have. It is literally a domino effect - the universal metaphor for small causes with disproportionate results. Our instincts for system behaviors may be enough to give us an intuitive feel for the results of any single improvement, but in this case we are asking not about the fall of a single domino, but rather about how the dominos are arranged. We are asking whether the tipping of one domino is likely to result in an isolated fall, two isolated falls, a small handful of toppled dominos, or whether it will knock over the entire chain.
If I may be permitted to adopt the antipolarity of "conservatism" - i.e., asking how soon things could conceivably happen, rather than how late - then I must observe that we have no idea where the point of open-ended self-improvement is located, and furthermore, no idea how fast progress will occur after this point is reached. Lest we overestimate the total amount of intelligence required, it should be noted that nondeliberate evolution did eventually stumble across general intelligence; it just took a very long time. We do not know how much improvement over evolution's incremental steps is required for a strongly self-improving system to knock over dominos of sufficient size that each one triggers the next domino. Currently, I believe the best strategy for AI development is to try for general cognition as a necessary prerequisite of achieving the domino effect. But in theory, general cognition might not be required. Evolution managed without it. (In a sense this is disturbing, since, while I can see how it would be theoretically possible to bootstrap from a nondeliberative core, I cannot think of a way to place such a nondeliberative system within the human moral frame of reference.)
It is conceptually possible that a basic bound rules out all improvement of effective intelligence past our current level, but we have no evidence supporting such a bound. I find it difficult to credit that a bound holding for minds in general on all physical substrates coincidentally limits intelligence to the exact level of the very first hominid subspecies to evolve to the point of developing computer scientists. I find it equally hard to credit bounds that limit strongly self-improving processes to the characteristic speed and behavior of weakly self-improving processes. "Human equivalence", commonly held up as the great unattainable challenge of AI, is a chimera - in the sense of being both a "mythical creature" and an "awkward hybrid". Infrahuman AI and transhuman AI are both plausible as self-consistent durable entities. Human-equivalent AI is not.
Given the tremendous architectural and substrate differences between humans and AIs, and the different expected cognitive advantages, there are no current grounds for depicting an AI that strikes an anthropomorphic balance of domain competencies. Given the difference between weakly recursive self-improvement and strongly recursive self-improvement; given the ladder effect and domino effect in self-enhancement; given the different limiting subjective rates of neurons and transistors; given the potential of minds-in-general to expand hardware; and given that evolutionary history provides no grounds for theorizing that the Homo sapiens sapiens intelligence range represents a special slow zone or limiting point with respect to the development of cognitive systems; therefore, there are no current grounds for expecting AI to spend an extended period in the Homo sapiens sapiens range of general intelligence. Homo sapiens sapiens is not the center of the cognitive universe; we are a noncentral special case.
Under standard folk psychology, whether a task is easy or hard or extremely hard does not change the default assumption that people undertaking a task do so because they expect positive consequences for success. AI researchers continue to try and move humanity closer to achieving AI. However near or distant that goal, AI's critics are licensed under folk psychology to conclude that these researchers believe AI to be desirable. AI's critics may legitimately ask for an immediate defense of this belief, whether AI is held to be five years away or fifty. Although the topic is not covered in this paper, I personally pursue general cognition as a means to seed AI, and seed AI as a means to transhuman AI, because I believe human civilization will benefit greatly from breaching the upper bounds on intelligence that have held for the last fifty thousand years, and furthermore, that we are rapidly heading toward the point where we must breach the current upper bounds on intelligence for human civilization to survive. I would not have written a paper on recursively self-improving minds if I believed that recursively self-improving minds were inherently a bad thing, whether I expected construction to take fifty years or fifty thousand.
| Next: | Conclusion |
| Up: | Levels of Organization in General Intelligence |
| Prev: | 2.7: The deliberation level |