General Intelligence and Seed AI 2.3
Creating Complete Minds Capable of Open-Ended Self-Improvement

Welcome to General Intelligence and Seed AI version 2.3.  The purpose of this document is to describe the principles, paradigms, cognitive architecture, and cognitive components needed to build a complete mind possessed of general intelligence, capable of self-understanding, self-modification, and recursive self-enhancement.
 
Multi-page version:   http://singinst.org/GISAI/
Single-page version:   http://singinst.org/GISAI.html
Printable version:   http://singinst.org/printable-GISAI.html

©2001 by the Singularity Institute for Artificial Intelligence, Inc.  All rights reserved.

Preface

"General Intelligence and Seed AI" is a publication of the Singularity Institute for Artificial Intelligence, Inc., a nonprofit corporation.  You can contact the Singularity Institute at institute@singinst.org.  To support the Singularity institute, visit http://singinst.org/donate.html.  The Singularity Institute is a 501(c)(3) public charity and your donations are tax-deductible to the full extent of the law.  The seed AI project is presently in the design/conceptualization stage and no code has yet been written; additional funding is required before the project can be launched.

"General Intelligence and Seed AI" is written in informal style.  Academic readers, readers seeking a more technical explanation, or readers who prefer a more formal style, may wish to read "Levels of Organization in General Intelligence" instead.

This is a near-book-length explanation.  If you need well-grounded knowledge of the subject, then we highly recommend reading °GISAI straight through.  However, if you need answers immediately, see the Singularity Institute pages on AI for introductory articles.

GISAI is a work in progress.  As of version 2.3, the sections "Paradigms" and "Mind" are complete and self-contained.  The section "Cognition" is in progress and may contain references to unimplemented topics.  As additional topics are published, the minor version number (second digit) increases.

Words defined in the Glossary look like this:
    "A °seed AI is an AI capable of self-understanding, self-modification, and recursive self-enhancement."


Executive Summary and Introduction

Please bear in mind that the following is an introduction only.  It contains some ideas which must be introduced in advance to avoid circular dependencies in the actual explanations, and a general summary of the cognitive architecture so you know where the ideas fit in.  In particular, I am not expecting you to read the introduction and immediately shout:  "Aha!  This is the Secret of AI!"  Some important ideas are described, yes, but just because an idea is necessary doesn't make it sufficient.  Too many of AI's past failures have come of the trophy-hunting mentality, asking which buzzwords the code can be described by, and not asking what the code actually does.
This document is about general intelligence - what it is, and how to build one.  The desired end result is a self-enhancing mind or "seed AI".  Seed AI means that - rather than trying to build a mind immediately capable of human-equivalent or transhuman reasoning - the goal is to build a mind capable of enhancing itself, and then re-enhancing itself with that higher intelligence, until the goal point is reached.  "The task is not to build an AI with some astronomical level of intelligence; the task is building an AI which is capable of improving itself, of understanding and rewriting its own source code.  The task is not to build a mighty oak tree, but a humble seed."  (From 1.1: Seed AI.)

General intelligence itself is huge.  The human brain, created by millions of years of evolution, is composed of a hundred billion neurons connected by a hundred trillion synapses, forming more than a hundred neurologically distinguishable areas.  We should not expect the problem of AI to be easy.  Subproblems of cognition include attention, memory, association, abstraction, symbols, causality, subjunctivity, expectation, goals, actions, introspection, caching, and learning, to cite a non-exhaustive list.  These features are not "emergent".  They are °complex functional adaptations, evolved systems with multiple components and sophisticated internal architectures,  whose functionality must be deliberately duplicated within an artificial mind.  If done right, cognition can support the thoughts implementing abilities such as analysis, design, understanding, invention, self-awareness, and the other facets which together sum to an intelligent mind.  An intelligent mind with access to its own source code can do all kinds of neat stuff, but we'll get into that later.

Different schools of AI are distinguished by different kinds of underlying "mindstuff".  Classical AI consists of "predicate calculus" or "propositional logic", which is to say suggestively named °LISP tokens, plus directly coded procedures intended to imitate human formal logic.  Connectionist AI consists of neurons implemented on the token level, with each neuron in the input and output layers having a programmer-determined interpretation, plus intervening layers which are usually not supposed to have a direct interpretation, with the overall network being trained by an external algorithm to perform perceptual tasks.  (Although more biologically realistic implementations are emerging.)  Agent-based AI consists of hundreds of humanly-written pieces of code which do whatever the programmer wants, with interactions ranging from handing data structures around to tampering with each other's behaviors.

Seed AI inherits connectionism's belief that error tolerance is a good thing.  Error tolerance leads to the ability to mutate.  The ability to mutate leads to evolution.  Evolution leads to rich complexity - "mindstuff" with lots of tentacles and interconnections.  However, connectionist theory presents a dualistic opposition between °stochastic, error-tolerant neurons and the crystalline fragility of code or assembly language.  This conflates two logically distinct ideas.  It's possible to have crystalline neural networks in which a single error breaks the chain of causality, or stochastic code in which (for example) multiple, mutatable implementations of a function point have tweakable weightings.  Seed AI strongly emphasizes the necessity of rich complexity in cognitive processes, and mistrusts classical AI's direct programmatic implementations.

However, seed AI also mistrusts that connectionist position which holds higher-level cognitive processes to be sacrosanct and opaque, off-limits to the human programmer, who is only allowed to fool around with neuron behaviors and training algorithms, and not the actual network patterns.  Seed AI does prefer learned concepts to preprogrammed ones, since learned concepts are richer.  Nonetheless, I think it's permissible, if risky, to preprogram concepts in order to bootstrap the AI to the point where it can learn.  More to the point, it's okay to have an architecture where, even though the higher levels are stochastic or self-organizing or emergent or learned or whatever, the programmer can still see and modify what's going on.  And it is necessary that the designer know what's happening on the higher levels, at least in general terms, because cognitive abilities are not emergent and do not happen by accident.  Both classical AI and connectionist AI propose a kind of magic that avoids the difficulty of actually implementing the higher layers of cognition.  Classical AI states that a LISP token named "goal" is a goal.  Connectionist AI declares that it can all be done with neurons and training algorithms.  Seed AI admits the necessity of confronting the problem directly.

In the human brain, there's at least one multilevel system where the higher levels, though stochastic, still have known interpretations: the visual processing system.  Feature extraction by the visual cortex and associated areas doesn't proceed in a strict hierarchy with numbered levels (seed AI mistrusts that sort of thing), but there are definitely lower-level features (such as retinal pixels), mid-level features (such as edges and surface textures), and high-level features (such as 3D shapes and moving objects).  Together, the pixels and attached interpretations constitute the cognitive object that is a visual description.  It's also possible to run the feature-extraction system in reverse, activate a high-level feature and have it draw in the mid-level features which draw in the low-level features.  Such "reversible patterns" are necessary-but-not-sufficient to memory recall and directed imagination.  Memory and imagination, when implemented via this method, can hold rich concepts that mutate interestingly and mix coherently.  A mental image of a red sausage can mutate directly to a mental image of a blue sausage without either storing the perception of redness in a single crystalline token or mutating the image pixel by independent pixel.  °David Marr's paradigm of the "two-and-a-half dimensional world", multilevel holistic descriptions, is writ large and held to apply not just to sensory feature extraction but to categories, symbols, and other concepts.  If seed AI has a "mindstuff", this is it.

Seed AI also emphasizes the problem of sensory modalities (such as the visual cortex, auditory cortex, and sensorimotor cortex in humans), previously considered a matter for specialized robots.  A sensory modality consists of data structures suited to representing the "pixels" and features of the target domain, and codelets or processing stages which extract mid-level and high-level features of that domain.  Sensory modalities grant superior intuitions and visualizational power in the target domain, which itself is sufficient reason to give a self-modifying AI a sensory modality for source code.  Sensory modalities can also provide useful metaphors and concrete substrate for abstract reasoning about other domains; you can play chess using your visual cortex, or imagine a "branching" if-then-else statement.  Sensory modalities provide a source of computational "raw material" from which concepts can form.  Finally, a sensory modality provides intuitions for understanding concrete problems in a training domain, such as source code.  This makes it possible for the AI to learn the art of abstraction - moving from concrete problems, to categorizing sensory data, to conceptualizing complex methods, and so on - instead of being expected to swallow high-level thought all at once.

Sensory modalities are the foundations of intelligence - a term carefully selected to reflect necessity but not sufficiency; after you build the foundations, there's still a lot of house left over.  In particular, a codic modality does not write source code, just as the visual cortex does not design skyscrapers.  When I speak of a "codic" sensory modality, I am not extending the term "sensory modality" to include an autonomous facility for writing source code.  I am using "modality" in the original sense to describe a system almost exactly analogous to the visual cortex, just operating in the domain of source code instead of pixels.

Sensory modalities - visual, spatial, codic - are the bottom layer of the AI, the layer in which representations and behaviors are specified directly by the programmer.  (Although avoiding the crystalline fragility of classical AI is still a design goal.)  The next layer is concepts.  Concepts are pieces of mindstuff, which can either describe the mental world, or can be applied to alter the mental world.  (Note that successive concepts can be applied to a single target, building up a complex visualization.)  Concepts are contained in long-term memory.  Categories, symbols, and most varieties of declarative memory are concepts.  Concepts are more powerful if they are learned, trained, or otherwise created by the AI, but can be created by the programmer for bootstrapping purposes.  (If, of course, the programmer can hack the tools necessary to modify the concept level.)  The underlying substrate of the concept can be code, assembly language, or neural nets, whichever is least fragile and is easiest to understand and mutate; this issue is discussed later, but I currently lean towards code.  (Not raw code, of course, but code as it is understood by the AI.)

Concepts, when retrieved from long-term memory, built into a structure, and activated, create a thought.  The archetypal example of a thought is building words - symbols - into a grammatical sentence and "speaking" them within the mind.  Thoughts exist in the RAM of the mind, the "working memory" created by available workspace in the sensory modalities.  During their existence, thoughts can modify that portion of the world-model currently being examined in working memory.  (Not every sentence spoken within the mind is supposed to describe reality; thoughts can also create and modify °subjunctive ("what-if") hypotheses.)  Thoughts are identified with - supposed to implement the functionality of - the human "stream of consciousness".

The three-layer model of intelligence is necessary, but not sufficient.  Building an AI "with sensory modalities, concepts, and thoughts" is no guarantee of intelligence.  The AI must have the right sensory modalities, the right concepts, and the right thoughts.

Evolution is the cause of intelligence in humans.  Intelligence is an evolutionary advantage because it enables us to model, predict, and manipulate reality, including that portion of reality consisting of other humans and ourselves.  In our physical Universe, reality tends to organize itself along lines that might be called "°holistic" or "°reductionist", depending on whether you're looking up or looking down.  "Which facts are likely to reappear?  The simple facts.  How to recognize them?  Choose those that seem simple.  Either this simplicity is real or the complex elements are indistinguishable.  In the first case we're likely to meet this simple fact again either alone or as an element in a complex fact.  The second case too has a good chance of recurring since nature doesn't randomly construct such cases."  (Robert M. Pirsig, "Zen and the Art of Motorcycle Maintenance", p. 238.)

Thought takes place within a causal, goal-oriented, "reductholistic" world-model, and seeks to better understand the world or invent solutions to a problem.  Some methods include:  °Holistic analysis:  Taking a known high-level characteristic of a known high-level object ("birds fly"), and using °heuristics (thought-level knowledge learned from experience) to try and construct an explanation for the characteristic; an explanation consists of a low-level structure which gives rise to that high-level characteristic in a manner consistent with all known facts about the high-level object ("a bird's flapping wings push it upwards").  Causal analysis:  Taking a known fact ("my telephone is ringing") and using heuristics to construct a causal sequence which results in that fact ("someone wants to speak to me").  Holistic design:  Taking a high-level characteristic as a design goal ("go fast"), using heuristics to reduce the search space by reasoning about constraints and opportunities in possible designs ("use wheels"), and then testing ideas for specific low-level structures that attempt to satisfy the goals ("bicycles").

Both understanding and invention are fundamentally and messily recursive; whether a bicycle works depends on the design of the wheels, and whether a wheel works depends on whether that wheel consists of steel, rubber or tapioca pudding.  Hence the need for °heuristics that bind high-level characteristics to low-level properties.  Hence the need to recurse on finding new heuristics or more evidence or better tools or greater intelligence or higher self-awareness before the ultimate task can be solved.  Solving a problem gives rise to lasting self-development as well as immediate solutions.

When a sufficiently advanced AI can bind a high-level characteristic like "word-processing program" through the multiple layers of design to individual lines of code, °ve can write a word-processing program given the verbal instruction of "Write a word-processing program."  (Of course, following verbal instructions also assumes speech recognition and language processing - not to mention a very detailed knowledge of what a word-processing program is, what it does, what it's for, how humans will use it, and why the program shouldn't erase the hard drive.)  When the AI, perhaps given a sensory modality for atoms and molecules, can understand all the extant research on molecular manipulation, °ve can work out a sequence of steps which will result in the construction of a general nanotechnological assembler, or tools to build one.  When the AI can bind a high-level characteristic like "useful intelligence" through the multiple layers of designed cognitive processes to individual lines of code, ve can redesign °vis own source code and increase vis intelligence.

Developing such a seed AI may require a tremendous amount of programmer effort and programmer creativity; it is entirely possible that a seed AI is the most ambitious software project in history, not just in terms of the end result, but in terms of the sheer depth of internal design complexity.  To bring the problem into the range of the humanly solvable, it is necessary that development be broken up into stages, so that the first stages of the AI can assist with later stages.  The usual aphorism is that 10% of the code implements 90% of the functionality, which suggests one approach.  Seed AI adds the distinction between learned concepts and programmer-designed concepts.  If so, the first stage might be an AI with simplified modalities, preprogrammed simple concepts, low-level goal definitions, and perhaps even programmer-assisted development of the stream-of-consciousness reflexes needed for coherent thought.  Such an AI would hopefully be capable of manipulating code in simple ways, thus rendering the source code for concepts (and in fact its own source code) subject to the type of flexible and useful mutations needed to learn rich concepts or evolve more optimized code.  The skeleton AI helps us fill in the flesh on the skeleton.

...
..
.

Have you got all that?

Good.

Take a deep breath.

We're ready to begin.

1: Paradigms

1.1: Seed AI

It is probably impossible to write an AI in immediate possession of human-equivalent abilities in every field; transhuman abilities even more so, since there's no working model.  The task is not to build an AI with some astronomical level of intelligence; the task is building an AI which is capable of improving itself, of understanding and rewriting its own source code.  The task is not to build a mighty oak tree, but a humble seed.

As the AI rewrites itself, it moves along a trajectory of intelligence.  The task is not to build an AI at some specific point on the trajectory, but to ensure that the trajectory is open-ended, reaching human equivalence and transcending it.  Smarter and smarter AIs become better and better at rewriting their own code and making themselves even smarter.  When writing a seed AI, it's not just what the AI can do now, but what it will be able to do later.  And the problem isn't just writing good code, it's writing code that the seed AI can understand, since the eventual goal is for it to rewrite its own assembly language.  (1).

If "recursive self-enhancement" is to avoid running out of steam, it's necessary for code optimization or architectural changes to result in an increment of actual intelligence, of smartness, not just speed.  Running an optimizing compiler over its own source code (2) may result in a faster optimizing compiler.  Repeating the procedure a second time accomplishes nothing, producing an identical set of binaries, since the same algorithm is being run - only faster.  A human who fails to solve a problem in one year (or solves it suboptimally) may benefit from another ten years to think about the problem; even so, an individual human may eventually run out of ideas.  An individual human who fails to solve a problem in a hundred years may, if somehow transformed into an Einstein, solve it within an hour.  Faster unintelligent algorithms accomplish little or nothing; faster intelligent thought can make a small difference; better intelligent thought makes the problem new again.

If each rung on the ladder of recursive self-enhancement involves a leap of sufficient magnitude, then each rung should open up enough new vistas of self-improvement for the next rung to be reached.  If not, of course, the seed AI will have optimized itself and used up all perceived opportunities for improvement without generating the insight needed to see new kinds of opportunities.  In this case the seed AI will have stalled, and it will be time for the human programmers to go to work nudging it over the bottleneck.  Ultimately, the AI must cross, not only the gap that separates the mythical average human from Einstein, but the gap that separates homo sapiens neanderthalis from homo sapiens sapiens.  The leap to true understanding, when it happens, will open up at least as many possibilities as would be available to a human researcher with access to vis own neural source code.

A surprisingly frequent objection to self-enhancement is that intelligence, when defined as "the ability to increase intelligence", is a circular definition - one which would, they say, result in a sterile and uninteresting AI.  Even if this were the definition (it isn't), and the definition were circular (it wouldn't be), the cycle could be broken simply by grounding the definition in chess-playing ability or some similar test of ability.  However, intelligence is not defined as the ability to increase intelligence; that is simply the form of intelligent behavior we are most interested in.  Intelligence is not defined at all.  What intelligence is, if you look at a human, is more than a hundred °cytoarchitecturally distinct areas of the brain, all of which work together to create intelligence.  Intelligence is, in short, modular, and the tasks performed by individual modules are different in kind from the nature of the overall intelligence.  If the overall intelligence can turn around and look at a module as an isolated process, it can make clearly defined performance improvements - improvements that eventually sum up to improved overall intelligence - without ever confronting the circular problem of "making itself more intelligent".  Intelligence, from a design perspective, is a goal with many, many subgoals.  An intelligence seeking the goal of improved intelligence does not confront "improved intelligence" as a naked fact, but as a very rich and complicated fact adorned with less complicated subgoals.

Presumably there is an ultimate limit to the intelligence that can be achieved on a given piece of hardware, but if the seed AI can design better hardware, the cycle continues.  To be concrete, if a seed AI is smart enough to chart a path from modern technological capabilities to °nanotechnology - to the hardware described in K. Eric Drexler's Nanosystems - this should be enough computing power to provide thousands or millions of times the raw capacity of a human brain.  (3).  Whether the cognitive and technological trajectory beyond this point continues forever or tops out at some ultimate physical limit is basically irrelevant from a human perspective; nanotechnology plus thousands of times human brainpower should be far more than enough to accomplish whatever you wanted a transhuman for in the first place.

This scenario often meets with the objection that a lone AI can accomplish nothing; that technological advancement requires an entire civilization, with exchanges between thousands of scientists or millions of humans.  This actually understates the problem.  To think a single thought, it is necessary to duplicate far more than the genetically programmed functionality of a single human brain.  After all, even if the functionality of a human were perfectly duplicated, the AI might do nothing but burble for the first year - that's what human infants do.

Perceptions have to coalesce into concepts.  The concepts have to be strung together into thoughts.  Enough good thoughts have to be repeated often enough for the sequences to become °cached, for the often-repeated subpatterns to become reflex.  Enough of these infrastructural reflexes must accumulate for one thought to give rise to another thought, in a connected chain, forming a stream of consciousness.  Unless we want to sit around for years listening to the computer go ga-ga, the functionality of infancy must be either encapsulated in a virtual world that runs in computer time, or bypassed using a skeleton set of preprogrammed concepts and thoughts.  (Hopefully, the "skeleton thoughts" will be replaced by real, learned thoughts as the seed AI practices thinking.)

Human scientific thought relies on millennia of accumulated knowledge, the how-to-think °heuristics discovered by hundreds of geniuses.  While a seed AI may be able to absorb some of this knowledge by surfing the 'Net, there will be other dilemnas, unique to seed AIs, that it must solve on its own.

Finally, the autonomic processes of the human mind reflect millions of years of evolutionary optimization.  Unless we want to expend an equal amount of programming effort, the functionality of evolution itself must be replaced - either by the seed AI's self-tweaking of those algorithms, or by replacing processes that are autonomic in humans with the deliberate decisions of the seed AI.

That's a gargantuan job, but it's matched by equally powerful tools.

1.1.1: The AI Advantage

The traditional advantages of computer programs - not "AI", but "computer programs" - are threefold:  The ability to perform repetitive tasks without getting bored; the ability to perform algorithmic tasks at greater linear speeds than our 200-°hertz neurons permit; and the ability to perform complex algorithmic tasks without making mistakes (or rather, without making those classes of mistakes which are due to distraction or running out of short-term memory).  All of which, of course, has nothing to do with intelligence.

The toolbox of seed AI is yet unknown; nobody has built one.  This page is more about building the first stages, the task of getting the seed AI to say "Hello, world!"  But, if this can be done, what advantages would we expect of a general intelligence with access to its own source code?

The ability to design new sensory modalities.  In a sense, any human programmer is a blind painter - worse, a painter born without a visual cortex.  Our programs are painted pixel by pixel, and are accordingly sensitive to single errors.  We need to consciously keep track of each line of code as an abstract object.  A seed AI could have a "codic cortex", a sensory modality devoted to code, with intuitions and instincts devoted to code, and the ability to abstract higher-level concepts from code and intuitively visualize complete models detailed in code.  A human programmer is very far indeed from vis ancestral environment, but an AI can always be at home.  (But remember:  A codic modality doesn't write code, just as a human visual cortex doesn't design skyscrapers.)

The ability to blend conscious and autonomic thought.  Combining °Deep Blue with °Kasparov doesn't yield a being who can consciously examine a billion moves per second; it yields a Kasparov who can wonder "How can I put a queen here?" and blink out for a fraction of a second while a million moves are automatically examined.  At a higher level of integration, Kasparov's conscious perceptions of each consciously examined chess position may incorporate data culled from a million possibilities, and Kasparov's dozen examined positions may not be consciously simulated moves, but "skips" to the dozen most plausible futures five moves ahead.  (5).

Freedom from human failings, and especially human politics.  The tendency to rationalize untenable positions to oneself, in order to win arguments and gain social status, seems so natural to us; it's hard to remember that rationalization is a °complex functional adaptation, one that would have no reason to exist in "minds in general".  A synthetic mind has no political instincts (6); a synthetic mind could run the course of human civilization without politically-imposed dead ends, without °observer bias, without the tendency to rationalize.  The reason we humans instinctively think that progress requires multiple minds is that we're used to human geniuses, who make one or two breakthroughs, but then get stuck on their Great Idea and oppose all progress until the next generation of brash young scientists comes along.  A genius-equivalent mind that doesn't age and doesn't rationalize could encapsulate that cycle within a single entity.

Overpower - the ability to devote more raw computing power, or more efficient computing power, than is devoted to some module in the original human mind; the ability to throw more brainpower at the problem to yield intelligence of higher quality, greater quantity, faster speed, even difference in kind.  Deep Blue eventually beat Kasparov by pouring huge amounts of computing power into what was essentially a glorified search tree; imagine if the basic component processes of human intelligence could be similarly overclocked...

Self-observation - the ability to capture the execution of a module and play it back in slow motion; the ability to watch one's own thoughts and trace out chains of causality; the ability to form concepts about the self based on fine-grained introspection.

Conscious learning - the ability to deliberately construct or deliberately improve concepts and memories, rather than entrusting them to autonomic processes; the ability to tweak, optimize, or debug learned skills based on deliberate analysis.

Self-improvement - the ubiquitous glue that holds a seed AI's mind together; the means by which the AI moves from crystalline, programmer-implemented skeleton functionality to rich and flexible thoughts.  In the human mind, °stochastic concepts - combined answers made up of the average of many little answers - leads to error tolerance; error tolerance lets concepts mutate without breaking; mutation leads to evolutionary growth and rich complexity.  An AI, by using probabilistic elements, can achieve the same effect; another route is deliberate observation and manipulation, leading to deliberate "mutations" with a vastly lower error rate.  What are these mutations or manipulations?  A blind search can become a heuristically guided search and vastly more useful; an autonomic process can become conscious and vastly richer; a conscious process can become autonomic and vastly faster - there is no sharp border between conscious learning and tweaking your own code.  And finally, there are high-level redesigns, not "mutations" at all, alterations which require too many simultaneous, non-backwards-compatible changes to ever be implemented by evolution.

If all of that works, it gives rise to self-encapsulation and recursive self-enhancement.  When the newborn mind fully understands vis own source code, when ve fully °understands the intelligent reasoning that went into vis own creation - and when ve is capable of °inventing that reason independently, so that the mind contains its own design - the cycle is closed.  The mind causes the design, and the design causes the mind.  Any increase in intelligence, whether sparked by hardware or software, will result in a better mind; which, since the design was (or could have been) generated by the mind, will propagate to cause a better design; which, in turn, will propagate to cause a better mind.  (7).  And since the seed AI will encapsulate not only the functionality of human individual intelligence but the functionality of evolution and society, these causes of intelligence will be subject to improvement as well.  We might call it a "civilization-in-a-box", an entity with more "hardware" intelligence than Einstein (8) and capable of codifying abstract thought to run at the linear speed of a modern computer.

A successful seed AI would have power.  A genuine civilization-in-a-box, thinking at a millionfold human speed, might fold centuries of technological progress into mere hours.  I won't beat the point to death.  I've done so in my other writings - Staring into the Singularity, in particular.  It's just that the fundamentalpurpose of transhuman AI differs from that of traditional AI.

The academic purpose of modern prehuman AI is to write programs that demonstrate some aspect of human thought - to hold a mirror up to the brain.  The commercial purpose of prehuman AI is to automate tasks too boring, too fast, or too expensive for humans.  It's possible to dispute whether an academic implementation actually captures an aspect of human intelligence, or whether a commercial application performs a task that deserves to be called "intelligent".

In transhuman AI, if success isn't blatantly obvious to everyone except trained philosophers, the effort has failed.  The ultimate purpose of transhuman AI is to create a °Transition Guide; an entity that can safely develop °nanotechnology and any subsequent ultratechnologies that may be possible, use transhuman °Friendliness to see what comes next, and use those ultratechnologies to see humanity safely through to whatever life is like on the other side of the Singularity.  This might consist of assisting all humanity in upgrading to the level of superintelligent Powers, or creating an operating system for all the quarks in the Solar System, or something completely unknowable.  I believe that, as the result of creating a °Friendly superintelligence, involuntary death, pain, coercion, and stupidity will be erased from the human condition; and that humanity, or whatever we become, will go on to fulfill to the maximum possible extent whatever greater destiny or higher goals exist, if any do.

To return to Earth:  There will undoubtedly be many milestones, many interim subgoals and interim successes, along the path to superintelligence.  The key point is that while embodying some aspect of cognition may be useful or necessary, it is not an end in itself.  Treating facets of cognition as ends in themselves has led traditional AI to develop a sort of "trophy mentality", a tendency to value programs according to whether they fit surface descriptions.  (One gets the impression that if you asked certain AI researchers to write the next Great English Novel, they'd write a 20-page essay on toaster ovens and then tear off through the streets, shouting:  "Eureka!  It's in English!  It's in English!")  My hope is that the lofty but utilitarian goals of seed AI will lead to the habit of looking at every piece of the design and saying:  "Sure, it sounds neat, but how does it contribute materially to general intelligence?"  After all, if an aspect of cognition is duplicated faithfully but without understanding its overall purpose, it's a matter of pure faith to expect it to contribute anything.

But that brings us to the next section, "Thinking About AI".

1.2: Thinking About AI

AI has, in the past, failed repeatedly.  The shadow cast by this failure falls over all proposals for new AI projects.  The question is always asked:  "Why won't your project fail, like all the other projects?  Why did the previous projects fail?  Does your theory of general intelligence explain the previous failures while predicting success for your own efforts?"  Actually, anyone can explain away previous failures and predict success; all you have to do is assert that some particular new characteristic is the One Great Idea, necessary and sufficient to intelligence.  The real question is whether a new approach to AI makes the failure of previous efforts seem massively inevitable, the predictable result of historical factors; whether the approach provides a theory of previous failures that is satisfyingly obvious in retrospect, makes earlier errors look like natural mistakes that any growing civilization might make, and thus "swallows" the historical failures in a new theory which leaves no dangling anxieties.

Okay.  I won't go quite that far.  Still, AI has an embarassing tendency to predict success where none materializes, to make mountains out of molehills, and to assert that some simpleminded pattern of suggestively-named °LISP tokens completely explains some incredibly high-level thought process.  Why?

Consider the symbol your mind contains for 'light bulb'.  In your mind, the sounds of the spoken words "light bulb" are reconstructed in your auditory cortex.  A picture of a light bulb is loaded into your visual cortex.  Furthermore, the auditory and visual cortices are far more complex, and intelligent, than the algorithm your computer uses to play sounds and MPEG files.  Your auditory cortex has evolved specifically to process incoming speech sounds, with better fineness and resolution than it displays on other auditory tasks.  Your visual cortex does not simply contain a 2D pixel array.  The visual cortex has specialized processes that extract David Marr's "two-and-a-half dimensional world" - edge detection, corner interpretation, surfaces, shading, movement - and processes that extract from this a model of 3D objects in a 3D world.  "About 50 percent of the cerebral cortex of primates is devoted exclusively to visual processing, and the estimated territory for humans is nearly comparable."  (°MITECS, "Mid-Level Vision".)

In the semantic net or Physical Symbol System of classical AI, a light bulb would be represented by an atomic LISP token named light-bulb.

NOTE: I say "LISP tokens", not "LISP symbols", despite convention and accepted usage.  Calling the lowest level of the system "symbols" is a horrifically bad habit.

Some of the problem may be explained by history; back when AI was being invented, in the 1950s and 1960s, researchers had tiny little machines that modern pocket calculators would sneer at.  These early researchers chose to believe they could succeed with "symbols" composed of small LISP structures, cognitive "processes" with the complexity of one subroutine in a modern class library.  They were wrong, but the need to believe produced approaches and paradigms that sank AI for decades.

Previous AI has been conducted under the Physicist's Paradigm.  The development of physics over the past few centuries - at least, the dramatic, stereotypical part - has been characterized by the discovery of simple equations that neatly account for complex phenomena.  In physics, the task is finding a single bright idea that explains everything.  Newton took a single assumption (masses attract each other with a force equal to the product of the masses divided by the square of the distance) and churned through some calculus to show that, if an apple falls towards the ground at a constant acceleration, then this explains why planets move in elliptical orbits.  The search for a similar fits-on-a-T-Shirt unifying principle to fully explain a brain with hundreds of °cytoarchitecturally distinct areas has wreaked havoc on AI.

"Heuristics are compiled hindsight; they are judgemental rules which, if only we'd had them earlier, would have enabled us to reach our present state of achievement more rapidly."  (Douglas Lenat, 1981.)  The heuristic learned from past failures of AI might be titled "Necessary, But Not Sufficient".  Whenever neural networks are mentioned in press releases, the blurb always includes the phrase "neural networks, which use the same parallel architecture found in the human brain".  Of course, the "neurons" in neural networks are usually nothing remotely like biological neurons.  But the main thing that gets overlooked is that it would be equally true (not very) to say that neural networks use the same parallel architecture found in an earthworm's brain.  Regardless of whether neural networks are Necessary, they are certainly Not Sufficient.  The human brain requires millions of years of evolution, thousands of modules, hundreds of thousands of adaptations, on top of the simple bright idea of "Hey, let's build a neural network!"

The Physicist's Paradigm lends itself easily to our need for drama.  One great principle, one bold new idea, comes along to overthrow the false gods of the old religion... and set up a new bunch of false gods.  As always when trying to prove a desired result from a flawed premise, the simplest path involves the Laws of Similarity and Contagion.  For example, the "neurons" in neural networks involve associative links of activation.  Therefore, the extremely subtle and high-level associative links of human concepts must be explained by this low-level property.  Similarly, any instance of human deduction which can be written down (after the fact) as a syllogism must be explained by the blind operation of a ten-line-of-code process - even if the human thoughts blatantly involve a rich visualization of the subject matter, with the results yielded by direct examination of the visualization rather than formal deductive reasoning.

In AI, the one great simple idea usually operates on a low level, in accordance with the Physicist's Paradigm.  Reasoning from similarity of surface properties is used to assert that high-level cognitive phenomena are explained by the low-level phenomenon, which (it is claimed) is both Necessary and Sufficient.  This cognitive structure is a full-blown fallacy; it contains the social drama (one brilliant idea, new against old) and the rationalization (reasoning by similarity of surface properties, sympathetic magic) necessary to bear any amount of emotional weight.  And that's how AI research goes wrong.

There are several ways to avoid making this class of mistake.  One is to have the words "Necessary, But Not Sufficient" tattooed on your forehead.  One is an intuition of causal analysis that says "This cause does not have sufficient complexity to explain this effect."  One is to be instinctively wary of attempts to implement cognition on the token level.  (One is learning enough evolutionary psychology to recognize and counter ideology-based thoughts directly, but that's moving off-topic...)

One is introspection.  Human introspection currently has a bad reputation in cognitive science, looked on as untrustworthy, unscientific, and easy to abuse.  This is totally true.  Still, you can't build a mind without a working model.  It is necessary to know, intuitively, that classical-AI propositional logic - syllogisms, property inheritance, et cetera - is inadequate to explain your deduction that dropping an anvil on a car will break it.  You should be able to see, introspectively, that there's more than that going on.  You can visualize an anvil smashing into your car's hood, the metal crumpling, and the windshield shattering.  (9).  Clearly visible is vastly more mental material, more cognitive "stuff", than classical-AI propositional logic involves.

The revolt against the Physicist's Paradigm can be formalized as the Law of Pragmatism:

The Law of Pragmatism
Any form of cognition which can be mathematically formalized, or which has a provably correct implementation, is too simple to contribute materially to intelligence.

The key words are "contribute materially".  An architecture can be necessary to thought without accounting for the substance of thought.  The Law of Pragmatism says that if a neural network's rules are simple enough to be formalized mathematically, than the substance of any intelligent answers produced by that network will be attributable to the specific pattern of weightings.  If the pattern of weightings is created by a mathematically formalizable learning method, then the substance of intelligence will lie, not in the learning method, but in the intricate pattern of regularities within the training instances.

We can't be certain that the Law of Pragmatism will hold in the future, but it's definitely a heuristic in the Lenatian sense; if only we'd known it in the 1950s, so much error could have been avoided.  The Law of Pragmatism is one of the tools used to determine whether an idea is Necessary, But Not Sufficient.  (11).

°GISAI proposes a mind which contains modules vaguely analogous to human sensory modalities (auditory cortex, visual cortex, etc.).  This does not mean that you can design any old system which can be described as "containing modular sensory modalities" and then dash off a press release about how your company is building an AI containing modular sensory modalities.  That's the trophy mentality I was talking about earlier.  A modular, modality-based system is Necessary, But Not Sufficient; it is also necessary to have the right modules, in the right sensory modalities, using the right representation and the right intuitions to process the right base of experience to produce the right concepts that support the right thoughts within the right larger architecture.

When you think of a light bulb, the syllables and phonemes of "light bulb" are loaded into your auditory cortex; if you're a visual person, a generic picture of a light bulb - the default exemplar - appears in your visual cortex.  Let's suppose that some AI has reasonably sophisticated analogues of the auditory cortex and visual cortex, capable of perceiving higher-level features as well as the raw binary data.  This is clearly necessary; is it sufficient to understand light bulbs in the same way as a human?

No.  Not even close.  When you hear the phrase "triangular light bulb", you visualize a triangular light bulb.

NOTE: Please halt, close your eyes, and visualize a triangular light bulb.  Please?  Pretty please with sugar on top?

How do these two symbols combine?  You know that light bulbs are fragile; you have a built-in comprehension of real-world physics - sometimes called "naive" physics - that enables you to understand fragility.  You understand that the bulb and the filament are made of different materials; you can somehow attribute non-visual properties to pieces of the three-dimensional shape hanging in your visual cortex.  If you try to design a triangular light bulb, you'll design a flourescent triangular loop, or a pyramid-shaped incandescent bulb; in either case, unlike the default visualization of "triangle", the result will not have sharp edges.  You know that sharp edges, on glass, will cut the hand that holds it.

Look at all that!  It requires a temporal, four-dimensional understanding of the light bulb.  It requires an appreciation, a set of intuitions, for cause and effect.  It requires that you be capable of spotting a problem - a conflict with a goal - which requires means for representing conflicts, and cognitive reflexes derived from a goal system.

Look at yourself "looking at all that".  It requires introspection, reflection, self-perception.  It requires an entire self-sensory modality - representations, intuitions, cached reflexes, expectations - focused on the mind doing the thinking.

For you to read this paragraph, and think about it, requires a stream of consciousness.  For you to think about light bulbs implies that you codified your past experiences of actual light bulbs into the representation used by your long-term memory.  The visual image of the light bulb, appearing in your visual cortex, implies that a default exemplar for "light bulb" was abstracted from experience, stored under the symbol for "light bulb", and triggered by that symbol's auditory tag of 'light bulb'.  And this exemplar can even be combined with the learned symbol for "triangle".  You have formed an adjective, "triangular", consisting of characteristics which can be applied to modify the visual and design substance of the light-bulb concept.  For you to visualize a light-bulb smashing, with an accompanying tinkling noise, requires synchronization of recollection and reconstruction across multiple sensory modalities.

I've mentioned many features in the last paragraphs; none of them are emergent.  None of them will magically pop into existence on the high level "if only the simple low-level equation can be found".  In a human, these features are °complex functional adaptations, generated by millions of years of evolution.  For an AI, that means you sit down and write the code; that you change the design, or add design elements (special-purpose low-level code that directly implements a high-level case is usually a Bad Thing), specifically to yield the needed result.

In short, the design in GISAI is simply far larger, as a system architecture, than any design which has been previously attempted.  It's large enough to resemble systems of the complexity described in the 471 articles in °The MIT Encyclopedia of the Cognitive Sciences.  (12).  You'll appreciate this better after reading the rest of the document, of course, but when you have done so, I expect that seed AI will look too different from past failures for one to reflect on the other.  Fish and fowl, apples and oranges, elephants and typewriters.  There is still the possibility that any given seed AI project will fail, or even that seed AI itself will fail - but if so, it will fail for different reasons.

2: Mind

2.1: World-model

Intelligence is an evolutionary advantage because it enables us to model, predict, and manipulate reality.  This includes not only Joe Caveman (or rather, Pat Hunter-Gatherer) inventing the bow and arrow, but Chris Tribal-Chief outwitting his (13) political rivals and Sandy Spear-Maker realizing that the reason her spears keep breaking is that she's being too impatient while making them.  That is, the "reality" we model includes not just things, but other humans, and the self.  (14).

A chain of reasoning is important because it ends with a conclusion about how the world works, or about how the world can be altered.  The "world", for these purposes, includes the internal world of the AI; when designing a bicycle, the hypothesis "a round object can traverse ground without bumping" is a statement about the external world.  The hypotheses "it'd be a good idea to think about round objects", or "the key problem is to figure out how to interface with the ground", or even "I feel like designing a bicycle", are statements about the internal world.

From an external perspective, cognitive events matter only insofar as they affect external behavior.  Just so, from an internal perspective, the effect on the world-model is the punchline, the substance.  This is not to say that every line of code must make a change to the world-model, or that the world-model is composed exclusively of high-level beliefs about the real world.  The thought sequences that construct a what-if scenario - a °subjunctive fantasy world - are altering a world-model, even if it's not the model of the world.  A "vague feeling that there's some kind of as-yet unnamed similarity between two pictures" is part of the content of the AI's beliefs about the world.  The code that produces that intuition may undergo many internal iterations, acting on data structures with no obvious correspondence to the world-model, before producing an understandable output.

What makes a pattern of bytes - or neurons - a "model"?  And what makes a particular statement in that model "true" or "false"?  (15).  The best definition I've found is derived from looking at the cause of our intelligence:  "Intelligence is an evolutionary advantage because it enables us to model, predict, and manipulate reality."  Models are useful because they correspond to external reality.

I distinguish four levels of binding:

These definitions raise an army of fundamental issues - °time, causality, °subjunctivity, goals, searching, invention - but first, let's look at a concrete example.  Imagine a °microworld composed of Newtonian billiard balls - a world of spheres (or circles), each with a position, radius, mass, and velocity, interacting on some frictionless surface (or moving in a two-dimensional vacuum).  (16).

The "world-model" for an AI living in that °microworld consists of everything the AI knows about that world - the positions, velocities, radii, and masses of the billiard balls.  More abstract perceptions, such as "a group of °three billiard balls", are also part of the world-model.  The prediction that "billiard ball A and billiard ball B will collide" is part of the world-model.  If the AI imagines a situation where four billiard balls are arranged in a square, then that imaginary world has its own, °subjunctive world-model.  If the AI believes "'imagining four billiard balls in a square' will prove useful in solving problem X", then that belief is part of the world-model.  In short, the world-model is not necessarily a programmatic concept - a unified set of data structures with a common format and °API.  (Although it would be wonderfully convenient, if we could pull it off.)  The "world-model" is a cognitive concept; it refers to the content of all beliefs, the substance of all mental imagery.

Returning to the billiard-ball world, what is necessary for an AI to have a "model" of this world?

In the last case, the AI may have been able to manipulate each of the six billiard balls as a separate object, or each action may have affected multiple balls simultaneously, requiring a more complex planning process.  The important thing is that "creating two symmetrical groups of three billiard balls" is not something that would happen by chance, or be uncovered by a blind search.  For the AI to create a structure of billiard balls, it will need °heuristics - knowledge about rules - that not only link outcomes to actions, but reverse the process to link actions to outcomes.

Suppose that a cue ball travelling south at 4 meters/second, bumping into a billiard ball travelling south at 2 meters/second, results in the cue ball and the billiard ball travelling south at 3 meters/second.  Suppose, furthermore, that these rules are contained within the AI's internal model of the environment, so that if the AI visualizes a cue ball at {8.2, 6} of radius 1 travelling south at 4 m/s, and a ball at {8.2, 10} of radius 1 going south at 2 m/s, the AI will visualize the balls bumping one second later at {8.2, 11}, and the two balls then travelling south at 3 m/s.  It's a long way from there to knowing - consciously, °declaratively - that two balls in general bumping at 4 m/s and 2 m/s while going in the same direction will travel on together at 3 m/s.  It's an even longer way to knowing that "if billiard ball X bumps into billiard ball Y, then they will continue on together with the average of their velocities".  And it's a still longer way to reversing the rule and knowing that "to get a group of two balls travelling together with velocity X, given billiard ball A with velocity Y, bump it with billiard ball B having velocity (2X - Y)".  Finally, to close the loop, this last high-level rule must be applied to create a particular hypothesized action in the world-model, and the hypothesized action needs to be taken as a real action in external reality.

Without jumping too far ahead, there are a number of properties that a world-model needs to support high-level thought.  It needs to support °time - multiple frames or a temporal visualization - with accompanying extraction of temporal features.  It needs to support predictions and expectations (and an expectation isn't real unless the AI notices when the expectation is fulfilled, and especially when it is violated).  The world-model needs to support hypotheses, °subjunctive frames of visualization, which are distinct from "real reality" and can be manipulated freely by high-level thought.  (By "freely manipulated", I mean a direct manipulative binding; choosing to think about a billiard ball at position {2, 3} should cause a billiard ball to materialize directly within the representation at {2, 3}, with no careful sequence of actions required.)  And for the visualization to be useful once it exists, the high-level thought which created the billiard-ball image must refer to the particular image visualized... and the reference must run both ways, a two-way linkage.

°Time, expectation, comparision, °subjunctivity, visualization, introspection, and reference.  I haven't defined any of these terms yet.  (Most are discussed in 3: Cognition, although you can jump ahead to Appendix A: Glossary if you're impatient.)  Nonetheless, these are some of the basic attributes that are present in human world-models, and which are Necessary (But Not Sufficient) for the existence of high-level features such as causality, intentionality, °goals, memory, learning, association, focus, abstraction, categorization, and symbolization.

NOTE: I mention that list of features to illustrate what will probably be one of the major headaches for AI designers:  If you design a system and forget to allow for the possibility of expectation, comparision, °subjunctivity, visualization, or whatever, then you'll either have to go back and redesign every single component to open up space for the new possibilities, or start all over from scratch. Actualities can always be written in later, but the potential has to be there from the beginning, and that means a designer who knows the requirements spec in advance.

Interlude: The Consensus and the Veil of Maya

In a rainbow, the physical frequency of the light changes smoothly and linearly with distance (19).  Yet, when you look at a rainbow, you see colors grouped into bands, with relatively sharp borders.  And it's not just you.  Everyone sees the bands.

It gets worse.  Consider:  The frequency of light is a linear, scalar, real number.  The visible frequencies of light rise linearly from red to blue, bounded by infrared and ultraviolet.  But if you look at a color wheel on your computer, you'll see that it's a wheel.  Red to orange to yellow to green to blue to... purple? ... and back to red again.  Where does purple come from?  It's a color that doesn't exist, seemingly added on afterwards to turn a linear spectrum into a circle!

It turns out the color purple and the bands in a rainbow are both artifacts of the way humans perceive color space, which in turn is a result of the way our visual cortex has evolved to distinguish objects in the ancestral environment and maintain color constancy under natural lighting.  (For more about this, see "The Perceptual Organization of Colors" in "The Adapted Mind".  It's definitely a cool article.)

The color purple, and the bands in the rainbow, aren't real.  But everyone sees them, so you can't just call them hallucinations.  I prefer to strike a happy compromise and say that purple and rainbows exist in the Consensus.  Nobody actually lives in external reality, and we couldn't understand it if we did; too many quarks flying around.  When we walk through a hall, watching the floor and walls and ceiling moving around us, we're actually walking through our visual cortex.  That's what we see, after all.  We don't see the photons reflected by the walls, and we certainly don't see the walls themselves; every single detail of our perception is there because a neuron is firing somewhere in the visual system.  If the wrong neuron fired, we'd see a spot of color that wasn't there; if a neuron failed to fire, we wouldn't see a spot of color that was there.  From this perspective, the actual photons are almost irrelevant.  Furthermore, all the colors in the hall you're walking through are technically incorrect due to that old color-space thing.  Heck, you might even walk past something purple.

This is the point where the philosopher usually goes off the solipsistic deep end.  "It's all arbitrary!  Nothing is real!  Everything is true!  I can say whatever I want and nobody can do a thing about it, bwahaha!"  I hate this whole line of thinking.  If I ever start sounding like this, check my forehead for lobotomy scars.

The Consensus usually has an extremely tight °sensory, °predictive, and °manipulative binding to external reality.  No, it doesn't work 100% of the time, but it works 99.99% of the time, so the rules are just as strict.  Just because you can't see external reality directly doesn't mean it isn't there.

Everything you see is illusion, the Veil of Maya.  Where Eastern philosophy goes wrong is in assuming that the Veil of Maya is hiding something big and important.  What lies behind the illusion of a brick is the actual brick.  The vast majority of the time, you can forget the Veil of Maya is even there.

Nor does our residence in the Consensus grant the Consensus primacy over external reality.  The Consensus itself is just another part of reality.  That's how reality binds the Consensus; it's just one part of reality affecting another part, under the standard rules of interaction imposed by the laws of physics.  External reality existed before the patterns in reality known as "humans" or "the Consensus".  People who ignore external reality on the grounds that "all truth is subjective" tend to have their constituent quarks assimilated by the quark-patterns we call "tigers".

However, sometimes it's important to remember that tigers only exist in the Consensus.  Suppose someone asks you for a definition of a "tiger", and you give them a definition that works 99.99% of the time - "big orange cat thingy with stripes".  Then whoever it is paints a tiger green and says, "Ha, ha!  Your definition is wrong!"  What I would do in this case is give a more precise definition based on genetics, behavior patterns, and so on, but then you have cyborg tigers and mutant tigers.  At that point, it becomes important to remember that it's "just" the Consensus.  You shouldn't expect things in the Consensus to have perfect mathematical definitions.  Evolution doesn't select for tigers, or tiger-perceiving minds, that have philosophically elegant definitions; evolution selects whatever works most of the time.

So why does the Consensus work?  Because of a fundamental rule of °reductholism:  Forget about definitions.  Anything true "by definition" is a tautology, and bears no relation to external reality - does not even refer to external reality.

Forget about definitions, and if you find that some cognitive perception is inherently °subjective or °observer-dependent - that the perception relies on qualities that exist only in the mind of the observer - then relax and accept it as being useful to intelligence most of the time, and don't go into philosophical fits.  It isn't real, after all, so why should you worry?

Hey, that's life in the Consensus.

DEFN: Consensus:  The Consensus is the world of shared perceptions that humanity inhabits.  Things in the Consensus aren't really really real, but they usually correspond tightly to reality - enough to make the rules about what you can and can't say just as strict.  What distinguishes the Consensus from actual reality is that there is no a priori reason why things should be formalizable, philosophically coherent, or unambiguous.

2.2: Sensory modalities

A human has a visual cortex, an auditory cortex, a sensorimotor cortex - areas of the brain specifically devoted to particular senses.  Each such "cortex" is composed of neural modules which extract important mid-level and high-level features from the low-level data, in a way determined by the "laws of physics" of that domain.  The visual cortex and associated areas (20) are by far the best-understood parts of the brain, so that's what we'll use for an example.

Visual information starts out as light hitting the retina; the resulting information can be thought of as being analogous to a two-dimensional array of pixels (although the neural "pixels" aren't rectangular).  "Low-level" feature extraction starts right in the retina, with neurons that respond to edges, intensity changes, light spots, dark spots, et cetera.  From this new representation - the 2D pixels, plus features like edges, light spots, and so on - the lateral geniculate nucleus and striate cortex extract mid-level features such as edge orientation, movement, direction of moving features, textures, the curvature of textured surfaces, shading, and binocular perception.  This information yields °David Marr's two-and-a-half-dimensional world, which is composed of scattered facts about the three-dimensional properties of two-dimensional features - this is a continuous surface, this surface is curving away and to the left, these two surfaces meet to form an edge, these three edges meet to form a corner.

Finally, a 3D representation of moving objects is constructed from the 2.5D world.  Constraint propagation:  If the 3D interpretation of one corner requires an edge to be convex, then that edge cannot be concave in another corner.  Object assembly:  Multiple surfaces that move at the same speed, or that move in a fashion consistent with rotation, are part of a single object.  Consistency:  An object (or an edge, or a surface) cannot simultaneously be moving in two directions.

The resulting 3D representation, still bound to the 2.5D features and the 2D pixels, is sent to the temporal cortex for object recognition and to the parietal cortex for spatial visualization.

The visual cortex is the foundation of one of the seven senses.  (Yes, at least seven.  In addition to sight, sound, taste, smell, and touch, there's proprioception (the nerves that tell us where our arms and legs are) and the vestibular sense (the inner ear's inertial motion-detectors).  (21).)  The neural areas that are devoted solely to processing one sense or another account for a huge chunk of the human cortex.  In the modular partitioning of the human brain, the single most common type of module is a sensory modality, or a piece of one.  This demonstrates a fundamental lesson about minds in general.

Classical AI programs, particularly "expert systems", are often partitioned into microtheories.  A microtheory is a body of knowledge, i.e. a big semantic net, e.g. propositional logic, a.k.a. suggestively named LISP tokens.  A typical microtheory subject is a human specialty, such as "cars" or "childhood diseases" or "oil refineries".  The content of knowledge typically consists of what would, in a human, be very high-level, heuristic statements:  "A child that is sick on Saturday is more likely to be seriously ill than a child who's sick on a schoolday."

How do the microtheory-based modules of classical AI differ from the sensory modules that are common in the human mind?  How does a "microtheory of vision" differ from a "visual cortex"?  Why did the microtheory approach fail?

There are two fundamental clues that, in retrospect, should have alerted expert-system theorists ("knowledge engineers") that something was wrong.  First, microtheories attempt to embody high-level rules of reasoning - heuristics that require a lot of pre-existing content in the world-model.  The visual cortex doesn't know about butterflies; it knows about edge-detection.  The visual cortex doesn't contain a preprogrammed picture of a butterfly; it contains the feature-extractors that let you look at a butterfly, parse it as a distinct object standing out against the background, remember that object apart from the background, and reconstruct a picture of that object from memory.  We are not born with experience of butterflies; we are born with the visual cortex that gives us the capability to experience and remember butterflies.  The visual cortex is not visual knowledge; it is the space in which visual knowledge exists.

The second, deeper problem follows from the first.  All of an expert system's microtheories have the same underlying data structures (in this case, propositional logic), acted on by the same underlying procedures (in this case, a few rules of °Bayesian reasoning).  Why separate something into distinct modules if they all use the same data structures and the same functions?  Shouldn't a real program have more than one real module?

I'm not suggesting that data formats and modules be proliferated because this will magically make the program work better.  Any competent programmer knows not to use two data formats where one will do.  But if the data and processes aren't complex enough to seize the programmer by the throat and force a modular architecture, then the program is too simple to give rise to real intelligence.

Besides, a single-module architecture certainly isn't the way the brain does it.  Maybe there's some ingenious way to represent auditory and visual information using a single underlying data structure.  If we can get away with it, great.  But if no act of genius is required to solve the very deep problem of getting domain-specific representations to interact usefully, if the problem is "solved" because all the content of thought takes the form of propositional logic, if all the behaviors can fit comfortably into a single programmatic module - then the program doesn't have enough complexity to be a decent video game, much less an AI.  (22).

We shouldn't be too harsh on the classical-AI researchers.  Building an AI that operates on "pure logic" - no sensory modalities, no equivalent to the visual cortex - was worth trying.  As Ed Regis would say, it had a certain hubristic appeal.  Why does human thought use the visual cortex?  Because it's there!  After all, if you've already evolved a visual cortex, further adaptations will naturally take advantage of it.  It doesn't mean that an engineer, working ab initio, must be bound by the human way of doing things.

But it didn't work.  The recipe for intelligence presented by GISAI assumes an AI that possesses equivalents to the visual cortex, auditory cortex, and so on.  Not necessarily these particular cortices; after all, Helen Keller (who was blind and deaf, and spoke in hand signs) learned to think intelligently.  But even Helen Keller had proprioception, and thus a parietal lobe for spatial orientations; she had a sense of touch, which she could use to "listen" to sign language; she could use the sensory modalities she had to perceive signed symbols, and form symbols internally, and string those symbols together to form sentences, and think.  (23)  Some equivalent of some type of "cortex" is necessary to the GISAI design.

"Cortex" is a specifically neurological term referring to the surface area of the brain, and therefore I will use the term "sensory modality", or "modality", instead of cortex.

DEFN: Modality:  Modalities in an AI are analogous to human cortices - visual cortex, auditory cortex, et cetera - enabling the AI to visualize processes in the target domain.  Modalities capture, not high-level knowledge, but low-level behaviors.  A modality has data structures suited to representing the target domain, and °codelets or processing stages which extract higher-level features from raw data.

Why does an AI need a visual modality?  Because the human visual cortex and associated neuroanatomy - our visual modality - is what makes our thoughts of 2D and 3D objects real.  Drew McDermott, in Artificial Intelligence Meets Natural Stupidity, pointed out that, just because a LISP token is labeled with the character string "hamburger", it does not mean that the program understands hamburgers.  The program has not even noticed hamburgers.  If the symbol were called G0025 instead of hamburger, nobody would ever be able to figure out that the token was supposed to represent a hamburger.

When two objects collide, we don't just have a bit of propositional logic that says collide(car, truck); we imagine two moving objects.  We model 2D pixels and 3D features and visualize the objects crashing together.  The edges touch, not as touch(edge-of(car), edge-of(truck)), but as two curves meeting and deforming at all the individual points along the edge.  You could successfully look at a human brain and deduce that the neurons in question were modelling edges and colliding objects; this is, in fact, what visual neuroanatomists do.  But if you did the same to a classical AI, if you stripped away the handy English variable names from the propositional logic, you'd be left with G0025(Q0423, U0111) and H0096(D0103(Q0423), D0103(U0111)).  No amount of reasoning could bind those cryptic numbers to real-world cars or trucks.

Furthermore, our visual cortex is useful for more than vision. Philosophy in the Flesh (George Lakoff and Mark Johnson) talks about the Source-Path-Goal pattern (24) - a trajector that moves, a starting point, a goal, a route; the position of the trajector at a given time, the direction at that time, the actual final destination... Philosophy in the Flesh also talks about "internal spatial 'logic' and built-in inferences":  If you traverse a route, you have been at all locations along the route; if you travel from A to B and B to C, you have traveled from A to C; if X and Y are traveling along a direct route from A to B and X passes Y, then X is further from A and closer to B than Y is.

These are all behaviors of spatial reality.  Classical AI would attempt to capture descriptions of this behavior; i.e. "if travel(X, A, B) and travel(X, B, C) then travel(X, A, C)".  The problem is that the low-level elements (pixels, trajectors, velocities) making up the model can yield a nearly infinite number of high-level behaviors, all of which - under the classical-AI method - must be described independently.  If A is-contained-in B, it can't get out - unless B has-a-hole.  Unless A is-larger-than the hole.  Unless A can-turn-on-its-side or the hole is-flexible.  Trying to describe all the possible behaviors exhibited by the high-level characteristics, without directly simulating the underlying reality, is like trying to design a CPU that multiplies two 32-bit numbers using a doubly-indexed lookup table with 2^64 (around eighteen billion billion) entries.

Real CPUs take advantage of the fact that 32-bit numbers are made of bits.  This enables transistors to multiply using the wedding-cake method (or whatever it is modern CPU designs use).  A 32-bit number is not a monolithic object.  The numerical interpretation of 32 binary digits is not intrinsic, but rather a high-level characteristic, an observation, an abstraction.  The individual bits interact, and yield a 32-bit (or 64-bit) result which can then be interpreted as the resulting number.  The computer can multiply 9825 by 767 and get 7535775, not because someone told it that 9825 times 767 is 7535775, but because someone told it about how to multiply the individual bits.

A visual modality grants the power to observe, predict, decide, and manipulate objects moving in trajectories, not because the modality captures knowledge of high-level characteristics, but because the modality has elements which behave in the same way as the external reality.  An AI with a visual modality has the potential to understand the concept of "closer", not because it has vast stores of propositional logic about closer(A, B), but because the model of A and B is composed of actual pixels which are actually getting closer.  (25).

Source-Path-Goal is not just a visual pattern.  It is a metaphor that applies to almost any effort.  Force and resistance aren't just people pushing carts, they're companies pushing products.  Source-Path-Goal applies not just to walking to Manhattan, but a programmer struggling to write an application that conforms to the requirements spec.  It applies to the progress of these very words, moving across the screen as I type them, decreasing the distance to the goal of a publishable Web page.  Furthermore, the visual metaphor is in many cases a useful metaphor, one which binds predictively.  (26).  A metaphor is useful when it involves, not just a similarity of high-level characteristics, but a similarity of low-level elements, or a single underlying cause.  (See previous footnote.)  The visual metaphor that maps the behavior of a programming task to the Source-Path-Goal pattern (a visual object moving along a visual line) is useful if some measure of "task completed" can be mapped to the quantitative position of the trajector, and the perceived velocity used to (correctly!) predict the amount of time remaining on the task.

Of course, one must realize that having a visual modality is Necessary, But Not Sufficient, to pulling that kind of stunt.  In such cases, noticing the analogy is ninety percent of the creativity.  The atomic case of such noticing would consist of generating models at random, either by generating random data sets or by randomly mixing previously acquired models, until some covariance, some similarity, is noticed between the model and the reality.  And then the AI says "Eureka!"

Of course, except for very simple metaphors, the search space is too large for blind constructs to ever match up with reality.  It is more often necessary to deliberately construct a model - in this case, a visual model - whose behaviors correspond to reality.  Discussion of such higher-level reasoning doesn't belong in the section on "sensory modalities", but being able to "deliberately construct" anything requires a way to manipulate the visual model.  In addition to the hardware/code for taking the external action of "draw a square on the sheet of paper", a mind requires the hardware/code to take the internal action of "imagine a square".  The consequence, in terms of how sensory modalities are programmed, is that feature extraction needs to be reversible.  Not all of the features all of the time, of course, but for the cognitive act of visualization to be possible, there must be a mechanism whereby the perception that detects the "line" feature has an inverse function that constructs a line, or transforms something else into a line.

Feature reconstruction is much more difficult to program than feature extraction.  More computationally intensive, too.  It's the difference between multiplying the low-level elements of "7" and "17", and reconstructing two low-level elements which could have yielded the high-level feature of "119".  This may be one of the reasons why thalamocortical sensory pathways are always reciprocated by corticothalamic projections of equal or greater size; for example, a cat has 10^6 neural fibers leading from the lateral geniculate nucleus to the visual cortex, but 10^7 fibers going in the reverse direction.  (27).

Even a complete sensory modality, capable of perception and visualization, is useless without the rest of the AI.  "Necessary, But Not Sufficient," the phrase goes.  A modality provides some of the raw material that concepts are made of - the space in which visualizations exist, but nothing more.  But, granting that the rest of the AI has been done properly, a visual modality will create the potential to understand the concept of "closer"; to use the concept of "closer", and heuristics derived from examining instances of the concept "closer", as a useful visual metaphor for other tasks; and to use deliberately constructed models, existing in the visual modality, to ground thinking about generic processes and interactions.  (In other words, when considering a "fork" in chess or an "if" statement in code, it can be visualized as an object with a Y-shaped trajectory.)

Is a complete visual modality - pixels, edge detectors, surface-texture decoders, and all - really necessary to engage in spatial reasoning?  Would a world of Newtonian billiard balls, with velocities and collision-detection, do as well?  It would apparently suffice to represent concepts such as "fork", "if statement", "source-path-goal", "closer", and to create metaphors for most generic systems composed of discrete objects.  The billiard-ball world has significantly less representative power; it's harder to understand a "curved trajectory" in spacetime if you can't visualize a curve in space.  (28).  But, considering the sheer programmatic difficulty of coding a visual modality, are metaphors with billiard balls composed of pixels that superior to metaphors with billiard balls implemented directly as low-level elements?

Well, yes.  In a visual modality, you can switch from round billiard balls to square billiard balls, visualize them deforming as they touch, and otherwise "think outside the box".  The potential for thinking outside the box, in this case, exists because the system being modeled has elements that are represented by high-level visual objects; these high-level visual objects in turn are composed of mid-level visual features which are composed of low-level visual elements.  This provides wiggle room for creativity.

Consider the famous puzzle with nine dots arranged in a square, where you're supposed to draw four straight lines, without lifting pen from paper, to connect the dots.  (29).  To solve the puzzle one must "think outside the box" - that is, draw lines which extend beyond the confines of the square.  A conventional computer program written to solve this problem would probably contain the "box" as an assumption built into the code, which is why computers have a reputation for lack of creativity.  (30).  A billiard-ball metaphor, even assuming that it could represent lines, might run into the same problem.

I suspect that many solvers of the nine-dot problem reach their insight because a particular configuration of tried-out lines suggests an incomplete triangle whose corners lie outside the box.  "Seeing" an "incomplete triangle" is an optical illusion, which is to say that it's the result of high-level features being triggered and suggesting mid-level features - in this case, some extra lines that turn out to be the solution to the problem.  Sure, you can make up ways that this could happen in a billiards modality, but then the billiards modality starts looking like a visual cortex.  The point is that, for our particular human style of creativity, it is Necessary (But Not Sufficient) to have a modality with rich "extraneous" perceptions, and where high-level objects in the metaphor can be made to do unconventional things by mentally manipulating the low-level elements.  (Even so, it would make development sense to start out with a billiards modality and work up to vision gradually.)

There are two final reasons for giving a seed AI sensory modalities:  First, the possession of a codic modality may improve the AI's understanding of source code, at least until the AI is smart enough to make its own decisions about the balance between slow-conscious and fast-autonomic thought.  Second, as will be discussed later, thoughts don't start out as abstract; they reach what we would consider the "abstract" level by climbing a layer cake of ideas.  That layer cake starts with the non-abstract, autonomic intuitions and perceptions of the world described by modalities.  The concrete world provided by modalities is what enables the AI to learn its way up to tackling abstract problems.

NOTE: One of the greatest advantages of seed AI - second only to recursive self-improvement - is going beyond the human sensory modalities.  It's possible to create a sensory modality for source code.  The converse is also true:  Various processes that are autonomic in humans - memory storage, symbol formation - can become sensory modalities subject to deliberate manipulation.

In programmatic terms, any program module with a coherent set of data structures and an API, which could benefit from higher-level thinking, is a candidate for transformation into a modality with world-model-capable representations, feature extraction, reversible features to allow mental actions, and the other design characteristics required to support concept formation.

2.3: Concepts

2.3.1: Modality-level, concept-level, thought-level

Modalities in the human brain are mostly preprogrammed, as opposed to learned.  (Human modalities require external stimuli to grow into their preprogrammed organization, but this is not the same as learning.)  Individual neural signals can have meanings that are visible and understandable to an eavesdropper.  Programmers may legitimately take the risk of creating modalities through deliberate programming, with low-level elements that correspond to data structures, and human-written procedures for feature extraction.

Within °GISAI, the term concept is used to refer to the kind of mental stuff that exists as a pattern in the modality.  A learned sequence of instructions that reconstructs a generic, abstracted "light bulb" in the visual modality is a concept.  Symbols, categories, and some memories are concepts.  (Despite common usage, "concept" might technically refer to non-declarative mental stuff such as a human cognitive reflex or a human motor skill.  However, in a seed AI, where everything is open to introspection, it makes sense to call the equivalents of human reflexes or skills "concepts".)  Concepts are patterns, learned or preprogrammed, that exist in long-term storage and can be retrieved.

A structure of concepts creates a thought.  The archetypal example, in humans, is words coming together to form sentences.  Thoughts are visualized; they operate within the RAM of the mind, the "workspace" represented by available content capacity in the sensory modalities, commonly called "short-term memory" or "working memory".  (The capacity of working memory in AIs is not determined by available RAM, but by available CPU capacity to perform feature extraction on the contents of memory.  If you have the data structures without the feature extraction, the AI won't notice the information.)  Thoughts manipulate the world-model.

In humans, at least, it's hard to draw clean boundaries between thoughts and concepts.  (31).  The experience of hearing the word for a single concept, such as "triangle", is not necessarily a mere concept; it may be more valid to view it as a thought composed of the single concept "triangle".  And, although some concepts are formed by categorizing directly from sense perception, more abstract concepts such as "three" probably occur first as deliberate thoughts.  We'll be discussing both types in this section.

2.3.2: Abstraction is information-loss; abstraction is not information-loss

In chemistry, abstract means remove; to "abstract" an atom from a molecule means to take it away.  Use of the term "abstract" to describe the process of forming concepts implies two assumptions:  First, to create a concept is to generalize; second, to generalize is to lose information.  It implies that, to form the concept of "red", it is necessary to ignore other high-level features such as shape and size, and focus only on color.

This is the classical-AI view of abstraction, and we should therefore be suspicious of it.  On the other hand, our mechanisms for abstraction can learn the concept for "red".  In a being with a visual modality, this concept would consist of a piece of mindstuff that had learned to distinguish between red objects and non-red objects.  Since redness is detected directly as a low-level feature, it shouldn't be very hard to train a piece of mindstuff to thus distinguish - whether the mindstuff is made of trainable neurons, evolving code, or whatever.  A neural net needs to learn to fire when the "red" feature is present, and not otherwise; a piece of code only needs to evolve to test for the presence of the redness feature.  At most, "red" might also require testing for solid-color or same-hue groupings.  Given a visual modality, the concept of "red" lies very close to the surface.

Of course, to have a real concept for "red", it's not enough to distinguish between red and non-red.  The concept has to be applicable; you have to be able to apply it to visualizations, as in "red dog".  You also need a default exemplar (32) for "red"; and an extreme exemplar for "red"; and memories of experiences that are stereotypically red, such as stoplights and blood.  (For all we know, leaving out any one of these would be enough to totally hose the flow of cognition.)  Again, these features lie close to the surface of a visual modality.  "Red" would be one of the easiest features to make reversible, with little additional computational cost involved; just set the hue of all colors to a red value.  (Although hopefully in such a way as to preserve all detected edges, contrasts, and so on.  Making everything exactly the same color would destroy non-color features.)  The default exemplar for red can be a red blob, or a red light; the extreme exemplar for red may be the same as the default exemplar, or it may be a more intensely red blob.  And the stereotypically red objects, such as stoplights and blood, are the objects in which the redness is important, and much remarked upon.

(33).

For the moment, however, let's concentrate on the problem of forming categories.  The conventional wisdom states that categorization consists of generalization, and that generalization consists of focusing on particular features at the expense of others.

We'll use the microdomain of letter-strings as an example.  To generalize from the instances {"aaa", "bbb", "ccc"} to form the category "strings-of-three-equal-letters", the information about which letter must be abstracted, or lost, from the model.  Actually, this misstates the problem.  If you lose that information on a letter-by-letter basis, then "aaa" and "aab" both look like "***".  What's needed is for the letter-string modality to first extract the features of "group-of-equal-letters", "number=3", and "letter=b", after which the concept can lose the last feature or focus on the first two.  If the second feature, "number", is also lost, then the result is an even more general concept, "strings-of-equal-letters".  Of course, this concept is precisely identical to the modality's built-in feature-detector for "group-of-equal-letters", which again points up that only very simple conceptual categories, lying very close to the surface of the modality's preprogrammed assumptions about which features are important, can be implemented by direct information-loss.

To examine a more complex concept, we'll look at the example of "three".

2.3.3: The concept of "three"

To a twenty-first-century human, trained in arithmetic and mathematics, the concept of "three" has enormous richness.  It must therefore be emphasized that we are dealing solely with the concept of "three", and that a mind can understand "three" without understanding "two" or "four" or "number" or "addition" or "multiplication".  A mind may have the concept "three" and the concept "two" without noticing any similarity between them, much less having the aha! that these concepts should go together under the heading "number".  If a mind somehow manages to pick up the categories of groups-of-three-dogs and groups-of-three-cats, it doesn't follow that the mind will generalize to the category of "three".

To think about infant-level or child-level AIs, or for that matter to teach human children, it's necessary to slow down and forget about what seems "natural".  It's necessary to make a conscious separation between ideas - ideas that, to humans, seem so close together that it takes a deliberate effort to see the distance.

Just because the AI exists on a machine performing billions of arithmetical operations per second doesn't mean that the AI itself must understand arithmetic or "three".  (John Searle, take note!)  Even if the AI has a codic modality which grants it direct access to numerical operations, it doesn't necessarily understand "three".  If every modality were programmed with feature-extractors that counted up the number of objects in every grouping, and output the result as (say) the tag "number: three", the AI might still fail to really understand "three", since such an AI would be unable to count objects that weren't represented directly in some modality.  An AI that learns the concept of "three" is more likely to notice not just three apples but that °ve (the AI) is currently thinking three thoughts.  A preprogrammed concept only notices what the programmer was thinking about when he or she wrote the program.

What is "three", then?  How would the concept of "three" be learned by an AI whose modalities made no direct reference to numbers - whose modalities, in fact, were designed by a programmer who wasn't thinking about numbers at the time?  How can such a simple concept be decomposed into something even simpler?

There's an AI called "°Copycat", written by Melanie Mitchell and conceived by Douglas R. Hofstadter, that tries to solve analogy problems in the microdomain of letter-strings.  If you tell Copycat:  "'abc' goes to 'abd'; what does 'bcd' go to?", it will answer "'bce'".  It can handle much harder problems, too.  (See °Copycat in the glossary.)  Copycat is a really fascinating AI, and you can read about it in Metamagical Themas, or read the source code (it's a good read, and available as plain text online - no decompression required).  If you do look at the source code, or even just browse the list of filenames, you'll see the names of some very fundamental cognitive entities.  There are "bonds", "groups", and "correspondences".  There are "descriptors" (and "distinguishing descriptors") and "mappings", and all sorts of interesting things.

Without going too far into the details of Copycat, I believe that some of the mental objects in Copycat are primitive enough to lie very close to the foundations of cognition.  Copycat measures numbers directly (although it can only count up to five), but that's not the feature we're interested in.  Copycat was designed to understand relations and invent analogies.  It can notice when two letters occupy "the same position" in a letter-string, and can also notice when two letters occupy "the same role" in a higher-order mental construct.  It can notice that "c" in "abc" and "d" in "abd" and "d" in "bcd" all occupy the same position.  It can understand the concept of "the same role", if faced by an analogy problem which forces it to do so.  For example:  If "abc" goes to "abd", what does "pqrs" go to?  Copycat sees that "c" and "s" occupy the same role, even though they no longer occupy the same numerical position in the string, and so replies "pqrt".

Correspondences and roles and mappings are probably autonomically-detected features on the modality-level (as well as being very advanced concepts in cognitive science).  Intuitive, directly perceived correspondences allow two images in the same modality to be compared, and that is a basic part of what makes a modality go.

These intuitions obey certain underlying cognitive pressures (also modeled by the Copycat project):  If two high-level structures are equal, then the low-level structures should be mapped to each other.  Symmetry, which - very loosely defined - is the idea that each of these low-level mappings should be the same.  If one is reflected, they should all be reflected, and so on.  Completeness:  You shouldn't map five elements to each other but leave the sixth elements dangling.

Copycat shows an example of how to implement this class of cognitive intuitions using conflict-detectors, equality-detectors, and a feature called a "computational temperature".  Roughly speaking, conflicts raise the temperature and good structures lower the temperature.  The higher the temperature, the more easily cognitive perceptions break - the more easily groups and bonds and mappings dissolve.  Lower temperatures indicate better answers, and thus answers are more persistent - perceived pieces of the answer in the cognitive workspace are harder to break.  Copycat's intuitions may not have the same flexibility or insight as a human consciously trying to solve a "symmetry problem" or a "completeness problem", but they do arguably match a human's unconscious intuitions about analogy problems.  Each low-level built-in cognitive ability has its analogue as a high-level thought-based skill, and it is dangerous to confuse the standards to which the two are held.

We now return to the concept of "three".  We'll suppose for the moment that we're operating in a Newtonian billiard-ball modality, and that we want the AI to learn to recognize three billiard balls.

The first concept learned for "three" might look like this:

The mental image on the left is an "exemplar" (or "prototype"), attached to the three concept and stored in memory.  The mental image on the right is the target, containing the objects actually being counted.  The concept of "three" is satisfied when correspondences can be drawn between each object in the three-exemplar and each object in the target image.  If the target image contains two objects, a dangling object will be detected in the three-exemplar image, and the concept will not be satisfied.  If the target image contains four objects, then a dangling object will be detected in the target image.  (34).

This isn't a full answer to the "problem of three", of course.  A full answer would also consider the question of how to computationally implement a "unique correspondence" in a non-fragile way; how to distinguish each object from the background; how to apply the three-concept to a mental image formerly containing two or four objects to yield a new mental image containing three objects; how to retrieve the exemplar from memory; how to extend the intuition of "unique correspondence" across modalities.  And the type of mindstuff needed to implement these instructions in a non-fragile way; and how the exemplar and concept were created or learned in the first place.

In fact, the problem of three is so complicated that it would probably be first solved by conscious thought, and compiled into a concept afterwards.  This adds the problem of figuring out how the thoughts got started; what types of task would force a mind to notice "three" and evolve a definition like that above; and how the skill gets compiled into a pattern.  Also, an understanding of three that generalizes from the concept "three billiard balls" to the concept "three groups of three billiard balls" means asking what kind of problem would force the generalization.  It means asking how the generalization would take place inside the thought-based skill or mindstuff-based concept; how the need to generalize would translate into a cognitive pressure, and how that pressure would apply to a piece of the mindstuff-code, and how that piece would correctly shift under pressure.  And then there are questions about moving towards the adult-human understanding of "three", such as noticing that it doesn't matter which particular billiard ball A corresponds to which billiard ball B.

However, the diagram above does constitute a major leap forward in solving the problem.  It is a functional decomposition of three, one that invokes more basic forces such as unique correspondence and exemplar retrieval.  It is a concept that could be learned even by an AI whose programmers had never heard of numbers, or whose programmers weren't thinking about numbers at the time.  It is a concept that can mutate in useful ways.  By relaxing the requirement of no dangling objects in the exemplar, we get "less than or equal to three".  By relaxing the requirement of no dangling objects in the target image, we get "greater than or equal to three".  By requiring a dangling object in the target image, we get "more than three".  By comparing two images, instead of a exemplar and an image, we get "same number as" (35), and from there "less than" or "less than or equal to".

In fact, examining some of these mutations suggests a real-world path to threeness.  The general rule is that concepts don't get invented until they're useful.  Many physical tasks in our world require equal numbers of something; four pegs for four holes, and so on.  The task of perceiving a particular number of "holes" and selecting, in advance, the correct number of pegs, might force the AI to develop the concept of corresponding sets, or sets that contain the same number of objects.  The spatial fact that two pegs can't go in the same hole, and that one peg can't go in two holes, would be a force acting to create the perception of unique (one-to-one) correspondences.  "Corresponding-sets" would probably be the first concept formed.  After that, if it were useful to do so, would come a tendency to categorize sets into classes of corresponding sets, when it was useful to do so; after that would come the selection of a three-exemplar and the concept of three.

The decomposition of three in the above graphic is not the most efficient concept for three.  It is simply the most easily evolved.  After the formation of the exemplar-and-comparision concept for three would come a more efficient procedure:  Counting.

To evolve the counting concept requires that the counting skill be developed, which occurs on the thought-level, which thought in turn requires a more sophisticated concept-level depiction of three.  It requires that one and two have also been developed, and that one and two and three have been generalized into number.  Once this occurs, and the AI has been playing around with numbers for a while, it may notice that any group of three objects contains a group of two objects.  It may manage to form the concept of "one-more-than", an insight that would probably be triggered by watching the number of a group change as additional objects are added.  It might even notice that physical processes which add one object at a time always result in the same sequence of numerical descriptions:  "One, two, three, four..."

If multiple experiences of such physical processes can be generalized, and an exemplar experience of the process selected and applied, the result might be a counting procedure like that taught to human children: Tag an object as counted and say the word 'one'; tag another object as counted and say 'two'; tag another object as counted and say the word that, in the learned auditory chanting sequence, comes after 'two'; and so on.  Do not re-count any object that has already been tagged as "counted".  The last word said aloud is the number of the group.  This method is more efficient than checking unique correspondences, and the method also reflects a deeper understanding of numbers.

Finally, once "three" has been used long enough, it's likely that a human brain evolves some type of neural substrate for seeing threeness directly.  That is, some piece of the human visual modality - probably the object-recognition system in the temporal lobe, but that's just a wild guess - learns to respond to groups of three objects.  (Larger numbers like "five" or "six" are harder to recognize directly - that is, without counting - unless the objects are arranged in stereotypical five-patterns and six-patterns, like those on the sides of dice.)  The analogue for an AI might be a piece of code (or assembly language, or a neural net - you know, mindstuff) that counts items directly.

However, even if the AI eventually creates a highly-optimized counting method, implemented directly, the previous definitions of the concept will still exist.  When new situations are encountered, new situations that force the extension of the concept, the mind can switch from the optimized method to the methods that reflect underlying causes and underlying substrate.  If necessary, the problem can rise all the way to the level of conscious perception, so that the deliberate, thought-level methods - the thoughts from which the concepts first arose - are used.  The experiences that underlie the original definition, the experience of noticing the definition, the experience of using the definition - all can be reviewed.  This is why a concept is so much richer, so much more powerful, if it's learned instead of preprogrammed.  It's why learned, rich concepts are so much more flexible, so much likelier to mutate and evolve and spin off interesting specializations and generalizations and variations.  It's why learned concepts are more useful when a mind encounters special cases and has to resort to high-level reasoning.  It's why high-level cognitive objects are vastly more powerful, more real, than the flat, naked "predicate calculus" of classical AI.

Thus the idea of "information-loss" or "focus" is cast in a different light.  Sure, calling something a three-group, or placing it into the three-category, can be said to "lose" a lot of information - in information-theoretical terms, you've moved from specifying the distinct and individual object to specifying a member of the class of things that can be described by "three".  In classical-AI terms, you've decided to focus on the feature called "number" and not any of the other features of the object.  But to label a rich, complex, multi-step act of perception "information loss" borders on perversion.  Seeing the "threeness" of a group doesn't destroy information, it adds information.  One perceives everything that was previously known about the object, and its threeness as well; nor could that threeness be "focused" on, until the methods for perceiving threeness were learned.

2.3.4: Concept combination and application

"When you hear the phrase "triangular light bulb", you visualize a triangular light bulb...  How do these two symbols combine?  You know that light bulbs are fragile; you have a built-in comprehension of real-world physics - sometimes called "naive" physics - that enables you to understand fragility.  You understand that the bulb and the filament are made of different materials; you can somehow attribute non-visual properties to pieces of the three-dimensional shape hanging in your visual cortex.  If you try to design a triangular light bulb, you'll design a flourescent triangular loop, or a pyramid-shaped incandescent bulb; in either case, unlike the default visualization of "triangle", the result will not have sharp edges.  You know that sharp edges, on glass, will cut the hand that holds it."
        -- 1.2: Thinking About AI
How do the concepts of "triangular" and "light-bulb" combine?  My current hypothesis involves what might be called "°reductionist energy minimization" or "°holistic network relaxation", a conflict-resolution method that takes cues from both the "potential energy surface" of chemistry and the "computational temperature" of Copycat.

Neural networks, when perturbed, are known to seek out what might be called "minimal-energy states".  A network-relaxation model of concept combination could be computationally realistic - an operation that neurons can accomplish in the 200 operations-per-second timescale.  My current hypothesis for the basic neural operation in concept-combination is the resonance.  A neural resonance circuit - perhaps not a physical, synaptic circuit, but a virtual message-passing circuit, established by one of the higher-level neural communication methods (binding by neural synchrony, maybe) - can either resonate positively, reinforcing that part of the concept-combination, or resonate negatively, generating a conflict.  My guess at the network-relaxation method resembles the "potential energy surface" of chemistry in that multiple, superposed alternatives are tried out simultaneously, so that the minima-seeking resembles a flowing liquid rather than a rolling ball.

The high-level, salient facets of the concepts being combined are combined first.  These high-level features then visualize the mid-level features; if no conflict is detected, the mid-level features visualize the low-level features.  If a conflict is detected at any level, the conflict propagates back up to the conflicting high-level or mid-level features causing the problem.  Who wins the conflict?  The more salient, more important, or more useful feature - remember, we're talking about combining two concepts, each with its own set of features along various dimensions - is selected as dominant, and the network relaxation algorithm proceeds.  When one concept modifies another, the "more salient" feature is the one specified by the concept doing the modifying.  (Note also that, in casual reading, not all the facets of a concept may be important, just as you don't fully visualize every word in a sentence.  Only the facets that resonate with the subject of discussion, with the paragraph, will be visualized.)

In the case of "triangular light bulbs", "triangular" is an adjective.  The concept for "triangle" or "triangular" is modifying the concept of "light bulb", rather than vice versa.  The default exemplar for "light bulb" - that is, an image of the generic light bulb - is loaded into the mental workspace, including the visual facet of the exemplar being loaded into the visual cortex.  Next, the concept for "triangular" is applied to this mental image.

The concept of "triangular", as it refers to physical objects, has a single facet:  It alters the physical shape of the target image.  Note that I say "physical shape", not "visual shape".  The default exemplar for "light bulb" is a mental image - not a mental picture, but a mental image; in GISAI, an "image" means a representation in any modality or modalities, not just the visual cortex.  The "light bulb" exemplar is an image of a three-dimensional bulb-shaped object, made of glass, having a metal plug at the bottom, whose purpose is to emit light.  It is this multimodal mental image that "triangular" modifies, not just the visual component of the image.  In particular, the "shape" facet of the light-bulb concept, the facet being modified, is a high-level feature describing the shape of the three-dimensional physical object, not the shape of the visual image.  Thus, modifying the light-bulb shape will modify the mental image of the physical shape, rather than manipulating the 2-D visual shape in the visual cortex.

The "triangular" concept, when applied along the dimension of "shape", manipulates the mental image of the light bulb, changing the 3D model to be triangle-shaped.  However, since the image of a flat light bulb fails to resonate, "triangle" automatically slips to "pyramid".

(I'm not sure whether this conflict is detected at the mid-level feature of "flat light bulb", or whether a flat light bulb actually begins to visualize before the conflict is detected.  The slippage happens too fast for me to be sure.  I suspect that "triangular" has slipped to "pyramidal" before, when applied to three-dimensional mental images; for neural entities, anything that happens once is likely to happen again.  Neurons learn, and neural thinking wears channels in the neurons.  It could be that the non-flatness of light bulbs is salient because of their bulbous shape, and that this resonance with non-flatness causes "triangular" to slip to "pyramidal" before the concept is even applied.)

Pyramids are sharp.  I know, from introspection, that the "sharp pyramidal light-bulb" got all the way down to the visual level before the conflict was noticed.  (The conflict rose to the level of conscious perception, but was resolved more or less intuitively; I didn't have to "stop and think".  So this is probably still a valid example of concept-level processes.)  The particular conflict:  Sharp glass cuts the person who holds it.  We've all had visual experience of sharp glass, and the associated need for visual recognition and avoidance; t