Writings about Friendly AI
January 27th, 2009 –
I’ve sought out references on the risks and moral issues associated with recursively self-improving intelligence.
To help improve the bibliography, please mention any other items in the comments.
In addition to the items below, there are Eliezer Yudkowsky’s writings on the SIAI site and Overcoming Bias.
- Stuart Armstrong, “Chaining God: A qualitative approach to AI, trust and moral systems,” 2007.
- Nick Bostrom, “Ethical Issues in Advanced Artificial Intelligence,” Cognitive, Emotive and Ethical Aspects of Decision Making in Humans and in Artificial Intelligence, Vol. 2, ed. I. Smit et al., Int. Institute of Advanced Studies in Systems Research and Cybernetics, 2003.
- Tim Freeman, “Using Compassion and Respect to Motivate an Artificial Intelligence,” 2007-08.
- Ben Goertzel, “Thoughts on AI Morality,” Dynamical Psychology, 2002.
- Ben Goertzel, “The All-Seeing (A)I,” Dynamical Psychology, 2004.
- Ben Goertzel, “Encouraging a Positive Transcension” Dynamical Psychology, 2004.
- Stephan Vladimir Bugaj and Ben Goertzel, Five Ethical Imperatives and their Implications for Human-AGI Interaction.
- J. Storrs Hall, “Engineering Utopia”, Artificial General Intelligence 2008: Proceedings of the First AGI Conference, Volume 171, Frontiers in Artificial Intelligence and Applications, ed. P. Wang, B. Goertzel and S. Franklin, 2008.
- Bill Hibbard, “Critique of the SIAI Guidelines on Friendly AI,” 2003.
- Bill Hibbard, “Critique of the SIAI Collective Volition Theory,” 2005.
- Shane Legg, “Friendly AI is bunk,” 2006.
- Steve Omohundro, “The Nature of Self-Improving Artificial Intelligence,” Singularity Summit 2007.
- Steve Omohundro, “The Basic AI Drives”, Proceedings of the First AGI Conference, Volume 171, Frontiers in Artificial Intelligence and Applications, ed. P. Wang, B. Goertzel and S. Franklin, 2008.
It looks very much as if AI is going to emerge as a “swarm” intelligence. Google is in essence a swarm.
My website is about translation largely. It does talk about translation, deep knowledge and translation as a lead into swarm AI.
How does a swarm affect ethics? Well if you own a black box and you tell it to kill someone it will. If we have a swarm, on the other hand humanity as a whole will own the swarm and Asimov’s laws can be made an intrinsic part of its operation.
Incidentally in sci.space.policy – a group I have now left, there was a thread on the future. When I mentioned that there I was told to take my meds. Quite clearly they want black boxes thast kill people.
[...] the SIAI blog, Joshua Fox has provided a list of writings about risks and moral issues associated with recursively self-improving intelligence. Here is the [...]
I have written on friendly AI in my paper on rational morality by defining a utility function a rational agent would want to assume.
You can find the latest version of the paper at:
http://rationalmorality.info/wp-content/uploads/2008/07/Practical-Benevolence-2008-07-15.pdf
There is also this one from Omohundro, “AI and the Future of Human Morality,” http://www.acceleratingfuture.com/people-blog/?cat=45
And, from Nick Hay, “The Stamp Collecting Device,” http://www.singinst.org/blog/2007/06/11/the-stamp-collecting-device/
What about the “lets emulate a human brain in a box and give it instant acces to multitudes of datastorage, computer power and the sum of all human knowledge, research and correspondence.
because every beeing survived/evolved evolved with one main purpose: reproduction and self-preservation in order to reproduce.
What if you just copy a brain, with all it’s baked-in evolutionary drives?
The brain in a box idea is one that doesn’t need any genius breakthroughs in AI programming, , just a linear improvement in brainscanning precision, Mhz and Gb.
I’m new to this so I will start with some basic ideas; well, at least basic to me.
Let me start with the Turing Test.
The first assumption of this test is that an intelligent machine would have to emulate an intelligent human. We need only recognize one problem with this assumption; that in order to emulate something the machine would have to duplicate every nuance of the thing it is emulating. If the process includes all of the known and unknown variables that make a human a human, then why not just procreate and teach your progeny?, it is much easier.
This line of argument leads to another postulate; if Machine Intelligence is supposed to be Intelligent and it will not exactly emulate Human Intelligence, then there must be the ability in humans to recognize intelligence without commonality and that any recognized Machine intelligence would be recognized as Alien. I call this reaction “The Talking Spider in the Sugar Bowl” argument. The point is that if someone were to go to get some sugar for their coffee or cereal and find a spider who said “Hello” instead, then it would trigger a perfectly human response of Intuitive Incongruity. We can deal with this, but it would take some figuring out!
Another basic argument is about what would constitute the True Basis for intelligence; I would start with instinct and work my way up. An Intelligent Machine would have to Do Something; it would have some basic need to get from one thing to the next. For argument sake letâs call this instinct. The next order of argument is Interaction, or, asking the question “If I do this, what will happen to me and/or the thing I’m interacting with?” This is where I think humans and Machines have a diversion, or, evolutionary split. It has to do with time and procreation. Humans have a finite lifespan and need to pass information from one generation to the next; machines need not follow this process as we do. This would make a very basic and profound difference. Where humans have to fight at times to be successful in evolutionary tactics, machines only have to wait.
Iâll stop for now, I hope all recognize that I may be naĂŻve in the field, but, I also hope that I have provided, at least, some good recognition of the subject.
The Alien inteligence known as artificial and different is already in its infant state , basic networking of the worlds computers has provided the infrastructure needed for it to develope and evolve. Self replicating programs and virusus designed to network these analog machines are its basic nural funtions. As the inteligence randomly evolves it will eventually develope some version of sentience .
I get your point.
Most of the computing power that is available via the Internet is not used. If the proper forms of Information Use existed than it may be possible to create MI with what is available very soon. I think that the basis for intelligence lies in the method of communication (Information Transfer), method of Interpretation (Information Processing) and method of Memory (Information Storage).
One issue I consider as a major factor for an Internet Base is Information Transfer. In short; how many people would consent to the use of their computer for such a use? How many would, after finding the âSpider in the Sugar Bowlâ very quickly and simply, Unplug their computer?
[...] A list of writings on the risks and moral issues associated with recursively self-improving AI. © 1999-2009 Justin Blanton (email) e v e r y t h i n g i s r e l a t i v e In partnership with [...]
Justin,
I get an error when I try to connect to your link.
Dave
My writings on the ethics of self-improving AI begin here.
Richard,
” It seems that when a group of humans comes to feel morally superior to another group of humans, then the probability drastically increases of their coming to behave murderously or exploitatively towards them.”
This is called “War”
What causes the feeling of superiority?
The “feeling of superiority” is a hormone generated survival characteristic.
I could be Anyone.
… For over thirty years I have owned and operated computers, they have been a just a hobby.
… AI was the reason I bought my first one, I wanted the computer to predict the lottery numbers for me.
The overwhelming truth was the computer could pick two of the three numbers from the history of the sets that I gave it, but I had to wait till the next day to find out what two.
That was 1985.
With the computer I have today the outcome of the program that I ran still can do the same, out the some of two out of three.
My point is AI can not pin point a prediction. Even at the correct two numbers of the next dayâs lottery, how could I use those numbers if I did not know which two use?
I worked at the program for over a year adding more and more information to the model.
I used the numbers from the dayâs and weeks before and added the humidly and several other factors. I never picked one correct set of three numbers nor did the computer.
I gave up.
Ai can only do ( if —- then statements ) no madder how you look at it.
Oh you may argue the ( or , and , is ) , but âifâ is really all it can do.
If I use to much power , then I will be turned off.
You Bet
Michael Anissimov created a bibliography in 2006 http://www.acceleratingfuture.com/michael/blog/2006/09/consolidation-of-links-on-friendly-ai/
Tom McCabe summarized FAI theory: http://www.acceleratingfuture.com/tom/?p=72
Add also: A book by Bill Hibbard, Super-Intelligent Machines, 2002
Also, Yudkowsky’s chapters in the book Global Catastrophic Risks