| Next: | Appendix A: Friendly AI Guides and References | Bookmark | |
| Up: | Creating Friendly AI | Monolithic | |
| Prev: | 5: Miscellaneous |
There is a core of unknowability at the center of the Singularity, and it's important not to get so wrapped up in the details of how we think we can affect Friendliness that we forget about that horizon. There may be philosophical crises that are simply beyond our ability to anticipate, not in the way that Socrates couldn't have anticipated NASDAQ or nanotechnology, but in the way that a dog couldn't have anticipated NASDAQ or nanotechnology.Beyond the Singularity lies the unknown. The risk cannot be eliminated. The risk was implicit in the rise of the first human to sentience. The risk will be faced regardless of whether the first transhuman lives tomorrow or in a million years, and regardless of whether that transhuman is an uploaded fleshly human or a Friendly seed AI. What gets sent into that Horizon, in whatever form vis or his or her mindstuff takes, will be something humanly understandable; the challenges that are faced may not be. All that can be asked of us is that we make sure that a Friendly AI can build a happier future if anyone or anything can; that there is no alternate strategy, no other configuration of mindstuff, that would do a better job.
In striving to create an AI, we are not striving to create a predictable tool. We are striving to create a messenger to send on ahead to find humanity's destiny, and the design requirement is that ve handle any problem, any philosophical crisis, as well and as altruistically as a human.
The ultimate safeguard of Friendliness beyond the Singularity is a transhumanly smart Friendly AI.
| Next: | Appendix A: Friendly AI Guides and References |
| Up: | Creating Friendly AI |
| Prev: | 5: Miscellaneous |