Research Program

Background and Aims

SIAI will engage in a first phase of research that involves both theoretical analysis and tool building in the attempt to develop a viable framework for safe advanced AI. SIAI's hope is to bring about a world in which the vast potential benefits of this technology can be safely enjoyed by all of humanity.

The SIAI Research Program consists of two phases:

Phase I, 2007 – 2011: Develop a unified mathematical and conceptual framework for studying safe, beneficial AGI; and create software tools and technologies to assist with AGI design, engineering, and instruction.

Phase I mathematical and conceptual research will include:

  • The use of probability theory, decision theory, and algorithmic information theory to study AGI ethics.
  • The development of a shared ontology for discussing cognitive concepts important for describing AGI designs and their behaviors.

Phase I software tools and technologies will include:

  • A simulation world for AGIs.
  • A collection of IQ and ethical test scenarios for AGI systems, implemented in the simulation world.
  • A language for communicating with AGIs about ethics and other subjects with minimal ambiguity.
  • Knowledge resources allowing AGIs to more easily represent and reason about their own structure and dynamics.
  • Key cognitive technologies designed specifically to serve as flexible components within safe, beneficial AGI architectures.

Phase II, 2011 – : If viable, design and engineer safe, beneficial AGI, utilizing the theories, tools, and technologies developed in Phase I.

Both of our phases will involve a combination of in-house research and intensive collaboration with researchers in academia and industry.