The Singularity Institute Blog

Peter de Blanc (Temple University), an SIAI Visiting Fellow, has recently published the final version of his most recent paper, titled “Convergence of Expected Utility for Universal Artificial Intelligence“. The paper, based on earlier work in the field of expected utility theory by Marcus Hutter (Australian National University) and SIAI Research Fellow Eliezer Yudkowsky, proves mathematically that any unbounded, perception determined, computable utility function cannot assign a defined utility to any action, assuming a Solomonoff-like prior. The abstract of the paper is as follows:

“We consider a sequence of repeated interactions between an agent and an environment. Uncertainty about the environment is captured by a probability distribution over a space of hypotheses, which includes all computable functions. Given a utility function, we can evaluate the expected utility of any computational policy for interaction with the environment. After making some plausible assumptions (and maybe one not-so-plausible assumption), we show that if the utility function is unbounded, then the expected utility of any policy is undefined.”

Peter de Blanc has done research with the Singularity Institute since 2006, when he participated in the “Summer of AI” research program along with Nick Hay (UC Berkeley), Marcello Herreshoff (Stanford University) and Eliezer Yudkowsky. During the summer of 2009, he was a Singularity Institute Summer Fellow, helping SIAI researchers work on problems in the fields of Friendly AI, rationality and decision theory. His personal website, Space and Games, can be found at http://www.spaceandgames.com/.