"Utility function security in artificially intelligent agents" by Roman V. Yampolskiy
 

Utility function security in artificially intelligent agents

Roman V. Yampolskiy, University of Louisville

Abstract

The notion of wireheading, or direct reward centre stimulation of the brain, is a well-known concept in neuroscience. In this paper, we examine the corresponding issue of reward (utility) function integrity in artificially intelligent machines. We survey the relevant literature and propose a number of potential solutions to ensure the integrity of our artificial assistants. Overall, we conclude that wireheading in rational self-improving optimisers above a certain capacity remains an unsolved problem despite opinion of many that such machines will choose not to wirehead. A relevant issue of literalness in goal setting also remains largely unsolved and we suggest that the development of a non-ambiguous knowledge transfer language might be a step in the right direction. © 2014 Taylor & Francis.