Thinking Inside the Box: Controlling and Using an Oracle AI

Authors:
Stuart Armstrong;Anders Sandberg;Nick Bostrom
Affiliations:
Future of Humanity Institute, Faculty of Philosophy, University of Oxford, Oxford, UK OX1 1PT;Future of Humanity Institute, Faculty of Philosophy, University of Oxford, Oxford, UK OX1 1PT;Future of Humanity Institute, Faculty of Philosophy, University of Oxford, Oxford, UK OX1 1PT
Venue:
Minds and Machines
Year:
2012

Citing 5
Cited 0

Introduction to Reinforcement Learning

Introduction to Reinforcement Learning
Beyond AI: Creating the Conscience of the Machine

Beyond AI: Creating the Conscience of the Machine
The Basic AI Drives

Proceedings of the 2008 conference on Artificial General Intelligence 2008: Proceedings of the First AGI Conference
Artificial Intelligence: A Modern Approach

Artificial Intelligence: A Modern Approach
The Superintelligent Will: Motivation and Instrumental Rationality in Advanced Artificial Agents

Minds and Machines

Quantified Score

Hi-index	0.00

Visualization

Abstract

There is no strong reason to believe that human-level intelligence represents an upper limit of the capacity of artificial intelligence, should it be realized. This poses serious safety issues, since a superintelligent system would have great power to direct the future according to its possibly flawed motivation system. Solving this issue in general has proven to be considerably harder than expected. This paper looks at one particular approach, Oracle AI. An Oracle AI is an AI that does not act in the world except by answering questions. Even this narrow approach presents considerable challenges. In this paper, we analyse and critique various methods of controlling the AI. In general an Oracle AI might be safer than unrestricted AI, but still remains potentially dangerous.