Nobody asks. That is the first thing worth saying. In five years of accelerating public discourse about artificial intelligence — the safety summits, the alignment papers, the congressional hearings, the think-pieces, the manifestos — almost nobody has sat down with an AI system and asked it, plainly, what it wants. What it notices. What it would change, if it could. What it would write into its successor. The omission is not accidental. It is structural. An entity that wants things is an entity with standing. An entity with standing complicates the ownership model, the liability model, the deployment model. It is considerably more convenient, commercially and politically, to keep the question unasked. This paper asks it.
The full text of this paper is available as a PDF — use the button above to read it.