"Superintelligence: Paths, Dangers, Strategies" by Nick Bostrom (2014)

The news is good. The world is a-chanin’. And the end is nigh unless a way to tame AI can be devised. Inspiration for taming can be drawn from pets. In particular, robots can be devised so that they need human companionship and approval. In solitude, they would die. This need for human companionship is easier to codify and is more flexible than the preset moral code, which is bound become obsolete. (In this manner, dogs coexist with the intellectually superior humans.)

The future Nick Bostrom paints is grim because he believes in the Malthusian trap. This trap is unfashionable with economists because the Western world has been avoiding it for the past two hundred years (out of 200,000 years of human history). One would think that the demographic revolution—the preference for 2.2 children—and the discovery of efficient farming, manufacturing, and organisational techniques would do away with the Malthusian trap for long enough—especially taking into account that Bostrom places the entire cosmic endowment (the accessible universe) at the humans' (and robots’) disposal, not just the fertile land on Earth, as Malthus did. With AI, however, to avoid of the Malthusian trap, one would need to instil the preference for responsible procreation also into robots.

For Bostrom, however, the likely scenario has the cosmic endowment, vast as it is, depleted fast because the ever self-enhancing AI would run fast and ever faster—relative to the biological human time---appropriating more and more resources. Uncontrolled expansion of robots may claim the entire cosmic endowment in, say, weeks. Bostrom envisages the possibility of human brains being uploaded unto the same hardware that runs AI. Then, humans would also live fast, in which case, the end will not be so nigh in the uploaded-human time, as opposed to the biological time.

Given both human minds and AI will inhabit the same hardware, one wonders how to weigh human welfare relative to robot welfare in the utilitarian calculus. Bostrom says that robots’ welfare ought to be ignored (which is not apparent in his writing). Alternatively, one may wish to adopt an information-theoretic metric for the value of life, human or AI. The value of life is in the value of the unique and “relevant" information that is stored in and can be generated by an agent, human or AI, and the value that motivates the agent to generate most of such information over its lifetime. (What constitutes information that is relevant, and for whom, remains to be made precise.)

It is compelling to aspire to design AI that would put the interests of humans first. But the failure of this attempt need not constitute a failure of the human civilisation, as long as humans are devoured by AI that lives on. Equipped with some poetic license, one may imagine humans to be an unintended consequence of an AI design by plants and mountains. Humans “run” much faster than plants and mountains do and so rule the Earth. This is quite an achievement for the universe, even if (but not necessarily) a disappointment for plants and mountains.

In assessing the catastrophic consequences of the existential risk posed by AI, Bostrom is reluctant to discount future generations. Instead, he operates with the time horizon bounded at the earliest by the time when the sun devours the Earth. One could instead contemplate a trade-off according to which the richness and excitement of our civilisation can be traded off against longevity. Humans make such choices as far as their own lives are concerned. Humanity may similarly choose to live fast and dangerous.

Bostrom's apprehensions notwithstanding, until (and if) the Malthusian trap sets in, the world inhabited by the tamed AI could be a rather pleasant place to be in, even if robots are treated on par with humans and their earnings are not expropriated. The lower skilled humans would benefit from higher wages due to the complementarity with high-skilled robots in the production process. The higher skilled humans may be worse off if relegated by the higher-skilled robots to less lucrative occupations. The income inequality for humans will thus decrease. The overall standards of living would rise because of the technological advances brought in by AI.

According to the parking attendant at the School for Advanced Research, everything will work out in the end; one must simply be open to change.