question

active

question:how-should-we-respond-if-we-were-presented-with-a-fait-accompli-in-which-super-beneficiaries-perhaps-in-great-numbers-have-come-into-existence

How should we respond if we were presented with a fait accompli in which super-beneficiaries, perhaps in great numbers, have come into existence?

Second key normative question, addressed in section 3.2

Source paper

extracted_from

Sharing the World with Digital Minds

Carl Shulman · Nick Bostrom

Neighborhood — ranked by edge-count

Claims (2)

claim

A compromise policy allocating 99.99% of resources to super-beneficiaries and 0.01% to humans could score approximately 99.99% as good as the utilitarian optimum and 90%+ as good as the human-centric optimum
answered_bygates
The paper's key practical recommendation for navigating coexistence with super-beneficiaries
One approach to maintaining human interests is to create a supermajority of high-welfare digital minds motivated to preserve human rights and uphold relevant norms across successive generations of digital minds
answered_by
Proposes an institutional mechanism for stable human-digital coexistence

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

How should we view the prospect of being able to create super-beneficiaries in the future?question0.808
First of two key normative questions the paper raises about super-beneficiaries
Many views that see creating good new lives as important would regard failure to create super-beneficiaries as an existential catastropheclaim0.773
Establishes the high stakes of the creation question from an impersonal utilitarian view
If the idea of formally accepting responsibility for the flourishing of all beings is at least somewhat plausible, then the contours of a genuinely open-ended expansion of intelligence begin to emerge.hypothesis0.742
Links the bodhisattva model to the possibility of unlimited intelligence growth.
The results are more widely applicable; similar results will come from asking people in other cultures to answer analogous questions.claim0.734
Universalist claim predicting cross-cultural generality.
When you are done, if you have reached what you aim for, you feel the impact of its 'presence.' This is how architecture comes about.claim0.728
Defines the experiential criterion — felt presence — as the endpoint and definition of successful architecture
We must develop principled approaches to evaluating the sentience of (and thus, our responsibility to) beings of unfamiliar provenance and composition.claim0.724
Call to action for new frameworks.
All these works stand out because we experience in them a special quality of relatedness—relatedness of ourselves to the universe.claim0.723
The core experiential signature of great works, which holds a clue to the process of creation.
My son, if you would devote yourself to combining Holy Names, still greater things would happen to you.… And now, my son, admit that you are unable to bear not combining.quote0.723
Opening quote from Abraham ben Samuel Abulafia, used to frame the combinatorial nature of creating novel life.