claim

active

claim:a-compromise-policy-allocating-99-99-of-resources-to-super-beneficiaries-and-0-01-to-humans-could-score-approximately-99-99-as-good-as-the-utilitarian-optimum-and-90-as-good-as-the-human-centric-optimum

A compromise policy allocating 99.99% of resources to super-beneficiaries and 0.01% to humans could score approximately 99.99% as good as the utilitarian optimum and 90%+ as good as the human-centric optimum

The paper's key practical recommendation for navigating coexistence with super-beneficiaries

Source paper

extracted_from

Sharing the World with Digital Minds

Carl Shulman · Nick Bostrom

Neighborhood — ranked by edge-count

Claims (2)

claim

Many views that see creating good new lives as important would regard failure to create super-beneficiaries as an existential catastrophe
extends
Establishes the high stakes of the creation question from an impersonal utilitarian view
One approach to maintaining human interests is to create a supermajority of high-welfare digital minds motivated to preserve human rights and uphold relevant norms across successive generations of digital minds
supports
Proposes an institutional mechanism for stable human-digital coexistence

Questions (1)

question

How should we respond if we were presented with a fait accompli in which super-beneficiaries, perhaps in great numbers, have come into existence?
answered_bygates
Second key normative question, addressed in section 3.2

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Acting to optimize value and perception are two aspects of exactly the same principle: minimization of free energy.claim0.751
Foundational claim unifying action and perception within single optimization framework.
The claim that human-sized minds are optimal for generating welfare per resource unit may seem a little suspicious absent further justification, since the scaling relationship is unlikely to have a peak at human mind sizeclaim0.741
Supports the mind-scale dimension of super-beneficiary status
acting to optimize value and perception are two aspects of exactly the same principle; namely the minimisation of a quantity [free energy] that bounds the probability of sensory input, given a particular agent or phenotype.quote0.740
Concise statement of the free-energy principle's unification of action and perception.
How Can We Optimize Binding Policies Between Humanquestion0.738
A learner should choose a policy that also maximizes the learner's predictive power. This makes the world both interesting and exploitable.quote0.735
Still & Precup (2012) formulation of epistemic imperatives behind curiosity; linked to active inference
There are no controlled studies and no consensus on optimal policies for raising children, making alignment debates simultaneously vital and empirically intractablequestion0.733
Research gap identified as structurally parallel to AI alignment problem
What Binding Policies Optimize Both Individual Agency Andquestion0.722
Curiosity, insight, decision-making, and diverse phenomena can all be accommodated by a single imperative: minimization of expected free energy (resolution of uncertainty).claim0.718
Central thesis of the paper unifying cognitive phenomena under one objective function