Optimal timing for superintelligence – Marginal REVOLUTION


There is a new paper by Nick Bostrom with that title:

Developing superintelligence is not like playing Russian roulette; it is more like undergoing risky surgery for a condition that will otherwise prove fatal. We examine optimal timing from a person-affecting stance (and set aside simulation hypotheses and other arcane considerations). Models incorporating safety progress, temporal discounting, quality-of-life differentials, and concave QALY utilities suggest that even high catastrophe probabilities are often worth accepting. Prioritarian weighting further shortens timelines. For many parameter settings, the optimal strategy would involve moving quickly to AGI capability, then pausing briefly before full deployment: swift to harbor, slow to berth. But poorly implemented pauses could do more harm than good.

Via Nabeel.




Source link

  • Related Posts

    Kathy Ruemmler leaves Goldman Sachs amid Epstein files fallout

    The top lawyer at Goldman Sachs, Kathy Ruemmler, said Thursday that she would leave the prestigious Wall Street firm amid controversy over her email correspondence with Jeffrey Epstein, which was…

    Japan, US Far Apart on $550 Billion Fund Projects, Akazawa Says

    (Bloomberg) — The US and Japan remain far apart over which projects should be funded first under the framework of a $550 billion investment vehicle even as the two sides…

    Leave a Reply

    Your email address will not be published. Required fields are marked *

    You Missed

    Prosecutor Seeks Dismissal of Charges Against Man Shot by ICE

    Amid disappointing earnings, Pinterest claims it sees more searches than ChatGPT

    Amid disappointing earnings, Pinterest claims it sees more searches than ChatGPT

    Arsenal: Will Arsenal’s end-of-season form haunt them again?

    Arsenal: Will Arsenal’s end-of-season form haunt them again?

    Alila Mayakoba is open in Mexico: Book from 21,000 Hyatt points

    Alila Mayakoba is open in Mexico: Book from 21,000 Hyatt points

    WATCH: Jordan Binnington quiets the critics in Canada’s shutout win over Czechia

    Kathy Ruemmler leaves Goldman Sachs amid Epstein files fallout

    Kathy Ruemmler leaves Goldman Sachs amid Epstein files fallout