Infinite numbers have long troubled utilitarian ethicists: if the universe is boundless, then there is infinite positive utility and infinite negative utility, and we can’t even define our current state, let alone affect it. Oxford philosopher Toby Ord proposes a solution: use hyperreal numbers, a mathematical construct in which infinite sums behave more like finite ones (for example, the sum of infinitely many twos is twice as big as the sum of infinitely many ones). As with so many philosophical ideas, this solves some existing problems while creating some new ones (pinpointing some hyperreals requires choosing something called an “ultrafilter”, but this choice is underdetermined). Still, Elias Schmied calls this a real philosophical advance, of the sort which comes along only rarely and makes genuine headway into a previously-hopeless-seeming problem. I’m just surprised there’s a way of thinking about infinities under which they behave logically. Why didn’t we start with this one, instead of being told that infinity + infinity = infinity and we should just accept it? Claude tries to answer my questions.

That is from Scott Alexander (many more links at the link, many on AI safety).