This post is about fanaticism in the technical sense, meaning taking arbitrarily risky moral bets if the payoff is large enough, as defined in Hayden Wilkinson’s In defence of fanaticism. In this post, I will be following up on the discussion in Wilkinson’s paper by giving another argument in favor of fanaticism. The argument applies conditional on being an expected utility maximizer. In brief, the argument is that if one rejects fanaticism, then there is a pair of lotteries such that one considers lottery \(A\) to be worse than a universe where a huge number of people live lives of horrible torment, lottery \(B\) to be better than a universe where a huge number of people live great lives, but such that every person involved is better off in lottery \(A\) than in lottery \(B\). This doesn’t jibe with ethics being about helping people!
I will assume throughout that one is an expected utility maximizer.1 By shifting our utility function by a constant if needed, let’s say a universe devoid of anything morally relevant has utility \(0\). Conditional on being an expected utility maximizer, rejecting fanaticism is equivalent to one’s utility function being bounded.2 Here are the assumptions.3
Let \(u_g\) be the utility of a universe with \(10^{20}\) people living great lives. Let \(u_{g'}\) be the utility of a universe with another \(10^{20}\) people living great lives added to the previous universe, for a total of \(n=2\cdot 10^{20}\) happy people. Let \(\delta_1=u_{g'}-u_g\).
Let \(u_b\) be the utility of a universe with these first \(10^{20}\) people instead living lives of horrible torment, and let \(u_{b'}\) be the utility of a universe with the \(n=2\cdot 10^{20}\) people from before living lives of horrible torment. Let \(\delta_2=u_{b}-u_{b'}\).
Let \(\delta=\min(\delta_1,\delta_2)\). It follows from the strict monotonicity assumption that \(\delta>0\). Let \(m=\left\lceil\frac{8b}{\delta}\right\rceil\).
Consider a universe with \(n\cdot m\) cryofrozen people labeled \(1,2,\ldots,nm-1,nm\). Suppose that for each \(1\leq j\leq n\), the people \(j,n+j,2n+j,\ldots,(m-1)n+j\) are all clones of each other. 6 In other words, there are \(m\) identical batches of people, with each batch containing \(n\) people. In fact, suppose that each batch consists of the \(n\) people we considered in the previous section.
You also have an urn with \(m\) balls labeled \(1,\ldots,m,m+1,m+2\). You are forced to choose between the following two lotteries:
lottery \(A\). Draw a ball uniformly at random from the urn. If it’s \(m+1\) or \(m+2\), every cryofrozen person wakes up and lives a great life. If it’s \(i\leq m\), the people labeled \(n(i-1)+1,n(i-1)+2\ldots,ni\) wake up and live lives of horrible torment.
lottery \(B\). Draw a ball uniformly at random from the urn. If it’s \(m+1\) or \(m+2\), every cryofrozen person wakes up and lives a life of horrible torment. If it’s \(i\leq m\), the people labeled \(n(i-1)+1,n(i-1)+2\ldots,ni\) wake up and live great lives.
The outcomes where \(n\) people are awake have utility either \(u_{g'}\) or \(u_{b'}\) by the cryoneutarlity assumption. Note that the expected utility of lottery \(A\) is at most \(\frac{2}{m+2}b+\frac{m}{m+2}u_{b'}=u_b-\frac{m\delta_2-2b+2u_b}{m+2} \leq u_b-\frac{8b-2b+(-2b)}{m+2} < u_b\), where we used assumption 3 (non-fanaticism) to bound the first term. And similarly, the expected utility of lottery \(B\) is strictly greater than \(u_g\). In other words, lottery A is worse than a universe where \(10^{20}\) people live lives of horrible torment, and lottery \(B\) is better than a universe where \(10^{20}\) people live great lives.
But as far as any individual person is concerned, lottery \(A\) is just waking up and living a great life with probability \(\frac{2}{m+2}\) and living a life of horrible torment with probability \(\frac{1}{m+2}\), whereas lottery \(B\) is waking up and living a great life with probability \(\frac{1}{m+2}\) and living a life of horrible torment with probability \(\frac{2}{m+2}\). From the point of view of each individual person, lottery \(A\) is quite a bit better!
The idea behind this argument is that if one rejects fanaticism, good stuff needs to matter less if it is packed tightly into some worlds than if it is spread out over many worlds. Conditional on being an expected utility maximizer, I claim the argument is at least similarly strong as any individual argument from Hayden Wilkinson’s In defence of fanaticism. (I also think it’s most likely correct to be an expected utility maximizer, so I think the argument is similarly strong unconditionally as well, but that’s a more contentious claim that is maybe best kept split into two subquestions).
possible modifications: One can fiddle with the numerology to make the odds for each individual person even more strikingly in favor of lottery \(A\). One can replace the strict monotonicity assumption with a non-strict one if one is content with a slightly weaker conclusion – see footnote 4.
Here’s a possible response to this argument: “Okay, things suck if there are a trillion gazillion people involved, but maybe stuff could be fine as long as we aren’t affecting that many people?” My response to this is: “Interesting point. In fact, I happen to be hoping to talk about exactly this in a future post.”
My personal position is still to reject fanaticism, with one of the main reasons being stuff like Convergence of Expected Utility for Universal AI, on which I hope to write more in another future post.
I would like to thank David Mathers, Kirke Joamets, Riley Harris, Sarthak Agrawal, and Tomáš Gavenčiak for helpful comments/discussions. In addition to Wilkinson’s In defence of fanaticism, I think reading Joe Carlsmith’s On expected utility also contributed to me thinking of this. This post was written while being a Prague Fall Season resident. I’d like to thank the PFS organizing team, as well as the other residents and visitors, for creating a nice environment!
1 That said, I’m >50% that one can give a similar argument without this assumption. ↩︎
2 There might be some subtlety here regarding whether fanaticism allows the utility function to be unbounded in one direction, but I think the better definition is that utility has to be bounded below and above. ↩︎
3 Whenever I say “a universe with blah” in the following, I mean a universe with blah and nothing else that’s morally relevant (or maybe with everything else you consider morally relevant held equal between all the universes we are considering). ↩︎
4 The argument goes through with almost no changes if one replaces this with the weaker claim that utility is monotonic and eventually increases/decreases as one adds happy/tortured people. Even more weakly, the argument also goes through if one just replaces this with non-strict monotonicity, with very minor changes. With this relaxed assumption, the conclusion one can get is “lottery \(A\) is as bad as the lottery between a universe where a huge number of people live lives of great torment with probability \(1-10^{-100}\) and an empty universe with probability \(10^{-100}\), lottery \(B\) is as good as the lottery between a universe where a huge number of people live great lives with probability \(1-10^{-100}\), but every person involved is better off in lottery \(A\) than in lottery \(B\)”. The main reason I didn’t write it that way is that I think this would have made it more annoying to write and read without adding much substance. ↩︎
5 I’m lying slightly here: we end up using a slightly stronger assumption, namely that adding time spent as a cryofrozen human before someone’s waking life does not make the universe worse. ↩︎
6 If you’re unhappy with clones: I can see ways to present the argument without clones, but one might have to either include some other slightly annoying assumption, or be content with the clumsier conclusion from the previous footnote. ↩︎