Thursday, April 2, 2020

Cognitive Hazards of Existential Risk

Amid the world-changing catastrophe and lockdown we are all experiencing now, people are wasting no time arguing about “who saw this coming and who took it seriously first.” Just today I read TUOC’s back and forth https://theunitofcaring.tumblr.com/post/614319297954709504/shlevy-theunitofcaring-slatestarscratchpad and Scott’s post about existential risk more broadly https://slatestarcodex.com/2020/04/01/book-review-the-precipice/ and those are among people I respect.

We could get lost in the weeds on the object level. My impression is that everyone “took this seriously” before some people they know/read, and after other people they know/read, and so your own impression of your success there is more about which group of people you choose to focus on. Myself, I was arguing to my boss that our office needs to act on this faster on March 3rd, which is well after some people, and well before many government officials. I don’t think it says much about me. Rationalist blogs took it seriously before Vox took it seriously before CNN took it seriously before the President, and every group’s villains take the opportunity to dunk on those later in the chain while their heroes lament.

So let’s talk about the meta level instead. Or why I think all the slamming on people for not seeing this coming is pointless and won’t benefit us in the long run anyway.

Rationalism, and many of its adjacent groups, prize very highly the trait of “taking ideas seriously.” This means talking about principles or threats not just in some “this may happen some day to other people” sense and a symbol to chatter about, but changing their own life based on the conclusions these ideas lead to. And in some ways, especially regarding your core ethical principles, I think this is very useful. 

But there seems very little discussion of the very good reason that most people don’t do this. Picture yourself as a politician, a leader of the country or at least your community. In the past few years you have been told by highly intelligent people that you respect about the following major problems that will explode any moment: global warming, AI expansion, EU breakdown and financial interdependence, terrorist attacks with nuclear or biological weapons, resource crisis, white nationalism, ebola, the breakdown of the middle class and/or nuclear family, social unrest and police brutality, and the end of late stage capitalism. All of these are very serious problems that very serious people tell you you need to act on NOW or society will end, and as far as you can tell, are equally convincing. 

You could expend all your credibility on the first one to cross your desk, and be forgotten in a week. And actually, many politicians do. We forget them. (Or they are like Marco Rubio, chasing forever after the latest big thing https://twitter.com/mattyglesias/status/1244683450742947842 )

Successful politicians, ones who stick around, offer a Very Serious and Wide-Ranging Plan Addressing the Roots of this Crisis and Calling for Urgent Action, which often involves expert task forces and grants to some organizations, and then forget about it because nothing will happen on their plan and they know it. (In fact you can credit Joe Biden for doing this exact thing on January 27 https://www.usatoday.com/story/opinion/2020/01/27/coronavirus-donald-trump-made-us-less-prepared-joe-biden-column/4581710002/ ). This is the opposite of “taking ideas seriously.” This is showing attention to a problem for the sake of others’ judgment, without having to change your life or direct action much at all.

Economists have a pithy version of this attitude, as applied to people who try to short a company that they just KNOW is over-valued: “The market can stay irrational longer than you can stay solvent.” (And indeed, many investors who saw the 2008 financial crisis coming and tried to bet against it… bet in 2006 and 2007 and lost all their money first.)

You can’t just take every world-changing threat seriously at face-value and adjust your portfolio likewise. You will lose all your money, all your credibility, and have a broken shell of a life.

And most people, on a “meta-rational level”, actually know this? That you have to have some dissonance between your stated beliefs and how you guide your day to day existence, and that letting every change in the former affect your latter will throw you wildly off course. So they nod soberly at the new articles, maybe post online about it and use it as a reason to contribute $100 to their already favorite politician, maybe spend an equivalent amount on some consumer good that makes them feel prepared (gold against a financial crisis, prepping supplies, etc) and move on with their lives. And in most cases *this is very good for them.*

The problem being of course, that most cases are not all cases, and the one risk that proves out destroys all your hard work too.

The sort of competition-as-a-virtue ideology that the leading edge of our culture has now (from Wall St. to Silicon Valley) actually makes this worse. Even if you have an entirely accurate picture of future risk, if you have a competitor that just *ignores* or undervalues future risk, and doesn’t budget and plan for it, they can just beat you by price (or whatever.) Only entities that don’t have to worry about the short term can really prepare for future existential risk, and until now, our cultural ethos was very much against those sorts of entities, deriding them as traditional, monopolistic, and unaccountable (which they often were.)

The answer isn’t to just “ignore all future risks.” But it’s also really not to take a prideful stance of insisting *all* long-tail risks need to be treated with immediate seriousness. If you do this, you will find yourself very disappointed: as all your followers either ignore you, or lose all their money and sanity on the many other claims on their attention.

Unfortunately we actually need to analyze upcoming risks honestly. Most will never affect us. Some actually will. This is, well, incredibly hard. In particular a sense of proportion is very valuable here. (Rationalists like the linked SSC post will go on about the vital difference between .00001 and .01 chance, but frankly I think most people don’t react differently to .5 and .1 chance currently.) 

So let’s be honest about how people are actually coming to take this crisis seriously: it’s not because an expert presented them with a scary chart. People, especially top decision makers, are used to seeing scary charts. It is when the Real of the experience actually starts to hit them in the face. Almost every state put off any action until sick people were already in their state and multiplying by (if small) quickly increasing numbers.

You can’t even decide “well I will trust the experts in the field or this one particular expert I respect”, since well, the most common failure mode of experts is to vastly overstate the importance of their field.

There is an interesting side-tangent here of the difference between people who are “very online” and those not. For the past year the political internet had pondered this distinction, as twitter/etc discourse failed to resemble American voting outcomes so starkly, even among the Democratic primary voters. The implication was that by being “too online” we were stewing in so much self-referential memetic influences that we had lost touch with what “real people” thought, and so being very online was bad for your mental hygiene. In the past couple months, the lines of these categories persisted, but now it was the “very online” who were dramatically right about something relative to people just listening to talk radio and reading local papers.

--

(*Yes, this post uses the term “existential risk” for things that will just kill dozens of millions of people and raise unemployment to 20%, and not the technical meaning which would require the obliteration of all human life. Existential risk more commonly refers to “that would coerce drastic changes on you”, such as a neighboring army that can topple your government, without needing the whole complete extermination of humanity thing.)