Climate Model Output Uncertainties Not Communicated to Research Users: Comments on the CWG Report II

I was originally going to do three posts on the  DOE CWG report, but the series is expanding.  Today I’m going to talk about a series of disconnects between climate research and potential users of climate research, including some other ways of thinking about and communicating uncertainties.

Regional Models Fit For What/Whose Purpose?

From the CWG report:

The IPCC acknowledges limitations in the accuracy of regional climate model outputs. This example shows that users need to assess model projections carefully on a case-by-case basis since local biases might be sufficiently large that the models are simply not fit for purpose. As has recently been noted by two leaders of the modeling community (emphasis added)
… for many key applications that require regional climate model output or for assessing large-scale changes from small scale processes, we believe that the current generation of models is not fit for purpose. (Palmer and Stevens 2019)

I became curious as to “fit for what purpose” and “who decides?”.  So I looked at Palmer and Stevens (open access) which turns out to be a “science op-ed” pointing out that models are not as helpful as they could be, with the perspective of needing lots more bucks to be better.  The authors think that overstating model accuracy has led to lack of investment in making them better.

As climate scientists, we are rightfully proud of, and eager to talk about, our contribution to settling important and long-standing scientific questions of great societal relevance. What we find more difficult to talk about is our deep dissatisfaction with the ability of our models to inform society about the pace of warming, how this warming plays out regionally, and what it implies for the likelihood of surprises.

My bold.  And this is where “fit for purpose” comes in.

The deemphasis of this type of information, while helpful for focusing the reader on the settled science, contributes to the impression that, while climate models can never be perfect, they are largely fit for purpose.However, for many key applications that require regional climate model output or for assessing large-scale changes from small-scale processes, we believe that the current generation of models is not fit for purpose.Figs. 2 and 3 develop this point further by showing how, on the regional scale and for important regional quantities (7), these problems are demonstrably more serious still, as model bias (compared with observations) is often many times greater than the signals that the models attempt to predict.

Again, my bold. You might be thinking.. uh-oh.. whose purpose? Who decided? Is the purpose for the climate science community or for users of models?  Were the users asked what their needs were? Why are we using regional models to inform land management decisions if we haven’t figured out what purposes they are fit for? And how can people write papers about impacts on plants, like corn or forest trees, whose growth depends on micro-not even regional -climates if the climate models are not “fit for purpose” in some undefined way?

Stories:  Long ago, I was chatting with forest economist Richard Haynes one day, and he mentioned something that stuck with me.   He had worked on (I think it was the ICEBMP scientific assessment) and said “those biologists develop models and don’t do sensitivity analysis on their assumptions.” It stuck with me because I thought it was interesting how different disciplines operate with apparently different views of treating uncertainties.  Years later, a co-worker and I (we were the Climate Change folks  in R-2) went to the Temple of Climate (NCAR in Boulder) to talk to folks there who were collaborating with the BLM (on a pointless research project IMHO but whatever).  I asked the question “do y’all do sensitivity analysis on your assumptions?” and they said they couldn’t, because the models were too big and it would have taken too much computing power.  That’s the time it became clear to me that we were being asked at the Forest Service to incorporate model outputs in our management, without the uncertainties being provided with the information.

Uncertainties are hard. I get it.  Economists have various ways of dealing with it in terms of math.  We forest genetics folks gave projections with caveats in words.. like “you can expect to get this kind of increase in growth on plantations in this area, managed like this, unless conditions change, like hurricanes or bugs or ….” so the uncertainties were communicated directly to the decision maker. In fact, it reminds me of an apocryphal story of the Timber Years in Region 6.  The story was that genetic improvement could have been put into models, and Pete Theisen, the Regional Geneticist, was asked if we could cut more now, based on those projections.  His answer was no, we have to wait and see.  We’ll see more about this idea in the next post.

My point here is that many disciplines have developed ways of dealing with uncertainties. Often these are explicitly communicated via words, or math, or both, to users of the information.  Or the expert just says something like “that’s our best guess.” And the user of the information would balance that with other factors, each with their own associated uncertainties. But perhaps, as the Palmer and Stevens paper suggests, in the rush to influence policy, some climate scientists have not been upfront about the uncertainties around their projections, and especially those that matter most to us..at regional and local spatial scales.  What to do? Some ideas in the next post.