Interesting read from Pielke’s blog! Ought to be required reading for scientists. Excerpt below….
Against Mathiness, Part 2
10 principles for effective use of math in policy research
- Use real-world variables
Policy research is more useful and relevant when it focuses on real-world variables. It is very easy for us researchers to study proxies for real-world variables or dimensionless indices in search of statistical or scientific significance. However, translating the practical meaning of those variables back to the real-world may not be particularly straightforward or even possible. .
Consider how the Intergovernmental Panel on Climate Change (IPCC) confused itself over a study of measurements of hurricanes, mistakenly converting trends in measurements of hurricanes to making claims about trends in hurricanes (which I documented here and here). The urge to use proxies for the thing-we-really-want-to-say-something-about often arises because the real-world variable does not give the results we want or expect. If you want to study hurricanes, study hurricanes. If hurricanes don’t give the results you want, that says something important — say it and don’t go looking for work-arounds.
Thanks for posting this, Steve!
I think about that a lot with using wildfire acreage as a dependent variable. If we want to know what causes “bad” fires as opposed to “good” fires, we can’t put them all together.
But scientists can’t study bad ones as opposed to good unless someone puts them in those categories. So it can be kind of a chicken and the egg thing.
Money is in climate and (bad) wildfires
There is no data on good vs. bad
We have a feeling for it and could interview fire suppression folks as to the important parameters that made a bad one bad… but that doesn’t employ models and satellites, so… not a preferred scientific tool. It seems a bit like the “streetlight effect” for science.
Coincidentally, LSE just sent this review of Escape from Model Land out in its newsletter.
https://blogs.lse.ac.uk/impactofsocialsciences/2024/01/05/escape-from-model-land-how-mathematical-models-can-lead-us-astray-and-what-we-can-do-about-it-review/
A good example of failing to use real-world variables is when the Forest Service claims that logging will modify fire and therefore save carbon or save habitat. This assumes an unreasonably high probability that wildfire will in fact interact with the vegetation treatment. In the real world, there is a low probability that feul reduction logging will interact with wildfire and therefore a low probability that the alleged benefits of fuel reduction will produce the assumed benefits for carbon and habitat.
In short, If you assume 100% probability of fire, then logging looks great, but if you assume a real-world (low) probability of fire, logging looks dumb.
What about the probability of fires that are over 200,000 acres? I would think that would skew your opinions. When such huge fires burn, the probability of a particular piece of land burning goes way up. The Caldor Fire burned over 90% of the Placerville Ranger District. Of course, some of those acres have been re-burned.
“because the real-world variable does not give the results we want or expect”
Pielke is drawing a conclusion about what appears to be dishonesty in research – I’m wondering what real-world data he is using for this?
I don’t think it’s dishonesty.. but the science community has narratives that are more or less popular and scientists also have their personal druthers which affects the questions they ask, the tools they use, and how they analyze them. Some scientists actually write that they believe their job is to influence others, not to look for the truth such as it is. E.g. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4758414/
‘Those who cannot prove beliefs, will try to sway opinions.’
Hi Larry: Who is the source of this aphorism? Also, please note that if a belief is proven to be true, then it becomes a fact and is no longer a belief — at least I believe that is a fact.
I just made it up, but there ARE people out there who practice the concept. Particularly those folks who ‘study’ something, somehow ignoring important realities that don’t fit the scope of their beliefs. Example: “All wildfires are good for the environment.”
(Actually, I wanted to alter the old quote: “Those who cannot do, teach.” )
“Anyone who can make you believe absurdities can make you commit atrocities.” This has been attributed to Voltaire, but is not a direct quote.
https://www.cato.org/publications/commentary/origins-warning-from-voltaire#