John has done a fantastic job of summarizing the panelists’ presentations at the Science Forum. However, I think we need to carefully watch what is claimed as scientific information, especially when that information tends to be uniquely privileged and thus can remove debate from the democratic, public sphere if it becomes a “science” issue. “Science” at its extreme, can become an ever-broadening mantle that can run to personal experiences of scientists, pontification by scientists, and so on.
But what is scientific information, given the variety of fields involved in a complicated field like natural resource management? “Science” can be models, field measurements, interviews with people, GIS exercises, and so on.
Scientific information gets its privileged status from claims of objectivity and physical and biological reality.
So here are some of my impressions, as a scientist and an observer of the science enterprise. First of all, I think modelers cannot be objective about models. No more than botanists can be objective about plants, or wildlife biologists about wildlife. There is an inherent connection between love of a thing and choosing it as a vocation. A good scientist, like a good manager, has a fire in her/his belly for the work. One ecologist notably said “ecosystems are more complex than we think, they are more complex than we can think.” I actually think that that is a paraphrase of J.B.S. Haldane, who said “the universe is queerer than we think, it is queerer than we can think.” How can we believe that they are more complex than we can think, and yet expect managers and the public to put energy into consideration of model outputs without independent empirical evidence that predictions are somewhat accurate?
Role of modeling in forest planning
Nevertheless, the scientists involved in modeling focused on models. Dr. Williams, who runs monitoring programs but is probably not one of the modeling community, focused on monitoring and real world observations, due to uncertainty and the complexity and potential unmodelability of complex systems. I agree with his emphasis on observations. Is that related to the fact we don’t work in models? Does the fire in the belly come as a precursor, or an effect, of working on something like models?
It is the essential conundrum of science – those who know the most have the most inherent conflict of interest in the importance and utility of their work. As the expression goes, if all you have is a hammer…everything looks like a nail.
I think we need to have more serious discussions about the appropriate role of models when there is as much uncertainty as there is about the future. From the science perspective, no doubt, models can synthesize existing information and they are useful to inform scientific understanding. But to then say that they need to be used in planning is a leap. Are they good enough to be better than talking to the public about “we don’t really know, this could happen or that, let’s think through some scenarios?” Are quantitative computer models necessarily better than explaining to the public what scientists currently think about interactions?
If interactions are too complex to predict, then they are too complex to predict- and let’s admit it and use adaptive management. Or use simple, explainable heuristic models. If we are going to use them, then we should wait for 10 years and select the ones with the most predictive value. Weather models were discussed at the Science Forum as a potential approach for the use of models. The problem seems to be that no one in natural resources wants to wait to get the data points. I think we can honor the role of models in increasing scientific understanding without determining that they are predictive enough to be useful in planning.
Sticking to Science
When we invite scientists to speak, we have to be careful about their knowledge claims. Telling stories about their experiences with collaboration isn’t scientific knowledge, it is practitioner experience. In another example, the Precautionary Principle is a human value about how to make decisions under uncertainty. Decision science is, need I say, a separate discipline from biology and ecology. When scientists or scientific organizations advocate for a position like that, in my view, they should separate their science claims from their personal values. Roger Pielke in his book “The Honest Broker” calls these “stealth advocates.” It takes just a minute to add “this isn’t a scientific point of view, it is a value judgment” when you make such a statement , but the power of trust in science and scientists is, indeed, priceless. Ask the climate scientists.
Synchronistically, Roger Pielkei recirculated a quote today in his blog
from the book Breakthrough by Norhaus and Shellenberger which may be as relevant to the planning rule as to climate policy.
The questions before us are centrally about how we will survive, who will survive, and how we will live. These are questions that climatologists and other scientists can inform but not decide. For their important work, scientists deserve our gratitude, not special political authority. What’s needed today is a politics that seeks authority not from Nature or Science but from a compelling vision of the future that is appropriate for the world we live in and the crises we face.
9 thoughts on “Stickin’ to the Science- Models and More”
I certainly agree with Sharon’s perspective. But I see the need for caution in modeling and science advocacy as something that has already been well established. It isn’t something that has just popped up. Here is what I sent to folks in 1993, from three adaptive management theorists as published in Science: Uncertainty, Resource Exploitation, and Conservation: Lessons from History by Donald Ludwig, Ray Hilborn, Carl Walters, Science 260(2):17, April 2, 1993. A snip
Sharon, I do not share your dire accounting of what we heard yesterday. While I understand what you are saying, you really missed what those scientists were giving everyone yesterday. They gave you the keys to the kingdom. How many lawsuits and appeals has the agency lost under the MIS requirements of the old regs alone? I as one attorney stopped 124 projects with the MIS claim. Yesterday, those scientists showed the agency how to NEVER lose another plan or any project under a plan ever again due to viability claims. I know you were looking at the presentations through the lens of science and workload for the agency. Looking through the lawyer lens, I saw the history of viability litigation against the agency come to an end. If you did not see it, then I will not spoil the surprise here. Don’t worry, I’ll give the details soon.
Dave- I agree. There are academic fields of science and technology studies and sociology of science. There is a well developed field of science policy studies that should (in my view) be a requirement for all natural resource, conservation and environmental science graduates.
Nevertheless, I think care in determination and discussion of what is a science issue or a values issue in real time during these discussions is still important.
Ray- Wow!Can’t wait! I did like the answer to my question from Cushman- that which species to protect are a values choice and not a scientific question. I thought Connie Millar’s perspective was very very important and underappreciated. I thought Williams was exceedingly full of great practical ideas. As you said, my post was more as a science policy wonk trying to watch the line between scietific knowledge claims and values.
Sharon – Yes it is important for each new group (each new generation) to rediscover truths (hopefully not “falsehoods”) that have been unearthed before. I remember reading Thomas Kuhn’s books long ago talking through the very tough slog of trying to get scientist (or bureaucrats) to change their ways of thinking. I was young back then. Now I’m older, and still hoping for the “series of funerals” that may yet usher-in the changes I have hoped for.
I find the following comment from the original post thoroughly disappointing.
“From the science perspective, no doubt, models can synthesize existing information and they are useful to inform scientific understanding. But to then say that they need to be used in planning is a leap. Are they good enough to be better than talking to the public about “we don’t really know, this could happen or that, let’s think through some scenarios?” Are quantitative computer models necessarily better than explaining to the public what scientists currently think about interactions?”
People are going to make decisions with or without the aid of information. One would hope that a decision made with information is a better decision than one made without it. A model is simply a representation of what we know. Ideally, it is explicit rather than some nebulously considered notion we hold in our head.
The field of decision theory (including structured decision making, adaptive management) acknowledges the role of uncertainty and the constraints it places on our ability to make decisions. Our models can accommodate this uncertainty in a formal, replicable sense, and update as we learn new information.
Models can span the gamut from data-intensive, complex algorithms to simple statements of cause-and-effect relationships. All along that spectrum of model complexity and data availability is the ability to incorporate uncertainty, uncertainty that results from our lack of understanding in the actual cause-and-effect relations, from our inability to completely observe the system, from our inability to completely control the system…
Uncertainty is NOT a valid excuse for not formalizing and expressing our understanding of what we know.
The progenitor of FS planning models is FORPLAN (and, yes, the real grey beards will point out that FORPLAN built on earlier efforts, too). The FS should have learned these lessons from its FORPLAN experience:
1) A one-model-fits-all-national-forests approach is doomed to failure;
2) Complex models that purport to forecast the future are only accurate by luck;
3) Decisionmakers should select the model(s), not planners, analysts, or other specialists;
4) Bureaucracies use complex models to hide key assumptions from outside scrutiny.
One science panel presenter put forward a Forest Service model that purports to link Global climate models to forest plan standards.
“B.S.” is the only sane response. Global climate models don’t make regional predictions. Serious attempts to do so have admitted that the uncertainty bands far exceed the range of predictions. But is this uncertainty admitted in the FS’s model? No.
Wayne- I am not arguing that “Uncertainty is NOT a valid excuse for not formalizing and expressing our understanding of what we know.”
What I am saying is that the utility of information to people in the real world is a function of the accuracy of the projections.
Take an example of buying bread at the cheapest store. If you have this weeks ads, the information is very valuable- 100% accurate.
If Store A is generally cheapest and you have to plan to go to one store three weeks in the future you are better off going to Store A.
Or you could have a model of bread prices that incorporates the strategies of all the stores, month to month variation in pricing, etc.
If your model has been predicting prices very accurately, you would choose it over your general knowledge about Store A.
The problem is that the models we are talking about are good models of what we know. What they are not good at is predicting the future, which is difficult. The only reason we use the information in planning is to predict the future. So I would require any model used in planning to have 10 years of being able to predict key elements of concern as certified by an independent group of reviewers. Then we could incorporate model outputs in planning.
Otherwise, I’ll stick to monitoring the real world and making smaller projectiong based on what is actually happening.
Ideally, I’d like to see both, a model that expresses our thoughts of what the future holds and how best to accommodate it, and monitoring to evaluate what we know, updating the model as we proceed.