An assessment is the gathering and integrating of information relevant to the planning area from many sources and the analysis of that information to identify a need to change a plan or to inform how a new plan should be proposed. – section 219.5(a)(1) of the proposed Forest Service planning rule
It is a synthesis of information in support of land management planning to determine whether a change to the plan is needed. Assessments are not decisionmaking documents but provide current information on select issue. – section 219.19 of the proposed Forest Service planning rule
This is the first of a series of posts about possible approaches to preparing an assessment for a National Forest/Grassland Plan revision under the proposed Forest Service planning rule. (It is based on some informal conversations that Peter Williams and I have had with folks inside and outside the Forest Service, but nothing here reflects official Forest Service policy or the deliberations of the team working on the planning rule.)
The proposed rule expects a process that integrates both science and collaboration: “the objective of this part is to guide the collaborative and science-based development, amendment, and revision of land management plans.” (219.1(c)). Under the rule, an assessment must be collaborative and science-based, just as the overall plan revision process, because it brings together many sources of information, including social, economic, and ecological, whether qualitative or quantitative. Moreover, the subsequent process must rely on information from an assessment if the process is to be collaborative and science-based.
Although one immediate purpose of an assessment is to identify whether a need for change exists, the second, equally important purpose of an assessment is to inform design of the subsequent forest planning process that will propose specific changes to the plan if a determination is made that a need for change does exist.
Under this definition, an assessment is both a product and a process.
The product is a report similar to an “Analysis of the Management Situation” or other scoping documents under the 1982 planning rule. It documents “existing and potential future conditions and stressors” that subsequently will be the foundation for the revision’s Environmental Impact Statement. It describes the Forest in the context of the broader ecosystem, and what’s going on in the States and counties within and surrounding the Forest.
The process involves convening multiple parties at multiple scales to determine if the current Forest Plan is working by answering a set of assessment questions derived from the rule.
This rather long list of questions has the potential to be quite lengthy, so they need to first be screened to determine if they are relevant to the particular forest. Screening questions would include:
Assessment Goal | Coarse Screening Question |
Need for change in plan components or monitoring program | Is the information needed to inform and develop plan components (i.e., Is this a Forest Plan issue, not a program planning issue or a project issue)? 219.6(b)(1) |
Is the resource present? 219.7(b)(2)(ii) | |
Is the resource important? 219.7(b)(2)(ii) | |
Is addressing the resource within the authority of the Forest Service? 219.8, 219.9, 219.10, 219.11 | |
Is addressing the resource within the capability of the plan area? 219.8, 219.9, 219.10, 219.11 | |
Is addressing the resource within the fiscal capability of the unit? 219.10 | |
Is there an emerging public issue that needs be addressed? 219.6 | |
Design of process for revising a plan or monitoring program | Is the information needed to understand the discrete roles, jurisdictions, responsibilities, and skills of interested and affected parties? 219.4(a) |
Is the information needed to understand the expectations regarding the accessibility of the process, opportunities, and information? 219.4(a) | |
Is the information needed to determine the scope, methods, forum, and timing of public participation opportunities? 219.4(a)(1) | |
Is the information needed to develop required plan components (219.6(b)(1)), including information needed to inform design of the public notification and participation process? 219.7(c)(1) |
In answering the questions, technical information is essential, but an assessment under the rule should not merely be a technical process – it is fundamentally participatory, drawing on information and knowledge from multiple sources and multiple participants. During an assessment, the most accurate, reliable, and relevant scientific information is synthesized from governmental and non-governmental sources. But the process is also about clarifying values, because an important step is to identify why a particular National Forest/Grassland is important to the participants. One reason for clarifying values is that the knowledge being sought includes how a new plan should be proposed. That is a process-oriented goal. To meet such a goal in a way that is appropriate for the local situation, the assessment must seek to understand procedural preferences—values—of stakeholders, including but not limited to those of Forest Service personnel. The second specific assessment purpose is worth highlighting again: the goal of an assessment under the proposed planning rule is to gather and integrate information that informs design of a participatory and collaborative process should one be needed to change the plan.
Part 2 will describe how an assessment might be conducted.
I have long-argued that the Forest Service mistakes forest-level planning (also forest-level assessments and monitoring exercises) for the real task of assessment, policy making (including program development), and monitoring (with associated learning and course-correction policy making). See, e.g. Why three levels of planning? This effort seems to continue that mistake. In making this mistake, the Forest Service unduly burdens forest-level managers with tasks and responsibilities they can’t possibly fulfill in any meaningful way.
When dealing with assessments, monitoring, and other tasks of adaptive management, I find it much better to frame assessment questions around science and policy questions (who whats to know what, and why do they want to know it), then to sort scale, scope AND decision fora according to what is needed/desired, when, where, by whom, and for what purpose? Remember Wicked problem frames and Garbage Can decision making?. Specifically this:
Dave – If I understand the garbage can decision making model, it describes this crazy anarchic environment involving people with shifting issues and feelings looking for opportunities for decisions, and other people with shifting solutions looking for issues to solve, and finally, shifting decision makers. The problems and solutions are all dumped into a process like forest planning, which becomes the “garbage can” for all this stuff. Meanwhile, all of us, forest planners, forest supervisors, the public, politicians, etc. all jump in the garbage can, and whatever decisions are made is dependent upon what problems, solutions, and decisionmakers happened to be present and playing around in the garbage. Welcome to the world of forest planning!
So one approach to addressing the problem wickedness of forest planning is maybe to set up some other garbage cans, like travel management planning or fire management planning, or something at a bigger scale like statewide planning or ecosystem planning.
One good thing about the proposed planning rule is that it doesn’t rule out a regional forester, Chief, Undersecretary, or Secretary being the decision maker for a plan revision or amendment (see section 219.2(b)(3)). Also, the so-called “third level of planning” – project and activity planning – is such a huge category that it can encompass everything from a forest-wide oil and gas decision to a campground relocation.
The proposed rule also seems to imply that there will be “government or non-governmental assessments” out there to draw from, and the examples given in section 219.6(b)(2) seems to imply that these will be at a broader scale. Also, the rule’s broad-scale monitoring strategy would naturally lead to broad-scale assessments. In turn, I can certainly imagine a situation where broad-scale assessments might lead to broad-scale decisions.
The problem I see with the Draft Planning Rule, and also your “How to do Assessments…” is that there is but a hint of how to deal with the Wickedness both you and I see. In both, there is still a hope that somehow a forest plan can be a lynchpin in any “adaptive governance” that may be in play. What is needed in advance of more directives to forest-level practitioners is for the agency to define the adaptive governance approach in total, then to find out what parts of the context/contribution puzzle might be dealt with at the forest level relative to other levels (not necessarily associated with administrative units). Instead, as usual, the Washington Office dumps all on the field. They have done it in data-base development, in planning, in monitoring and evaluation and more. This practice needs to stop. The forests simply cannot deliver what is asked of them, and in reality the “ask” itself makes less and less sense when ecological concerns are at issue.
My approach, by contrast, seeks to address “wickedness” directly, by first admitting the problem (which the Forest Service never has), then by allowing “garbage cans” (what I like to call “decision-containers”) to be positioned as needed, rather than in force-fit manner, i.e. “national, forest, and project”. My supposition is that assessment decision-containers ought to be most often structured at geographical scales higher than forests, and likely set up in “scope” not to include all things. But we can leave that to the stuff of “outcomes” from publicly deliberated problems/resolution strategies. To give one example, it likely makes little sense to try to structure assessments that deal with say, Gray Wolves (or “large carnivores”) and migratory songbirds in the same assessment. So too with “monitoring and evaluation.” And in neither case does it make sense to deal with assessments at a forest level.
The Forest Service needs to design an adaptive management strategy for all this, yet refuses to even acknowledge such in any formal “rule making” sense. Or at least none I’ve seen. Instead it clings tightly to the “three levels.” I find three levels of planning/decision-making to be wildly “off base.” They made sense (sort of) way back in the olden days when the intent was to to manage a piece of ground (e.g. a national forest) for sustained yield of wood products. But since maintenance of biodiversity is the biggest issue faced by forest (and other ecosystem) managers, the forest-level primary focus makes no sense, at least not to me.
I make a big issue of this since I spent my career trying to help forests make sense of the over-burdened-requirements that the Washington Office always heaped on them, while letting Research and to a large extent the Regions off the hook. (That is, I made my career that way while fighting an never-ending-stream of nonsense directives from the Washington Office.)
One final point, for now: The “Analysis of the Management Situation” in the 1982 never made sense. To resurrect such now as a means to sell this stuff leaves me scratching my head, bewildered.
In regard to Dave’s final point, I confess to having played a modest role in the 1982 Analysis of the Management Situation — the “benchmarks.” These theoretical analyses were intended to demonstrate the opportunity costs, in timber volume foregone, associated with environmental protection. I explained to the in-coming Deputy Undersecretary of Agriculture, Doug MacCleery, how this analysis could be constructed. When he wrote the 1982 rules, he incorporated that conversation. Sorry about that.
The AMS discussion could be a red herring. John’s post explains an assessment under the proposed rule as being a product similar to an AMS. That doesn’t sound like an argument for an AMS under some other name. Also, consider how different this AMS discussion might be if the FS, under the 1982 regulations, had conducted an actual analysis of the management situation, rather than the arcane and marginally relevant exercise that an AMS became. A largely descriptive exercise conducted largely internally and supported by conjecture and unsupported assumptions, Andy Stahl, what were you thinking? Just kidding! Seriously, though, if an AMS originally had just focused on an analysis/assessment of the management situation/context, how much better off might we all have been?
Today, collaborative processes often begin with what’s known as an “situation analysis”. It’s a diagnostic undertaking, not a descriptive one. Perhaps that’s more closely aligned with what John’s saying in the post. It’s not a resurrection of an idea that was a distortion in the first place; it’s a reassertion of what the original idea might have been had the distortion not occurred. Specifically, it could have been a collaborative assessment of the current situation, possibly even including current preferences for future, so as to determine whether a need for change exists.
Also, I’m left wondering how Dave’s suggestion of emergently placed garbage cans of ideas is really all that different from John’s argument that the three levels of planning allow us to collaboratively identify needs for change such that those needs emerge from those diagnostic discussions? Aren’t the resulting needs for change at three basic levels, including a level above that of a forest or grassland, emergent and self-organizing based on the collaborative discussions, much as Dave argues would happen with garbage cans? Has me thinking Dave and John could be talking past each other on this point.
After all, addressing wicked problems successfully–meaning we don’t allow ourselves to become high-centered and not make any decision and we avoid the temptation to make a premature decision–often means stepping back and looking at one scale above and one scale below the focal problem. Seems like scale-dependent puzzle management that Dave suggests (I’d suggest “puzzle solving” is actually at odds with the idea of wicked problems because a wicked problem is characterized by an inability to establish agreement on what a solution would be and how to tell when one exists).
Nevertheless, while an understanding of wicked problems seems promising, there are some cautions to consider too. For example, in the case of forests and grasslands, addressing wicked problems would mean looking up at a broad/contextual scale and down at a narrower project scale. Elegant simplicity if done well. And, if done well, those basic concepts are communicated in an understandable, plain-language way, much as John has tried to do. Clearly, quite promising.
The caution is that, while the idea of wicked problems is powerful and has been in the natural resource management literature for 25 years, its still not the most accessible idea for some of those most needed to make the changes. John’s blog entries on assessment may have more attention to wickedness and wicked problems than a preliminary read suggests. At some point, perhaps a cross-walk would be good between the blog ideas and the concepts of wickedness, wicked problems, and other good concerns raised over the past 30 years of land management. Such an exercise in retroductive or abductive reasoning could be interesting and valuable, even if those terms aren’t entirely needed.
Andy, no need to apologize for the past..I think we all thought differently in 1982. I remember going on a timber field trip where OSU professors said that the latest science showed that clearcuts should be bigger to be more efficient (south central Oregon). And Weyerhaeuser was precommercially thinning lodgepole.. I think there’s probably enough lack of prescience to go around.
An Optimist muses, in #5:
First, the “garbage cans” are decision-making fora, not just idea jars. But that’s just a quibble. On to bigger matters:
To begin to assess whether or not John and I are “talking past each other,” I need to know much more about how the Forest Service intends to conduct “all lands” assessments/monitoring exercises (and with whom). Where is the “rule-making” or “policy-making” on that? And how does that, arguably larger undertaking, interrelate with relatively narrower forest plan updating? From both this little post (as 1 of 5 posts on “How to Assess”), and from the background paper John sent me, which needs to be referenced (as a pdf of doc) as backdrop for these posts I, get little indication of how this broader contextual framing feeds into the narrower forest planning exercise. Instead I get “optimistic” prognostications as to how people can collaborate (at multiple scales) to help an individual forest think about revising its plan.
Don’t get me wrong, I love the ideas of collaboration and deliberative democracy, but I believe that the contextual framing here is wrong, as it is in the Draft Planning Rule. Better to get directly to adaptive governance as per recommendations of the Clinton era Committee of Scientists (See Why Three Levels of Planning?), leaving forest-level planning as a minor play made with a robust information system of assessment, monitoring, and standard-setting information and policy that is largely done apart from (and at scales broader than) a national forest. It is in these setting, much more than in what we’ve called forest planning where “science” is a main driver. Yet we continue to hear calls to hold forest supervisors accountable for using “best science”. Such banter reminds me of holding a car dealership accountable for wind-tunnel-related design flaws in the models they show and sell. But that’s a matter for another post, e.g. here.
I take little comfort in optimism about people being brought together to deal with vexing forest problems at such local levels (too many arenas, too few, actors, too little time). I’ve been involved in too many of these, and feedback long ago led me to believe that we have to “look up a scale for context and management” as recommended in Allen, Tainter, and Hoekstra’s Supply side Sustainability. Besides I believe it a waste of effort anyway because of what I call the myth of the omnipotent forest supervisor. We all know that what forest supervisors do re: land management planning/policy-making is not done in an isolation booth, even one filled with forest-level collaborators. There are program managers (Regional and National) pushing (often with budget strings attached) on individual forest supervisors, usually via the Regions. There are local Congressional folks in the mix (usually via phone calls so as to leave no tracks), so too with Administration folks (remember Mark Rey and Jim Lyons and the multiple visits/phone calls to the field level folks?).
It is OK to be optimistic, but remember an adage that says, “Hope springs eternal!” Left unsaid, is that human “optimism,” unchecked by “pessimism” and other reality checks often leads to things like The Charge of the Light Brigade, and The Battle of the Little Bighorn, also thirty years of failed forest planning.