A few weeks ago, I heard Chief Moore say something like “the Forest Service is considering alternative measures for fuel reduction/wildfire resilience based on outcome not outputs.” This is somewhat related to our earlier discussion about timber targets.
I remembered that RVCC (Rural Voices for Conservation Coalition) had done some work on this, and the RVCC folks were kind enough to dig two reports up for me. So it’s worth discussing and feel free to share any other reports or thoughts in the comments. Discussions of performance measures do not normally get the blood flowing in many people I know. Retirees may just roll their eyes and say “Thank Gaia I don’t have to think about this anymore or sit in meetings or read stuff about it…” Nevertheless, here goes
It looks like there were two different thoughtful efforts. One (2020) was called “Implementing Outcome-based Performance Measures Aligned with the Forest Service’s Shared Stewardship Strategy.”
We’ll cover the other one (2022 with many of the same notable players) in the next post. This was a joint project of U of Oregon’s Ecosystem Workforce Program and RVCC with support from the Forest Service. The below is an excerpt of part of the paper. The whole report is worth reading, and has many other points worthy of discussion.
5. Guiding principles and recommended next steps
In this section our goal is to provide guidance to the agency for moving forward with revising performance measurements in accordance with the Shared Stewardship Strategy. Our suggestions are derived from recommendations from the literature, stakeholder feedback, and our own experiences working with the agency
5.1 Internal agency considerations to prepare for performance measure redesign
The agency must define and communicate a clear purpose and audience for new performance measures prior to moving forward. We suggest that the agency consider the following questions and recommendations before requesting input from stakeholder partners.
It would be nice if all efforts required “communicating a clear purpose and audience prior to moving forward.”
Implementing Outcome-Based Performance Measures Aligned with the Forest Service’s Shared Stewardship Strategy 13
• What decisions and changes are new performance measures intended to inform? Whose behavior will change, and at what levels of the agency, as a result of the new performance measures? Be cautious of defining too many goals for performance measures. Composite priorities, such as those that are often referenced together in Shared Stewardship (e.g., cross-boundary, geographic prioritization, partnership), may require distinctly different performance measures.• Will new performance measures replace or complement existing measures? New performance measures will not exist in a vacuum independent of existing measures, particularly timber volume and fuels reduction acre targets. As noted in the literature review, easily measured and defined goals and associated performance measures are likely to crowd out those with more complexity. Furthermore, if new performance measures have no connection to budgets or staff performance reviews, they are unlikely to motivate or institutionalize new bureaucratic behavior. The distinction between performance measures should be clarified internally within the agency and externally for partners prior to moving forward.
• Who are the intended audiences (e.g., WO,Congress, OMB, states, community partners) and what would be meaningful to them? A single performance measure is unlikely to meet
the needs of all possible audiences. Counts of partnership agreements, for instance, may help signal progress to Congressional audiences, but are unlikely to be particularly meaningful to local stakeholders or state implementation partners. We encourage dialogue with intended audience(s) to ensure performance measures are meaningful to those parties.• What investments will the agency be able to make in data collection and management? Utilizing existing data may be necessary and preferable in the short term; however, new performance measures will likely require some level of new data collection. We encourage the agency to recognize that updating existing databases and creating new fields, if not whole new data systems, is likely needed to meaningfully report on partnership outcomes.
• At what scale does the agency want to implement new performance measures? The recommendations and considerations offered below apply broadly across most or all scales, but
performance measure design and implementation will look different at varying scales. For instance, the principle of inclusivity may look different if a performance measure is intended to evaluate a District or District Ranger compared to a Region or Regional Forester.We also recommend that the agency make revised performance measures one part of a broader strategy to ensure that incentives and policies within the agency align with the intent of the Shared Stewardship Strategy. In particular, we suggest the agency convene a series of workshops for academic partners and practitioners who specialize in United States public lands forest governance and policy to consider options for broader reform efforts within the agency (e.g., reforming the National Forest Management Act, incentive structures within the agency, long-term visioning). We further recommend that the agency convene a structured meeting of national partners to further develop recommendations for implementing revised performance measures.
I’d only add that partnership-ish measures at the landscape scale should perhaps be coordinated in such a way that landscapes with intermix of BLM and FS should consider collecting the same kind of information and consider developing similar performance measures in those areas, if performance measures occur at the landscape scale. I think partners, neighbors and taxpayers would thank you.
I don’t think it matters much anymore what measures are used, the FS is so lax at holding managers accountable, there is little utility in pretending it means anything. I was once told the “racehorse” story of FS accomplishments; “we tie our major successes and performance to the “racehorses” of the Agency. It matters not, what the job is, the racehorses will run their race because that’s what racehorses do”. Pretty true, from my experience.
True story: I once was chastised by the RF for overspending our timber budget $600,000 for the year. The RF communicated to me it would affect my performance, and I needed a good justification for my action. I told him to talk to his Director of Timber. Puzzled, the RF ask why would the Director would know? I told him the Director ask me to do not only my Forests timber program for the year, but also an adjacent Forests program. I said “I can’t do that for nothing”….. The other Forest was not capable, or couldn’t do their own program, so we stepped up. Here is the good part, the other Forest Sup got a higher performance appraisal than did I! 🤣🤣🤣
Made me no difference; “they’re off and running…….”🤣
Jim, I think it’s not just managers who are hard to hold accountable. I’ve been on both sides.. trying to hold employees accountable and having my own performance changed drastically from one year to the next..doing the same stuff the same way, but targeted by new superiors for unknown reasons. Also there are unit reviews.. which I remember people taking seriously in my early career and now are sometimes just “opportunities for conversation.”
Region 2 is the only place I’ve ever seen a Regional Forester Review of National Forests. Maybe they occurred elsewhere, I just never saw it. That is a pretty high level (process heavy) exercise to look at a set of measurable, and not so measurable, elements. Probably worthwhile but questionable as to worth the effort.
As for Tony’s response of “projects, or measures meeting desired future conditions”, I assumed everyone augured in on timber sale reviews and Rx burn reviews. Do they not? It was probably the best effort I ever encountered to measure which way the “stick” moved; linking Planning with implemented results. I think Region 8 handled that exercise very well. And, it was all resource inclusive, not just timber and burning.
The FS lost its ability to actually measure individual performance when they went to the system that required the supervisors superior to review prior to meeting with the employee being rated. Really? That and ER and LMR were scared to death of actual written documentation on performance…..
A relatively simple place to start for evaluating outcome-based performance measures is comparing a project’s or program’s end result against the local land management plan’s “desired conditions”. I had felt in my career that targets became more about getting something done and less about whether the outcome of “getting something done” actually aligned with the “desired conditions” described in the LMP. No doubt…this will require more work to make a credible conclusion that an outcome is actually fulfilling a “desired condition.”
I’m not sure… every recent EA I’ve read (about vegetation) has a section talking about how it moves toward DCs.
That’s the point…it is written as if it is automatically assumes that a project’s completion WILL result in the outcome desired. Many times that is true, but to substantiate “outcome-based performance measures”, evidence is needed and that is where this “outcome-based” idea falls short – where is the evidence?
If this refers to performance related to partnerships and. Stewardship partners then it is probably a good thing. But should not get bogged down in to much analysis-main thing is that the FS person and his/her supervisor agree upon the measure and that it is measurable in some sense. It should not replace hard targets. There must be accountability
There is a need to have such performance standards after the RF. Receives his/hers the RF needs to set down and spell out expectations with those they directly supervise and so on down the line – this should not replace hard targets for most activities. There has to be accountability it should not develop into a swamp hole of meetings
Etc
I agree that “hard targets” help with accountability to ensure that budgets are are invested appropriately. However, if a line officer’s/manager’s performance is premised on whether things get done rather than things getting done well (ie, a project’s completion actually makes the environmental condition better (or at least not make it worse)), then line officers/managers will realize what helps THEM advance is more important than improving the land….which calls into question the land ethic of those line officers/managers.
Tony, maybe we’ve had this discussion before.. but I’d be happy if all federal employees would follow law and regulations. To me it’s unethical to substitute my own judgment for what is the correct thing to do for that of the system, such as it is. My example is if you worked for, say, Medicare. Suppose ethically you believed that everyone has a right to heathcare and started signing people up who didn’t meet the requirements. That way lies chaos. IMHO.
Point well taken…an element of integrity is needed to achieve a “greater good” that is bounded by laws/regulations. Unbounded advocacy can indeed lead to chaos…no different than the belief that NFs should primarily be managed for timber…that advocacy is not supported by the laws.