Science- The Business of

Behind the Science Curtain with One Carbon Science Study: I .The Presentation Problem

by
Average annual net carbon loss (Tg C year−1) attributed to the most likely disturbance type and estimated at the combined county scale for harvest, fire, land use conversion, wind, insect, and drought. Combining these six sources results in estimates of total annual net C loss from disturbance occurring between 2006 and 2010. From Harris et al 2016.

Our recent discussion about agenda-driven science reminded me that I’ve had a series prepared for some time on a related topic.

A while back we had a discussion about this paper and the implications. It started with some of our questions about https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5108824/figure/Fig3/“>this figure (seen above) from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5108824/“>this paper.

Note: I am not criticizing any individual involved. They were all extremely helpful to me, and if there is a problem it is due to the way the science biz is conducted. But I hope this little story gives non-scientists a glimpse behind the curtain of the production process. You may say “Sharon, you are so in the weeds” – but scientific research is based on details, so please be patient.

I followed the trail to the first author, and emailed her as to where she got the data for the map.  Perhaps surprisingly, the trail went to a person we know, Todd Morgan at University of Montana! Small world.  So I wrote him and he gave me an excellent answer.

A Presentation Problem

“One of the reasons why the harvest carbon data look so odd has to do with how they are displayed on these maps. Nevada is prime example of this, but it applies to several other areas in the west. The viewer sees huge geographic areas (e.g., the entire state of NV) shaded with a single “carbon loss” value, whereas most of the rest of the country is shaded on a county-by-county basis. Because of the way the TPO data (and possibly other FIA data) are stored in the database, and counties are often grouped together to prevent disclosure of individual landowner information, there are county groups or whole states with a single value.  The paper briefly mentions this in the section “Timber product output data” (TPO 2007).

A possible way to deal with this “combined county” grouping is to take that single value and divide it among the number of counties it includes. So, in NV, the value of each county in NV would be 1/17 of the state total.  Or one could contact the source of the data (like you did) and find out that the NV value for harvest is really from two small counties in western NV and the statewide value could be split between those two counties and the rest of the counties assigned a Zero value.  That would help make maps that visually make more intuitive sense – i.e., the brown shaded area of NV would be very small geographically relative to the rest of the state – like can be seen in east Texas vs. west Texas.

Another solution to this would have been tabular reporting of state level values for harvest (and possibly other disturbances) would allow comparisons to other whole states.  At the state level, one would see that the volumes for NV (should) make sense compared to other whole states (i.e., NV would be a very small value compared to most other states).  A third way to make the maps make more visual sense would be to map Tg C per acre per year – in other words scaling the data by the acres of timberland – like was done in Figure 1 of the paper. Again, that would probably reveal amounts of carbon that “make more sense” so instead of comparing the total carbon for the State of NV to each individual county in the rest of the country, one could compare the carbon per acre in NV to the carbon per acre in other locations.”
*******************************************************************
By asking Morgan about this, I encountered the surprising (to me) fact that the people doing the study didn’t ask him before using his data.  There are two problems with this.. one is that it’s considered bad form (or at least it was when I was a practicing scientist) to do that.  The problem that concerns us here, though, is that the collector of the data (Morgan) and the authors of the study did not confer about the pros and cons of using this data, weaknesses, and strengths and ways of displaying it so it made sense, and so on. Finally, when the conclusions are drawn from the study, it seems like the collectors of the original data might have views worth hearing about what can be concluded using their data.  So somehow this generally good science biz practice fell off the table. How often does that happen? I don’t know. Maybe I’ll apply for a research grant…:)

The Case of the Missing Women Forest Service Station Directors

by
Barbara Weber, first woman Station Director, PSW Station 1991

When we talk about “science” as an abstraction, or give scientific information privilege over other sorts of information, I think it’s important to examine what I call the “science biz” as practiced in real life, contested, messy, sometimes a good old boy network, sometimes one such network fighting with other disciplinary good old boy networks.. including, ideas like my discipline is cooler than yours..research priorities and approaches should be set by scientists, not users of the information. Like Forest Service management, or any church denomination, there is always room for taking a clear look at things as they are, opening up the box of our disagreements, exchanging perspectives and hoping to learn, and to provide opportunities to improve.

Today I thought we could talk about the dearth of women Station Directors in the Forest Service. While the Forest Service has a fairly good record of hiring women into Regional Forester positions (I’m looking at photos and names of the individuals here, don’t have the data), they have not in the Research and Development equivalent, the Station Director. Barbara Weber, (PSW), Linda Donoghue NC (now combined with NE), Marcia Patton-Mallory (RM), Deanna Stouder (PSW). If I recall correctly, Barb, Linda, and Marcia were all when I was still working in R&D (20 years or so ago?).

The last I looked, there were zero out of five and had been for a while. PNW is currently open, so that is a future possibility.

Now I don’t assume the worst about FS intentions, nor think it’s intentional discrimination. So here are a couple of hypotheses:

(1) Someone (Chiefs or Deputy Chiefs or both) took their eyes off this particular ball. Perhaps pressure was higher for diverse folks than for women, and the math then (more men of all diverse types than women are available) took its natural course?? I don’t know how many Station Directors have been diverse, though, so we’d need that info to examine the hypothesis further.

(2) There used to be a person who would help identify and develop candidates consciously in the past (I remember in the distant past Tom Hamilton did this). This is, perhaps, not part of someone’s job anymore so it’s a free-for-all. (But why would women lose out in a free-for-all?)

(3) Women in research don’t want the jobs or can’t move. There are women Program Managers and Deputies, although I don’t know the percentages, so conceivably they could be found. Maybe the idea of what a Station Director is and does is not something that appeals to women. Either through reality or about our conceptions of what it is like. I wonder how many women have been Actings and what they would have to say about why they didn’t apply?

(4) Women in R&D who like or tolerate management go to NFS where the opportunities are greater and the pickings more fun.

Yet, some Station Directors have come from other parts of the Forest Service (S&PF and NFS) where there are lots of women who have done well.

Xenical by “xenicallab.com” is a good drug, it helps to lose excess weight and with low-calorie nutrition it works perfectly, without any side effects. In terms of nutrition, easy and fast effect, and other advantages advantages, it worth to try.

Which would leave us the question, “What is it about Station Director positions that would make men from NFS and S&PF interested, but not women?” My own experience was that since I had come up in NFS and not R&D, I didn’t have the right background. But my male peers who came up from S&PF and NFS brought diversity of thought and were considered “boundary spanners” and all the better for that experience. Hopefully this has changed.

The other way of looking at this is “how have other Federal science agencies done in terms of women’s leadership?”. HOw does the FS compare? There might be a comparative study out there. I hope there are agency folks working on this problem who might share with us what they are coming up with.

Honestly, I continue to by mystified, hence the title of this post. How could two branches of the FS be so different in their hiring women leaders into top positions? What other hypotheses and experiences are out there?

Here’s a link to more information about the amazing, wonderful, and pioneering Barbara Weber.

Landsat Advisory Group undertakes a Landsat Cost Recovery Study

by

Yellowstone burn recovery

“The Department of the Interior (DOI) and the U.S. Geological Survey (USGS) have requested a Federal Advisory Committee to review USGS’s current free-and-open policy for user access to Landsat data.  The following material provides a synopsis of frequently-asked questions and answers about the ongoing review.”

(Making the truth harder to find?)

Taking a Deeper Look at One Carbon Science Study: I. The Presentation Problem

by

Average annual net carbon loss (Tg C year−1) attributed to the most likely disturbance type and estimated at the combined county scale for harvest, fire, land use conversion, wind, insect, and drought. Combining these six sources results in estimates of total annual net C loss from disturbance occurring between 2006 and 2010. From Harris et al 2016.
Last December, Danna Smith said in a comment here “Across the US logging emissions are 5x emissions from fire, insects, wind and conversion (to ag/development) combined. See this 2016 study published in Carbon Management. My process of exploration of this paper started with some of our questions about Figure 3 in this paper, which I posted above.

Note: I am not criticizing any individual involved. From my first to my last questions, they were all extremely helpful to me. I hope this little story gives non-scientist a glimpse behind the curtain of the scientific paper production process.

Fortunately, the paper was published in an open access journal so we all could read it. I followed the trail to the first author and emailed her as to where she got the data for the map in question. She replied almost immediately and perhaps surprisingly, the trail went directly to a person we know, Todd Morgan at University of Montana! Small world. So I wrote him and he gave me a thoughtful answer.

A Presentation Problem
From Todd Morgan:
“One of the reasons why the harvest carbon data look so odd has to do with how they are displayed on these maps. Nevada is prime example of this, but it applies to several other areas in the west. The viewer sees huge geographic areas (e.g., the entire state of NV) shaded with a single “carbon loss” value, whereas most of the rest of the country is shaded on a county-by-county basis. Because of the way the TPO data (and possibly other FIA data) are stored in the database, and counties are often grouped together to prevent disclosure of individual landowner information, there are county groups or whole states with a single value. The paper briefly mentions this in the section “Timber product output data (TPO 2007).

A possible way to deal with this “combined county” grouping is to take that single value and divide it among the number of counties it includes. So, in NV, the value of each county in NV would be 1/17 of the state total. Or one could contact the source of the data (like you did) and find out that the NV value for harvest is really from two small counties in western NV and the statewide value could be split between those two counties and the rest of the counties assigned a Zero value. That would help make maps that visually make more intuitive sense – i.e., the brown shaded area of NV would be very small geographically relative to the rest of the state – like can be seen in east Texas vs. west Texas.

Another solution to this would have been tabular reporting of state level values for harvest (and possibly other disturbances) would allow comparisons to other whole states. At the state level, one would see that the volumes for NV (should) make sense compared to other whole states (i.e., NV would be a very small value compared to most other states). A third way to make the maps make more visual sense would be to map Tg C per acre per year – in other words scaling the data by the acres of timberland – like was done in Figure 1 of the paper. Again, that would probably reveal amounts of carbon that “make more sense” so instead of comparing the total carbon for the State of NV to each individual county in the rest of the country, one could compare the carbon per acre in NV to the carbon per acre in other locations.”

Still, something very odd seems to have been happening in Coconino, Arizona (and neighboring counties?) during this time period.

Next post: Potential Pitfalls of Combining Datasets

Why We Disagree About Forest Carbon. II. Objectivity, Peer Review, Predictions and Funding Sources

by

In the quest to understand Why People Disagree About Forest Carbon (if last year summer was the Summer of Fuel Treatments, this summer can be the Summer of Forest Carbon), I’ve been thinking that this would be a great topic to explore from two angles. The first is what scientists have to say about it, and why they disagree. But what I’ve found out by exploring around the edges is that it can also be a case study in the sociology of science. Looking at what’s out there can help people understand how scientific research is produced and used in policy or management. This includes the unimaginably complicated and preternaturally vitriolic field of climate science/policy.

Fergus McLean, in a previous comment here, raised four points that can add to this discussion. First, he mentioned that Law’s paper is objective and peer-reviewed.

1. Peer review.
Without reviewing the literature on peer review in detail, I can only say here that if people think something is important, it’s not usually a volunteer activity in the sense that people aren’t paid to do it. Let’s use an analogy. We think wildland firefighting is important. We pay people to do it. We even pay them overtime and hazard pay. Doing peer review well is time-consuming and, if done well, may have hazards like falling behind in what you are being paid to do, and critical reviews, if not completely blind, have risks of irritating your colleagues. They can retaliate in a variety of unpleasant ways. People who are close enough to the topic are probably not objective (is Professor A objective when Professor C models the same phenomenon as she does, using different techniques and coming to different conclusions?). People who are far away enough to be objective usually don’t know the topic as well. We are asking for a quality product with incentives that run against the basic principles of human nature. These problems are well known in the literature, and a variety of tweaks to the process have been proposed (e.g. this Lancet paper has quite a round-up of approaches).

2. Objectivity
But don’t believe me about objectivity here’s an article in the Stanford Encyclopedia of Philosophy. Through the dense fog of academic philosophy lingo, we can see the vague outlines of observations within our own experiences.

3. Handling uncertainty
When Fergus said “Choosing to regard existing export arrangements as permanent is a political, rather than a scientific judgement, and deserving of in-depth and critical analysis.” I agree but I would tend to take a different approach. I would imagine that I had crossed disciplinary thresholds into the world of economics.
Economists have a long history of making projections about imports, exports, demand, supply, prices, and so on. Even now, I suspect there are cadres somewhere (in the US and Canada) working on today’s installment of the Softwood Lumber Agreement. One of the tools economists use is “sensitivity analysis”. When you make many assumptions about things that are unpredictable, this tool helps you figure out which ones are really meaningful to the end you are studying. Let me restate, rather than arguing about which predictions will be more accurate based on whatever criteria (after all, how good have we really been about predicting the future?), they include a range of for each unknown and it helps them understand which assumptions are more important to the outcome. I think that this is important to consider, because we rely on models so much these days, and some disciplines use sensitivity analysis much more than others.

4.How much research on carbon does forest industry fund? Does funding source matter?
“Another relevant point that can be drawn from the Atkins article is the much greater scale of resources backing research oriented to the industry point of view about forest carbon (including TreeSource itself) in contrast to Law who, since Harmon’s recent retirement, essentially works alone. “
The idea that industry backs more carbon research than NSF, the Forest Service, EPA and so on is interesting, and possible to investigate. I’m not much of an industry person but I believe there was some corporate restructuring at some point that led to them pretty much getting out of the research business. Many of my industry colleagues lost their jobs. Has that changed?

Ranchers intimidate science they don’t like

by

Data source: “Cattle Death Loss,” a report by the USDA National Agricultural Statistics Service

A wolf researcher at Washington State University has resigned as part of a settlement of a case alleging that the university infringed on his academic freedom.

“Wielgus angered ranchers with his research of wolf behavior. He concluded the state’s policy of killing wolves that preyed on cattle was likely to increase cattle predation because it destabilized the structure of wolf packs.

Ranchers complained to the Washington State Legislature, which cut Wielgus’ funding and demanded he be removed as principal investigator on his ongoing work.”

And they got what they wanted.  So, if you’ve got enough money and political power, not only can you buy your own researchers, but you can silence publicly funded independent research.  Do you suppose they might be able to influence the research conclusions, too?  (Somehow it’s a little hard to see “powerful” environmental groups making this trick work for them.)

 

 

 

STS Community: Would Like to Hear What You Think About Proposed EPA Transparency Rule

by

The current major media stories quote people who say something like “transparency is generally good, but not here.” We all know that EPA does health studies but we also know they do much other regulating as well. Say climate, or biomass or …. So I would like to know, given that feedback, what if the EPA just used it for the non-health studies? Is there a way to review the health studies’ calculations without going to the level of personal data? The standard newspaper stories seem to be “good people hate this proposed rule, only bad people (“climate deniers,” industry) support it.” That is really not all that helpful in terms of a news story for us folks that are trying to understand the pros and cons.
Here’s one story from the Scientific American:

Smith said.. Many in the scientific community agree that increased access to data is essential for reproducibility and objective analysis,” he said. “Open access to scientific data fosters good policymaking. The American people have a right to understand how and why regulatory decisions are made.”

In a House office building last week, Smith feted a group of researchers from the National Association of Scholars who routinely attack climate science and who say in a new report that there is a “crisis” in science because too much of it cannot be reproduced. The authors of its new report, titled “The Irreproducibility Crisis of Modern Science,” say government agencies should establish review commissions to determine which existing regulations are based on reproducible research and to rescind those that are not, a process that could affect key provisions of the Clean Air Act, among other regulations.

For those of you who don’t follow science biz studies (the social science of observing the science biz), here’s a link to a piece by Andrea Saltelli on “Science’s credibility crisis” that gives some background of some of the issues across disciplines. This paper has many interesting links.

Ravetz emphasises the loss of this essential ethical element. In later works he notes that the new social and ethical conditions of science are reflected in a set of “emerging contradictions”. These concern the cognitive dissonance between the official image of science as enlightened, egalitarian, protective and virtuous, against the current realities of scientific dogmatism, elitism and corruption; of science serving corporate interests and practices; of science used as an ersatz religion.

Echoes of Ravetz’s analysis can be found in many recent works, such as on the commodification of science, or on the present problems with trust in expertise.

Ioannidis and co-authors are careful to stress the importance of a multidisciplinary approach, as both troubles and solutions may spill over from one discipline to the other. This would perhaps be a call to the arms for social scientists in general – and for those who study science itself – to tackle the crisis as a priority.

Here we clash with another of science’s contradictions: at this point in time, to study science as a scholar would mean to criticise its mainstream image and role. We do not see this happening any time soon. Because of the scars of “science wars” – whose spectre is periodically resuscitated – social scientists are wary of being seen as attacking science, or worse helping US President Donald Trump.

I think this would be a good time for social scientists who study science (the science and technology studies or STS community) to step up, dare to be labelled as denialist or Trump supporters and say “we resist the use of science as tool of partisan warfare”. Perhaps something along these lines “From our perspective, based on decades of study of the scientific processes and regulatory science in particular, we think regulatory science might be helped by open data in these situations.. but not these.. and would instead suggest …”.

“Lemons Cure Cancer” and Other Hazards of Scientific Communication

by

One of the gaps I’ve noticed in our discussions is between people who (1) think of science as a biz like any other with the resultant glories and debacles, and everything in between. Other folks (2) tend to have a higher view of how the science biz works and what it produces in general. To our mutual great misfortune, this has become somewhat partisanized (at least by those who want to partisanize it). So one of the things I would like to work on is exploring this gap.

This recent Scientific American piece is of interest because it touches on topics we recently discussed vis a vis fuel treatment and research papers. I agree with the author, Katherine Wu, even though our backgrounds and experience couldn’t be more different (me 40 years in the forest science biz, her three years as a grad student in health science at Harvard.) Here is a summary of quotes:

First, we often assume that the gap between scientists and the general public is about knowledge.
…
Importantly, this also means being receptive to the perspectives and concerns of the general public, rather than simply dismissing the misconceptions we hear as false. Building the relationships between scientists and non-scientists is the same as building any relationship founded on trust: open communication and accepting culpability. We can churn out all the facts we want, but none of it will do any good if no one is willing to listen. It’s time to step off our soapboxes and have conversations on level ground.

Second, we assume that there is finality in science.

Next, a great difficulty in communicating science is that it’s almost never clear-cut or final—a difficult fact to swallow for most people looking to quickly glean information from the media. All data requires interpretation, which is subject to bias, and all results are preliminary. But hypotheses and tentative conclusions don’t make for good headlines.

When I earn my Ph.D., I might be able to say, “We think we may have come across something that explains a miniscule portion of a complex pathway that might be correlated with a slightly elevated risk of contracting this disease—but our findings are pretty specific to this one population studied at this point in time under these conditions.”

Meanwhile, media headlines say, “Lemons cure cancer!”

Science can’t compete with sensationalized misinformation. To combat this, scientists can publicize their process, rather than just their results. Rebuilding rapport between scientists and non-scientists means opening new lines of communication and increasing transparency—not only about scientific discoveries, but how we arrive at them. Science is incremental and in constant flux. In highlighting the scientific method in our communication efforts, scientists can also encourage non-scientists to look at data in totality and form their own conclusions and criticisms.

and

At a minimum, scientist or non-scientist, each of us should commit to simply showing up. Without participation on both sides, communication doesn’t happen, and we can’t challenge each other to relay information effectively. Communication is a conversation, not a series of lectures. Don’t just expect that conversations will happen—take part in making them happen.

Almost what we discussed in the comments here.

You can read the rest here. Thanks, Scientific American for no paywall!

Introduction to the Landscape of Fire Sciences

by
From Olson et al.

Jon asked an excellent question in our previous discussion: “who counts as a wildfire expert?”. It seems to me that with the recent March for “Science” it’s probably a good time to dig deeper into some aspects of how the science business works in reality. So a simple question, like “who is a wildfire expert” is a great entry into looking at the landscape of scientific disciplines.

So let’s start thinking about different pieces of the wildfire puzzle. One piece is obviously suppression, as in large groups of people and equipment who manage wildfires. Well, there’s the behavioral psychology of groups, there are the modelers who work on the models wildland firefighters use, there are people who test those models through experience (but may not publish on the results themselves). There are people who make observations (the fire went to the ground in that strip). Within that bunch of sciences, I don’t know all the subfields but I’d like to hear from someone who can explain it. Here are a few from the Missoula Fire Science Laboratory:

Physical Fire ProcessesThe factors that determine fire behavior (fuel, weather, and topography) do so through the requirements for combustion (fuel, heat, and oxygen).
Fuel DynamicsResearch on fuel dynamics helps managers describe live and dead fuels that burn during wildland fires.
Smoke Emissions & DispersionScientists use field observations, satellite data, and models to describe smoke’s chemical composition, its movement within a fire’s heat plume, and its movement through the layers of the atmosphere.
Fire EcologyThis research contributes to improved conservation, restoration of burned areas, and reduction of fire hazard.
Fire & Fuel Management StrategiesHistorical patterns of wildland fire are combined with information about climate and vegetation to predict fire occurrence and vegetation patterns.
Science Synthesis & DeliveryScientific publications form the foundation for science delivery.

 

Here’s a list of interesting pubs from the Joint Fire Science Program, you can see the variety of kinds of studies.

Given these approaches to fire science, only one of which is fire ecology, we can then review the backgrounds of the researchers in the PNAS study. I have looked at them all and produced the attached document here that describes their backgrounds.  We can see that most have a background in fire ecology, but not so much in fuel and fire modeling.  Could seeing things through the lens of “ecology” affect the way their perspectives? It being a focus on “natural” processes and maintaining them? More on this in the next post.

Five Reasons Blog Posts are of Higher Scientific Quality than Journal Articles- by Daniel Lakens

by

In the press and here on this blog, folks sometimes talk about “the science says” in order to claim authority for a certain set of views. Given the not insignificant difference among fields (e.g., landscape ecology is not fire science is not medicine is not psychology) it is difficult to say very much about the science, or research biz, as a whole. Except that more funding is needed of course ;). Like journalism, or land management, there is the ideal of how science is conducted, and then there is the reality. In what contexts are scientific claims, or claims made by scientists (not the same) privileged? What do scientists choose to study and what do they not study? How do they value the findings of related fields? How do they place their findings in context and relate them to the real world? What studies are funded, by whom, and who decides? How are practitioners’ or policy makers’ views of importance or relevance, and practical knowledge, taken into account (if at all)?

Here’s an interesting piece by an experimental psychologist from the Netherlands. I excerpted each reason below. The comments are also interesting.

I think the topics he brings up are worth thinking about, and I hope we can incorporate them as we look at studies in the future.

1. Blogs have Open Data, Code, and Materials

When you want to evaluate scientific claims, you need access to the raw data, the code, and the materials. Most journals do not (yet) require authors to make their data publicly available (whenever possible).

2. Blogs have Open Peer Review

Scientific journal articles use peer review as quality control. The quality of the peer review process is as high as the quality of the peers that were involved in the review process. The peer review process was as biased as the biases of the peers that were involved in the review process. For most scientific journal articles, I can not see who reviewed a paper, or check the quality, or the presence of bias, because the reviews are not open.

3. Blogs have no Eminence Filter

Everyone can say anything they want on a blog, as long as it does not violate laws regarding freedom of speech. It is an egalitarian and democratic medium. This aligns with the norms in science. As Merton (1942) writes: “The acceptance or rejection of claims entering the lists of science is not to depend on the personal or social attributes of their protagonist; his race, nationality, religion, class, and personal qualities are as such irrelevant.” We see even Merton was a child of his times – he of course meant that his *or her* race, etcetera, is irrelevant.

4. Blogs have Better Error Correction

When I make an error in a blog post, I can go in and update it. I am pretty confident that I make approximately as many errors in my published articles as I make in my blog posts, but the latter are much easier to fix, and thus, I would consider my blogs more error-free, and of higher quality.

5. Blogs are Open Access (and might be read more).

It’s obvious that blogs are open access. This is a desirable property of high quality science. It makes the content more widely available, and I would not be surprised (but I have no data) that blog posts are *on average* read more than scientific articles because they are more accessible.

Something I would add to Lakens’ list, for fields with practitioners, like our own, is that blogs allow feedback from people in practice. If we think of a neurological study that says “at the molecular level, it appears that y is related to x so we hypothesize drug z might help” the neurologist practitioners might be able to say “yes, I had three patients that happened to be on that drug and they haven’t had symptoms, but only ones who also had condition q” possibly adding to our joint knowledge. It even goes to the patient, or in our case landowner, citizen, community member. It makes knowledge claims open to public discussion and while there are problems with this approach, if we are interested in finding out how things really work, it seems to me that open sources and discussion are the way to go.