1. Introduction

1.1 Project Justification

Fire emissions and smoke impacts from wildland fires are a growing concern due to increasing fire season severity, the public’s dwindling tolerance of smoke, more rigorous air quality regulation and fire’s role in climate change issues.  Unfortunately, as identified in JFSP RFA 2008-01 Task 6, while a number of model solutions are available to address these issues, a “lack of quantitative information on the limitations of smoke and emissions models impedes the use of these tools for real-world applications.” 

This proposal directly addresses both the need for rigorous, quantitative assessment of all available smoke and emissions models and the need to translate such information into usable guidance for use by decision-makers and regulators. 

These issues have been identified in several major recent reports as being critical for real-world application of smoke and emissions models.  Participants of the two JFSP Smoke Management and Air Quality Roundtables expressed that “practitioners often do not know the accuracy of models they are using” and “do not understand the gaps in available data, proper uses and limits of models, or uncertainty of results” (JFSP, 2007).  The interagency Joint Action Group (JAG) convened to produce a National Wildland Fire Weather Needs Assessment came to similar conclusions, recognizing the need to define the current state of the science and the limitations of fire emissions and smoke models, and to compile more comprehensive observation data to further their development and validation (OFCMSSR, 2007).

To address these issues, we propose to

• create an open standard for comparing smoke and emissions models both against each other and against real-world observations for use now and into the future through a model intercomparison project;
• perform rigorous evaluations of current publicly available models through a sequence of standard case studies identified by the open standard; and
• translate results into user-accessible guidance as to which models perform best under which circumstances.

This project will complement and expand on other efforts, notably JFSP Project #07-2-3-07 [PIs: Solomon and Larkin, completion date 1/2009] designed to intercompare fuel loading and consumption models.  This project differs from other efforts in that it

• creates an open and ongoing model intercomparison structure;
• compares many more types of models (not just fuel loading and consumption); and
• evaluates models against real-world observations rather than just against each other.
  

This work will directly address several problems identified by the recent JFSP Smoke Roundtables related to Compatibility of Multiple Tools (Prob. #2);  Accuracy and Application of Tools (Prob. #3); Fire and Climate Change (Prob. #4); Tracking Emissions (Prob. #7); and Measuring Impacts (Prob. #10).  The work relates to a number of solutions identified in the JFSP report, specifically:

• create a study plan for validating existing models and sub-models (Solution #3A),
• provide guidance from which appropriate models can be selected (Solution #2A),
• validate fuel consumption models with existing observations (Solution #7C),
• perform an assessment of models (Solutions #10D and #10C), and  
• identify greenhouse gas emissions and their uncertainties (Solution #4B).
  

Results of the project proposed here will directly assist decision makers such as fire managers who make go/no-go decisions and air quality regulators.  The project results will allow decision makers and regulators to directly choose the appropriate model, to understand the strengths and weaknesses of the model or model combination chosen in comparison to other options, and to understand the range of model uncertainty.

1.2 Project Objectives

This project has two major objectives:

1. Creation of an ongoing, open access Smoke and Emissions Model Intercomparison Project (SEMIP) run by a governing board of scientists and users; and
2. Completion of the first round of evaluations under SEMIP including creating user guidance tailored to specific model applications.

Both objectives will be guided by a science advisory board and overseen by a governing board created for this project consisting of scientists and users (discussed in Section VII, Science Delivery and Application).
In creating the model intercomparison project (Objective A), we will

• A1.    create an open standard for evaluating models including performance metrics and open criteria for inclusion;
• A2.    identify and lay out several standard case studies to be used by all who wish to submit their model results and gather all relevant observational data for evaluating these cases;
• A3.    develop a phased strategy for evaluating these results and producing summary guidance;
• A4.    obtain approval from the project’s advisory board and the JFSP Board for the standards, process, study plan, and proposed outputs; and
• A5.    unify and distribute all data sets necessary to allow anyone to run the standard cases using their own model and to allow others to perform their own model evaluations.

In performing the evaluations (Objective B), we will

• B1.    include a number (22) of known, publicly available models (see Objective B, Methods), as well as any other models whose results were submitted under the open standard;
• B2.    perform model-to-model intercomparisons and model-to-observation comparisons for five specific standard case scenarios identified in Objective A;
• B3.    identify, as much as possible, the periods, locations, and circumstances under which each model performs best; and
• B4.    create application-specific guidance for decision makers and regulators in the form of instructions, reports, interactive web sites, and training sessions.

The project’s successful outcome will

• provide an ongoing unified benchmark for emissions, smoke, and component model evaluations;
• provide guidance on which models to use when and their uncertainties for air quality forecasters, emissions inventory creators, fire managers, and others who rely on smoke and emissions models; and
• define areas of future research including observational campaigns and model improvements.

1.3 Background

Modeling the emissions and transport of smoke from fires is highly complex.  In practice, it requires the sequential linking (implicitly or explicitly) of several sub-process modeling steps, including fuel loading, fuel consumption, smoke emissions, plume rise, and transport/dispersion (O’Neill et al., 2007).  Within each sub-process, several models have been developed, and there is little information about the uncertainties inherent in each model or about how they compare.  Therefore, for any specific case, air quality and fire managers are often unsure which models are appropriate and how much those models should be trusted.

Worse, in each of these steps, the choice of model can have a substantial, often defining, impact on the resulting calculation (e.g., Larkin et al., 2007).  For example, Figure 1 shows the effect of different plume geometry assumptions on modeled ground-level pollutant concentrations.  Additionally, combinations of models can interact in non-obvious ways.  For these reasons, comparison and evaluation is important at a variety of levels (Figure 2).  Examination at a variety of levels also allows additional observation types to be used, creating additional real-world/model evaluations.

This type of comparison project has been performed in other fields with considerable success.  The IPCC (Intergovernmental Panel on Climate Change) 1990 and 1992 reports emphasized the need for establishing a unified process by which global climate (circulation) models (GCM) could be inter-compared, undergo a rigorous evaluation with a standard set of performance metrics, and be validated against a standard set of observational datasets (Gates, 1992, 1999).  The Atmospheric Model Intercomparison Project (AMIP) was then started with seed money to choose the standard case studies and their corresponding observation data sets, to determine the appropriate performance metrics, and to start the intercomparison of GCM models.  To this day, AMIP remains the definitive source for on-going GCM model evaluation.  The precedent set forth by AMIP led to many more model intercomparison projects, such as the unification of land-surface parameterization schemes (Henderson-Sellers et al., 1993), the paleoclimate modeling intercomparison project (Joussaume et al., 1999), and the global atmospheric tracer transport project (Gurney et al., 2002), to name a few.  Each model intercomparison project sets up standard cases for all models to run and allows any and all parties to submit results from their models for these cases.  Models are then tested against each other and against all observational data to determine both inter-model errors and model performance characteristics.

The proposed SEMIP is based on the principles of these prior efforts.  We will begin by including the component models and model chains that form a variety of smoke and emissions calculations currently in use and publicly available.  In particular, we will begin by including the underlying model components used to create a variety of emission inventories and smoke impact predictions available today, specifically:  national air quality forecasts from the National Weather Service’s (NWS) CMAQ, Missoula Fire Lab’s WRF-Chem, and STI’s CMAQ systems;  national and regional smoke forecasts from the NWS HYSPLIT and the Fire Consortia for the Advanced Modeling of Meteorology and Smoke (FCAMMS); Air Sciences, Inc.’s Smoke Impact Spreadsheet (SIS); Oklahoma State University’s OK-FIRE; Washington State University’s AIRPACT;  global air quality forecasts from NASA GEOS-Chem;  as well as emissions inventories from the EPA National Emissions Inventory, the WRAP Fire Emissions Tracking System, and NCAR. Other proprietary systems, such as that of Baron Advanced Meteorology Systems for air quality forecasting, will also be included should such organizations make their modeling results available.