What further guidance can be given for addressing the criteria considered in the Science Panel Review?

The science review of an application is a thorough and rigorous evaluation by experts knowledgeable in the research areas relevant to the proposed project. It is critical that the content of the Narrative and the Appendix fully address how the proposed research is responsive to the evaluation criteria. The Narrative must follow the order of the science panel evaluation criteria listed within the NOFO. Further guidance is offered here, per criterion:

Project goals, objectives, and specific aims: The goals, objectives or specific aims criterion provides a framework for the applicant to move from a statement of broader purpose (such as reducing toxic exposure) to providing precise, measurable, and/or behavioral details representing the plan to achieve these goals (such as increased time lapsed from suppression to overhaul). The applicant’s goals, objectives, or specific aims should drive the study design and project methods. The methods of measurement and analyses must follow the applicant’s goals, objectives, or specific aims and should lead to answers to the proposed question(s).

Literature review: The literature review demonstrates that the scientific team is knowledgeable about what has been studied in the past. Literature reviews typically identify gaps in knowledge or what is lacking or needed to advance the science to the next level. This expertise suggests the ability to do applied research in the proposed area. Importantly, the literature review makes evident how the proposed research will contribute to the existing body of knowledge in this area. Without a strong literature review, the relevance of this project will be less evident.

Project methods: Project methods describe the overall approach to the study. The methods will identify the study design, such as a randomized control trial or a large-scale fire test series design. Related information, such as how to select participants or structures, contents, and firefighting methods utilized, respectively, should be described. It is important to justify the scientific and technical rigor of the research and merit of the project. Plans to recruit and retain human subjects should be described, where applicable. Where human subjects are involved, describe plans for submission to the IRB.

Project measurements: Project measurements include the more macro and micro aspects of the research. The applicant may specify established instruments to collect data and implement the project. The applicant should discuss the data to be collected to evaluate the performance methods, technologies, and products proposed. The applicant should demonstrate the measurement methods and equipment selected for use are appropriate and sufficient to successfully deliver the proposed project objectives. A table format may be useful to handle extensive lists of measurements and portray timing. Study flow diagrams and timelines can be helpful. Sufficient information is needed to support the feasibility of the project.

Project analysis: Project analysis may include the statistical plan for determining achievement of the project aims. Quantitative and qualitative statistical methods as well as power analyses, as appropriate to the study design, are typically expected. Specify within the analysis plan what data will be analyzed, the statistical methods that will be used, the sequence of steps, and interactions, as appropriate. For example, where fire departments rather than individual firefighters are the unit of analysis then sampling, measurement strategies, and the analyses would reflect that study design. The analysis plan should specify what constitutes significant or successful results.

Dissemination and implementation: The dissemination and implementation criterion requires applicants to indicate how the results of the proposed project (if successful) would be disseminated to the fire service and scientific communities. Not only does this criterion cover dissemination of results, but also how the results might move to implementation, that is, moving from research to practice. The applicant should show how the issues for dissemination and implementation have been taken into consideration throughout the project and in development of their project product. For example, a product made available online would improve access and adoption. Also, providing the research result to relevant firefighter standards organizations to promote national adoption is a proven means to implement successful research results. The development of dissemination and implementation plans does not intend to suggest that the applicant would be responsible for all the subsequent activities.