Merit Indicators for the Review of Applications to NFRF Exploration Competitions

Using the Matrices

For each criterion, the reviewer is asked to consider relevant elements, which may include a few, several or all of the elements as outlined in the matrices, as well as some not listed. The matrices are intended to be used as a guide. A proposal does not have to be rated High against all elements to receive a high rating for the criterion overall. For example, when considered under the High Reward criterion, a project might have the potential to significantly improve the health of individuals suffering from a rare disease. The size of the community impacted would therefore be small, but the extent of the impact would be great. Even though the small size of the community is under the Low heading, the overall rating for High Reward could be High if, on balance, the reviewer thinks the potential impact is significant.

Criterion: Interdisciplinarity

Projects that meet the NFRF program’s expectations for interdisciplinarity can be defined by elements including but not limited to the following.

	Pass	Fail
Novelty of perspective	Pushes the boundaries in terms of interdisciplinarity, integrating two or more disciplines that are not commonly combined.	Proposes an interdisciplinary approach where there is a long tradition and/or established co-operation/collaboration/interaction between the disciplines.
Novelty of approach	Proposes the application or adaptation of tools/methods/techniques from one discipline to solve a problem in another discipline. (This may also apply for projects where there is a history of collaboration between the disciplines.)	The proposed tools/methods/techniques are already in use in or easily applied to the second disciplinary area, requiring little adaptation or development.
Project design	Designed from an interdisciplinary perspective.	The project is an interdisciplinary component “added on” to a more conventional project or program of research.
Other		The application did not adequately establish the interdisciplinary nature of the project.

Applications are assessed for “Interdisciplinarity” and must receive a “pass”.

Sub-criterion: Fit to Program

“Fit to program” is a sub-criterion of “Interdisciplinarity”.

	Pass	Fail
Fit to program	Proposes a project that pushes the boundaries of what can be funded through the agencies, according to their mandates and existing suite of programs, as a result of the interdisciplinary approach and high risk nature.	The scope of the proposed project (subject and approach) fits within the parameters of the mandate and existing suite of programs of one or more of the agencies.

Applications are also assessed for “Fit for program” and must receive a “pass”.

Criterion: Equity, Diversity and Inclusion

	Pass	Fail
Analysis of context	Shows understanding of EDI considerations/systemic barrier(s) in the context of the research team. Concrete and specific examples are cited in analysis. Demonstrates a strong commitment to EDI overall.	Analysis of context is generic and/or does not point to one or more systemic barriers. Evidence of commitment to EDI overall is lacking.
Concrete practice for each area	Lists at least one concrete practice that targets the specific context listed for each area.	A concrete practice is not listed for one or more of the areas, or the concrete practice(s) listed is/are not related to the context that was described.
Implementation	Provides a description of how the concrete practice has been/will be realistically implemented.	No description or unclear description is provided of how the concrete practice will be implemented. The implementation plan is unrealistic.
Impact	Explains how the concrete practice will impact EDI and how it will be measured.	Gives no indication of how the impact will be measured. Does not explain the anticipated impact of the concrete practice on EDI or how it will be measured.

Criterion: High Risk

Projects that are high risk can be defined by elements including but not limited to the following.

	High	Medium-High	Medium-Low	Low
Unique directions	Addresses a completely new theory.	Addresses a novel concept that bridges theories established in different fields.	Addresses a novel concept closely linked to established theories.	Represents an incremental or “logical next step” approach.
Challenging current paradigms	Aims to radically challenge accepted theories.	Aims to challenge accepted theories.	Will test established theories.	Aims to reinforce established theories.
Enhancing our understanding	Aims to extraordinarily enhance our understanding of a complex and challenging issue and/or significantly enhance our understanding of multiple complex and challenging issues.	Aims to significantly enhance our understanding of a complex and challenging issue and/or notably enhance our understanding of multiple complex and challenging issues.	Aims to notably enhance our understanding of a complex and challenging issue.	Aims for an incremental advancement in our understanding of a complex and challenging issue.
Novel interdisciplinary approaches	Is at the interface between disciplines, requiring a novel interdisciplinary approach (i.e., two or more disciplines that are not commonly combined). Goes beyond established approaches of any single discipline, bringing together disparate disciplines in new ways.	Crosses disciplinary boundaries and integrates approaches from two or more disciplines.	Crosses disciplinary boundaries, using approaches from one or more disciplines.	Crosses disciplinary boundaries, involving two (or more) disciplines that are closely related or commonly crossed. (The interdisciplinary approach is established.)
Development or adaptation of methods and techniques	Involves the development of novel methods or techniques.	Involves the adaptation of methods and techniques to a new field.	Involves the application of proven methods and techniques in a new context.	Involves proven methods or techniques.
Other				The application did not adequately establish the high risk nature of the project.

Criterion: High Reward

Projects that are high reward are those with the potential for outcomes that can be defined by elements including but not limited to the following.

	High	Medium-High	Medium-Low	Low
Broad impact	Significant societal, economic, technological or health impact.	Notable societal, economic, technological or health impact.	Minor societal, economic, technological or health impact.	No societal, economic, technological or health impact.
Reach	Impacts a large and diverse community or communities.	Impacts a large community.	Impacts a significant community.	Impacts a limited community.
	Significant impact on numerous fields or applications. Developed techniques/methodology will improve research in all integrated disciplines.	Impacts numerous fields or applications. Developed techniques/methodology will improve research in several integrated disciplines.	Impacts primarily one field or application. Developed techniques/methodology will improve research primarily in one discipline.	Limited impact on field or application.
Impact on research or the research community	Resolves a long-standing issue, debate or critical question(s).	Contributes to resolving a long-standing issue, debate or critical question(s).	May contribute to resolving a long-standing issue or debate.	Unlikely to affect a long-standing issue or debate.
	Opens a new area of discovery or changes the direction of thought in a discipline or disciplines	Identifies a new area for discovery or challenges the direction of thought in a discipline or disciplines.	May identify a new area for discovery or a direction to pursue to challenge the direction of thought in a discipline or disciplines.	Unlikely to identify a new area for discovery or challenge the direction of thought in a discipline.
	Will lead to ground-breaking advances in the area and/or significant advancements in current knowledge, methods and/or technologies.	Will lead to significant advancements in the area and/or advancements in current knowledge, methods and/or technologies.	May lead to significant advancements in the area and/or advancements of current knowledge, methods and/or technologies.	Unlikely to lead to significant advancements in the area or in current knowledge, methods and/or technologies.
Other				The application did not adequately explain the value of the potential outcomes of the project.

Criterion: Feasibility

Projects that are feasible can be defined by elements including but not limited to the following.

	High	Medium-High	Medium-Low	Low
Objectives	The proposed research project is clearly presented and objectives are clearly defined.	The proposed research project is presented and objectives are described.	The proposed research project lacks clarity. Objectives are described.	The proposed research project, as presented, lacks clarity. Objectives are not clearly described and/or there are concerns about the likelihood of being able to achieve them.
Building on current knowledge or prior art	The application demonstrates that the research team is aware of current and relevant research, prior art or knowledge. The proposed research may challenge paradigms but is built off sound principles.	The application demonstrates that the research team is aware of most current and relevant research, prior art or knowledge. Knowledge of some developments might be lacking, but this does not impact the feasibility of the proposed research.	The application demonstrates that the research team is lacking awareness of relevant research, prior art or knowledge in one aspect/discipline related to the project.	The proposed project does not seem to take into account current and relevant research, prior art or knowledge (e.g., proposing approaches that have been tested and failed).
Workplan	The proposed workplan, including methodological approach, is well described, reasonable and likely to be achievable within the proposed time frame.	The proposed workplan, including methodological approach, is reasonable and likely to be mostly achievable within the proposed time frame.	The proposed workplan is reasonable. The methodological approach is lacking detail. The project objectives might be met on time.	The proposed workplan is not reasonable/feasible. The methodological approach is missing or flawed. It is unlikely that the project objectives will be met on time.
Research team	The application clearly demonstrates that the research team has the required expertise in all relevant disciplines to meet objectives.	The application demonstrates that the research team likely has the required expertise in all relevant disciplines to help meet objectives.	The application demonstrates that the research team has most of the required expertise, though some aspects may be missing or insufficiently described.	The application does not clearly demonstrate that the research team has all the required expertise to complete the work.
Resources	The research team has acquired or has concrete plans to acquire the necessary resources to complete the work. All resources have been described.	The research team has acquired or has concrete plans to acquire the necessary resources to complete the work, though some aspects have not been well described.	The application demonstrates that the research team has acquired or has concrete plans to acquire most of the resources to complete the work, though some aspects may be missing or insufficiently described.	The application does not clearly demonstrate that the research team has acquired or has concrete plans to acquire the necessary resources to complete the work.
GBA+/SGBA	GBA+/SGBA has been integrated into the methodological approach (if applicable). The impact on the methodological approach and/or design has been clearly described.	GBA+/SGBA has been integrated into the methodological approach (if applicable). The impact on the methodological approach and/or design has been described.	GBA+/SGBA has been integrated (if applicable). The impact on the methodological approach or design has not been described.	GBA+/SGBA considerations apply to the project, even though the applicant indicated that they do not. They have not been integrated into methodological approach or design.
Indigenous research	Active engagement and reciprocity with First Nations, Inuit and Métis are present and clearly described.	Engagement and reciprocity with First Nations, Inuit and Métis are present for Indigenous research and described.	There is minimal engagement and reciprocity with First Nations, Inuit and Métis for Indigenous research.	Engagement and reciprocity with First Nations, Inuit and Métis appear to be lacking or have not been described.