2021 Exploration: Merit Indicators for the Review Process


For each criterion, the reviewer is asked to consider relevant elements, which may include a few, several or all of the elements as outlined in the matrices, as well as some not listed. The matrices are intended to be used as a guide. A proposal does not have to be rated Exceptional against all elements to receive an Exceptional rating for the criterion overall. It is left to the discretion of the reviewer to balance assessments of individual elements to provide an overall rating per criterion.

For the High Risk, High Reward and Feasibility evaluation criteria, the matrices indicate four ratings: Exceptional, Very Good, Fair and Poor. However, members can use a seven-point scale in their assessments, selecting ratings that fall between the four described. The seven-point rating scale for the High Risk, High Reward and Feasibility evaluation criteria is as follows:

  • Exceptional (described)
  • Excellent (between Exceptional and Very Good)
  • Very Good (described)
  • Good (between Very Good and Fair)
  • Fair (described)
  • Inferior (between Fair and Poor)
  • Poor (described)

Whether a project meets the New Frontiers in Research Fund (NFRF) program’s expectations for interdisciplinarity can be defined by elements including, but not limited to, the following. Please note that a pass for this criterion does not require the application to pass all elements.

Pass Fail
Novelty of perspective Pushes the boundaries in terms of interdisciplinarity, integrating two or more disciplines that are not commonly combined.

Proposes an interdisciplinary approach where there is a long tradition and/or established co-operation/collaboration/interaction between the disciplines.

The project is multidisciplinary. It involves more than one discipline but there is a lack of integration between the different disciplinary perspectives or approaches.

Novelty of approach Proposes the application or adaptation of frameworks/tools/methods/techniques from one discipline to solve a problem in another discipline. (This can also apply to projects where there is a history of collaboration between the disciplines.) The proposed frameworks/tools/methods/techniques are already in use in, or easily applied to, the second disciplinary area, requiring little adaptation or development.
Project design Designed from an interdisciplinary perspective.

The project is an interdisciplinary component “added on” to a more conventional project or program of research.

The project is designed from a multidisciplinary perspective, where work in several disciplines will be conducted separately rather than through an integrated approach.

Other - The application did not adequately establish the interdisciplinary nature of the project.

All applications are assessed for interdisciplinarity and must receive a pass to be considered for funding.

Interdisciplinarity subcriterion: Fit to Program

Fit to Program is a subcriterion of Interdisciplinarity.

Pass Fail
Fit to Program Proposes a project that pushes the boundaries of what can be funded through the agencies, according to their mandates and existing suites of programs, as a result of its high-risk nature and/or interdisciplinary approach. The scope of the proposed project (subject and approach) fits within the parameters of the mandate and existing suite of programs of one or more of the agencies.

All applications are assessed for Fit to Program and must receive a pass to be considered for funding.

To meet the NFRF program’s expectations for equity, diversity and inclusion (EDI), projects must pass each of the following elements.

Pass Fail
Analysis of context

Clearly demonstrates understanding of EDI considerations/systemic barriers within the context of the research team.

Provides a clear explanation of the team’s specific challenges/opportunities related to EDI.

Cites examples in the analysis.

Demonstrates a strong, broad-based commitment to EDI.

Fails to demonstrate an understanding of EDI considerations/systemic barriers within the context of the research team.

Provides an analysis of context that is generic and/or not aligned with best practice and/or that does not point to one or more systemic barriers.

Lacks evidence of a commitment to and understanding of EDI overall.

Concrete practice for each area Clearly identifies, at minimum, one concrete practice specific to the context of the research team for each area.

Does not provide a concrete practice for one or multiple areas, and/or provides concrete practices irrelevant to the context of the research team.

Challenges are not discussed.

Implementation

Provides a clear and realistic explanation of how the concrete practice(s) has been/will be implemented.

Considers implementation challenges.

Lacks an implementation plan or provides an unclear description of the implementation plan.

Provides an unrealistic implementation plan.

Impact Explains how the concrete practice(s) will impact EDI and describes a suitable methodology for measuring success, including specific evaluation criteria. Does not explain the anticipated impacts that the concrete practice will have on EDI, nor any method for measuring its success.

For information on how the rating scale is used in assessments, refer to Using the Matrices.

How high risk a project is can be defined by elements including, but not limited to, the following:

Exceptional Very Good Fair Poor
Unique directions Develops a completely new theory or paradigms. Develops a novel concept that bridges established theories in different fields. Develops a novel concept closely linked to established theories. Represents an incremental or “logical next step” approach.
Challenging current paradigms Aims to radically challenge accepted theories or paradigms. Aims to challenge accepted theories or paradigms. Aims to test established theories or paradigms. Aims to reinforce established theories or paradigms.
Enhancing our understanding Aims to extraordinarily enhance our understanding of a complex and challenging issue and/or significantly enhance our understanding of multiple complex and challenging issues. Aims to significantly enhance our understanding of a complex and challenging issue and/or notably enhance our understanding of multiple complex and challenging issues. Aims to notably enhance our understanding of a complex and challenging issue. Aims to incrementally advance our understanding of a complex and challenging issue.
Novel interdisciplinary approaches

Is at the interface between disciplines, requiring a novel interdisciplinary approach (i.e., two or more disciplines that are not commonly combined).

Goes beyond established approaches of any single discipline, bringing together disparate disciplines in new ways.

Crosses disciplinary boundaries and integrates approaches from two or more disciplines. Crosses disciplinary boundaries using approaches from one or more disciplines. Crosses disciplinary boundaries, involving two (or more) disciplines that are closely related or commonly crossed. (The interdisciplinary approach is established.)
Development or adaptation of methods and techniques Will develop novel methods or techniques. Will adapt existing methods or techniques to a new field. Will apply proven methods and techniques in a new context. Will use proven methods or techniques.
Other - - - The application does not adequately establish the high-risk nature of the project.

For information on how the rating scale is used in assessments, refer to Using the Matrices.

Projects that are high reward are those with the potential for outcomes that can be defined by elements including, but not limited to, the below.

For the “broad impact” and “reach” elements, projects that take place in a single or limited number of geographic communities; with a particular subpopulation or subpopulations; or that focus on a rare disease, can receive Very Good or Exceptional ratings when the potential impact on those affected is significantly high, or if the project team identifies potential applicability/transferability of research results to other contexts.

Examples include research undertaken with a single Indigenous community on entrepreneurship and health; with rural, coastal communities on climate change and governance; or with individuals who have a rare chronic illness. In these cases, the size of the community or communities or populations impacted may be small, but the extent of the potential impact is great. As such, this kind of project could be rated favourably by reviewers under the broad impact section. Reach could also be considered Exceptional or Very Good if the proposal clearly articulates how these findings may be applicable to other contexts.

Exceptional Very Good Fair Poor
If successful, the project is likely to result in:
Broad impact Significant economic, scientific, artistic, cultural, social, technological or health impact. Notable economic, scientific, artistic, cultural, social, technological or health impact. Minor economic, scientific, artistic, cultural, social, technological or health impact. No economic, scientific, artistic, cultural, social, technological or health impact.
Reach Strong impact on a single or small number of unique communities or subpopulations, with lessons for other contexts, or strong impact on large or multiple communities. Impact on a single or small number or unique communities or subpopulations, with lessons for others, or impact on large or multiple communities. Limited impact on a single or small number of unique communities or subpopulations, with either limited or no lessons for others, or limited impact on multiple communities. No meaningful impact on any unique communities, with limited or no lessons for others, and no impact on multiple communities.

Significantly impacts numerous fields or applications.

Developed techniques/methodology will improve research in all integrated disciplines.

Impacts numerous fields or applications.

Developed techniques/methodology will improve research in several integrated disciplines.

Impacts primarily one field or application.

Developed techniques/methodology will improve research primarily in one discipline.

Has limited impact on a field or application.
Impact on research or the research community Resolves a long-standing issue, debate or critical question or questions. Contributes to resolving a long-standing issue, debate or critical question or questions. May contribute to resolving a long-standing issue or debate. Unlikely to affect a long-standing issue or debate.
Opens a new area of discovery or changes the direction of thought in a discipline or disciplines. Identifies a new area for discovery or challenges the direction of thought in a discipline or disciplines. May identify a new area for discovery or a direction to pursue to challenge the direction of thought in a discipline or disciplines. Unlikely to identify a new area for discovery or challenge the direction of thought in a discipline.
Will lead to ground-breaking advancements in the area and/or significant advancements in current knowledge, methods and/or technologies. Will lead to significant advancements in the area and/or advancements in current knowledge, methods and/or technologies. May lead to significant advancements in the area and/or advancements of current knowledge, methods and/or technologies. Unlikely to lead to significant advancements in the area or in current knowledge, methods and/or technologies.
Other - - - The application did not adequately explain the value of the potential outcomes of the project.

For information on how the rating scale is used in assessments, refer to Using the Matrices.

A project’s feasibility can be defined by elements including, but not limited to, the following:

Exceptional Very Good Fair Poor
Objectives The proposed research project is clearly presented and its objectives are clearly defined. The proposed research project is presented adequately and its objectives are sufficiently described. The proposed research project lacks clarity. Objectives are minimally described. The proposed research project, as presented, lacks clarity. Objectives are not clearly described and/or there are concerns about the likelihood of being able to achieve them.
Building on current knowledge or prior art

The application demonstrates that the research team is aware of current and relevant research and prior art or knowledge.

The proposed research may challenge paradigms, but is built on sound principles.

The application demonstrates that the research team is aware of most current and relevant research, and prior art or knowledge.

Knowledge of some developments might be lacking, but this does not impact the feasibility of the proposed research.

The application demonstrates that the research team lacks awareness of relevant research or prior art or knowledge in one aspect/discipline related to the project. The proposed project does not seem to take into account current and relevant research and prior art or knowledge (for example, proposing approaches that have been tested and failed).
Work plan The proposed work plan, including the methodological approach, is well described, reasonable and likely to be achievable within the proposed time frame. The proposed work plan, including the methodological approach, is described, reasonable and likely to be mostly achievable within the proposed time frame. The proposed work plan is reasonable. The methodological approach is lacking detail. The project objectives might be met within the proposed time frame. The proposed work plan is not reasonable/feasible. The methodological approach is missing or flawed. It is unlikely that the project objectives will be met within the proposed time frame.
Research team The application clearly demonstrates that the research team has the required expertise in all relevant disciplines to meet the objectives. The application demonstrates that the research team likely has the required expertise in all relevant disciplines to help meet the objectives. The application demonstrates that the research team has most of the required expertise, though some aspects may be missing or insufficiently described. The application does not clearly demonstrate that the research team has all the required expertise to complete the work.
Resources The research team has acquired or has concrete plans to acquire the necessary resources to complete the work. All aspects have been described. The research team has acquired or has concrete plans to acquire the necessary resources to complete the work. Some aspects have not been well described. The application demonstrates that the research team has acquired or has concrete plans to acquire most of the resources to complete the work. Some aspects may be missing or insufficiently described. The application does not clearly demonstrate that the research team has acquired or has concrete plans to acquire the necessary resources to complete the work.
GBA+* GBA+ has been integrated into the methodological approach (if applicable). The impact on the methodological approach and/or design has been clearly described. GBA+ has been integrated into the methodological approach (if applicable). The impact on the methodological approach and/or design has been described. GBA+ has been integrated into the methodological approach (if applicable). The impact on the methodological approach or design has not been described. GBA+ considerations apply to the project, even though the applicant indicated that they do not. They have not been integrated into methodological approach or design.

Indigenous research*

Refer to the SSHRC Guidelines for the Merit Review of Indigenous Research

Co-creation, co-leadership and co-ownership with First Nations, Inuit and/or Métis Peoples are clearly integrated in the project’s design. The methodological approach and/or theoretical framework successfully incorporate(s) key considerations of SSHRC’s Merit Review of Indigenous Research. Attention to equitable processes and procedures for fair and respectful inclusion of Indigenous communities and their perspectives is evident. Active engagement and reciprocity with First Nations, Inuit and/or Métis Peoples are present and clearly described. Key considerations of SSHRC’s Merit Review of Indigenous Research have been incorporated into the methodological approach and/ or theoretical framework of the project. There is modest engagement and reciprocity with First Nations, Inuit and/or Métis Peoples present for Indigenous research or described. There is minimal incorporation of SSHRC’s key considerations for the Merit Review of Indigenous Research. Engagement and reciprocity with First Nations, Inuit and/or Métis Peoples appear to be lacking or have not been described. Key considerations of SSHRC’s Merit Review of Indigenous Research have not been incorporated into the methodological approach and/or theoretical framework of the project.

* The overall rating for the Feasibility criterion cannot be higher than the rating of any given element in the matrix. For example, if a reviewer considers most of the elements to be Very Good, but Poor as it relates to either Indigenous research or GBA+, then the overall rating for Feasibility cannot be higher than Poor.

Date modified: