2022 Exploration: Merit Indicators for the Review Process
Using the matrices
For each criterion, the reviewer is asked to consider relevant elements, which may include a few, several or all of the elements as outlined in the matrices, as well as some not listed. The matrices are intended to be used as a guide. A proposal does not have to be rated Exceptional against all elements to receive an Exceptional rating for the criterion overall. It is left to the discretion of the reviewer to balance assessments of individual elements to provide an overall rating per criterion.
For the High Risk, High Reward and Feasibility evaluation criteria, the matrices indicate four ratings: Exceptional, Very Good, Fair and Poor. However, members can use a seven-point scale in their assessments, selecting ratings that fall between the four described. The seven-point rating scale for the High Risk, High Reward and Feasibility evaluation criteria is as follows:
- Exceptional (described)
- Excellent (between Exceptional and Very Good)
- Very Good (described)
- Good (between Very Good and Fair)
- Fair (described)
- Inferior (between Fair and Poor)
- Poor (described)
Criterion: Interdisciplinarity
Whether a project meets the New Frontiers in Research Fund (NFRF) program’s expectations for interdisciplinarity can be defined by elements, such as the following. Note that a pass for this criterion does not require the application to pass all elements.
| Pass | Fail | |
|---|---|---|
| Novelty of perspective | Pushes the boundaries in terms of interdisciplinarity, integrating two or more disciplines that are not commonly combined. | Proposes an interdisciplinary approach where there is a long tradition and/or established co-operation/collaboration/interaction between the disciplines. The project is multidisciplinary. It involves more than one discipline but there is a lack of integration between the different disciplinary perspectives or approaches. |
| Novelty of approach | Proposes the application or adaptation of frameworks/tools/methods/techniques from one discipline to solve a problem in another discipline. (This can also apply to projects where there is a history of collaboration between the disciplines.) | The proposed frameworks/tools/methods/techniques are already in use in, or easily applied to, the second disciplinary area, requiring little adaptation or development. |
| Project design | Designed from an interdisciplinary perspective. | The project is an interdisciplinary component “added on” to a more conventional project or program of research. The project is designed from a multidisciplinary perspective, where work in several disciplines will be conducted separately rather than through an integrated approach. |
| Other | - | The application did not adequately establish the interdisciplinary nature of the project. |
All applications are assessed for interdisciplinarity and must receive a pass to be considered for funding.
Interdisciplinarity subcriterion: Fit to Program
Fit to Program is a subcriterion of Interdisciplinarity.
| Pass | Fail | |
|---|---|---|
| Fit to Program | Proposes a project that pushes the boundaries of what can be funded through the agencies, according to their mandates and existing suites of programs, as a result of its high-risk nature and/or interdisciplinary approach. | The scope of the proposed project (subject and approach) fits within the parameters of the mandate and existing suite of programs of one or more of the agencies. |
All applications are assessed for Fit to Program and must receive a pass to be considered for funding.
Criterion: Equity, Diversity and Inclusion in Research Practice (referred to as EDI in previous Exploration competitions)
To meet the NFRF program’s expectations for equity, diversity and inclusion in research practice (EDI-RP), projects must pass each of the following elements.
| Pass | Fail | |
|---|---|---|
| Analysis of context | Shows understanding of EDI-RP considerations / systemic barriers in the context of the research team. Provides a clear explanation of the team’s specific challenges/opportunities related to EDI-RP. Cites concrete and specific examples in the analysis. Demonstrates a strong commitment to EDI-RP overall. |
Fails to demonstrate an understanding of EDI-RP considerations / systemic barriers in the context of the research team. Analysis of context is generic and/or does not point to one or more systemic barriers. Evidence of commitment to EDI-RP overall is lacking. |
| Concrete practice for each area | Lists at least one concrete practice that targets the specific context listed for each area. | Fails to list a concrete practice for one or more of the areas, or the concrete practices listed are not related to the context that was described. Challenges are not discussed. |
| Implementation | Provides a description of how the concrete practice has been or will be realistically implemented. Considers implementation challenges. |
Provides no or an unclear description of how the concrete practice will be implemented. The implementation plan is unrealistic. |
| Impact | Explains how the concrete practice will impact EDI and how it will be measured. | Gives no indication of how the impact will be measured. Fails to explain the anticipated impact of the concrete practice on EDI or how it will be measured. |
Criterion: High Risk
For information on how the rating scale is used in assessments, refer to Using the Matrices.
Exploration grants are intended to fund high-risk research. Researchers are encouraged to think “outside of the box,” undertake research that defies current paradigms, and bring disciplines together in unexpected ways and from bold, innovative perspectives. It is recognized that innovation carries risk and, consequently, it is expected that a number of funded projects will fail to meet their objectives.
How high risk a project is can be defined by elements including, but not limited to, the following:
| Exceptional | Very Good | Fair | Poor | |
|---|---|---|---|---|
| Unique directions | Develops a completely new theory or paradigm. | Develops a novel concept that bridges established theories or paradigms in different fields. | Develops a novel concept closely linked to established theories or paradigms. | Represents an incremental or “logical next step” approach. |
| Challenging current paradigms | Aims to radically challenge accepted theories or paradigms. | Aims to challenge accepted theories or paradigms. | Aims to test established theories or paradigms. | Aims to reinforce established theories or paradigms. |
| Enhancing our understanding | Aims to extraordinarily enhance our understanding of a complex and challenging issue and/or significantly enhance our understanding of multiple complex and challenging issues. | Aims to significantly enhance our understanding of a complex and challenging issue and/or notably enhance our understanding of multiple complex and challenging issues. | Aims to notably enhance our understanding of a complex and challenging issue. | Aims to incrementally advance our understanding of a complex and challenging issue. |
| Novel interdisciplinary approaches | Is at the interface between disciplines, requiring a novel interdisciplinary approach (i.e., two or more disciplines that are not commonly combined). Goes beyond established approaches of any single discipline, bringing together disparate disciplines in new ways. |
Crosses disciplinary boundaries and integrates approaches from two or more disciplines. | Crosses disciplinary boundaries using approaches from one or more disciplines. | Crosses disciplinary boundaries, involving two (or more) disciplines that are closely related or commonly crossed. (The interdisciplinary approach is established.) |
| Development or adaptation of methods and techniques | Will develop novel methods or techniques. | Will adapt existing methods or techniques to a new field. | Will apply proven methods and techniques in a new context. | Will use proven methods or techniques in an established context or field. |
| Other | - | - | - | The application does not adequately establish the high-risk nature of the project. |
Criterion: High Reward
For information on how the rating scale is used in assessments, refer to Using the Matrices.
Projects that are high reward are those with the potential for outcomes that can be defined by elements including, but not limited to, the below.
For the “broad impact” and “reach” elements, projects that take place in a single or limited number of geographic communities; with a particular subpopulation or subpopulations; or that focus on a rare disease, can receive Very Good or Exceptional ratings when the potential impact on those affected is significantly high, or if the project team identifies potential applicability/transferability of research results to other contexts.
Examples include research undertaken with a single Indigenous community on entrepreneurship and health; with rural, coastal communities on climate change and governance; or with individuals who have a rare chronic illness. In these cases, the size of the community or communities or populations impacted may be small, but the extent of the potential impact is great. As such, this kind of project could be rated favourably by reviewers under the broad impact section. Reach could also be considered Exceptional or Very Good if the proposal clearly articulates how these findings may be applicable to other contexts.
| Exceptional | Very Good | Fair | Poor | |
|---|---|---|---|---|
| If successful, the project is likely to result in: | ||||
| Broad impact | Significant economic, scientific, artistic, cultural, social, technological, health or environmental impact. | Notable economic, scientific, artistic, cultural, social, technological, health or environmental impact. | Minor economic, scientific, artistic, cultural, social, technological, health or environmental impact. | No economic, scientific, artistic, cultural, social, technological, health or environmental impact. |
| Reach | Strong impact on a single or small number of unique communities or subpopulations, with lessons for other contexts, or strong impact on large or multiple communities. | Impact on a single or small number or unique communities or subpopulations, with lessons for others, or impact on large or multiple communities. | Limited impact on a single or small number of unique communities or subpopulations, with either limited or no lessons for others, or limited impact on multiple communities. | No meaningful impact on any unique communities, with limited or no lessons for others, and no potential for impact on multiple communities. |
Significantly impacts numerous fields or applications. Developed techniques/methodology will improve research in all integrated disciplines. |
Impacts numerous fields or applications. Developed techniques/methodology will improve research in several integrated disciplines. |
Impacts primarily one field or application. Developed techniques/methodology will improve research primarily in one discipline. |
Has limited impact on a field or application. | |
| Impact on research or the research community | Resolves a long-standing issue, debate or critical question or questions. | Contributes to resolving a long-standing issue, debate or critical question or questions. | May contribute to resolving a long-standing issue or debate. | Unlikely to affect a long-standing issue or debate. |
| Opens a new area of discovery or changes the direction of thought in a discipline or disciplines. | Identifies a new area for discovery or challenges the direction of thought in a discipline or disciplines. | May identify a new area for discovery or a direction to pursue to challenge the direction of thought in a discipline or disciplines. | Unlikely to identify a new area for discovery or challenge the direction of thought in a discipline. | |
| Could lead to ground-breaking advancements in the area and/or significant advancements in current knowledge, methods and/or technologies. | Could lead to significant advancements in the area and/or advancements in current knowledge, methods and/or technologies. | Is less likely to lead to advancements in the area and/or advancements of current knowledge, methods and/or technologies. | Unlikely to lead to significant advancements in the area or in current knowledge, methods and/or technologies. | |
| Other | - | - | - | The application did not adequately explain the value of the potential outcomes of the project. |
Criterion: Feasibility
For information on how the rating scale is used in assessments, refer to Using the Matrices.
While a focus on high risk may seem at odds with feasibility, risk must be related to the idea being proposed, and not to a lack of a concrete plan or inability to execute the activities.
A project’s feasibility can be defined by elements including, but not limited to, the following:
| Exceptional | Very Good | Fair | Poor | |
|---|---|---|---|---|
| Objectives | The proposed research project is clearly presented and its objectives are clearly defined. | The proposed research project is presented adequately and its objectives are sufficiently described. | The proposed research project lacks clarity. Objectives are minimally described. | The proposed research project, as presented, lacks clarity. Objectives are not clearly described and/or there are concerns about the likelihood of being able to achieve them. |
| Building on current knowledge or prior art | The application demonstrates that the research team is aware of current and relevant research and prior art or knowledge. The proposed research may challenge paradigms, but is built on sound principles. |
The application demonstrates that the research team is aware of most current and relevant research, and prior art or knowledge. Knowledge of some developments might be lacking, but this does not impact the feasibility of the proposed research. |
The application demonstrates that the research team lacks awareness of relevant research or prior art or knowledge in one aspect/discipline related to the project. | The proposed project does not seem to take into account current and relevant research and prior art or knowledge (for example, proposing approaches that have been tested and failed). |
| Work plan | The proposed work plan, including the methodological approach, is well described, reasonable and likely to be achievable within the proposed time frame. | The proposed work plan, including the methodological approach, is described, reasonable and likely to be mostly achievable within the proposed time frame. | The proposed work plan is reasonable. The methodological approach is lacking detail. The project objectives might be met within the proposed time frame. | The proposed work plan is not reasonable/feasible. The methodological approach is missing or flawed. It is unlikely that the project objectives will be met within the proposed time frame. |
| Research team | The application clearly demonstrates that the research team has the required expertise in all relevant disciplines to meet the objectives. | The application demonstrates that the research team likely has the required expertise in all relevant disciplines to help meet the objectives. | The application demonstrates that the research team has most of the required expertise to meet the objectives, although some aspects may be missing or insufficiently described. | The application does not clearly demonstrate that the research team has all the required expertise to meet the objectives. |
| Resources | The research team has acquired or has concrete plans to acquire the necessary resources to complete the work. All aspects have been described. | The research team has acquired or has concrete plans to acquire the necessary resources to complete the work. Some aspects have not been well described. | The application demonstrates that the research team has acquired or has concrete plans to acquire most of the resources to complete the work. Some aspects may be missing or insufficiently described. | The application does not clearly demonstrate that the research team has acquired or has concrete plans to acquire the necessary resources to complete the work. |
| Equity, diversity and inclusion in research design (referred to as GBA+ in previous Exploration competitions) | EDI-RD considerations have been integrated into the methodological approach (if applicable). The impact on the methodological approach and/or design has been clearly described. | EDI-RD considerations have been integrated into the methodological approach (if applicable). The impact on the methodological approach and/or design has been described. | EDI-RD considerations have been integrated (if applicable). The impact on the methodological approach or design has not been described. | EDI-RD considerations apply to the project, although the applicant indicated that they do not. EDI-RD considerations have not been integrated into methodological approach or design. |
Indigenous research* Refer to the SSHRC Guidelines for the Merit Review of Indigenous Research |
Co-creation, co-leadership and co-ownership with First Nations, Inuit and/or Métis Peoples are clearly integrated in the project’s design. The methodological approach and/or theoretical framework successfully incorporate(s) key considerations of SSHRC’s Merit Review of Indigenous Research. Attention to equitable processes and procedures for fair and respectful inclusion of Indigenous communities and their perspectives is evident. | Active engagement and reciprocity with First Nations, Inuit and/or Métis Peoples are present and clearly described. Key considerations of SSHRC’s Merit Review of Indigenous Research have been incorporated into the methodological approach and/ or theoretical framework of the project. | There is modest engagement and reciprocity with First Nations, Inuit and/or Métis Peoples present for Indigenous research or described. There is minimal incorporation of SSHRC’s key considerations for the Merit Review of Indigenous Research. | Engagement and reciprocity with First Nations, Inuit and/or Métis Peoples appear to be lacking or have not been described. Key considerations of SSHRC’s Merit Review of Indigenous Research have not been incorporated into the methodological approach and/or theoretical framework of the project. |
* The overall rating for the Feasibility criterion cannot be higher than the rating of the element in the matrix for EDI-RD and/or Indigenous research. For example, if a reviewer considers most of the elements to be Very Good, but Poor as it relates to either Indigenous research or EDI-RD, then the overall rating for Feasibility cannot be higher than Poor.
- Date modified: