COVID-19 Update

COVID-19: Impact on SSHRC programs, experts database and perspectives from our community.


Evaluation of the Canada First Research Excellence Fund—Management Response and Action Plan

Context

The Canada First Research Excellence Fund (CFREF) supports postsecondary institutions in their efforts to become global research leaders. The program helps Canadian institutions compete with the best in the world for talent and partnership opportunities, and to make breakthrough discoveries. With this funding, emerging opportunities can be seized and the greatest strengths of the institutions can be put forward with large scale, transformational and forward-thinking institutional strategies.

The Government of Canada’s CFREF program has thus far provided an investment of $1.25 billion to 17 leading Canadian institutions (18 grants total) through two inaugural competitions in 2015 and 2016. Together, these grants represent the largest investment by the federal government to support the implementation of scientific and institutional strategies that allow grantees to strengthen their world-class position in multidisciplinary fields of research.

The CFREF program is a tri-agency initiative of the Social Sciences and Humanities Research Council (SSHRC), the Natural Sciences and Engineering Research Council and the Canadian Institutes of Health Research (the agencies). It is administered by the Tri-agency Institutional Programs Secretariat (TIPS), which is housed within SSHRC. This is the first evaluation of the CFREF program since it was launched. The evaluation was conducted simultaneously with the midterm review of the grants from the first competition. The midterm review focused on the progress of individual grants, while the evaluation’s focus was on the program itself. The evaluation used multiple lines of evidence in order to examine the following questions:

  1. To what extent does CFREF continue to address a unique need and align with government priorities?
  2. How, and to what extent, have institutions implemented structures and processes for prioritizing funding towards research in CFREF priority research areas?
  3. To what extent have funded institutions created or strengthened partnerships, collaborations and infrastructure to enhance research capacity?
  4. To what extent has high-calibre, diverse and interdisciplinary research talent been attracted, retained and trained?
  5. To what extent is the design and delivery of CFREF effective and efficient?

Comments from Management

The CFREF program provides the government with a tool that helps competitively selected Canadian postsecondary institutions turn their key strengths into world-leading capabilities to scale institutional research and science capacity to a global level. The CFREF program is unique as a vehicle to strategically invest in priority areas of research, as a support to train the next generation of scientific leaders and leverage funding beyond the award to continue to boost research-related outcomes. This report highlights the synergies made possible by the combination of broad-based research and infrastructure funding and strategic investments that solidify and expand the research capacity at Canadian institutions.

The evaluation report concludes that the program is performing well to date and is delivering on its objectives. Management notes that the funded grants have met the immediate outcomes (years 0 to 3) by creating and strengthening partnerships and collaborations; prioritizing funding to CFREF-related research activities; and attracting and retaining high-calibre, diverse and interdisciplinary research talent. Many initiatives have a promising head-start on the intermediate outcomes (years 3 to 7), such as producing discoveries and insights; sharing discoveries and insights with knowledge users; and creating enhanced research infrastructure as well as training environments and opportunities. Nonetheless, management acknowledges that it is still early in the program to draw conclusions regarding broader long-term differential impacts as compared to investments in the core funding opportunities of the agencies at an equivalent level. It is expected that the next program evaluation in five years will provide more insight on these impacts.

An equitable, diverse and inclusive (EDI) Canadian research enterprise is essential to create excellent, innovative and impactful research. Program management is strongly committed to ensure access to its programming and to ensure that the goals and benefits of EDI are achieved in the research ecosystem. Midterm reviews and the evaluation report show promising practices, but indicate that improvements could be made with regard to EDI in research design and in the research environment. While each CFREF institution is responsible for developing and implementing its EDI plan, management acknowledges that there remains significant work to be done to increase representation of individuals from the four designated groups (women, persons with disabilities, Indigenous Peoples and members of visible minorities) within the teams and their governance structures.

This evaluation report, as well as the ongoing midterm review for the CFREF grants, aligns with the recommendation from the Fundamental Science Review to undertake an evaluation of the program by providing the agencies and the federal government with the facts and evidence on which to inform their considerations for future competitions. These discussions will continue as we address the recommendations of this evaluation report through the management response and action plan.

The following list provides brief descriptions of the recommendations in the evaluation report:

  1. Improve alignment of the CFREF program with government priorities on EDI, and support for early career researchers, by:
    1. Continuing to ensure that grantees have implemented plans related to the representation of individuals from the four designated groups and monitoring the participation of these groups;
    2. Clarifying the CFREF program’s role and expectations of grantees in supporting early career researchers, given that it is a current priority for the government.
  2. Continue to track the rate at which grants are being expended and consider no-cost extensions as required, especially as the COVID-19 pandemic may cause additional delays.
  3. Strengthen monitoring and reporting activities undertaken by grantees, in order to improve the ability to understand and assess longer term impacts, by:
    1. Reviewing the annual progress and midterm report templates to ensure that key definitions are clarified;
    2. Improving the utility of the Performance Measurement Plans for both TIPS and grantees; and
    3. Instituting an end-of-grant report, based on the current model for the mid-term report, in order to better understand and document outcomes and results achieved over the life of each grant.
  4. Further enhance communications and support to applicants and grantees by:
    1. Ensuring that comprehensive guidance is provided by TIPS to funding applicants; and
    2. Maintaining sustained communication with grantees.

This report provides the research community and Canadians with information and analysis on the data collected thus far by the program through annual reporting and midterm reviews as well as through this evaluation exercise itself, which included key interviews with institutional personnel, researchers, administrators and others. Management is pleased to see that grantees are progressing well in many areas, such as engaging participants, leveraging funds from a variety of sources and developing world-leading research capacity. The recommendations made in the evaluation report will help guide how key elements of the program, including reporting, EDI, monitoring, compliance, are managed going forward. In addition, the feedback received through the evaluation report will guide future design enhancements to the CFREF program and new competitions.

At the time the evaluation was conducted, the COVID-19 pandemic had only started. In the last months, it has become apparent that the pandemic has impacted researchers, students and administrators and has caused delays to the research activities for many CFREF grants. Management acknowledges that these challenges have a variable impact, depending on the fields of research and size or geographical location of institutions.

We are pleased to present the attached action plan, which provides specific information on the management response to the recommendations contained in the evaluation report.

Valérie Laflamme, PhD
Associate vice-president
Tri-Agency Institutional Programs Secretariat
Social Sciences and Humanities Research Council

Dominique Bérubé, PhD
Vice-president
Research
Social Sciences and Humanities Research Council

Contact Information:

Programs: Dominique Lalonde, director, Institutional Programs, TIPS

Management response: Valérie Laflamme, associate vice-president, TIPS

Evaluation: Susan Morris, director, Evaluation, SSHRC

Evaluation of the Canada First Research Excellence Fund—Management Response and Action Plan

Recommendation Agree/
Disagree
Proposed action Responsibility Target date for completing proposed action

Recommendation 1

Improve alignment of the CFREF program with government priorities on Equity, Diversity and Inclusion (EDI), and support for early career researchers (ECRs), by:

 

Agree

 

Management agrees that program improvements be made with regard to EDI to align with tri-agency and Canada Research Coordinating Committee (CRCC) priorities.

   

a) Continuing to ensure that grantees have implemented plans related to the representation of individuals from the four designated groups and monitoring the participation of these groups. If the distribution of CFREF participants does not improve on pace with program expectation, consider implementing more specific guidance or EDI targets in future competitions; and

Agree

a) The following actions will be taken:

  • Update program requirements, literature (e.g., update CFREF Administration Guide) and guidelines to better align them with EDI priorities.

Director, Institutional Programs, TIPS, and Director, Policy, Performance and Equity and Diversity, TIPS

Priority: High
Timeline: Fall 2021*
*Contingent on timelines for new competition

  • Integrate EDI as a merit review criteria and require a CFREF- specific EDI action plan for future competitions.

Priority: Medium
Timeline: Fall 2022

b) Clarifying the CFREF program’s role and expectations of grantees in supporting ECRs, given that it is a current priority for the government.

Agree

b) The following action will be taken:

  • Propose options to the program’s governance structure to better define the program’s expectations for grantees to support ECRs in future competitions. 

Associate Vice-President, TIPS

Priority: High
Timeline: Fall 2021*
*Contingent on timelines for new competition
Ongoing

Recommendation 2

Continue to track the rate at which grants are being expended and consider no-cost extensions as required, especially as the COVID-19 pandemic may cause additional delays.

 

Agree

 

Many factors contribute to the rate at which grants are being expended, including the COVID-19 pandemic.

These extensions, if required, would provide institutions with flexibility and more time to fully invest CFREF grant funds.

The following actions will be taken:

   
  • Continue to monitor financial reports annually and provide program management with an up-to-date summary each year at the conclusion of the annual reporting exercise.

Director, Institutional Programs, TIPS

Priority: Medium
Timeline: Ongoing

  • Ensure that program design and timelines align with institutional planning to support effective financial management.

Director, Institutional Programs, TIPS

Priority: Medium
Timeline: Fall 2021
*Contingent on timelines for new competition

Recommendation 3

Strengthen monitoring and reporting activities undertaken by grantees, in order to improve the ability to understand and assess longer term impacts, by:

 

Agree

 

Management agrees that improvements to program monitoring and reporting activities would better support applicants and grantees. Improvements may also enhance the program’s ability to analyze and compare data in order to inform future policy or program direction including future competitions.

The following actions will be taken:

   

a) Reviewing the annual progress and midterm report templates to ensure that key definitions are clarified, and that the same format is used for common data elements across these reporting tools in order to enhance consistency in reporting and comparability of data;

Agree

a) Explore and implement changes to annual reporting (AR) and midterm reporting (MR) templates to ensure that these templates are aligned with one another. In addition, the changes should reflect any new reporting requirements for future competitions and continue to be consistent with common data elements to enhance the program’s ability to demonstrate the impacts of the program.

Director, Institutional Programs, TIPS, and Director, Policy, Performance and Equity and Diversity, TIPS

Priority: Medium
Timeline for AR: April 2023*
(Ready for fiscal year 2022-23 reporting)
Timeline for MR: April 2026*
*Contingent on grant start dates following CFREF Competition 3 (TBC)

b) Improving the utility of the performance measurement plans for both TIPS and grantees by requiring applicants to clearly articulate what the grant is expected to achieve in the short- and long-term and how (i.e., its post-grant legacy), and to identify relevant grant-specific performance indicators based on the grant’s transformational logic, in addition to common CFREF program-level indicators; and

Agree

b) Develop an approach to address performance measurement plans with grantees. Ensure that each grantee is clear on the purpose, objectives and short- and long-term outcomes in order to sustain global research excellence post-award.

Director, Institutional Programs, TIPS, and Director, Policy, Performance and Equity and Diversity, TIPS

Priority: High
Timeline: September 2022

c) Instituting an end-of-grant report, based on the current model for the midterm report, in order to better understand and document outcomes and results achieved over the life of each grant.

Partially agree**

c) Consider the value of creating an end-of-grant report template that captures the cumulative data and narrative on the outcomes, successes and sustainability of the critical research capacity developed over the course of the CFREF award.

**Management notes that the program terms and conditions do not require an “end-of-grant” report. However, there may be value in exploring the benefits of such a reporting requirement for this program.

Director, Institutional Programs, TIPS, and Director, Policy, Performance and Equity and Diversity, TIPS

Priority: Medium
Timeline: April 2023

Recommendation 4

Further enhance communications and support to applicants and grantees by:

 

Agree

 

Management agrees that communicating and engaging on a regular and frequent basis with applicants and grantees is important. In addition, more feedback and interaction through annual and midterm reporting stages will better position all parties to work together in achieving program objectives.

The following actions will be taken:

 

Director, Institutional Programs, TIPS

 

a) Ensuring that comprehensive guidance is provided by TIPS to funding applicants, should there be a new competition;

Agree

a) Provide updated information on application requirements, deadlines and eligibility for any future competitions through its website. Engage applicants directly through webinars and key stakeholder forums (e.g., conferences).

Director, Institutional Programs, TIPS

Priority: High
Timeline: September 2021*
*Contingent on timelines for new competition

b) Maintaining sustained communication with grantees during the implementation phase of their grant.

Agree

b) Develop a stakeholder engagement plan that provides current and future grantees with a calendar of key dates, events and deliverables. The program will also engage in regular follow-ups, site visits and provide feedback to address reporting requirements and assist grantees.

Director, Institutional Programs, TIPS

Priority: Medium
Timeline: April 2023

Date modified: