PDF Version

---

List of Tables

Table 1:	SSHRC Institutional Grants Awards and Grant Expenditures, from 2002 to 2011
Table 2:	Aid to Small Universities Applications, Awards and Grant Expenditures, from 2002 to 2014
Table 3:	Evaluation Issues and Evaluation Questions
Table 4:	Evaluation Methods and Group Responsible
Table 5:	SSHRC Institutional Grants and Aid to Small Universities Expenses, as Reported by Institutions
Table 6:	Total Number of SSHRC Institutional Grant- and Aid to Small Universities-Funded Projects, by Funding Cycle
Table 7:	Funded Faculty, by Discipline, for the SSHRC Institutional Grants and Aid to Small Universities Funding Opportunities
Table 8:	Proportion of Students Funded, by Level of Study
Table 9:	Categories of Contribution for the SSHRC Institutional Grants Funding Opportunity
Table 10:	SSHRC Institutional Grants and Aid to Small Universities Program Expenditures, Administrative Expenditures, and Relative Ratios, for Fiscal Year 2008-09 to 2011-14

List of Acronyms

EQ	evaluation question
ASU	Aid to Small Universities
FAR	final activity report
GRF	general research fund
R&D	research and development
RSF	Research Support Fund
SIG	SSHRC Institutional Grants
SSHRC	Social Sciences and Humanities Research Council
SOA	statement of account
ST&I	science, technology and innovation

---

Executive Summary

Overview

This report presents the results of an evaluation of the Social Sciences and Humanities Research Council’s (SSHRC) institutional research capacity grants. It is a subprogram that falls under the Insight program in SSHRC’s program alignment architecture, and comprises two funding opportunities: SSHRC Institutional Grants (SIG), and Aid to Small Universities (ASU).

This is a routine evaluation of SSHRC’s institutional research capacity grants. The last evaluation of the SIG and ASU funding opportunities was approved in January 2011. Therefore, the current evaluation was required as per the evaluation coverage requirements stipulated in Section 42.1(1) of the Financial Administration Act^{Footnote 1} and the Treasury Board Policy on Evaluation^{Footnote 2} (2009). The evaluation was designed to address the five core evaluation issues stipulated in the Treasury Board Policy on Evaluation (2009), which fall within two broad categories: relevance and performance. The current evaluation also addressed specific issues related to the design and delivery of the funding opportunities.

Data from multiple lines of evidence were collected, analyzed and synthesized, in order to assess the relevance and performance of the funding opportunities through five evaluation questions. The responsibility for data collection was shared between evaluators from an external consulting firm, Goss Gilroy Inc., and evaluators from SSHRC’s Evaluation Division. The evaluation methods and groups responsible for data collection are presented in Table 4. These methods were mapped to the evaluation questions and indicators in the evaluation matrix (see Appendix A).

Key Findings

The main findings from the evaluation study are presented below, by evaluation question (EQ).

Relevance (EQ1): To what extent do SIG and ASU funding opportunities continue to address a demonstrable need and are responsive to the needs of Canadian institutions?

There do not appear to be other funding opportunities that provide funding to institutions to support small-scale research projects in the social sciences and humanities. The SIG and ASU funding opportunities are meeting a specific institutional need by providing flexible institutional grants to support a large number of small-scale research projects. Institutions highlighted that, in the absence of the SIG and ASU funding opportunities, they would be unable to fund the same number of small-scale research projects.

Relevance (EQ2): To what extent are SIG and ASU funding opportunities objectives consistent with the federal government priorities and SSHRC’s, and SSHRC’s strategic outcomes?

The SIG and ASU funding opportunities are aligned with the federal government’s science, technology and innovation strategy and SSHRC’s strategic outcomes. However, the fit of the SIG and ASU funding opportunities within SSHRC’s program architecture is less clear.

Relevance (EQ3): Is there a role or responsibility for the federal government in delivering SIG and ASU funding opportunities?

The role of the federal government in delivering the SIG and ASU funding opportunities is confirmed.

Performance (EQ4): What contributions have SIG and ASU funding opportunities made to the achievement of outcomes?

Institutions are using SIG and ASU funds to support a large number of small-scale research projects in a variety of social sciences and humanities disciplines at Canadian institutions. In particular, the SIG and ASU funding opportunities are supporting mostly early and mid-career researchers in the creation and the dissemination of knowledge, while contributing to the training of students and postdoctoral researchers in the social sciences and humanities. There is only limited anecdotal evidence that the SIG and ASU funding opportunities are contributing to developing, maintaining or strengthening research capacity at institutions (for SIG and ASU) and in areas of importance identified in institutions’ strategic plans (for ASU only).

Performance (EQ5): To what extent are SIG and ASU funding opportunities cost-efficient?

Both the SIG and ASU funding opportunities are being delivered in a cost-efficient manner. The SIG funding opportunity is more cost-efficient compared to the ASU funding opportunity, due to the differences in how the funding is awarded to institutions (allocation vs. merit review).

Design and Delivery (EQ6): To what extent are the SIG and ASU funding opportunities delivered as planned?

Institutions are mostly satisfied with the tools and guidelines provided by SSHRC. Institutions use varied approaches internally in order to award SIG and ASU funds to researchers; however, it appears that most institutions are not consistently promoting the funds as originating from SSHRC. The current reporting forms for SIG and ASU are not meeting SSHRC’s information needs in an efficient way.

Conclusions

Overall, the evaluation confirmed the relevance of SSHRC’s institutional research capacity grants, as they appear to be meeting a specific need by providing institutions with funding for small-scale research projects. The evaluation findings do suggest, however, that revisions to the funding opportunities’ objectives, and reconsidering their fit within SSHRC’s program architecture, would improve the relevance of SSHRC’s institutional grants. In terms of performance, the SIG and ASU funding opportunities are contributing to the immediate and intermediate outcomes of SSHRC’s Insight program.

More specifically, the evaluation findings revealed that institutions are using SIG and ASU funds to support social sciences and humanities researchers to engage in research and research-related activities, and to support students and postdoctoral researchers engaged in social sciences and humanities research training. In addition, there is evidence that the SIG and ASU funding opportunities are contributing to the Insight program’s intermediate outcome of producing and mobilizing research knowledge. However, the extent to which the SIG and ASU funding is contributing to developing or strengthening research capacity within postsecondary institutions is less evident. Finally, the evaluation findings seem to suggest that the relevance and performance of the funding opportunities could be improved by implementing several design and delivery changes, including clarifying the funding opportunities’ objectives, providing increased guidance to institutions regarding the use of the funds, and improving the format and content of the final activity reports.

Recommendations

Based on the findings and conclusions of the current evaluation, and following discussions with the Evaluation Advisory Committee and the Departmental Evaluation Committee, four recommendations are proposed for the evaluation of SSHRC’s institutional research capacity grants.

Recommendation 1: It is recommended that program management continue to monitor institutions’ use of SIG and ASU funds, with particular attention to the amounts that remain unspent after the three-year grant period.

Recommendation 2: It is recommended that program management review the objectives of SSHRC’s institutional research capacity grants (SIG and ASU), with specific emphasis on defining the current expected result (intermediate outcome) of developing, maintaining or strengthening institutional research capacity.

Recommendation 3: It is recommended that program management revise the application and end-of-grant reporting forms currently used for SSHRC’s institutional research capacity grants (SIG and ASU).

Recommendation 4: In line with SSHRC’s program architecture renewal exercise, it is recommended that program management consider whether efficiencies could be gained by streamlining and combining the budgets of its two current institutional research capacity grants funding opportunities (SIG and ASU) into one funding opportunity—SSHRC Institutional Grants.

---

1.0 Introduction

This report presents the results of an evaluation of the Social Sciences and Humanities Research Council’s (SSHRC) institutional research capacity grants. It is a subprogram that falls under the Insight program in SSHRC’s program alignment architecture, and comprises two funding opportunities: SSHRC Institutional Grants (SIG), and Aid to Small Universities (ASU). The evaluation was conducted by SSHRC’s Evaluation Division, with input from the Evaluation Advisory Committee, which consisted of program staff and management involved in the delivery of the SIG and ASU funding opportunities, and representatives from SSHRC’s Corporate Strategy and Performance Division.

The document is organized as follows: Section 1 presents an introduction and the evaluation context—which describes the rationale for this evaluation, and the overall evaluation strategy. Section 2 outlines the profiles of the SIG and ASU funding opportunities. Sections 3 and 4 present the evaluation findings and conclusions. Section 5 provides the recommendations.

1.1 Insight Program

SSHRC funds research and research training that builds knowledge about people, past and present, with a view toward creating a better future for citizens of Canada and the world. From questions of family and culture to concerns about jobs and employment, research about people—how we live, what we think and how we act—informs new knowledge on and insights into the issues that matter most to Canadians. The Insight program aims to support and foster excellence in social sciences and humanities research that is intended to deepen, widen and increase our collective understanding of individuals and societies, as well as to inform the search for solutions to societal challenges.

The objectives of the Insight program are to:

• build knowledge and understanding from disciplinary, interdisciplinary and/or cross-sector perspectives through support for the best researchers;
• support new approaches to research on complex and important topics, including those that transcend the capacity of any one scholar, institution or discipline;
• provide a high-quality research training experience for students;
• fund research expertise that relates to societal challenges and opportunities;
• mobilize research knowledge, to and from academic and non-academic audiences, with the potential to lead to intellectual, cultural, social and economic influence, benefit and impact.

The Insight program is composed of two subprograms: individual, team and partnership research grants; and SSHRC institutional research capacity grants. The current evaluation focused on SSHRC’s institutional research capacity grants, which consist of two funding opportunities: SIG and ASU.

1.2 SSHRC Institutional Grants Funding Opportunity

Objectives

The purpose of the SIG funding opportunity is to assist Canadian postsecondary institutions to:

• develop, increase or strengthen research excellence in the social sciences and humanities;
• assist, in particular, researchers embarking on their research career to become competitive in grant competitions at the national level;
• assist established researchers with modest funding requirements, or those wishing to reorient and strengthen their research programs; and
• support national and international knowledge dissemination and collaboration.

Program Delivery and Expenditures

SSHRC provides annual grants to eligible institutions for three-year terms. The annual grant to each institution is calculated according to the following formula:

• $50 for each faculty member whose discipline falls within SSHRC's mandate; plus
• a payment based on the university's average performance in all SSHRC research support programs in the previous three years of competitions, calculated at the rate of:

• 23 per cent of the first $100,000 awarded;
• 20 per cent of the next $400,000 awarded; and
• 14 per cent of the remainder.

The method of calculation recognizes multi-institutional grants, in order to distribute credit for performance to all co-applicants. Although the awards are granted for a three-year period, SSHRC, since 2002, makes the calculation annually and adjusts the value of the grant accordingly. Also, SSHRC applies an adjustment factor, in order to bring the total grants into the allocated budget for the program. SSHRC guarantees a minimum grant of $5,000 to each eligible institution.

It should be noted that a review process was introduced in 2015 for SIG; grants awarded using the new SIG merit review process are not included in this evaluation, as they are out of scope.

The following table provides information on SIG expenditures and awards from 2002 to 2011. The SIG funding opportunity represented 1.5 per cent of SSHRC’s total expenditures in 2013-14.

Table 1: SSHRC Institutional Grants Awards and Grant Expenditures, from 2002 to 2011

1.3 Aid to Small Universities Funding Opportunity

Objectives

The main objective of this funding opportunity is to enable small universities to develop and strengthen focused research capacity in the social sciences and humanities. The ASU funding opportunity supports the following eligible activities:

• start-up costs / partial funding of research centres;
• stipends to doctoral students, provided the program of study is related to the ASU grant;
• agenda-setting seminars;
• visiting scholars (travel and stipend);
• organization of colloquiums or symposiums; and
• seed funding for collaborative research or the development of partnerships.

Program Delivery and Expenditures

The ASU grants are awarded on a competitive basis for a three-year period. The maximum value of a grant is $30,000 per year for three years. Each university must justify its grant request; the amount requested is subject to review.

The ASU program is not intended to provide sustaining support to institutions, nor is it intended as an alternative to SSHRC's national grants programs. Therefore, SSHRC limits ASU support of research infrastructure, such as centres and institutes, to six years.

Information on the number of ASU awards and expenditures from 2002-14 is summarized in Table 2. The ASU funding opportunity represented 0.2 per cent of SSHRC’s total expenditures in 2013-14.

Table 2: Aid to Small Univiersities Applications, Awards and Grant Expenditures, from 2002 to 2014

Applications/Awards	2002	2005	2008	2011	2014
Number of Applications	36	33	40	40	51
Number of Awards	23	21	22	21	20
Total Grant Expenditures ($)	1,748,548	1,783,682	1,834,999	1,808,908	1,791,881

1.4 Insight Program Logic Model

Along with SSHRC’s other Insight program funding opportunities, SIG and ASU contribute to the achievement of the outputs and outcomes described in the Insight program logic model (see Appendix C). The SIG and ASU funding opportunities contribute to all of the Insight-program outputs. These consist of “Grants for individual researchers and teams” and “Program literature, communication products, presentations and other interactions with the [social sciences and humanities] community.”

Among the immediate outcomes, “Individuals and teams engage in research and research-related activities” and “[social sciences and humanities] students and postdoctoral researchers are engaged in [social sciences and humanities] research training” were the outcomes areas examined through the evaluation. In addition, two intermediate outcomes were assessed, consisting of “Research is produced and mobilized” and “Research capacity is developed or strengthened within postsecondary institutions.” A full description of these Insight program outcomes is included in Appendix D.

---

2.0 Evaluation Scope, Issues and Questions

This is a routine evaluation of SSHRC’s institutional research capacity grants. The last evaluation of the SIG and ASU funding opportunities was approved in January 2011. Therefore, the current evaluation was required as per the evaluation coverage requirements stipulated in Section 42.1(1) of the Financial Administration Act^{Footnote 3} and the Treasury Board Policy on Evaluation^{Footnote 4} (2009).

The evaluation was designed to address the five core evaluation issues stipulated in the Treasury Board Policy on Evaluation (2009), which fall within two broad categories: relevance and performance. The current evaluation also addressed specific issues related to the design and delivery of the funding opportunities. Table 3 presents the evaluation questions that were developed to address each evaluation issue.^{Footnote 5}

Table 3: Evaluation Issues and Evaluation Questions

Relevance
Issue 1: Continued need for the funding opportunities	Q1. To what extent do SIG and ASU funding opportunities continue to address a demonstrable need and are responsive to the needs of Canadian institutions?
Issue 2: Alignment with government priorities	Q2a. To what extent are SIG and ASU funding opportunities objectives consistent with the federal government priorities? Q2b. To what extent are SIG and ASU funding opportunities objectives consistent with SSHRC’s strategic outcomes?
Issue 3: Alignment with federal roles and responsibilities	Q3. Is there a role or responsibility for the federal government in delivering SIG and ASU funding opportunities?
Performance (effectiveness, efficiency and economy)
Issue 4: Effectiveness of the funding opportunities	Q4. What contribution have SIG and ASU funding opportunities made to the achievement of outcomes?
Issue 5: Demonstration of efficiency and economy	Q5. To what extent are SIG and ASU funding opportunities cost-efficient?
Design and Delivery
Issue 6: Design and delivery	Q6. To what extent are the SIG and ASU funding opportunities delivered as planned?

2.1 Evaluation Methods

Data from multiple lines of evidence were collected, analyzed and synthesized, in order to assess the relevance and performance of the funding opportunities. The responsibility for data collection was shared between evaluators from an external consulting firm, Goss Gilroy Inc., and evaluators from SSHRC’s Evaluation Division. The evaluation methods and groups responsible for data collection are presented in Table 4. These methods were mapped to the evaluation questions and indicators in the evaluation matrix (see Appendix A).

Table 4: Evaluation Methods and Group Responsible

Evaluation Method	Group Responsible
Document and literature review (n=64) Internal documents (n=49) External documents (n=15)	Evaluation Division
Key informant interviews (n=24) SSHRC management and staff (n=5) Institutional research officers (n=19) Follow-up interviews with institutional research officers (n=4)	Goss Gilroy Inc.
Administrative data review ASU applications (n=80); from 2008-11 to 2011-14 SIG and ASU Aaards (n=194); from 2008-11 to 2011-14 SIG and ASU general research fund (GRF) data (n=64) for the 2008-11 funding cycle	Evaluation Division
File review (n=345) ASU final activity reports (FARs) (n=273) for the 2008-11 and 2011-14 funding cycles SIG FARs (n=71) for the 2008-11 funding cycle (includes excel spreadsheets provided by institutions, listing the small, funded projects)	Evaluation Division
Cost-efficiency analysis	Evaluation Division
Overall integrative analysis and reporting	Evaluation Division

Challenges and Mitigation Strategies

In an effort to calibrate the scope, the current evaluation relied primarily on secondary data—i.e., data that had already been collected from institutions in the form of applications, final activity reports (FARs) and statements of account (SOAs). A limited number of key informant interviews were conducted in order to complement the analysis of secondary data. Challenges were experienced when reviewing the available reports, given that some were only available in hard copy, which had to be entered into a spreadsheet manually, while others contained narratives that required coding prior to the analysis phase. Notwithstanding the challenges experienced when reviewing the secondary data, all available applications, FARs and SOAs for the timeframe covered were included in the sample. As a result, the evaluation was able to provide evidence not previously available on how institutions are using the SIG and ASU funds.

---

3.0 Evaluation Findings

This section presents the findings and conclusions from the evaluation study, by evaluation question. The main conclusions for each evaluation question are presented at the start of each section, and are followed by the specific findings that support the conclusions.

3.1 Relevance (EQ1): To what extent do SIG and ASU funding opportunities continue to address a demonstrable need and are responsive to the needs of Canadian institutions?

Key Findings: There do not appear to be other funding opportunities that provide funding to institutions to support small-scale research projects in the social sciences and humanities. The SIG and ASU funding opportunities are meeting a specific institutional need by providing flexible institutional grants to support a large number of small-scale research projects. Institutions highlighted that, in the absence of the SIG and ASU funding opportunities, they would be unable to fund the same number of small-scale research projects.

Continued need

Recent federal budgets have confirmed the federal government’s commitment towards supporting social sciences and humanities research, announcing increased funding for SSHRC to support advanced research in the social sciences and humanities. Along with new funding for SSHRC, there have also been changes in the research context both at institutions and at SSHRC. Both research officers and SSHRC management and staff highlighted that additional funding for research in the social sciences and humanities from the federal government has prioritized funding research that involves partnerships. In addition, interview respondents also noted that the success rates are lower in recent years for Insight Grants and Insight Development Grants competitions, compared to for earlier, similar funding opportunities (Standard Research Grants and Research Development Initiatives).

These two main changes in the research context have resulted in limited availability of funding targeted at supporting small-scale research projects in the social sciences and humanities. In fact, all institutional representatives expressed the importance of receiving funding through the SIG and ASU funding opportunities. Various reasons were identified to justify the need for these types of institutional grants. For example, research officers reported that the funds were required in order to support smaller-scale / short-term research projects that do not require large grants; and to provide seed funding to help researchers develop the necessary capacity to become competitive for future SSHRC grants. Furthermore, institutional representatives highlighted that, in the absence of the SIG and ASU funding opportunities, they would be unable to fund the same number of small-scale research projects.

Institutional expenditure information shows that most large and medium institutions are spending the majority of their SIG funds during the three-year funding cycle. However, one-third of small institutions had remaining SIG and ASU funds at the end of the three-year funding cycle. More specifically, the SIG administrative data review showed that 65 per cent of SIG-funded institutions (2008-11) had unspent SIG funds after the three-year grant period. In other words, 47 SIG-funded institutions did not spend, on average, 15 per cent of their total awarded amount by the end of the grant period. This amounted to just over $2 million for the 2008-11 funding cycle. It is interesting to note that small SIG-funded institutions had the largest proportion of unspent funds (39 per cent), compared to the large (13 per cent) and medium (12 per cent) SIG-funded institutions.

The analysis of the ASU expenditures revealed a similar pattern: 77 per cent of ASU-funded institutions had not spent all of their ASU funds by the end of the three-year grant period. This means that 17 (out of 21) ASU-funded institutions did not spend, on average, 29 per cent of the total awarded amount. This amounted to $409,061 for the 2008-11 funding cycle. The number of ASU applications, along with the number of awards and funding amounts, remained stable across the two funding cycles (2008-11 and 2011-14).

Given that the evaluation found a strong perceived need on the part of institutions for the SIG and ASU funding opportunities, it was interesting to discover that some institutions (in particular small institutions) are not spending all of the funds within the three-year grant cycle. In an effort to further explore the unspent funds findings, a number of follow-up interviews were conducted with key informants who had previously participated in the initial key informant interviews. Four out of six contacted institutions agreed to participate in a short (10-minute) follow-up phone call. The main objective of the follow-up interviews was to gather specific anecdotal information about why institutions are not spending all of the SIG and ASU funds.

Although only four follow-up interviews were conducted, similar information emerged across the interviews. The competitive process that institutions are required to establish in order to award SIG funds was cited as one of the reasons institutions were unable to spend all of their funds. Their ability to award small seed grants through a competitive process was dependent on the number of applications received, which, in some cases, was insufficient, leaving the institutions with unspent funds. For ASU-funded institutions, two small institutions indicated that they had misjudged the planning process, and were unable to implement the activities outlined in their applications because certain elements ended up not being feasible due to the limited grant funds. It should also be noted that all four institutions indicated that they did not perceive an urgency to spend the funds during the three-year funding cycle, because there are no consequences for not spending the funds.

A number of evaluation findings suggest that the grant amounts that small institutions are receiving through the ASU funding opportunity may not be sufficient to strengthen institutional research capacity in areas identified in the institutions’ strategic plans. First, only a portion (15 per cent) of the ASU-funded institutions clearly identified that they used the funds to support their institutions’ strategic areas of research. Most frequently, they reported using funds to support small projects where the link between the institutions’ strategic research plans and the research funded was not obvious.

Interestingly, the minority of institutions that clearly illustrated that they used the funds to support research in areas of strategic importance tended to report awarding larger grants. In addition, one-quarter of ASU-funded institutions reported combining their ASU funds with other sources of funds, in order to provide grants to their researchers. The issue that the amount of ASU funds received by small institutions is too small to address significant institutional needs was identified in the previous evaluation. Therefore, the ASU grant amounts provided to small institutions may not be in line with some of the funding opportunity’s expected outcomes.

Similar funding opportunities

A scan of SSHRC’s current suite of funding opportunities found that there are funding opportunities that share some similar objectives to those of SSHRC’s institutional research capacity grants. The document review findings noted that SSHRC’s Insight Development Grants funding opportunity does share some similarities to the SIG and ASU funding opportunities in terms of objectives. For example, similar to the SIG and ASU funding opportunities, the Insight Development Grants funding opportunity supports short-term research projects that are in their initial stages, knowledge mobilization activities, and the training of students. Although commonalities were noted between the Insight Development Grants funding opportunity and the two funding opportunities considered in this evaluation, there are some key objectives and design and delivery features of the SIG and ASU funding opportunities that are different from the Insight Development Grants funding opportunity. These include the institutional nature of the SIG and ASU funding opportunities, which is reflected in the program objectives that indicate that SIG and ASU funds should contribute to developing, strengthening and/or maintaining the social sciences and humanities research capacity at the institution. The ASU funding opportunity objectives also contain an additional component that requires institutions to use the funds to strengthen institutional research capacity in areas identified in the institutions’ strategic plans. Since the SIG and ASU funds are awarded to institutions and not individual researchers, the number of small-scale research projects funded by the SIG and ASU funding opportunities is much greater compared to by the Insight Development Grants funding opportunity. Over a three-year period, SIG and ASU funds were used to support 5,615 research-related projects (from 2008-11), whereas Insight Development Grants supported 881 research projects, each for a three-year period (from 2011 and 2013). It should be noted that the average grant size is also much larger for the Insight Development Grants, compared to that for SIG and ASU. Interestingly, all interview respondents indicated that they were not aware of any other funding opportunities that have the same objectives to those of the SIG and ASU funding opportunities—to support small-scale research projects.

3.2 Relevance (EQ2): To what extent are SIG and ASU funding opportunities objectives consistent with the federal government priorities, and SSHRC’s and SSHRC’s strategic outcomes?

Key Findings: The SIG and ASU funding opportunities are aligned with the federal government’s science, technology and innovation strategy and SSHRC’s strategic outcomes. However, the fit of the SIG and ASU funding opportunities within SSHRC program architecture is less clear.

Consistency with federal government and SSHRC priorities

As a federal research funding agency, SSHRC contributes to the Government of Canada’s science, technology and innovation (ST&I) strategy, in order to further government priorities in enhancing the economic, social and cultural development of Canada, its communities and its regions. By funding the research projects in the social sciences and humanities, SIG and ASU contribute to the “Knowledge” and “People” pillars of the current ST&I strategy^{Footnote 6}.

Along with other SSHRC funding opportunities, the SIG and ASU opportunities also contribute to the objectives of the Insight program. These include: helping Canadian institutions build knowledge and understanding about people, societies and the world; supporting researchers and students; funding research infrastructure; and providing other resources. The Insight program, together with SSHRC’s other two umbrella programs (Talent and Connection),^{Footnote 7} should result in the achievement of SSHRC's “Canada is a world leader in social sciences and humanities research and research training” strategic outcome.

While the documents reviewed support the consistency of the SIG and ASU funding opportunities with the government’s and SSHRC’s strategic outcomes, SSHRC management and staff expressed differing opinions regarding the extent to which SIG and ASU were consistent with federal government priorities and SSHRC’s strategic outcomes. A minority of interviewees indicated that allowing universities the flexibility to support research that has not yet been deemed “excellent” through a merit review process is not consistent with SSHRC’s approach to funding research. In addition, this minority of interviewees also indicted that building the capacity of small institutions is not explicitly aligned with SSHRC’s objectives. It is important to note, however, that the majority of interviewees felt that SIG and ASU are well aligned with SSHRC’s strategic plan and objectives.

Alignment with SSHRC’s program architecture

The findings from the document review noted that the fit of the SIG and ASU funding opportunities within SSHRC’s current program architecture is not necessarily clear. In 2010, SSHRC undertook a program architecture renewal, with the goal of reducing complexity and eliminating overlap in program objectives, in order to achieve greater overall program coherence. A number of SSHRC management and staff explained that the focus had been on streamlining SSHRC’s larger funding opportunities (i.e., those with larger budgets and higher profiles). Therefore, the SIG and ASU funding opportunities’ fit within the new program architecture renewal had not been considered to any great extentt that time, it was acknowledged that these funding opportunities would be revisited at a later date.

3.3 Relevance (EQ3: Is there a role or responsibility for the federal government in delivering SIG and ASU funding opportunities?

Key Findings: The role of the federal government in delivering the SIG and ASU funding opportunities is confirmed.

Role of the federal government

The 2010 evaluation of the Insight funding opportunities (i.e., Standard Research Grants and Research Development Initiatives) highlighted that the onus of responsibility for funding social sciences and humanities rests mostly with the federal government, because the amount of industry funding for social sciences and humanities researchers is not as large as it is for researchers in the natural sciences, engineering and health. According to a Statistics Canada report, the business enterprise provides almost half of research and development (R&D) funding in Canada; however, the natural sciences and engineering sector accounted for more than 90 per cent of the total R&D expenditures.^{Footnote 8} The vast majority of social sciences and humanities research is funded by the federal government and educational institutions. SSHRC program management and institutional representatives agreed that delivering the SIG and ASU funding opportunities is an appropriate role for the federal government. Institutions confirmed that SSHRC funding (through the SIG and ASU funding opportunities) is the main source of funding for small-scale social sciences and humanities research projects at their institutions. From the SIG and ASU file review, it is clear that institutions are using the SIG and ASU funds to support small-scale social sciences and humanities research projects.

3.4 Performance (EQ4): What contributions have SIG and ASU funding opportunities made to the achievement of outcomes?

Key Findings: Institutions are using SIG and ASU funds to support a large number of small-scale research projects in a variety of social sciences and humanities disciplines at Canadian institutions. In particular, the SIG and ASU funding opportunities are supporting mostly early and mid-career researchers in the creation and the dissemination of knowledge, while contributing to the training of students and postdoctoral researchers in the social sciences and humanities. There is only limited anecdotal evidence that the SIG and ASU funding opportunities are contributing to developing, maintaining or strengthening research capacity at institutions (for SIG and ASU) and in areas of importance identified in institutions’ strategic plans (for ASU only).

Research knowledge is produced and mobilized

The evaluation findings confirm that both SIG and ASU funds are being used by institutions to support small-scale research projects through research awards, travel grants, salaries for students, and other research-related project expenses. The proportions identified in the institutions’ SOAs are slightly different for the two funding opportunities (see Table 5).

Table 5: SSHRC Institutional Grants and Aid to Small Universities Expenses, as Reported by Institutions

Expense	SSHRC Institutional Grants		ASU
	$ millions	%	$ millions	%
Research Awards to Faculty	14.7	56	0.822	31
Travel Awards	6.6	25	0.670	25
Salaries for Students	2	8	0.349	13
Other Expenditures (materials, workshops, etc.)	2.5	11	0.700	28
Total	25.8	100	2.541	100%

It is important to note that both SIG and ASU-funded institutions reported combining multiple sources of funding (SIG, ASU and GRF) to cover the expenses reported in the SOAs. For SIG-funded institutions, approximately 82 per cent of the total expenditures were covered by a combination of SIG, ASU and GRF funds. For ASU-funded institutions, approximately 42 per cent of the total reported expenditures were covered by a combination of SIG, ASU and GRF funds. It is interesting to note that this does not align with the findings from the interviews; when interview respondents were asked whether or not they combine SIG and ASU funds, they indicated that they rarely do.

The evaluation findings confirmed that institutions are spending SIG funds on supporting research to conduct small-scale research projects. The file review revealed that from 2008 to 2011 the funded institutions used SIG funds to support 5,342 small-scale projects in various social sciences and humanities disciplines (see Table 6). Further analyses showed that the average grant amount per funded small-scale project was $2,590 (with a minimum of $50, and maximum of $67,500). The ASU file review also found that the funded institutions supported 273 small-scale projects during the two funding cycles. Given the differences in program design elements, it is not surprising that the average grant amount for an ASU-funded small-scale projects was higher than for SIG ($11,725, with a minimum of $460 and maximum of $90,000).

Table 6: Total Number of SSHRC Institutional Grant- and Aid to Small Universities-Funded Projects, by Funding Cycle

Funding opportunity	2008-11		2011-14		Total # of Projects
	# of Projects	Average amount	# of Projects	Average amount
SSHRC Institutional Grants	5,342	$2,590	*	*	5,342
Aid to Small Universities	153	$11,510	120	$11,898	273

The review of the FARs revealed that slightly more researchers in the social sciences, as compared to in the humanities, received SIG and/or ASU funding for small-scale research projects during the two funding cycles (see Table 7).

Table 7: Funded Faculty, by Discipline, for the SSHRC Institutional Grants and Aid to Small Universities Funding Opportunities

Main Discipline*	SIG (2008-11)		ASU (2008-14)
	# of Faculty	%	# of Faculty	%
Social Sciences	1,877	52	113	53
Humanities	1,692	47	80	38
Interdisciplinary	12	0.3	20	9
Total	3,581	100	213	100

The Association of Universities and Colleges of Canada’s study of faculty in higher education reported that, of the 40,800 full-time faculty employed at Canadian universities, “Social sciences (including education) account for the largest share of full-time faculty with almost 14,200 faculty members. There are approximately 8,100 in the humanities (including arts).”^{Footnote 9}

Despite the similarity in the number of researchers funded in each discipline (1,877 faculty members in social sciences versus 1,692 faculty members in humanities), the SIG funding opportunity supported a higher proportion of humanities faculty (21 per cent) compared to social sciences faculty (13 per cent).

The file review analysis also determined that almost one-quarter of SIG-funded projects involved the dissemination of research-related products (23 per cent). The SIG FAR does not require institutions to provide further information on the types of dissemination activities undertaken. Additional information is requested in the ASU FAR; a review of this information found that researchers spent ASU funds on the following dissemination activities: presentations (26 per cent), publications (24 per cent) and conferences (12 per cent).

Development of research capacity

For the purposes of the current evaluation study, the development of research capacity was assessed by calculating the numbers and descriptions of researchers supported (e.g., rank and field of study) and the numbers of students and postdoctoral researchers trained. In addition, both the key informant interview data and the information provided in the FARs were analyzed to gather information on institutions’ perceptions of the the SIG and ASU funding’s impact on the development of research capacity at their institutions. It is important to note that research capacity in the context of SIG and ASU funding is not defined in the program literature, and a definition of research capacity is not provided to institutions when they complete the FARs.

Based on the file review analysis, the ranks of the professors supported through the SIG and ASU funding opportunities were: assistant professors (1,433, or 40 per cent), followed by associate professors (1,253, or 35 per cent), and full professors (895, or 25 per cent). This is not surprising, given that the file review and the key informant interviews found that most institutions were using the funds to provide seed grants for researchers, and such funding would be more relevant for professors at the assistant or associate level.

As part of their small-scale project grants, researchers also supported students, postdoctoral researchers and other research personnel. This is evidenced by information collected in the SOAs and the FARs. For ASU, there were 580 students, postdoctoral researchers and other research personnel supported from 2008-11 to 2011-14 (see Table 8). On average, this is 290 students and postdoctoral researchers for each of the two funding cycles. Institutions provided $670,000 in salaries to students, postdoctoral researchers and nonstudents over the two funding cycles.

The amount spent on students, postdoctoral researchers and nonstudents increased by 17 per cent between the two funding cycles (2008-11 and 2011-14). This is likely due to an increase in student salaries. Given that ASU funds small institutions, it is not surprising that the majority of the funds for student salaries were provided to bachelor’s and master’s students (59 per cent).

The SIG file review showed that institutions spent $2 million on student salaries for the two funding cycles, 2008-11 and 2011-14. This amount increased by 19 per cent from the 2008-11 to the 2011-14 funding cycles. Medium and large institutions spent most of their funds on master’s students, whereas small institutions funded mostly undergraduate students.

Table 8: Proportion of Students Funded, by Level of Study

Funding Opportunity	Undergraduate		Master’s		Doctoral		Postdoctoral Researchers		Nonstudents		Total
	#	%	#	%	#	%	#	%	#	%	#
SSHRC Institutional Grants	*	23	*	40	*	14	*	2	*	19	*
Aid to Small Universities	287	49	183	32	60	10	35	3	15	6	580

In the SIG FARs, institutions are asked to provide a short narrative indicating how the SIG funds have contributed to their institution’s global research strategy. Theses narratives were reviewed and qualitatively coded into six categories. The categories, along with the proportion of institutions that identified each category, are listed in Table 9.

Table 9: Categories of Contribution for the SSHRC Institutional Grants Funding Opportunity

Category	% of Institutions
Contributing to enhancing researchers’ future grant applications	80
Developing research collaborations	68
Initiating a research program	65
Developing innovative research areas	27
Developing specific research expertise	15
Promoting a culture of research	14

As expected, the majority of institutions indicated that the funds went towards providing seed funding to their researchers. This corresponds with the information collected when the lists of small-scale funded projects were analyzed. Although institutions reported using the funds to enhance future SSHRC grant applications, we are unable to ascertain whether any future grant applications were successful.

Similar to the SIG FAR, the ASU FAR collects narrative information on how the grants have contributed to enhancing funded institutions’ research capacity in areas of importance identified in the institutions’ strategic plans. From this review of narratives, it was found that approximately 15 per cent of ASU-funded institutions are spending funds on research activities that have a clear alignment with the areas of importance identified in the institutions’ strategic plans. This includes funding research activities such as creating/developing research centres and/or research programs, international networks of research institutes, research clusters, research partnerships and larger research initiatives. The remaining institutions (85 per cent) describe funding small-scale activities such as short-term research projects, speaker series, dissemination activities, travel, materials and student stipends.

Key informant interviews were conducted in an effort to explore in more depth how the SIG and ASU funds have helped institutions develop, increase or maintain their research capacity. When asked, most research officers responded that SIG and ASU funding has, to a significant or great extent, helped their institution increase its research capacity. When asked to explain this further by providing additional details, four out of 19 SIG-funded institutions reported that researchers who were awarded SIG funds during the last funding period later received a larger SSHRC grant (these four institutions reported a range between 25 and 100 per cent of SIG-funded researchers subsequently receiving a SSHRC grant). Furthermore, one large university had calculated that, for the 2011-14 funding cycle, their institution’s SIG funding had a 750 per cent leveraging rate. Finally, some research officers also highlighted the contribution of SIG funding to longer-term outcomes, such as their university getting its first, or adding another, Canada Research Chair.

3.5 Performance (EQ5): To what extent are SIG and ASU funding opportunities cost-efficient?

Key Findings: Both the SIG and ASU funding opportunities are being delivered in a cost-efficient manner. The SIG funding opportunity is more cost-efficient compared to the ASU funding opportunity, due to the differences in how the funding is awarded to institutions (allocation vs. merit review).

Cost-efficiency analysis

Results from the program-efficiency analysis showed that, overall, both the SIG and ASU funding opportunities are delivered in a cost-efficient manner.^{Footnote 10} The ratios indicated that, on average, it cost 29 cents to deliver $100 of SIG funding, and $3 to deliver $100 of ASU funding, over the two funding cycles. That the SIG funding opportunity is more cost-efficient than the ASU funding opportunity can be explained by the different mechanisms used to award the funding: i.e., an allocation (SIG) vs. an adjudication process (ASU). It is important to note that, for the period under review, the SIG funding opportunity had not yet introduced a merit review process.

Another important caveat is that, since both the SIG and ASU funding opportunities are awarded every three years, the costs associated with delivering the funding opportunities are mostly incurred in the competition year. In non-competition years, the costs associated with delivering the funding opportunities are minimal. As a result, in competition years, the cost-efficiency ratios increase and it costs 46 cents to deliver $100 of SIG funding, and approximately $9 to deliver $100 of ASU funding. Not surprising was the finding that the SIG funding opportunity is more cost-efficient compared to the ASU funding opportunity. This is explained by the differences in how the funding is awarded to institutions (allocation vs. merit review). The cost-efficiency ratios displayed in Table 10 are for six years (two competition cycles).

Table 10: SSHRC Institutional Grants and Aid to Small Universities Program Expenditures, Administrative Expenditures, and Relative Ratios, for Fiscal Year 2008-09 to 2011-14

SSHRC Institutional Grants	2008-09	2009-10	2010-11	2011-12	2012-13	2013-14
Total operating expenditures	$14,278	$17,758	$23,613	$11,629	$11,774	$11,145
Grant funds awarded	$5,188,000	$5,188,000	$5,188,000	$5,184,000	$5,185,500	$5,183,000
Total program costs	$5,202,278	$5,205,758	$5,211,613	$5,195,629	$5,197,274	$5,194,145
Efficiency ratio (% of operations money / grant money)	0.28	0.34	0.46	0.22	0.23	0.22
Efficiency ratio (% of operations money / total program costs)	0.27	0.34	0.45	0.22	0.23	0.21

Aid to Small Universities	2008-09	2009-10	2010-11	2011-12	2012-13	2013-14
Total operating expenditures	$242	$99	$56,385	$55	$54	$52,239
Grant funds awarded	$605,208	$614,573	$615,218	$608,939	$589,969	$580,000
Total program costs	$605,450	$614,672	$671,603	$608,994	$590,023	$632,239
Efficiency ratio (% of operations money / grant money)	0.04	0.02	9.17	0.01	0.01	9.01
Efficiency ratio (% of operations money / total program costs)	0.04	0.02	8.39	0.01	0.01	8.26

Given that the SIG and ASU funding opportunities are SSHRC’s only funding opportunities that provide institutional grants for small-scale projects, it was difficult to find an appropriate comparison for the cost-efficiency ratios. Therefore, the funding opportunity chosen for comparison cannot be directly compared to the SIG and ASU funding opportunities; however, it does provide a sense of how the SIG and ASU funding opportunities’ cost-efficiency ratios compare to another institutional funding opportunity delivered by SSHRC. For SIG, the Research Support Fund (RSF; formerly the Indirect Costs Program) was chosen because both funding opportunities provide institutional grants that involve an allocation/formula-based calculation to determine the funding each eligible institution will receive. The findings show that SIG is slightly less cost-efficient than the RSF, as it costs 17 cents to deliver each $100 of RSF funding. While both SIG and the RSF can be considered extremely cost-efficient, it is not surprising that the RSF is more cost-efficient, given that there are significant economies of scale associated with RSF, as RSF program expenditures are roughly $300 million (compared to $15 million for the SIG funding opportunity).

3.6 Design and Delivery (EQ6): To what extent are the SIG and ASU funding opportunities delivered as planned?

Key Findings: Institutions are mostly satisfied with the tools and guidelines provided by SSHRC. Institutions use varied approaches internally in order to award SIG and ASU funds to researchers; however, it appears that most institutions are not consistently promoting the funds as originating from SSHRC. The current reporting forms for SIG and ASU are not meeting SSHRC’s information needs in an efficient way.

Institutional satisfaction with Social Sciences and Humanities Research Council tools and guidelines

The majority of research officers interviewed indicated that they were satisfied with the tools and guidelines provided by SSHRC for the administration of the SIG and ASU funding opportunities. Overall, they indicated that the guidelines are clear, accessible and easy to follow. When asked, a few institutions suggested some opportunities for improvement, such as: institutions should be asked to report on higher-level outcomes instead of individual transactions; SSHRC should provide the specific list of eligible activities in the funding opportunity description: and SSHRC should clarify the definition of “faculty member.” Finally, official language minority community institutions indicated that they were satisfied in their experiences communicating with SSHRC.

Perceptions of adjudication criteria/process at the Social Sciences and Humanities Research Council and institutions

Research officers at institutions were asked for their opinions regarding SSHRC’s adjudication criteria and selection process for the ASU funding opportunity. Overall, research officers were satisfied with SSHRC’s ASU selection process. In particular, they expressed that it was well-known and understood and that feedback received regarding unsuccessful proposals was clear and straightforward.

Researcher officers also identified aspects of their institution’s process that they considered most effective, such as having a review process that mirrors SSHRC’s, including having a high-quality review committee. They also suggested a few areas for improvement at their institutions, including more consistently applying the award criteria across years, and disbursing funds to researchers more quickly.

SSHRC Institutional Grants and Aid to Small Universities visibility

Research officers were also asked about the visibility of the SIG and ASU funding opportunities. Research officers and administrators at institutions appear to be well aware of the SIG and ASU funding opportunities. However, since most institutions rename their internal funding opportunities to something other than “SSHRC SIG,” it is likely that, in many instances, funded researchers are unaware that they received SSHRC funds. The last evaluation also suggested that the level of awareness of SIG and ASU among researchers (funded and unfunded) varied.

Institutions’ grant allocation strategies

As part of the key informant interviews, research officers were asked to elaborate on how they allocate or award the SIG and ASU grants at their institutions. Almost all those interviewed indicated that they award the SIG and/or ASU funds through their institution’s research office; however, a few large institutions indicated that faculties were provided with a portion of the funds, and that deans were responsible for administering the funds within their faculty.

The analysis of the SIG FARs showed that half of the institutions used their own criteria to support the evaluation of proposals (36 institutions, or 51 per cent of the total; slightly higher for large institutions, at 61 per cent). The competitive process that institutions employed to award the SIG funds included calls for proposals and a review committee based on an established set of criteria (59 institutions, or 83 per cent of the total).

In contrast, the review of ASU FARs revealed that approximately half of the institutions (18 institutions, or 45 per cent of the total) allocated the ASU funds directly to their departments. Only one-third of the ASU-funded institutions mentioned establishing a competition-based process (13 institutions, or 33 per cent of the total). Note that the SIG funding opportunity requires that the funds be awarded through a competitive process; however, this is not a requirement for the ASU funding opportunity.

Key informant interviewees commented that, given the large number of small-scale research projects that are funded through the SIG and ASU funding opportunities, it would not make sense, from an efficiency perspective, for SSHRC to adjudicate such a large number of small-scale research proposals.

Satisfaction with the current delivery model and alternative delivery models

Research officers were asked to comment on their satisfaction with the current delivery models for the SIG and ASU funding opportunities. They were generally satisfied with the current delivery models, and did not suggest any substantial changes to the funding opportunities. They did, however, take the opportunity to point out the importance of providing institutions with flexibility around how and when to use the funds. When SSHRC management and staff were asked the same questions, differing views emerged regarding how much discretion institutions should have in deciding how and for what purposes to allocate SIG and ASU funds. Overall, respondents indicated satisfaction with the current model; however, the following options for improving the delivery of the SIG and ASU funding opportunities were suggested:

• eliminate the SIG and ASU funding opportunities and roll the funds into the Insight Grants envelope; and
• combine the SIG and ASU funding opportunities into one funding opportunity and implement a formula-based allocation method, weighting the smaller institutions favourably, as done in the RSF.

Quality of information collected in the final activity reports and other data collected by institutions

Research officers indicated that they had not experienced any major challenges in completing the SIG and ASU applications and FARs. They did, however, speak to the need to improve the relevance of information collected in the FAR templates in a manner that better reflects the objectives and expected outcomes of the funding opportunities. In fact, some research officers, as well as SSHRC management and staff, expressed interest in focusing more on outcome-level data, as opposed to the disaggregated activities-level data currently reported. Most research officers indicated that it is important to balance the benefits of high-quality information with the administrative burden on institutions. The evaluators for the current evaluation noted a need to streamline the reporting tools, to allow for more consistent reporting across funded institutions.

Research officers were also asked to provide information regarding any additional data that they collect for the SIG and/or ASU funding opportunities. About half of the research officers interviewed mentioned that their institution collects information in addition to what is reported in FARs for the funding opportunity. In most of these cases, institutions request reports from researchers who have received SIG funds. These reports require researchers to describe project outcomes in further detail, including instances where they applied for external funding after completing the SIG-funded project. In contrast, research officers at ASU-funded institutions indicated that they did not collect any information in addition to what is provided to SSHRC.

---

Conclusions

Overall, the evaluation confirmed the relevance of SSHRC’s institutional research capacity grants, as they appear to be meeting a specific need by providing institutions with funding for small-scale research projects. The evaluation findings do suggest, however, that revisions to the funding opportunities’ objectives, and a reconsideration of their fit within SSHRC’s program architecture, would improve the relevance of SSHRC’s institutional grants.

In terms of performance, the SIG and ASU funding opportunities are contributing to the immediate and intermediate outcomes of SSHRC’s Insight program. More specifically, the evaluation findings revealed that institutions are using SIG and ASU funds to support social sciences and humanities researchers in engaging in research and research-related activities and in supporting students and postdoctoral researchers engaged in social sciences and humanities research training. In addition, there is evidence that the SIG and ASU funding opportunities are contributing to the Insight program intermediate outcome of producing and mobilizing research knowledge. However, the extent to which the SIG and ASU funding is contributing to developing or strengthening research capacity within postsecondary institutions is less evident.

Finally, the evaluation findings seem to suggest that the relevance and performance of the SIG and ASU funding opportunities could be improved with the implementation of several design and delivery changes, including by clarifying the funding opportunities’ objectives, providing increased guidance to institutions regarding the use of the funds, and improving the format and content of the FARs.

---

5.0 Recommendations

Based on the findings and conclusions of the current evaluation, and following discussions with the Evaluation Advisory Committee and the Departmental Evaluation Committee, four recommendations are proposed for the evaluation of SSHRC’s institutional research capacity grants.

Recommendation 1:

It is recommended that program management continue to monitor institutions’ use of SIG and ASU funds, with particular attention to the amounts that remains unspent after the three-year grant period.

In addition to monitoring institutions’ use of SIG and ASU funds, program management should consider developing and communicating guidelines related to the amount of SIG and ASU funds that can remain unused at the completion of the grant period.

Recommendation 2:

It is recommended that program management review the objectives of SSHRC’s institutional research capacity grants (SIG and ASU), with specific emphasis on defining the current expected result (intermediate outcome) of developing, maintaining or strengthening institutional research capacity.

SSHRC should examine determining what results it expects institutions to achieve with respect to this intermediate outcome. Consideration should be given to the size of institution and the amount of the award when determining/defining the expected results.

Recommendation 3:

It is recommended that program management revise the application and end-of-grant reporting forms currently used for SSHRC’s institutional research capacity grants (SIG and ASU).

The application forms and reporting forms should be more clearly aligned, so that institutions are able to report systematically on results achieved. The reporting forms should be completed electronically and should request a small number of specific quantitative information from institutions (e.g., number of projects funded, number of students trained, etc.). In addition, open-ended questions should be minimal in the new reporting forms. Once the intermediate outcomes have been defined (see recommendation 1), the reporting forms should contain questions that allow SSHRC to accurately measure the expected results.

Recommendation 4:

In line with SSHRC’s program architecture renewal exercise, it is recommended that program management consider whether efficiencies could be gained by streamlining and combining the budgets of its two current institutional research capacity grants funding opportunities (SIG and ASU) into one funding opportunity—SSHRC Institutional Grants.

Should the funding opportunities be combined, an allocation formula could be developed, such that small institutions would be favoured. Although applications would be required, the adjudication for the new funding opportunity could be similar to the criteria-based merit review recently implemented for SIG, where institutions’ proposals are deemed satisfactory or unsatisfactory based on a set of criteria aligned with the funding opportunity’s revised objectives/outcomes (see recommendation 1).

---

Appendix A: Evaluation Matrix

Issues and Evaluation Questions	Indicators	Source of Data
		Document/ Literature Review	Review of Activity Reports	Interviews with SSHRC Staff	Interviews with Research Officers	Review of Statements of Accounts	SSHRC’s Database and Statistics	Cost-Efficiency Analysis
Relevance
Issue 1: Continued Need for the Funding Opportunities
Question 1 To what extent do SIG and ASU funding opportunities continue to address a demonstrable need and are responsive to the needs of Canadian Institutions?	1.1 Description and assessment of the need for SIG and ASU in the current university research context (trends in overall grant funding in the social sciences and humanities, number of applications, number and amount of SIG/ASU grants for the last two funding cycles, and perceptions of stakeholders)(SIG and ASU)	X		X	X	X
	1.2 Trend in the number of small institutions (ASU)						X
	1.3 Proportion of unspent funds compared to the institutions’ GRF, by size of institution (SIG and ASU )					X	X
	1.4 Evidence of relative importance of SIG and ASU in comparison to other SSHRC and/or external funding opportunities (SIG and ASU)	X		X	X
Issue 2: Alignment with Government Priorities
Question 2 2a. To what extent are SIG and ASU funding opportunity objectives consistent with the federal government priorities? 2b. To what extent are SIG and ASU objectives consistent with SSHRC’s strategic outcomes?	2.1 Degree of alignment with federal government priorities, as articulated in recent documents or communications (SIG and ASU)	X		X
	2.2 Degree of alignment with SSHRC’s strategic outcomes (SIG and ASU)	X		X
Issue 3: Alignment with Federal Roles and Responsibilities
Question 3 Is there a role or responsibility for the federal government in delivering SIG and ASU funding opportunities?	3.1 Evidence that the funding opportunities are consistent with federal roles and responsibilities (SIG and ASU)	X		X	X
Performance (effectiveness, effinciency and economy)
Issue 4: Effectiveness of the Funding Opportunities
Question 4 What contribution have SIG and ASU funding opportunities made to the achievement of outcomes?	4.1 Assessment of the extent to which research knowledge is produced and mobilized
	4.1.1 Total amount of grant funds allocated to research activities and other expenses (travel, materials and supplies, salaries and stipends, dissemination events, equipment, etc.), by funding opportunity, by competition cycle (SIG and ASU)					X
	4.1.2 Total amount of grant funds invested in noneligible areas, by funding opportunity, by competition cycle (SIG and ASU)		X			X
	4.1.3 Total number of research and research-related activities (i.e., grants and grant amounts) supported by funding cycle (SIG and ASU)		X
	4.1.4 Evidence that institutions have developed or strengthened research capacity in a particular area of the social sciences and humanities (e.g., researchers refine competencies and skills; number and amount of grants increases) (ASU)		X		X
	4.1.5 Total number and types of research activities undertaken using SIG funds that contribute to the institution’s global research strategy (SIG)		X
	4.1.6 Total number and types of research activities supported, by funding cycle (SIG)		X
	4.1.7 Total number and types of research contributions (i.e., dissemination activities), by competition cycle (ASU)		X
	4.2 Assessment of the extent to which social sciences and humanities trainees enhanced their research and knowledge mobilization kills
	4.2.1 Number and types of students and fellows supported (e.g., undergraduate, masters, doctoral, and postdoctoral by competition cycle) (ASU)		X
	4.2.2 Total amount of grant funds spent on students (Canadian and foreign, by level of study) and non-student salaries, by funding opportunity, by competition cycle (SIG and ASU)					X
Issue 5: Demonstration of Efficiency and Economy
Question 5 To what extent are SIG and ASU funding opportunities cost-efficient?	5.1 Ratio of administrative costs to total program expenditures, for SIG and ASU and comparable programs (SIG and ASU)							X
Design and Delivery
Issue 6: Design and Delivery
Question 6 To what extent are the SIG and ASU funding opportunities delivered as planned?	6.1 Number and description of policy and communication products, tools, and guidelines produced by SSHRC, by funding opportunity (SIG and ASU)	X		X	X
	6.2 Level of satisfaction of institutions that used the tools and guidelines, by funding opportunity (SIG and ASU )			X	X
	6.3 Perceptions of the adjudication criteria/process at SSHRC and institutions (SIG and ASU)			X	X
	6.4 Institutions’ perceptions of SIG and ASU visibility, by funding opportunity (SIG and ASU)			X	X
	6.5 Description of how institutions award the SIG and ASU funds, by size of institution, and by type of selection process (e.g., use of the 4A list, selection criteria, availability of matching funds, best practices) (SIG and ASU)		X	X	X
	6.6 Opinions on alternative delivery mechanisms for SIG and ASU (SIG and ASU)			X	X
	6.7 Quality of information collected in the FARs (SIG and ASU)		X	X	X
	6.8 Other information or data collected by institutions, in addition to what is reported in the FARs and SOAs, that could enable SSHRC to report on outcomes (SIG and ASU)			X

---

Appendix B: Social Sciences and Humanities Research Council Program Alignment Architecture

Download Image

---

Appendix C: Draft Insight Logic Model

Download Image

---

Appendix D: Insight Program Description

a) Outputs

These are the direct products or services generated from Insight program activities. These outputs consist of the following:

Grants for individual researchers and teams

The outputs of the Insight funding opportunities developed, delivered, and promoted by SSHRC, are grants to individual and team researchers at Canadian postsecondary institutions in the social sciences and humanities to conduct research and research-related activities. In the case of Insight Development Grants and Insight Grants, the grants are directly awarded to the individuals and teams. In the case of the Aid to Small Universities (ASU) and SSHRC Institutional Grants (SIG) funding opportunities, funds are provided to institutions, which then allocate the funds internally to individuals and teams.

Program literature, communication products, presentations and other interactions with the social sciences and humanities community

A number of outputs other than grants are produced in order to achieve the desired outcomes of the Insight funding opportunities. Information about new and/or existing funding opportunities is communicated through literature aimed at applicants and merit reviewers, to share with them the opportunity descriptions, objectives, eligibility, application information, review processes and evaluation criteria. Guidelines and policies are created or updated as required, and are integrated into the merit review process, to communicate SSHRC’s priorities and visions. For instance, guidelines for effective research training were developed to emphasize the importance that SSHRC places on student training and skills development. Finally, through outreach and engagement, SSHRC generates communication products and provides advice to potential applicants.

b) Immediate Outcomes

These are the external consequences directly attributable to the outputs. They are expected to occur in the early stages of the award. They may be measured in the applications and in end-of-grant reports.

Individuals and teams engage in research and research-related activities

As a result of the funding provided through the Insight Development Grants, Insight Grants, SIG, and ASU funding opportunities, individuals and/ or teams of researchers engage in research and research-related activities, such as develop and enhance theoretical frameworks, methodologies and networks, as well as collect and analyze data. In addition, in the case of SIG and ASU, this may involve activities that occur at the end of research project, such as presenting at conferences, given that these funding opportunities do not necessarily fund entire research projects, but components of them.

Social sciences and humanities students and postdoctoral researchers are engaged in social sciences and humanities research training

As a result of the funding provided through research and research-related grants to individuals and teams, students and postdoctoral researchers are engaged in research and research-related activities. As per guidelines to applicants and merit reviewers, a research training component must be included in almost all Insight Development Grants- and Insight Grants-funded projects, as well as in some of the ASU- and SIG-funded activities. In general, the salaries of students and postdoctoral researchers are paid through the award; however, in some instances, students and postdoctoral researchers who are actively involved in a research project may have their salary paid through a fellowship or another grant.

c) Intermediate Outcomes

These are the external consequences expected to occur once the immediate outcomes have been achieved. These outcomes are expected to occur around the end of the award period. They may be measured in the end-of-grant reports.

Research is produced and mobilized

As a result of the research engaged in by individuals and teams, new knowledge is produced and mobilized through a variety of means (e.g., publications, presentations, social media) to academic and non-academic audiences. The knowledge produced has the potential to lead to intellectual, cultural, social, and economic benefits at this stage, because it is made available for knowledge receptors, both academic and non-academic.

Given the different objectives of the funding opportunities, the expectations for the production and mobilization of knowledge are different. Since the Insight Grants funding opportunity funds mature research projects, and the Insight Development Grants funds the initial stages of research, it is expected that there will be greater evidence of research contributions through Insight Grants than Insight Development Grants. In some instances, the research activities engaged in by recipients of SIG and ASU will lead to a distinct research contribution. such as a peer-reviewed journal article or a presentation.

Research capacity is developed or strengthened within postsecondary institutions

As a result of the research and research-related activities undertaken with ASU funding, and also as a result of the ASU guidelines and merit review process, small postsecondary institutions develop or strengthen their research capacity in a particular area of the social sciences and humanities. Developing and strengthening research capacity in this context implies that faculty members further refine their competencies and skills related to conducting research. This may manifest itself through an increase in the number and size of grants that researchers are awarded.

As a result of the research and research-related activities undertaken with Insight Development Grants funding, and of the merit review process, which recognizes the importance of supporting emerging scholars, early-career researchers are expected to subsequently become more competitive in other grant competitions.

While funding from the SIG and Insight Grants funding opportunities develops research capacity within Canadian postsecondary institutions, it is not an explicit objective of the funding opportunities.

---

Footnotes