SSHRC’s Aid to Scholarly Journals (ASJ) funding opportunity supports the dissemination and readership of original research results in the social sciences and humanities (SSH) via grants to Canadian scholarly SSH journals. The ASJ disburses more than $3 million in funding annually, awarded through periodic competitions, to approximately 160 journals. The ASJ is part of SSHRC’s Insight Program.
The ASJ evaluation was conducted jointly alongside that of the Awards for Scholarly Publications (ASP) program. Both funding opportunities provide financial support to enable the dissemination of Canadian SSH research results.
This evaluation was initiated in response to a request from SSHRC senior management to evaluate the ASJ and ASP. The overall purpose of the evaluation was to inform decisions by senior and program management regarding the future of ASJ and ASP in a changing environment. The last evaluation of the ASJ, also conducted as a combined evaluation with ASP, was completed in 1985 by Touchstone Policy & Program Evaluation Inc. For the current evaluation, separate evaluation reports on the ASJ and ASP were submitted. This management report addresses the final evaluation report for the ASJ.
The evaluation focused on questions of relevance, performance, cost-efficiency and alternatives. It covered the period 2008-17 for performance-related questions and 2008-18 for questions of relevance, cost-efficiency and alternatives. Particular attention was paid to questions of relevance and alignment given changes within ASJ and the scholarly journal publishing ecosystem since the last evaluation of the program.
Program management commends the evaluation team on the conduct of the evaluation. Program management is pleased that the evaluation conclusions demonstrate ASJ’s continued relevance to Canadian scholarly SSH journal publishing and the continued suitability of its funding model. ASJ funding enables journals to increase their publishing capacity. It also contributes to the publication of research on topics and issues of national interest. The results highlight the way ASJ funding augments SSHRC’s other investments in research and researchers in Canada.
The evaluation findings also confirm ASJ’s contribution to the dissemination and discoverability of SSH research. The ASJ has continually incentivized innovation and learning of best practices in scholarly publishing, from an initial pilot program aimed at supporting open access (OA) publication in 2007 to the transition to exclusively funding OA journals in its last competition in 2018. As part of this transition, the objectives of the program were also adjusted to include: leveraging significant federal investments in digital scholarly publishing infrastructure; assisting journals to take advantage of developments in digital technologies and innovative practices; and encouraging the transition to OA models of publishing. Data from this evaluation provide a useful baseline for future performance assessment in these areas.
Program management recognizes that Canadian scholarly SSH journal publishing operates within a broader research ecosystem and that the norms governing the assessment of research and researchers within this ecosystem largely determine scholarly publishing practices. As described by the evaluation, the ASJ on its own does not have the capacity to effect holistic change in scholarly journal publishing practices. In tandem with incentives offered within the ASJ, SSHRC continues to work with the Canadian SSH scholarly journal and researcher communities to advance shared priorities in research assessment and scholarly publishing through other means. Program management will continue to engage with the Canadian scholarly journal publishing community throughout the implementation of the Action Plan set out below.
| Recommendation |
Response |
Action |
Responsibility |
Target Date |
|
1. Continue to offer ASJ funding.
- Canadian journals remain important to the SSH research system in Canada. ASJ is demonstrating positive outcomes that would be challenging to reproduce by other means. ASJ is a relatively small investment for SSHRC under a cost-efficient model.
- Alternate revenue sources for OA journals are currently limited or present equity issues. Perceived risk of loss of public funding for OA, i.e., if programs are discontinued, is reported to inhibit OA adoption in the sector.
|
Agree |
1. Continue to offer ASJ funding via the current funding model. As concluded by the evaluation, ASJ is a cost-efficient funding opportunity whose effects would be difficult to reproduce by other means. It also augments SSHRC’s other investments in research and researchers.
|
Not applicable |
Not applicable |
|
2. Adjust ASJ’s objectives to align with the Insight Program, considering ASJ’s key strengths and capacity.
- Review the Insight Program’s Program Information Profile (PIP) to identify points of alignment between ASJ and Insight objectives and goals.
- Consider ASJ’s primary and potential outcomes given its niche and value as described by stakeholders, and its capacity.
- Update ASJ’s objectives to reflect how these intersect or have potential to align.
|
Agree |
2a. Conduct a review of Insight Program’s PIP and the ASJ’s primary and potential outcomes and capacity, as described in the evaluation report, to identify potential points of alignment. |
Research Grants Portfolio / Corporate Strategy and Performance |
June 2022 |
2b. Update ASJ’s objectives in light of identified points of alignment with the Insight Program and the evaluation’s findings concerning ASJ’s outcomes and capacity. |
Research Grants Portfolio |
October 2022 |
|
3. Ensure operational alignment of ASJ to updated objectives.
- Review adjudication criteria against equity, diversity and inclusion (EDI) commitments.
- Take first steps to collect data related to EDI, with a focus on monitoring for indicators related to equitable access to ASJ funding.
- Review adjudication criteria and performance indicators against the recommendations of the San Francisco Declaration on Research Assessment (DORA) declaration, to further support conditions for equitable participation.
|
Agree |
3a. Review eligibility and adjudication criteria with a view to:
- Ensuring alignment with updated core priorities and objectives, as identified in Action 2a above;
- Ensuring conditions for equitable participation, with specific reference to SSHRC’s EDI, official languages and DORA commitments.
|
Research Grants Portfolio / Corporate Strategy and Performance, with Evaluation Division providing technical assistance |
June 2023 |
3b. Explore options for gathering information on ASJ funding in relation to EDI. Areas for consideration include, but are not limited to:
- Alignment with corporate EDI data collection strategies and program priorities and objectives, as identified in Action 2b;
- Identification of methods best suited for capturing journals’ current and potential EDI performance;
- Journals’ capacity to address EDI and EDI data collection.
|
Research Grants Portfolio / Corporate Strategy and Performance, in consultation with Programs Planning and Operations Division and Information and Innovation Solutions |
June 2023 |
|
4. Update ASJ’s logic model.
|
Agree |
4. Create an updated logic model that more accurately depicts ASJ’s objectives as identified in Action 2b, and ASJ’s anticipated outcomes and performance indicators. |
Research Grants Portfolio / Corporate Strategy and Performance, with Evaluation Division providing technical assistance |
October 2022 |
|
5. At the corporate level, consider using ASJ as a vehicle to further advance key council priorities.
Options include:
- Given that the current investment in ASJ is not sufficient to make more than a very modest contribution to the amount, quality and accessibility/discoverability of published Canadian research, SSHRC could raise the grant amount.
- ASJ can contribute to SSHRC’s learning as an organization in key priority areas through close engagement with the sector and by keeping communication lines open. This is especially applicable in areas that are developing rapidly, such as EDI, research assessment, open research and open access, equitable and sustainable OA resourcing.
|
Partially agree |
5a. Continue to contribute to organizational knowledge in the area of scholarly publishing.
SSHRC has on-going investments that can be used to advance on this item: |
Research Grants Portfolio / Corporate Strategy and Performance / Research Partnerships Portfolio |
|
- Leverage the results of the study of the financial situation of Canadian scholarly journals, jointly commissioned by SSHRC and the Fonds de recherche du Québec – Société et culture, submitted on September 30, 2020.
|
Research Grants Portfolio |
June 2022 |
- Address the outcomes of the community engagement activities carried out after the release of the report on the past, present and future of scholarly communication commissioned by SSHRC as part of its strategic planning (Momentum 2020-2022).
|
Research Grants Portfolio / Corporate Strategy and Performance |
December 2022 |
- Explore options for leveraging knowledge gained from the Pan-Canadian Knowledge Access Initiative (PCKAI) to support the production of incentives and research outputs encouraging journals and researchers to join the OA community. The PCKAI aims to provide greater access to scholarly journal publications to increase the discoverability and impact, in Canada and internationally, of Canadian SSH research.
|
Research Partnerships Portfolio |
December 2023 |
5b. Explore whether and how ASJ can contribute to the policy requirements of the forthcoming Tri-agency Data Management Policy. |
Research Grants Portfolio / Corporate Strategy and Performance |
April 2022 |