Knowledge gaps in evaluating the effectiveness and impacts of Living Labs focused on environmental and agricultural sustainability

Printable version

About the project

Living Labs (LLs) bring together end-users, researchers, practitioners and other collaborators to co-develop and test innovative practices, technologies and solutions. They can advance user-centric solutions to complex environmental issues, including agricultural sustainability. Agriculture and Agri-Food Canada (AAFC) launched a five-year Living Laboratories Initiative focused on agroecosystems (2018), and a 10-year Agricultural Climate Solutions program to expand this LL national network (2021). The program launch presents opportunity for this project to inform this national initiative.

Despite the promise of the LL approach for addressing complex challenges, there is a gap in knowledge surrounding how to measure and evaluate the performance of a given LL process and its wider social and environmental impacts. Understanding evaluation can support effective implementation of LLs by building on successes and failures, and facilitate learning through common measures, approaches and knowledge sharing—key information for AAFC’s national initiative.

This knowledge synthesis project used mixed methods, including scoping literature review and expert workshops targeting LL practitioners to address the gap in LL evaluation in the context of environmental and agricultural sustainability.

Project objectives:

1. Synthesize best practices for evaluating impacts and effectiveness of LLs, drawing upon insights from related multi-stakeholder initiatives;
2. Identify knowledge gaps and strengths of LLs focused on environmental or agricultural sustainability to develop an agenda for future research and action; and
3. Build and engage a network of cross-sectoral LL experts, stakeholders and partners interested in LL for sustainability to solidify Canada as a leader in this field.

Key findings

1. The scoping review identified 41 articles focused on measuring the impact of LLs (56% peer-reviewed articles, 34% proceedings/conference papers, 1% books). Findings showed few studies on generalizable approaches or frameworks evaluating the impact of LLs and even fewer in the agricultural or sustainability sector. The dominant method for evaluation used in the literature was comparative qualitative using case studies. The main purpose of evaluation was improvement of the particular LL functioning (67%) rather than its wider impacts. A common entry point for evaluation among the studies in our final dataset was action research where participants of the LL helped develop the metrics and indicators that would be used to evaluate the LL.
2. The workshop involved three phases conducted in both English and French to develop a research agenda: i) surveys identifying key research themes to inform subsequent phases; ii) virtual workshops where experts further discussed and elaborated on research themes; and iii) asynchronous expert feedback and validation of finalized research agenda.
3. Workshops resulted in a future research agenda for evaluating and understanding impacts of LLs for sustainability. They also mobilized knowledge from the scoping review, bringing it to practitioner communities. The final LL research agenda outlined 50 research questions grouped under seven main research questions, as outlined below.

Policy implications

1. Findings from the scoping review and research agenda may be leveraged as a guide and collaborative platform to support effective implementation of LLs such as AAFC’s nation-wide LL network initiative. The research agenda can be used by researchers and LL actors to collaborate on future research, develop strategic directions and focus on evaluation to promote wider societal and environmental impacts over standardized reporting needs.
2. The diversity of actors and their definitions, expectations and perspectives surrounding the effectiveness and impacts of LLs should be considered in the evaluation process. This inclusion can support the development of relevant metrics for evaluating social and sustainability impacts from LLs in an equitable way, which leads to improved and effective evaluation processes.
3. The research agenda offers nuances for research and operations that capture the diverse nature of social and environmental impacts of LLs over time and at different scales, while also offering potential for greater qualitative and participatory methods in the evaluation and reporting of LL processes and outcomes. Actioning and consideration of the themes from the agenda can open spaces in policy to co-design questions and methods, co-create processes and co-evaluate impacts and outcomes.

Further information

Read the full report

Contact the researchers

Vivian Nguyen. Assistant Professor, Institute of Environmental and Interdisciplinary Science, Carleton University; vivian.nguyen@carleton.ca

Date modified:

2022-02-08