Assessing social learning? Four monitoring specialists provide some answers

Last month, the Climate Change Communication and Social Learning (CCSL) project organized an evidence-gathering workshop to better unpack what social learning is, and particularly how to assess and monitor it.

Here, four monitoring and evaluation (M&E) specialists who particiapted in the meeting reflect on some of the issues around social learning assessment and the M&E framework in the evidence-gathering workshop background paper.

• Barbara van Mierlo is associate professor at the Knowledge, Technology and Innovation Group at Wageningen University.
• Claire Hutchings is head of evaluation at Oxfam GB.
• Dr. Georgina Cundill is Senior Lecturer in the Department of Environmental Science at Rhodes University in South Africa.
  
  
• Richard Taylor is asenior researcher at Stockholm Environment Institute in Oxford. He is part of the M&E team at SEI and he is also editor of the adaptation decision-making initiative on weADAPT.

They agreed to share some of their experiences and insights . . .

What is your interest in social learning and what brings you to this workshop?

(Barbara van Mierlo)

I was suggested to attend by a colleague of mine Marc Schut, who had to be elsewhere and then was invited by Liz Carlile. The CCSL initiative seemed a bold one and I was curious to see what the people were up to and if my experience could be of any help.

Personally, I am specifically interested in the role of interactive learning in transformative change and ways to support such learning.

The line of my research revolves around bounded initiatives of groups who aim for system innovation. This means that consumption and production practices, social relations, norms, values, regulation and policy change together with a change of technological artefacts and physical infrastructures that have emerged with earlier unsustainable practices. An example would be a change from the current Dutch energy sector to a sustainable one in which renewable energy is produced by households, local citizen groups and local governments. Or, the sought for fundamental change of researchers’ roles in the CGIAR.

While acknowledging the limited contribution of single initiatives, the aim of my work is to study as well as enhance their significance and role in transformative change towards sustainability.

Specific topics are: the characteristics of learning processes that are relevant for system innovation; the role of communication in such learning, and the ability of bounded initiatives to contribute to transformative change and their interaction with the existing systems.

(Claire Hutchings)

I am the head of evaluation in Oxfam GB, and in particular, provide thought leadership on the evaluation of ‘hard to measure benefits’ including empowerment, advocacy, resilience etc.  I was asked to participate in the workshop by Oxfam’s Global Advisor – Climate Change Adaptation, Resilience, who oversees Oxfam’s work on the ASSAR project, as he felt that my experience supporting the measurement of similar projects may be relevant.  For me, social learning is most akin to Participatory Action Research, done well, and I have always been interested in this approach and in particular in the often unintended benefits it offers in terms of building the capacity of communities and individuals to analyse their situation and collectively problem solve, often not limited to the specific issue that the project may be focused on.

(Georgina Cundill)

I have been interested in the potential contribution of social learning theory to participatory research and collaborative management since about 2005. Between 2005 and 2010 I went through the rather disorientating experience of trying to monitor social learning in communities using the existing literature. Frustrations with the varied interpretations of what social learning was lead myself and colleagues to call for greater conceptual and practical clarity, and for empirical evidence from the field. This CCSL initiative is aimed at doing just that, and this is why I wanted to support your efforts. It is crucially important work.

(Richard Taylor)

My interest developed through working on M&E. Previously, colleagues of mine had projects using social learning and – though they were very enthusiastic about the approach – to an outsider I still felt it was a tricky area because there is so much literature on the subject, and it seems you have to be very immersed in your study to use social learning! Yet, the main drawback seemed to me to be the difficulty to observe and measure social learning. Then in 2010 two things happened that made my mind up for me – firstly my institute started using the Outcome Mapping (OM) approach for project planning and monitoring and I became a supporting ‘node’ for that, and secondly I started working on an FP7 project using social learning as one of the methods/approaches. Since then, we found social learning concepts increasingly useful for reflecting on the work of our group. An exampleis an article on ‘learning across locales’ describing the lessons drawn from  our work on weADAPT website.

In your (recent) personal experience and work, what has been useful to monitor or  assess social learning (tools, ideas, areas to focus on etc.)?

(Barbara)

In the past nine years, we have developed the Reflexive Monitoring in Action (RMA) methodology in close cooperation with practitioners. It aims to contribute to the emergence of new social practices and rules, by stimulating reflection and learning. It helps to monitor and evaluate progress ex durante. The guide can be found on: http://edepot.wur.nl/149471

RMA has a wide international uptake in among other areas, agriculture, natural resource management and health.

Some specific RMA tools are helpful to monitor learning, like the Dynamic Learning Agenda (see the guide) and a tool we are currently developing to define moments of learning in the regular meetings of innovation initiatives, rather than specificly-organized learning meetings.

The concept of system learning has shown to be helpful to assess learning relevant for system innovation. It goes beyond system thinking and stresses the importance of redefining barriers into opportunities and taking systemic actions.

(Claire)

As I noted at the workshop, I think that the group needs to get clarity on whether they interested in social learning as a process by which they can more effectively, perhaps more sustainably, realise particular outcomes – in this case related to Climate Change – or whether they are interested in social learning as a means by which individuals and communities can gain new skills that will enable them to analyze their situation and collectively problem solve.  If the former, then the focus will be on identifying indicators around some common interim and final outcomes around Climate Change that will enable you to check that these are materialising – monitoring the social learning process becomes about monitoring implementation.  If the latter, then the focus will be on identifying characteristics of community cohesion, capacity etc. that are explicitly about social learning outcomes, with the work on Climate Change simply a catalyst for building these.

(Georgina)

I have experimented with monitoring both the ‘background conditions’ that might support social learning, and the evidence that social learning is actually happening (based on Reed et al’s 2010’s definition). I have found the former useful for reflexive process monitoring that supports social learning, and the latter useful for improving our understanding of the opportunities and limits of social learning as a normative goal in participatory practice.

(Richard)

Before we started using OM I didn’t fully realise that we had approaches that could in fact measure social learning and make it more tangible through focusing attention to changes in behaviour. That is when it got interesting!  My institute went a step further to develop a custom-built web-based OM system called PMEC (Planning Monitoring Evaluation and Communication); read about it here.  So in 2010 I had a steep learning curve – both the OM concept and getting used to the computer system and helping with feedback to improve it.

One thing that happened is that some projects that were not set up with behaviour change in mind started to look a bit more like social learning processes because of the M&E planning (using OM), and this sometimes also generated new ideas for the research.

What is the next frontier for you in this domain? What are you interested in finding out more about?

(Barbara)

A major challenge is to explore further ways to translate new insights from action research into practical tools that are simple, yet effective in stimulating innovation initiatives to embark on innovative pathways.

A conceptual challenge is to disentangle the relations between reflection, learning and institutional change, rather than presume a positive, uni-linear relation between them.

(Claire)

I would be more interested in supporting some thinking about social learning outcomes, irrespective of thematic focus.

(Georgina)

We need a much stronger understanding of the implications of social learning theory for participatory practice. Not only for how we implement projects and engage people, but also for what kinds of outcomes we should expect when we go this route. I’d like to see us applying a more empirical lens to understanding these opportunities and limits through experience on the ground.

(Richard)

For me there is still much to learn about social learning (for example we had an interesting conversation about triple loop learning) and also how people are applying it in climate adaptation projects is even more interesting.  Institutionally, we are now starting to see the results of the hard work that went into PMEC. For example, the ability to generate reports from our data. To use visualisation software to map who is working on what across the institute. But there are new things to consider. For example, a challenge is linking our internal system to external communications to improve efficiency in getting information out of the system (and avoid replication of work!), so I would also like to know how others plan to communicate the lessons of social learning.

Read workshop notes and related materials

Discover some additional materials mentioned by these specialists:

Social learning for adaptation: A handbook for practitioners and action researchers:

http://weadapt.org/knowledge-base/transforming-governance/social-learning-for-adaptation

Documentary on our use of theater for transformation in a social learning process

http://vimeo.com/user22953453/videos

Social learning publications:

Cundill, G., Lotz-Sisitka., H., Mukute, M., Belay, B., Shackleton, S., Kulundu, I. 2014. A reflection on the use of case studies as a methodology for social learning research in sub Saharan Africa. NJAS. 69: 39-47 URL: http://www.sciencedirect.com/science/article/pii/S1573521413000067

Cundill, G. and Rodela, R. 2012. A review of assertions about the processes and outcomes of social learning in natural resource management. Journal of Environmental Management. 113: 7- 14. URL: http://www.sciencedirect.com/science/article/pii/S030147971200429X

Rodela, R., Cundill, G. and Wals, A. 2012. Methodological underpinnings of social learning research in natural resource management: a review. Ecological Economics 77: 16-26. URL: http://www.sciencedirect.com/science/article/pii/S0921800912000961

Cundill, G., Cumming, G., Biggs, D. and Fabricius, C. 2012. Soft systems thinking and social learning for adaptive management. Conservation Biology. 26(1): 13-20. URL: http://onlinelibrary.wiley.com/doi/10.1111/j.1523-1739.2011.01755.x/full

Cundill, G. 2010. Monitoring social learning processes in adaptive comanagement: three case studies from South Africa. Ecology and Society 15(3): 28. [online] URL: http://www.ecologyandsociety.org/vol15/iss3/art28/

Reed, M. S., A. C. Evely, G. Cundill, I. Fazey, J. Glass, A. Laing, J. Newig, B. Parrish, C. Prell, C. Raymond, and L. C. Stringer. 2010. What is social learning? Ecology and Society 15(4): r1. [online] URL: http://www.ecologyandsociety.org/vol15/iss4/resp1/

Cundill, G. and Fabricius, C. 2009. Monitoring in adaptive co-management: Towards a learning based approach. Journal of Environmental Management, 90: 3205-3211. URL: http://www.sciencedirect.com/science/article/pii/S0301479709001510

Shackleton, C.M., G. Cundill and A.T. Knight. 2009. Beyond just research: experiences from southern Africa in developing social learning partnerships. Biotropica, 41(5): 563-570. URL: http://onlinelibrary.wiley.com/doi/10.1111/j.1744-7429.2009.00559.x/abstract