The Blagrave Trust is funding a study of outdoor education.
Here is a blog on the study written by Giving Evidence, who along with UCL will be conducting the research.
Systematic review of evidence to inform funding practice
“What is known about what works and what doesn’t? What can we learn from the existing literature and experience of other organisations about what works and what doesn’t – and for whom and in what circumstances – which can help us make better funding decisions?”
These questions are the genesis of a study of outdoor education for 8-25 year olds commissioned by the Blagrave Trust, a family foundation which supports disadvantaged young people in southern England. Giving Evidence is working on the study in partnership with the EPPI Centre (Evidence for Policy and Practice Information and Co-ordinating Centre) at UCL, which routinely does systematic reviews of literature in particular areas to inform decision-makers.
We’re excited to do this project because those original questions, and this way of answering them, pertain to any area of charitable funding, service delivery or social policy. The charitable sector focuses a lot on doing monitoring and evaluation – i.e., producing research – but is weirdly less unconcerned about using research already created by others. We’ve written about this before, and will do so again: using systematic reviews of the existing literature could save a lot of time and money and significantly upgrade performance. There seems to be appetite in the outdoor learning sector to hear and heed the findings from research.
Study aims and logic
This particular study will:
– Categorise the various outdoor learning activities in the UK, in order to give funders a more coherent sense of the sector as a whole and see their options;
– Identify the various outcomes which organisations running those various activities are measuring, i.e., what providers seem to be seeking to achieve; and
– Assess the designs of individual evaluations and the standard of evidence offered in total for different types of outdoor learning.
Such a study can be very valuable. Most obviously it can guide providers and funders to the most effective interventions.
It can also guide the research effort within the sector, and hence reduce research waste. In many sectors, donors and operators collectively spend a lot on research but with no co-ordination about where it is spent. Often the spend goes where activity is the greatest. By contrast, it is better to identify areas which most need additional research. That depends in part on the ‘evaluate-ability’ of the various interventions, i.e., whether they are ready to be evaluated. This is useful because very often, NGOs and others evaluate work:
– before it is ready: when it’s still in a pilot stage and too unstable for an investigation of its causal effects. Pilots should just be monitored and the beneficiaries consulted. And/or
– after evaluation ceases to be useful: because the intervention has been adequately evaluated and its effects known. By analogy, we don’t evaluate all medical drugs forever: eventually we’re pretty confident that we understand a drug’s effect and the trials stop.
The priorities for research are interventions which are stable enough to evaluate, and of course in which enough people are interested, i.e., where the research findings could influence enough activity to be worthwhile.
Third, by assessing the quality of existing research, we hope to raise awareness of research quality – i.e., that some types of study are more reliable than others. Poor quality research normally means that the design doesn’t allow researchers to distinguish the effects of the programme from other factors and from chance.
Both these problems – evaluation at the wrong time, and bad evaluation – are major causes of research waste.
Timing and getting involved
We expect to work on this until the Autumn. We will then publish the findings and work with the Blagrave Trust and others to mobilise players in the sector around them.
We’d love to hear from you if you have studies about outdoor learning in the UK which looks at the causal connection between activities and clearly-stated outcomes. Please send to admin@giving-evidence.com