Measuring the impact of what we do is becoming increasingly important for the arts sector, especially when our work crosses over into the realms of education, health and well-being. Everyone’s budgets are being squeezed, so spending is increasingly scrutinized and targeted. Arts organisations are beginning to dip a toe into the unfamiliar world of commissioning where the more experienced players have reams of hard data at their fingertips about the effectiveness of their approach. We (in the arts) tend to have less of this – although not necessarily because we are poor at evaluating our programmes. For many years high quality project evaluation has taken place, and it is often required by enlightened funders such as the National Foundation for Youth Music who have pushed an outcomes focussed approach in order to motivate the sector to think about what it is doing and why. But demonstrating impact in the arts is often more complex than a robust evaluation: the criteria for demonstrating real change requires us (i) to know where we are starting from, (ii) to have a structured and defined programme of activity for the target group, and (iii) to be able to identify the arts activity as the cause of any change in the participants (ie it usually means needing some sort of control group). Each of these three elements is full of challenges, especially for small and medium-sized organisations with limited funding to spend on expensive research-led evaluations.
So, as a sector what can we do? First, don’t panic! We don’t all have to become expert evaluators, or engage expensive consultants, overnight. Lets remember what the arts is brilliant at: moving people, telling stories, engaging people, and finding creative ways of meeting challenges...
So, second – use the power of well-presented feedback from our participants to tell us about what’s changed for them as a result. This anecdotal evidence may not be quantitative – but its power should not be forgotten, especially if presented in a professional and creative way.
Third, don’t try and evaluate everything. It will be more useful to get some good quality evidence for a project’s impact on one or two areas of a participants’ life than to show in a vague way how it has impacted on every single part of their existence.
Fourth, if you can’t generate hard evidence yourself, use someone else’s. There is plenty of high-quality academic research out there about the power of the arts to positively impact on all sorts of areas of life. Much of this can be accessed freely. Of course you need to acknowledge your sources – but doing so will also demonstrate that you are aware of the research that is relevant to your work.
Fifth, don’t promise what you can’t deliver. There is a temptation to say that the arts can solve the world’s ills – and although there may be evidence that supports the notion that some type of arts programme may help with some specific problems over a certain period of time, this might not apply to any arts project solving any problem. There are interesting discussions at the moment in the music sector led by London Music Masters, Pro Bono Economics and Sound Connections around impact and measurement. It is something the sector needs to (and is beginning to) take seriously in a landscape that is increasingly based on the commissioning of services to meet key outcomes. The balance between shouting about the ability of the arts to help with literacy, numeracy, school attendance (etc) must be balanced with the importance of maintaining the validity and worth of art in its own right. Because if we put all our eggs in the “do music because it’ll help you with X” basket – and it fails to live up to our claims, or to the success of a non-arts approach – then the arts will be firmly off the commissioners’ radar for a long time to come.
Sixth, work together more, following the example set by the organisations meeting with London Music Masters. Lets be brave and open and tell other organisations in the sector about what we’re doing, what we’re trying to do, and whether it worked or not. Lets learn from each other – at a practitioner but also organisational level. Lets pool our resources, share our data and reports, and if necessary find some common means of measuring key outcomes which many organisations could use, in order to generate a statistically significant body of evidence of the impact of the life-affirming work we do in the arts sector.
Creative Futures’ first Impact Report, through which we’ve tried to summarize, describe and quantify the impact of our programmes during 2014-15, is published this week. We will be working on refining and strengthening our reporting mechanisms in coming years.