The title is a quote from Oscar Wilde, arguably the creator of truisms so true that they now feel clichéd. This particular quote has taken on a new resonance for me recently, as I have been thinking about how we might ever capture the ‘true value’ of our national improvement work.
Another ‘truism’ I have come across amongst many who work in national improvement is that a huge amount of what is achieved is ‘invisible’, ‘immeasurable’ and ‘intangible’. It cannot be reduced to a single number, or a matter of pounds and pence. Many of us feel this to be true. We can see hints of our value in the testimonials and case studies we have collected, and we know this doesn’t capture the half of it. But you can forgive the cynicism of the uninititated. Language like ’invisible value’ smacks to many of the Emperor’s New Clothes, and nobody wants to be taken for a fool. So, against an ever more challenging ask of delivering financial balance in the health and care sector, do we commit to know the price of everything and risk missing the real value of our improvement work?
If we do, we might fail to see the value in things which are incredibly important, no longer investing in them as a result, with the damage only being seen some way down the line when it’s too late. How can we make sure that the value of our work is clear to all? And how can we be sure that we are delivering value in all our activities, even when our role is enabling others to meet their improvement goals, to build their own improvement capability, or to connect with others in the system?
Evaluation is almost always HARD to do well. Siobhan Campbell (Chair of the Cross-Government Evaluation Group) set out some of the huge challenges around policy evaluation in a frank plenary to the Social Research Association annual conference in December 2016 (click here for her presentation). Add in an improvement angle and it becomes even more complex:
Pressure on budgets brings a keen focus on monetised value and justifying spend, but Return on Investment ratios and other traditional approaches can often ignore the less ‘tangible’ value of our work
Proving that outcomes are a result of our actions is almost impossible when operating in complex systems
We are often enablers of change – working with and through our partners and it is therefore difficult to capture full immediate impact
System change takes time - much more time than our reporting cycles generally allow for
We often lack internal expertise and confidence to take on these challenges
People are passionate about improvement! Great, but this often leads to them taking action before designing in evaluation, and asking questions about impact too late
The Sustainable Improvement Team at NHS England has been giving this some serious thought. Whenever we have raised this with others (e.g. Expo 2016, SRA annual conference 2016) it has been clear that we are not alone in struggling with some of these challenges, and there are certainly no silver bullets on offer. However, in response we have started to come up with some underpinning principles for how we might most helpfully think about our value, and a practical approach that we think might help, borrowing heavily from the worlds of developmental evaluation and the experiences of others in tackling this problem.
Be clear about what you are trying to achieve, and understand what changes (intended and unintended) you expect to see
There is no one absolute measure of value - focus on creating a ‘good enough’ account of value at a point in time
Judgements of value can (and should) be made by those involved, not only by 'objective' third party evaluators
Involve stakeholders in defining and evaluating value at meaningful intervals
Be honest about contribution and attribution - we will hardly ever prove causality, but let’s have a meaningful conversation about our contribution
Don’t dismiss the elements of value that are harder to capture
Keep it simple and transparent
Focus on learning, not just accountability
Our Emerging Approach
What do you think?
Do you have a similar or alternative approach?
Can we improve on our approach?
Could an approach like this work for you?
Join our Community of Interest to find out more, access our resources, help us refine this work and get it into practice. It’s open to everyone and you can request to join here.