Insight and evaluation: Do your communications really work?

In a world of complex and ever-changing media use, where distrust and cynicism of authority is rising and the vocal few are riding a wave of populism, how can you be sure your communications really work? Neil Wholey, Member of the Government Communications Service Evaluation Council, explains.


Future Comms digital banner

Key points

  • Use an evaluation framework and create a culture of insight evaluation across the organisation (not just in communications).
  • Be ambitious and don’t confuse SMART objectives with strategy.
  • Use insight and evaluation to ensure your organisation and community work together.

Confidence from evidence

Bridging the perceptions gap between the self-evident common sense of the public and the self-evident professional sense of those running public services is a key challenge local councils face.

Excellent communications brings your organisation and community together. It cannot deliver alone and must work with policy, service delivery and the community itself. Insight and evaluation provides the support to enable you to set the right strategy, tackle difficult objectives and give you confidence that delivery and change is happening. 

How does this work? After all great communications is surely just common sense and not difficult. Anyone can do it. I’m sure you could write a half-decent tweet or press release. You’re also clever enough to see through any communications that is too glossy or creative as the superficial spin it surely is. You know what good communications looks like and don’t need some fancy evaluation to try and pretend some grand strategic impact. Just get the right information out there, make sure there are no typos, and stop any negative press coverage?

But are you sure? Considering the way that reputations, careers and organisations can be destroyed by bad communications how lucky do you feel? Can you really rely on your own gut instinct of what works and the reassurances of those in the communications team?

You probably know more than most about what your council does and how well it is performing. The people you serve don’t. They just want you to get on with the job. After all running a local council is just common sense and not difficult. Anyone can do it. Keep the streets clean, mend potholes, give housing to the right people and don’t waste money on silly things such as communications. The public can see through your bureaucratic council-speak waffle and the bland puff press releases that sell your own self-importance. Just tell the truth, don’t make any mistakes, and listen to what the people are saying.

The nub of the issue is how do you help the public understand and scrutinise a complex world to help it evolve, rather than reacting against it with anger? And how do you encourage your organisation to actively reduce complexity and simplify the world, rather than reacting with frustration or distain at the idea of doing so? The simple answer is to use insight and evaluation to drive change.

Evaluation theory emerged to help decision-makers deal with a complex world. It is not specific to communications; rather it seeks to gather evidence together to give you confidence that interventions into complex problems are working. The language of evaluation will be familiar, as it helps to tell a story. The GCS Evaluation Framework sets out such a framework clearly for the public sector, and there are more resources available from the International Association for the Measurement and Evaluation of Communications (AMEC). 

Or if you want to inspire the troops think about General Montgomery’s one page battle plan for D-Day. It is a clear to-do list of activity, but also sets out a clear timing and order. By delivering against this plan he would be given the reassurance that the overall objective was being met.

Let’s set out a clear but difficult objective; we will bring the community and council together to address the budgetary and service challenge we face together. It’s hardly a SMART objective. We can accept it is strategic as the impact of getting it right is huge, but how would you measure it and is it achievable, relevant or timely?  

This is why I’m not a big fan of SMART objectives to drive strategy. They tend towards a clear manageable deliverable; we will do X by Y and Z will happen. This can crush the life out of a difficult ambitious objective by reducing it to what is perceived to be easier or has supposedly worked before. SMART objectives have to be part of an evaluation framework.

With an evaluation framework we start with the inputs. This is the planning stage where we try and understanding the issue in hand. We gain insights by talking with people about the broader context. This is not just a conversation within your office but across your organisation and the community. What is the challenge? What would an improved situation feel like? What needs to change in terms of everyone’s behaviour and attitudes? What is the most ambitious thing you could do? What are the limitations? What budget do you have? Who has different perspectives on things? What has worked elsewhere?

From this you are unlikely to identify just one brilliant idea that solves all your problems. A complex problem is likely to have a range of possible solutions. Some you think will work, others you are less sure of, and some you worry will cause more damage than good. With evaluation you weight up the risks. You create a portfolio of a range of different actions, some will be communications but others could be changes to service delivery or an aspect of policy. You create a vision that logically brings them together, a sense of a shared mission and a campaign to achieve the overall objective.

Communications at its very best is highly sophisticated in understanding needs and deploying just the right activities to achieve overall objectives, no matter how ambitious. But it is also brilliant at taking that sophistication and translating it into a vision and campaign that is easy to understand. It helps you lead your organisation and engage with your community.

This then leads us to the agreed outputs, your planned activity. With excellent planning at the input stage you are fairly confident that these outputs will work, or at least know the risks involved. Volume and quality do matter to give you some reassurance. You can’t say we have this huge mission and then issue only a couple of tweets. But similarly you can’t just rely on volume and just do lots of stuff and hope some of it sticks.

For each of your outputs you need to think how you will measure the outtake, how people feel as a result, and outcomes of what they do as a result. This is the bit where you put your SMART objectives, in analysing individual elements of your overall campaign. This can be through a combination of different bits of evidence. Have you received positive comments from key influencers in the community? Did you get positive feedback from a short survey done at the end of an event? Did they sign up to a call to action? Has service use changed? What have you learnt from the activities?

I could wax lyrical about the importance of statistical reliability and robust evaluation, but my advice is to focus on evidence that gives you and senior decision-makers confidence and is proportional.

You need to create a culture where everyone is gathering evidence, big and small, on what is working or not. Let them be inquisitive, let them be proud and share experiences and insights and let them change and innovate based on what they are learning.

The robustness of evaluation will improve as more people positively challenge and learn from it. The evaluation framework brings it all together by building collective confidence that actually this is working and that the overall objective can be achieved, no matter how difficult.