Benchmarking resident satisfaction data

Understanding resident or customer views is a key element of assessing the effectiveness of an authority, alongside cost and performance information. Furthermore, understanding resident satisfaction and being able to make informed comparisons can strengthen local accountability and be a key part of the sector’s approach to managing its own performance.

The LGA conducts regular national telephone polling of resident’s satisfaction with local government, three times a year. In addition, regional level results are available on a yearly basis. This guidance outlines a set of resident satisfaction questions that councils can use in their local surveys, and guidance on the conditions that must be met, to enable the results to be benchmarked against these national and regional results.

This is a flexible approach, designed to suit local circumstances and minimise the cost to councils of collecting the data. It allows councils to use only those questions that are of interest locally and fit these questions in with existing survey plans, whilst also helping to maximise the benefit and insight that can be gained from this data by making it broadly comparable.

The questions were developed in consultation with the sector and with assistance from Ipsos Mori, including cognitive testing. In addition, the document contains additional questions which are not part of the benchmarking, but which councils may find useful to use in their own local surveys.

This guidance has two elements:

• A ready to use question set which is cognitively tested, developed in consultation with the sector and with advice from Ipsos Mori. These questions can be used by any council using any survey method, regardless of whether they would like to benchmark these or not.
• For councils conducting general population telephone surveys and who wish to compare their results, a set of criteria that would need to be met to ensure that these questions can be robustly benchmarked against the LGA’s national and regional comparator figures.

It is important to note that this is a guide to benchmarking and not a guide to conducting a resident survey. Councils should follow standard recognised good practice when carrying out surveys locally.

The full set of questions can be found below. They cover the following areas:

• satisfaction with the local area as a place to live
• satisfaction with the way the council runs things
• value for money
• council responsiveness
• how well informed the council keeps residents
• advocacy
• community identity
• community safety
• trust in the council
• satisfaction with services
• community cohesion
• anti-social behaviour.

Benchmarking of these results should be part of a wider approach to understanding and responding to local communities. Benchmarking provides context but is only one element of this approach and helps raise lines of enquiry rather than providing answers.

Councils should seek to include additional questions in their local surveys that might help diagnose what is driving their results locally. For example, questions on what needs improving in the area might be of limited use for benchmarking but could usefully be included locally to identify areas for improvement.

The criteria laid out below describe the conditions that would need to be met for a council to be able to make a meaningful comparison of its survey results against the national and regional LGA polling figures.

These are based on the findings of a review by Ipsos MORI, which identified a set of conditions that would need to be met to ensure that any comparisons being made are robust, fair and valid, and that any data is of a good quality that will stand up to scrutiny. The criteria were then refined based on an extensive consultation with local authorities. 

Note: The question set and guidance were initially developed as part of a project to enable council to council benchmarking (called ‘Are You Being Served’). The following document summarises and addresses the key issues raised by consultees and explains the changes that were made to the guidance as a result of the consultation: Residents' satisfaction: consultation response. Whilst the primary purpose was to look at council to council benchmarking, most of the document remains relevant to national and regional benchmarking, and it also gives background on how the questions were developed. 

It will be important to refer to this guidance right at the outset of planning any local survey activity, as some of the requirements will influence early decisions about issues such as sampling and data collection method.

Data collection methods

The LGA’s national and regional figures are obtained using a telephone poll. As the mode of data collection can have a marked impact on results, intra–mode comparisons are not desirable. Therefore, we recommend that only councils using a telephone survey method should benchmark against the LGA figures.

Sampling

Only surveys that use all adult residents as the sample frame are appropriate for benchmarking (i.e. general population surveys). Service, topic or area specific surveys that only draw a sample from a subset of the general population will not be appropriate for benchmarking.

Two types of sampling will be suitable for benchmarking, in addition to a census approach:

• Random sampling (whereby all population members/households have a random one in ‘n’ chance of being selected) is favoured for all methods of data collection.
• Quota sampling is also acceptable.

Authorities that have obtained satisfaction data through residents’ panels will not be able to benchmark this. This is because there is strong evidence to suggest a considerable response bias among panel members, which makes it inadvisable to compare data collected via this method with general population data.

Weighting

Where a random sampling or census approach has been used, data should then be weighted to the known profile of the local population. Variables that are typically used for weighting in residents’ surveys include age, gender and social grade (or work status as a proxy for social grade). 

This guidance does not specify exactly which variables the data should be weighted on. The purpose of weighting is to make a sample representative if it is not already. What is appropriate will depend very much on the way the survey was sampled and on local circumstances. 

Don’t know and refuse to answer

For benchmarking, it is important that the ‘don’t know’ and ‘refuse to answer’ options for each question are dealt with in the same way between surveys. Therefore councils who wish to compare their survey results to the LGA figures will need to take the following approach:

• Do not include an explicit reference to ‘don’t know’ or ‘refuse to answer’ in the answer option list.
• The interviewer can however code these answers if they are given spontaneously.

Question ordering

Testing has identified that respondents’ answers to certain perception-based questions may be influenced by questions asked earlier in a survey (this is known as ‘context’ and ‘position’ effects). For example, if a question about general satisfaction with an area is asked after a series of questions about crime, this is likely to elicit a more negative response than for a different authority that preceded this with questions about parks and leisure. Therefore, for each question, advice on question ordering is given for councils who wish to compare to the national and regional polling results.

Full question text

This section outlines the full text and answer options of the questions, as well as guidance on the modifications that would be needed in two tier areas and any definitions that should be included. For each question, it is also stated whether an LGA benchmark figure (national and regional) is available.

Definition of local area

Many of the questions in this set ask respondents about their ‘local area’. Please include the following text at the start of your survey to ensure that this is being interpreted consistently:

“Throughout this survey we ask you to think about ‘your local area’. When answering, please consider your local area to be the area within 15 – 20 minutes walking distance from your home.”

Question 1: Satisfaction with the local area

Introducing the council

The following preamble should be inserted before the first question that asks specifically about a named council. A slightly adjusted preamble will be used depending on which scenario your survey fits:

• In the case of a questionnaire designed to understand perceptions of a single tier council:

“Your local area receives services from [name of council]. [Name of council] is responsible for a range of services such as refuse collection, street cleaning, planning, education, social care services and road maintenance.”

• In the case of a questionnaire in a two-tier area, designed to understand perceptions of a district only, or a county only:

“Your local area receives services from two councils, [name of district council] and [name of county council]. EITHER This survey asks about [name of district council], which is responsible for services such as refuse collection, street cleaning and planning OR This survey asks about [name of county council] which is responsible for services such as education, social care services and road maintenance.”

• In the case of a questionnaire designed to understand perceptions of both the district and county:

“Your local area receives services from two councils, [name of district council] and [name of county council]. [Name of district council] is responsible for services such as refuse collection, street cleaning and planning. [Name of county council] is responsible for services such as education, social care services and road maintenance.”

Question 2: Satisfaction with the local authority

Two tier areas: If this questionnaire is being conducted by a district and county in partnership, repeat the question.

Question 3: Value for money

Two tier areas: If this questionnaire is being conducted by a district and county in partnership, please use this slightly different introductory text:

In considering the following two questions, please think about the range of services that both [name of district council] and [name of county council] provide to the community as a whole, as well as the services your household uses. It does not matter if you do not know all of the services [name of district council] and [name of county council] provide to the community. We would like your general opinion.

Then ask ‘To what extent do you agree or disagree that [name of council] provides value for money?’ twice.

Question 4: Council responsiveness

Two tier areas: If this questionnaire is being conducted by a district and county in partnership, repeat the question.

Question 5: Informed about the council

Two tier areas: If this questionnaire is being conducted by a district and county in partnership, repeat the question

Question 6: Advocacy

Two tier areas: If this questionnaire is being conducted by a district and county in partnership, repeat the question.

Question 7: Community identity

Question 8a: Community safety after dark

Question 8b: Community safety during the day

Question 9: Trust in the council

Two tier areas: If this questionnaire is being conducted by a district and county in partnership, repeat the question.

Question 10: Satisfaction with services

Note: This question was not included in the original ‘Are You Being Served’ technical review but is now included in the LGA polling.

Question 11: Community cohesion (ethnicity)

Question 12: Community cohesion

Question 13: Anti-social behaviour

If you have a query on any aspect of this guidance, please email [email protected].

Councils might want to get involved with their regional Local Area Research and Intelligence Association (LARIA) group, to provide mutual support and discuss the technical issues associated with conducting resident satisfaction surveys: https://laria.org.uk/. LARIA also has a Knowledge Hub (community of practice) group: https://khub.net/group/lariagroup.