Lived experience and decision-making: an applied approach to using, connecting and representing qualitative evidence

The challenge

The LGA’s own Cornerstones of Culture report promotes the idea of ‘resilient place’, evoking the importance of place attachment and civic pride to communities. This language is consistent with policymakers’ renewed emphasis on people’s lived experience in places. Yet generating evidence for this lived experience—for feelings, connections, and values—is not easy. Quantitative data can be reductive, and qualitative data is often difficult to aggregate.

It is therefore unsurprising that the report identifies ‘data and evidence’ as one of the six barriers facing local, public-funded culture. The authors note that data is important for targeting investments, for valuing economic contributions, and for developing shared languages and methods in partnership. The report then translates these barriers into three key data needs: 

• standard core data that could set effective benchmarks; 
• consistent evaluation methods that could understand the impact of cultural activity on communities; 
• and consistent analytical models that could guide decision-making. 

The Neighbouring Data project, based in the Southampton Institute for Arts and Humanities (SIAH), responds to these challenges by focusing on the use, connection and representation of qualitative data. We are particularly interested in how lived experience exceeds data silos: how it brings into conversation data about neighbourhoods with neighbouring data on culture, place, pride, health and wellbeing. This article outlines the potential role of qualitative data in understanding lived experience and the wider opportunities it presents for place-based decision-making.

Data opportunities

Our research found that qualitative data provides four important opportunities for improving place-based decision-making: 

One-- it enables rich, detailed and surprising insights into the relationships between culture and communities: it tells local authorities what communities value, and why. It provides the types of insights that survey data, for example, never could. 

Two-- it crosses and transcends the service silos of local authorities: carefully gathered qualitative data can make meaningful connections between health, culture and inclusion, for example, to assist with investment decisions. 

Three-- it produces stories that provide local authorities with opportunities to meaningfully connect with their communities, in ways that prove they’ve been heard and responded to. 

Four-- it is important because funders and commissioners require it: the expectation to increase ‘pride in place’, for example, was embedded as an ‘exploratory metric’ in the technical annex of the Levelling Up White Paper.

Drawing on our previous studies of place, culture, pride, and wellbeing, Neighbouring Data examined how qualitative data could be used, connected and represented. It included three main strands: 

• First, we explored data experiences and highlighting the challenges of using qualitative data, drawn from interviews and focus groups with local authorities;
• Second, we examined data observatories, interrogating existing models for organising, accessing and sharing qualitative data; 
• Third, we collaborated with creative practitioners on experiments that tested new approaches to the visualisation, narration and maintenance of qualitative data.

Findings for local government

Our work on data experiences revealed the diverging and messy models that exist for qualitative data. There is repeatable but expensive qualitative data collection: this evidence is obtained using standard methods, such as targeted, semi-structured interviews and focus groups. There is artefact data generated using creative methods: this data might include audio, visual, literary and written material. There is also a significant amount of what one local authority officer neatly described as ‘vibes’. This category includes the ocean of qualitative information—social media, doorstep conversations, open comments on surveys, graffiti, and emails—in which local authorities swim. 

Our first finding concerned support for local authorities in recognising, characterising and navigating these different forms of qualitative evidence. One, they need more capacity and skills to deliver rigorous, targeted qualitative approaches. Two, they need to implement strategies for using the surfeit of existing and ‘found’ qualitative data. Existing models for data observatories, both actual portals and theoretical interventions, suggest approaches for dealing with this variety of qualitative material. 

Our second finding emphasised the importance of developing core and standard protocols for data biographies. Catherine D’Ignazio explains that data biographies are ‘stories of how the dataset came to be in the world’. Instead of following typical data analysis—where you acquire a dataset and work forwards to ascribe meaning to the data—a data biography requires describing how the data came into existence before you analyse it. This process does key work because it demonstrates the provenance of data, it foregrounds the limitations and partiality of data, and it provides opportunities to make the data searchable and connectable. 

Our third finding is that the layering of data found in environmental observatories, including categories such as ‘hard/official’ and ‘soft/citizen’, produces new dynamic connections between sources and importantly helps to establish the provenance of data. We think that crowd-sourced, citizen qualitative data is rich in potential as it provides capacity for councils: it contributes to community engagement and democratic participation in ways that improve the usefulness of qualitative data itself. Co-produced community data is important because it reinforces the connections between cultural data and other forms of data, a process which accurately reflects that many assets and services are dependent on one another. We therefore use the term neighbouring data to refer to different communities and to different types of adjacent and related data.

Our fourth finding acknowledges the significant challenge of using, connecting and representing the vast range of qualitative data forms. We think that there are vital understandings to be gained through this variety of data, but questions of accessibility, ethics and data ownership complicate known issues of capacity, sustainability and data literacy. Our experiments have suggested models that can visualise the connections between datasets. Modes of presentation should preserve the richness of qualitative data, articulating its purpose and offering comparisons with other data. Visualisations can make qualitative data understandable, compelling and sustainable.

Empowering councils

For the Neighbouring Data project, we invited creative practitioners to find ways of connecting different types of qualitative data, including narratives, images, poems, videos and maps. This data was drawn from research collected across different University of Southampton projects working in and with communities across the UK.

With one creative practitioner, we developed a model for using qualitative data called the Neighbourhood Insight Engine. This model deployed a hypertextual approach to different datasets, making connections between a range of qualitative responses, which enabled non-linear and non-hierarchical reading experiences.

We then tested the Neighbourhood Insight Engine with community groups, arms-length bodies, local authorities and policymakers, who saw the potential that the model offered to explore data as part of the decision-making process.

The Engine responds to a key concern in decision-making. Data literacy is often understood as the need for communities to understand data that is presented to them, so that they can respond to decisions that have already happened. The Engine flips this process: it encourages communities to have an active role in using, connecting and representing the data that informs decision-making.

This work offers opportunities for local authorities to engage more substantially with communities. As survey respondents, citizens are limited to providing snippets of fragmented data. Instead, the Engine indicates a much more robust tool for citizen engagement. Local authorities should note the parallel conversations on social media data, which embrace digital literacy not only as reading data but as working with data. 

The Cornerstones of Culture report understands the wider community ‘as participants in and leaders of culture’. It references, too, the need for a ‘community voice’ and to ensure ‘that a diverse range of voices are heard when decisions are being made’. To achieve these aims, we need: new ways of using qualitative data about lived experiences; support and recognition for communities to work with, understand and represent qualitative data; and innovative approaches for connecting qualitative data about lived experience with other, neighbouring forms of data.

Next steps

We recognise the need for co-produced protocols and guidelines that can inform the use, representation and connection of qualitative data. We know that qualitative data gives insights into lived experience, and we understand the clear challenges that this data offers the sector. We are therefore developing principles and tools that provide new ways to use qualitative data about people’s lived experiences, in partnership with the communities that generate this data and the citizens who have these experiences.