Innovative Approaches to Evaluating Impact: Testing The ‘Evaluative Performance’ Method and Approach in Transdisciplinary Environments

Project team

Dr Charlie Ingram (PI)

Collaborators

Coventry University (lead)
Royal Holloway, University of London (Project partner)
University of Amsterdam (Project partner)
EnergyREV
Coventry Biennial
Festival Republic

Duration

1/4/23 - 1/4/25

Project overview

Recent developments such as the global pandemic, gender violence, Black Lives Matter, and not least the climate crisis, have accelerated the search for participatory methods of engagement, participation, and impact across a number of research disciplines.

Within these disciplines, citizen voice and lived experience are sought to deepen understanding behind behavioural change as a result of policy or project intervention strategies. Within this there are limitations, for example, the UK Civil Service in 2021 recognises that ‘most policymakers do not consistently have the skills, incentives, or infrastructure to find new evidence about citizens’ (Knight, 2021). It is also argued that policy solutions are too often based on stakeholder opinion, and rarely are the voices and experiences of citizens used to evidence policymaking (UK Civil Service, 2020, 63). Therefore, policy engagement methodologies as the basis of evidence gathering are themselves generating ‘marginalised groups’, ‘the seldom heard’, and creating barriers to access for citizens.

This process began with my development of a new practice research methodology – ‘Evaluative Performance’ – and its use within Coventry City of Culture 2021. It’s successful reception suggests potential, and this will be my project’s intellectual and practical focus. In particular, this research project will test ‘Evaluative Performance’ and its ability to evaluate impacts associated with intervention projects of various scales and disciplines. Specifically, how arts-based methods of data collection, analysis and dissemination can contribute to the need for valuing the voices of citizens in policymaking, and a variety of impact strategies.
‘Evaluative Performance’ is the use of headphone verbatim practice as a participatory way to capture the needs and experiences of those ‘on the ground’ that are hard to reach by policymakers and impact evaluators. A key unique identifier in this method, is the inclusion of the research participant in their own representation in performance. For reference, headphone verbatim is a sub-genre of verbatim theatre practice whereby ‘in both rehearsal and performance, the actors wear headphones, through which they hear the audio script. They then repeat that script as immediately and exactly as possible, including… every stammer, pause, and repetition’ (Wake, 2013).

Building on my own networks, this project will run ‘Evaluative Performance’ investigations in different research settings: Contemporary art, engineering and computer science, equality and diversity initiatives and anti-social behavioural interventions at live events.

Project objectives

To document and evaluate the ‘lived experience’ of citizens and project beneficiaries as a result of intervention projects of various scales, types, and fields of influence
To test by way of practice research, the ‘Evaluative Performance’ research and evaluation model as a method of measuring impact and its applicability to multiple disciplines
To disseminate findings of this study with academic and non-academic audiences via headphone verbatim theatre performance, and review how these dissemination strategies are received in both rehearsed and non-rehearsed scenarios
To provide evidence for arts-based participatory research methodologies to be used in impact evaluation professionals and policymakers outside arts and culture disciplines

Impact statement

Implication and impacts of this research are expected to benefit academic and non-academic audiences. This research will contribute to multiple areas of scholarship including understanding impact, evaluation methodologies, policymaking, citizen science, behavioural studies, theatre and performance studies, cultural studies, and practice research. The emphasis on performance outputs, conference presentation and journal publications will enable the research to have benefits for the wider academic community, while policy briefs will benefit those working in policymaking and evaluating project impacts. This research will raise the profile the ICC and that of a new research centre, showcasing it as a spearhead of a progressive, forward thinking, impact focussed institution of research.

Participants engaging in case study projects are estimated to benefit from this research. Through a participatory method of evaluation, participants will be able to feel like their voice is being heard at a time when policymaking is failing to listen.

Theatre makers stand to benefit from this research, as the project aims to make a case for a more participatory approach to evaluation practices, which, if taken up post-project, a wider array of artists will gain creative opportunities through developing research partnerships. An example of this successful relationship, and the impacts thereof can be seen from the Coventry Creates project in 2020, whose success has allowed it to continue for a second funding round.

Policymakers are anticipated to benefit from this research through the development of several outputs that will demonstrate the contribution theatre arts can make towards current challenges to capturing the citizen experience, and further recommend ways to provide policymakers with methodological advances that break down previous barriers to citizens.

Outputs

Two peer-reviewed journal articles – continuing to develop my publications portfolio. Recommended journals are the Journal of Research Practice, Theatre Topics, the International Journal of Cultural Policy, and European Urban and Regional Studies among others. Further, there are joint-author collaboration opportunities on this project with Dr Samia La Virgne from Royal Holloway University
At least two conference presentations – online or in person participation as a presenting researcher, building from already established relationships with IFTR and the Southampton Institute for Arts and Humanities. This will take place either at a policy focussed or a theatre and performance focussed conference event such as TaPRA, the International Behavioural Public Policy Conference and the Diversity, Equity and Inclusion Conference
Briefing paper – produced for the benefit of policymakers, a briefing paper will provide contributions to the professional sector, thereby encouraging changemaking at an institutional level. This paper will be based on research findings and provide advice to break down barriers to access citizens in policymaking
‘Evaluative Performance’ workshop presentations – these performances will be documented and be distributed alongside all other outputs mentioned above, further corroborating research findings through practical demonstrations of methodological practice
At least one collaborative research funding bid – Through engaging with the collegiality culture of a research centre, it is my intention to collaborate on and support the development of further research projects within the centre outside this research project, and continue to be an active facilitator of knowledge exchange amongst staff and students across CU
Engagement in ECR development activity – Within and beyond the Doctoral College, including contributing to emerging networks both internally and externally, in and outside of my current research discipline, thereby expanding my experience as an emerging participatory researcher
Contribution to CU research culture – Through engaging with: further research collaboration opportunities with staff, postgraduate and undergraduate supervision opportunities, contributions to progress review panels and continue as a leading voice in centre-based knowledge sharing events