Monitoring and Evaluation in Art for Social Change

Why this resource?

There has been growing interest in better understanding and documenting whether and how creative community-based initiatives can bring about social change. However, for some community groups, the topic of evaluation can be fraught with frustration.

This resource seeks to clarify some of the diverse theories, methods and techniques that may be useful for evaluating such projects.

This resource resource assumes minimal previous knowledge of the topics presented. Although recognizing that project development and evaluation are intrinsically linked, this toolkit is for the evaluation itself rather than how to develop the project.

Community-Based Initiatives & Social Change

There is increasing integration of the arts in community-based social change initiatives. This toolkit draws on examples and concepts in the community-engaged art field, but the information on evaluation can be applied across a wide range of community-based projects and programs, including in the areas of health promotion, social justice, and environmental sustainability, among others.

There is much debate about what constitutes social change – or what are the pathways to equity-sensitive social transformation.

Who is this toolkit for?

This toolkit is for anyone interested in evaluating a community-based health promotion or social equity initiative, whether you are a member of a community organization, an artist, a community organizer, an advocate, an activist, a health promotion practitioner, a community development planner, a funder, or a researcher.

How to use this tool

There are many ways to use this tool: For example, you could:

	Follow the 7-step Evaluation Guide
	Go through the Basic Evaluation Mini-Course
	Choose one of the animated scenarios
	Seek examples that relate to the art form, outcome, issues, target group or methods you are looking for
	Try out the quiz
Or even just:	Use the search command to find specific topics Download other guides

The tool is designed to link to the materials you need. (For example, from the scenarios, you can access details of how to do focus groups, design surveys, find existing questionnaires, etc.)

‘Art for Social Change’

Regardless of the art form or tool used, ‘art for social change’ engages the members of an identified community using creative imagination and expression to work out identity, shared values and aspirations. It is hoped that participation in this kind of creative engagement would lower the threshold for many forms of social participation and social action, and would help people to find new ways to see and be engaged in the world.

While there is great potential for individual learning and development using the arts, issues affecting individuals are always considered in relation to group awareness and group interests. Building skills for social change requires meaningful participation in shaping collaboratively created work.

(Reference: Goldbard, A. (2006). New creative community. The art of cultural development. Oakland, California: New Village Press)

Photo from Santa Fe University.

Critical considerations before you start an evaluation

There are critical debates surrounding the use of arts in social change and in community-engaged practices that are important to consider before approaching the issue of evaluation. Although these considerations may not have definitive answers or solutions, they are nonetheless important to be aware of and to consider in the practice and evaluation of art for social change.

Art as a Tool
One consideration is the issue of using the arts as a tool to “solve” other problems. This views arts as a means to an end in achieving other societal goals and objectives, which raises concerns about reducing the arts to an instrumental tool rather than having intrinsic value on its own. As Clive Gray writes,

“…there is a burden of expectation that cultural policies should provide a host of solutions to problems that are originally economic, social, political or ideological (or some combination of these).”
- Gray, 2007

Viewing the arts as a tool to achieve other sectors’ goals may also establish the arts as being a secondary and supporting sector to other areas of public policy. Gray also writes:

“Given that these other policy areas generally have the structural strengths that the arts and cultural policy sectors normally lack, particularly in terms of political salience and support, it is not surprising to find them in a dominant position in policy terms, with arts and cultural policies adopting a secondary, contributory, position in comparison with them.”

“Consumer”-Driven Individualization
Moreover, the focus on evaluating arts as a means for other sectors’ approaches to social change can lead to the individualization of the outcomes at the expense of the collective public good. Gray predicts:

“…there will be an increasing emphasis on the benefits of public policies for individual ‘consumers’ at the expense of the broader social collectivity, leading to the development of a fragmented set of policies that are increasingly focused on the demands and interests of individuals.”
-Gray, 2007

What is “good quality” art?
Another consideration is the tension between what constitutes quality in traditional arts as compared to in community-based arts. For instance, the traditional arts community may have established criteria for what makes “good art,” whereas community members participating in an art for social change project may not have the traditional training or skill to create art that would align with such criteria. As Belfiore notes,

“…it will not come as a surprise that quite often issues of quality have been the cause of friction between major funding bodies… and community arts groups. On the one hand, the former value quality and excellence in the arts and make of them important criteria for subsidy. On the other, community arts activities, which are mainly participatory, place more emphasis and value on the artistic process—with its empowering effects—rather than the artistic product... Despite the formal recognition, on the part of the arts “establishment” of the intrinsic value of participatory arts in the community, quality is still a delicate issue in the relationship between the national funding bodies and community arts groups.” -Belfiore, 2002

In response to the challenge of applying traditional quality criteria to art for social change, Animating Democracy developed a framework consisting of 11 attributes to help understand and evaluate excellence in art for social change. The 11 attributes in the Aesthetic Perspectives: Attributes of Excellence in Arts for Change are:

Coherence
Commitment
Communal Meaning
Cultural Integrity
Disruption
Emotional Experience
Openness
Resourcefulness
Risk-taking
Sensory Experience
Stickiness

For more information about the Aesthetic Perspectives, visit the Animating Democracy website.

Complexity
Another important consideration is the complexity of social change. The historical, political, cultural, and social contexts, the people involved in the project (both participants and organizers), power dynamics, short-term impacts, long-term impacts, and unanticipated consequences all interact in a complex interplay surrounding the project and its evaluation. Social change itself is inherently large-scale, complex and long-term – all factors which can be at odds with the practicalities of evaluation, which requires a limited scale, definable constructs and measures, and a finite length of time. In reconciling the complexity of social change with the practicalities of evaluation, it is helpful to consider the underlying theory of social change, which can determine program-specific indicators that can be used to evaluate the program’s impact within the bigger picture of social change.

References

Animating Democracy. (2017). Aesthetic Perspectives: Attributes of Excellence in Arts for Change. Retrieved from http://www.animatingdemocracy.org/aesthetic-perspectives.

Belfiore, E. (2002). Art as a means of alleviating social exclusion: Does it really work? A critique of instrumental cultural policies and social impact studies in the UK. International Journal of Cultural Policy, 8(1), 91-106.

Gray, C. (2007). Commodification and instrumentality in cultural policy. International Journal of Cultural Policy, 13(2), 203-215.

Types of ‘Art For Social Change’ Projects

There are 3 major types of art projects that can be said to be aiming to generate social change:

Artist creation: the artist(s) create their own artwork, and the social change content is in the work itself (e.g., Pablo Picasso’s Guernica)
Group problem solving: the artist(s) act as facilitators using art as part of a problem solving process, where group art creation is not the final goal of the project (e.g., image theatre to improve teamwork in a workforce)
Group artwork creation: the artist(s) act as facilitators for group art creation for social good (e.g., see the Examples page)

This guide assumes that your project being evaluated is going to fall into the category of “group artwork creation.” It's important to clarify that such projects are not focused solely on the production of art, but on the process of making art together, for example to work out identity, shared values, and aspirations.

Photos by Isengart, Medialab Prado, Jumblies Theatre distributed under a CC BY 2.0 license.

Community-engaged art and its various terms

“Art as a community-based, collectively driven activity is probably as old as art-making itself, and indeed, as the very catalyzing of ‘community’ as such. In recent years, a variety of terms have been used, each with its own nuanced goals and practices. These include: socially engaged art, community arts, community cultural development, social arts, and participatory arts. In addition, the practices of arts education, creative arts therapies, popular education and creative leadership processes are closely related. While a series of networks of those who practice community-based art have now been established and a body of literature addressing shared concerns is currently being amassed, the boundaries of this community of practice and research, is necessarily porous.”

(Reference: Spiegel, JB (2018) Social Circus, Buen Vivir: A critical inquiry into one bold political vision within a social arts global movement in The Art of Collectivity: Social transformation, Buen Vivir and Ecuador’s social circus in global perspective. McGill-Queens University Press.)

From Evaluation to Research

There is a continuum from “operational process evaluations to show funders how funds have been spent” to “inquiry for more in-depth understanding of the social impact of a project that contributes new knowledge to the field”. Some authorities consider only the former to be “evaluation” and the latter to be “research” best conducted by academic scholars. It may be wise to see this as a continuum rather than a dichotomy (either one or the other).

If the intention is to publish in a peer-reviewed journal to inform world knowledge, this would be to the right of the continuum and would require research ethics approval (a requirement for publishing in a peer-reviewed academic journal). Alternatively, if the evaluation is being conducted for the organization's own internal use and not intended for a broader audience, this would be to the left of the continuum.

Regardless of the position along the continuum, it is important to conduct the research/evaluation ethically and systematically so it can be used and interpreted by other people.

More about Terminology

The words “evaluator” and “researcher” are used interchangeably in this tool, for simplicity.

Similarly, the words “data” and “information” are used interchangeably; both mean the results of quantitative, qualitative and/or arts-based evaluation or research.