When things go ‘wrong’

When conducting the case studies, it was more common for an assessment plan to be altered than it was for it to remain as originally proposed. This is partly because not everything you need to know can be known in advance and what is already known may not be the most pertinent information when it comes to designing an effective, collaborative social value assessment.

It is perhaps best to think of an assessment as an open process of exploration. It is still important to have an initial plan, but try to:

• Respond to new information and emerging understandings. In most cases, the proposed approaches and methods will need to undergo some degree of adjustment and in a few cases they will require a complete rethink. This may be part of a co-design discussion with community organisations, in response to an unanticipated opportunity, or when something that you try does not work out.
• Remain flexible over exactly what, where and when research activities take place.
• Expect to be surprised. Unplanned opportunities and unexpected findings can open up new networks and avenues of enquiry. Opportunities and alternative methods will emerge as your understanding of the situation grows. These might come to light in the course of your planned activities or as the result of a serendipitous coincidence or unexpected invitation.

Working through the complexities of a participatory multi-method approach and responding to the situations that arise will ultimately help your understanding. Documenting your process and why alterations to methods were necessary can be useful for reflection and learning.

Depending on the nature of the problem, also see the questions below.

The methods proposed at the outset of an assessment will by necessity be based on incomplete understandings of the context. Once engaged in the research, it may well become apparent that one or more of the intended activities is not possible or appropriate (either entirely or as originally planned). This may be decided in collaboration with communities or be a practical decision taken in response to the realities on the ground.

• Consider why the activity has been unsuccessful. Was it a practical barrier, such as timing or venue, or was it a question of trust or appropriateness?
• What can you learn from this? Negative responses or unsuccessful techniques are themselves revealing, identifying absences and potential challenges to participation in other research activities.
• Does it make sense to try the activity again, perhaps with different community, or in combination/sequenced with another method?
• How might you adjust your approach? Do you need to add in more time?
• Don’t take it personally (but do engage in some self-reflection). Discussions with other members of a team or community representatives can help identify why an approach or activity has been unsuccessful and identify alternatives. Consider collaborations with others if the limitations are due to your profile.

• Does this matter? What does the material you are obtaining reveal about social values?
• What were you anticipating? Consider your assumptions about the heritage and communities.
• If people are participating but seem unwilling to share their personal reflections, you may need to revisit your consent procedures and reflect on the power dynamics within the process. Are you able to create a safe space for sharing? Are people clear on how their information will be used?

The emphasis within an assessment is on identifying the diversity of values for communities with interests in a site, not achieving consensus on a single or dominant interpretation. The case study assessments found that processes of valuing were highly contextualised and that common terminology may mask quite diverse values and practices.

• Try to identify where or when differences are expressed. Are the same participants expressing different values in different contexts and/or is there a tension between different groups?
• Avoid the urge to simplify or tidy-up your findings. The depth of the understanding may depend on navigating a messy complexity, but that’s OK.