¹University of Bristol, UK

I am grateful to Professor Ormerod for taking the time and trouble to comment on my article concerning the evaluation of PSMs (White, 2006). I am also delighted he found it interesting and insightful.

The article had three aims: to provide a modest discussion of the main issues that have be-devilled the OR community in addressing evaluation; to adopt a moderate or pragmatic approach to evaluation (which is an attempt to preserve a systematic attitude towards evaluation at the same time as recognizing the importance of actors' meanings and incorporating them into the evaluation); and finally to reflect on and analyse the evaluation findings to generate middle-range theory(ies) in order to understand what characterizes successful PSM interventions. In meeting these aims the paper did not undertake a comprehensive review of the literature, only a selection of works were included and doubtless many important ones were excluded. Having said this, there were many influences drawn on for the article, particularly the work of Ormerod and his position on pragmatism. Another influence was the work of Keys (1997) whose insight—using concepts drawn from the sociology of scientific knowledge (eg Latour, 1987), is that OR processes are complex interventions (ie systematic or purposeful action by an agent to create change or bring about improvements) and complex approaches are required to understand them.

It is clear that complex interventions require more than just descriptive methods to evaluate what works for whom and how. Also, there are no 'one-size-fits-all' approaches. Understanding will vary depending on specific situations. It often involves watching how the intervention is done, looking at participation in the intervention and what is produced, talking to stakeholders, and asking lots of how and why questions. To be honest, stakeholders are often somewhat nervous about discussing questions about an intervention. This is particularly so with the recipients of an intervention. It is often a tricky process, involving much negotiation and confidence building.

Most reported evaluations of PSMs in the literature are provided by practitioners and are often based on personal reflections. While it might be difficult to generalize from these accounts, they are, however, important in that they provide valuable insights for other people to share and review. Clearly, many of these practitioners are highly experienced and their reflections should be valued. I think more reflexive accounts of interventions should be encouraged such as ones provided by Ormerod (eg 1998) and Connell (2001). I would be particularly interested in ones which explore relationship issues such as trust and stakeholders' social networks. This would chime with an insight described in my article, which saw that PSM interventions involve bespoke relationship building. Here, further investigation of the intervention could look to see if these relationships can be described and whether they have a bearing (or not) on the success (or not) of an intervention.

Finally, I would like to thank Professor Ormerod for referring me to his viewpoint on Connell's paper (Ormerod, 2001). I particularly liked his description of the distinction between the stakeholders. I agree that an important category of stakeholders is the done without and it should not be under-estimated how valuable this group is. One member of this group, I think, is the academic community and the needs of this audience have to be satisfied. In my view, to facilitate this, evaluation of complex interventions requires a complex translation of insights about practice, which may be achieved by constructing middle-range theories that steer a course between abstraction and reflections on practice.