¹Warwick University, UK

I found White's paper on evaluating PSMs both interesting and insightful (White, 2006). I am sure it will be of great help to those who try to evaluate PSMs and other methods in the future. The case described involved engaging normally reticent citizens some of whom required interpreters to participate in English. This was clearly a formidable challenge and White is to be congratulated on a successful intervention.

In the past I have also tried to evaluate PSMs and published case studies on the use of PSMs in Sainsbury's, Palabora, Richard's Bay, PowerGen, Rio Tinto and Parliament (references can be found in Ormerod, 2005) and elsewhere. The Sainsbury's project was followed up after 5 years had elapsed (Ormerod, 1996a). The follow-up showed that, as well as the success of PSMs in engaging the interest, enthusiasm and creativity of the participants at the time, the strategies derived were implemented and were showing the benefits anticipated (with some pluses and minuses). The paper about Palabora also specifically addresses the issue of evaluation. Overall I concluded that participative exercises supported by PSMs are effective in supporting strategy development, in gaining commitment to implementation and, in the right circumstances, in delivering value for money (Ormerod, 2005).

In the detailed accounts of all the projects there is much to support White's observation that the methods (PSMs and others) successfully engaged participants who found them helpful. In particular I found that a mixture of activities calling on the different senses—drawing, speaking, watching, listening, asking, thinking—helped participants engage with the issues at hand, develop creative solutions and gain confidence in abstract reasoning and design. The PSMs provide a common way of presenting and capturing ideas; this aided communication and understanding. As the participants produced the diagrams (the conceptual models of SSM, for instance), the events were largely self-documenting. The publishing of 'books' of the results after each event helped consolidate the outcomes and established the feeling that the event had been significant, that something substantial had been achieved. It also, of course, provided a repository of knowledge for later stages.

An overview of the evaluation of the projects can be found in a Viewpoint (Ormerod, 2001) on Connell's paper (Connell, 2001). I concluded that, when eliciting the subjective views of stakeholders for evaluation purposes, a distinction should be drawn between the doers (the consultants/facilitators), the done for (the clients), the done with (the participants), the done to (those outside the project who were interviewed), and the done without (others in the organisation and in society more generally who were not involved in the project but may be affected by its outcome). It is this latter category of stakeholder that is likely to contain the strongest critics of the project and it is to my regret that I have never sought to capture their views.

Interviews and questionnaires were used for evaluation purposes. However, the continuing use of the methods involved provides powerful additional evidence that the methods are perceived as effective by those involved (particularly the done for and the done with). In the case of Sainsbury's, management believed that the participants had learned a lot about how to manage in general and subsequently asked me to run their senior management development courses, which I did for many years. I also helped them with a number of other projects using PSMs, for instance, to help the logistics function develop a strategy, to help a new team develop a home shopping strategy (Ormerod, 1997, pp 52–54), and to help the logistics function of Homebase (a Sainsbury's subsidiary) redesign itself after the take over of Texas. The managers at RTZ (the operators and part owners of the Palabora and Richard's Bay operations), also showed a continuing interest in the use of soft approaches. For instance, I was involved in systems definition and development (Ormerod, 1995, pp 97–99) and IS strategy development at the Hamersley iron ore mining complex in Australia (Ormerod, 1996b). It was an ex-RTZ manager at PowerGen who initiated my involvement there. The managers in all these cases had been involved in the use of the methods and perceived value in deploying them in a variety of different circumstances.

SSM would seem to be the most widely used PSM in practice, the take-up being mainly in the systems development arena. More generally, it seems likely that PSMs will only become widely used if they either become part of the defined set of methods of consultants (internal or external) or they become embedded in systems development methodologies. Published evaluations help this process but in the end practitioners will make up their own minds based on experience.

White's paper represents a significant step forward in the difficult quest of evaluating soft OR (PSMs) interventions. I look forward to reports of further applications and development of the approach by White and others.