Introduction

The traditional view of science has come under sustained attack from postmodernism and other relativist positions. As a result the assumed authority of science as a body of established or justified knowledge has been undermined. Technology has not traditionally made such strong claims, but for some, because of its close association with science, it is similarly tainted. Critical rationalism (CR), a philosophical position based on the proposition that knowledge can be falsified but never justified, provides a potential defence against such attacks. In an earlier paper, I concluded that Karl Popper, the originator of CR, had established a strong (albeit contested) position in the philosophy of science but had not similarly developed an approach that could be used to guide OR practice (Ormerod, 2009, p 459).

Given that many in OR maintain that OR offers a rational and objective approach, the purpose of this paper is to explore some recent attempts to extend the CR position into practice and draw out the implications for OR. The paper has a pragmatic rather than a theoretical philosophical aim: its purpose is to offer a practically orientated perspective to help the reader judge what kind of guidance they might gain from CR in their own research or practice.

The CR position has been part of (and was for a time central to) the debate about the nature of science. Popper took Hume's generally accepted observation, that empirical evidence can only be used to falsify and never to prove a theory, and placed it at the centre of his approach to scientific discovery. A summary of Popper's ideas can be found in Magee (1973) or more briefly in Ormerod (2009), which also summarises some of the critical attacks on Popper's position within the philosophy of science. Ormerod (2010b) examines the dispute between inter alia Popper's objectivist, and Bayesian subjectivist approaches to making rational inferences. Criticism of Popper's views on practice can be found in Ulrich (1983, 2006a). A restatement and defence of CR can be found in Critical Rationalism (Miller, 1994). In this paper advantage will be taken of Miller's more recent book, Out of Error, which contains a number of essays describing his continued development of CR (Miller, 2006). Both of Miller's books contain many references to critics of CR. Out of Error can be taken as an up-to-date statement of CR and both of Miller's books include some observations on practice. Further discussion of practice can be found in Miller's more recent working papers (Miller, 2009a, 2009b, 2010).

In contrast to the philosophy of science, the philosophy of professional practice per se is not so well established as a field of philosophical endeavour. However, professional practices are simply human activities; the way people act in everyday life, the way that they interact with others, and the way that their beliefs about the world influence their actions, are subjects that have been addressed by many philosophers (and sociologists). Thus in terms of Kant's distinction between ‘theory’ and ‘practice’, both areas have received a good deal of attention. In addressing ‘theory’, the content of natural science and the activities of scientists have been central to the debate; in addressing ‘practice’ the focus moved to social science but relatively little attention has been paid to professional practitioners, their development and use of theory and the way that they advise decision-makers. OR has attracted the attention of some philosophers interested in developing the philosophy of practice (notably Churchman and Schainblatt, 1965a, 1965b; Ackoff, 1978, 1979a, 1979b and Ulrich, 2007, 2011a, 2011b). In the UK some OR and systems researchers, aiming to improve OR practice, have drawn on various philosophical, sociological and psychological ideas (for instance, Checkland, 1981; Eden, 1988; Mingers, 1997; Keys, 1998; Jackson, 2000 and Midgley, 2000). However, apart from Boothroyd's use of Popper's ideas on theory (Boothroyd, 1978; Ormerod, 2010a), the potential application of CR to OR practice remains largely unexplored.

Given the part that CR played in the development of the philosophy of natural science, an obvious next step for philosophers of a CR persuasion is to examine how it can be brought to bear in fields of professional practice closely related to natural science such as engineering and technology. The aim of this paper is to examine the progress that has been made and to explore its relevance for OR. Could the insights of CR lead to a better understanding of OR? Should OR change its approach in the light of CR or can CR be safely ignored?

The label ‘critical rationalism’ introduces two terms that are controversial. The use of the term ‘critical’ here derives from opposition to ‘positivism’; scientists, according to CR, cannot argue positively for a particular theory, they can only criticise theories and eliminate those that are false. The term ‘rationalism’ indicates that only strictly logical arguments should be deployed; it is assumed that rationalism is to be preferred to dogmatism. Logical here means consistent with deductive logic; inductive logic is held to be logically flawed. ‘Rational’ is thus taken to mean taking arguments and their conclusions seriously; arguments are the subject matter of philosophical logic. The aim of this paper is to examine the attempt to apply rational deductive thinking to practice post-Popper. Philosophers take the terms ‘theory’ and ‘practice’ to distinguish speculating about the facts of the world (theory) and deciding what to do, what action to take (practice). This Kantian meaning of the words is commonly adopted in philosophical discourse and will be used here. It does not accord with everyday English usage nor the use in OR to denote all that practitioners do.

For the purposes of this paper the examination of CR starts with the paradigmatic activity of natural science, namely developing knowledge of the natural world. It then turns to engineering; the paradigmatic activity is taken to be the making of human artefacts. It is only when it comes to OR that an examination of a full range of activities is attempted, from technical development to policy analysis. In reality scientists are involved in engineering-type activities, engineers are involved in scientific-type activities and both get involved in the creative development of ideas and policy analysis considered here only from the perspective of OR. In taking science to mean natural science, the wider consideration of the social sciences is set aside. An alternative approach, not taken here, would be to start with social science, move to social work and then to consider the role of CR in social policy analysis.

The paper is structured as follows. In the next (second) section of the paper the ‘problem of induction’ is briefly introduced. This is the problem that CR originally set out to address. Induction assumes that statements about the future (Kant's ‘theories’) can be derived from observations of the past: it seems to permeate our thinking and is logically flawed. Popper's original propositions are set out in relation to science and the subsequent restatement of CR by Miller is introduced in terms of fallibilism, negativism and scepticism.

The third section introduces practice, using the example of putting up an umbrella to introduce the structure of action and decisions to act (a focus on action is widely adopted in philosophy and social science theory (Ormerod, 2010c)). The fourth section considers the CR analysis of engineering taking bridge building as an example. This is, in an important way, a rather weak test (Popper in contrast demands the strongest possible test) of the proposition that CR can provide the basis for understanding and informing professional practice. It is a weak test because engineering is dominated by technical, instrumental considerations and at this point in the paper the wider social and political aspects are set aside. Nevertheless, engineering illustrates how a mature, successful practice discipline has addressed the issues in a practice area that has been used by CR to test their ideas. Further, the point has been made in the OR literature that OR is itself a technological rather than scientific activity (Keys, 1989, 1991); engineering may thus be considered a stepping-stone from science to OR for the purposes of this paper.

The fifth section, attempting a rather stronger test, examines how CR might be applied in OR; the test is more demanding because it examines OR's engagement in the full range of instrumental (how to achieve a given operational end), communicative (how to determine appropriate ends through discussion) and strategic (how to develop policies and deploy resources) reasoning. Thus for the purposes of this paper it is taken that OR is more than science or technology. In the fifth section the three domains (science, technology and OR) are compared and discussed in relation to the perspective of CR (It is not the intent to provide a rounded comparison of scientific, engineering and OR endeavours.) It is concluded that it may be possible as CR suggests to drive out inductive and justificatory claims in OR, but CR needs to develop its ideas further on (i) the development and use of theory-in-practice, (ii) the nature and use of critique, and crucially (iii) the role of subjectivity in decision-making.

Science from a CR perspective

The CR experience of addressing the problem of induction in basic science is central to CR thinking. Even though induction is only one of a number of problems faced by professionals in practice, it is inevitably the point of departure for CR. To understand the CR approach to practice it helps therefore to understand their approach to science. Popper argued that the same epistemological considerations apply to knowledge of social facts as well, but here, for simplicity of explanation, natural science will be the focus. The problem of induction and the difficulty it has presented to those engaged in the philosophy of science can be traced through the writings of David Hume, Karl Popper and David Miller. David Hume first identified the problem of induction (from the observation of only white swans it cannot be concluded that all swans are white). Karl Popper recognised that falsification could be used to avoid the problem; on the basis of this he developed the epistemology and methodology of CR. David Miller has been prominent in taking the programme forward post-Popper, plugging some holes in Popper's defence of the CR approach to science and developing a CR approach to practice. The problem of induction threatens to invalidate much of OR's analytical armoury; every time we make statements about the future, induction would seem to be implicitly invoked. If induction is indeed flawed then OR has to find a way to conduct its analysis differently or at least understand its analytical approaches in a different way. CR aspires to offer such a way out.

The problem of induction

According to The Oxford Companion to Philosophy:

Induction has traditionally been defined as inference from the particular to the general. More generally an inductive inference can be characterized as one whose conclusion, while not following deductively from its premises, is in some way supported by them or rendered plausible in the light of them. Scientific reasoning from observations to theories is often held to be the paradigm of inductive reasoning. Most philosophers hold that there is a logical problem about induction; its classic statement is found in Hume's Enquiry Concerning Human Understanding. (Cohen, 1995, pp 405-406)

The Oxford Companion entry points out that many philosophers have challenged Hume's position. Some have asserted that it is part of what we mean by rationality to operate in accordance with inductive procedures. Others have argued that induction is justified by past successes. Yet others have proposed what is known as a pragmatic justification: not that induction will lead to the truth, but if there is a truth to be known, inductive procedures are the best way of getting to it (Cohen, 1995, p 405).

The strict critical rationalist rejects all such views; for them, induction is logically not defensible, it is to be avoided in all its forms:

[T]here is no inductive logic. Facts, observations, experiments—these serve in science not as a foundation, or even as a support, for the theoretical superstructure, but only as tests of its correctness. Although no accumulation of experiences can verify a universal theory, or even provide grounds for supposing it to be true, one counterexample, if upheld, will falsify it. … Our knowledge can grow, provided we forgo the traditional demand for theories that are verified, or proved, or justified, …. (Miller, 2006, p 6)

The critical rationalist solution to the problem of induction

Miller explains that Popper's revolutionary doctrine for theoretical science was that science could be conceived as a rational enterprise that makes no appeal to induction and has no use for justification (Miller, 2006, p 113). What we call scientific knowledge cannot be knowledge in the traditional empiricist sense; that is, it cannot be derived from experience by induction, or by any other method; and far from having a foundation in experience, it consists largely of unsupported conjectures or guesses. As Kant recognised, knowledge precedes experience: it is only knowledge that is susceptible to modification in the light of experience that is genuinely open to empirical investigation. This is Popper's criterion of demarcation between science and non-science. Only those hypotheses that are empirically refutable, or empirically falsifiable, can count as scientific. Decisive and unquestionable refutation is not necessary, but susceptibility to removal is. Science may therefore admit hypotheses that are not strictly speaking falsifiable (such as statistical hypotheses) if it has rules that govern their rejection. Indeed, it may also allow falsified hypotheses to be retained, provided that there is some weight of negative evidence that would eventually cause them to be banished (Miller, 2006, p 86). To provide scientists with some guidance in how to conduct their research, Popper suggested a number of methodological conventions such as: design experiments to falsify hypotheses instead of confirming them; choose experiments that provide the severest test possible; use ad hoc additions to the original hypotheses sparingly, preferably not at all.

As a description of the activities of science Popper's position came under sustained attack (Ormerod, 2009). Even the basic logic of the approach has been the subject of claim and counter claim. Miller has been at the forefront of the defensive action. In order to sustain the basic position some of Popper's original contentions have had to be strengthened, modified or abandoned. The result of the defensive tightening of the logic has been a stricter, more austere CR. Miller (2006, pp 51–58) explains that there are at least three levels at which CR takes a stand against traditional approaches:

  1. i)

    Fallibilism: Fallibilism starts from the insight that universal hypotheses cannot be verified, but they can be falsified. The denial of certainty or conclusiveness is, according to Miller, ‘an epistemological advance, but not a very great one’ (p 58).

  2. ii)

    Negativism: The second advance on traditional approaches is the method of conjectures and refutations. Positivists seek positive confirming evidence when subjecting a theory to empirical investigation; negativists seek if possible to falsify the theory. Miller explains: ‘falsificationalists are interested only in relations between theories and the world, most importantly correspondence and lack of correspondence but also in subsidiary properties such as explanatory power and problem-solving ability, whilst positivists (and justificationists in general) are as much interested in relations between the theories and ourselves and the evidence we have in our possession, and especially degree of support, or degree of confirmation that it provides’ (p 55). Critical rationalists place the emphasis on criticism and negative argument.

  3. iii)

    Scepticism: Scepticism here does not mean the denial of realism. Rather as Miller explains: ‘A fallibilist is one who repudiates the quest for conclusive justification and certainty. A sceptic is one, like Hume and Popper, who repudiates also the quest for partial justification’ (p 72). CR aims to combine rigorous scepticism with common-sense realism.

According to CR what is important is whether or not a theory is true. We can guess whether a theory is true and we have no reason to discard it unless we find that it is inconsistent with the truth of accepted test statements (p 57). Thus although ultimate justification is not possible, it is possible to distinguish between more or less well-founded claims to knowledge; science survives the sceptical assault. Both technology and OR seem on the face of it to be both less negative and less sceptical in nature. How they appear to make practical progress without falling foul of logical pitfalls will be explored in the following sections.

The application of CR to practice

An example of practice: the action of putting my umbrella up

When it started to rain I put my umbrella up. Why? My immediate purpose is to avoid getting wet. Why? I prefer not to get wet; I value keeping dry. Will the umbrella help me achieve my aim? The action of putting the umbrella up is based on the theory that the umbrella keeps me dry when it rains. I have in the past found this to be the case. I put my umbrella up. It is windy and the rain is now driving nearly horizontally. My legs get wet. I stand in a doorway. I keep dry. But I am no longer moving towards my destination. I realise I value mobility. My new theory is that the umbrella gives me dry mobility in the rain except when the wind is blowing strongly. I notice that some people wear raincoats in order to have their hands free to hold their children's hands or carry the shopping. I adjust my theory to take account of my new observations (I recognise that the theory applies only if I have a hand free to carry the umbrella). I have to go out again. It is not raining but it looks though it might. If it remains sunny I will be hot and look foolish in a raincoat. I realise that I value staying dry, I value mobility, I value holding my children's hands, being able to carry the shopping or my briefcase, not getting too hot, not looking foolish. I forecast that it will rain. I decide on the raincoat. It rains. A passing car splashes my feet and ankles. In future I must choose a route to avoid pavements near busy roads. Next time I decide to take the car, pay the congestion charge, and use the time saved to give some thought to the problem of global warming. I value my time … and of course the environment.

The moral of the tale is that even the simplest action involves theorising, forecasting, generating options, anticipating consequences, identifying values, and deciding what to do. When concerned simply about keeping dry the issue is instrumental action. When I start to worry about the environment I am thinking in terms of strategic action. Every day, using our highly developed common sense, we handle complex decision processes related to our actions, both instrumental and strategic. Generally these decision processes remain unarticulated but every so often we examine them to adjust to new circumstances or new information, or to respond to some minor disaster that we wish to avoid in future. This is the process of articulation that Boothroyd draws attention to and makes central to his conception of OR. He points out that whereas at any one time scientists concern themselves with sets of theories (some theories and their supporting theories), engineers (and by analogy OR practitioners), in order to achieve some desired consequences, have to select for attention, sets of theories, sets of proposals, sets of actions and sets of consequences (Boothroyd, 1978; Ormerod, 2010a, p 1089). To address this complex of issues from a logical perspective is a daunting task for the CR programme. Nevertheless, the CR philosophers have proceeded by picking out some aspects where they believe their approach, originally developed for science, can be brought to bear on practice.

The pragmatic problem of induction

In his Objective Knowledge (see also the discussion of the ‘problem of tomorrow’ in Realism and the Aim of Science (Popper, 1983, p 62f )) Popper tries to formulate the logical challenge of practice:

[A] man of practical action always has to choose between some more or less definitive alternatives, since even inaction is a kind of action. But every action presupposes a set of expectations; that is theories about the world. Which theory shall the man of action choose? Is there such a thing as a rational choice? This leads us to the pragmatic problem of induction:

  Pr1:

Upon which theory should we rely for practical action, from a rational point of view?

  Pr2:

Which theory should we prefer for practical action, from a rational point of view?

My answer to Pr1 is: From a rational point of view, we should not ‘rely’ on any theory, for no theory has been shown to be true, or can be shown to be true. My answer to Pr2 is: But we should prefer as basis for action the best-tested theory. (Popper, 1972, pp 21–22)

Popper continues: ‘in spite of the “rationality” of choosing the best-tested theory as a basis for action, this choice is not “rational” in the sense that it is based on good reasons for expecting that it will in practice be a successful choice: there can be no good reasons in this sense, and this is precisely Hume's result’. According to Miller this passage has become notorious. ‘Although Popper's advice to act on the best tested theory is not usually contested …, few have been persuaded that this choice is rational unless some concession is made to induction; unless, that is, we assume some “link between past confirmation and future reliability” (Zahar, 1997, p 145), or we assume that “[i]f a theory T1 was more successful up to now than theory T2, then it is—not logically necessary but—probable, that T1 is also going to be more successful in the future than T2 (Schurz, 2002)” ’(Miller, 2006, p 113).

At about the same time Miller notes that a significant change of emphasis emerged in Popper's thinking. Popper ‘suggested that what is paramount when we need to act, and is the target of our critical assessment, is not the best-tested theory in our possession, but the best proposal for action—that is, the proposal that has most resolutely survived criticism’ (p 113). In this new formulation of the problem of pragmatic induction, hypotheses are replaced by proposals. However, the problem for deductivists remains, induction still seems to play a part. Miller observes that because the change of emphasis was ignored ‘the general consensus was that Popper's non-inductive solution to the pragmatic problem could not be made to work’ (p 113). The problem as formulated by Miller is that ‘if his [the agent's] choice is guided by the best-tested scientific (or commonsense) theories available to him, then it looks as if he is assuming that what was successful in the past will remain successful in the future. This is induction. If he renounces induction, then he is not allowed to assume that “the future resembles the past”, and his choice of what action to take in the future (as Hume made evident) is unconstrained. The rational agent is thus compelled to become an inductive agent’ (p 115). If this argument were to hold the CR proposition is defeated.

Can the deductivist position of the critical rationalist be rescued? The problem is that ‘[w]hat is being claimed is that the non-inductive agent's choice is quite unfettered by information about the past. It takes account of nothing, it is arbitrary, a matter of taste not of judgement’ (p 115). Thus any method of practical testing would be pointless and previous success would be irrelevant for our future actions. Miller suggests that ‘[t]he solution to the problem so formulated is quite straightforward. It suffices to recall that if an agent accepts some set Y of propositions, then, whether he appreciates it or not, he accepts all the logical consequences of Y. In particular, if the agent accepts some laws of nature or other spatio-temporally universal generalization [or presumably some lesson from experience] … to be true, then, like it or not, he accepts that in some respect the future will resemble the past. … No metaphysical principle of induction is needed to generate such predictions, since all the necessary content is provided free by the laws of nature (or other generalizations) that the agent accepts or assumes’ (p 115).

This seems to me to be an important insight because without it the gap between (i) how we actually cope with having to make practical decisions (by making assumptions about the future based on the past) and (ii) the logical rejection of induction (which thereby forbids a general assumption about the relationship between past and future) would be too wide to bridge. It has important consequences for the way that hypotheses and proposals are formulated in both technology and OR. We can also note that it is the agent who decides to accept a set of propositions; another agent may accept a different set of propositions. Subjectivity would seem to enter the picture. In the domain of science the subjectivity of individual scientists is converted into objectivity by invoking the collective judgement of the community of scientists; it is the community of scientists that ultimately determines which hypotheses can be considered to have been falsified. How one domain of practice, engineering, handles these and other issues is now examined.

Engineering from a critical rationalist perspective

Engineering is an example of practice and, given its apparently close relationship to science, for critical rationalists it holds the prospect of being fertile ground. At this point in our exploration of the issues we can simply note that an example of less instrumentally orientated practice (which is likely to bring into focus more social and political issues) could be expected to be more demanding for CR.

The nature of engineering

At its simplest, engineering can be depicted as a cycle of (i) inventing (guessing a solution to a problem), (ii) doing (making the invention), (iii) reflecting (assessing the success or otherwise of the solution), and (iv) inventing (guessing improved or new solutions). However, engineering has evolved to such a degree that such a simple description of trial and error would hardly suffice today. In bridge building, for example, a number of distinct activities would now be recognised including feasibility studies, design, construction, maintenance, and performance in use. Furthermore, at each stage of the engineering cycle proposals are subjected to detailed analysis. Thus in the invention stage many proposals can be weeded out by analysis-before-action and in the reflection stage, the factors affecting performance or failure can be assessed by analysis-after-action.

In order to analyse, the bridge engineer requires (i) a proposed design (of a man-made artefact, the bridge), (ii) a statement of the function to be performed (for instance, to support future rail, road and pedestrian traffic while providing clearance for shipping), (iii) a specification of the natural phenomena that the bridge will have to contend with (temperature, wind, earthquakes, flooding, and some not so natural phenomena such as impacts from ships), and (iv) the properties of materials to be used. The designer of a bridge will decide early on whether to use steel, concrete or timber. For a long span bridge the choice will be steel. The properties of steel are therefore of paramount importance to the designer of long span bridges. In choosing steel the engineer inherits a long history of the use of steel as a structural material. This history involves both advances in the production techniques in the iron and steel industry and innovations in the use of steel by structural designers. At every stage in the process leading to a bridge in place the engineers have to concern themselves with the function to be performed, safety during every stage of manufacture, construction and use, and cost. Innovation is also a driving motivation and aesthetics may be important. Bridge engineers want to build better bridges in every sense.

In essence, to analyse steel structures the long span bridge engineer only needs to know just a few empirical facts: the strength, elasticity and the temperature coefficient of expansion of the steel used. The strength and elasticity of steel can be ascertained by conducting a simple test to destruction on a sample a few inches long. The extension is measured as a load is applied. The plotted results (extension or strain against load) invariably show an approximately linear relationship. The conclusion is that steel is elastic and Hooke's law applies. Unfortunately we have to question this conclusion when it is observed that as the sample approaches failure the points stray from the line. Engineers overcome this problem of non-linearity by restricting the range of stress (strain) over which the elastic property can be assumed to apply approximately and by ensuring that the design keeps the stresses within that range. (In Popper's terms an ad hoc qualification has been added to the hypothesis restricting its use: Popper considered ad hoc qualifications undesirable because in his view they reduce falsifiability and impede progress.) A series of dramatic failures have over the years drawn attention to further inadequacies in the basic assumptions about the properties of steel; engineers have to be aware of the susceptibility to brittle fracture at low temperatures, to the tendency to ‘creep’ at high temperatures, to radiation, and so on. These problems give rise to a plethora of ad hoc qualifications attached to the basic theory that structural steel is reliably elastic whose properties can be simply established with a tensile test. The engineering response has been to make changes in design, production, and site practice. The continuing use of structural steel for bridge building demonstrates that, whereas scientists searching for universal truth would long ago have given up the hypothesis that steel was elastic with a measurable strength, engineers have doggedly persisted with it. One advantage of doing so was that it made stress analysis of the structure possible in practice.

Stress analysis (the equivalent of modelling in OR terms) plays a crucial function because it allows options to be criticised on paper in the design office: the cost is low as no physical activities are involved. However, the process of analysis itself is based on additional simplifying assumptions such as ‘frictionless pin joints’, ‘plane sections remain plane under bending’ and specified design loads. With these assumptions stress analysis becomes a powerful, flexible tool that structural engineers can be trained to deploy. The calculations are nearly always approximately right as attested by the results; for instance, the measured deflection of a bridge under load can be compared with the calculated predictions.

The above account of the properties of structural steel and the design of long span bridges introduces some of the intellectual and physical challenges of one particular example of engineering endeavour: to design bridges, engineers have to choose between imagined options taking into account the client brief and the requirements of safety, economy and aesthetics (and the impact on the environment and society); engineers continuously innovate to take advantage of new structural forms, new materials, new machines, new scientific discoveries and new methods of analysis and design; a balance has to be struck between the aspiration to be innovative and the need to be cautious (risk averse) particularly where safety is involved; to analyse the structures, engineers need to know the characteristics of the materials to be used and the loads to be withstood (traffic, wind, floods, earthquakes and so on); analysis of design proposals in the office enables some to be discarded before any physical activity takes place; engineers use approximations to make analysis possible and materials usable; analysis and new materials allow more daring designs to be attempted; this daring, bolstered by confidence in analysis, pushes up against the limitations of the assumptions until some unexpected failure occurs; engineers learn from these failures; testing of components or models enables some possible modes of failure to be anticipated and avoided; engineering knowledge and knowhow is related to the need to act in specific situations; best practice is gleaned from past experience (in particular from failure) and laid down in codes of practice, standards, procedures, recipes, operating manuals and so on.

The role of experience

According to CR, the rational procedure for engineers is to guess at proposals-for-action (conjectures) and then criticise (attempt to refute) them. The criticism is based on (yet-to-be-refuted) scientific hypotheses. Deciding how to act, Miller says, ‘is always a matter of guesswork, as everything about the future is. But if we persevere in our criticism, sometimes we can avoid implementing very bad guesses concerning how to implement our goals’ (Miller, 2006, p 121). However, engineers would not accept that their proposals, even their initial ideas, are blind guesses. Even when asked for a quick reaction engineers will draw on their experience of past calculations, difficulties and failures, and the impact of standards and recommended procedures. Miller's response to this point is: ‘Our guesses are not random, of course, but informed; which means only that they are guesses informed by earlier guesses. … However richly our guesses are informed by what is known, they know not (are blind to) what is not known’ (Miller, 2005, p 68). It seems to me that the critical rationalist needs to develop a fuller explanation of how agents learn, gain experience, develop craft skills and so on. Given that they are prepared to countenance connections between past and future within a theory this should not present insuperable difficulties. Thus the engineer's rule of thumb can be treated as a yet-to-be-refuted theory, a theory within which it is assumed that the theory will hold in the future.

Consider the case where an agent has to choose between two technological approaches T1 and T2, both of which can in theory meet the requirements. T1 has been successfully used several times but T2 is an idea that has not been tried before. The cautious engineer would opt for T1 on the grounds that he knows it will work, it is tried and tested, and is therefore a safer choice. From an objectivist point of view he would have no grounds for doing so. Both T1 and T2 have the logical status of unrefuted conjectures: either approach can be taken, as there can be no ‘good reasons’ for preferring T1. In fact, following the rule that the best tested (T1) should be preferred would have the consequential effect of ruling out innovation, inventiveness and ingenuity; this would conflict with one of engineering's core aims, which is to invent new and better ways of doing things; the engineer is both practical man and innovator. In Deductivist Decision Making Miller revisits this problem concluding that there are after all no reasons not to choose T1 because if T2 is tested (a course of action that CR would urge) and is successful, the position remains the same; there is no reason why T1 should not be chosen (nor that it should be). If on the other hand the test on T2 fails, then it simply confirms that T1 was the better choice. Clearly, there is no reason not to choose T1 (Miller, 2010, pp 15-16). It seems to be possible and highly desirable to develop this type of analysis to account for the role of learning and experience in the success of engineering.

The role of codes and standards

Another problem for the engineer is that there are in principle an infinite number of yet-to-be-refuted scientific hypotheses that could and should be used to criticise a proposal and equally an infinite amount of practical experience to tap into. An individual engineer does not have the time for an extended winnowing process for every judgement made. A method is therefore required to narrow down the field to those that for practical engineering purposes can be treated as established laws. The response of engineers has been to develop codes of practice, standards, procedures, manuals, analytical methods, and physical tests. These are written to apply in delineated areas of engineering practice. Miller stridently declares: ‘The instrumentalist whitewash, which pleads that a refuted theory remains “true within its field of application”, is not countenanced by falsificationists’ (Miller, 2006, p 31). This would seem to conflict with the engineering approach as instanced by the assumptions made about structural steel described above: the assumption of elasticity, for instance, is pragmatically instrumental. Truth for the falsificationist means universally true. ‘True within a field of application’ is for them a contradiction. The answer seems to lie in accepting that judgements can be made and that a hypothesis can be therefore judged true within a defined field of application; the defined field becomes an ad hoc qualification.

When those that set the standards are faced with competing scientific hypotheses they will decide which should be reflected in the standards and what reliability should be placed on them. These are decisions, which according to CR cannot be justified; but in practice they are judged by the engineers who write the standards according to their performance. The resulting proposals are subject to criticism in terms of (i) their accuracy, (ii) the appropriateness of the ad hoc additions, (iii) their use in practice, and (iv) whether they will hold good in the future. In other words they are criticised both in terms of epistemic (pertaining to truth) and pragmatic (pertaining to usefulness) values. Similar judgements are made about abstract ‘instruments’ derived from many other sources (experience, failure, engineering research and so on). In so far as all such instruments (codes of practice, rules of thumb and so on) can be discarded if not fit for purpose, they can be viewed as scientific. However, this is not the science of universal truth, it is rather the pragmatic exploitation of local, temporary stabilities under controlled conditions. The knowledge of how such stabilities can be created and exploited (for instance, how to create a concrete mix of desired strength and workability from the locally available sources of gravel, sand, cement and water at the time of construction) is not the same as knowledge of the natural world sought by science. It is thus better identified as different by giving it a different name such as engineering knowhow or theory-for-practice. The critical rationalist's deductivist outlook can still hold. Statements of engineering knowhow can be conjectured and refuted, but the criticism must now be on the basis of pragmatic as well epistemic values. In taking this code-writing approach engineers have created an ecosystem (a micro world) that takes account of science but for much of the time immunises the practicing engineers from interaction with it in their daily activities. With a careful analysis of the logic of theory-for-practice CR could make an important step towards addressing the problem of practice: it would be a crucial step in addressing the epistemological aspect. That it has yet to do so, in my view, amounts to a serious shortcoming.

In summary, reliance on experience would seem to be a crucial feature of engineering. CR insists that proposals are merely blind guesses. While logically correct, this seems to rule out the possibility that engineers’ guesses on engineering issues are likely to be less blind than those of the layman. CR needs to develop its theory to account for the fact that relying on experience (both individual and collective) is logical. To help individual engineers bring the collective experience of past scientific and engineering effort to bear, engineering communities have developed a two-tier system of codes of practice to provide practitioners with ‘theories-for-practice’ to work with. These theories are those found to hold within specified conditions when specified precautionary measures are adhered to in defined areas of application. The theories are only universal in that potentially they can be applied in any part of the world (hence the development of international codes and standards). Legal and quasi-legal principles and rules embedded in codes of practice and standards play an important role in engineering and these have to be accommodated in any theory of practical action. As we have seen these ‘theories-for-practice’, in contrast to the theories of science, reflect a compromise between epistemic and pragmatic values.

Engineers are not limited to their defining instrumental activities; they also have to consider wider aspects. A large bridge will have political, economic, and social significance. For instance, the Bosphorus Bridge in 1973 provided a direct link between Asian and European Turkey for the first time since the invading King Xerxes built a temporary pontoon bridge in 480 BC. Any large bridge will also have aesthetic, social and environmental impacts: for instance, the manufacture of the structural steel, decreased journey distances and traffic generated all have implications for carbon emissions; sites of scientific interest may be disturbed; the bridge may attract suicide attempts; the bridge may become an iconic emblem of the city; and it may have military significance (the Bosphorus Bridge, for instance, had to provide clearance for the Russian fleet based in the Black Sea to pass under it). Furthermore, the account so far has implicitly depicted the engineer as an isolated individual engaging in the paradigmatic activity of making a human artefact (a bridge). In practice engineers work in teams, engage in collaborative activities and participate with others in wider policy debates about the development of, for instance, energy, transport and water management systems. However, many different disciplines become engaged in the same sort of activities; for the purpose of this paper these will be discussed in terms of the involvement of OR.

Operational research from a critical rationalist perspective

OR was born out of the idea that a scientific approach to management problems would yield more objective and rational (and therefore better) decisions. It was never clear precisely what this should be taken to mean and as a consequence any attempt to pin down an authoritative definition of the subject has been contentious. Despite this (or perhaps because of it) OR in the early years was seen to be successful and grew quite rapidly. As the professional practice matured it settled on a variety of somewhat disparate activities. These activities can be characterised in various ways, none of which would attract universal agreement. It is easier to agree that we can describe various types of OR activity and that one can talk about broad ‘archetypes’ of OR rather than a single all embracing definition of OR. I have previously proposed three archetypes of OR practice, namely ‘smart bits’, ‘helpful ways’ and ‘things that matter’ (Ormerod, 1997). While making no claim that these are either exclusive or exhaustive, in this instance they are helpful in separating out those activities that are similar in nature to engineering (smart bits), those in which individuals and groups engage in the investigation of (yet-to-be defined) issues (helpful ways), and those that consider wider societal questions (things that matter). This section will therefore address the relevance of CR for each of the three archetypes in turn.

Smart bits

‘Smart bits’ refers to algorithms embedded in other systems such as forecasting, scheduling and credit scoring. However, I take it also to include most quantitative modelling activities including simulation and business what … if models. Virgin Media, for instance, ‘employs techniques such as clustering, time series forecasting, optimisation and various forms of regression …’ (Doel, 2010). The models themselves are in part deductive but they depend on assumptions (premises) based on empirical data or subjective views on assumed facts about the world. The deductive (mathematical) reasoning poses no problems for CR. However, judgements about the structure and scope of the models and algorithms are involved, and the derivation of data to populate the model goes right to the heart of the debate about induction. How can this process of generating the assumptions be described in CR terms, if indeed it can be?

The critical rationalist would say that when modellers (in collaboration with their clients) make assumptions they first make blind guesses. The chosen parameter, relationship or structure remains after it and other possible guesses have been subjected to criticism. Implicitly there will be an assumption that the chosen parameter, relationship or structure will hold in the future. If, for instance, the parameter in question is a cost there will have been acceptance of a rule that this particular cost has been properly estimated and will remain the same in the future or will change according to some specified relationship, which it is accepted will apply in the future. The relationship between the past and the future is imbedded in the many such accepted relationships but, as CR insists, not in some all embracing inductive principle.

Statistical techniques are frequently used in establishing the parameters for these types of models. CR rejects the statistical theory of inverse inferences:

The dream of rules for inferring universal laws from brute facts, and rules for inferring causes from effects, is realized in statistics as the theory of inverse inference, as it is known; that is, a technique for inferring the composition of a population from the composition of a sample drawn from it. But unfortunately for their patrons … they are nothing but conjectures or guesses… (Miller, 2009a, p 14)

The use of epistemic probabilities (but not physical probabilities of frequency or propensity) is similarly rejected. Miller (2010, p 3) takes a quote from Keynes (an inductionist) as one of the principles of his approach:

… no knowledge of probabilities, less in degree than certainty, helps us to know what conclusions are true. (Keynes, 1921)

Whether the use OR makes of statistical and probabilistic techniques can be rescued in the eyes of the critical rationalist is a moot point. The issue lies at the centre of the fierce debate between the critical rationalists and the Bayesians (Miller, 1994, Chapter 6; Ormerod, 2010b). The methods of classical statistics (shorn of their justificationist embellishments) can presumably be accommodated within individual theories (in the same way that the inductive relationship between past, present and future can be handled as discussed above) by adding an appropriate assumption: in other words some statement can be added, for instance, that it is assumed that the samples taken are indeed representative of the whole being considered.

In conducting ‘smart bits’ analysis the data used usually contains many taken-for-granted assumptions. Thus cost estimates usually conform to the host organisation's cost-accounting practices, which will themselves be based on standard accounting conventions determined by a standard setting body. For accountants these conventions perform a somewhat similar function to the codes of practice in engineering; they provide a consistent (though not true) basis for making comparisons and taking decisions. In some circumstances these may be brought into question (for instance, by instead examining opportunity costs), but usually costs as measured by the accountants are taken at face value as theories-for-practice. Thus, although the analyst will be very concerned that the logic captured in a model correctly reflects the logic of the operations being considered, the validity of the abstract accounting models of cost that support the cost assumptions are usually taken for granted. Like the designers of artefacts for mass production, the designer of algorithmic models can test a prototype and adjust it until the desired performance is achieved. For instance, when introducing their new ordering system in their supermarket stores, Sainsbury's phased the introduction in order to compare the performance of those stores with the new system against those with the old. If no improvements were apparent, implementation was paused, the reasons for the lack of improvement analysed, and adjustments to the system made (Ormerod, 1996, pp 116–117).

It seems that in smart-bits-OR it may be possible to adopt a critical rationalist approach and claim to be logically rational given that the problem of rational prediction using classical statistics can be resolved satisfactorily as indicated above by wrapping inductive assumptions in individual theories. The main casualty would be Bayesian statistical methods: these should be avoided according to CR.

Helpful ways

‘Helpful ways’ are deployed when OR consultants are invited (contracted) by a client to examine an issue, problem or opportunity. This may be conducted by an individual or team of consultants engaging in a process that can be likened to a doctor's diagnosis, a forensic scientist's investigation or a social workers case analysis. It also alludes to those interventions where facilitation is supplied to groups of people to help them discuss, debate and negotiate some issues or problems. These two helpful approaches to getting to grips with an issue will be referred to as investigative-mode and facilitative-mode.

The investigative-mode can be illustrated by my first experience of conducting an OR investigation. My problem was the low utilisation of equipment (expensive assets) in one of the National Coal Board's production areas. The crude statistics suggested that the poor utilisation was the result of machines lying idle once production had stopped (for geological reasons) and the equipment was left at the coalface waiting to be salvaged (taken to the surface or deployed to a new face). The utilisation problem thus became the sub-problem of salvage. The problem of salvage led to an examination of the way that operations were planned. Did the plans include salvage? The answer was yes (a simple matter of fact—the plans were there), so attention turned to observing how men were deployed to different tasks at the beginning of each shift at different pits and asking the officials in charge how they decided what to do. Were the salvage plans being implemented? The answer was generally no (a conclusions that would seem to rely on induction; particular observations led to a more general conclusion). This led to the examination of how the relative priority of the different tasks ought to be determined according to the mine's circumstances; in other words those operations that affected the salvage of the equipment were considered in the light of other tasks such as the tunnelling to provide access to new areas of coal and the mining of coal itself. It was only then that ways of reducing the time taken to salvage equipment (proposals arrived at by invention or guessing) could be considered and, with the help of some simple models, the consequences of each proposal evaluated. In other words each proposal was criticised with the aid of analysis very much as an engineering design would be analysed. In discussing possible solutions with the management it became apparent that any changes would have implications for the relationship between the manager of the mine and the management of the area: questions of autonomy and accountability, not only for the utilisation of assets but also for the safety of operations. Thus a different set of values came into play. Only the managers could make the decision in the light of the tensions that would undoubtedly arise from new organisational arrangements (they decided not to adopt the changes suggested).

One way of flushing out such wider considerations earlier in the intervention is to arrange meetings at which key players discuss the issues. There are various ways of structuring such interactions and the OR investigator may revert to analysis between meetings. In these cases the meetings are used to enhance the investigative-mode; they are an alternative to multiple one-to-one interviews. However, a further step can be taken if emphasis in the meetings is placed on the participants engaging with the issues with the OR consultants taking on the role of facilitator. Facilitation of the interactions may be supported by methods that are variously referred to as ‘soft OR’, ‘soft systems’ or ‘problem structuring methods’, but they don’t have to be. Models may be involved but they may be qualitative in nature reflecting the subjective views of participants rather than some abstracted aspect of reality.

In these facilitative-mode interventions it is the participants' understanding, their beliefs and ultimately their agreements to act that are at stake. Many universal and spatio-temporally local laws (some trivial, some important) are taken for granted while differences of interpretation, beliefs and desires are the focus of attention. In contrast, in investigative-mode it is the role of the investigator (the OR consultant) that is central, the opinions of those interviewed are taken as evidence; the consultant will try to maintain an objective, neutral stance but this is an ideal and subjective views will play a part. Decision-makers may choose to take account of the analysis, or they may ignore it. Ultimately it is the decision-maker's beliefs and values that are expressed in his or her decision. (The critical rationalist would rather say beliefs constrain a decision.)

The important thing from a CR point of view is not to allow the inherent subjectivity of decision-making to pollute the possibility of maintaining a strict approach to what can be said about facts based on the evidence. In the case of science it can be assumed that intersubjective consensus within the scientific community can guide choices. However, for OR there is hardly ever the time or opportunity to develop an intersubjective consensus across a wide community. It is OR's ambition to help individual and small groups of decision-makers formulate and hone their beliefs in the light of the anticipated consequences of their decisions and the ‘facts of the situation’ as the decision-makers perceive them. Such formulations, once articulated, can be subject to critical debate. If the relevant people are involved, the decision-making process can be said to be rational.

There are clearly different definitions of rationality at work here. Both critical rationalists and advocates of helpful ways OR conclude that the prospect of rationality lies in the process of reaching a decision rather than in the decision reached. Both address proposals to act. In practical decisions the critical rationalist seeks logical rationality in relation to the true (epistemic, pragmatic, aesthetic and ethical) consequences of proposed actions, with the values taken as given. The helpful ways practitioner seeks what might be termed practical rationality through the involvement of relevant people who reflect both on the facts and the values related to the issue in hand and bring their beliefs to bear. The role of values is crucial in both schemes. CR implicitly takes values (in the mining example above, cost, safety, accountability and so on) as sources of criticism; advocates of soft OR also make values the object of criticism, to be debated and explored at the same time.

The subjectivist approaches of helpful-ways-OR can be understood in terms of Popper's three worlds: the physical world is described as World 1; the world of conscious human processes is called World 2; the world of the objective creations of the human mind is called World 3. World 3 is the world of theories, including false theories, and the world of scientific problems, including questions to do with the truth or falsity of various theories (Popper, 1999, pp 23–35). For example, a bridge is part of World 1, but it is built according to a plan that draws on theories. These plans and theories reside in World 3. The engineers’ thoughts about the bridge during the design process, and those of the engineers and workers during construction, reside in World 2. Whereas smart-bits-OR similarly concentrates on treating the objects of interest as occupying World 1 using theories from World 3, helpful-ways-OR concentrates on the thought process of those involved in decision-making (World 2) and their relationship with the physical world (World 1) and theories (World 3).

All three worlds are considered by Popper to be real. In addressing the issue of practice, CR concentrates on the relationship between World 1 and World 3 and deliberately excludes consideration of World 2, the world of psychology and subjectivity. In effect, it is trying to repeat the approach developed in relation to science. In science the aim was to understand how universal truths could be pursued despite the individual ideas and biases, the human characteristics, of individual scientists. Universal truths are not affected by individual beliefs. In engineering, the individual beliefs of engineers might be the source of proposals, but the aim is to find the best proposals for action despite the individual beliefs; ideally, the best engineering proposals are those that would be recognised as such by all engineers but realistically the beliefs and attitudes of individual engineers come into the picture. In helpful-ways-OR the aim is to concentrate on the World 2 beliefs of the participants in relation to World 1 reality. By capturing the thoughts of the participants in text or (generally qualitative) model form, ideas and beliefs are effectively moved from World 2 into World 3. In other words ideas and beliefs are elicited and recorded, so that they can be subjected to criticism and analysis. It is anticipated that participants will learn and adjust their beliefs, at least sufficiently to understand the point of view of others. The aim is not to find the best proposal, or even a consensus, but to seek agreement to act or agreement to explore the issues further.

The emphasis in helpful ways therefore shifts to consideration of the process of engagement. How wide should the scope of the project be, who should be involved, and what status (for instance, as an expert, or as someone affected, or as someone responsible for the consequences) should each participant be afforded is, according to Ulrich, a matter for critical analysis; he refers to this as boundary analysis. Boundary analysis sits within the framework of critical systems heuristics (Ulrich, 1983, 1987). Within helpful-ways-OR it seems that boundary analysis could be a starting point for critical rationalists. Both critical system heuristics and CR recognise that rationality lies in the process; both emphasise the fundamental importance of a critical approach. The choice of a boundary is itself an action; both would therefore advocate criticism of proposed boundary choices. Whether, in general, the use of the methods of ‘soft OR’, dealing as they do with subjective beliefs, would be fertile ground for CR is a moot point; it requires CR to recognise the crucial role of subjectivity in decision-taking. However, these methods are often used to structure problems and are described as problem structuring methods (PSMs). The results therefore could be considered by CR as part of the creative, inventive activity with the intention being to generate proposals (guesses). The CR approach then comes into its own when the outcomes of PSMs are subjected to greater scrutiny, as they surely will need to be if the proposals are to be implemented.

Things that matter

‘Things that matter’ refers to policy formation on socially important issues, sometimes referred to as ‘policy analysis’ or ‘systems analysis’, or as the application of ‘strategic OR’. Policy advice can be seen as arguing for or against one or a number of policy options and OR consultants may be engaged in helping clients develop their arguments. Miller takes an argument to mean a structure that consists of premises and conclusions, not just any fragment of narrative. Such arguments he says are often erroneously used to establish, prove, support, or otherwise make plausible the doctrines we hold (Miller, 2006, p 64). The question posed by the critical rationalist is: in what manner does rational argument advance or promote the search for truth? Since logic is the theory of argument, it is a question for logic (p 65). The primary purpose of argument Miller suggests is not to persuade, to add knowledge, or to justify, but rather its purpose is ‘to criticise or to probe or to eliminate the propositions that we are interested in, not to provide reasons either for or against these propositions’ (p 65). For those whose task it is to advise on the pros and cons of proposals, this position is not at first sight encouraging. However, CR does allow arguments to defend against criticism. Thus the merits of a proposal can be displayed by pointing out the flaws in (in other words, criticising) the criticisms of it (Miller, 2006, p 79).

Walker (2009), drawing on Mayer et al (2004), suggests that six styles of policy analysis can be identified (see Figure 1). The top half of the hexagon is described as primarily object-orientated, focussing on systems, policy, measures, and models; the bottom half is subject-orientated, focussing on people (policymakers, stakeholders, experts) and their interactions in a policy process. The traditional OR contribution to policy analysis has been object-orientated. The many examples in The Handbook of Systems Analysis (Miser and Quade, 1985, 1988; Miser, 1995) exemplify the ‘rational style’.

Figure 1
figure 1

The policy analysis styles (Walker, 2009).

Full size image

OR analysts adopting a ‘rational style’ need to avail themselves of relevant ‘facts’. On important issues, however, it is likely that many of the key ‘facts’ will be established by scientists (as in the case of epidemics such as swine flu or foot and mouth) or engineers (as in the case of building a new power station or a new rail link). The OR consultant will therefore develop models to help a policy advisor (perhaps a civil servant) evaluate (predict) the consequences of different possible policies given the key facts and relationships provided by the scientists and engineers and assuming lots of other facts and relationships as given. Where uncertainties persist about factual (including scientific) and other assumptions (including the response of those affected by the policy), different assumptions about the future can be adopted (scenarios) and the performance of the different potential policies under different sets of assumptions can be explored using the assumed relationships. In critical rationalist terms the policies (guesses, inventions) are criticised according to their predicted consequences (safety, health, environmental, economic, political, social, aesthetic and so on).

Criticism plays a heightened role in public policy development; independent experts, interest groups, journalists, politicians, members of the public have their say in a variety of public (and private) arenas including parliament, newspapers, TV, blogs, conferences and formal public inquiries. The ‘rational style’ of Figure 1 attempts to provide an objective, logical approach to anchor the debate. The aim of the analysis is to avoid the dominance of personal beliefs through an emphasis on rational process. Public policy formulation can thus (at times) be seen as an attempt to act according to the logic of CR; in the swirl of political debate the logical, objective approach gives something that policy advisors can hold onto in the face of strongly held subjective beliefs of the protagonists. It also holds out the prospect that the positions adopted (the decisions made) can be defended at least in terms of the process followed. However, CR would not countenance any claim that the outcome is, as a consequence, rational because this is always taken to mean that it is justified.

My own work on strategy and policy in the National Coal Board (later British Coal Corporation) was primarily conducted in the ‘rational style’ but, as the modelling effort was embedded in the policy formulation and decision-making process, it also included adopting the ‘client-advisory’ and the ‘argumentative’ styles (Plackett et al, 1982; Ormerod and McLeod, 1984; Ormerod, 2010c). Most of the activity was driven by the needs of processes outside the control of OR. Policy analysts may choose to adopt a ‘client advisory style’ or may be expected by clients to do so. Generally, the OR advice will take the form ‘if you want to achieve your declared aims then each alternative proposed strategy/policy is likely to result in the following consequences according to our models and these assumptions’. This, of course, gives rise to questions: are these really the aims you should pursue; have we considered all the relevant consequences, good and bad; are the assumptions acceptable; how can we assess which proposals are attractive given that different consequences are different in kind and affect different people in different ways? These sorts of questions, crucial in policy analysis, seem at first sight to be out of the reach of logical and scientific reasoning. However, OR has developed methods (in particular, multi-criteria decision analysis) to help decision-makers clarify their views (Belton and Stewart, 2002).

OR has also supported policy analysis by adopting the styles identified in the lower half of Figure 1. Here the attention is on the ‘subject-orientated’ issues of process, interaction and participation. Examples of such policy analysis can be found in Planning Under Pressure (Friend and Hickling, 2005): for instance, national environmental policy-making in the Netherlands, private-public decision-making in Sweden, plutonium management in the UK, and neighbourhood renewal in Rome. The documentation produced from such projects can be open to scrutiny, but claims to rationality lie in the participation involved, the process adopted and the agreed quality of the resulting proposals rather than any claim to be objective and logical.

These examples illustrate that things-that-matter OR can be conceptually decomposed into activities that are similar to the smart bits and helpful ways activities except in the degree of detail, documentation and transparency required for (socially) important decisions. However, one of the distinguishing features of ‘things that matter’ is the need for a much greater critical scrutiny of the aims and values deployed in the establishment of facts and the evaluation of consequences: wider societal aims cannot be assumed or taken as given. It is the central importance of these issues in ‘things that matter’ (and in many helpful ways engagements) that takes OR beyond the paradigms of either natural science or technology. To date CR has not addressed the identification and selection of values in either substantive or processual terms; in other words it has said little or nothing about what ought to matter or how it is (or ought to be) determined in practice. This is, however, an area that many argue can be subjected to considerations of logic and rationality. Whether this current limitation of CR can be rectified will be addressed in the following discussion.

Discussion

The paradigmatic aim of natural science is to understand the natural world; the results of natural science are the ‘laws of nature’ in the form of refutable but unrefuted hypotheses; the concern is that the ‘laws’ are spatio-temporally universal; the driving force is curiosity. The paradigmatic aim of engineering is to harness nature in the service of man: the results are artefacts designed and made by humans; the concern is that the artefacts should be functional, safe, aesthetically pleasing and affordable; the driving force is invention and practical problem solving to get things done. The paradigmatic aim of OR is to help managers address the problems and issues arising in the human activity systems they manage; the result is improved decisions; the concern is that the help given should be rational and practical; the driving force is problem solving (in a wide sense).

It is clear that natural science, engineering and operational research are very different practical activities. Scientists are located in laboratories, universities, government institutions and commercial organisations. They research natural phenomena for evidence of universal truths about regularities in nature. Questions are formulated, hypotheses and models are proposed and experiments are conducted. Results are published in journals and are criticised and discussed by an established community of scientists. In engineering, the work is conducted in design offices, in factories and maintenance facilities, on construction sites and at operational facilities (such as oil refineries, nuclear power stations and harbours). The work is seldom written up for publication but the resulting artefacts are there to be used. Engineering best practice is captured in codes of practice, standards, operating manuals and training material. The engineering community as a forum for exchanging ideas and experiences is less developed than science. In operational research, consultants produce models and reports to assists clients in their decision-making. They are located in consultancies, commercial organisations and government departments. Practitioners seldom write about their work. Codes of practice play little or no part and consultants usually have to rely on personal reflection and client reactions for critical appraisal.

CR has yet to settle on a compelling narrative to describe practice, as CR would see it, in terms of its underlying logic. This would seem to be because the efforts of critical rationalists have been concentrated on applying to practice (such as engineering) what has been learnt in the context of science, rather than considering practice per se. The depiction of design as blind guesses winnowed by relentless criticism based on science does not seem to capture the essence of engineering design activity. Similarly the avoidance of positive arguments in policy analysis seems at times artificial. After all, it seems sufficient to say that the merit of a holiday in Rome is that you could visit the Vatican, and the merit of a holiday in Egypt is that you could visit the pyramids. To criticise Rome as a destination because it lacks pyramids seems an unnecessarily convoluted way of making the same point. In the early stages of the development of a theoretical position such problems are to be expected. In the last century there were many such arguments stimulated by Popper's reformulation of the essential task of science. It is a mistake to think that the mature position developed for natural science can be simply extended to practice. It is necessary to consider practice afresh.

To be taken seriously in any area of endeavour CR has two tasks. First, it has to show that activities that have a long history of successfully fulfilling their purpose are acting in a manner consistent with CR precepts: the success has to be explained. Second, it has to show that, once the subject area has been shorn of all atavistic inductive, justificatory and subjective thinking, CR can provide a philosophy of action that can be practically deployed. In the domain of natural science CR has addressed both tasks (though its position is strongly contested and currently not in vogue). In engineering, it can point to the relentless criticism of proposals through testing, analysis and experience-in-use to account for engineering success. Engineers have circumnavigated the problems of scientific uncertainty and met the pragmatic demands of solving engineering problems by the judicious use of ad hoc qualifications, control of the conditions and the adoption of codes of practice and the like. They have created a disciplined approach, which is not infallible but can point to practical success. In the domain of engineering critical rationalists are in the process of making their case (it is work in progress).

OR can also point to its successes, but even those engaged in OR are not convinced that all of its activities are demonstrably successful. In which case, it is better to consider the various activities of OR rather than the subject as a whole. For example, in the case of smart-bits-OR it could hardly be denied that those aspects of OR that rely on logic such as scheduling, optimisation, critical path analysis and the basic logical approach of problem solving are successful in what they do. Other models that rely on statistics and forecasting are also successful and demonstrably so, but according to CR care must taken to declare the associated assumptions about the relationship between the sample and the whole and between the past and the future. In contrast, it is difficult to maintain that helpful-ways-OR, whether in investigative-mode or facilitative-mode, is so obviously successful that CR is obliged to account for its success; it would in any case be a difficult task because CR aims to avoid or at least tame subjectivity. However, the focus in helpful-ways-OR is on the process, and CR insists that processes, which are designed to be critical, can be deemed rational even though the outcomes cannot. Policy advice may involve object-orientated or subject-orientated approaches (see Figure 1). Object-orientated styles can strive for a logical, objective approach along critical rationalist lines; their success can therefore be described in CR terms. Subject-orientated styles run up against the same difficulties as helpful ways, the problem of subjectivity.

Table 1 lists some of the activities that might be involved in different types of OR intervention and indicates how each activity might be viewed from a critical rationalist perspective. The table provides a tentative agenda for further investigation. Five topics are picked out for further consideration here, namely guesses, theory-for practice, logical choice, the nature of critique and subjectivity.

Table 1 The logical status of OR activities
Full size table

Guesses

Generating guesses (options) from which to choose is an important activity in managerial decision-making. OR may support the gathering of relevant ideas from members of the client's organisation, from external experts, and other sources (for instance, other companies, other industries, other countries). It may also help in the generation of new ideas by facilitating workshops designed for that purpose. Furthermore, the very act of analysing, structuring and modelling the decision may well throw up options previously rejected or not considered. Nor is it only options that need to be guessed. There may be some speculation about the values that should be included in the analysis and again the intervention may uncover other possible values for consideration. Guesses may also be required, about the factors that could influence future events, prior to any analysis. CR has no objection to any method being used to generate guesses. It is thus an area where inductive thinking could be used so long as the guesses are subsequently subjected to criticism. The CR contribution is to insist on relentless criticism once guesses have been put forward for consideration. As we have seen, one approach could be to treat all soft OR as creating guesses, thereby placing it outside the jurisdiction of CR. Naming soft OR methods as ‘problem structuring methods’, hints at such an orientation.

Theory-for-practice

Engineers have adopted a two-level approach to theory-for-practice: the writers of the codes examine all the evidence and come to a view as to what should generally be taken to be the case. Clearly such guidance cannot cover all possible circumstances and some interpretation is left to the engineers engaged in design, manufacture, construction and so on. In so far as the principles are encoded in the law, engineers may find themselves taken to court and, even if codes are not legal requirements, engineers may be held liable if they have not followed ‘best practice’.

The surprising thing is that, although in the world of practice engineering standards, accounting rules, medical protocols and the like proliferate, there is little use made of such a two-level structure in OR. The attempts to develop and apply ethical codes for OR practice is one example, but beyond that it is difficult to think of OR principles that might be universally accepted. The fate of past attempts to produce a definition of OR suggests that any attempt to agree principles would be fraught with difficulty. One could conclude that the difficulties are too great or that experience shows that such principles are just not needed. However, engineering codes are not written for engineering in general but for specific tasks within specific sub-domains within engineering. At a more disaggregated level the task for OR becomes easier. For instance, Robinson (2008a, 2008b) suggests values that should be applied to the development of simulation models. However, on the whole it is likely that OR practitioners will continue to work without formal guidance and must develop their own (local, temporary, contingent) knowledge base; epistemological difficulties thus impinge directly, making adherence to the strictures of CR relevant. Declarations of implicit assumptions are thus crucial.

Logical choice

Most OR is concerned with helping managers make choices. The choice process can be formulated in decision and game theoretic terms; options are evaluated in terms of their consequences against possible scenarios. OR supports a rational approach in terms of (i) identifying relevant facts (often using statistics to interpret the data), (ii) inventing plausible scenarios reflecting uncertainty about the future, (iii) identifying the aims of the decision makers, and (iv) evaluating the consequence of implementing each option in terms of the identified aims (Ormerod, 2010c). Can a critical rationalist perspective support the decision choice model? To answer this question we can refer back to Miller's analysis of technological choices (T1, T2) introduced earlier in the paper. It would seem that choosing the tried and tested technology is the dominant choice. An operational researcher would come to similar conclusions. He or she might go further and attach a cost to the further testing or alternatively make past thoroughness and severity of testing one of the criteria in the evaluation of the choices. OR analysts may well attach probabilities to the success and failure of the two proposals and calculate the utility of the two options. If so, they are taking a Bayesian approach. Miller comments:

A comparison [of the deductivist approach] with the decision that a Bayesian decision maker would reach in similar circumstances draws useful attention to the fact that the deductivist approach here sketched, by ignoring all questions of utility and value, is rather oversimplified, though not damagingly so. … Where the deductivist differs from the Bayesian is in being unswayed by considerations of probability, and in evaluating the available techniques [options] solely in terms of how well criticized are the hypotheses that they will be effective on the next occasion. (Miller, 2010, p 16)

Clearly, CR challenges OR's approach in so far as it uses (subjective) probabilities and utilities. A response could be to make clear the basic assumption needed to allow logical analysis to proceed,

The nature of critique

As we have seen critique plays a pivotal role in the CR approach. In scientific investigation epistemic values (pertaining to truth) are all-important. In engineering epistemic values are joined by pragmatic and economic values and the imperatives of safety. Beyond immediate instrumental concerns, engineers may need to consider aesthetic and ethical issues as appropriate for the task in hand. This could include more broadly environmental, social, economic and political values. For OR the values to be considered are similarly epistemic, pragmatic, and ethical (taken to include environmental, social, economic and political values); it would be unusual but possible for aesthetics to also be a concern. One way of ensuring that the logical concerns raised by CR are prominent in OR thinking is to include a value pointing to logical analysis in the values that OR pursues. Thus OR is defined as helping clients take good decisions in the light of their epistemic, logical, pragmatic, and ethical values.

The ultimate aim or objective of any enterprise usually consists of a complex of values often not explicitly formulated nor shared equally by all members of the enterprise; the emphasis placed on different values can be seen as reflecting the interests of the people involved. Nor is the concern with values confined to ultimate aims or objectives. The means used to achieve ends will also have implications that need to be examined in terms of the values being pursued. The imperative of vigorous critique has therefore to be extended to values. When engaged in helpful ways and things that matter, as has already been noted, OR has a role in eliciting and critiquing the values. For instance, this would be a major concern in multiple criteria decision analysis (MCDA) projects (Belton and Stewart, 2002). From an OR perspective we can either wait and see whether the advocates of CR are prepared to engage with individual beliefs and values or whether we should consider CR embedded in something else that does; for instance, in the approach I favour, critical pragmatism (Ulrich, 2003, 2006b, 2007).

Subjectivity

CR, as we have seen, addresses the rationality of statements about factual matters but has chosen to have little or nothing to say about subjective beliefs and is determined to keep uncontrolled or uncontrollable subjectivity out of epistemology. However, notwithstanding attempts to take decisions in a rational manner, managerial decisions to act are, in the event, subjective in nature. In considering the application of the critical rationalist perspective it seems inevitable that one is driven to invoke at some point a subjective judgement or decision. How can these be accommodated from the perspective of CR? For science, the strategy of CR is to avoid considerations of personal beliefs by invoking the idea of a community of scientists. A community can be assumed to come to an intersubjective view as to whether a particular hypothesis, or complex of hypotheses, should be rejected; individual opinions, beliefs and idiosyncrasies are thus subsumed. This process may take a considerable time and involve a lot of resources, but it works for science. The merit of invoking a wider community is that it draws on the experience and expertise of those engaged in the field, it codifies their deliberations, and provides opportunities for discussion and criticism; the results are transparent and can be taken to provide an objective view. Engineers on the other hand have to take many, local decisions, relatively quickly, often with limited resources. As we have seen they have developed a ‘proxy community’ in the form of engineering knowhow or theory-for-practice written down in codes of practice and so on, sources that can be instantly and cheaply accessed. The combination of codified standards and operating procedures, and uncodified experience and common sense allows engineers to proceed with some confidence.

Managerial decision-makers and their OR helpers generally do not have the time or opportunity to consult a wider community and cannot call on codified manuals to address their disparate issues (this is a gap that the managerial literature tries to exploit with fads and fashions). One strategy supported by OR is to provide advice in the form of algorithms and models; this is the smart bits approach. OR consultants provide the assurance that the models are objective in the sense of unbiased, properly researched and can be defended if necessary. Where assumptions are made the client's acquiescence is sought; most clients are happy to back their own judgement and determine what is acceptable. The reliance on the professionalism of the OR analyst and the client's judgement is, from a critical rationalist perspective, clearly a second-best solution. An alternative strategy, the facilitative-mode helpful ways approach, has been to design and facilitate a ‘micro community’ in order to simulate, in a local context, the merits of scrutiny by a wider community. To simulate the desirable attributes of a community, helpful ways engage participants in a process of interviews and workshops, which provide opportunities for debate and criticism; the deliberations are captured in cognitive maps, conceptual models and other outputs. The ‘simulation’ involves engaging the subjective beliefs of individual participants; helpful-ways-OR addresses how this can be best achieved. Again this is a second-best solution but it places less reliance on the objectivity of the analyst and the individual judgement of the client. What are called here second-best solutions clearly involve subjective judgements, but an attempt is made to tame them.

CR holds that rationality lies in the process of considering options rather than in the choice made: the choice process can be claimed to be rational but the choice made cannot. This is a point also made by Ulrich and others who research the process of OR. In practice there are usually a number of people involved in a managerial decision and there are also the concerns of those not directly involved to be considered. There may be others who have something to contribute. It is rational to consider who should be involved in such a decision-making process and what that involvement should be. For instance, Ulrich identifies inter alia the roles of the expert, the witness and the affected (Ulrich, 1983, 1987). Thus like CR, Ulrich sees rationality in practice lying in the process; he therefore locates rationality in the choice of boundaries within which the issue is to be considered, the choice of who should be involved, and the status or role of all those who are involved. The question of boundaries doesn’t arise in the critical rationalist scheme for science as the focus is on universal truth, the deliberations are transparent, and all scientists can participate in the community. In managerial decision-making the process is usually neither transparent nor collective; it is one of OR's aims to make it more transparent (at least to those involved) in order to improve and support it. Another aim might be to make decision-making more democratic. However, ultimately decisions are often taken by an individual or a small group (such as an executive committee). In such circumstances individual subjective beliefs about the aims, the means and the consequences will be important and cannot be assumed away. Where things that matter are concerned the process is generally opened up to public scrutiny involving politicians, the media, independent experts and individual citizens; the community is involved, the scientific and democratic ideals can converge. Very often, for things that matter, the design of such processes is out of OR hands.

The fact that subjectivity inevitably enters the proceedings could be taken to indicate that the CR endeavour fails in the domain of managerial decision-making and hence OR. However, if the logical, objective rationality that is sought by CR cannot be assured, it can at least be pursued. One strategy for OR is therefore to always strive to achieve the objective, logical rationality of a deductivist approach, while accepting that some subjective, elements may be required to achieve this end in practice; OR needs to be clear as to when and why such a tactic is necessary. A second strategy is essentially to accept the Bayesian solution(s) to the problem of induction by embracing a subjectivist approach with an assumption declared that makes logical analysis possible (Ormerod, 2010b). The third strategy is to accept that there are distinct camps within OR, namely the deductivists who stick to smart bits type activities, the statisticians and forecasters who have carefully developed specialised techniques, and the subjectivists who gather under the banner of ‘soft OR’ or ‘problem structuring methods’. However, as we have seen, even the simplest factual statements have recourse to subjectivity at some point; equally those engaged in soft OR have hardly abandoned rational thinking.

A more radical strategy would be to turn the CR approach on its head: instead of avoiding subjectivity, the philosophic logic of CR could be applied to it. While there is a logical basis for rejecting induction and justification, the avoidance of subjectivity (to concentrate on epistemology rather than psychology) is simply a choice originally introduced by Popper as a research strategy; in the case of science it proved productive. For practice, it has the effect of making the most important element of action, the actor deciding what action to take, beyond the reach of CR. This need not be the case: subjective beliefs can be subject to logic. Take the example of putting the umbrella up when it rains. I put the umbrella up when it rains because (i) I believe it will keep me dry and (ii) I value keeping dry. The act can be described as logical because I have acted according to my beliefs. Beliefs are clearly crucial. Furthermore, if several values are in play (keeping dry, personal mobility, sartorial elegance, preserving the environment) and several actions are being compared (umbrella, shelter, raincoat, drive) it makes sense to examine whether the beliefs in the values are consistently applied.

Where the logical process reaches the point where people are required to make judgements (the community of scientists have to accept that a theory has been falsified; the authors of engineering codes of practice have to determine which standards to adopt; the micro-communities of ‘helpful ways’ have to form a view as to what to do next; a democratic process has to ensure that the interests of citizens are respected in consideration of ‘things that matter’), the participants need to act logically in terms of their beliefs; their beliefs should be informed by all the data and analysis they can obtain (with implications for who they should involve and believe). If CR cannot countenance subjectivity beyond recognising human fallibility and invoking the ideal of intersubjectivity, the conclusion is clear: CR doesn’t provide an adequate overall philosophical approach for OR and it must at most assume a subordinate role. If it can countenance subjectivity, CR may yet provide an adequate basis for OR, one that accords with the basic belief of most OR practitioners that OR should try to be logical at all times.

Conclusions

In moving its attention from theory to practice CR has to consider the implications for both the theory (theory-for-practice) and practice (action) itself. Induction is rife within OR but this can be made logically acceptable by always containing statements about the assumed relationship between parts and the whole, and between the past and the future within the theories and proposals being considered. Decision-makers can then judge whether to accept such assumptions or not. If the theory is accepted so are the assumptions. Thus the CR concern to avoid a transcendent acceptance of induction can be met. This is an important step. Also the avoidance of justificatory claims is logically correct and should be applied.

It may be possible for OR to avoid the problem of induction and to desist from making justificatory claims but subjectivity is an integral part of managerial decision-making and cannot be ignored or assumed to be eliminated by the pursuit of an intersubjective ideal. The critical rationalist tries to minimise the subjective element and in many respects this is precisely what OR consultants try to do on a day-to-day basis: they help decision-makers place more weight on evidence and analysis and less on personal guesses and intuition. However, there will always be an important element of subjective judgement involved. Furthermore, CR has not yet paid the same attention to the values deployed in criticism as it has to the epistemological issues. OR academics engaged in research into the process of OR have collectively given much more attention to this issue, both by developing MCDA approaches and by taking a social science perspective.

Table 1 provides an agenda for a more comprehensive evaluation of the role of CR in OR. Three areas in particular need more elaboration:

  1. 1)

    the development and use of theory-in-practice or practice knowhow;

  2. 2)

    the role of subjectivity in decision-making; and

  3. 3)

    the nature and practice of critique.

CR as currently constituted does not yet deliver its promise to provide a logically based deductivist understanding of practice that can rebuff the postmodern and relativist onslaught on rationality. Critical pragmatism, the stance advocated by Ulrich, currently pays more attention to actual practice and thereby provides a more comprehensive response. Moreover, it is able to accommodate the strictures of CR where they can be usefully applied. The approach of CR, based as it is on logic, should appeal to the reflective operational researcher who recognises the importance of objectivity and logical rationality to practitioners and clients. An appreciation of CR as presented in this paper will also help them to recognise some of the difficulties of holding to such a commitment in practice and remind them of the assumptions that must be declared in even the most basic predictions and proposals for clients.