Design science research in Europe

Design science research – relevant and rigorous?!

Design-oriented research has a long tradition in Europe. While design science research is the dominating IS research paradigm in the German-speaking countries (Lange, 2006; Wilde & Hess, 2007; Becker et al., 2008), a large number of design-oriented researchers can also be found in the Nordic countries, the Netherlands, Italy, and France, just to name the biggest communities. In many other European countries, however, design-oriented IS research is less visible. While behavioural IS research aims at ‘truth’, i.e., at the exploration and validation of generic cause–effect relations, IS design science research aims at ‘utility’, i.e., at the construction and evaluation of generic means–ends relations. Both behavioural research and design research are paralleled by ‘meta’ research: Social sciences and design science reflect the respective processes, products, roles, etc. in order to provide guidance for researchers and to specify the respective approach's rigour.

For behavioural IS research, statistical significance is established as a clear and common measure of its results’ rigour – but the relevance of these results varies. Since design-oriented IS research is aimed at the construction of ‘better’ IS-related problem solutions, utility for practice is established as a clear and common measure of its results’ relevance – but the rigour of its construction and evaluation varies.

In behavioural research as well as in design research, researchers aim to reach the ‘Pasteur quadrant’ (Stokes, 1997), i.e., to combine the highest standards of rigour with a high level of relevance (Winter, 2007). As a consequence, high-quality design science research has not only to be relevant, but also to be rigorous. The rigour of design-oriented IS research, however, is less well defined and less commonly accepted than its behavioural counterpart. Methodology research in social sciences has matured over decades. Methodology research in design science, in contrast, is still struggling to amalgamate findings from engineering (which is even denied ‘science’ status by some), computer science, management, and also from social sciences into a consistent methodology.

The discussion of design science and design research for IS has gained momentum with the help of the DESRIST conference series (Chatterjee & Hevner, 2006; Chatterjee & Rossi, 2007; Vaishnavi & Baskerville, 2008), with special tracks on design research at major IS conferences as well as with special issues of IS journals on design research – like this special EJIS issue. Most IS design science research contributions refer to Simon's foundational ‘Sciences of the Artificial’ (1969), Nunamaker et al. (1990), Walls et al. (1992), March & Smith (1995), and to the Hevner et al. MISQ paper (2004). While numerous contributions have been made to the justification of design, the typology of artefacts, or specific problem solutions, rigour-related aspects are not yet sufficiently standardized across the IS design science research community. One example is the lack of a commonly accepted reference process model for design research: proposals include ‘build – evaluate – theorize – justify’ (March & Smith, 1995), ‘identify a need – build – evaluate – learn and theorize’ (Rossi & Sein, 2003), ‘develop/build – justify/evaluate’ (Hevner et al., 2004), or ‘problem identification and motivation – objectives of a solution – design and development – demonstration – evaluation – communication’ (Peffers et al., 2006). Another example is the debate whether constructs, models, methods and instantiations are the only acceptable artefact types, or whether theories and/or ‘testable design product hypotheses’ should be included (Walls et al., 1992; Venable, 2006) or not (March & Smith, 1995). Although the IS community agrees in general that information systems comprise organizational (human) as well as technical (software) components, it is argued whether ‘pure’ organizational artefacts are acceptable outcomes or whether only IT-related artefacts are allowed results of IS design research. A final example for the need of rigour improvement in IS design science research is the lack of commonly accepted, specific evaluation guidelines for the different artefact types.

Design research vs design science

An analysis of IS design science research exhibits two different types of contributions (Hevner et al., 2004): On the one hand, artefact construction and artefact evaluation are reflected on a generic level. The majority of contributions, on the other hand, describe the construction and evaluation of specific artefacts. While Cross (2001) designates these two categories as ‘science of design’ and ‘design science’, respectively, we prefer the designations ‘(IS) design science’ vs ‘(IS) design research’. While design research is aimed at creating solutions to specific classes of relevant problems by using a rigorous construction and evaluation process, design science reflects the design research process and aims at creating standards for its rigour.

Not every artefact construction, however, is design research. ‘Research’ implies that problem solutions should be generic to some extent, i.e., applicable to a set of problem situations. The tradeoff between the level of solution generity and the problem scope is addressed by situational artefacts. Since a unique solution is not applicable to many problem situations without generalizations, which diminish its solution power, situational adaptations should be incorporated in order to preserve application value (i.e., solution specificity) while covering a broad problem scope. This concept has its roots in the situational support of management decisions (Fiedler, 1964; Simon, 1976, 1977). Based on the concept of situational artefacts, it can be differentiated between their construction and evaluation on the one hand, and their problem-specific adaptation on the other hand. While the former can be regarded as design research in its narrower sense, the latter should better be designated solution engineering. The differentiation between solution construction and solution adaptation has its roots in (reference) modelling (Fettke & Loos, 2005), i.e., the stream of design research focussing on artefacts of type ‘(reference) model’. It can be applied for other IS design research streams, too: Method engineering (= design research) should be differentiated from method-based solution engineering, and construct engineering (= design research) should be differentiated from construct-based solution engineering. Since instantiations are problem specific by definition, instantiation design is always solution engineering; it may include adaptation of situational (reference) models, situational methods, and situational constructs.

Design science research artefacts

March and Smith's differentiation of constructs, models, methods, and instantiations as artefact types (1995) is commonly accepted in IS design science research (Vahidov, 2006; vom Brocke & Buddendick, 2006). Constructs constitute the ‘language’ to specify problems and solutions. Models use this language to represent problems and solutions. Methods describe processes which provide guidance on how to solve problems. Instantiations are problem-specific aggregates of constructs, models, and methods. Information systems can be interpreted as aggregates comprising specific instantiations of

• constructs (e.g., modelling primitives implemented by meta models of modelling tools),

• models (e.g., process models implemented as workflows), and

• methods (e.g., project methods implemented during software package introduction).

These artefact types are, however, covered with different intensities: An analysis of the first and second DESRIST conferences shows that most IS design science research is aimed at models and in particular instantiations, while methods are covered rarely. If methods are covered at all, the construction and evaluation of algorithms or mathematical/statistical techniques dominate, while only very few contributions address the construction and evaluation of systematic and purposeful behaviour guidelines or the reflection of this process (Bucher & Winter, 2008).

It is important to understand the artefact types of IS design research not as separate concepts, but as an interdependent system. Chmielewicz's (1994) classification of research approaches in social sciences may serve as a foundation to explain such dependencies. Chmielewicz differentiates between ontology building, theory building, technology, and philosophy. The respective artefact types are concepts, cause–effect relations, means–ends relations, and normative statements. Illustrations of his taxonomy usually use the pyramid metaphor: Applicable ontology and meta models constitute the foundation for formulating theories. Valid cause–effect relations constitute the foundation for identifying means–end relations. This ‘technology’ constitutes the foundation for choosing desirable ends, i.e., for normative actions.

Figure 1 illustrates the relationships of the Chmielewicz research approach (and implied artefact type) pyramid (left) to March and Smith's design science research artefacts (right). Concepts correspond to constructs, means–ends relations correspond to models and methods, and normative statements (= chosen desirable ends) correspond to instantiations.

Figure 1

Chmielewicz's research approaches vs IS design science research artefact types (Gericke & Winter, 2008).

The comparison supports two interesting insights:

• 1 Theories can and should be integrated into the interdependent system of IS design science research artefact types. Although opposed by March & Smith (1995) and therefore drawn in a dotted lozenge in Figure 1, the Chmielewicz pyramid shows that theories are necessary to close the gap between technology (which is the core of design science research) and ontology building (which creates the terminological and meta model foundations). Although theory building is not design science research, theories as ‘intermediate’ artefacts need to be included in the system of relevant artefacts for IS design science research.

• 2 Models and methods are more closely interrelated than other artefact types of IS design science research. It has in fact been argued that models and methods are ‘two views of the same coin’ (Bucher et al., 2008). While models focus on the result perspective – and imply procedural aspects –, methods focus on procedural aspects – and imply results. If models and methods are so closely related, construction and evaluation standards should also be aligned. The mutual transfer of situational concepts and adaptation mechanisms from models to methods and vice versa, which has been outlined in Becker et al. (2007) and Winter & Schelp (2006), could – and should – be facilitated on a common conceptual foundation.

Towards an IS design science research framework

The various differentiations introduced above can be combined into an analysis framework for IS design science research:

• Differentiation (1) separates IS design science (i.e., reflection and guidance of artefact construction and evaluation processes) from IS design research (i.e., construction and evaluation of specific artefacts).

• Differentiation (2) separates the reflection of IS artefact construction and IS artefact evaluation. This differentiation is necessary because all process models separate the ‘build’ from the ‘evaluate’ phase.

• Differentiation (3) separates IS design research in its narrower sense (i.e., construction and evaluation of situational artefacts) from IS solution engineering (i.e., problem-specific adaptation and utilization of situational IS artefacts).

• Differentiation (4) separates construction, evaluation, and adaptation processes with regard to their respective object. All four processes that result from differentiations (2) and (3) are separated into construct, model, method, and instance sub-processes. Further reflection is necessary to validate whether the differentiation between constructs, models, methods, and instantiations is equally important for construction, evaluation, and adaptation processes. While an artefact-specific differentiation of the evaluation process is quite straightforward for reasons of effectiveness, it may be useful to find as many artefact-neutral elements as possible in the construction and adaptation processes in order to make design science research more efficient.

Figure 2 illustrates the resulting analysis framework for IS design science research. Since instantiations (e.g., actual information systems) are usually problem-specific and not situational artefacts, the instance portion of differentiation (4) for design research construction/evaluation processes is not applicable (and therefore in brackets).

Figure 2

Proposed analysis framework for IS design science research (Gericke & Winter, 2008).

The road ahead

Although most information systems researchers are affiliated to business schools, IS research should not be limited to behavioural, social science-style research. While theory building is important and necessary to explain real-world phenomena, this knowledge also needs to be put into action in order to solve real-world problems. In both IS research paradigms, relevance as well as rigour needs to be maximized in order to reach the ‘Pasteur quadrant’. It cannot be denied that behavioural IS research significantly outperforms IS design research regarding commonly accepted, well-defined rigour standards.

IS design research seems to be reducing and ultimately closing this gap. European IS research communities can significantly contribute to this aim by bringing in their long design research tradition and their achievements: It is no surprise that the world market leader in enterprise business software as well as the world market leader in enterprise modelling software does not only have their company roots in Europe, but also have heavily drawn and are still closely related to local, design-oriented IS research communities. Similar contributions and relationships exist for participative design, enterprise engineering, enterprise architecture, method engineering, and many other design science research ‘flavours’. The rising number of IS chairs, IS students, and IS industry research labs at many European universities as well as the growing volumes of IS industry research grants indicates that design science research is also instrumental in keeping IS research significant.

The current IS design science research challenges should be addressed from such a position of strength: The rigour of IS design has to be clear enough to establish reliable and transparent quality controls, ultimately leading to the emancipation of design-oriented IS research journals in the ‘A’ and ‘B’ journal segment, and/or even better allowing for the coexistence of both types of contributions in leading IS research outlets. IS design researchers need clear guidelines for artefact construction quality and artefact evaluation quality. A commonly accepted framework of concepts and processes has to emerge in order to serve as a basis for state-of-the-art analyses and to allow for discovering synergies and necessary differentiations. Finally, links to behavioural research need to be systematically developed in order to integrate knowledge about problem contexts, solution types, and instruments into design-oriented IS research.

Applying the framework: this special issue's contribution

The outputs of design science research should be useful. The six articles of this special issue on design science research are therefore classified using the proposed framework of analysis.

• 1 The paper ‘Client as Designer in Collaborative Design Science Research Projects: What Does Social Science Design Theory Tell Us?’ by Weedman reflects on the artefact construction process without being specific regarding certain artefact types. Her finding that there are fewer problems caused by differences in disciplinary cultures than by the difference between design worlds and client worlds contributes mostly to the reflection of artefact constructions, although implications for artefact evaluation could be derived.

• 2 The paper ‘On Theory Development in Design Science Research: Anatomy of a Research Project’ by Kuechler and Vaishnavi explores the relation between theory building and design science research. They report on creating/refining ‘kernel theory’ components as well as ‘design theory’ components through the process of developing and testing process models (and design rules). Regarding the proposed framework, they contribute to the reflection of artefact construction and evaluation, with special emphasis on the relations between ‘kernel theories’ (‘theories’ according to the Chmielewicz classification) and ‘design theories’ (which come close to our understanding of methods).

• 3 The paper ‘Design of Emerging Digital Services: A Taxonomy’ by Williams, Chatterjee, and Rossi examines 12 leading digital services and develops a design taxonomy for classifying such services. They propose two broad classification dimensions, a set of fundamental design objectives and a set of fundamental service provider objectives. Not unlike this very portion of the special issue introduction evaluates the proposed framework by classifying the six papers, the authors evaluate the utility of the proposed taxonomy by classifying three leading digital services. Regarding the proposed IS design science research framework, this contribution can be characterized as construction and evaluation of a generic model with certain construct portions.

• 4 The paper ‘Designing Enterprise Integration Solutions – Effectively’ by Umapathy, Purao, and Barton investigates whether design knowledge for enterprise integration in the form of patterns can be reused to develop systems integration solutions, and whether such reuse leads to more effective design outcomes. They construct a method artefact that assists designers in developing integration solutions based on design strategies represented in integration patterns, and evaluate this artefact to assess whether it meets the intended goals. It is therefore a good example for the construction and evaluation of generic method (with model components) artefacts in design research.

• 5 The paper ‘Secure Activity Resource Coordination: Empirical Evidence of Enhanced Security Awareness in Designing Secure Business Processes’ by D’Aubeterre, Iyer and Singh develops a ‘Secure Activity Resource Coordination’ artefact, which supports modelling business process models characterized by the secure exchange of information within and across organizational boundaries. They also present an empirical evaluation of the proposed artefact against other business process modelling proposals to demonstrate the utility of the proposed design method. Their work can be classified as construction and evaluation of a generic model (as a method component) in design research.

• 6 The paper ‘Essence: Facilitating Software Innovation’ by Aaen proposes an approach to facilitate creativity and innovation in agile software development. A specific facility is described, and a specific method is outlined, which are intended to support certain software innovations. Aaen also reports on first evaluation results. Regarding the proposed framework of analysis, this research can be regarded as problem-specific adaptation of generic artefacts (drawn from existing work). While its focus is on an actual instance, the paper also touches construct, model, and method adaptation.

Figure 3 illustrates this classification of the six papers.

Figure 3

Special issue paper classification using the proposed analysis framework.

This framework-based analysis of the six examples of design science research in this special issue shows that the examples contribute across a broad variety of the possible contribution areas. Two papers primarily contribute in the area of design science, as distinguished from design research. One of these papers is focused on method (as opposed to construct, models, or instantiations) and deals with both construction and evaluation. Of the four papers that have more focus on design research, three deal mainly with construction and evaluation of artefacts that are models and methods (one of these also contributes toward instantiation). The final paper involves problem-specific constructs, models, methods, and instantiations.

The framework provides a means for comparing the different kinds of contributions, and also for identifying the types of contributions that proceed from design science research. The analysis demonstrates the usefulness of the framework for descriptive analysis. Future work is necessary to determine if the framework is also useful in a normative sense, to help in designing research. This normative notion of a heuristic value in the framework recognizes that research design is itself a potential subject area for the study of design science research.