Introduction

The design of Information Technology (IT) artifacts is at the core of information systems (IS) research. While IS research has been criticized for an over-emphasis on behavioral and managerial aspects (Orlikowski & Iacono, 2001; Iivari, 2007) and for a perceived lack of practical relevance (Hirschheim & Klein, 2003), it has also been argued that the design of IT artifacts intended to solve organizational problems must remain a focus of the discipline (Hevner et al, 2004; Winter & Albani, 2013). Since the 1990s, numerous scholars have worked to develop the foundations of good design science research in information systems (DSRIS), for instance, in the form of general conceptualizations (March & Smith, 1995), guidelines (Hevner et al, 2004), or methodologies (Peffers et al, 2008).

Theory is of growing importance in DSRIS. Some scholars, most prominently March & Smith (1995), argued that the goal of DSRIS was to build and evaluate IT artifacts. Following that approach, theory is something that lies at the heart of behavioral science research and thus outside the realm of DSRIS. Here, theory simply inspires the building of an artifact; however, the pure focus on artifacts has been heavily criticized for being too narrow for this socio-technical discipline (Mckay & Marshall, 2005; Carlsson, 2010; Kuechler & Vaishnavi, 2012a). Arguments have been made for including a theoretical perspective in design science (for instance, Walls et al, 1992; Gregor, 2006; Gregor & Jones, 2007; Gregor, 2009) in order to capture knowledge with respect to artefact design and construction. With both IT artifacts and so-called design theories, DSRIS has two powerful instruments at hand for capturing design knowledge (Kuechler & Vaishnavi, 2012a).

What is IS design theory (ISDT)? With the emergence of theory in DSRIS and the widespread use of the term design theory, several conceptualizations have emerged (see, for instance, Venable, 2006 for an overview). Baskerville & Pries-Heje (2010) collect some assumed characteristics of an ISDT, including being of a prescriptive nature (Walls et al, 1992), practical (Goldkuhl, 2004) and the basis for action (Gregor & Jones, 2007), principals-based (Markus et al, 2002), and a dualist construct. As for the latter, the supposed dualism refers to design theories covering both principles of form and function as well as principles of implementation (Gregor & Jones, 2007; similar dualist conceptualizations can be found in, for instance, Walls et al, 1992).

Baskerville & Pries-Heje (2010) make an argument to eliminate the prevalent dualist assumption and distinguish between two types of design theory: design practice theory that explains how to construct an IT artifact and explanatory design theory (EDT) that explains why a certain component is constructed into an artifact. For them, both types of design theory have intrinsic value, and for the purpose of effectively focusing one’s research efforts, can be addressed independently. The explanatory EDT-type question of why a certain component is constructed into an artifact can also be found in Gregor’s (2009) conceptualization of the exterior mode of design theorizing. The notion of exterior refers to the ‘purposefulness of IT artifacts’ (p. 4) and to ‘what happens as artifacts exist and are used in their external environment’ (p. 7). She argues that an exterior mode design theory can include concrete propositions such as: A system with feature X will perform better on measurement M than a system without feature X. In a similar vein, Kuechler & Vaishnavi (2012a) develop the notion of a DREPT (Design-Relevant Explanatory/Predictive Theory) that explains why an artifact has the effects it does. All of these conceptualizations have in common that they seek to explain why an artifact with certain features has the effects it does (Kuechler & Vaishnavi, 2012a) and why these features should be constructed into the artifact in the first place (Baskerville & Pries-Heje, 2010).

In this context, an ISDT will entail statements that relate features of IT artifacts with purposes and goals (Gregor, 2009); however, the literature lacks a clear conceptualization of this relationship, which may lead to issues with respect to interpreting evaluation results and refining both the artifact and the underlying theories based on these results. If cause–effect relationships are not fully explored, artifact construction is conceptually separated from the theoretical base and evaluation results cannot be traced back to particular features. Furthermore, existing notions of design theory have been criticized as being too complex and denying important characteristics of normal theory (Baskerville & Pries-Heje, 2010). With the focus on explaining why an artifact has the effects it does, ISDT has a strong resemblance to behavioral science theory. While this holds the potential to create and exploit synergies between theorizing efforts in design and behavioral science, the literature does not yet provide a comprehensive discussion of how to realize such potential.

While the literature has already attempted to shed light on the relationships between the different theoretical components of an ISDT, there remain some challenges with respect to fully capturing and understanding these relationships. First, the role of kernel theories remains somewhat blurry. Baskerville & Pries-Heje (2010) position general requirements and general components as the two core elements of an EDT. These correspond to the notion of meta-requirements and meta-design (Walls et al, 1992, 2004). While seeing them as loosely connected to the core theory, Baskerville & Pries-Heje (2010) do not explicitly include kernel theories in their conceptualization. This raises the question of how justificatory knowledge is related to and used within the EDT. Second, the relationship between the prescriptive and the descriptive aspects of the theory are not yet fully explored. Baskerville & Pries-Heje (2010) state that an EDT has a role in both the explanation and the construction of an artifact. It is considered ‘a constructive theory, prescriptive on the one hand, while remaining available as explanatory theory, descriptive on the other hand’ (p. 275). However, how does the abstractly described relationship between general requirements and general components look in detail if the theory serves a descriptive and a prescriptive function simultaneously?

Against this background, this paper pursues the research objective of detailing the relationships between general requirements and general components of an EDT considering both its descriptive and prescriptive functions. In this context, we develop a framework that explicates this relationship. We explore commonalities between theorizing in design and behavioral science, and on this basis, explore mutual learning potential.

To achieve these objectives, the remainder of this paper is structured as follows. First, we discuss the role of theory in design science, unveiling prevalent conflict lines and opening issues with respect to EDT. We then present a framework for explanatory ISDT that integrates existing works and applies concepts and terminology from behavioral science and the field of structural equation modeling. On that basis, we are able to unearth and map major issues and challenges in explanatory IS design theorizing. The case of electronic feedback systems is then presented as a concrete example. The final sections are concerned with discussing the implications and limitations of our research and providing an outlook for future research.

The role of theory in design science research

The role of theory in DSRIS is a controversial discussion. March & Smith (1995) argue that while theory is important for designing artifacts, theorizing does not need to be considered an integral part of design science research activities themselves. They state that both the construction and evaluation of IT artifacts are part of DSRIS, while theorizing and justifying are part of the natural sciences. They recognize, however, that a certain relationship between the two streams exists and propose that ‘theorizing in IT research must explicate those characteristics of the IT artifact operating in its environment that make it unique to IT and require unique explanations’ (March & Smith, 1995, p. 259). Similarly, Hevner et al (2004) see the IT artifact as the major result of the DSRIS. They argue that while behavioral science research should focus on truth, the main goal of design science research should be utility, recognizing that both goals are inseparable because ‘truth informs design and utility informs theory’ (Hevner et al, 2004, p. 82). Still, theorizing is conceptually separated from the activities of building and evaluating and is explicitly excluded from DSRIS.

Contrary to this view, Walls et al (1992) propose the idea of an ISDT, defined as a ‘prescriptive theory based on theoretical underpinnings which says how a design process can be carried out in a way which is both effective and feasible’ (Walls et al, 1992, p. 37). They differentiate design theories from normative theories, stating that normative theories contend what an agent should do, while design theories address how to achieve the particular goal (p. 41). These how-to statements are also referred to as design principles (Markus et al, 2002). Similarly, Gregor (2006) argues that IS theories exist for design and action (Theory type V) that are concerned with ‘the principles of form and function, methods, and justificatory theoretical knowledge that are used in the development’. She sees this theory type as being strongly related to theories of explanation and prediction (Theory type IV) because the knowledge of people as well as IT capabilities strongly informs the design and implementation of new IT artifacts. Arguing for the need for this theoretical view on design, Gregor & Jones (2007) state that ‘seeking to express IS design knowledge as theory provides a sounder basis for arguing for the rigor and legitimacy of IS as an applied discipline, in comparison with the older, more traditional disciplines in the natural sciences’ (Gregor & Jones, 2007, p. 314).

Design theories are commonly seen as dualist constructs, which relates to design being both a product as well as a process (e.g., Walls et al, 1992). In this context, Walls et al (1992) differentiate between design product and design process. Similarly, Gregor & Jones (2007) speak of principles of form and function in contrast to principles of implementation. Breaking up this dualist structure, Baskerville & Pries-Heje (2010) introduce two distinct components of an ISDT. The EDT explains why a component is being constructed into an artifact and the design practice component explains how to construct the artifact. While their conceptualization provides a simple and elegant perspective on design theory, there are certain challenges that must be addressed. These include: (1) the role of underlying kernel theories, (2) the descriptive and prescriptive nature of the EDT, and (3) issues with respect to artifact evaluation.

Challenges for EDT

The role of kernel theories in EDT

There have been several efforts to describe the relationship between kernel theories and design theories. Early research has stated that the construction of an ISDT is possible even without a complete understanding of the underlying micro theories (Simon, 1981). Similarly, Venable (2006) argues that the inclusion of justificatory knowledge in the ISDT is not required, while Gregor & Jones (2007) state that it should be included, even if it is incomplete. The strongest claim in this respect was made by Iivari (2007, p. 11), who argued that ‘without a sound kernel theory it is not justified to speak about “design theory” ’. While there now seems to be general agreement that kernel theories are valuable for the development of an ISDT (e.g., Walls et al, 1992; Kuechler & Vaishnavi, 2008; Baskerville & Pries-Heje, 2010), a detailed description of how their knowledge may be included in the conception of the ISDT is lacking. In reference to the original framework of Walls et al (1992); Kuechler & Vaishnavi (2012a, 2012b, p. 346) state that there is ‘no guidance on how the kernel theory relates to or suggests the prescribed meta-design and/or design method’.

This issue is especially relevant with respect to the explanatory aspect of design theory as ‘the explanatory information may borrow theoretical information from the natural, social, or design sciences’ (Kuechler & Vaishnavi, 2012a, 2012b, p. 396). In their definition of EDT, however, Baskerville & Pries-Heje (2010) do not explicitly include kernel theories. They view them as ‘separate background theories that form the assumption space for the explanatory design theory’ (p. 274) and state that only general requirements and general components as well as their relationships make up the core theory. Thus, the question remains as to how justificatory knowledge can be theoretically related to the explanatory aspect of design theory. Goldkuhl (2004) makes the point that the relationship between the prescribed action and the goal at the level of design theory mirrors that of cause and effect at the level of the kernel theory. Drawing on this understanding and focusing on the explanatory aspects of design theory, Kuechler & Vaishnavi (2012a, 2012b) argue that in order to fully capture justificatory knowledge, a more abstract type of design theory is required. Consequently, they propose the notion of a DREPT that augments the traditional how information content of an ISDT with explanatory information explaining why an artifact has certain effects (Kuechler & Vaishnavi, 2012a). These theories, which lie between the mostly explanatory and predictive kernel theories and the mostly prescriptive ISDT and include knowledge from both extremes, have also been referred to as high-level design theories (Kuechler & Vaishnavi, 2012b) or mid-range theories (Kuechler & Vaishnavi, 2008). Within this paper, we further explicate and formalize the relationship between kernel theory constructs and the EDT. To this end, we adopt concepts and terminology from structural equation modeling as frequently used in behavioral science.

The prescriptive and descriptive nature of EDT

Baskerville & Pries-Heje (2010) state that ‘explanatory design theory has a role both in the explanation and construction of design artifacts’ (p. 275). It can be both prescriptive and descriptive depending on the purpose of application, that is, whether it is used before, during, or after artifact construction (Baskerville & Pries-Heje, 2010). If this is the case, however, the relationship between the two components of the theory, that is, general requirements and general components, must be defined in more detail. It must be made explicit how this relationship can be both prescriptive and descriptive simultaneously. Kuechler & Vaishnavi (2012a, 2012b) solve this duality by moving the explanatory statement with respect to the design function to a new theoretical entity (i.e., the DREPT) and separating it from the prescriptive aspect, which they see as core element of the ISDT. While we believe this is a valid and useful approach, we see the need for an even simpler and more applicable EDT framework that is able to integrate both perspectives into one theoretical construct.

Evaluation and refinement of the EDT

In their conceptualization, Baskerville & Pries-Heje (2010) state that an EDT serves roles before, during, and after artifact construction. With respect to the post-construction period, they see the major role of the EDT in describing and explaining the artifact’s features as being teleological. The literature has proposed that testing and refining design theory is an important function in design science. Testing the applicability of design theory in practice is key to increasing relevance in IS research (Rosemann & Vessey, 2008). This is why testable design (product) hypotheses are often included as part of the ISDT. In this context, Carlsson et al (2010) state that ‘based on the results [of the evaluation], the outcome may be reflected on and the design theory may be refined’ (p. 117). Baskerville & Pries-Heje (2010) exclude the notion of testable design hypotheses as described by Walls et al (1992) from their conceptualization of EDT. They state that ‘hypotheses are deduced from the theory, and while possibly important for testing, are not essential to the theory itself’ (p. 274). This raises questions as to what the described deduction process may look like and how potential evaluation results may inform the refinement of the EDT. EDT operates on the relationship between general requirements and general components. Depending on the level of abstraction and the complexity of the artifact, however, a diverse set of components may be related to a diverse set of requirements. While this allows for a simple view of the entire theory, cause–effect relationships remain unclear. As there is no direct relationship between a specific meta-requirement and the component that addresses this requirement, the deduction of design hypotheses, and consequently the evaluation and refinement of the theory, may become more complicated. We argue that testable design hypotheses can be seen as a relationship between general requirements and general components, and thus should be included in the concept of an EDT.

Framing explanatory ISDT

Development of a framework

With the focus on the explanatory aspect, design theory has a closer resemblance to behavioral science theory. Benbasat (2010) stated that the main focus of design science may be how to design well based on well-defined principles, ontologies, whereas behavioral research in the context of human computer interaction might focus more on the evaluation of the resulting designs. It is commonly agreed upon that artifact evaluation provides empirical evidence that may lead to revision or confirmation of the design theory in general (March & Smith, 1995; Gregor, 2006; Carlsson et al, 2010) and its explanatory aspects in particular (Kuechler & Vaishnavi, 2012a). Thus, to address the open issues and challenges with respect to EDT, we will adopt structures from behavioral science. Particularly, we frame ISDT with the help of terminology and concepts from the field of structural equation modeling (Thompson et al, 2009; Wetzels et al, 2009).

Underlying our argument is the following definition:

An explanatory IS design theory seeks to inform a designer about which features should be included in an artifact and why. Structurally, it consists of two or more connected hypotheses, while a single hypothesis in its basic form describes the relationship between an independent variable (cause) and a dependent variable (effect). To fulfill its informative function for a designer, at least one of the hypotheses of an explanatory IS design theory must include an independent variable that can be systematically manipulated through the design of an artifact. In principle, explanatory IS design theories constitute normative theories, which means that at least one dependent variable is regarded as desirable or undesirable.

Visually, we conceptualize the explanatory aspect of the design theory with independent and dependent variables and their causal relationships as arrows. We focus on providing the means to visualize and better understand the relationship between the descriptive aspect of the EDT, that is, the justificatory knowledge embedded in it, and its constructive aspects, that is, the prescriptive elements. This is especially beneficial as design artifacts usually incorporate multiple design features targeting different causes (and potentially also different effects). Our framework helps us to grasp these complex designs and thus may assist in both design and design theorizing. In line with our defined focus on explanatory ISDT (Gregor, 2009; Baskerville & Pries-Heje, 2010; Kuechler & Vaishnavi, 2012a), factors concerning the design process (including principles of implementation or design practice aspects) are not included.

Inner model

At the core of the framework (Figure 2; Table 1 for definitions) lies the inner model, which may be based on either kernel theory constructs and relationships or the researcher’s experience. Kuechler & Vaishnavi (2012a) state that ‘the explanatory or predictive knowledge may originate in kernel theories or in experience-based insights (evidence-based justification), but it always exists; to suggest otherwise is to imply design is a random process’ (p. 398). In the former case, the model basically consists of explanatory or predictive statements from the kernel theories that explain the cause and effect relationship between the particular independent and the dependent variable. The reasoning is first and foremost deductive, that is, the designer assumes that a generic kernel theory will hold for the specific artifact implementation. In the latter case, the variables are not formally stated but only exist in the mind of the designer in the form of (unvalidated) hypotheses or thoughts. Here, the reasoning is foremost inductive or abductive, moving from observations in other projects (experience) or by means of creative imagination to implicit and tentative hypotheses with respect to the functioning of the artifact. These different forms of reasoning in DSRIS have been discussed in detail in the literature, for example, by Fischer & Gregor (2011). In both cases, the inner model comprises the justificatory knowledge and explains why the artifact has the effects it does and why certain features should be constructed into the artifact in the first place. This form of representation has particular advantages. First, it exposes independent and dependent variables and their relationships as major constituents of the justificatory knowledge as being derived from kernel theory and relates them to the other components of design theory. This explicates and visualizes the role of kernel theory in EDT. Second, it allows for multiple causes for the same effect – the implications of which will be discussed later. Third, our approach includes justificatory knowledge as part of the design theory and not as a separate theoretical entity. This allows us to better visualize the dual nature of design theories, that is, that they include both prescriptive (direct links between meta-design and meta-requirements) as well as descriptive (inner model) aspects (Baskerville & Pries-Heje, 2010).

Table 1 Key terminology of the proposed design theory framework

Outer model

The outer model is concerned with the relationships between the latent (dependent or independent) variables and their manifestations (measurements or design items). The outer model consists of two parts, the design model and the measurement model. The design model is concerned with the relationship between the independent latent variables, their corresponding meta-designs or general components and potential manifest design items, that is, instantiations within the artifact. Similarly, the measurement model is concerned with the dependent latent variables in the inner model, their corresponding meta-requirements or general requirements and potential measurements that may be used to assess whether the requirements are fulfilled. Thus, mapping terminology from EDT, we conceptualize general components as the class of all alternative design items addressing one independent kernel theory construct. Similarly, we define general requirements as the class of evaluation criteria for the dependent variable. Thus, testable design product hypotheses that are ‘used to test whether the meta-design hypotheses satisfies the meta-requirements’ (Walls et al, 1992) directly connect the design model and the measurement model. They reside within the outer model because they do not include any justificatory knowledge, that is, they state only that a particular class of artifacts will address a specific class of goals and do not express why this is the case. This knowledge is governed by the latent variables and their relationships in the inner model. This is valuable for the understanding of EDT as it allows us to trace back each meta-design or instantiation to its roots, that is, the latent variable on which it is focused. Furthermore, the derived framework allows us to identify issues with respect to design theorizing and map (see Figure 1) existing discussions on potential sources of error design research.

Figure 1
figure 1

Explanatory ISDT components and their relationships.

Identifying and mapping potential sources of error in design theorizing

The literature has addressed different errors with respect to design theorizing, particularly regarding evaluation. The DSRIS literature widely acknowledges the need to clearly differentiate between the ISDT and its particular instantiation by means of an IT artifact. In this context, Gregor & Jones (2007) differentiate between theory (components) and instantiations, which they consider ‘a different type of thing altogether’ (p. 320). Baskerville et al (2007) identified several risks associated with artifact building (the instantiation process from an ISDT) and its evaluation. Regarding artifact building, it is possible that the developed, untried instantiation may be ineffective, inefficient, or inefficacious (errors C2, C3, C4) at solving the organizational problem (Baskerville et al, 2007). Thus, there may be a conceptual gap between the prescriptive statements within the ISDT and the actual implementation, which in turn, may lead to errors in the evaluation. The improper application of the meta-design or the design method as well as the potentially improper instantiation of the solution technology may lead ‘to evaluation of something other than the correct solution technology or the ISDT as stated’ (errors D-4 and D-5) (Baskerville et al, 2007, p. 3).

Even if the abstract ISDT is followed correctly, however, many different instantiations may evolve, all addressing the same ISDT constructs. Voigt et al (2012) state that recommendations developed from design theories often must be interpreted before they can be put into action by means of an actual implementation; however, interpretation has seldom been conceptualized and addressed by DSRIS. As yet, the literature remains rather generic when describing the relationship between ISDT and artifacts. For instance, Lee et al (2011) conceptualize abstraction, that is, deriving common concepts or ideas from an instance problem, and de-abstraction, that is, the instantiation of abstract concepts by means of a specific artifact (which may still be partly imaginary), as two important theorizing activities. However, it is left open how exactly such de-abstraction should be performed or how the resulting artifact instance relates to the underlying theory constructs. One exception is the work by Kuechler & Vaishnavi (2012a, 2012b) introducing the concept of models, that is, implementation schemes, as mediating steps between ISDTs and the implemented artifact. These models are also incorporated into the conceptual framework for design and engineering in IS, as developed by Winter & Albani (2013). It is stated that models incorporate the final design details and also recognized that, in their case, ‘many alternative choices […] could have been made yielding many different models from a single ISDT’ (Kuechler & Vaishnavi, 2012a, p. 351). The authors show that alternative solutions must be considered as additional potential sources of error.

This discussion can be mapped to our framework. In terms of the design model, the designer has a certain amount of creative freedom in regard to the actual implementation of a functionality associated with a (latent) design variable. This is because kernel theory constructs are often quite generic and allow for a plethora of different solutions. As noted above, all of these different solutions form a class of artifacts hypothesized to meet a certain set of goals, and thus can be understood as the meta-design (Walls et al, 1992). The designer must interpret the independent latent construct to arrive at a manifest design choice (Kuechler & Vaishnavi, 2012b). Thus, different designers may arrive at different IT artifact features, all of which could be argued to speak to the latent design variable. This affects the generalizability of results from artifact evaluation and is especially relevant because evidence coming from this evaluation may not only feed back into the design theory but may also be used to refine the associated kernel theories (Kuechler & Vaishnavi, 2008). Carlsson et al (2010, p. 111) claimed that ‘IS design science research should develop practical design knowledge and theory to be used to solve classes of IS problems’. They stated that this should include the development of abstract knowledge that can be used to design and implement specific IS initiatives. Because an ISDT can only be evaluated by means of particular design instantiations, however, one must be careful when making generalizations about the findings. It can be argued that the results are only valid for this one particular instantiation, and especially that the feedback to the kernel theory is problematic and must be considered carefully. In the context of evaluating prescriptive statements, van Aken (2004) states that it is ‘impossible to prove its effects conclusively, but it can be tested in context, which in turn can lead to sufficient supporting evidence’ (p. 227). He states that an ISDT consists of technological rules in the form of ‘If you want to achieve Y […], then something like action X will help’ (p. 227). Here, it is important to note that the term ‘something like action X’ implies that there may be several actions, that is, design possibilities, targeted toward the same goal. This leads to the first issue associated with IS design theorizing, which has also been termed the creative leap (Kuechler & Vaishnavi, 2012a):

Issue 1:

  • Conceptual distance between a latent independent variable (cause) and its corresponding design items.

This ambiguity between latent and manifest variables can also appear in the measurement model, the second part of the outer model:

Issue 2:

  • Conceptual distance between a latent dependent variable (effect) and its corresponding measurements.

While the independent variables, that is, the causes within the kernel theory, may be addressed by differing designs, the latent dependent (effect) variables may also be measured differently. Thus, the evaluating designer or researcher has a certain freedom in how to assess whether the artifact meets the specified requirements. This conceptual distance is usually lower than the one described by Issue 1, as kernel theories likely provide measurement constructs for their dependent variables. However, there might be competing measurements addressing the same theoretical concept. The choice of these measurement items may significantly affect the evaluation results and thus require thorough argumentation.

In terms of structural equation modeling, the described aspects can be compared with issues of convergent validity. Competing designs should show similar effects on the independent variable; otherwise, they must be reconsidered. The conceptual distances described by Issue 1 and Issue 2 together show the difficulty in linking the evaluation results of a particular IT artifact feature to the design theory, or even further back, to the underlying kernel theories. This is also evident when examining the testable design product hypotheses in the model. Because they reside within the outer model, testing design product hypotheses or prescriptive statements may be subject to both of the issues described earlier.

Another issue relates to the fact that independent variables impacting the same dependent variable may influence each other. Relationships among these variables may be even more complex. Each independent latent variable can potentially be addressed by different design items (see Issue 1). Current models (e.g., Goldkuhl, 2004; Kuechler & Vaishnavi, 2012a, 2012b) take into account one cause–effect relationship in the kernel theory/mid-range theory. Interdependencies between different causes for one effect, and thus between statements addressing the same goal, are not yet accounted for.

Issue 3:

  • Potential interdependence of simultaneously implemented design items.

This relates to issues of discriminant validity of latent variables. The effects of a particular design item may not always be clearly distinct with respect to one independent variable, but may simultaneously target multiple ones.

Implications for the IS design theorizing process

The discussion of conceptual issues also has implications for the IS design theorizing process. In the current literature, a testable design product hypothesis is usually considered to be related to an on or off situation, meaning that an artifact with a particular feature is hypothesized to perform better on a certain measure than one without it. For instance, following Gregor (2009, p. 7), exterior mode design theory can include propositions such as A system with feature X will perform better on measure M than a system without feature X (emphasis added). Such hypotheses may be empirically tested, and if validated, be inverted to arrive at a design proposition suggesting to include feature X to improve measure M. This does not take into account equifinality, however, or different ways to arrive at a design goal. Examining our framework, the two following distinct aspects can be distinguished with respect to alternative designs: (1) Design choices addressing the same independent variable (and, thus, also the same dependent variable) in the kernel theory. This has been discussed under the term creative leap and refers to Issue 1. (2) Design choices addressing different independent variables in the kernel theory but targeted toward the same effect (dependent variable).

Current ISDT conceptualizations do not yet take into account potential knowledge that stems from comparing alternative designs. Speaking in terms of Gregor (2009), what if the proposition that we wish to test relates to systems with two competing features that are both related to the same theoretical grounding and are both targeted at addressing the same measure: A system with feature X will perform better on measure M than a system with feature Y? Most behavioral science (kernel) theories, the basis for ISDT (Iivari, 2007), do not contain a single cause–effect relationship. Instead, they are rather complex and include sets of variables that furthermore may be connected using mediation or moderation. Thus, because the statements of the ISDT may correspond to either one of these relationships, a variety of ISDT statements may exist that are all targeted toward the same design goal. Similarly, the solution space for each of these prescriptive statements may be rather large, that is, there may be several ways in which to implement each design principle. For fundamental and critical design decisions, both aspects must be considered unhelpful for the designer and may result in lock-in situations and potentially sub-optimal designs. This is especially relevant because it has been proposed that human designers are likely to settle for good or satisfactory solutions rather than optimal ones (Simon, 1996). While this is not an issue in most cases, optimality may be desirable in other ones. Thus, competing or interdependent cause–effect relationships in the kernel theories may lead to the two following issues in design theorizing:

  1. 1)

    Optimal design is random. Because there is no comparison of design decisions, there is no deterministic way to arrive at an optimal design or even to determine the better one among a set of design options. Current frameworks are unable to capture the knowledge generated from comparing artifacts with respect to a certain evaluation dimension, that is, the knowledge of which of the competing design alternatives is more effective to reach a certain goal or improve a certain measure. For critical design decisions, this may become an issue.

  2. 2)

    Path dependencies lead to lock-in situations. Different design dimensions often influence each other. Thus, if a fundamental design decision is taken, the solution space for other design decisions is limited. For instance, the choice of technological frameworks in programming often determines the way certain functionalities may be implemented and thus the design of the final product, for example, in terms of look-and-feel. Depending on the measure that the designer is trying to address with his design, this limitation of possible solutions leads to the issue that the later, the design may only be optimized locally, that is, within this specific design path. This is related to Issue 3 as discussed above.

It is commonly accepted that designers take many design decisions without formally evaluating alternatives a priori. Gregor (2009, p. 7) states that ‘experimentation in design research can have a different role from that usual in the theory testing cycle of the H-D method. […] In anything more than a trivial design problem the designer will make very many design decisions and it would be infeasible to test every design decision by conducting a formal experiment’. In that light, the design theorizing process builds upon experimentation and trials. While we agree with Gregor (2009) that it is infeasible to test all alternative designs, we argue that there may be situations when a thorough and formal evaluation of design alternatives is necessary. For instance, for medical systems where user errors must be minimized as far as possible, the evaluation of competing designs with respect to this dimension may be desirable. The following case shows that comparing different designs may also be desirable in the context of high-level design decisions that impact the solution space for other (lower-level) design decisions.

The exemplary case of electronic feedback

Derivation of design principles from kernel theories

To illustrate the mentioned relationships and issues, we will discuss the design of an electronic feedback system as a case. Here, we understand feedback interventions as ‘actions taken by (an) external agent(s) to provide information regarding some aspects of one’s task performance’ (Kluger and DeNisi, 1996, p. 255). Feedback has been found to be an important element in motivating employees and improving overall workforce performance (Moss & Martinko, 1998). Here, the usefulness of the feedback is influenced by the level of social interaction with the person giving feedback (Earley, 1986). Thus, designing a useful feedback system is strongly related to improving social presence, that is, the ‘warmth’ of the communication (Short et al, 1976) within the tool. To inform our design, we seek kernel theories constructs that were found to positively influence social presence in computer mediated communication. Here, two major factors can be derived from social presence literature: (1) human embodiment and (2) media richness.

Human embodiment: In the context of feedback, the sender can either be human (e.g., the supervisor) or the task performance may be evaluated and directly output by an information system. In the IS literature, there are several examples of human embodiment to increase perceived social presence (Cyr et al, 2009; Hess et al, 2009; Qiu & Benbasat, 2009). Here, studies have examined both human senders (e.g., Zhu et al, 2010) and the artificial imitation of humans. For the latter, examples include text-to-speech (TTS) output (e.g., Qiu & Benbasat, 2009) or the use of recommendation agents (e.g., Hess et al, 2009). Thus far, however, no comparison of both possibilities has been provided. Thus, as there is evidence that both types of human embodiment positively influence social presence, no final ‘best design’ can be derived from the kernel theories with respect to this dimension. On the basis of the kernel theory, however, we can formulate the following design principle:

DP1: An electronic feedback system should feature some sort of human embodiment in order to increase social presence for the receiver.

Media richness: For the second factor, media richness, the situation is different. Here, several studies have shown that higher media richness, that is, the ability of a medium to convey social cues, leads to higher perceived social presence (e.g., Yoo & Alavi, 2001; Aljukhadar et al, 2010; Zhu et al, 2010). However, while early studies on social presence and media richness suggest that there are significant differences between text, audio, and video conditions (Short et al, 1976; Daft et al, 1987), more recent studies do not consistently confirm these results. In a recent study of media richness in Website design, for example, only the difference between text and video was found to be significant (Aljukhadar et al, 2010). On the basis of the findings, we formulate the following design principle:

DP2: An electronic feedback system should give feedback using a rich medium in order to increase social presence for the receiver.

Figure 2 shows the applied ISDT framework. The inner model shows the two independent variables, human embodiment and media richness, and their hypothesized cause–effect relationship with the dependent construct of social presence. The design model shows that for the case of human embodiment, there are two design alternatives that may both satisfy the meta-requirement of increasing social presence. Thus, both design principles, that is, prescriptive statements, can be visualized as arrows from the meta-designs for human embodiment and media richness to the meta-requirement. They span the (latent) inner model and do not provide any (explicit) information with respect to why the artifact will yield a higher social presence. This information is only implicitly given because the design principles are derived deductively from existing theory. The following section will discuss the framework in terms of the conceptual issues as developed in the previous section.

Figure 2
figure 2

ISDT framework applied to the case of electronic feedback.

Discussion of conceptual issues in ISDT

Discussion of Issue 1: Conceptual distance between a latent independent variable (cause) and its corresponding design items. With respect to the human embodiment construct, the only information that could be drawn from existing kernel theories was that some sort of human embodiment increases social presence. In the existing studies, this embodiment is either a real person or an artificial representation of a human. As there have been no comparisons between the two choices, which option will lead to higher perceptions of social presence has not yet been theoretically validated. While it may be hypothesized that the human setting will increase the level of perceived social presence as computers are just trying to imitate humanness, this is not yet empirically supported. Thus, with respect to the kernel theories, both design options may fulfill the meta-requirement. As no ‘ideal design’ can be derived, we as designers must choose which option to implement.

Discussion of Issue 2: Conceptual distance between a latent dependent variable (effect) and its corresponding measurements. There are several measurements of social presence available in the literature. For instance, Zhu et al (2010) use a seven item measurement based on Short et al (1976), while other studies frequently adopt the 5-item measurement as developed by Gefen & Straub (2003). As evaluation results may differ among these measures, the design researcher should defend why a particular measure was chosen. In our case, the most popular measure is the one developed by Gefen & Straub (2003), which has been widely applied in IS literature (Cyr et al, 2009; Al-Natour et al, 2011).

Discussion of Issue 3: Potential interdependence of simultaneously implemented design items. In the case context, the design decision with respect to the human embodiment influences the solution space for instantiations of the media-richness construct. For instance, audiovisual feedback in the human scenario could be a video-chat while a possible instantiation in the artificial scenario would be an avatar. Thus, the design decision with respect to human embodiment may influence the result of the artifact evaluation later on. For instance, in their study on recommendation agents, Qiu & Benbasat (2009) found that the difference between text and audio is only significant if a human voice is applied instead of a TTS algorithm. As no comparison of the perception of social presence has been conducted with respect to the two possibilities, and thus, there is no conclusive knowledge on this aspect available in the kernel theories, a designer can only ‘guess’ which one will yield better results. Because the decision influences the solution space for other design decisions (in our case the one for media richness), however, this may lead to lock-in situations with respect to the final artifact.

Comparative design evaluation

On the basis of the kernel theories, a designer may struggle with the question of which basic design path to pursue. Thus, if the design is required to be ‘optimal’, it may be beneficial to compare several alternatives with respect to the dependent variable that needs improvement. While it is not feasible to conduct this process for all design decisions (Gregor, 2009), it may be especially valuable for critical ones, or as in our case, for those decisions that have implications for other design dimensions. In addition, trying out different design alternatives, that is, the simultaneous implementation and subsequent evaluation of different artifact ‘versions’, has become easier because of agile development methods and rapid prototyping. However, designers may choose to evaluate the design alternatives in a way that is less formal and structured. For our case, we are able to conduct a formal experiment.

Methodology and data collection

To find the ideal design with respect to social presence, we implemented three different media-richness levels in both conditions for human embodiment and conducted an experimental evaluation. The different media-richness levels were added to visualize implementation possibilities within our framework. Here, a designer could have implemented only the highest media-richness level as this dimension is completely explained by the kernel theory. Table 2 shows the design instantiations that were tested as part of the experiment.

Table 2 Design instantiations

We used the social presence measure developed by Gefen & Straub (2003) to measure the dependent variable. The items used are depicted in Table 3. Feedback was given for a simple IT-based anagram solving task. These tasks have frequently been utilized to represent simple/routine work tasks in both IS and psychological studies, especially in those related to social presence (Aiello & Svec, 1993; Davidson & Henderson, 2000; Park & Catrambone, 2007). We applied a 3 × 2 within-subject design according to the two treatment dimensions, that is, each participant had to complete all six treatments. We iterated the treatment order to prevent biases because of training effects. In each setting, the participants had to solve 20 anagrams, receiving external performance related feedback after each 25% of task completion. The feedback message included information on the number of correctly solved anagrams and the average time taken to complete one anagram within the set. We conducted the study with 43 participants (average age: 23.9, 17 females, 25 males). The perceived social presence levels in each condition were compared among each other to identify the most effective artifact design with respect to this dimension. To this end, we applied a mean-based comparison using t-tests for the evaluation of significances.

Table 3 Measurement items for perceived social presence

Results

With respect to the human embodiment dimension, we found evidence that human feedback leads to significantly higher levels of social presence than feedback messages generated artificially. Regarding the media-richness dimension, our findings support the kernel theories. Here, the audio condition led to significantly higher levels of social presence than the text condition in both the artificial and the human condition. Similarly, participants reported significantly higher social presence levels for video compared with audio feedback. Table 4 shows the means and standard deviations for all experiment conditions while the t-test results are presented in Table 5.

Table 4 Means and standard deviations
Table 5 Mean differences and t-test results

Therefore, video call functionality was found to maximize social presence levels in the context of electronic feedback. Thus, with respect to the two dimensions tested in the experiment, we were able to identify the most promising artifact design.

Discussion

Implications for DSRIS

Our work contributes to design science research in various ways. First, we develop and apply an EDT framework making use of terminology and concepts that stem from the field of structural equation modeling (including the inner and outer model, latent and manifest variables, and measurement model) and develop additional ones (design model, design items). In this context, we explicate the relationships between the different theory components of explanatory design theories and show that the core of these theories closely resembles behavioral theory. Second, with the help of that framework, we identify and discuss major challenges for explanatory IS design theorizing. Particularly, we visualize the implications of (kernel) theory variables that are often latent, which means that they are not directly designable (Issue 1) or observable (Issue 2). Design science researchers face a creative leap (Kuechler & Vaishnavi, 2012a) when translating kernel theory variables into concrete IT artifact features. By means of our framework, we visualize this creative leap and explore its impact with respect to interpreting evaluation results and refining both the artifact and the underlying (kernel) theories. In addition, we highlight that the creative leap in IS design theorizing not only exists at the end of designing IT artifacts but also at the end of effect measurement. Moreover, we introduce theorizing about alternative design items, two or more potential IT artifact features that speak to the same latent kernel theory variable.

Our work contributes to the explanatory shift in IS design theorizing and opens up new research perspectives. There have been numerous calls for the inclusion of kernel theory (from natural and social sciences) in design theorizing (Walls et al, 1992; Gregor & Jones, 2007; Iivari, 2007). With the concepts of mid-range theories (Kuechler & Vaishnavi, 2008), DREPT (Kuechler & Vaishnavi, 2012a), exterior mode design theory (Gregor, 2009), and the framework presented in this paper, we are coming closer to achieving that goal as well as finding ways to operationalize this inclusion and translation. This leads to a point at which behavioral science research and explanatory IS design science research seems to be difficult to distinguish. From our perspective, the difference between behavioral theory and EDT lies in their individual purposes. In behavioral research, the focus is on developing and testing theory, that is, hypothesizing and testing the relationship between different constructs. The actual design instantiation is just a means to this testing. In design science research, however, the focus is to assist designers in building artifacts. An EDT answers the question of why an artifact has certain effects and thus provides knowledge that can be used for artifact construction targeted toward a particular outcome or goal. Thus, theory could be considered a means to the end of assisting in artifact design.

This discussion also has implications for design science methodology, particularly artifact evaluation. First, it must be recognized that different evaluation measures, although developed to target the same effect, may yield different results. Hevner et al (2004) state that measures and evaluation metrics are crucial components of DSRIS and that ‘[d]esign-science researchers must constantly assess the appropriateness of their metrics’ (p. 88). Our framework suggests that it may be beneficial to evaluate artifacts along several competing measures, especially if the findings must be as accurate as possible. Second, it is important to identify interdependencies among different (classes of) design items early in the design process as fixing one item may limit the solution space.

Our case study on electronic feedback systems shows how to assess the value of these design item alternatives and thus addresses the creative leap. More generally, we add to Kuechler & Vaishnavi’s (2008, 2012a, 2012b) work on bridging between rather abstract kernel theories and concrete IT artifacts in design theorizing. In addition, we identify the challenge of interdependent artifact features (Issue 3) and present an approach that takes such interdependence into account. We contribute to explanatory IS design by theorizing the evaluation of alternative designs and provide an exemplar of a design theory that contains statements in the form of ‘A system with feature X will perform better on measurement M than a system with feature Y or Z’.

Implications for designing electronic feedback

On the basis of the concrete example, we also contribute to the body of (explanatory) ISDT. An electronic feedback system (=class of systems) with a video functionality (=IT artifact feature) will perform better on social presence (=goal/effect) than those with text or audio-only functionality. If video functionality is not in the solution space (e.g., for reasons of bandwidth), audio functionality leads to higher social presence than text functionality. This holds true for feedback by humans as well as artificial agents. Moreover, given the concrete designs chosen, if feedback in an electronically mediated communication is provided by human agents, social presence will (still) be higher compared with artificial agents.

Our example is able to contribute to the different explanatory approaches to IS design theorizing (Gregor, 2009; Baskerville & Pries-Heje, 2010; Kuechler & Vaishnavi, 2012a). First, we ‘theorize about artifacts in use’ (Gregor, 2009, p. 8) and contribute a set of interrelated statements linking concrete IT artifact features to specific measures. This can be considered exterior mode design theorizing. Second, we contribute to developing a DREPT for electronic feedback systems as we explain ‘why the artifact has the effects it does’ (Kuechler & Vaishnavi, 2012a, p. 396). We make use of kernel theories (e.g., Short et al, 1976) and IS theories (e.g., Cyr et al, 2009; Zhu et al, 2010) and translate their dependent and independent variables to IT artifact-relevant constructs (Kuechler & Vaishnavi, 2012a). In terms of EDT (Baskerville & Pries-Heje, 2010), we show that general design components (human embodiment and media richness) can be related to general requirements (social presence). With our contribution to EDT, we seek to ‘explain[ ] why a component is being constructed into an artifact’ (p. 275). The unique trait of Baskerville & Pries-Heje’s (2010) conceptualization of EDT is that it remains on the level of general components and does not describe concrete IT artifact features. Overall, we find an explanatory shift in IS design theorizing that focuses on explaining why an IT artifact has the effects it does.

Taking our example of electronic feedback systems, from a design science perspective, we evaluated six competing IT artifact designs with respect to the general requirement of social presence. From a kernel theory perspective, we evaluated the effect of two variables on social presence by means of different implementations. So is this (explanatory) design science? Here, like Gregor (2009), we argue that the IT artifact needs to be central to design theory and second Kuechler & Vaishnavi (2012a), who see the defining criterion for a design theory (dependent and independent variables) as being clearly rooted in the technology domain. Taking into account these arguments, this is in fact a piece of design science research; however, is it also behavioral science research? While we can debate the extent of the contribution, it clearly attempts advancement in the field of social presence theory by testing interrelated theoretical statements about human embodiment as well as media richness. As a result, we see how structurally similar theorizing in these research streams can be. It becomes obvious that the major difference lies in how the researchers position their work and to what body of knowledge they attempt to actively contribute. This may pose a challenge for that type of research. The work can be easily assessed by two quite different sets of research standards: Does it have enough to say about how to design innovative (IT) artifacts? Are the design implications non-trivial? And further, is the advancement of (here: social presence) theory significant enough? Is it empirically rigorous enough? We see, however, that spanning the (partly artificial) boundaries between these different research traditions is potentially the most fruitful avenue to make use of dispersed design-relevant knowledge and ultimately to build better information systems.

Limitations and outlook

This work leaves several issues open and has limitations that should be addressed by future research. We referred, for instance, to exterior mode design theory (Gregor, 2009), EDT (Baskerville & Pries-Heje, 2010), and DREPT (Kuechler & Vaishnavi, 2012a) and built a case for a greater explanatory shift in IS design theorizing. With cross-referencing between these papers being largely absent (most possibly because of publication timing issues), future research could attempt to integrate these works on a deeper level than an applied piece such as the present one can do. Moreover, we were able to present a single case example, electronic feedback systems, but the described issues and possible solutions will become more informative in the light of more cases. It could also prove revealing to apply the framework presented in this paper to IT-oriented research that is located close to the behavioral science end of the spectrum. In addition, future studies could also focus on further implications our work has for design science methodology and integrate these aspects into existing methodological frameworks.