Introduction

With the proliferation of many outlets for publishing IS research in recent years, there appears to be an increasing trend to conducting research on research. This phenomenon is longstanding in the IS literature – indeed many of the seminal papers in the IS discipline could be classified as 'research on research' (e.g., Culnan, 1986, 1987; Markus & Robey, 1988; Orlikowski & Baroudi, 1991; Orlikowski & Iacono, 2001). Over the past two decades, there have been 35 published studies that have addressed topics such as the rankings of IS journal outlets (Doke & Luke, 1987; Koong & Westrofer, 1989; Gillenson & Stutz, 1991; Holsapple et al., 1993, 1994a, 1994b; Jackson and Nath, 1989; Nord & Nord, 1995; Walstrom et al., 1995; Hardgrave & Walstrom, 1997; Walczak, 1999; Mylonopoulos & Theoharakis, 2001; Katerattanakul & Han, 2003; Lowry et al., 2004), the 'most prolific' IS scholars (Trower, 1995; Im et al., 1998; Athey & Plotnicki, 2000; Claver et al., 2000; Huang & Hsu, 2005), the most 'productive' IS research institutions (e.g., Vogel & Wetherbe, 1984; Lending & Wetherbe, 1992; Eom, 1994), the most widely cited IS papers (Walstrom & Leonard, 2000; Whitman & Woszczynski, 2003) or the most prolific researchers in specialty areas within the IS discipline, such as decision support systems (DSS) (Holsapple et al., 1995; Forgionne & Kohli, 2001), group support systems (GSS) (Holsapple & Luo, 2003), ecommerce (Bharati & Tarasewich, 2002), expert systems (Cheng et al., 1994), accounting IS (Daigle & Arnold, 2000), or reviews of the top researchers or journals in fields such as library and information science (Meho & Spurgin, 2005), computer science (Katerattanakul et al., 2003), and operations management (Goh et al., 1996, 1997; Barman et al., 2001). IS researchers have even identified IS scholars' views of the best IT practitioner journals (Hsieh et al., 2001), and have developed sophisticated mathematical procedures for identifying the highest-quality IS journals (e.g., Cooper et al., 1993; Kleijnen & van Groenendaal, 2000; Forgionne & Kohli, 2001), including a formula for consolidating the results of nine prior journal ranking studies (Rainer & Miller, 2005).

Studies assessing the most productive or 'prolific' researchers and institutions have existed in the IS literature since the dawn of our discipline (Vogel & Wetherbe, 1984). In recent years, such studies have proliferated – with one U.S. scholar alone (Clyde Holsapple) co-authoring eight such studies (Holsapple and Luo, 1993, 1994a, 1994b, 1995, 2003; Cheng et al., 1994; Goh et al., 1996, 1997). Given the questions that have been raised regarding the excessive emphasis on rankings of prestigious MBA programs or other professionals schools (Stake, 2004), one might question whether such an emphasis on identifying and ranking the most 'prolific' IS researchers and most productive institutions is truly valuable to faculty and students in the IS field, or whether this borders on 'form over substance' (Gioia & Corley, 2002). We were especially intrigued by the fact that the latest such installment in the genre of studies that rank IS researchers based on numbers of published articles (Huang & Hsu, 2005) yielded a list of the 'Top 30' most prolific IS scholars that features no Europeans and just two women scholars – both in North America. In this paper, we examine the research methodology of this recent study, comparing it to prior studies that constructed similar rankings of the most productive IS scholars and observing that, by choosing what to count (and what not to count), implicit assumptions pervade such studies – with these assumptions directly (but subtly) influencing the results that ensue. Specifically, we examine how ranking studies tend to exclude large segments of the IS researcher population – in this case, European scholars – who may be extremely productive (Galliers & Whitley, 2002), although they may be slighted by the specific outlets counted in such studies which have, to date, exclusively focused on North American journals.

Critique of studies of 'prolific' IS researchers

In her renowned essay titled 'Do Categories Have Politics?', Lucy Suchman (1994) draws on speech–act theory to develop the central argument that all categories are political. How we classify persons, objects, and events – including what is or is not counted – rests on a series of political decisions that both reflect and, in turn, influence the allocation of power. Referencing the work of sociologist Harvey Sacks on the politics of categories and labeling, Suchman (2002, pp. 96–97) argues that the process of categorizing objects and events serves as: ... a fundamental device by which all members of any society constitute their social order.... [T]he sociologist Harvey Sacks was concerned ... with the role that categorization plays in contests over the control of [group members'] social identities.... Sacks' analysis identifies the relation of categorization devices to social identity, including assessments of persons' adherence to the moral and aesthetic sensibilities associated with a particular category. It points as well to the ways in which categorization can be ... a resource in the development of more elaborated and formalised systems of social control.

In this vein, it is important to consider the merits of constructing lists of most prolific scholars, as well as the types of categories that are used in constructing them – categories such as 'high-quality IS journals' or 'scholarly journals,' in general. In their recent ranking of the 'Top 30' most productive IS scholars (see Appendix A), Huang & Hsu (2005) counted IS articles published between 1999 and 2003 in a set of 12 journals which were largely consistent with those counted in prior studies that ranked the most productive IS scholars and/or institutions (e.g., Lending & Wetherbe, 1992; Im et al., 1998; Athey & Plotnicki, 2000; Lowry et al., 2004). The journals employed in Huang and Hsu's study are summarized in Table 1, along with cross-references to several prior studies that evaluated the quality of IS journals. In comparing the set of journals used by Huang and Hsu to a similar study by Athey & Plotnicki (2000) five years earlier, we see that they expanded the set of journals slightly: first by adding Decision Support Systems, a specialized journal focusing on DSS research, and second, by adding the Journal of the AIS to their list (an electronic journal published by the Association of Information Systems, first published in 2000). Other than these two additions, their list was consistent with that of Athey & Plotnicki (2000).

Table 1 - Journals employed by Huang and Hsu (2005) and comparative rankings in prior studies of IS journal quality.

Full table

Although Huang and Hsu's journal list is generally consistent with the prior study of IS researcher productivity by Athey & Plotnicki (2000), as well as with earlier studies that ranked IS journal quality (summarized in Table 1), what is overlooked in Table 1 is the fact that the most recent journal ranking study in Table 1 (Mylonopoulos & Theoharakis, 2001) ranked European Journal of Information Systems (EJIS) as a high-quality journal, but neither EJIS or any other European journal were counted in any of these studies ranking the most productive IS scholars. In the recent global survey conducted by Mylonopoulos & Theoharakis (2001), EJIS was ranked 11th by respondents worldwide; Information Systems Journal (ISJ), Journal of Strategic Information Systems (JSIS), and Information Technology & People (ITP) were ranked number 16, 20, and 27, respectively. We summarize the results of many prior studies that ranked IS journals, scholars, and institutions in Appendix B.

Not only do we consider it problematic that EJIS and these other leading European scholarly journals were excluded from researchers' publication counts, but a cursory review of the journal titles that were included in Huang and Hsu's study reveals a host of other problems. In our opinion, all such studies of the 'most productive scholar' genre mask several unexamined assumptions about the choice of 'high-quality' IS journals. These include assumptions about which journals are 'high-quality,' but, more fundamentally, assumptions about what counts as a scholarly IS journal in the first place – as opposed to journals in other disciplines that publish some IS research. As we discuss below, many of the journals included in Huang and Hsu's study are popular and widely read by practitioners, but they are not scholarly IS journals. Some others represent journals that primarily serve academic researchers in disciplines other than IS, although they sometimes publish IS research. Moreover, if the list of top journals used in such studies is simply expanded incrementally based on the lists used in prior studies, then the problem of 'path dependence' emerges (Whitley & Galliers, 2005). By this we refer to the fact that the authors' choice of journals to examine in any given study of high-quality IS journals will be influenced by whatever journals were in the study before that, and so forth, going back in time to the earliest studies of this genre (e.g., Vogel & Wetherbe, 1984). The problem of path dependence is reflected by the fact that authors who examine publications in high-quality journals often just incrementally update the list of 'top' journals from the prior study. If all authors simply update such lists incrementally – going back to the first studies that identified the most-prolific IS researchers (e.g., Remus, 1989, 1991) or the institutions with most productive researchers (e.g., Lending & Wetherbe, 1992; Vogel & Wetherbe, 1984) – most of which were published before the emergence of journals such as Information Systems Research (in 1990), European Journal of IS, JSIS (both in 1991), ISJ (in 1995), and Journal of the AIS (in 2000), then they will necessarily overlook these newer but possibly higher-quality outlets – in favor of journals whose primary attribute may be their longevity. We consider it surprising and problematic that the authors of the most recent studies identifying the most prolific IS researchers neglected to include the leading European IS journals (such as EJIS, ISJ, JSIS, and ITP), despite the fact that all of these high-quality European journals were founded prior to 1995, and the studies in question counted publications from the mid-to-late 1990s (in the case of Athey & Plotnicki (2000)) or well into the beginning of the new millennium (for Huang & Hsu (2005)). Indeed, given the fact that several of these European IS journals frequently rank among the 'Top 10' scholarly IS journals worldwide, according to Galliers & Meadows (2003), it is problematic that they have been either intentionally or unwittingly overlooked in these most recent studies of IS researcher productivity.

The other problem with studies that employ lists of high-quality journals, which are expanded incrementally from prior studies (as do Huang and Hsu, by adding Decision Support Systems and Journal of AIS to their list, but not seriously challenging the journals that already comprise the list of so-called top journals), is that these authors never consider the question of whether a given journal may have 'slipped' in the rankings – perhaps due to the emergence of other, higher quality journals, or due to other editorial policy changes. As is well known in the case of Communications of the ACM (CACM) (see Dennis et al., 2006, p. 9), this journal transformed its editorial policies and target readership during the mid-1990s to serve a practitioner audience, rather than a scholarly one. As noted by Nord & Nord (1995, pp. 38, 40) in their study, 'Journal Status Assessment and Analysis,' CACM 'restructured its format to include shorter articles with fewer references [in the early 1990s, but] ... Perhaps the CACM change with the most significant impact is the increase to about 90 percent its articles published [by] practitioners. This may affect its rankings in future studies.' The fact that by 1997, all articles in CACM were held to very terse word limits (3,000 words) along with severe constraints on the total number of citations (maximum of 12 citations), seems to have been overlooked by many who have classified high-quality IS journals over the past five years, and who still continue to rank CACM as a 'Top 5' IS journal (Mylonopoulos & Theoharakis, 2001; Lowry et al., 2004; Rainer & Miller, 2005). In general, since these studies are based on researchers' perceptions of journal quality, there may be a delay in the adjustment of such perceptions to reflect the realities of such editorial policy changes. It is perhaps also noteworthy that nearly all of the studies that have evaluated and provided rankings of IS journals over the past decade were themselves published in CACM (Walstrom et al., 1995; Mylonopoulos & Theoharakis, 2001; Katerattanakul et al., 2003; Schwartz & Russo, 2004; Barnes, 2005; Rainer & Miller, 2005). ISWorld provides a comparison of MIS journal rankings in different studies at: http://www.isworld.org/csaunders/rankings.htm.

We believe there are several unexamined assumptions related to the politics of how IS journals are categorized as 'high-quality scholarly journals' vs 'low-quality scholarly journals,' or as 'niche-area scholarly journals' vs 'popular practitioner outlets.' We believe that many of the journal ranking studies have conflated the notion of journal name recognition (which tends favor practitioner outlets and scholarly journals with broad coverage) with the level of journal quality – at the expense of some very high-quality journals that serve niche areas, for example, database management, design research, or interpretive research. We believe that it is important for researchers who are conducting studies of IS journal quality – or even those of us simply reading such studies – to ask the questions: 'What are scholarly IS journals?', 'What are popular, high-impact practitioner journals, which have different review criteria for publishing their contents?' and 'Can we distinguish scholarly IS journals from scholarly publication outlets in other disciplines?' We state these questions explicitly because, in the analyses that led to the creation of recent lists of 'highly prolific' IS researchers (Im et al., 1998; Athey & Plotnicki, 2000; Huang & Hsu, 2005), these authors counted publications in journals that might best be regarded as high-impact practitioner magazines (e.g., Harvard Business Review (HBR), Sloan Management Review, and CACM), as well as scholarly journals that focus on other disciplines, such as operations research or operations management (e.g., Management Science and Decision Sciences), or computer science (IEEE Transactions on Software Engineering) and which consequently, publish few papers by IS researchers. We question the inclusion of these practitioner magazines and non-IS, academic outlets in counting publications of IS researchers, at the expense of more focused scholarly IS journals, including several leading European journals (EJIS, ISJ, JSIS and ITP). We are especially strong in our critique of the set of journals selected in the two most recent studies (Athey & Plotnicki, 2000; Huang & Hsu, 2005) – particularly given the fact that the first set of authors featured data indicating that some of these journal outlets publish under 20% of their studies by IS scholars. For example, Athey & Plotnicki (2000) presented data showing that the fraction of articles having one or more IS scholars as authors was as follows: Harvard Business Review (0.5%), Sloan Management Review (18%), CACM (15%), Management Science (12%), and IEEE Transactions on Software Engineering (14%). The only one of these journals where the proportion of papers with at least one IS scholar as authors exceeded the 20% threshold was Decision Sciences (for which Athey and Plotnicki reported that 36% of its papers contained at least one IS academic author). We thus challenge the inclusion of high-impact, but non-scholarly magazines that primarily serve the general management community (HBR and Sloan Management Review) or IT practitioner community (CACM), as well as academic outlets written by and for scholars in fields other than IS (e.g., Management Science and IEEE Transactions on Software Engineering). In contrast, based on the data provided by Athey & Plotnicki (2000), we consider Decision Sciences to be a journal that routinely publishes articles by and for IS researchers, as well as for scholars from related disciplines, such as operations research and operations management.

The authors of these recent North American rankings of most prolific IS scholars note that the choice of journals will affect which authors and academic institutions are ranked most highly. For instance, Athey & Plotnicki (2000, p. 11) assert that 'the importance of the decisions as to which journals to include ... cannot be overemphasized.' Given this statement and the fact that several leading European journals began publication in the early-to-mid 1990s, it is surprising that Athey and Plotnicki neglected to include European scholarly journals in their study. Perhaps not surprising, was the fact that there were no European scholars listed among the 30 most productive scholars in any of these studies of most productive IS researchers. In the interest of full disclosure, we acknowledge that several scholars from outside of North America have appeared in these results, but they were from countries such as Hong Kong (Patrick Chau), and Singapore (K.K. Wei, Bernard Tan, and Kar-yan Tam). There were, however, no Europeans appearing among the list of top-ranked IS researchers in these studies. Given evidence that European IS scholars publish much more frequently in European IS journals than in North American journals – and vice-versa for North Americans (Galliers & Meadows, 2003), this neglect of European IS journals would seem to ensure the absence of European scholars from the set of results.

A comparable gender analysis of Huang and Hsu's 'Top 30' most prolific researchers raises similar questions, especially given the fact that women comprise a large proportion of the authors of scholarly IS research today. We note that, in Huang and Hsu's study, covering 1999–2003, only two women from North American universities were featured among the 'Top 30' scholars (Ritu Agarwal and Anne Massey), whereas in Athey and Plotnicki's study (covering publications from 1992 to 1997), five women appeared in the list of the 24 most productive scholars (Sirkka Jarvenpaa, Iris Vessey, Bernadette Szajna, Maryam Alavi, and Wanda Orlikowski). We believe that the choice of journals to include (and which others to exclude) may also influence the balance of male and female scholars appearing in the set of results, since some scholarly journals have a higher proportion of women authors than other journals.

With regard to the politics of the category 'scholarly IS journal,' Athey and Plotnicki note that the creation of such lists of prestigious journals tends to favor well-known, generalist publications, even if they are directed to a practitioner audience (such as HBR or CACM) over high-quality academic journals that specialize in a particular niche area of scholarly research: The standard methodology of all these studies [that identified the top IS journals] was a survey instrument. The very nature of a survey eliminates any specialized journal as a top tier publication. For example, a researcher specializing in database would consider the ACM Transactions on Database Systems a premier journal, [but] due to the general survey audience in the above studies, only journals of general interest [to IS scholars] would receive the necessary votes to be classified as top-tier. – Athey & Plotnicki (2000, p. 6).

Thus, high-quality specialized outlets, such as the 'IEEE Transactions ...' or 'ACM Transactions ...' journals, tend to score much lower in surveys of journal quality, as rated by IS scholars, compared to generalist, practitioner journals (e.g., HBR, CACM), and below other general, scholarly IS journals that may have less rigorous criteria for review and acceptance (e.g., Information & Management or Journal of Computer Information Systems). There is also the question of whether some leading North American journals are biased toward positivist, empirical research. Several recent studies have shown that North American IS journals publish relatively little qualitative and/or interpretive studies, relative to their European counterparts (Walsham, 1995; Chen & Hirschheim, 2004), and that papers published in European IS journals (Galliers & Meadows, 2003) and European Ph.D. dissertations (Evaristo & Karahanna, 1997) are more likely to be conceptual or qualitative in nature, rather than positivist, theory-testing studies. One conclusion is that if North American journals are not amenable to qualitative, interpretive, or critical theory studies – or if they have only recently become more open to publishing such studies (see Markus & Lee, 1999), then European authors will be underrepresented in counts of articles published in North American journals during the mid-to-late 1990s. Some scholars have even suggested that certain leading North American journals are still not wholly receptive to alternative research methods (Chen & Hirschheim, 2004). Introna & Whittaker (2004) invoke Foucault's (1977) notion of 'regimes of truth' in alluding to MIS Quarterly's publication practices which, they argue, have historically been biased against interpretive and critical research. Based on their keynote address at the 2002 ECIS conference analyzing who publishes where and what they cite, Galliers and Whitley report 'interesting patterns that strongly suggest that European researchers have a different research profile to those reported elsewhere' (2002, pp. 11–12). This is supported by evidence that European Ph.D. dissertations tend to employ different research methods and leverage distinct reference disciplines, compared to their U.S. counterparts (Evaristo & Karahanna, 1997). In fact, Galliers and Meadows, in analyzing patterns of cross-citations between European and North American scholars refer to the IS community as 'a discipline divided .... marked by a distinct parochialism along national, or at least, regional lines' (Galliers & Meadows, 2003, p. 108). Their conclusion echoes a comment 10 years earlier by Suomi (1993), who – even then – noted a pattern where European IS scholars often cite their North American counterparts who, in turn, almost completely ignore them.

This leads to the key objective of our study: to identify the most productive IS researchers from a recent five-year interval, but based on a geographically balanced set of quality journals serving the global IS academic community. We examine whether the set of IS researchers yielded by our analysis differs from the lists generated solely from the previous studies that were limited to North American journals (Im et al., 1998; Athey & Plotnicki, 2000; Huang & Hsu, 2005). We pose the question that, if scholars who conduct these types of studies acknowledge that 'the importance of the decisions as to which journals to include ... cannot be overemphasized' (Athey & Plotnicki, 2000, p. 11), then shouldn't a more balanced set of journals yield different results, in terms of the geographic distribution and possibly the gender distribution of scholars who appear in the results?

Research methods

To answer these questions, we employed the bibliographic repository that was recently created and made available to the IS community by a team of faculty and Ph.D. students at Georgia State University in the U.S.A. (Chua et al., 2002). The GSU bibliographic repository (GSU/BR) consists of a comprehensive database of all publications in IS, information science, computer science, and related journals. With regular updates every six months since its original release in 2003, we consider the GSU/BR to be a valuable resource for research. In order to essentially replicate Huang and Hsu's study, in terms of the time period covered, but with a more balanced set of international, scholarly IS journals, we limit ourselves to the same number of journals employed by Huang and Hsu (12 journals), yet we substituted scholarly IS journals (primarily European journals) for the six journals that we omitted (Table 2).

Table 2 - Journals used in our study.

Full table

In modifying the list of journals employed by Huang & Hsu (2005), we chose to exclude the high-impact practitioner publications that Huang and Hsu included in their study: HBR, Sloan Management Review, and CACM. Moreover, we agree with Claver et al. (2000, p. 182) who characterize publications such as Management Science, Academy of Management Journal, and IEEE Transactions on Software Engineering as being outlets 'which, though not specialising in IS, regularly publish articles regarding this area.' We also agree with their logic that it makes sense to exclude these non-IS journals when evaluating scholarly IS productivity: ... we decided not to study the second group of journals (non-IS), although some of the landmark papers were published there ... there may be complete issues (or even volumes) of these journals containing no IS articles, which would distort a chronologically exhaustive study of research on IS (Claver et al., 2000, p. 182).

In addition to excluding these practitioner journals, we also excluded two scholarly journals that mainly publish research in other disciplines, such as operations research/operations management (Management Science) and computer science (IEEE Transactions on Software Engineering). We also omitted Journal of the AIS, a relatively new electronic journal that did not publish many articles before 2001. After excluding these six journals, we added six other, internationally recognized scholarly journals that focus on IS research. This includes EJIS, ISJ, ITP and JSIS. In order to have the same total number of journals as Huang and Hsu, we added two additional journals that publish high-quality IS research of a general nature – despite having misleading titles which suggest that they are niche journals (Database for Advances in Information Systems and IEEE Transactions on Engineering Management). The latter two journals are both scholarly IS journals focusing on general topics of IT deployment, use, and management (although IEEE TEM also contains articles about R&D management and technology commercialization, in addition to IS research).

Perhaps even more critical than the question of what journals to count is the question what types of publications to count. Recent studies of 'most prolific researchers' rely on online databases that typically count every possible article (whether it is a research paper or not) as a publication. While some authors have explicitly stated that they excluded non-research articles, such as book reviews (Athey & Plotnicki, 2000), other researchers do not always specify whether they excluded non-research articles. While it is fairly easy to perform online searches by author names in databases such as ABI/Inform or Science Direct, the results obtained from these searches often included other types of publications, in addition to research articles. For example, the GSU/BR (which we use for our study) does not distinguish between research articles and other genres of articles, such as book reviews, letters to the editor (often called 'issues & opinions' pieces), editors' introductory comments, or guest editors' introductions to special issues.

One by-product of this failure to distinguish among different genres of research papers vs non-research papers, is that journals that feature many special issues (e.g., Journal of MIS (JMIS)), special sections (MIS Quarterly), or those featuring a regular 'introduction from the editor' column (e.g., Database for Advances in IS, EJIS, and MIS Quarterly), or book reviews (e.g., EJIS, ITP) will tend to overstate the publication counts by their editors, guest editors, and book reviewers. While all scholarly journals have occasional special issues – which often begin with the guest editors' introductory comments – some journals begin every issue with brief introductory comments from the Editor, and sometimes feature as many as three special issues or special sections per year. Although these editorial introductions are valuable for journal readers, counts of authors' publications that fail to distinguish between editorial introductions and peer-reviewed research articles do a disservice to the practice of counting scholars' research contributions. For example, one highly regarded journal routinely uses the same researchers as guest editors for periodic special issues, while another seeks to incite controversy by asking noted scholars to write short research commentaries. In both cases, we consider that such practices unnecessarily inflate the publication counts of specific guest editors and/or senior scholars who have the honor of serving in these roles.

Any analysis of researchers' productivity should seek to 'filter' the lists of publications carefully, so as not to conflate scholarly research articles with these other types of articles. Although this pattern is certainly not typical of all journals, some journals featured as many as eight 'special issues' or 'special sections' during the five-year period covered in our study. At a rate of two or three guest editors per special issue, on average, we estimate that over 20 article 'counts' result from these very short introductory comments in one journal alone (JMIS) – which may have a noticeable effect on our results. We consider it important to treat such non-research articles differently than peer-reviewed research articles. Unfortunately, the GSU/BR treats all types of articles alike and hence, we had to add a special 'article type' field to the database containing the GSU/BR to identify such short, non-peer-reviewed articles. After we conducted queries using terms such as 'Editor's Comments,' 'Introduction to Special Issue,' or 'Book Review' in the paper titles – or searched on page length to identify articles below five pages – we found that some authors appearing in our result list indeed had their publication counts substantially inflated due to such non-research articles. Many of these authors were highly respected senior scholars – those who had risen to the role of Editor-in-Chief or regular guest editor for various journals. Even after adjusting their totals counts to omit these non-research articles, some of these authors still appeared at the top of our result set (e.g., Mark Keil, Dorothy Leidner, and Jay Nunamaker). In some other cases, however, most of the author's publications were deleted after adjusting for these non-research articles, with the result that some scholars no longer appeared in our result set of authors with three or more publications.

After eliminating non-research articles, we ran several queries on the GSU/BR to identify researchers with at least two articles in the set of 12 journals during 1999–2003. There were 560 authors with two or more publications. We then manually coded data about each author's gender (based on personal knowledge, and by viewing website photographs) and the continent/country in which they were employed during most of the 1999–2003 time period. We chose country/continent of employment (rather than the author's nationality or country of origin) for several reasons: while the vast majority of IS faculty are employed in the same continent in which they were born, in those cases where nationality and country of employment differ, the latter can easily be verified by examining biographical details within the publications themselves. In contrast, nationality cannot be verified in any source. We know of a few cases where authors moved between continents during the period under examination; we dealt with these exceptions by coding the authors as belonging to the continent/country where they spent the majority of time during the interval from 1999 to 2003.

For authors for whom we were uncertain about their gender, we were able to resolve this uncertainty by viewing photographs on their websites or, for some journals, articles containing photographs. Since this process of gathering and coding data for each author's gender and continent of employment was labor intensive, we were unable to do so for every researcher who had published one or more papers in this set of journals (a list that amounts to more than 3000 researchers). Since this coding process was so time-consuming (requiring that we consult several additional databases containing IS researcher information such as the ISWorld Faculty Directory and the AIS membership database), we restrict our analysis below to authors who published three or more papers in the set of journals during 1999–2003. This limited our results to 240 IS researchers. In a related study, where we analyze whether gender differences exist in terms of IS researchers' productivity (Gallivan & Benbunan-Fich, 2006), we focus on a larger result set of 560 authors with two or more publications in the same journals during this interval.

One additional methodological consideration is how to measure the productivity of the researchers (Chua et al., 2003). Previous studies in the field of scientometrics typically feature at least two different methods for counting numbers of publications – either normal counts (with each publication counted as 1 for all authors, regardless of number of co-authors) or adjusted counts (with each publication counted as a fraction of a paper, depending on the number of authors – for example, 1/3 of a publication, for a paper with three authors). Prior studies found that their results were rather sensitive to different counting methods – especially for authors with many co-authored publications and few solo-authored publications, or vice-versa.

Based on a set of queries that we ran with the GSU/BR, we generated a list of researchers with three or more research publications in these 12 journals during 1999–2003, using normal counts and then we computed the adjusted counts by crediting each author with a fractional count for any jointly authored papers. As mentioned above, we coded the demographic characteristics of the resulting authors in terms of gender and the continent/country where they were employed. Finally, we analyzed the geographic and gender distribution of authors in our results – both for the overall set of 240 authors with three or more publications, and also for the 'Top 30' authors in our result set. We compared our list of 'Top 30' researchers to the two prior studies (Athey & Plotnicki, 2000; Huang & Hsu, 2005), allowing us to analyze the effects of substituting the four European journals and two other scholarly journals that we added for the practitioner journals and other, non-IS journals that we deleted.

Results

We examined the demographic attributes of all 240 authors who published three or more articles in any of the 12 journals. Unfortunately, due to space limitations, we are unable to include the full list of author names (however, the list is available upon request from the authors). Our full set of 240 researchers includes 42 women (17.5% of all authors), 67 authors employed outside of North America (27.6%), and specifically 32 authors from Europe (13.3%). We note that the ratio of authors from Asia and Australia-New Zealand (combined) was 14.6%, just slightly larger than the ratio of authors from Europe (13.3%). Table 3 provides a summary of demographic data for the set of 240 authors with three or more journal publications from 1999 to 2003.

Table 3 - Results for authors with three or more publications.

Full table

We also analyzed the average productivity of authors from each continent, and also the average productivity of the men and women appearing in the full set of 240 authors. When analyzed by continent of employment, North American authors dominate the list with 72% of total authors on the list, and 72.6% of total papers published. The figures for the ratio of authors from other continents appear in Table 3. Note that the fraction of total papers published by authors from various geographic regions is very close to the proportion of authors from these regions – which means that, for authors who appear on this list of 240 researchers, there were no geographic differences in average researcher productivity. For example, if we had found that researchers from Asian countries represented 12% of authors, but had published 15% of total papers, this would indicate that authors from Asian countries were more prolific, on average, than other authors. (In fact, the Asian authors represented 12.1% of authors on the list and they were responsible for 12.4% of the total papers in the sample.) The fact that we did not observe any differences between the ratio of authors from each continent vs their proportion of papers published is due to the fact that, on a continent-by-continent basis, the authors from the four different continents represented in our results were equally productive.

Women represented 17.5% of the researchers on the list and accounted for 16.3% of the total articles published (based on normal counts) and 15.7% of total articles published (based on adjusted counts). The mean number of articles per author is slightly lower for the 42 women (4.17 articles per woman) than for the 198 men (4.55 articles per man), and the standard deviation is much smaller for the women (SD=1.51) than for the men (SD=2.75). This lower standard deviation for women reflects the fact that women tend to be clustered more densely in the middle of the set of 240 authors (rather than in the very top or very bottom of the list). We obtained similar results for our analysis of adjusted counts, where the average publication statistics are again somewhat lower for the 42 women (1.81 papers) appearing in the result set than for the 198 men (2.07 papers), a difference of 13%. So, the women who appear on our list of 240 most productive IS researchers have, on average, 8% (based on normal counts) or 13% (based on adjusted counts) fewer publications than the men on our list. We consider the issue of research productivity by gender in more depth in a related study (Gallivan & Benbunan-Fich, 2006).

In terms of examining both continent of employment and gender simultaneously, we found relatively few women from outside North America among the 240 most productive scholars. One notable exception is Dorothy Leidner, who was the most prolific female author (with eight publications using normal counts and 4.75 using adjusted counts). Nearly all the remaining 41 women were either employed in the U.S.A. or Canada (the latter including Yolande Chan and Blaize H. Reich), with eight women from outside of North America, including five from the U.K. (Chrisanthi Avgerou, Margi Levi, Shirin Madon, Sue Newell, Jacky Swan), two from Asia (Lai Lai Tung and Christina Soh), and one from Australia/New Zealand (Shirley Gregor).

Table 4 shows the 'Top 32' most prolific IS researchers (those with seven or more publications based on normal counts). The first column lists the total number of normal count articles appearing in the 12 IS journals for the same time period as Huang &Hsu's study, while the second column shows the adjusted counts. The list is sorted by normal counts (i.e., total number of papers), followed by adjusted counts, then by the authors' surnames. In all cases where two or more authors have identical numbers of total counts and adjusted count papers, these are shown as ties. (Note that the rank order of authors would be different if we sorted first by adjusted counts, and then by normal counts.) This set of the 32 most productive researchers features four authors from Europe (Phillip Powell, Dorothy Leidner, Zahir Irani, and Kalle Lyytinen) and five authors from Asia (Patrick Chau, Bernard Tan, Kar-Yan Tam, Thompson Teo, and Robert Davison). This 'Top 32' list includes four women (Dorothy Leidner, Ritu Agarwal, Anne Massey, and Mitzi Montoya-Weiss) and 28 men. In demographic terms, our results differ from those published by Huang & Hsu (2005), which had no Europeans scholars, three men from Asia, and just two women (both from North America).

Table 4 - List of 'Top 32' most productive authors (with seven or more publications) (sorted by normal count, then adjusted count, then authors' surnames).

Full table

We conducted additional analyses to see whether continent of employment or gender was statistically significant in explaining the total number of publications for the authors in our list of 'Top 240' authors with three or more publications. The results for our loglinear regression analysis (based on Poisson distributional assumptions) found no significant results. In our analyses, we used dummy variables to represent the authors' location (with separate variables for North America, Europe, Asia, and Australia/ New Zealand) as well as for gender. We found no statistically significant effect for continent. We interpreted this to mean that, although non-North American authors represent just 28% of the authors in this dataset, for those authors that do appear in the dataset, they are just as likely to have published a large number of articles as the North Americans – as shown, for instance, by Philip Powell (nine papers), Zahir Irani (eight papers), and several other authors with seven or eight papers. While non-North American authors represent 28% of the authors who published three or more papers in our overall result set (as shown in Table 4), and nine of the 32 authors (28.2%) in our 'Top 32' authors, the non-North Amercians are a much larger proportion of the authors in our result set, compared to the 13% reported by Huang & Hsu (2005).

Similarly, our analysis of gender revealed no statistical difference between the number of publications by men and women. Although women represent a minority of the authors in the overall result set of 240 researchers (17.5%), for those women who do appear in the result set, they are just as likely as the men to have published many articles. In order to truly know whether continent of employment or gender have an effect on the likelihood of publishing in the set of journals we counted, we would need to have data about all IS researchers worldwide who have published 0, 1 or 2 papers in these journals during the period in question. Unfortunately, we are unable to perform this type of analysis because we lack a comprehensive database that lists all IS researchers, along with gender and the continent where they are employed. We estimate that coding this information for authors who have published just one or more papers in these journals would require hundreds of hours. Moreover, coding this information for IS researchers who have not published any papers in these journals would be nearly impossible, as the IS discipline lacks a comprehensive listing of all Ph.D. graduates and/or university faculty. Although the GSU/BR contains information about publications, the demographic data regarding authors' gender and continent/country of employment must be manually gathered and coded from other sources, such as the ISWorld Faculty Directory, the AIS membership database, and other sources.

Table 5 compares our results with the results of earlier studies of most productive IS researchers. The prior study by Huang & Hsu (2005) identified two women, no European authors and three male Asian authors in their 'Top 30' researchers with at least eight normal count publications. While Huang and Hsu's study was conducted for the identical time period as our study, we note that two prior studies (Im et al., 1998; Athey & Plotnicki, 2000) covered earlier time periods; therefore, we would not expect their results to exactly match our own. Among their 'Top 24' most productive IS researchers, Athey & Plotnicki (2000) identified just one researcher from outside of North America (Patrick Chau from Hong Kong) and five women (Sirkka Jarvenpaa, Iris Vesey, Bernadette Szajna, Maryam Alavi, and Wanda Orlikowski), of whom only Sirkka Jarvenpaa was ranked in their 'Top 10.' Likewise, Im et al. (1998) identified only two researchers from outside North America, and two women (Sirkka Jarvenpaa and Iris Vessey).

Table 5 - Comparison of our results to prior studies of most productive IS scholars.

Full table

We first compare our list of the 'Top 32' researchers (those with at least seven normal count publications) to these earlier studies; then we compare the demographic attributes for our overall set of 240 authors with three or more publications to these prior studies. Our list of the 'Top 32' researchers features nine authors from outside North America (28%), and a total of four women (12.5%). The women include Dorothy Leidner (who we have coded as European, since she was on the faculty at INSEAD, in Paris, for most of the time period of our study) and three North American women – Ritu Agarwal, Anne Massey, and Mitzi Montoya-Weiss. Of the 28% of the sample that were employed outside of North America, there were two authors from the U.K. (Philip Powell and Zahir Irani), one from France (Dorothy Leidner), and one from Finland (Kalle Lyytinen). It also includes five authors from Asia, including Hong Kong (Patrick Chau, Kar Yan Tam, and Robert Davison), and Singapore (Thompson Teo and Bernard Tan). In comparing our 'Top 32' results to those of the prior studies, we see that 28% of our highest-ranked authors are from outside North America, compared to just 4–13% in the prior studies. A total of 12.5% of the authors on our 'Top 32' list were employed in European universities, compared to 0% in the prior studies. Finally, our study featured 12.5% women, compared to values ranging from 6.7% (Huang & Hsu, 2005) to 20.8% (Athey & Plotnicki, 2000) in these prior studies.

To examine the influence of the four European journals separately in our list of researchers, we conducted a separate analysis based on counts of articles published in the four European journals alone (EJIS, ISJ, ITP, and JSIS). Table 6 shows the 34 scholars who published three or more papers in these European journals. Of these authors, 22 were from Europe, with just seven from North America, three from Asia (Thompson Teo, Robert Davison, and Shan Pan), and two from New Zealand (Paul Cragg and Michael Myers). Seven of the 34 authors who published three or more papers in the European journals are women (21%) – a somewhat higher ratio than we found for the set of 12 journals overall (where the ratio of women was 17.5%). The highest number of normal count publications in the European journals was six papers each by Thomas Kern and Philip Powell. In comparing this list of most productive authors in European journals alone (Table 6) to our list of authors with seven or more total publications in any of the 12 journals (in Table 4), there were eight authors who appeared in both lists: one woman (Dorothy Leidner) and seven men (Mark Keil, Zahir Irani, Philip Powell, Robert Davison, Al Lederer, Thompson Teo, and Richard Baskerville). In addition to their European publications, each of these authors also published several papers in the larger set of North American journals during the same period.

Table 6 - Authors with three or more publications in European journals (sorted by normal count, then adjusted count, then authors' surnames).

Full table

Discussion

We analyzed the most productive IS researchers for the same time period as a recent study published in Communications of the AIS (Huang & Hsu, 2005), but with a different set of journals – which included four European IS journals, but omitting practitioner-oriented publications, and scholarly publications which primarily serve disciplines other than IS (e.g., Management Science and IEEE Transactions on Software Engineering). Our analysis, based on a set of 12 geographically balanced, scholarly IS journals produced a substantially different – and more diverse – set of 'most productive researchers' than that reported in earlier studies. Our result set is more diverse, both in terms of the continent in which the researchers were employed (27.6% of researchers in our 'Top 240' list of researchers were from outside North America vs 14.3% in Huang and Hsu's (2005) study, and in terms of gender (18% women in our study, compared to 6.7% in Huang and Hsu's study). We believe that the changes we made in terms of what to count (e.g., substituting four European IS journals and two other international, scholarly journals for the practitioner outlets and non-IS journals counted in Huang and Hsu's study, as well as our exclusion of non-research articles) has influenced our results, making the resulting set of authors more diverse.

With respect to continent of employment, the addition of the four European journals seems to have enhanced the diversity of our sample by adding more researchers from Europe to our result set (which consists of 12.5% Europeans and a total of 28% non-North Americans in our 'Top 32' list, compared to no European authors in Huang and Hsu's 'Top 30'). However, despite the fact that four of the 12 journals we included are European scholarly journals, still less than 14% of the authors in our 'Top 32' and in our full result set of the leading 240 IS scholars are European. The results of the separate analysis of productivity in European journals alone suggest that the inclusion of these outlets in our basket helped to increase the list of European authors who appeared in our set of the 'Top 240' IS researchers worldwide. Below, we consider additional questions raised by our analysis.

Our results have important implications for any future productivity analysis of IS scholars. The first issue deals with how to determine productivity itself. The selection of journals and time period, the type of publications and the method of counting all play a role in the set of individuals who comprise the result set. By comparing our results with those of the prior 'most productive researcher' studies, we have shown that different methods for identifying scholarly IS journals – and for deciding what constitutes a research contribution – can generate a more-or-less diverse set of researchers, in terms of geographic location and gender. While most researcher productivity studies have paid a great deal of attention to which journals to include in their 'basket' of high-quality journals, most of these studies have ignored the issue of what types of publications to count. In our study, we have specifically focused on peer-reviewed research articles, discounting non-research articles, such as book reviews, editors' introductions, short articles that critique other published studies, etc.

Given the fact that ours is the first study of IS researcher productivity to include several high-quality European journals – leading to the result of a more diverse set of authors – it follows that careful consideration must be given to judging researchers' productivity according to criteria that are geographically neutral. This impartial process is made more critical in light of recent evidence suggesting the existence of a methodological and epistemological divide between U.S. and European journals and scholars (Walsham, 1995; Avgerou et al., 1999; Galliers & Meadows, 2003; Katerattanakul & Han, 2003; Chen & Hirschheim, 2004). Evaristo & Karahanna (1997, p. 33) underscore this theme in their review of European vs U.S. dissertations when they conclude that: Given the predominantly positivist research tradition in North America and interpretive research tradition in Europe, it seems reasonable to presume two different sociological paradigms. If indeed, North American and European researchers belong to two different sociological paradigms, this could pose problems in terms of reviewing and publishing in MIS journals and conferences as well as in sharing knowledge across research traditions.

Despite the fact that only four European authors appeared on our list of the 'Top 32' researchers with seven or more publications (Table 5), European authors still constitute a minority of authors in our results. Two-thirds of the journals in our 'basket' are considered to be North American journals (eight out of 12), and apparently, European scholars publish much less frequently in North American journals than do North Americans – a phenomena that was explored at length in a recent essay by Galliers & Meadows (2003). Why might this be the case? One potential explanation suggested by Galliers and Meadows was that the IS community is a 'discipline divided' – that is, marked by considerable 'parochialism' at the country or regional level. By this, they meant that North American IS scholars tend to publish their research in North American journals and cite North American authors, while European authors tend to publish their work in European journals and cite works in those journals. While Galliers and Meadows were not the first to draw attention to differences between European and North American styles of research (Walsham, 1995; Evaristo & Karahanna, 1997), their arguments are consistent with other studies showing that North American scholars generally slight European IS publications by neglecting to cite them (Suomi, 1993; Katerattanakul & Han, 2003). Another observation is that, when European IS researchers publish scholarly articles, they tend to have a very different pattern of research citations than their North American peers. Along these lines, Galliers & Whitley (2002) found that Europeans cite other European authors much more than do North Americans (at least those who publish at ECIS conferences). In addition, Galliers & Whitley (2002) showed that the sources European IS scholars cite most often are books – for instance, seven of the 11 most-cited sources at ECIS conferences were books (Checkland, 1981; Earl, 1989; Giddens, 1990; Davenport, 1992; Hammer & Champy, 1993; Walsham, 1993; Yin, 1994), with nearly all the other most frequently cited sources being practitioner articles in Harvard Business Review (Cash & Konsynski, 1985; Porter & Millar, 1985; Hammer, 1990). In contrast, Galliers and Whitley found that few scholarly articles were frequently cited in ECIS conference papers – just one scholarly article appeared among the top 11 most frequently cited sources (Malone et al., 1987 in CACM). The most popular academic articles that were cited in ECIS conferences tended to be papers about how to conduct case studies or interpretive research (Benbasat et al., 1987; Lee, 1989; Walsham, 1995), although none of the latter, scholarly articles were among the 10 most-frequently cited sources. These results demonstrate that European IS scholars do indeed employ different research methods and reference disciplines, compared to North American authors – at least, based on doctoral dissertations (Evaristo & Karahanna, 1997) and the sources cited most frequently in ECIS conference papers (Galliers & Whitley, 2002). Similar arguments have been made by other authors (e.g., Walsham, 1995; Chen & Hirschheim, 2004; Introna & Whittaker, 2004; Larsen & Levine, 2005), showing that European journals have traditionally been more open to interpretive, case study, and critical theory research, although some North American journals have, in recent years encouraged more submissions along these lines (Markus & Lee, 1999).

The separate analysis that we presented of articles in European journals alone (Table 6) shows that European researchers publish more frequently than North Americans do in European outlets (74% of the 34 authors publishing three or more papers in the four European journals are European or were employed by European universities). This result is consistent with the claim that European researchers are aware of the philosophical differences between the North American research paradigm (generally, quantitative and positivist) and their own (generally qualitative and interpretive or critical theory), and hence they mostly submit to outlets that have a record of publishing the style of research that they conduct (Galliers & Meadows, 2003). Unfortunately, since we lack data on the number of journal submissions by authors from various countries, we cannot critically test this explanation. (Publication data can be analyzed by the authors' country of employment, while actual submission data by country are not available anywhere, to our knowledge.) Whether this outcome is predominantly the result of self-selection (European authors eschewing North American outlets, in favor of European journals) or due to a reviewing bias (European manuscripts being rejected at higher rates by North American journals), our results attest to the aforementioned 'parochialism' (Galliers & Meadows, 2003) when it comes to the gap in research methods and epistemological beliefs between Europe and North America: 74% of the most prolific authors publishing in European journals are European and 72% of the most prolific authors publishing in a set of 12 (primarily North American) journals are North American or employed in North America.

Limitations

These results must be interpreted in light of out study's limitations. As we acknowledged above, the selection of journals and the timeframe being examined determine to a great extent the list of most productive researchers. Thus, any change in the 'basket' of journals or in the time period under investigation will produce a different set of results. Given this sensitivity of results to the timeframe and basket of journals selected, it would be interesting to conduct a study to see whether the proportion of papers published by Europeans in North American journals has increased over the past decade, or conversely, whether the proportion of papers published by non-Europeans in European journals has increased in recent years. While recent studies (e.g., Chen & Hirschheim, 2004; Larsen & Levine, 2005) have shown changes in the types of studies published in several IS journals over the past decade (specifically focusing on methods and epistemology), it would be interesting to build upon their work by conducting a trend analysis of authors' nationality in each journal to see if there have been changes over time.

A second limitation is that our method of determining the most productive researchers is based on normal counts and adjusted counts of number of papers published instead of other metrics for counting research impact, such as citation counts – as employed by Katerattanakul & Han (2003), and advocated by other authors (Mylonopoulos & Theoharakis, 2001; Galliers & Meadows, 2003). Obviously, a different process of identifying research contributions, such as weighting papers based on how often they are cited would alter the results to produce a set of IS researchers with the greatest impact (Walstrom & Leonard, 2000; Whitman & Woszczynski, 2003).

A third limitation is that our analysis focused on researchers who published at least three papers in the set of 12 journals that we analyzed, rather than the much larger set of authors who published just one or more papers in these journals. We made this decision to restrict our analysis to authors with three or more papers, since the effort involved in coding gender and continent of employment for authors with fewer papers would be enormously time-consuming. It is possible, however, that a slightly different pattern of results would emerge if we had set a lower threshold for inclusion in our study – such as just one publication, or just two publications in this set of journals – or in other journals.

Fourth, we focused on journal publications as our measure of scholarly productivity, and clearly, the results would differ greatly if we had focused on other scholarly outputs, such as authorship of research monographs or conference proceedings. It is obvious that, within European universities, there is a much stronger tradition of researchers publishing books compared to North American universities – or, at least in North American business schools, specifically. A cursory analysis that we conducted regarding the number of books published by the European authors appearing in the list of 'Top 32' researchers, vs those from other continents, suggests that many of the European IS scholars publish books on a regular basis, in comparison to the North American authors, who publish few, if any books. Apparently, there is much greater recognition attached to academics publishing books in European business schools, compared to North America. While our study was limited to journal articles only, another interesting area for follow-up research would be to evaluate the numbers of books published by researchers from different geographical areas, and to assess their impact (based on citation counts). Galliers & Whitley, (2002) have conducted research along these lines, although it would be interesting to expand their analysis beyond ECIS conferences, to see how often scholarly books are cited in IS journals. Despite the limitations that we noted above, we believe that our study has shown that the details of what is counted in studies of IS researcher productivity has important implications for the set of authors who appear in the results. We hope that other researchers who conduct studies of scholarly productivity will carefully consider the issues that we raised above with regard to what journals (and what types of articles within them) should count as scholarly research contributions. As noted in Suchman's (1994) work on the politics of categorization, it is critical that we understand how the categories we use affect the outcomes that result from our efforts.

One final area of future research would be to investigate career development in academia. It may be the case that different reward systems and salary incentives per published refereed article (i.e., merit increases) exists between universities in North America, Europe, and other parts of the world. Likewise, future research should investigate whether these rewards for research – as opposed to teaching and service – are similar for men and women IS academics (Whitman et al., 1999). Although studies have been conducted to examine whether gender is a factor in the productivity of IS researchers (Hu & Gill, 2000; Gallivan & Benbunan-Fich, 2006), and to estimate the increase in compensation associated with journal publications (Gill, 2001), analyses of compensation, based on gender and geographic location have not been published, to our knowledge. This area of future inquiry may hold the potential to explain the observed patterns of productivity among IS researchers.

Conclusions

Our analysis of the most productive IS researchers between 1999 and 2003 in a set of 12 geographically balanced, scholarly IS journals produced a substantially different – and more diverse – set of 'most productive researchers' than that reported in earlier studies. Our result set is more diverse in terms of the continent in which the researchers were employed with 28% of researchers from outside North America, and in terms of gender (17.5% women). In particular, 13.3% of the researchers in our full set of 240 researchers and four of the 'Top 32' (12.5%) are European authors with seven or more publications each (Philip Powell, Dorothy Leidner, Zahir Irani, and Kalle Lyytinen).

References

1. Athey S and Plotnicki J (2000) An evaluation of research productivity in academic IT. Communications of the AIS 3(7), 1–20 (available online at http://cais.isworld.org ).
2. Avgerou C, Siemer J and Bjorn-Andersen N (1999) The academic field of information systems in Europe. European Journal of Information Systems 8(2), 136–153. | Article |
3. Barman S, Hanna MD and Laforge RL (2001) Perceived relevance and quality of POM journals: a decade later. Journal of Operations Management 19(3), 367–385. | Article |
4. Barnes SJ (2005) Technical opinion: assessing the value of IS journals. Communications of the ACM 48(1), 110–112. | Article |
5. Benbasat I, Goldstein DK and Mead M (1987) The case research strategy in studies of information systems. MIS Quarterly 11(3), 369–386. | Article |
6. Bharati P and Tarasewich P (2002) Global perceptions of journals publishing ecommerce research. Communications of the ACM 48(5), 21–26.
7. Cash J and Konsynski BR (1985) IS redraws competitive boundaries. Harvard Business Review 63, 134–142.
8. Checkland PB (1981) Systems Thinking, Systems Practice. John Wiley & Sons, New York.
9. Chen WS and Hirschheim R (2004) A paradigmatic and methodological examination of IS research from 1991 to 2001. Information Systems Journal 14, 197–235. | Article |
10. Cheng CH, Holsapple CW and Lee A (1994) The impacts of periodicals on expert system research. IEEE Expert 9(6), 7–14. | Article |
11. Chua C, Cao L, Cousins K, Mohan K, Straub DW and Vaishnavi V (2002) IS bibliographic repository: a central repository of research information for the IS community. Communications of the AIS 8(27), (available at cais.isworld.org/articles ).
12. Chua C, Cao L, Cousins K and Straub DW (2003) Measuring researcher production in information systems. Journal of the AIS 3(6), (available at jais.isworld.org/articles ).
13. Claver E, Gonzaalez R and Llopis J (2000) An Analysis of research in information systems (1981–1997). Information & Management 37(4), 181–195. | Article |
14. Cooper R, Blair D and Pao M (1993) Communicating MIS research: a citation study of journal influence. Information Processing and Management 29(1), 113–127. | Article |
15. Culnan MJ (1986) The intellectual development of management information systems, 1972–1982: a co-citation analysis. Management Science 32(2), 156–172.
16. Culnan MJ (1987) Mapping the intellectual structure of MIS, 1980–1985: a co-citation analysis. MIS Quarterly 11(3), 341–343. | Article |
17. Daigle RJ and Arnold V (2000) An analysis of the research productivity of AIS faculty. International Journal of Accounting Information Systems 1(2), 106–122. | Article |
18. Davenport T (1992) Process Innovation: Reengineering Work through Information Technology. Harvard Business School Press, Boston, MA.
19. Dennis AR, Valacich J, Fuller M and Schneider C (2006) Research standards for promotion and tenure in Information Systems. MIS Quarterly 30(1), 1–12.
20. Doke ER and Luke RH (1987) Perceived quality of CIS/MIS journals. Journal of Computer Information Systems 28(4), 30–33.
21. Earl MJ (1989) Management Strategies for Information Technologies. Prentice-Hall, New York.
22. Eom SB (1994) Ranking institutional contribution to decision support systems research: a citation analysis. Data Base 25(1), 35–42.
23. Evaristo JR and Karahanna E (1997) Is North American IS research different from European IS research. Data Base for Advances in Information Systems 28(3), 32–43.
24. Forgionne GA and Kohli RA (2001) Multiple criteria assessment of decision technology system journal quality. Information & Management 38(7), 421–438. | Article |
25. Foucault M (1977) Truth and Power: Power/Knowledge: Selected Interviews and Other Writings 1972–1977 (GORDON C, Ed), pp. 109–133, Pantheon Books, New York.
26. Galliers RD and Meadows M (2003) A discipline divided: globalization and parochialism in IS research. Communications of the AIS 11, 108–117 (available at cais.isworld.org/articles ).
27. Galliers RD and Whitley EA (2002) An anatomy of European IS research: ECIS 1993–2002: some initial findings. European Conference on Information Systems, Gdansk, Poland.
28. Gallivan MJ and Benbunan-Fich R (2006) Is there a gender gap in the research productivity of is faculty? Journal of the Association of Information Systems under review, 2006.
29. Giddens A (1990) The Consequences of Modernity. Stanford University Press, Stanford, CA.
30. Gill TG (2001) What's an MIS paper worth? An Exploratory Analysis. Data Base for Advances in Information Systems 32, 14–33.
31. Gillenson ML and Stutz JD (1991) Academic issues in MIS: Journals and books. MIS Quartery 15(4), 447–452.
32. Gioia D and Corley K (2002) Being good v. looking good: business school rankings and the Circean transformation from substance to image. Academy of Management Learning & Education 1, 107–120.
33. Goh C-H, Holsapple CW, Johnson LE and Tanner JR (1996) An empirical assessment of influences on POM research. Omega 24(3), 337–342. | Article |
34. Goh C-H, Holsapple CW, Johnson L and Tanner JR (1997) Evaluating and classifying POM journals. Journal of Operations Management 15(2), 123–138. | Article |
35. Hammer M (1990) Reengineering work – don't automate, obliterate. Harvard Business Review 68(4), 104–112.
36. Hammer M and Champy J (1993) Reengineering the Corporation: A Manifesto for Business Revolution. Harper Business, Boston, MA.
37. Hardgrave BC and Walstrom KA (1997) Forums for MIS scholars. Communications of the ACM 40(11), 119–124. | Article |
38. Holsapple CW, Johnson LE, Manakyan H and Tanner J (1993) A citation analysis of business computing research journals. Information & Management 25(5), 231–244. | Article |
39. Holsapple C, Johnson LE, Manakyan H and Tanner J (1994a) Business computing research journals: adjusted citation analysis. Journal of Management Information Systems 11, 131–140.
40. Holsapple CW, Johnson LE, Manakyan H and Tanner J (1994b) Business computing system research: structuring the field. Omega 22(1), 69–81. | Article |
41. Holsapple CW, Johnson LE, Manakyan H and Tanner J (1995) An empirical assessment and categorization of journals relevant to DSS research. Decision Support Systems 14, 359–367.
42. Holsapple CW and Luo W (2003) A citation analysis of influences on collaborative computing research. Computer Supported Cooperative Work 12(3), 351–366. | Article |
43. Hsieh C-T, Lu M-T and Lin B (2001) Assessing the quality of computer print magazines: practitioners' versus academics' perceptions. Industrial Management & Data Systems 101, 120–125.
44. Hu Q and Gill TG (2000) IS faculty research productivity: influential factors and implications. Information Resources Management Journal 13(2), 15–25.
45. Huang H-H and Hsu JSC (2005) An evaluation of publication productivity in IS: 1999 to 2003. Communications of the AIS 15(5), 555–564 (available at cais.isworld.org/articles ).
46. Im K-S, Kim K-Y and Kim JS (1998) An assessment of individual and institutional research in information systems. Decision Line. Available at www.bm.ust.hk/ismt/research/mis_rank_1996.pdf.
47. Introna L and Whittaker L (2004) Truth, journals and politics: the case of the MIS Quarterly. In Information Systems Research: Relevant Theory and Informed Practice (KAPLAN B, TRUEX D, WASTELL D and WOOD-HARPER T, Eds), pp 103–120, Kluwer Press, Dordrecht, Netherlands.
48. Jackson WM and Nath R (1989) Publication patterns of MIS researchers. Interfaces 11(2), 15–20.
49. Katerattanakul P and Han B (2003) Are European IS journals under-rated: an answer based on citation analysis. European Journal of Information Systems 12, 60–71. | Article |
50. Katerattanakul P, Han B and Hong S (2003) Objective quality ranking of computing journals. Communications of the ACM 46(10), 111–114. | Article |
51. Kleijnen JPC and Van Groenendaal M (2000) Measuring the quality of publications: new methodology and case study. Information Processing and Management 36(4), 551–570. | Article |
52. Koong KS and Westrofer HR (1989) Faculty usage of MIS journals: a survey. Journal of Computer Information Systems 30(1), 1–4.
53. Larsen TJ and Levine L (2005) Searching for management information systems: coherence and change in the discipline. Information Systems Journal 15, 357–381. | Article |
54. Lee AS (1989) A scientific methodology for MIS case studies. MIS Quarterly 13(1), 33–52. | Article |
55. Lending D and Wetherbe JC (1992) Update on MIS research: a profile of leading journals and U.S. universities. Database 24(1), 5–11.
56. Lowry PB, Romans D and Curtis A (2004) Global journal prestige and supporting disciplines: a scientometric study of information systems journals. Journal of the Association of Information Systems 5(2), 29–77.
57. Malone TW, Yates JA and Benjamin RI (1987) Electronic markets and electronic hierarchies. Communications of the ACM 30(6), 484–497. | Article |
58. Markus ML and Lee AS (1999) Special issue on intensive research in IS: using qualitative, interpretive, and case methods to study information technology. MIS Quarterly 23(1), 35–38.
59. Markus ML and Robey D (1988) Information technology and organizational change: casual structure in theory and research. Management Science 34(5), 583–598.
60. Meho LI and Spurgin K (2005) Ranking the research productivity of library & information science faculty and schools. Journal of the American Society for Information Science & Technology 56, 314–331.
61. Mylonopoulos NA and Theoharakis V (2001) Global perceptions of IS journals. Communications of the ACM 44(9), 29–33. | Article |
62. Nord JH and Nord GD (1995) MIS research: journal status assessment and analysis. Information & Management 29(1), 29–42. | Article |
63. Orlikowski WJ and Baroudi JJ (1991) Study Information Technology in organizations: Research approaches and assumptions. Information Systems Research 2(1), 1–28.
64. Orlikowski WJ and Iacono SC (2001) Research commentary: desperately seeking the IT artifact – a call to theorizing the IT artifact. Information Systems Research 12(20), 174–207.
65. Porter M and Millar V (1985) How information gives you competitive advantage. Harvard Business Review 12, 12.
66. Rainer RK and Miller MD (2005) Examining differences across journal rankings. Communications of the ACM 48(2), 91–94. | Article |
67. Remus W (1989) Articles published in the top four MIS journals: 1984–1988. MIS Interrupt 43, 5 (University of Calgary, Canada).
68. Remus W (1991) Articles published in the top four MIS journals: 1986–1990. MIS Interrupt 50, 2–3 (University of Calgary, Canada).
69. Schwartz RB and Russo MC (2004) How to quickly find articles in the top IS journals. Communications of the ACM 47(2), 98–101. | Article |
70. Stake JE (2004) Rankings are Dangerous! available at http://monoborg.law.indiana.edu/LawRank/.
71. Suchman L (1994) Do categories have politics? – the language/action perspective reconsidered. Computer Supported Cooperative Work 2(3), 177–190. | Article |
72. Suomi R (1993) On the nationality balance of authors and references in selected MIS journals. Information & Management 24(6), 339–347. | Article |
73. Trower JK (1995) Publications by researchers and institutions in two top IS journals: 1990–1994. Proceedings of the 1st Americas Conference on Information Systems Pittsburgh, pp. 23–25.
74. Vogel DR and Wetherbe JC (1984) MIS research: a profile of leading journals and universities. Data Base 16(1), 3–14.
75. Walczak S (1999) A reevaluation of Information Sytems publication forums. Journal of Computer Inforamtion Systems 40(1), 88–97.
76. Walsham G (1993) Interpreting Information Systems in Organisations. Wiley, Chichester.
77. Walsham G (1995) The emergence of interpretivism in IS research. Information Systems Research 6(4), 376–394.
78. Walstrom KA, Hardgrave BC and Wilson RL (1995) Three forums for management information systems scholars. Communications of the ACM 38(3), 93–106. | Article |
79. Walstrom KA and Leonard L (2000) Citation classics from the information systems literature. Information & Management 38, 59–72. | Article |
80. Whitley EA and Galliers RD (2005) An alternative perspective on citation classics: Evidence form the first ten years of the European Conference on Information Systems, Working paper, London School of Economics.
81. Whitman ME, Hendrickson AR and Townsend AM (1999) Academic rewards for teaching, research, and service: data and discourse. Information Systems Research 10(2), 99–109.
82. Whitman ME and Woszczynski A (2003) Exemplary work in IS research: past and present. In Handbook for IS Research (WHITMAN ME and WOSZCZYNSKI A, Eds), Idea Group Press, Hershey, PA.
83. Yin RK (1994) Case Study Research: Design and Methods, 2nd edn. Sage Press, Newbury Park, CA.

Appendices

Appendix A

Table A1

Table A1 - Most prolific researchers (from Huang and Hsu, 2005).

Full table

Appendix B

Table B1

Table B1 - Table of prior general studies of researcher productivity and journal quality.

Full table

Acknowledgements

The authors wish to thank Jing Guo, M.S. candidate in Actuarial Sciences at Georgia State University for her help with data collection, and also Sun Lim Lee, MBA student at Baruch College, City University of New York. We also wish to thank Cecil Chua, Nanyang Technological University in Singapore, for making the GSU Bibliographic Repository available to us, Dave Naumann from the University of Minnesota for providing an archival version of the ISWorld Faculty Directory, and Pete Tinsley, Executive Director of AIS, for access to AIS membership records. An earlier version of this paper was submitted to the IFIP 8.2 Working Group conference on Social Inclusion: Societal and Organizational Implications for Information Systems held in July 2006. This compares to Huang & Hsu's (2005) study for the same period of time, but with a different basket of journals, which featured no European authors among their "Top 30" list of most productive IS researchers.

About the authors

Michael J. Gallivan is an associate professor in the CIS Department at Georgia State University in the Robinson College of Business in Atlanta. He received his Ph.D. degree from the MIT Sloan School of Management. His research interests are in the management of IT personnel, including issues of gender and racial diversity, IT outsourcing, IT adoption, implementation, and training. His research has been published in MIS Quarterly Journal of Management Information Systems, Information & Management, IEEE Transactions on Professional Communication, Information Technology & People, and Information and Organization.

Raquel Benbunan-Fich is an associate professor of Information Systems at the Zicklin School of Business, Baruch College, City University of New York. She received her Ph.D. in Management Information Systems from Rutgers University, Graduate School of Management. Her research interests include computer-mediated group collaboration, technology-mediated learning, evaluation of Web-based systems, and research productivity of IS faculty. Her research has been published in Communications of the ACM, Decision Support Systems, IEEE Transactions on Professional Communication, Information & Management, International Journal of Electronic Commerce and other journals.