Skip to main content
SpringerLink
Log in

What Do the Worldwide Governance Indicators Measure?

  • Original Article
  • Published:
The European Journal of Development Research Aims and scope Submit manuscript

Abstract

As policymakers and researchers focus more on the question of the impact of governance in economic development, they have required measures of the quality of governance to set policy or to conduct analyses. A number of measures of the quality of governance have been created. Among these are the Worldwide Governance Indicators, which rank countries on six aspects of ‘good governance’. Critics have focused on problems of bias or lack of comparability that raise questions about the utility of these indicators. However, a more fundamental question is that of whether the indicators have ‘construct validity’ – whether they measure what they purport to measure. This paper considers the construct validity of the indicators and raises the question of whether researchers and policymakers are relying on wrong data, rather than poor data.

Les responsables politiques et les chercheurs ont besoin de mesures concrètes de la qualité de la gouvernance afin de pouvoir déterminer l’impact de celle-ci, en particulier par rapport au développement économique. Un certain nombre d’indicateurs ont récemment été créés, parmi lesquels les Indicateurs de gouvernance dans le monde de la Banque Mondiale, qui classent les pays à partir de six critères de « bonne gouvernance ». L’utilité de ces indicateurs a été mise en question pour des raisons de distorsion ainsi que des problèmes de manque de comparabilité. Cependant, une question plus fondamentale est celle de la validité théorique de ces indicateurs, c’est-à-dire, s’ils mesurent ce qu’ils prétendent mesurer. Cet article considère la validité conceptuelle de ces indicateurs et cherche à déterminer dans quelle mesure les chercheurs et les responsables politiques ne sont pas en train de se baser sur des données fausses, plutôt que des données insuffisantes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2

Similar content being viewed by others

Notes

  1. The indicators have had various names over the years, but this paper uses the name used on the World Bank's website and most recently by the authors.

  2. For a description of available data sources, see Gray (2007) Governance for economic growth and poverty reduction: Empirical evidence and new directions reviewed. http://www.gsdrc.org/docs/open/RET422.pdf.

  3. See the disclaimer at www.govindicators.org that reads ‘Disclaimer: The aggregate indicators do not reflect the official views of the World Bank, its Executive Directors, or the countries they represent. The WGI are not used by the World Bank Group to allocate resources or for any other official purpose.’

  4. Another necessary property of a measure of a construct is reliability. Validity and reliability are usually discussed together. A measure is ‘reliable’ ‘to the extent to which an experiment, test, or any measuring procedure yields the same results on repeated trials’ (Carmines and Zeller, 1979). The WGI will be reliable to the extent that the underlying individual indicators are reliable. Accordingly, the reliability of the WGI is beyond the control of the authors and will not be discussed here, except to note that the authors recognize the importance of measurement error and have paid considerable attention to minimizing the impact of such error.

  5. These predicted relationships were dubbed the ‘nomological network’ by Cronbach and Meehl (1955).

  6. A consistent estimator is one that has a greater probability of converging on the population parameter the greater the sample size. (Wooldridge, 2003). An inconsistent estimator does not; increasing the sample size is no guarantee of improved accuracy.

  7. A list of the underlying variables used to construct each governance indicator appears in the appendices of the study by Kaufmann et al (2008). Table 1 in this paper reproduces for convenience the list of variables that are used to construct the Voice and Accountability indicator.

  8. Confidential data are not available to researchers. While in theory researchers could purchase the proprietary data, many are likely to find it to be cost-prohibitive. For example, a one-year subscription to the Gray Area Dynamics data set produced by Merchant International Group for an individual academic researcher would cost $10 000 (private communication with Paddy Breiner, 20 February 2009); this is just one of the proprietary data sets. Further, as Schrank points out, the definitions and methods of collection of the underlying variables should also be available; where they themselves are aggregates, the raw data and the method of aggregation should be made available (private conversation with Andrew Schrank (8 March 2007)).

References

  • Adcock, R. and Collier, D. (2001) Measurement validity: A shared standard for qualitative and quantitative research. The American Political Science Review 95 (3): 529–546.

    Article  Google Scholar 

  • Andres, A.R. (2006) Software piracy and income inequality. Applied Economics Letters 13 (2): 101–105.

    Article  Google Scholar 

  • Apodaca, C. (2004) The rule of law and human rights. Judicature 87 (6): 292–299.

    Google Scholar 

  • Arndt, C. and Oman, C. (2006) Uses and abuses of governance indicators. Paris: OECD Development Centre.

    Google Scholar 

  • Bartholomew, D.J. (1995) Spearman and the origin and development of test theory. British Journal of Mathematical and Statistical Psychology 48: 211–220.

    Article  Google Scholar 

  • Bollen, K.A. (1989) Structural Equations with Latent Variables. New York: Wiley.

    Book  Google Scholar 

  • Bollen, K.A. (1993) Liberal democracy: Validity and method factors in cross-national measures. American Journal of Political Science 37 (4): 1207–1230.

    Article  Google Scholar 

  • Bollen, K.A., Glanville, J.L. and Stecklov, G. (2002) Economic status proxies in studies of fertility in developing countries: Does the measure matter? Population Studies 56: 81–96.

    Article  Google Scholar 

  • Burnside, C. and Dollar, D. (2000) Aid, policies, and growth. American Economic Review 90 (4): 847–868.

    Article  Google Scholar 

  • Burnside, C. and Dollar, D. (2004) Aid, policies, and growth. American Economic Review 94 (3): 781–784.

    Article  Google Scholar 

  • Campbell, D.T. and Fiske, D.W. (1959) Convergent and discriminant validation by the multitrait–multimethod matrix. Psychological Bulletin 56: 81–105.

    Article  Google Scholar 

  • Carmines, E.G. and Zeller, R.A. (1979) Reliability and Validity Assessment. Beverly Hills, CA: Sage Publications.

    Book  Google Scholar 

  • Carson, R.T. et al (1998) Referendum design and contingent valuation: The NOAA panel's no-vote recommendation. The Review of Economics and Statistics 80 (2): 335–338.

    Article  Google Scholar 

  • Cronbach, L. and Meehl, P. (1955) Construct validity in psychological tests. Psychological Bulletin 52 (4): 281–302.

    Article  Google Scholar 

  • Dakolias, M. (2006) Are we there yet? Measuring success of constitutional reforms. Vanderbilt Journal of Transnational Economics 39 (4): 1117–1231.

    Google Scholar 

  • Das, G.G. and Andriamananjara, S. (2006) Hub-and-spokes free trade agreements in the presence of technology spillovers: An application to the western hemisphere. Review of World Economics 142 (1): 33–66.

    Article  Google Scholar 

  • Dollar, D. and Kraay, A. (2003) Institutions, trade and growth. Journal of Monetary Economics 50 (1): 133–162.

    Article  Google Scholar 

  • Easterly, W., Levine, R. and Roodman, D. (2004) Aid, policies and growth: Comment. American Economic Review 94 (3): 774–780.

    Article  Google Scholar 

  • Eid, M., Lischetzke, T., Nussbeck, F.W. and Trierweiler, L.I. (2003) Separating trait effects from trait-specific method effects in multitrait–multimethod models: A multiple-indicator ct-c(m-1) model. Psychological Methods 8: 36–60.

    Article  Google Scholar 

  • Elkins, Z. (2000) Gradations of democracy? Empirical tests of alternative conceptualizations. American Journal of Political Science 44 (2): 293–300.

    Article  Google Scholar 

  • Faber, J. (1987) Measuring cooperation, conflict and social network of nations. The Journal of Conflict Resolution 31 (3): 438–464.

    Article  Google Scholar 

  • Fallon, J. and Richard, H. (1997) “The rule of law” as a concept in constitutional discourse. Columbia Law Review 97 (1): 1–56.

    Article  Google Scholar 

  • Filmer, D. and Pritchett, L. (2001) Estimating wealth effects without income or expenditure data – or tears: Educational enrollment in india. Demography 38 (1): 115–132.

    Google Scholar 

  • Forgas, J.P. (1980) Images of crime: A multidimensional analysis of individual differences in crime perception. International Journal of Psychology 15 (1): 287–299.

    Article  Google Scholar 

  • Forsythe, G.B., McGaghie, W.C. and Friedman, C.P.P. (1986) Construct validity of medical clinical competence measures: A multitrait–multimethod matrix study using confirmatory factor analysis. American Educational Research Journal 23 (2): 315–336.

    Article  Google Scholar 

  • Girjalva, T.C., Berrens, R.P., Bohara, A.K. and Shaw, W.D. (2002) Testing the validity of contingent behavior trip responses. American Journal of Agricultural Economics 84 (2): 401–414.

    Article  Google Scholar 

  • Gray, H. (2007) Governance for economic growth and poverty reduction: Empirical evidence and new directions reviewed, http://www.gsdrc.org/docs/open/RET422.pdf.

  • Hammond, K.R., Hamm, R.M. and Grassia, J. (1986) Generalizing over conditions by combining the multitrait–multimethod matrix and the representative design of experiments. Psychological Bulletin 100: 257–269.

    Article  Google Scholar 

  • Hart, D., Atkins, R. and Youniss, J. (2005) Knowledge, youth bulges, and rebellion. Psychological Science 16 (8): 661–662.

    Article  Google Scholar 

  • Hirschman, A.O. (1970) Exit, voice, and loyalty. Cambridge, MA: Harvard University Press.

    Google Scholar 

  • International Development Association. (2004) IDA's performance-based allocation system: IDA rating disclosure and fine-tuning the governance factor. Washington DC: International Development Association.

  • Iqbal, K. and Shah, A. (2008) How do the Worldwide Governance Indicators measure up? http://siteresources.worldbank.org/PSGLP/Resources/Howdoworldwidegovernanceindicatorsmeasureup.pdf.

  • Johnston, M. (2000) The new corruption rankings: Implications for analysis and reform. Paper prepared for Research Committee 24, International Political Science Association World Congress; 2 August, Quebec City, Canada.

  • Jung, M. (2006) Host country attractiveness for cdm non-sink projects. Energy Policy 34 (15): 2173–2184.

    Article  Google Scholar 

  • Kaufmann, D. and Kraay, A. (2002) Growth without governance. Economia 3 (1): 169–229.

    Google Scholar 

  • Kaufmann, D. and Kraay, A. (2003) Governance and growth: Causality which way? – evidence for the world, in brief. Washington DC: World Bank.

    Google Scholar 

  • Kaufmann, D., Kraay, A. and Mastruzzi, M. (2004) Governance matters III: Governance indicators for 1996, 1998, 2000, and 2002. The World Bank Economic Review 18 (2): 253–288.

    Article  Google Scholar 

  • Kaufmann, D., Kraay, A. and Mastruzzi, M. (2005) Governance matters IV: Governance indicators for 1996–2004. Washington DC: World Bank.

    Book  Google Scholar 

  • Kaufmann, D., Kraay, A. and Mastruzzi, M. (2006a) Governance matters V: Aggregate and individual governance indicators for 1996–2005. Washington DC: World Bank.

    Book  Google Scholar 

  • Kaufmann, D., Kraay, A. and Mastruzzi, M. (2006b) The Worldwide Governance Indicators project: Answering the critics. Washington DC: World Bank.

    Google Scholar 

  • Kaufmann, D., Kraay, A. and Mastruzzi, M. (2007) Governance matters VI: Aggregate and individual governance indicators 1996–2006. Washington DC: World Bank.

    Google Scholar 

  • Kaufmann, D., Kraay, A. and Mastruzzi, M. (2008) Governance matters VII: Aggregate and individual governance indicators 1996–2007. Washington DC: World Bank.

    Book  Google Scholar 

  • Kaufmann, D., Kraay, A. and Zoido-Lobatón, P. (1999a) Aggregating governance indicators. Washington DC: World Bank.

    Google Scholar 

  • Kaufmann, D., Kraay, A. and Zoido-Lobatón, P. (1999b) Governance matters. Washington DC: World Bank.

    Google Scholar 

  • Kaufmann, D., Kraay, A. and Zoido-Lobatón, P. (2002) Governance matters II: Updated indicators for 2000/01. Washington DC: World Bank.

    Google Scholar 

  • Kealy, M.J., Dovidio, J.F. and Rockel, M.L. (1988) Accuracy in valuation is a matter of degree. Land Economics 64 (2): 158–171.

    Article  Google Scholar 

  • Knack, S. (2006) Measuring corruption in Eastern Europe and Central Asia: A critique of the cross-country indicators. Washington DC: World Bank.

    Book  Google Scholar 

  • Kurtz, M. and Schrank, A. (2006) Growth and governance: Models, measures and mechanisms. Journal of Politics 69 (2): 538–554.

    Article  Google Scholar 

  • Kurtz, M. and Schrank, A. (2007) Growth and governance: A defense. Journal of Politics 69 (2): 563–569.

    Article  Google Scholar 

  • Laughland, A.S., Musser, W.N., Shortle, J.S. and Musser, L.M. (1996) Construct validity of averting cost measures of environmental benefits. Land Economics 72 (1): 100–112.

    Article  Google Scholar 

  • Liu, M.C. and San, G. (2006) Social learning and digital divides: A case study of internet technology diffusion. Kyklos 59 (2): 307–321.

    Article  Google Scholar 

  • Llamazares, I. (2005) Patterns in contingencies: The interlocking of formal and informal political institutions in contemporary Argentina. Social Forces 83 (4): 1671–1695.

    Article  Google Scholar 

  • McKenzie, D.J. (2005) Measuring inequality with asset indicators. Journal of Population Economics 18: 229–260.

    Article  Google Scholar 

  • Méon, P.-G. and Sekkat, K. (2005) Does corruption grease or sand the wheels of growth? Public Choice 122 (1–2): 69–75.

    Article  Google Scholar 

  • Meyer, B.D. (1995) Natural and quasi-experiments in economics. Journal of Business and Economic Statistics 13 (2): 151–161.

    Google Scholar 

  • Naude, W.A. (2004) The effects of policy, institutions and geography on economic growth in Africa: An econometric study based on cross-section and panel data. Journal of International Development 16: 821–849.

    Article  Google Scholar 

  • Neumayer, E. (2002) Do democracies exhibit stronger international environmental commitment? A cross-country analysis. Journal of Peace Research 39 (2): 139–164.

    Article  Google Scholar 

  • North, D. and Thomas, R.P.T. (1996) Rise of the western world: A new economic history. Cambridge: Cambridge University Press.

    Google Scholar 

  • Olken, B.A. (2006) Corruption perceptions v. corruption reality. NBER Working Paper No. 12428.

  • Patterson, M.L. (1990) On the construct validity and developmental course of rapport. Psychological Inquiry 1 (4): 320–321.

    Article  Google Scholar 

  • Perry, J.L. (1996) Measuring public service motivation: An assessment of construct reliability and validity. Journal of Public Administration Research and Theory: J-PART 6 (1): 5–22.

    Article  Google Scholar 

  • Pfeiffer, C. (2005) Media use and its impacts on crime perception, sentencing attitudes and crime policy. European Journal of Criminology 2 (3): 259–285.

    Article  Google Scholar 

  • Razafindrakoto, M. and Roubaud, F. (2006) Are international databases on corruption reliable? A comparison of expert opinion surveys and household surveys in Sub-Saharan Africa. Paris: DIAL.

    Google Scholar 

  • Seligson, M.A. (2006) The measurement and impact of corruption victimization: Survey evidence from Latin America. World Development 34 (2): 381–404.

    Article  Google Scholar 

  • Smith, G.T. (2005) On construct validity: Issues of method and measurement. Psychological Assessment 17 (4): 396–408.

    Article  Google Scholar 

  • Stolk, E.A. and Busschbach, J.J.V. (2001) The comparison of the euroqol and the health utilities index in patients treated for congential anomalies. The European Journal of Health Economics 2 (2): 54–59.

    Article  Google Scholar 

  • Studemund, A.H. (1997) Using econometrics: A practical guide. Reading, MA: Addison-Wesley Educational Publishers.

    Google Scholar 

  • Tamanaha, B.Z. (2008) The dark side of the relationship between the rule of law and liberalism. NYU Journal of Law and Liberty, Vol. 33; St. John's Legal Studies Research Paper No. 08-0096.

  • Thomas, M.A. (2007) The governance bank. International Affairs 83 (4): 729–745.

    Article  Google Scholar 

  • Underwood, B.J. (1957) Psychological Research. New York: Appleton-Century-Crofts.

    Google Scholar 

  • Van de Walle, S. and Bouckaert, G. (2007) Perceptions of productivity and performance in Europe and the United States. International Journal of Public Administration 30 (11): 1123–1140.

    Article  Google Scholar 

  • Wallace, R.B. and Herzog, A.R. (1995) Overview of health measures in the health and retirement study. The Journal of Human Resources 30: S84–S107.

    Article  Google Scholar 

  • Westen, D. and Rosenthal, R. (2003) Quantifying construct validity: Two simple measures. Journal of Personality and Social Psychology 94: 608–618.

    Article  Google Scholar 

  • White, H. (1982) Maximum likelihood estimation of misspecified models. Econometrica 50 (1): 1–26.

    Article  Google Scholar 

  • Williams, R.H., Zimmerman, D.W., Zumbo, B.D. and Ross, D. (2003) Charles Spearman: British behavioral scientist. Human Nature Review 3: 114–118.

    Google Scholar 

  • Wooldridge, J.M. (2003) Introductory Economics: A Modern Approach. US: South-Western.

    Google Scholar 

  • World Bank. (1994) Governance: The World Bank's experience. Washington DC: World Bank.

  • World Bank. (1998) Assessing aid – what works, what doesn’t, and why. Washington DC: World Bank.

  • World Bank. (2006a) Strengthening Bank Group engagement on governance and anticorruption. Washington DC: World Bank.

  • World Bank. (2006b) World Bank releases largest available governance data source. Washington DC: World Bank.

Download references

Acknowledgements

The initial draft of this paper, under the same title, was written and posted in 2006; a revision was posted in 2007. The author thanks Ray Koopman, Francis Fukuyama, David Armor, Daniel Kaufmann, Aart Kraay, Richard Messick, Anwar Shah, Michael Johnston, Omar Azfar, Stephen Knack, Simeon Djankov, Alice Fredericks and the anonymous referees of this journal for their thoughtful comments and insights, as well as her research assistant, Scott Merrill, for his able assistance.

Author information

Authors and Affiliations

  1. The Johns Hopkins University, USA

    M A Thomas

Authors
  1. M A Thomas
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Thomas, M. What Do the Worldwide Governance Indicators Measure?. Eur J Dev Res 22, 31–54 (2010). https://doi.org/10.1057/ejdr.2009.32

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1057/ejdr.2009.32

Keywords

Search

Navigation