Skip to main content
Log in

Measuring Governance: Implications of Conceptual Choices

  • Original Article
  • Published:
The European Journal of Development Research Aims and scope Submit manuscript

Abstract

This article examines challenges of measurement validity in aggregate governance indicators. We focus on three deleterious consequences of aggregating perception-based indicators in the absence of conceptual clarity: (i) the scant attention to content validity; (ii) the conflation of causes, characteristics and consequences of governance; and (iii) the underestimation of uncertainty. As an alternative, we present a Bayesian latent variable framework for measuring governance. This alternative formal statistical model offers several advantages: it is a principled method that allows the researcher to make explicit the conceptual choices in measuring governance, and it provides an assessment of uncertainty by letting measurement error (noise) in governance measures propagate into inferences. We argue that this multidimensional, disaggregated and transparent approach will lead to greater measurement validity and transparency about the uncertainty in governance measures. The conceptual and empirical analysis uses data from the World Bank Governance Indicators.

Abstract

Cet article traite des difficultés relatives à la validité de mesure des indicateurs globaux de gouvernance. Nous nous intéressons, en particulier, à trois conséquences néfastes de l’agrégation d’indicateurs fondés sur la perception, en l’absence de clarté conceptuelle: 1) le peu d’attention accordé à la validité du contenu; 2) une confusion entre les causes, les caractéristiques et les conséquences de la gouvernance, et 3) la sous-estimation de l’incertitude. Nous proposons, comme alternative, une modélisation bayésienne avec variables latentes pour mesurer la gouvernance. Ce modèle statistique formel présente plusieurs avantages: il s’agit d’une méthode raisonnée qui permet au chercheur d’expliciter les choix conceptuels en termes de mesure de gouvernance, et il fournit une évaluation honnête de l’incertitude en laissant l’erreur de mesure dans la mesure de la gouvernance se propager dans les inférences. Nous soutenons que cette approche multidimensionnelle, désagrégée et transparente permettra de renforcer la validité de la mesure et la transparence sur l’incertitude autour des mesures de gouvernance. L’analyse conceptuelle et empirique s’appuie sur des données tirées des indicateurs de gouvernance de la Banque mondiale.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3

Similar content being viewed by others

Notes

  1. See, for example, Acemoglu et al, 2001, Keefer and Knack, 1995, Fearon, 2011, Easterly et al, 2006, Denisova et al, 2009, Kaufmann and Kraay, 2002 and Dollar and Kraay, 2003.

  2. We use index/indices to mean highly aggregated, composite measures of governance, while indicator/indicators refers to specific, disaggregated elements of governance, for example, WGI is an index and an indicator might be frequency of household bribery.

  3. See Coppedge et al (2011) for similar measurement principles.

  4. However, see discussion below of WGI and the Quality of Governance Institute for important advances in this regard.

  5. As Kaufmann and colleagues point out, unobserved component models were pioneered in economics by Goldberger (1972), and are closely related to hierarchical and empirical Bayes models in statistics by Efron and Morris (1971, 1972). In this sense, they are very similar to our approach below.

  6. The WGI’s definition of governance was initially a minimalist one. However, with the increased availability of data indicators and their progressive inclusion in the index, it has changed. The 2007 update of the WGI defines governance as the traditions and institutions by which authority in a country is exercised. This includes the process by which governments are selected, monitored and replaced; the capacity of the government to effectively formulate and implement sound policies; and the respect of citizens and the state for the institutions that govern economic and social interactions among them (Kaufman et al, 2007, p. 7).

  7. A table with a list of the single items in the 2007 WGI dimensions is available upon request or by Kaufmann et al (2007).

  8. RL is defined as measuring perceptions of the extent to which agents have confidence in and abide by the rules of society and, in particular, the quality of contract enforcement, property rights, the police and the courts, as well as the likelihood of crime and violence (Kaufman et al, 2008, p. 7).

  9. More recently, the QoG Institute at Göteborg University in Sweden has compiled both a cross-sectional data set with global coverage pertaining to the year 2002 (or the closest year available) and a cross-sectional time-series data set with global coverage spanning the time period 1946–2010. This marks an advancement: instead of providing aggregate indicators as the WGI do, all the information of the indicators themselves is accessible to scholars in a disaggregated and transparent form.

  10. These nine are: Global Integrity Index, OECD Development Center African Economic Outlook, European Bank for Reconstruction and Development Transition, Business Environment and Enterprise Performance Survey, Asian Development Bank, Vanderbilt University Americas Barometer, Afrobarometer, Latinobarometro and Political Economic Risk Consultancy Corruption in Asia Survey. A table detailing missingness for each source indicator in 2007 is available from the authors.

  11. See Lee (2007) for a fully Bayesian approach to structural equation modelling.

  12. The results of the Bayesian latent variable model are very similar to the results of traditional factor analysis (results available on request). Traditional factor analysis techniques result in identification and computation challenges with missing values.

References

  • Acemoglu, D., Johnson, S. and Robinson, J. (2001) The colonial origins of comparative development: An empirical investigation. American Economic Review 91 (5): 1369–1401.

    Article  Google Scholar 

  • Adcock, R. and Collier, D. (2001) Measurement validity: A shared standard for qualitative and quantitative research. American Political Science Review 95 (3): 529–546.

    Article  Google Scholar 

  • Arel-Bundock, V. and Mebane, W. (2011) Measurement error, missing values and latent structure in governance indicators. Paper presented at the 2011 Annual Meeting of the American Political Science Association; 1–4 September, Seattle, WA.

  • Coppedge, M. et al. (2011) Conceptualizing and measuring democracy: A new approach. Perspectives on Politics 9 (2): 247–267.

    Article  Google Scholar 

  • Denisova, I., Eller, M., Frye, T. and Zhuravskaya, E. (2009) Who wants to revise privatization? The complementarity of market skills and institutions. American Political Science Review 103 (2): 284–304.

    Article  Google Scholar 

  • Department for International Development. (2001) Making Government Work for Poor People: Building State Capacity. Strategy Paper. London: Department for International Development.

  • Dollar, D. and Kraay, A. (2003) Institutions, trade, and growth. Journal of Monetary Economics 50 (1): 133–162.

    Article  Google Scholar 

  • Easterly, W., Ritzen, J. and Woolcock, M. (2006) Social cohesion, institutions, and growth. Economics & Politics 18 (2): 103–120.

    Article  Google Scholar 

  • Efron, B. and Morris, C. (1971) Limiting the risk of Bayes and Empirical Bayes estimators—part I: The Bayes case. Journal of the American Statistical Association 66 (336): 807–815.

    Google Scholar 

  • Efron, B. and Morris, C. (1972) Limiting the risk of Bayes and Empirical Bayes estimators—part II: The Empirical Bayes case. Journal of the American Statistical Association 67 (337): 130–139.

    Google Scholar 

  • Fearon, J. (2011) Governance and Civil War Onset. Prepared for the World Bank’s 2011 World Development Report on Conflict, Security, and Development.

  • Gelman, A. (2006) Prior distributions for variance parameters in hierarchical models. Bayesian Analysis 1 (3): 515–533.

    Google Scholar 

  • Gervasoni, C. (2006) Data set review: The World Bank’s Governance Indicators (1996–2004). APSA-CP Newsletter 17 (1): 17–20.

    Google Scholar 

  • Givens, D. (2012) Defining governance matters: A factor analytic assessment of governance institutions. Journal of Comparative Economics 41 (4): 1026–1053.

    Article  Google Scholar 

  • Goertz, G. (2006) Social Science Concepts: A User’s Guide. Princeton, NJ: Princeton University Press.

    Google Scholar 

  • Goldberger, A.S. (1972) Structural equation methods in the social sciences. Econometrica 40 (6): 979–1001.

    Article  Google Scholar 

  • Grindle, M.S. (2010) Good Governance: The Inflation of an Idea. Technical Report John F. Kennedy School of Government, Harvard University.

  • Grzymala-Busse, A.M. (2007) Rebuilding Leviathan: Party Competition and State Exploitation in Post-Communist Democracies. Cambridge, UK: Cambridge University Press.

    Book  Google Scholar 

  • Jackman, S. (2009) Bayesian Analysis for the Social Sciences. New York: John Wiley and Sons.

    Book  Google Scholar 

  • Janda, K. and Kwak, J.Y. (2011) Party Systems and Country Governance. Boulder, CO: Paradigm Publishers.

    Google Scholar 

  • Jong-sung, Y. and Khagram, S. (2005) A comparative study of inequality and corruption. American Sociological Review 70 (1): 136.

    Article  Google Scholar 

  • June, R. (2008) A User’s Guide to Measuring Corruption. Oslo, Norway: UNDP/Global Integrity.

    Google Scholar 

  • Kaufmann, D. and Kraay, A. (2002) Growth without governance. Economia 3 (1): 169–229.

    Google Scholar 

  • Kaufmann, D., Kraay, A. and Zoido-Lobatón, P. (1999a) Aggregating Governance Indicators. World Bank Policy Research Working Paper No. 2195. Washington, DC.

  • Kaufmann, D., Kraay, A. and Zoido-Lobatón, P. (1999b) Governance Matters. World Bank Policy Research Working Paper No. 2196. Washington, DC.

  • Kaufmann, D., Kraay, A. and Mastruzzi, M. (2007) Governance Matters VII: Aggregate and Individual Governance Indicators, 1996–2007. World Bank, World Bank Institute, Global Programs Division, and Development Research Group, Macroeconomics and Growth Team. Technical report. Washington, DC.

  • Kaufmann, D., Kraay, A. and Mastruzzi, M. (2008) Governance Matters VII: Aggregate and Individual Governance Indicators, 1996–2007. World Bank Policy Research Working Paper No. 4654.

  • Kaufmann, D., Kraay, A. and Mastruzzi, M. (2010) The Worldwide Governance Indicators: Methodology and Analytical issues. World Bank Policy Research Working Paper No. 5430.

  • Keefer, P. and Knack, A. (1995) Institutions and economic performance: Cross-country tests using alternative institutional measures. Economics and Politics 7 (3): 207–227.

    Article  Google Scholar 

  • Kurtz, M. and Schrank, A. (2007a) Growth and governance: Models, measures, and mechanisms. Journal of Politics 69 (2): 538–554.

    Article  Google Scholar 

  • Kurtz, M. and Schrank, A. (2007b) Growth and governance: A defense. Journal of Politics 69 (2): 563–569.

    Article  Google Scholar 

  • Kurtz, M. and Schrank, A. (2012) Perception and Misperception in Governance Research: Evidence from Latin America. In: K.S. Jomo and A. Chowdhury (eds.) Is Good Governance Good for Development? London: Bloomsbury Academic, pp. 71–96.

    Google Scholar 

  • Langbein, L. and Knack, S. (2010) The worldwide governance indicators: Six, one, or none? The Journal of Development Studies 46 (2): 350–370.

    Article  Google Scholar 

  • La Porta, R., Lopez-de-Silanes, F., Shleifer, A. and Vishny, R. (1999) The quality of government. Journal of Law, Economics, and organization 15 (1): 222.

    Article  Google Scholar 

  • Lee, S. (2007) Structural Equation Modeling: A Bayesian Approach, Vol. 680. New York: John Wiley & Sons.

    Book  Google Scholar 

  • Levendusky, M.S., Pope, J.C. and Jackman, S.D. (2008) Measuring district-level partisanship with implications for the analysis of US elections. The Journal of Politics 70 (03): 736–753.

    Article  Google Scholar 

  • Mainwaring, S., Scully, T. and Vargas Cullell, J. (2009) Measuring success in democratic governance. In: S. Mainwaring and T. Scully (eds.) Democratic Governance in Latin America. Stanford, CA: Stanford University Press.

    Google Scholar 

  • Manzetti, L. and Blake, C.H. (1996) Market reforms and corruption in Latin America: New means for old ways. Review of International Political Economy 3 (4): 662–697.

    Article  Google Scholar 

  • North, D.C. (1990) Institutions, Institutional Change and Economic Performance. Cambridge: Cambridge University Press.

    Book  Google Scholar 

  • Pemstein, D., Meserve, S.A. and Melton, J. (2010) Democratic compromise: A latent variable analysis of ten measures of regime type. Political Analysis 18 (4): 426.

    Article  Google Scholar 

  • Przeworski, A. and Teune, H. (1970) The Logic of Comparative Social Inquiry. Krieger, Malabar (reprint 1982), 74–87.

  • Quinn, K. (2004) Bayesian factor analysis for mixed ordinal and continuous responses. Political Analysis 12 (4): 338.

    Article  Google Scholar 

  • Rothstein, B. and Teorell, J. (2008) What is quality of government? A theory of impartial government institutions. Governance 21 (2): 165–190.

    Article  Google Scholar 

  • Sandholtz, W. and Koetzle, W. (2000) Accounting for corruption: Economic structure, democracy, and trade. International Studies Quarterly 44 (1): 31–50.

    Article  Google Scholar 

  • Thomas, M.A. (2010) What do the worldwide governance indicators measure? European Journal of Development Research 22 (1): 31–54.

    Article  Google Scholar 

  • Tommasi, M. (2004) Crisis, political institutions and economic reforms: The good, the bad, and the ugly. In: B. Tungodden, N. Stern and I. Kolstad (eds.) Toward Pro-Poor Policies – Aid, Institutions and Globalization. Oxford, UK: Oxford University Press, pp. 135.

    Google Scholar 

  • Treier, S. and Jackman, S. (2008) Democracy as a latent variable. American Journal of Political Science 52 (1): 201–217.

    Article  Google Scholar 

Download references

Acknowledgements

We thank Michael Coppedge, Aart Kraay, Zachary Elkins, Stephen Jessee, Kurt Weyland and Kenneth Janda for research advice and comments on earlier drafts.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Katherine Bersch.

Additional information

Supplementary Information accompanies on European Journal of Development Research website (http://www.palgrave-journals.com/ejdr)

Electronic supplementary material

Appendix

Appendix

In this Bayesian analysis, we take ‘CC’, one of the six WGI dimensions, as an example. Following Treier and Jackman (2008), corruption is treated as a latent, continuous unidimensional variable. Using a Bayesian framework, we compute the posterior density over all unknown parameters conditional on the indicators (that is, 2007 WGI second-level source indicators for corruption) and priors (Jackman, 2009, p. 438). We assume for this analysis that the indicators are on an interval scale. The WGI indicators for each country for 2007 are modeled as functions of the unobserved level of corruption, following the factor analytic model. Let i =1, …, n index countries and j=1, …, m index the WGI corruption source indictors. The equation and prior take the form:

where x i is the latent level of corruption in country i and y ij is the i-th country’s score on indicator j. The intercept is γj0, and as Jackman (2009, p. 129) points out in factor analysis it is typical to drop the intercept terms, as shifts in the mean level of any or all variables is of no consequence when we turn to an analysis of covariances between items. In item response theory, this is the difficulty parameter; in this case as the term was added, it is an ease parameter. Finally, γj1 is the factor loading, or item discrimination parameter, which conveys the extent to which the indicators of corruption are tapping the latent concept of corruption. Indicators strongly related to corruption will have discrimination parameters that are large in magnitude.

We specify vague priors for γj0 and γj1, the difficulty and discrimination parameters, respectively, to reflect the absence of prior information about these indicators. Following Gelman’s (2006) recommendation, we put a uniform prior on the standard deviation over a large range, 0–100. Furthermore, in order to estimate this model and to prevent shifts in location and scale for the latent traits, we constrain x i to have mean 0and a variance of 1.

We estimate the model using the Markov chain Monte Carlo (MCMC) algorithm in WinBugs. After discarding the first 10 000 iterations as burn-in, estimates and inferences are based on 50 000 iterations, thinned by 100, in order to produce 500 approximately independent draws from the joint posterior density. Standard MCMC diagnostics for the sample are consistent with Markov chain convergence.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Bersch, K., Botero, S. Measuring Governance: Implications of Conceptual Choices. Eur J Dev Res 26, 124–141 (2014). https://doi.org/10.1057/ejdr.2013.49

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1057/ejdr.2013.49

Keywords

Navigation