Skip to main content
SpringerLink
Log in

Validating integration and citizenship policy indices

  • Original Article
  • Published:
Comparative European Politics Aims and scope

Abstract

After a long period of mostly case study research, in the recent past various integration and citizenship policy scholars have started to compare a relatively large range of countries by constructing policy indices. As there are almost as many indices as there are such studies, one might wonder whether it matters which indices we use. The main goal of this article is to compare these indices and to investigate commonalities and differences. By means of content, convergent and construct validity tests, methodological strengths and weaknesses will be discussed. It will be shown that most of the indices consist of different components and are sometimes weakly related to each other even if they can be explained by similar factors. An overview of the current state of research in this field allows us to learn from the innovative ideas that have been developed so far and to discuss the next steps to be taken.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. A similar development has taken place in the immigration literature (see Helbling et al, 2013 and Bjerre et al, 2013). However, this article focuses on integration and citizenship policy indices as the data sets on immigration policies are not accessible and because most researchers in this field have only investigated sub-policy fields such as asylum or labor migration (for example, Thielemann, 2003; Hatton, 2004; Cerna, 2008; Ruhs, 2011; see, however, Givens and Luedtke, 2005 as well as Klugman and Pereira, 2009).

  2. By ‘index’ I understand a measurement that operationalizes in a quantitative way a social phenomenon and represents an aggregate of data. By ‘indicator’, ‘item’ or ‘variable’ I mean the most basic element of an indicator.

  3. In the remainder of this article, I use the abbreviations as they are used by the authors. If there is no abbreviation for an index I simply refer to its author. I mostly discuss indices that allow comparisons at the country level. In Switzerland, indices have been created for the subnational level that will be discussed at the margins of this article (Helbling, 2008; Manatschal, 2012; see also Manatschal and Stadelmann-Steffen 2013).

  4. Bollen (1989) also discusses criterion validity that involves the comparison of an index with some other generally accepted measure of the same concept. If there was already an index that was accepted by the research community as the most valid measurement, the validity of new indices could simply be assessed by comparing them. However, since researchers have just started to measure integration and citizenship policies, it is not surprising that there is so far no such generally accepted index. It appears that it becomes almost impossible to assess the criterion validity of the existing indices and that such tests need to be precluded at the current stage of research.

  5. As a consequence the number of data points (N) that can be compared varies a lot depending on which indices are analysed.

  6. More detailed analyses with scatter plots have, however, shown that the results are not driven by outliers.

  7. I have retained the naturalization subindices as conceptualized and measured by the authors: For the ICRI data I retained the ‘nationality acquisition’ subindex, for the LOI data the ‘naturalization’ subindex and for MIPEX the ‘access to nationality’ subindex.

  8. As the EUDO CITIZENSHIP Observatory has not constructed an index so far we cannot include the CITLAW indicators in these analyses.

  9. The LOI indices are excluded from these analyses as Waldrauch and Hofinger (1997) have not constructed an overall index. The same holds for the EUDO CITLAW Indicators. Jeffers et al (2012, p. 7) argue that their indicators cover more aspects than the other existing citizenship indices. According to them, aggregating all these different aspects would lead to an index that could not be interpreted in a straightforward way.

References

  • Bader, V. (2007) The governance of Islam in Europe: The perils of modelling. Journal of Ethnic and Migration Studies 33 (6): 871–886.

    Article  Google Scholar 

  • Banting, K. and Kymlicka, W. (eds.) (2006a) Multiculturalism and the Welfare State in: Recognition and Redistribution in Contemporary Democracies. Oxford: Oxford University Press.

    Book  Google Scholar 

  • Banting, K. and Kymlicka, W. (eds.) (2006b) Introduction. Multiculturalism and the welfare state: Setting the context. In: Multiculturalism and the Welfare State. Recognition and Redistribution in Conteporary Democracies. Oxford: Oxford University Press, pp. 1–45.

    Chapter  Google Scholar 

  • Banting, K. and Kymlicka, W. (forthcoming) Is there really a backlash against multiculturalism policies? New evidence from the multiculturalism policy index. Comparative European Politics.

  • Banting, K., Johnston, R., Kymlicka, W. and Soroka, S. (2006) Do multiculturalism policies erode the welfare state? An empirical analysis. In: K. Banting and W. Kymlicka (eds.) Multiculturalism and the Welfare State. Recognition and Redistribution in Conteporary Democracies. Oxford: Oxford University Press, pp. 49–91.

    Chapter  Google Scholar 

  • Bjerre, L., Helbling, M., Römer, F. and Zobel, M. (2013) Conceptualizing and Measuring Immigration Policies. A Comparative Perspective Manuscript under review.

  • Bollen, K.A. (1989) Structural Equations with Latent Variables. New York: John Wiley and Sons.

    Book  Google Scholar 

  • Bolliger, C. (2004) Spielt es eine Rolle, wer entscheidet? Einbürgerungen in Gemeinden mit Parlaments- und Volksentscheid im Vergleich. In: P. Steiner and H-R. Wicker (eds.) Paradoxien im Bürgerrecht. Sozialwissenschaftliche Studien zur Einbürgerungspraxis in Schweizer Gemeinden. Zürich: Seismo, pp. 43–61.

  • Brubaker, R. (1992) Citizenship and Nationhood in France and Germany. Cambridge, MA: Harvard University Press.

    Google Scholar 

  • Brubaker, R. (2010) Migration, membership, and the modern nation-state: Internal and external dimensions of the politics of belonging. Journal of Interdisciplinary History 41 (1): 61–78.

    Article  Google Scholar 

  • Bultmann, P.F. (1999) Lokale Gerechtigkeit im Einbürgerungsrecht. Berlin, Germany: Duncker&Humblot.

    Google Scholar 

  • Carmines, E.G. and Zeller, R.A. (1979) Reliability and Validity Assessment. Beverly Hills, CA and London: SAGE Publications.

    Book  Google Scholar 

  • Casper, G. and Tufis, C. (2003) Correlation versus interchangeability: The limited robustness of empirical findings on democracy using highly correlated data sets. Political Analysis 11 (2): 196–203.

    Article  Google Scholar 

  • Cerna, L. (2008) Towards an EU Blue Card? The delegation of National High Skilled Immigration Policies to the EU level. Oxford University, Centre on Migration, Policy and Society (COMPAS) Working Paper Series, July 2011.

  • Cinar, D., Hofinger, C. and Waldrauch, H. (1995) Integrationsindex. Zur rechtlichen Integration von AusländerInnen in ausgewählten europäischen Ländern. Political Science Series No. 25. Vienna, Austria: Institute for Advanced Studies.

    Google Scholar 

  • Coppedge, M. and Gerring, J. (2011) Conceptualizing and measuring democracy: A new approach. Perspectives on Politics 9 (2): 247–267.

    Article  Google Scholar 

  • Crepaz, M. (2006) ‘If you are my brother, I may give you a dime!’ Public opinion on multiculturalism, trust and the welfare state. In: K. Banting and W. Kymlicka (eds.) Multiculturalism and the Welfare State. Recognition and Redistribution in Conteporary Democracies. Oxford: Oxford University Press, pp. 92–117.

    Chapter  Google Scholar 

  • Dinesen, P. and Hooghe, M. (2010) When in Rome, Do as The Romans Do: The acculturation of generalized trust among immigrants in Western Europe. International Migration Review 44 (3): 697–727.

    Article  Google Scholar 

  • Dornis, C. (1999) Einbürgerung in Deutschland: Eine vergleichende Untersuchung der Verwaltungspraxis in verschiedenen Regionen. Demographie aktuell (15). Berlin, Germany: Humbold-Universität.

    Google Scholar 

  • Dornis, C. (2001) Einbürgerung in Deutschland: Die Verwaltungspraxis in verschiedenen Regionen im Vergleich. In: L. Akgün and D. Thränhardt (eds.) Integrationspolitik in föderalistischen Systemen (Jahrbuch Migration). Münster, Germany: Lit, pp. 63–99.

    Google Scholar 

  • Dronkers, J. and Vink, M.P. (2012) Explaining access to citizenship in Europe: How policies affect naturalisation rates. European Union Politics 13 (3): 390–412.

    Article  Google Scholar 

  • Elkins, Z. (2000) Gradations of democracy? Empirical tests of alternative conceptualizations. American Journal of Political Science 44 (2): 293–300.

    Article  Google Scholar 

  • Fleischmann, F. and Dronkers, J. (2010) Unemployment among immigrants in European labour markets: An analysis of origin and destination effects. Work, Employment and Society 24 (2): 337–354.

    Article  Google Scholar 

  • Givens, T. and Luedtke, A. (2005) European immigration policies in comparative perspective: Issue salience, partisanship and immigrant rights. Comparative European Politics 3 (1): 1–22.

    Article  Google Scholar 

  • Goodman, S.W. (2010) Integration requirements for integration’s sake? Identifying, categorising and comparing civic integration policies. Journal of Ethnic and Migration Studies 36 (5): 753–772.

    Article  Google Scholar 

  • Goodman, S.W. (2012) Measurement and interpretation issues in civic integration studies: A rejoinder. Journal of Ethnic and Migration Studies 38 (1): 173–186.

    Article  Google Scholar 

  • Hagedorn, H. (2001a) Föderalismus und die deutsche Staatsangehörigkeit: Die Einbürgerungspolitik der Bundesländer. In: L. Akgün and D. Thränhardt (eds.) Integrationspolitik in föderalistischen Systemen (Jahrbuch Migration). Münster, Germany: Lit, pp. 91–117.

    Google Scholar 

  • Hagedorn, H. (2001b) Einbürgerungspolitik in Deutschland und Frankreich. Leviathan 29 (1): 36–57.

    Article  Google Scholar 

  • Hammar, T. (1990) Democracy and the Nation State. Aliens, Denizens and Citizens in a World of International Migration. Aldershot: Avebury.

    Google Scholar 

  • Hatton, T.J. (2004) Seeking asylum in Europe. Economic Policy 19 (38): 5–62.

    Article  Google Scholar 

  • Helbling, M. (2008) Practising Citizenship and Heterogeneous Nationhood. Naturalisations in Swiss Municipalities. Amsterdam, the Netherlands: Amsterdam University Press.

    Book  Google Scholar 

  • Helbling, M. and Tresch, A. (2011) Measuring party positions and issue salience from media coverage: Discussing and crossvalidating new indices. Electoral Studies 30 (1): 174–183.

    Article  Google Scholar 

  • Helbling, M., Bjerre, L., Römer, F. and Zobel, M. (eds.) (2013) How to measure immigration policies. In: APSA Newsletter Migration and Citizenship. Issue 2 (forthcoming).

  • Herrera, Y.M. and Kapur, D. (2007) Improving data quality: Actors, incentives, and capabilities. Political Analysis 15 (4): 365–386.

    Article  Google Scholar 

  • Hooghe, M., Reeskens, T., Stolle, D. and Trappers, A. (2009) Ethnic diversity and generalized trust in Europe. A cross-national multilevel study. Comparative Political Studies 42 (2): 198–223.

    Article  Google Scholar 

  • Howard, M.M. (2006) Comparative citizenship: An agenda for cross-national research. Perspectives on Politics 4 (3): 443–455.

    Article  Google Scholar 

  • Howard, M.M. (2009) The Politics of Citizenship in Europe. New York: Cambridge University Press.

    Book  Google Scholar 

  • Huddleston, T. and Niessen, J. (2011) Migration Integration Policy Index III, British Council and Migration Policy Group.

  • Janoski, T. (2010) The Ironies of Citizenship: Naturalization and Integration in Industrialized Countries. New York: Cambridge University Press.

    Book  Google Scholar 

  • Janoski, T. (forthcoming) Foreigners acquiring citizenship in advanced industrialized countries: The complexities of measuring naturalization rates. Comparative European Politics.

  • Jeffers, K., Honohan, I. and Bauböck, R. (2012) CITLAW indicators: How to measure the purposes of citizenship laws. San Domenico di Fiesole: EUDO CITIZENSHIP Observatory, http://eudo-citizenship.eu/indicators/eudo-citizenship-law-indicators.

  • Kesler, C. and Bloemraad, I. (2010) Does immigration erode social capital? The conditional effects of immigration-generated diversity on trust, membership and participation across 19 countries, 1981–2000. Canadian Journal of Political Science 43 (2): 319–347.

    Article  Google Scholar 

  • Klugman, J. and Pereira, I.M. (2009) Assessment of National Migration Policies: An Emerging Picture on Admissions, Treatment and Enforcement in Developing and Developed Countries. United Nations Development Programme: Human Development Research Paper 2009/48.

  • Koning, E.A. (2011) Ethnic and civic dealings with newcomers: Naturalization policies and practices in twenty-six immigration countries. Ethnic and Racial Studies 34 (11): 1974–1994.

    Article  Google Scholar 

  • Koopmans, R. and Kriesi, H. (1997) Citoyenneté, identité nationale et mobilisation de l’extrême droite. Une comparaison entre la France, l’Allemagne, les Pays-Bas et la Suisse. In: P. Birnbaum (ed.) Sociologie des Nationalismes. Paris, France: Presses Universitaires de France, pp. 295–324.

    Google Scholar 

  • Koopmans, R., Statham, P., Giugni, M. and Passy, F. (2005) Contested Citizenship: Immigration and Cultural Diversity in Europe. Minneapolis, MN and London: University of Minnesota Press.

    Google Scholar 

  • Koopmans, R., Michalowski, I. and Waibel, S. (2012) Citizenship rights for immigrants: National political processes and cross-national convergence in Western Europe, 1980–2008. American Journal of Sociology 117 (4): 1202–1245.

    Article  Google Scholar 

  • Ludvig, A. (2004) Why should Austria be different from Germany? The two recent nationality reforms in contrast. German Politics 13 (3): 499–515.

    Article  Google Scholar 

  • Manatschal, A. (2012) Path dependent or dynamic? Cantonal integration policies between regional citizenship traditions and right populist party politics. Ethnic and Racial Studies 35 (2): 281–297.

    Google Scholar 

  • Manatschal, A. and Stadelmann-Steffen, I. (2013) Cantonal variations of integration policy and their impact on immigrant educational inequality. Comparative European Politics 11 (5): 671–695.

    Article  Google Scholar 

  • Marks, G., Hooghe, L., Steenbergen, M.R. and Bakker, R. (2007) Crossvalidating data on party positioning on European integration. Electoral Studies 26 (1): 23–38.

    Article  Google Scholar 

  • McLaren, L.M. (2010) Cause for Concern? The Impact of Immigration on Political Trust, Policy Network, http://www.policy-network.net/publications/3889/Cause-for-concern? The-impact-of-immigration-on-political-trust, accessed 20 September 2010.

  • Michalowski, I. and Van Oers, R. (2012) How can we categorize and interpret civic integration policies? Journal of Ethnic and Migration Studies 38 (1): 163–171.

    Article  Google Scholar 

  • Munck, G.L. and Verkuilen, J. (2002) Conceptualizing and measuring democracy. Evaluating alternative indices. Comparative Political Studies 35 (1): 5–34.

    Article  Google Scholar 

  • Niessen, J., Huddleston, T., Citron, L., Geddes, A. and Jacobs, D. (2007) Migrant Integration Policy Index. Brussels, Belgium: British Council of Migration Policy Group.

    Google Scholar 

  • Ray, L. (2007) Validity of measured party positions on European integration: Assumptions, approaches, and a comparison of alternative measures’. Electoral Studies 26 (1): 11–22.

    Article  Google Scholar 

  • Reichel, D. (2011) Do Legal Regulations Hinder Naturalization? Citizenship Policies and Naturalization Rates in Europe. European University Institute, Robert Schuman Centre for Advanced Studies, EUDO Citizenship Observatory. RSCAS Working Paper 2011/51.

  • Ruedin, D. (2011) The Reliability of MIPEX Indices as Scales, SOM. Working Papers no. 2011–03, http://www.som-project.eu, accessed April 2012.

  • Ruhs, M. (2011) Openness, Skills and Rights: An Empirical Analysis of Labour Immigration Programmes in 46 High and Middle Income Countries. Oxford University, Centre on Migration, Policy and Society (COMPAS). Working Paper Series, July 2011.

  • Skaaning, S.-E. (2010) Measuring the rule of law. Political Research Quarterly 63 (2): 449–460.

    Article  Google Scholar 

  • Teorell, J. and Lindstedt, C. (2010) Measuring electoral systems. Political Research Quarterly 63 (2): 434–448.

    Article  Google Scholar 

  • Thielemann, E.R. (2003) Does Policy Matter? On Governments' Attempts to Control Unwanted Migration, The Center of Comparative Immigration Studies, University of California, San Diego. Working Paper no. 112, December 2004.

  • Thränhardt, D. (2008) Einbürgerung Rahmenbedingungen, Motive und Perspektiven des Erwerbs der deutschen Staatsangehörigkeit. Bonn, Germany: Friedrich-Ebert-Stiftung.

    Google Scholar 

  • Tilly, C. (1995) Citizenship, identity and social history. International Review of Social History 40 (3): 1–17.

    Article  Google Scholar 

  • Vink, M. (2011) Naturalisation rates and rejection rates measure different phenomena, and have different problems. In: R. Bauböck and M. Helbling (eds.) Which Indices are Most Useful for Comparing Citizenship Policies? European University Institute, Robert Schuman Centre for Advanced Studies, EUDO Citizenship Observatory. RSCAS Working Paper, pp. 11–14.

  • Vink, M.P. and Bauböck, R. (2013) Citizenship configurations: Analysing the multiple purposes of citizenship regimes in Europe. Comparative European Politics.

  • Waldrauch, H. and Hofinger, C. (1997) An index to measure the legal obstacles to the integration of migrants. New Community 23 (2): 271–285.

    Google Scholar 

  • Weber, M. (1946) From Max Weber: Essays in Sociology. New York: Oxford University Press.

    Google Scholar 

Download references

Acknowledgements

I would like to thank Sara Goodman Wallace, Edward Koning, Ruud Koopmans, Enric Martinez-Herrera, Ines Michalowski, Didier Ruedin, Maarten Vink and the anonymous reviewers for their comments on previous versions of this article.

Author information

Authors and Affiliations

  1. Social Science Research Center Berlin (WZB), Germany

    Marc Helbling

Authors
  1. Marc Helbling
    View author publications

    You can also search for this author in PubMed Google Scholar

Appendix

Appendix

Table A1

Table A1 Data access and time periods for comparison
Full size table

Rights and permissions

Reprints and permissions

About this article

Cite this article

Helbling, M. Validating integration and citizenship policy indices. Comp Eur Polit 11, 555–576 (2013). https://doi.org/10.1057/cep.2013.11

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1057/cep.2013.11

Keywords

Search

Navigation