How League Tables and Rankings Impact on HEIs and HE: Institutional Perspectives

Our position is clearly the second...University in international rankings

Despite growing concerns about technical and methodological issues, there is a strong perception among university leaders that rankings help maintain and build institutional position and reputation, good students use rankings to 'shortlist' university choice, especially at the postgraduate level, and stakeholders use rankings to influence their own decisions about funding, sponsorship and employee recruitment. Benefits are seen to flow directly from high ranking. While many individual countries are developing their own rankings, worldwide rankings have greater penetration and significance. This is true even for countries that do not have national rankings. Accordingly, HE presidents and senior leaders are taking the results of rankings very seriously.

Fifty-eight per cent of respondents are unhappy with their current institutional rank, and 93% and 82% want to improve their national or international position, respectively. Seventy per cent of all respondents want to be in the top 10% nationally, and 71% want to be in the top 25% internationally. While answers depend on the level of 'happiness with their position', almost 50% of respondents use their institutional rank for publicity purposes, either in press releases, official presentations and their website (see example above). Positive rankings generate 'better marketing' and 'support of public opinion', while the converse is also true. A low ranking spawns 'negative publicity due to not being among 500 best world Shanghai-rated' forcing HEIs to 'waste our time on damage limitation'.

Impact on students and other stakeholders

While over 70% of respondents said the primary or original purpose of LTRS was to provide 'comparative information' to students, there is increasing evidence that consumers now include government, industry and other stakeholders, whose decisions are influenced by an institution's reputation (see Table 1). Respondents believe rankings are a critical factor underpinning and informing institutional reputation, and that 'reputation derived from league tables is a critical determinant for [ student] applicants'. While respondents' views vary in the degree to which they think students are 'more attracted to [ a] university because of high ranking', whether it is 'at the margins' only or more widespread, the general consensus is that 'student choice is influenced by ranking'. This is especially true for the international students, 'where status and prestige are considered in decision-making, although students give too much weight to rankings without knowing the methodology'. Parents use rankings as a 'benchmark for judging the best university', and advise their children accordingly.

Table 1 - Perception of how LTRS impact on key stakeholders.

Full table

Institutions believe that high rankings can boost their 'competitive position in relationship to government'. In turn, government and funding agencies are more favourably disposed to highly ranked HEIs, as evidenced through their 'support' and 'commitment', increased 'funding to promote teaching and research excellence' or facilitation of accreditation. Relatedly, governments use rankings to influence and incentivize institutional behaviour, for example, including the 'development contract ...to strengthen the use of performance indicators in the internal budgeting process'. Research funding agencies use rankings to 'distribute the money to universities with better reputation'. High standing both assures and reassures potential sponsors and benefactors, enabling them to associate their own corporate image with success: 'Benefactors don't want to help or be associated with losers, they want their image to be associated with winners only'. In binary systems, respondents claimed that benefactors only fund activities or facilities in universities. Employers also respond positively: 'degree holders from universities with good reputations have better chances to get a job'. New doors can be opened because of good rankings: 'those not open to us become more receptive'.

High-quality future faculty are also more attracted to highly ranked institutions, making 'recruitment easier because of their good reputation' because 'success breeds success'. Organizationally, good rankings have a positive impact on faculty morale, associated with 'pride' and 'honour', and on academic behaviour: increasing 'awareness of the importance of publishing high quality research' and making it 'easier to induce an improvement with a department head whose rankings have been declining'. Conversely, however, respondents complained that poor rankings impacted negatively on morale: 'Fine professors and programs (sic) are short changed having to read a ranking based on doubtfully relevant indicators' while good staff leave because they are disappointed by the rankings.

Respondents believe there is greater interest shown by potential institutional partners, for example willingness and ease to establish partnerships and collaboration with industry and other HEIs. High rankings made it 'easier to present the institution to partners and funders' while another said that high ranking provoked 'more interest from other institutions'. Over 40% of respondents admitted that peer-benchmarking was now integral to their own decision-making about whether to enter international collaboration, academic programmes, research or student exchanges, while 57% said they thought rankings were influencing the willingness of other HEIs to partner with them. Respondents said there was pressure to 'establish contact with reputed universities' while many 'international partners accept only universities above a certain level in rankings'. Critically, 34% of respondents said rankings were influencing the willingness of other HEIs to support their institution's membership of academic or professional organizations.

Institutional leaders claim they have experienced the benefits of high ranking — 'we feel an improvement': recruitment of better students and faculty, additional funding, accreditation, and external support is easier or much improved. While it is less clear that support has actually been withdrawn due to lower ranking, positive benefits are perceived as unlikely to accrue to institutions with poor ranking and poor rankings contribute to weakening an institution's position. 'Recent statistics...weakens [ our university] in the ongoing fusion [ merger] process in the country'. Another said there had 'been negative publicity due to not being among 500 best world Shanghai-rated'. There is concern that rankings present a 'simplistic picture' and perpetrate an 'arbitrary definition of quality', which generates and contributes to public confusion especially 'if rankings do not reflect the real quality difference'. These problems are amplified, when in stakeholders' minds, there appears to be a correlation between rank and quality.

Impact on higher education

At a macro-level, leaders are concerned LTRS are influencing developments in higher education and policy decision-making. Do LTRS aid transparency and accountability, and provide useful information to consumers/students and the general public or are they distorting academic values? Are LTRS helping HEIs set strategic goals and improve quality or encouraging HEIs to refocus their activities and resource allocation in order to gain a better rank — to become what is measured?

The overwhelming majority of respondents said LTRS did not provide a full overview of an institution but tended to favour the strengths of well-established universities, with an emphasis on research and postgraduate activity. In so doing, they helped establish a hierarchy that did little to promote or value institutional diversity or differentiation or represent the complexity of higher education activities. In an era when governments favour greater market-led competition between HEIs, respondents did not agree that LTRS encouraged fair competition, primarily because they are open to 'distortion, inaccuracies and obscurities'. On the question of whether rankings could 'make or break an HEI's reputation', respondents were evenly split.

On the other hand, HEIs think LTRS force/help institutions to be(come) more accountable, set strategic planning goals, and provide comparative information to students, parents and other stakeholders. They provide a methodology — albeit the quality of the methodology is contested — by which institutions can benchmark their own performance and that of other institutions. Yet, unlike a classification system — which categorizes or groups institutions according to type/mission — rankings do not and thus cannot help institutions either identify or align themselves with 'true peers' (Hazelkorn, 2007, 103–104).

Institutional (re)actions

Given the issues described above, how are HEIs responding? The IMHE/IAU study shows that over 56% of respondents have established a formal internal mechanism to review institutional rankings and their own institution's position. Of this group, 56% said rankings were reviewed by the Vice Chancellor, Rector or President, 14% by their Governing Authority, 7% each by their Senate or strategic planning group, and 5% at departmental level. As a result of this process, 63% of respondents say they have taken strategic, organizational, managerial or academic actions (see Table 2). While these 'headings' arguably overlap, they seek to identify different types of decision-making undertaken in response to rankings. Interestingly, only 8% indicated they had not taken any action — which suggests growing impact over time compared to the aforementioned US 2001 survey.

Table 2 - Actions taken by HEIs in response to LTRS.

Full table

Institutional leaders confirm they take rankings seriously, embedding them within their strategic planning processes at all levels of the organization, including Governing Authority, Senior Executive and School/College. Depending upon the institutional strategic objective vis-à-vis their current position, HEIs use ranking metrics to guide their own goals. This may mean setting student and faculty recruitment targets (e.g. specifying academic entry criteria, making conditions of appointment/promotion clearer and more transparent, appointing Nobel prize winners), indicating individual academic performance measurements (e.g. research activity and peer-review publications, programme development), setting school/college level targets, and/or continual benchmarking exercises. As one respondent stated: 'the improvement of the results has become a target in the contract between presidency and departments', while another confirmed they have 'developed a set of internal research output indicators...we do internal benchmarking'.

Some HEIs have restructured departments, invested in their organization's facilities or improved 'awareness and expertise in the Research and Innovation Office'. Many HEIs have established an institutional research office to collect data, monitor their performance, better present their own data in public or other official realms, and benchmark their peer's performance. Others have taken a more aggressive approach, using rankings as a tool to influence not just organizational change but influence institutional priorities. In this respect, both teaching programmes and research are mentioned. Respondents spoke of using ranking 'to drive activities at university, faculty and campus levels' and 'for internal budgeting'.

Arguably, the above examples indicate that rankings are having the desired public policy effect by encouraging HEIs to 'best the system' — in other words, to improve their ranking position by influencing the input metrics used by most ranking organizations. Other HEIs see it differently; they are effectively developing a survival strategy, which involves either consideration of 'unhelpful merger proposals' or spending substantial sums of money to 'bolster demand in key overseas markets to counter league tables'. Overall, their emphasis is on restoring internal morale and public confidence: 'spend time...restoring our damaged feelings'. For many, the focus is on better marketing and publicity, including the website, to 'inform [ others] about our strength' in light of the impact that LTRS are having on the general perception of the quality of higher education and particular institutions.

HE leaders readily acknowledge that perceptions of the impact — whether positive or perverse — is dependent upon institutional position and the level of 'happiness' with their position. Thus, respondents spoke of: Decent rankings may help raise/reassure awareness of institution/department/program and help support their activities.
Foreign universities are interested in the fact that we are one of the three best private universities in our country.
Installation of a privately funded department of real estate management by a benefactor/sponsor in response to rankings.

On the other hand, if an institution's ranking is considered poor, then there is likely to be an accumulation of negativity: Denial of collaboration because of a bad position in the Shanghai Ranking.
Local newspapers write that local government should not spend more money for our university.
Decline in enrolment.

What Other Evidence Tells Us about the Impact and Influence of LTRS on HEIs

You should hold a degree from a Times top 100 university ranked at no. 33 or higher. (Anon)

While there may be a distinction between perception and reality, evidence suggests the 'perception' held by international HEIs has much validity. This section refers to some of this other evidence from the academic literature and journalistic reportage in order to contextualize and assess their responses. That evidence concurs with the institutions' view that, in addition to students, there is a growing group of interested stakeholders who refer to and use — in varying but arguably increasing degrees — the results of LTRS to inform their own decision-making processes. Dichev (2001, 237) estimates the total audience for the special issue of USNWR is approximately 11 million people. Stakeholders include a wide 'non-consumer audience' who account for sales of 40% more than the traditional prospective student cohort market. Comprising 'state governments who are directly responsible for allocating funding to public institutions', alumni who have an interest and may be potential benefactors, and the voting public, these audiences have a direct influence on the amount of financial resources allocated to colleges, and their preferences may be reflected in tuition policy, admission criteria, the profile of the faculty, and the campus activities of a college. (Zhe et al., 2007, 2)

Impact on student choice

Because rankings were initially conceived and have continued to be promulgated as a critical source of transparent consumer information for students and their parents, this has been one of the main areas of contention. There is little doubt that the attractiveness of rankings is their simple, easy-to-understand format. They provide a fast, short-hand Q-mark, enabling the user to 'pre-sort' a group of HEIs prior to more in-depth inquiry (Contreras, 2007; http://www.insidehighered.com/views/2007/07/31/contreras). Yet, do rankings measure what users think they are measuring? Do they provide the kind of information and indicators that would be most helpful to students and their parents? Are students using rankings to help inform their decision-making and to what effect? The evidence to date is limited but trends are apparent.

Rankings are important for the vital and potentially lucrative international recruitment market, especially for mobile high-achieving postgraduate students and particular disciplines such as law, medicine and business (Berger, 2001; Sauder and Lancaster, 2006; Wedlin, 2006). A recent UK study confirmed that 92% of international students considered UK league tables important/very important to inform their choice, although the results seemed less conclusive for Australia. (Roberts and Thompson, 2007, 5, 18–20; http://www.theknowledgepartnership.com/
docsandpdf/leaguetablepressrelease.pdf). Relatedly, institutional ranking is a decisive factor for students seeking government sponsorship/scholarship to study abroad; governments use rankings as an indicator of 'value-for-money' (Clarke, 2007, 43). According to Salmi and Saroyan (2007, 52), scholarships in Mongolia and Qatar are restricted to students admitted to highly ranked international universities.

But rankings are also vital for the domestic undergraduate student market. In the US, rankings are important for high-ability and second-generation students, and students from Asian backgrounds (Hossler and Foley, 1995, 29; McDonough et al., 1998; Monks and Ehrenberg, 1999; Ehrenberg, 2004; Griffith and Rask, 2007). Spies (1978) argues that above-average students make choices based on non-financial factors, such as reputation. Students who have the financial ability to pay full fees — who are effectively free to choose —, and not reliant on government or other grants, are more likely to attend higher ranked colleges (even by a few places) than grant-aided students who appear to be less responsive to rankings. McDonough et al. (1998) says that while only 40% of US students use newsmagazine rankings, 11% said rankings were an important factor in their choice. This group may be small, but it is the vital high achieving group which HEIs and governments are keen to attract.

Research indicates strengthening usage among lower-income groups (McManus quoted in Roberts and Thompson, 2007, 18), but elite responsiveness among students and parents remains most significant (Machung, 1998). Attendance at the most select universities and colleges is seen to 'confer extra economic advantages to students, in the form of higher early career earnings and higher probabilities of being admitted to the best graduate and professional schools', albeit this may be more for 'under represented minority students and students from low-income families' (Ehrenberg, 2004). It also confers indirect benefits, such as connections to 'elites' and future decision-makers, membership of 'the right' social and golf clubs and schools, etc. Accordingly, there is some evidence that students have 'tried to increase the standing of their program in satisfaction-based rankings by sending back surprisingly upbeat surveys' (Clarke, 2007). In addition to 'exacerbat[ ing] the competition between institutions for top students...research indicates that applicant behaviour is very much conditioned by rankings' (Ehrenberg, 2004, 26); slight changes can 'cause perceptible ebbs and flows in the number and quality of applicants' (Dichev, 2001, 238; Sauder and Lancaster, 2006, 116), especially international students (Honan, 1995; Roberts and Thompson, 2007, 22; http://www.theknowledgepartnership.com/
docsandpdf/leaguetablepressrelease.pdf), albeit Schmalbeck (1998) suggests institutional reputation may be resilient to small or annual changes.

International research, however limited, supports the US experience. Overall, 61% of UK students referred to LTRS before making their choice, and 70% considered they were important/very important (Roberts and Thompson, 2007, 20; http://www.theknowledgepartnership.com/
docsandpdf/leaguetablepressrelease.pdf; see also Rolfe, 2003, 32–33). The Centre for Higher Education Development in Germany (CHE), which has been operating a student information system for several years, says 60% of prospective students 'know rankings and use rankings as one source of information among others' (Federkeil, 2007). Students taking professional focused programmes are more likely to use such information in contrast to students taking a traditional 'academic' programme. Clarke (2007) cites UK, German and New Zealand experiences that high-achieving students are more likely to use rankings to inform choice and high ranking leads to increased applications. While there was no evidence that lower ranked universities lose students, high ranking does have a positive impact on application numbers. Where a binary system exists, there is some evidence that ranking is accelerating social selectivity by sector (see Sauder and Lancaster, 2006, 122–124). For example, Ireland has witnessed strong migration out of its institute of technology sector in favour of universities as demographics change, and competition escalates the reputation race (Clancy, 2001, 56–57 and Table 17; Fitzgerald, 2006).

Most attention has focused generically on student choice and behaviour. There is, however, a need to distinguish between undergraduate and postgraduate students. The latter is likely to be more responsive to worldwide rankings given their maturity, career focus and capacity for mobility, in addition to increasing national and institutional anxiety to recruit these lucrative students who can also shore up national research and economic development strategies (Kallio, 1995).

Impact on recruitment, marketing and reputation

University administrators are 'most engaged and obsessively implicated' (Keller, 2007) with the collection of data used for rankings, and their aftermath. They are effectively 'caught between not wanting to place public emphasis on their ranking...and privately trying to avoid slipping' (Griffith and Rask, 2007). Both reactions are understandable given mounting evidence that rankings do impact on student numbers, quality of applicants, and institutional reputation, and that 'changes in rank have a significant influence on the applications and enrolment decisions of students...' (Monks and Ehrenberg, 1999, 10; see Wedlin, 2006; Roberts and Thompson, 2007). But is it the objective to increase student numbers or only 'recruit students who will be "assets" in terms of maintaining and enhancing their position in the rankings?' (Clarke, 2007, 38).

Monks and Ehrenberg (1999, 10; also Ehrenberg, 2001, 2) produce evidence of a direct correlation between high ranking and more applications. An institution whose rank improves can accept a smaller percentage of its applicants and thereby increase its selectivity — a metric used by USNWR and The Sunday Times. On the other hand, 'a less favourable rank leads an institution to accept a greater percentage of its applicants, [ leading to] a smaller percentage of its admitted applicants [ who] matriculate, and the resulting entering class is of lower quality, as measured by its average SAT [ college entry] scores' — and the circles repeats itself, leading to a downward spiral in terms of ranking position. They suggest that 'the growing popularity and influence of these rankings may also lead the institution to try to influence them'. Because the selectivity index is a key metric in the US, institutions have sought to influence the number of applicants it receives while still retaining the same number of available places. This may have the knock-on effect of 'creating a new second tier of elite institutions' benefitting from the 'overflow' and obsession with 'elite' institutions (Samuelson, 2006; Finder, 2007). It may also force HEIs to abandon distinctive missions — such as access programmes — that are not measured in rankings (Espeland and Sauder, 2007, 15).

Private institutions are better able to respond to ranking pressure, given their ability to 'adjust net tuition in response to changes in USNWR rank' (Meredith, 2004, 460). They are better able to use mechanisms, such as financial aid and investments, to influence 'student input' metrics (Brewer et al., 2002). They may also admit students on probationary or part-time basis so their (relatively) lower entrance scores will not be included in official data returns or discourage others (Ehrenberg, 2001, 7). Or, they may step-up their publicity and marketing efforts to encourage more students to apply. Other methods include using merit aid to 'purchase talent' or invest in 'image-enhancing facelifts', such as dormitories, fiber optic networks and sports facilities. Too many institutions now spend their resources aggressively recruiting students with high SAT or ACT scores and other conventional markets of achievement that correlate strongly with socioeconomic status. In turn, at many institutions those choices skew the allocation of financial aid from students with the great need to those with the most offers of admission. (Lovett, 2005)

Some of these actions are admirable but the effect may be to reduce the 'resources available for other activities, including those designed to recruit and retain students from traditionally underrepresented groups' (Clarke, 2007, 38; see also Meredith, 2004; Stake, 2006).

While European HEIs have not heretofore had a history of such investment, there are increasing reports that these 'added-value' facilities and sport scholarships are a critical factor in institutional marketing and strategic development, and national and international recruitment. Selectivity indices have also not been a significant element of other national or worldwide rankings, especially in Europe where equity and open recruitment has tended to be the norm. Nonetheless, even in systems, such as in Ireland, where student admissions are effectively 'blind' to subjective factors, there are suggestions HEIs have endeavoured to influence the process.

Impact on other stakeholders

There appears to be less hard evidence on how stakeholders — government/policymakers, funding agencies, employers, and sponsors — are, positively or perversely, being influenced by LTRS, but again a pattern is emerging (see Meredith, 2004, 457–458). Employers have long recognized the advantages of recruitment from specified institutions; the 'milk run' is a date in many large corporations' diary for graduate recruitment. US accounts claim law firms regularly use USNWR rankings to 'determine the threshold for interviews' (Espeland and Sauder, 2007, 19). A recent study by the University of Sussex for the Higher Education Funding Council for England (HEFCE) says employers rely strongly on institutional reputation gained via rankings in the Times Higher Education Supplement or implicit knowledge: 25% of graduate recruiters interviewed 'cited league tables as their main source of information about quality and standards' (University of Sussex, 2006, 87, 80, also 87–92; http://www.hefce.ac.uk/pubs/rdreports/2006/rd20_06/): It tends to be knowledge that we've gained over the years but I guess in terms of hard data we use The Times Higher Education...
We would look at The Sunday Times league table. And then just, I think, general opinion,...

While graduate success in the job market does reflect 'traditional status hierarchies', there is evidence that graduates of particular universities do especially well (Clarke, 2007). Employers use league tables as a method of pre-selection, targeting graduates of the same top 10 or 20 universities: '... as long as we keep taking graduates from those universities and those graduates come into the business and perform exceptionally well, there is no reason to change' (University of Sussex, 2006, 87).

Governments are similarly influenced, and acutely conscious of how rankings can be used to drive institutional behaviour (Baly, 2007). Ministers and policy directors refer to their institutions' 'world class' excellence as a statement of national pride and attractive selling-point to encourage inward investment. Not surprisingly, the Malaysian government felt compelled to establish a Royal Commission of Inquiry to investigate why the rankings of its two top universities fell by almost 100 places within a year (Salmi and Saroyan, 2007, 40; see also recent reaction in Australia, http://blogs.smh.com.au/newsblog/archives/016441.html). Ironically, poor rankings can also incentivize a government to spend more financial resources on poorer ranked institutions (Zhe et al., 2007, 22). Spiewak (quoted in van Raan, 2007, 94) reports that Deutsche Telekom used rankings to help select the location for professorial chairs. While formal rankings may not be used, private philanthropy in Ireland has gone, with minor exceptions, only to the university sector. Monks and Ehrenberg (1999, 6) similarly notes the correlation between high rankings and/or an improvement in position and increases in endowment per student.

Governing Boards and alumni are not immune either. Writing on the 'uses and abuses' of rankings, Levin (2002, 1) remarks: For many trustees, alumni, parents, and others, a high ranking means bragging rights on the golf course or grocery line — an ephemeral pleasure similar to when the football or basketball team wins a national ranking.

His study (2002, 12, 15) shows that 75% of university presidents believe their boards think they need to pay attention to USNWR rankings 'whether it likes it or not'; 36% felt rankings were important to attract students; 34% felt rankings were helpful for benchmarking; 33% said they misled students and their families; and 20% said they were worthwhile. Not surprisingly, 68% of boards did discuss rankings, with 71% doing so for half an hour or more. Honan (1995) reports that alumni from Hamilton College, NY, anxiously queried 'what was wrong after it fell out of the top 25 national liberal arts colleges in the USNWR rankings'.

These examples also illustrate how public opinion can be a vital agenda-setting media through which reputation is both reified and reinforced — even among those who themselves may not have been HE graduates (see Hossler, 1998, 164). Zhe et al. (2007, 19) argue that because 'state funding and tuition prices at public colleges are topics frequently discussed in many types of media, including newspapers, magazines, TV and radio, the public is increasingly sensitized to the issues', in addition to which rankings 'provide a focal point for the public to pay attention to higher education'.

Impact on institutional behaviour

HEIs are learning to (1) reap the benefits, (2) adjust/alter their institutional behaviour in order to better reap the benefits or (2) try to ignore all the fuss — often experiencing all three forms of 'reactivity' in reverse order. When rankings were introduced most administrators dismissed them...Over time, law schools learned that rankings were fateful, that people made important decisions using rankings, and schools began to invest heavily in improving rankings. This reinforced rankings' impact and legitimacy.... (Espeland and Sauder, 2007, 23–24) Monks and Ehrenberg (1999; see also Wedlin, 2006) outline a wide range of measures taken by HEIs, 'at all places in the selectivity game' (Ehrenberg, 2001, 4), to influence their rank, 'correcting their data in a way that leads to an improvement in their ranking, or by devoting resources to activities related to improving their rank that do not directly enhance educational quality'. Senior HE 'administrators consider rankings when they define goals, assess progress, evaluate peers, admit students, recruit faculty, distribute scholarships, conduct placement surveys, adopt new programs and create budget'. Whichever action they choose, 'rankings are always in the back of everybody's head' (Espeland and Sauder, 2007, 11). In many instances, consultants are hired to provide guidance.

There may be little public acknowledgement by HEIs that they have purposely designed a strategy to improve their rankings, but there is growing evidence to the contrary. Levin (2002, 6–7) recounts the 'case study' of Virginia Commonwealth University whose board adopted a goal to move from Tier 3 to Tier 2, and 'to develop a plan to make it happen'. Its institutional research office spent six months gathering data, and a vice president for institutional outreach was hired and put in charge of admissions, marketing and communications. Nine strategies were adopted including improving academic programmes, developing a marketing plan, enhancing publications and the website, increasing the proportion of faculty with terminal degrees, and increasing the proportion of full-time faculty. Similar to a recent offer by the board of Arizona State University, 'for every year VCU is ranked in Tier 2, the board has promised him a $25,000 bonus' (Chronicle of Higher Education, 25 May 2007; East Valley Tribune, 18 March 2007).

A memorandum to the board of a US highly ranked doctoral university detailed particular strategies to be adopted in order to achieve single digit ranking: these include (1) spend more money per student; (2) double the annual private fund-raising; (3) increase public funding; and (4) increase the endowment. According to Levin, university presidents have highlighted specific metrics for improvement, inter alia: 88% identified retention rates; 84% alumni-giving; 75% graduation rates; 71% entry scores; 63% faculty compensation; and 31% student-faculty ratio. More than 25% of presidents sought to improve educational expenditure, by effecting greater selectivity, increasing faculty salaries, creating new and better programmes, improving funding and use of resources, changing the hiring or promotional procedures and improving marketing. While only 7% mentioned improving research capacity, others recorded a shift in resources from teaching to research, marketing or merit scholarships (Espeland and Sauder, 2007, 25–27).

Impact on higher education

All commentators acknowledge that rankings must now be considered part of the fabric of higher education. For many, the difficulties with rankings are, as Altbach (2006; http://www.bc.edu/bc_org/avp/soe/cihe/newsletter/
Number42/p2_ Altbach.htm) states, not with the principle but with its practice. Because of the close correlation between rankings and reputation (see University of Sussex, 2006, 119; Espeland and Sauder, 2007, 14; Roberts and Thompson, 2007, 4), HEIs are looking more and more closely and strategically on how to improve their position. For many, this means focusing on the critical input indicators — but at what cost?

For example, focusing on student selectivity could undermine other educational and national objectives, such as widening access. 'Miles College in Alabama and Jackson State U in Mississippi both score low in USNWR rankings in part because they serve many financially needy students, who score lower on standardized tests than their more privileged peers do, and because the two institutions spend relatively little on each student'. However, if a different set of measurements is used, such as the US National Survey of Student Engagement, both score well above national average in several key categories, for example frequency of outside-the-classroom discussions, and promptness of feedback to students (Chronicle of Higher Education, 22 September 2006). Because there is little genuine movement year to year (Dichev, 2001) and '...certain institutions or types of institutions...rise to the top regardless of the specific indicators and weightings' (Usher and Savino, 2006, 32), to realistically effect any change requires vast financial investment. In this climate, public HEIs, especially those dependent on the public purse, have an impossible task competing. 'Existing input metrics favor (sic) private institutions, which are already well-endowed, over public ones' (Farrell and van der Werf, 2007).

Rankings inflate the academic 'arms race' locking HEIs into a continual 'quest for ever increasing resources' (Ehrenberg, 2004, 26), reinforcing the 'effects of market-based and competitive forces' (Clarke, 2007, 36), 'intensif[ ying] competitive pressures' and creating a 'global market' which places a 'growing emphasis on institutional stratification and research concentration' and establishes a worldwide norm for a 'good university' (Marginson, 2007, 132, 136). Universities which do not meet the criteria or do not have 'brand recognition' are effectively devalued or ignored (Machung, 1998, 13; Lovett, 2005). This is further promulgated by the fact that US community colleges, which comprise 45% of US undergraduate students, are not included in the USNWR rankings, while the CHE rankings list universities and fachhochschulen/universities of applied sciences separately (http://www.daad.de/deutschland/hochschulen/
hochschulranking/06543.en.html).

Others argue it is not all doom and gloom. Salmi and Saroyan (2007, 53) say rankings can be used in a constructive way, forming a vital part of institutional strategic planning and development. If rankings can prompt a retrospective analysis of institutional performance, leading to setting goals to support institutional and national visions, then they can be considered as having a positive impact towards improvement.

In a similar view, Samuelson (2004) argues that there is an important leveling process going on: the old elite...suffer because they can't accommodate everyone who's qualified...More good students and faculty must go elsewhere.... The new elite have gained more than the old have slipped. Ivy League schools, for example still dominate Rhodes scholarships, but less so. In the 1960s, their students won 39 percent; in this decade, that share is 27 per cent.

Academic improvements and campus developments are occurring because HEIs are using rankings to benchmark their performance and facilities, and engage in continued quality assurance.

What's Next?

League tables and rankings have gained popularity because they (appear to) fulfil particular purposes and needs. While initially many concerns were easily ignored or shrugged off with reference to either the individual institution's (poor) score or broader policy objectives (e.g. accountability, transparency, benchmarking and strategic planning), there is now increasing realization and evidence that LTRS are having an (unintentional) impact and influence on higher education institutions and higher education broadly. It is clear that they are equated in the minds of students, their parents and other key stakeholders with quality, and are now a significant factor shaping institutional reputation. A high rank is perceived as better, and to be placed in the top 100 worldwide is considered a strategic ambition for many governments and institutions. In this respect, there is increasing evidence that rankings are being used by governments as a policy instrument, a tool for speeding up reform of higher education, the creation of an elite group in different countries (e.g. Russia and Germany) similar to the UK Russell Group, forcing/encouraging mergers, etc. Some of this emanates from confusing rankings with evaluation and accreditation. Indeed, the speed of reform is likely to quicken as governments believe it will lead to more competitive and better (more highly ranked) HEIs. On the other hand, institutional leaders and governing authorities are using rankings as a management tool, to enforce change and internal restructuring, resource allocation, changes in academic practice, etc.

The extent to which all these changes can be rated as either positive or perverse is still debatable, but it is clear that rankings are a manifestation of the already competitive global market in which higher education operates, and are being used and perceived as such. Rankings have placed a new premium on status and elite institutions, reinforcing reputation and vice-versa, with a strong bias towards long-established and well-endowed institutions. Those who can afford to do well in this marketplace will benefit, while others may find their current position eroding or just plainly ignored. In policy and funding regimes which resource HEIs wholly or disproportionately through the public purse, there is a growing public intolerance for increases in public expenditure. Those institutions are coming under greater pressure to reduce costs and improve efficiencies, often translated into reducing unit cost. In these circumstances, it will be difficult to positively impact on the student–faculty ratio — one of the key input metrics — thus forcing greater institutional and sectoral differentiation according to social selectivity/stratification. The formation of worldwide networks and consortia of top elite HEIs [ e.g. Universitas 21, Coimbra Group, League of European Research Universities (LERU), Worldwide Universities Network (WUN), and International Alliance of Research Universities (IARU)] is also likely to further inflate the gap between elite and mass education as institutions use the rankings as a guide for partnership and collaboration, and governments form closer alliances with these cartels. Despite calls for greater inter-institutional collaboration, elite institutions are unlikely to see benefit in working with or helping 'lesser' institutions.

A world-class university is $1 billion–$1.5 billion-a-year operation, plus an additional $500 million if there is a medical school. This would require many HEIs increasing their overall funding by at least 40% (Usher, 2006, http://www.theglobeandmail.com
/servlet/story/RTGAM.20061030.
URCworldclassp28/BNStory/univreport06/home; Sadlak and Liu, 2007, 20). Few societies or (public) institutions can afford this level of investment, without sacrificing other social and economic objectives, such as widening access, institutional diversity, community partnerships, cross-institutional collaboration and resource sharing, and knowledge transfer (see Ehrenberg, 2001, 17–20). Developing countries and new HEIs are most vulnerable in this environment — as the kinds of metrics which propel institutions to the top are far behind achievement; thus, the wedge between elite and mass education is likely to be replicated on a global scale. The Matthew Effect² will become increasingly obvious, as HE is arguably restructured for the benefit of elite high-achieving students and their institutions.

What's next? Because of various factors identified above, different countries, regions, organizations and institutional groups are understandably endeavouring to establish their own counter-ranking systems. While this illustrates a positive determination to become engaged in the debate and process, this balkanization of rankings is counter-productive because worldwide rankings have established credibility with the major stakeholders that needs an appropriate response to be sustainable. How should quality be defined and measured, by whom and for what purpose? According to Levin (2002, 15), 77% of US presidents said better criteria should be developed to measure educational outcomes, and USNWR should stop measuring how much an institution spends or measuring reputation. Additional metrics might include accomplishments of graduates, student satisfaction, postgraduate training, and graduate employment rates.

Similar to the US study, IMHE/IAU respondents did not suggest any radical departure from existing metrics (Hazelkorn, 2007). Asked to chose from a range of commonly used indicators, respondents gave low 'marks' only to some, each of which is explicable by the fact that they are relevant to or beneficial to relatively few or specialist HEIs: alumni or private giving, investment, Nobel or similar prizes, and exhibitions and performances. Those metrics which received the majority of support (respondents could indicate all that apply) were teaching quality, student–faculty ratio, graduate employment, research (including publications, citations and income), Ph.D. students, finance, student life, selectivity, mission and the library. Rankings should be conducted by independent research organizations or accreditation agencies, or by non-governmental or international organizations — not by media organizations. Ideally, respondents favour institutional or publicly available data or that which has been gathered by questionnaires. Less than 20% favour peer review, which compares favourably with the US study which also found strong dislike for reputational questions. Despite criticism about the difficulty comparing whole institutions with different missions, 30% of respondents favour institutional reviews rather than programme or departmental level (21% respectively). Ultimately, the objective should be to enable student choice, provide accountability and enhance quality while giving a 'fair and unbiased picture of the strengths and weaknesses of a university'.

This paper set out to understand how LTRS are impacting on and influencing institutional behaviour and decision-making. The IMHE/IAU study provides the first wide-angle view as to what is happening, and how institutional leaders are responding. Their experience — as the 'foot soldiers' of this new HE world — is replicated throughout the literature, hence the gap between their perception and the reality is actually quite narrow. While most research on the impact and influence of LTRS is drawn from the US, there is much to learn from it. Indeed, the international experience differs only in detail. It is also clear that while universities, policymakers and stakeholders criticize and lampoon league tables and rankings, few can afford to ignore them — and most have incorporated them in some fashion into their strategic thinking if not their planning. It is, however, worth considering and reflecting on how an arguably innocuous consumer concept, promulgated by various media organizations, has been transformed into a policy instrument with wide ranging consequences for higher education.

Notes

¹ An earlier version of this paper was presented to the Higher Education in the 21st Century — Diversity of Missions conference (Dublin, 2007) and 3rd Meeting of the International Ranking Expert Group (Shanghai, 2007). Special thanks are extended to Richard Yelland, OECD/IMHE, and Eva Egron-Polak, IAU, who supported the research, and Amanda Moynihan, my research assistant.

² The 'Matthew Effect' is based on a line in St. Matthew's Gospel that says, 'For unto every one that hath shall be given, and he shall have abundance: but from him that hath not shall be taken away even that which he hath' (Matthew 25:29). This line has often been summarized as: 'The rich get richer, and the poor get poorer'.

References

1. Altbach, P.G. (2006) 'The dilemmas of ranking', International Higher Education 42(Winter).
2. Baly, A. (2007) 'What can we learn from Australia about ranking universities?', Presentation to Ranking Universities for Students: Pilot Project Results Conference, Brussels.
3. Berger, M. (2001) 'Why the USNWR law school rankings are both useful and important', Journal of Legal Education 51(4): 487–502.
4. Bowden, R. (2000) 'Fantasy higher education: university and college league tables', Quality in Higher Education 6(1): 41–60. | Article |
5. Brewer, D.J., Gates, S.M. and Goldman, C.A. (2002) In Pursuit of Prestige. Strategy and Competition in U.S. Higher Education, New Brunswick: Transaction Publishers.
6. Clancy, P. (2001) College Entry in Focus. A Fourth National Survey of Access to Higher Education, Ireland: Higher Education Authority.
7. Clarke, M. (2007) 'The Impact of Higher Education Rankings on Student Access, Choice and Opportunity', in IHEP (ed.) College and University Ranking Systems: Global Perspectives and American Challenges, Washington, DC: Institute for Higher Education Policy, pp. 35–47.
8. Contreras, A. (2007) 'The cult of speed', Inside Higher Education, 31 July.
9. Dichev, I. (2001) 'News or noise? Estimating the noise in the US News University rankings', Research in Higher Education 42: 237–266. | Article |
10. Dill, D.D. and Soo, M. (2005) 'Academic quality, league tables and public policy: a cross-national analysis of university ranking systems', Higher Education 49: 495–533. | Article |
11. Egron-Polak, E. (2007) 'Editorial', IAU Horizons, May.
12. Ehrenberg, R.G. (2001) 'Reaching for the Brass Ring: how the USNWR rankings shape the competitive environment in US higher education', Paper prepared for Macalester Forum on Higher Education.
13. Ehrenberg, R.G. (2004) 'Econometric studies of higher education', Journal of Econometrics 121: 19–37. | Article |
14. Espeland, W.N. and Sauder, M. (2007) 'Rankings and reactivity: how public measures recreate social worlds', American Journal of Sociology 113(1): 1–40. | Article |
15. Farrell, E.F. and van der Werf, M. (2007) 'Playing the rankings game', Chronicle of Higher Education, 25 May.
16. Federkeil, G. (2007) 'Rankings and quality assurance in higher education', Presentation to 3rd Meeting of the International Ranking Expert Group (IREG 3), Shanghai.
17. Finder, A. (2007) 'Ivy League admissions crunch brings new cachet to next tier', New York Times, 16 May.
18. Fitzgerald, G. (2006) 'ITs [ Institutes of Technology] suffer from change in student social base', The Irish Times, 11 March.
19. Griffith, A. and Rask, K. (2007) 'The influence of the USNWR collegiate rankings on the matriculation decision of high-ability students: 1995–2004', Economics of Education Review 26(2): 44–255. | Article |
20. Hazelkorn, E. (2006) 'Impact and influence of league tables and ranking systems on institutional decision-making', Presentation to the HRK/IMHE Symposium on Institutional Diversity: Rankings and Typologies in Higher Education, Bonn, December.
21. Hazelkorn, E. (2007) 'Impact and influence of league tables and ranking systems on higher education decision-making', Higher Education Management and Policy 19(2): 87–110.
22. Honan, J.P. (1995) 'Monitoring institutional performance', Priorities 5(Fall).
23. Hossler, D. (1998) 'Everybody wants to be Number One: The Effects of the Media's College Rankings', in G.I. Maeroff (ed.) Imaging Education: The Media and Schools in America, New York: Teachers College Press, pp. 161–177.
24. Hossler, D. and Foley, E.M. (1995) 'Reducing the Noise in the College Choice Process: The Use of College Guidebooks and Ratings', in R. Dan Walleri and M.K. Moss (eds.) Evaluating and Responding to College Guidebooks and Rankings, San Francisco: Jossey-Bass Publishers, pp. 21–30.
25. Kallio, R.E. (1995) 'Factors influencing the college choice decisions of graduate students', Research in Higher Education 36(1): 109–124. | Article |
26. Keller, T. (2007) 'Truth, lies and rankings', Inside Higher Education, 12 April.
27. Levin, D.J. (2002) 'The uses and abuses of the US news rankings', Priorities 20(Fall/Autumn).
28. Lovett, C. (2005) 'The perils of pursuing prestige', Chronicle of Higher Education, 21 January.
29. Machung, A. (1998) 'Playing the rankings game', Change (July/August): 12–16.
30. Marginson, S. (2007) 'Global university rankings: implications in general and for Australia', Journal of Higher Education Policy and Management 29(2): 131–142. | Article |
31. McDonough, P.M., Antonio, A.L., Walpole, M. and Pérez, L.X. (1998) 'College rankings: democratized college knowledge for whom?', Research in Higher Education 39(5): 513–537. | Article |
32. Meredith, M. (2004) 'Why do universities compete in the ratings game? An empirical analysis of the effects of the US News and World Report college rankings', Research in Higher Education 45(5): 443–461. | Article |
33. Monks, J. and Ehrenberg, R.G. (1999) 'U.S. News & World Report's college rankings: why they do matter', Change 31(6): 43–51.
34. Roberts, D. and Thompson, L. (2007) 'University League Tables and The Impact on Student Recruitment, Reputation Management for Universities', Working Paper Series No. 2. Leeds, Cambridge and Brisbane: The Knowledge Partnership.
35. Rolfe, H. (2003) 'University strategy in an age of uncertainty: the effect of higher education funding on old and new universities', Higher Education Quarterly 57(1): 24–47. | Article |
36. Sadlak, J. and Liu, N.C. (2007) 'Introduction to the Topic: Expectations and Realities of World-Class University Status and Ranking Practices', in J. Sadlak and N.C. Liu (eds.) The World-Class University and Ranking: Aiming Beyond Status, UNESCO-CEPES and Shanghai Jiao Tong University: Cluj University Press, pp. 17–24.
37. Salmi, J. and Saroyan, A. (2007) 'League tables as policy instruments: uses and misuses', Higher Education Management and Policy 19(2): 31–68.
38. Samuelson, R.J. (2004) 'In praise of rankings. Demographics and fierce competition have created a new class of elite schools. That turns out to be a good thing', Newsweek, 8 September.
39. Sauder, M. and Lancaster, R. (2006) 'Do rankings matters? The effects of US News & World report rankings on the admissions process of law schools', Law & Society Review 40(1): 105–134. | Article |
40. Schmalbeck, R. (1998) 'The durability of law school reputation', Journal of Legal Education 48(4): 568–590.
41. Spies, R.R. (1978) The Effect of Rising Costs on College Choice. A Study of the Application Decisions of High-Ability Students, Princeton, NJ: College Board Publication Orders.
42. Stake, J.E. (2006) 'The interplay between law school rankings, reputations, and resource allocation: ways rankings mislead', Indiana Law Journal 81(Winter): 229–269.
43. Turner, D. (2005) 'Benchmarking in universities: league tables revisited', Oxford Review of Education 31(3): 353–371. | Article | ISI |
44. University of Sussex School of Education. (2006) 'Needs of employers and related organisations for information about quality and standards of higher education', Report to HEFCE, UK.
45. Usher, A. (2006) 'Can our schools become world-class', The Globe and Mail, 30 October.
46. Usher, A. and Savino, M. (2006) 'A world of difference: a global survey of university league tables', Canadian Education Report Series. Toronto: Educational Policy Institute.
47. van Raan, A.F.J. (2007) 'Challenges in the Ranking of Universities', in J. Sadlak and N.C. Liu (eds.) The World-Class University and Ranking: Aiming Beyond Status, UNESCO-CEPES and Shanghai Jiao Tong University: Cluj University Press, pp. 87–121.
48. Wedlin, L. (2006) Ranking Business Schools: Forming Fields, Identities and Boundaries in International Management Education, Cheltenham Glos: UK Edward Elgar.
49. Zhe, J., Whalley, G. and Whalley, A. (2007) 'The power of information: how do U.S. News rankings affect the financial resources of public colleges?', Working Paper 12941, National Bureau of Economic Research, http://www.nber.org/papers/w12941.