Introduction

The European Society for Opinion and Market Research estimates that the use of mail surveys in ad hoc market research in the UK has increased from 5% in 1999 to 7% in 2003 (Kotler et al, 2001; ESOMAR, 2005). The importance of the technique continues to promote a steady flow of research into methods of achieving higher response rates to mail surveys in order to reduce survey costs and minimize the attendant problems of inadequate sample sizes and non-response bias (Cycyota and Harrison, 2002; Edwards et al, 2002; Erwin and Wheelright, 2002; Helgeson et al, 2002; Larson and Chow, 2003; Braunsberger et al, 2005; Díaz de Rada, 2005; Larson and Poist, 2004). In particular, prepaid monetary incentives have been studied and relatively consistent positive results have been reported in both consumer and organisational surveys (see Tables 1 and 2). Cash incentives may also increase respondents' favourable attitudes towards similar surveys, willingness to participate in a follow-up study (Singer et al, 1999) and speed of return (Gajaraj et al, 1990; Brennan, 1992) without necessarily biasing respondents (Weathers et al, 1993). However, this method can add significant cost to a project and there have been few systematic attempts to model the effects of different incentive sizes on response rates over a number of studies. One exception, a study by Jobber and Saunders (1988), was limited by a small sample size and the failure to distinguish between consumer and organizational populations. From a meta-analysis, Church (1993) found that the greater incentive value, the greater the increase in response, but the relationship may not be linear and called for further research on different incentive amounts. Thus, little is known about the relationship between the size of the incentive and the size of the increased response. For example, in the United States, a 25¢ (estimated value 89.9¢ in 2005) yielded a substantial increase in response, whereas larger amounts encourage little more response (Moser and Kalton, 1976), while Armstrong (1975) suggests 40¢ ($1.57 in 2005) is the optimum amount (current values based on Sahr (2005)). Furthermore, we do not know whether this relationship will be the same in widely different populations, for example, between consumers and organizational respondents, as there is some evidence of differential inducement effects being observed (Jobber, 1986; Dennis, 2003; Cycyota and Harrison, 2002; Roth and BeVier, 1998; Summers and Price, 1997). Professional respondents are likely to require higher incentives to effect the same change in response due to the competing demands placed on the respondents' working day, the value they place on their time, and possibly their perception of the value of the information to the survey sponsor. This leads us to propose

Table 1 - Monetary incentives in consumer/household mail surveys.

Full table

Table 2 - Monetary incentives in organizational mail surveys.

Full table

A second issue is differential response patterns. If they exist, researchers need to know which level of incentive maximizes the cost–benefit ratio and the effectiveness of individual techniques that might also provide an increased understanding of respondent motivation and behaviour (Dillman, 1972; Hansen, 1980; Hornik, 1981). Theoretical support for using prepaid monetary incentives is provided from three perspectives. Gouldner's (1960) norm of reciprocity states that people are more likely to help those who have helped them in some way. Giving a prepaid incentive builds a psychological obligation to reciprocate which is fulfilled by completing and returning the questionnaire. Secondly, uncomfortable levels of cognitive dissonance might be created by accepting the incentive without responding, which might increase respondent compliance (Festinger, 1957). To throw the money away would be wasteful, to accept it without completing the questionnaire would seem dishonest and to return the money to the sender would involve fruitless effort (Furse and Stewart, 1982). Thus, the simplest method of dispelling any dissonance is compliance. Thirdly, social exchange theory contends that participation in an exchange is determined by value assessments, based on cost–benefit analysis, made by the parties to the exchange (Groves and Couper, 1998). Although early studies argued that it is the giving of an incentive, not its value, that increases response rate (Moser and Kalton, 1976; Armstrong, 1975), others have shown response to increase significantly with larger incentives (eg Bolstein and James, 1990). If this were so, it would be useful to know the optimum size of the inducement. This leads us to propose

Given this, one objective of this study was to devise a statistical model to examine the effect of incentive size on mail-survey response in consumer and organizational populations. This will facilitate a comparison with the results of an earlier study by Jobber and Saunders (1988) based on 16 experiments conducted prior to 1986. A second objective is to develop a cost–benefit model of the impact of monetary incentives to enable researchers to determine the optimum incentive size. In doing this, we shall examine experimental evidence since 1975 to test the two hypotheses.

Modelling prepaid monetary incentives and response

The linear logit specification (1) proposed by Jobber and Saunders (1988) as a general model for mail-survey response rates has several features that recommend it to modelling the influence of monetary incentives:

where s is the number identifying each survey, where s=1, 2 ,..., S, P_s is the probability of a questionnaire being returned in survey s, n identifies each survey design element (such as questionnaire length, proportion of open ended questions) where D_ns is the value of survey design element n in survey s and a_n will be the coefficient of survey-design element once logs are taken to create a linear model.

The S-shaped response of the linear logit model fulfils Armstrong's (1975) three a priori assumptions about how incentives change mail-survey response rates:

1. There is no change in response rate when the monetary incentive is zero. This is achieved by making D_ns=M_s+1 where M_s is the value of the monetary incentive.
2. The predicted response rate increases asymptotically to a limit. The absolute limit must be 100%, although this assumes that 'everyone can be bought'.
3. There are diminishing marginal returns as each additional amount spent has less impact. The specification also contains a realistic feature that differs from Armstrong's exponential mode, namely:
4. The effect of an incentive depends upon the no-incentive response rate. This represents an incentive having less impact on a badly designed survey with a no-incentive response rate of, say, 2%, than on a better designed survey with a no-incentive response rate of, say, 20%. The asymptotic assumption already has incentives having a reduced effect when the no-incentive response rate is very high, for example, 95%.

The logit model also fulfills Armstrong's final request that the relationship be simple, although this is a general call for parsimony rather than an assumption for mail-survey response.

To analyse the impact of monetary incentives, the other influences on survey response are separated out to give

The no-incentive response rate (P_0s) then reduces to

Substituting (3) into (2) gives the marginal impact of monetary incentives with all other features of the survey eliminated

In Equation (4), the 'odds ratio', (P_0s/(1-P_0s)), represents the combined effect of all the survey-design elements other than the prepaid monetary incentive. The equation can be made linear, so it can be estimated using ordinary least squares regression using data from mail-survey experiments.

Adding two extra coefficients enables the specification to differentiate between the impact of monetary incentives on consumer and organizational respondents and to capture Armstrong's (1975) 'rule of thumb' that responses are constant for incentives over a small amount

where i and c are the coefficients of dummy variables for there being an incentive (I_s=e) or not (I_s=1) or a survey being of organizations (C_s=e) or consumers (I_s=1).

Taking logs produces a linear equation suitable for where U_s is an error term conforming to the usual assumptions of regression analysis

Data

The database used in this study was obtained from 36 experiments into the effect of prepaid monetary incentives conducted since 1975 (see Tables 1 and 2). As the studies are drawn from different years, it is necessary to standardize the economic value of each incentive. The 2002 dollar was used as the base (no studies were found after 2002) and the average annual rate of increase in USA consumer prices for the period 1975–2002 (US Department of Labor, 2004) was employed to standardize incentive values (see Tables 1 and 2). The year prior to the publication date was taken as the year of the study, unless other information was available. For the UK and New Zealand experiments, a similar process was followed after converting to US currency using standard exchange rate tables.

Results

A scatter plot of the experimental results (Figure 1) suggests several patterns, including:

1. a predominance of mail surveys with low response rates;
2. some organizational surveys achieving response rates as high as consumer surveys;
3. an average increase in survey response in the region of 20%;
4. consumer and organizational surveys showing similar responses to monetary incentives;
5. an indication from studies testing a range of monetary incentives that response rates increase with the incentive size (see Bolstein and James, 1990, Table 1).

Figure 1.

Increased response due to monetary incentives for consumers and organizations.

Full figure and legend (50K)

Estimating Equation (6) as a series of nested equations using the data in Tables 1 and 2 gave the following results:

^*t-tests and F-tests significant at 0.000 level

Comparison of the nested equations and the analysis of variance (Table 3) shows that the provision of an incentive (I_s) and the size of the incentive (M_s) significantly improve response rates. This supports the hypothesis H2 that as the magnitude of the incentive increases, so too does the response rate.

Table 3 - Analysis of variance of regression model for monetary incentive response.

Full table

However, the results do not support hypothesis 1 since the insignificance of the coefficient that differentiates between organizational and consumer surveys in Equation (7) does not allow us to reject the null hypothesis that organizational and consumer respondents respond the same to monetary incentives. The analysis of variance results underline the finding because adding the distinction between organizational and consumer samples does not significantly increase the variance explained.

Optimizing the monetary incentive

The amount to be spent on a monetary incentive depends upon the cost of mailing without an incentive. The cost (v) of a survey is

where n is the sample size required, P_M is the expected response rate with incentive M, q is the variable cost of mailing excluding the incentive, q₀ is the fixed cost of mailing excluding the incentive.

Substituting the generalized form of (8) into (10) shows the survey cost depends on the expected response rate without an incentive and the direct impact of the incentives

This differentiates with respect to M and equates to zero to show the optimum monetary incentive to be given. The intractable equation is

where I=e when M>0 and the results of differentiation are invalid for the trivial case where m=0.

This means the optimum monetary incentive (M) depends on: the variable cost of mailing a questionnaire excluding the incentive (q); the survey response rate with no monetary incentive (P₀); the coefficient for the mail incentive (m). Iⁱ is a constant reflecting the impact of providing any mail incentive. The equation can be approximated for M using Fourier analysis, but graphical methods are an attractive alternative for exploring the relationships. Figure 2 shows an almost linear relationship between the optimum incentive and the cost of mailing when the incentive-free mailing costs are above $2.50. Below that level, the results suggest that a minimal incentive would suffice to trigger the fixed component of the response to a monetary incentive (Iⁱ in Equation (5)). However, it is worth noting that none of the studies cited has tested the impact of incentives below 10¢. Although the survey response rate without an incentive has some influence on the optimum prepaid monetary incentive, its influence is small compared to the survey costs. This is indicated by the plots of two curves in Figure 2, which show the relationship between mailing costs and optimum incentive when the response rate without incentive is 10 and 50%, respectively.

Figure 2.

Optimum prepaid monetary incentive rates.

Full figure and legend (12K)

Discussion and conclusions

The study by Jobber and Saunders (1988) supported the use of monetary incentives, but concluded that incentive value made an insignificant contribution to the model. However, their work used a relatively small number of monetary incentive experiments. The inclusion of 20 additional experiments conducted since then provides a more robust data set from which to draw conclusions. The new model once again confirms the efficacy of using monetary incentives to increase response to mail surveys, but now incentive value does significantly affect response. This confirms recent work, which suggests that monetary incentives have the most effect on the decision process of moving respondents though a mail-survey response hierarchy of effect model from Attention through Intention, and Completion to Return (Helgeson et al, 2002). Furthermore, the larger sample size has allowed the testing of differences between consumer and organizational populations for which no support was found.

Using the minimization of overall survey costs, for a given sample size, as an objective of mail survey practice, the optimum monetary incentive was found to be a function of the variable costs of mailing a questionnaire excluding the incentive, the survey response rate with no incentive, and the coefficient for the mail incentive. As the cost of mailing (excluding an incentive) increases, so the size of the optimum incentive rises. Thus, when survey costs are low, the optimum incentive is low, since a large value incentive would represent a high proportion of overall costs. Conversely, when survey costs are high, a large incentive becomes feasible as it accounts for a smaller percentage of total costs. The curves displayed in Figure 2 can be used to determine optimum monetary incentive size. Once the variable costs of mailing a questionnaire have been calculated, the corresponding optimum incentive can be established. Although two curves are shown, representing two levels of response, in practical terms, the value of the optimum incentive does not differ dramatically between a high or low expected response rates. For mailing costs below $2.50 per questionnaire, a minimal incentive (eg 10¢) should be used.

References

1. Armstrong JS (1975). Monetary incentives in mail surveys. Public Opin Q 39: 111–116. | Article |
2. Armstrong JS and Yokum JT (1994). Effectiveness of monetary incentives: mail surveys to members of multinational professional groups. Ind Market Mngt 23: 133–136. | Article |
3. Balakrishnan PV, Chawla SK, Smith MF, and Micholski BP (1992). Mail survey response rates using a lottery prize giveaway incentive. J Direct Market 6: 54–59.
4. Bellizzi J and Hite RE (1986). Face-to-face advance contact and monetary incentive effects on mail survey return rates, responses differences and survey costs. J Bus Res 14: 99–106. | Article |
5. Bolstein R and James JM (1990). The effect of monetary incentives and follow-up mailings on the response rate and response quality in mail surveys. Public Opin Q 54: 346–361. | Article |
6. Braunsberger K, Gates R and Ortinau DJ (2005). Prospective respondent integrity behavior in replying to direct mail questionnaires: a contribution in overestimating nonresponse rates. J Bus Res 58: 260–268. | Article |
7. Brennan M (1992). The effect of a monetary incentive on mail survey response rates new data. J Marketing Res Soc 34: 173–177.
8. Brennan M, Hoek J and Astridge CA (1991). The effects of monetary incentives on the response rate and cost-effectiveness of a mail survey. J Marketing Res Soc 33: 229–241.
9. Burns AC and Hair JF (1980). Analysis of mail survey response from a commercial sample. Proceedings of American Institute for Decision Science, 12th Meeting (November), Vol. 1, Decision Sciences Institute, Atlanta, GA, pp 227–229.
10. Church AH (1993). Estimating the effect of incentives on mail survey response rates: a meta-analysis. Public Opin Q 57: 62–79. | Article | ISI |
11. Cycyota CS and Harrison DA (2002). Enhancing survey response rates at the executive level: are employee- or consumer-level techniques effective? J Mngt 28: 151–176.
12. Dennis Jr WJ (2003). Raising response rates in mail surveys of small business owners: results of an experiment. J Small Bus Mngt 41: 278–295. | Article |
13. Díaz de Rada V (2005). Measure and control of non-response in a mail survey. Eur J Marketing 39: 16–33. | Article |
14. Dillman DA (1972). Increasing mail questionnaire response in large samples of the general public. Public Opin Q 36: 254–257. | Article |
15. Dommeyer CJ (1988). How form of the monetary incentive affects mail survey response. J Marketing Res Soc 30: 379–385.
16. Edwards P et al (2002). Increasing response rates to postal questionnaires: systematic review. Br Med J 324(7347): 1183–1186. | Article | ISI |
17. Erwin WJ and Wheelright LA (2002). Improving mail survey response rates through the use of a monetary incentive. J Mental Health Counsel 24: 247–255.
18. ESOMAR (2005). Industry Report 2003. European Society for Opinion and Marketing Research: Amsterdam.
19. Festinger L (1957). The Theory of Cognitive Dissonance. Stanford University Press: Stanford, CA.
20. Friedman H and San Augustine A (1979). The effects of a monetary incentive and the ethnicity of the sponsor's signature on the rate and quality of response to a mail survey. J Acad Market Sci 7: 95–101.
21. Furse DH and Stewart DW (1982). Monetary incentives versus promised contribution to charity new evidence on mail survey response. J Marketing Res 19: 375–380. | Article |
22. Furse DH, Stewart DW and Rados DL (1981). Effects of foot-in-the-door, cash incentives and follow-ups on survey response. J Marketing Res 18: 473–478. | Article |
23. Gajaraj A, Faria A and Dickinson J (1990). A comparison of the effect of promised and provided lotteries, monetary and gift incentives on mail survey response speed and cost. J Marketing Res Soc 32: 141–162.
24. Gilbart E and Kreiger N (1998). Improvement in cumulative response rates following implementation of a financial incentive. Am J Epidemiol 148: 97–100. | PubMed |
25. Gilpatrick TR, Harmon RR and Tseng LPD (1994). The effect of a nominal monetary gift and different contacting approaches on mail survey response among engineers. IEEE T Eng Mngt 41: 285–290. | Article |
26. Goodstadt MS, Chung L, Kranitz R and Cook G (1977). Mail survey response rates: their manipulation and impact. J Marketing Res 14: 391–395. | Article |
27. Gouldner AW (1960). The norm of reciprocity: a preliminary statement. Am Sociol Rev 25: 161–178. | Article |
28. Groves RM and Couper MP (1998). Nonresponse in Household Interview Surveys. John Wiley & Sons: New York.
29. Hansen RA (1980). A self-perception interpretation of the effect of monetary and non-monetary incentives on mail survey respondent behaviour. J Marketing Res 17: 77–83. | Article |
30. Helgeson JG, Voss KE and Terpening WD (2002). Determinants of mail-survey response: survey design factors and respondent factors. Psychol Market 19: 303–328. | Article |
31. Hornik J (1981). Time cue and time perception effect on response to mail surveys. J Marketing Res 18: 243–248. | Article |
32. Hubbard R and Little EL (1988). Cash prizes and mail survey response rates: a threshold analysis. J Acad Market Sci 16(42): 42–44.
33. James JH and Bolstein R (1992). Large monetary incentives and their effects on mail survey response rates. Public Opin Q 56: 442–453. | Article |
34. Jobber D (1986). Increasing response rates to industrial mail surveys. Ind Market Mngt 15: 183–195. | Article |
35. Jobber D and Saunders J (1988). Modelling the effects of prepaid monetary incentives on mail-survey response. J Opl Res Soc 39: 365–372.
36. Jobber D, Birro K and Sanderson SM (1988). A factorial investigation of methods of stimulating response to mail surveys. Eur J Opl Res 37: 158–164. | Article |
37. Kotler P, Armstrong G, Saunders J and Wong V (2001). Principles of Marketing: Third European Edition. Prentice Hall: London.
38. Larson PD and Chow G (2003). Total cost/response rate trade-offs in mail survey research: impact of follow-up mailings and monetary incentives. Ind Market Mngt 37: 533–537. | Article |
39. Larson PD and Poist RF (2004). Improving response rates to mail surveys: a research note. Transportation J 42(4): 67–75.
40. London SU and Dommeyer CJ (1990). Increasing response to industrial mail surveys. Ind Market Mngt 19: 235–241. | Article |
41. McDaniel SW and Rao CP (1980). The effect of monetary inducement on mailed questionnaire response quality. J Marketing Res 17: 265–268. | Article |
42. Moser C and Kalton G (1976). Survey Methods in Social Investigation. Heineman: London.
43. O'Keefe T and Homer PM (1987). Selecting cost-effective survey methods: foot-in-door and prepaid monetary incentives. J Bus Res 15: 365–376. | Article |
44. Paolillo JGP and Lorenzi P (1984). Monetary incentives and mail questionnaire response rates. J Advertising 13: 46–48.
45. Parkes R, Kreiger N, James B and Johnson KC (2000). Effects on subject response of information brochures and small cash incentives in a mail-based case-control study. AEP 10: 117–124. | PubMed |
46. Peck J and Dresch S (1981). Financial incentives, survey response and sample representativeness: does money matter. Rev Public Data Use 9: 245–266.
47. Pressley MM and Tullar W (1977). A factor interactive investigation of mail survey response rates from a commercial population. J Marketing Res 14: 108–111. | Article |
48. Robin D and Walters G (1976). The effect on return rate of messages explaining monetary incentives in mail questionnaire studies. J Bus Commun 13(3): 49–54.
49. Roth PL and BeVier CA (1998). Response rates in HRM/OB survey research: norms and correlates, 1990–1994. J Mngt 24: 97–117.
50. Sahr RS (2005). Consumer price index (CPI) conversion factors 1800 to estimated 2015 to convert to dollars of estimated 2005. http://www.oregonstate.edu/Dept/pol_sci/fac/sahr/sahrhome.html (17 May).
51. Schneider KC and Johnson JC (1995). Stimulating response to market surveys of business professionals. Ind Market Mngt 24: 265–276. | Article |
52. Singer E, Groves RM and Corning AD (1999). Differential incentives: beliefs about practices, perceptions of equity, and effects on survey participation. Public Opin Q 63: 251–260. | Article |
53. Summers J and Price JH (1997). Increasing return rates to a mail survey among health educators. Psychol Rep 81: 551–554. | PubMed |
54. Tedin K and Hofstetter C (1982). The effect of cost and importance factors on the return rate for single and multiple mailings. Public Opin Quart 46: 122–128. | Article |
55. Tullar W, Pressley MM and Gentry D (1979). Towards a theoretical framework for mail survey response. Proceedings of the Third Annual Conference of the Academy of Marketing Science. Vol. 11. Academy of Marketing Science, pp 243–247.
56. US Department of Labor (2004). Bureau of Labor Statistics, http://data.bls.gov/servlet/SurveyOutputServlet (June 10).
57. Weathers PL, Furlong MJ and Solórzano D (1993). Mail survey research in counseling psychology: current practice and suggested guidelines. J Counsel Psychol 40: 238–244. | Article |