White Papers

Online Survey Response Rates and Times

Background and Guidance for Industry

Michael Braun Hamilton, Online Survey Analyst
support@supersurvey.com

Ipathia, Inc. / SuperSurvey
http://www.supersurvey.com
800-241-0279

Executive Summary

This white paper analyzes meta-data for 199 surveys conducted using the SuperSurvey^® cluster at http://www.supersurvey.com. These surveys all utilized our Auto-Generated Keys methodology for tracking of invitations and all were premium (paid) surveys. We analyze the meta-data to determine typical response rates and response times.

Conclusions:

Half of all surveys receive at least a 26% response rate, but due to the large degree of variation in response rates, this may be of little predictive value.
Half of survey responses arrive within one day, and nearly all (96.5%) arrive within 2 weeks.
Surveys invitations sent at the beginning of the workday achieve higher response rates and quicker response times.

Recommendations:

Utilize advanced tracking, such as SuperSurvey's^®Auto-Generated Keys, when responses are of high value.
Launch business surveys early in the day, and collect a minimum of two weeks for high-value surveys, or one week when speed is the primary concern. Firm-wide employee surveys may require more than two weeks to cover employee vacation time.
Contact SuperSurvey^® for guidance on conducting online surveys for high-value business, government, and research applications.

Introduction

SuperSurvey^® (http://www.supersurvey.com) has conducted online surveys for clients across industry and local and federal government, primarily in the US and Canada. Here we examine meta-data on a subset of those surveys to determine response trends for surveys using Auto-Generated keys, a tracking method typically used for higher-value professional surveys. These surveys include employee, customer, and conference trade show surveys. The analysis in this white paper provides industry with guidance on what to expect with online surveys and provides a baseline for comparison with the response rates and patterns that they receive on those surveys, along with recommendations for best practices.

Method

This paper analyzes a subset of the response meta-data compiled by the SuperSurvey^® system for surveys which used the site's e-mail invitation system and utilized Auto-Generated Keys to track individual invitees access to the survey.¹ We extracted from our logs the time and date of all invitations sent out by our email server, and the time and date of any corresponding response.² This meta-data sample consists of 199 surveys with a total of 523,790 invitations sent to potential respondents.

Response Rate

Average survey response rate - 32.52%

Median survey response rate - 26.45%

Total response rate - 13.35%

Average survey response rate (sample size < 1000) - 41.21%

Response rate is defined as the percentage of survey invitations that result in a response. Response rates vary greatly from survey to survey, affected by almost all aspects of the survey process.

The discrepancy be-tween these calculations of response rate is in part a result of the distribution of response rates with regards to sample size. Figure 1 shows response rates plotted against the number of invitations sent out. While smaller surveys have a relatively even distribution of response rates, surveys with larger sample sizes tend to have markedly lower response rates. These surveys with high sample sizes and low response rates are presumably large consumer surveys where a low response rate was expected and a larger sample size was necessary to obtain sufficient data.

If surveys with sample sizes greater than 1000 invitees are excluded, we have an average response rate of 41.21%. The distribution of these response rates is remarkably heterogeneous (See Figure 2), with a standard deviation of 29.40%. In general, response rates vary too much and rely on far too many factors for aggregate rates such as these to be of much predictive power.

Conclusions and recommendations:

Response rates vary greatly, but in our experience most surveys receive a 26% response or better.
Large invitation lists are associated with lower response rates. It is important to use as focused and high-quality a list as possible, (especially when acquiring sample on a paid basis.)
Using tracking methods such as SuperSurvey's^® Auto- Generated Keys can help in measuring and improving re-sponse. You can selectively follow up with non-responders in order to boost your response rate.

Response Time ³

Median Response Time (hh:mm:ss) – 16:26:44

25% Response Time (hh:mm:ss) – 02:24:15

75% Response Time (hh:mm:ss) – 82:33:58 (3 days, 10½ hrs)

95% Response Time (hh:mm:ss) – 294:28:36 (12 days, 6½ hrs)

Responses to online surveys arrive rapidly, a key component of the business case for online surveys. The median response time is 16½ hours — that is, 50% of respondents had responded in the first 16½ hours. The average time from sending an invitation to receiving a response is a little less than 3 days. (This average is pushed out by the fact that some of these surveys ran quite a long time and had responses coming in for up to 3 months.)

Figure 3 shows how the percentage of responses received rises rapidly for the first day or so and then begins to level off. This graph only show response times up to one week, but that covers over 87% of the responses. Table 1 breaks down the numbers further.

Table 1 - Survey Responses over Time

time	% response	time	% response	time	% response
5 min	0.99%	3 hrs	27.43%	4 days	77.82%
10 min	2.81%	6 hrs	35.05%	5 days	81.37%
15 min	4.71%	12 hrs	43.09%	1 week	87.45%
30 min	9.37%	24 hrs	59.42%	2 weeks	96.45%
1 hr	15.65%	2 days	67.93%	3 weeks	98.46%
2 hrs	22.99%	3 days	73.27%	4 weeks	99.25%

Conclusions and recommendations:

Over half of survey responses are likely to arrive in the first day.
Seven out of eight responses arrive within the first week
We recommend at least 2 weeks as a run time for surveys in which it is important to get a full response. This is especially true for firm-wide employee surveys, where employees may be on 2 week vacations.
One week should be a sufficient run time when speed is more of a concern than complete sample response.

Response and the Time of Day

Response rate and response time correlate strongly with the time of day at which invitations are sent. We find that that the highest response rates are for survey invitations sent out between 6:00 and 9:00 AM ET (Fig. 4).⁴ Likewise, this 3 hour period had some of the shortest average response times (Fig. 5). Response rates go down throughout the workday but remain above the mean until 3:00 PM ET. Average response times rise over the course of the workday (but fall again in the evenings).

Conclusions and recommendations:

Response rates and times are best for surveys sent out between 6:00 and 9:00 AM, at the beginning of the work day.
Response rates decline and response times increase over the course of the workday.
Though response times are quicker in the evenings, response rates are low.
Business surveys to be sent after 3:00 PM should wait until the next business day.

Next Step

Contact SuperSurvey^® for more information on successfully conducting your survey online, or visit our web application to begin building surveys right away at http://www.supersurvey.com.

SuperSurvey^®
800-241-0279
service@supersurvey.com
http://www.supersurvey.com

Notes:
1. When we use “survey” in this paper, we are referring to surveys that use Auto-Generated Keys.

2. For the purpose of this analysis, responses were defined as the respondent submitting at least one answer to a question on the survey.

3. Response times are figured as the difference in time between the invitation e-mail being sent out by our mail server and the respondent submitting the first answer to a question on the survey.

4. The times recorded by the SuperSurvey^® system are in Eastern Time (ET). It should be noted that many survey makers and respondents were located in other time zones, however.

Welcome!