• About
• Accreditation
• Compliance Assist
• HEOA
• Policies
• Program Review
• Qualtrics
• Student Achievement
• Student Learning Outcomes
• Surveys

  • First Destination-Undergraduate and Graduate
  • CIRP
  • NSSE
  • Senior Exit
  • ETS Proficiency Profile
• Wabash National Study
• Contact Us

Frequently Asked Questions (FAQ)

1. When and why should sampling be used?

2. How can response rates be improved?

When and why should sampling be used?

Sampling is a statistical tool used to gain representative information regarding a target population.  It works best for items like surveys, general skills/intelligence tests, and other information seeking methods.  The target population is the group of people who are able to answer the questions and to whom the results will apply (Kitchenham & Pfleeger, 2002).  Under ideal circumstances, the best way to gather representative data would be to administer the survey or test to your entire target population.  However, due to a number of reasons (cost, difficulty reaching total population, etc.); doing so is not always feasible.  When administering to the entire target population is not practical, sampling is an effective way to gain generalizable information for the whole target population (Kitchenham & Pfleeger, 2002).  Please note that sampling is only required when the entire target population cannot be reached.  (Ex. If your target population is under an n of 50, it is probably best to survey the entire population.)

In order to make sure a sample is valid and representative, it is important to use a probabilistic sampling method (simple random sample, stratified random sample, systematic sampling, or cluster-based sampling).  For more information regarding these sampling techniques, please review Principles of Survey Research Part 5: Populations and Samples (Kitchenham & Pfleeger, 2002).     

References
Kitchenham, B., & Pfleeger, S. (2002). Principle of Survey Research. Software Engineering, 17-20.

How can response rates be improved?

The task of maximizing response rates is challenging, to say the least. Over the past few years, response rates are trending downward and numerous studies are being conducted to address this issue (Baruch & Holtom, 2008).  Nonetheless, there are several measures to be taken to improve response rates.  Prior to describing some of these techniques, however, it is important to clarify one concern.  While response rate is valuable, representativeness should be the overall goal of any survey (Baruch & Holtom, 2008).  Even surveys yielding low response rates can be beneficial, if they are representative of the population to be analyzed.  Going forward, there are some very basic measures that can improve response rates including: keeping surveys short, avoiding complex and misleading questions, incorporating only those questions directly related to the focus area, and being consistent in design and question format (Kitchenham & Pfleeger, 2002).  Also, it is significant to engage the survey taker, both prior to and after implementation.  This could include building a rapport with the sample population [difficult with large populations], properly advertising the existence of the survey, and sending reminders.  Likewise, it is crucial to keep the actual survey instrument engaging.  When possible, and with reasonable constraints, make the survey instrument appealing (Puleston, 2011).  Incentives are yet another method used to increase response rates.  The literature is mixed on the effectiveness of incentives, but few studies show a decrease in response rate while using this approach (Porter & Whitcomb, 2003). Another vital matter is in relation to the amount of separate surveys being dispersed.  Survey fatigue is a real concern and can be limited by proper coordination of survey usage.  Piloting your survey is a crucial aspect of getting good response rates. Giving the survey a test run might emphasize a poorly constructed survey, as well as display areas for improvement (Puleston, 2011).  Lastly, it is important to be realistic in regard to desired response rates.  A host of factors can determine the likelihood of surveys being completed, many of which are outside the control of the surveyor [e.g. institutional and student demographics] (Porter & Umbach, 2006).

References
Baruch, Y., & Holtom, B. (2008). Survey Response Rate Levels and Trends in Organizational Research. Human Relations, 1139-1160.
Kitchenham, B., & Pfleeger, S. (2002). Principles of Survey Research Part 3: Constructing a Survey Intrument . Software Engineering Notes, 20-24.
Porter, S., & Umbach, P. (2006). Student Survey Response Rates Across Institutions: Why Do They Vary? Research in Higher Education, 229-247.
Porter, S., & Whitcomb, M. (2003). The Impact of lottery Incentives on Student Survey Response Rates. Research in Higher Education, 389-407.
Puleston, J. (2011). Improving Online Surveys. International Journal of Market Research, 557-560.