The availability of online tools such as Survey Monkey makes it easy to collect data online–too easy, in my opinion. In fact, the ease of data collection makes it quite likely that the unwitting user will collect misleading information.
Here’s just one way a survey can go wrong. Let’s say you want to know how people in your neighborhood view the upcoming U.S. presidential election. You create an online survey and send a message to folks in your neighborhood via Nextdoor, saying “Please take my online survey!” You get 500 responses, which seems like a big enough number to be credible.
Do these responses accurately reflect the views of people in your neighborhood?
Probably not, because:
- You’ve received responses only from people who use Nextdoor.
- You’ve received responses only from people who don’t mind taking the time to respond, either because they feel strongly about the election or because they have nothing better to do.
- If your message and survey are in English, you probably aren’t capturing the views of non-English speakers.
- By having an online survey, you aren’t capturing the views of people who don’t use the Internet.
In fact, it is quite likely that you’ll get a more accurate picture of the views of your neighbors by getting fewer responses, as long as those responses come from a representative sample of neighborhood residents. A statistician can help you figure out just how many responses you need to obtain for your desired level of accuracy.
Another way a statistician can help is to devise a strategy that will result in a representative sample. For example, one method would be to randomly select blocks in your neighborhood, then randomly sample addresses from those blocks, and then obtain responses from those addresses. Sure, this is more work, but if you truly want accurate data, this method is far better than simply putting up an online survey and getting responses from whoever happens to feel motivated to reply.