Confidence in results is related to many factors. The margin of sampling error, related to the sample size (the number of people the survey researchers spoke to), is one commonly used indication of how confident you can be that if the researchers had been able to ask the same question of every person instead of just those sampled, they would have gotten similar responses.
For example, if a survey was conducted with 1,000 American adults, it would have a margin of sampling error of plus or minus 3 percentage points. This means that if the researchers had been able to interview every American adult, instead of just the 1,000 sampled, the researchers could be 95% confident that the results they would have gotten from talking to every American adult would be no more than 3 points above or 3 points below the response percentage they got from interviewing only the sample (1,000 adults).
As a general rule of thumb, if 2,000 or more people were asked a question, the margin of sampling error is around+/-2 percentage points; for 800-1,000 it’s around +/-3 percentage points; for 500-700 it’s around +/- 4 points; for 400 it’s around +/-5; for 300 it’s around +/-6 points; for 200 it’s around +/- 7 points; and for 100 it’s around +/-10 points.
In addition to sampling error, surveys are subject to other forms of error, such as from nonresponse (people who did not answer the survey question – bias is related to how different people are who did not answer the survey questions from those who did answer the survey questions), question wording (such as ambiguous or confusing questions, or questions with words that some respondents may not have understood), and context effects (such as when respondents may be “clued in” to an answer by other questions that were just asked before that question).
For an explanation of how confident you should be in the results of particular surveys, see
The National Council on Public Polls' "20 Questions A Journalist Should Ask About Poll Results".
[Return to FAQ List] | [Submit New Question]
